Camera calibration determines optimal parameters for the cameras. These parameters include the focal length and the transform from one camera to the other. More on the parameters being optimised for camera calibration can be found on the input page.
The calibration requires two steps, creating the
nbs recording and using it to find the camera parameters.
On the robot, enable compressed image recording in
config/DataLogging.yaml. To edit the file, run
nano config/DataLogging.yaml. Make sure
true(it should be by default).
Ensure the Left and Right camera serial numbers are correct in their respective configuration files (
Right.yaml). The configuration files are found in the Camera module folder, with separate directories for each robot.
The serial numbers can be retrieved by first running
lsusband finding the bus and device numbers for the FLIR/Point Grey cameras. Then run
lsusb -s <bus number>:<device number> -vto find the serial numbers.
Next you need to figure out which serial number is left and which is right. Change the serial number for one camera to an incorrect number (this ensures it's not found when running test/camera). Enable
./test/cameraand test which camera is working by covering one camera and seeing the resulting image in the NUsight vision tab. Alternatively, you could unplug one camera rather than change the serial number. If the serial number is on the wrong camera configuration file, switch them.
Check the focus of the lenses using the NUsight vision tab. If it is not sharp, focus the camera. This involves pulling the camera out from the head, still plugged in, and loosening the three grub screws. Twist until the camera is focused.
Verify that the extrinsics look correct in NUsight. The extrinsics are the transform from the last neck motor to the camera. This is retrieved from the robot's URDF model, but differences in the real robots may result in required offsets. Offset values for both pitch and roll are in the camera configuration files. The offsets are in radians.
message.output.CompressedImagevalue greater than 0.
- Stand the robot up with
- Face the robot towards a straight horizontal line, such as a line of bricks on a brick wall.
./test/visualmeshand switch to the vision tab in NUsight
NUsight should show a blue horizontal line. Adjust the offsets until the line is drawn at a constant height (the height of the cameras) along the wall.
./test/cameraon the robot and hold up a board as shown in the below image. There is an appropriate board in the NUbots laboratory. Move the board around to cover the robot's view over time, taking into account that the lenses have a field of view. Move the board around at different distances and angles.
Stop the binary and copy the
nbsfile created into the NUbots directory on your computer.
To stop the program running, hit Ctrl + c.
Discover the name of the log file created. The file is in the
recordings/test/camera/folder on the robot. The name corresponds to the date and time it was created, according to the robot. Run
ls recordings/test/camera/to list all logs.
(optional) Rename the log file, making it easier to copy across to the computer. Change the name with the command
mv <old path> <new path>.
Run the command
scp <robot address>:<file path> .from the NUbots directory on your computer, not on the robot.
The robot can be turned off at this point. The recording you just made should be on your computer in the NUbots directory. Check that it is before continuing to the next section.
Run the following commands to install the dependencies of the camera calibration tool if you have not done this beforesudo apt updatesudo apt install protobuf-compiler libprotobuf-devsudo -H pip3 install --upgrade pipsudo -H pip3 install tensorflow opencv-contrib-python ruamel.yaml
Calibrate the cameras by running./b nbs calibrate_cameras <nbs file name> -c <camera config folder>
<nbs file name>is the name of the
nbsfile you just created.
<camera config folder>is the path to the folder with the two camera config files
Right.yaml. These exist in
The calibration routine changes the values in the camera configuration files.
Verify these values by running vision and checking the output in NUsight.
You should see the same blue line as before, correctly aligned with the wall. If the line is not aligned, you will need to adjust the camera offsets.
test/localisationrole and check that the field lines in the localisation view are parallel or perpendicular to each other. If the perspective does not look right, then the cameras are not calibrated correctly.
Repeat this process with other robots if needed.
Commit these changes in a new branch on GitHub and make a Pull Request. Read the Git guide if you are unsure about this step.