Data Recording and Playback
The NUbots system utilises NUClear's message passing architecture. Any messages that are emitted when the system is running can be recorded and saved as NUClear Binary Stream (NBS) files. These NBS files can later be filtered, played back (to simulate the running system), analysed, and converted to alternative formats for further processing.
The following steps detail how to record data. An example will be given throughout showing how to record sensors messages on a real robot.
Set up the role for recording. Most of the time this involves making minor changes to an existing role.
- The role must contain the module/s which emit the message/s you want to record.
- The role must contain the module
Set up the data logging configuration file. This involves making changes to
Add in the messages you want to log, and set their values to
In this example, the sensors message already exists in the file (
message.input.Sensors). Set the boolean to
truenext to this message.
If you would like to log a different message and it doesn't already have a corresponding line in the DataLogging configuration file, do the following:
- Find the
.protofile that defines that message, such as
- Find what the
Sensor.proto, it is
- Piece this together with the message name (
message Sensorsin the
.protofile) to get
- Add this to the configuration file with the other messages.
You can enable multiple messages at once, and they will all be recorded.
- Find the
Build the code and install it to a robot. To find out how to do this, visit the Getting Started page.
Run the binary for the role you want to record.
Locally in Docker: In the same location you built the code, run the binary using the command
./b run <role_name>.
On a real robot: SSH into the robot using
ssh nubots@<address>and run the binary built for the role you want to record. For our example, run
Stop the binary when you have finished recording by pressing Ctrl+C.
Retrieve the log file.
Locally in Docker: The log file will be in your NUbots directory under
On a real robot: You will need to move the file off the robot so that it is not lost and so that it can be used. Note that when someone else uses the robot any recordings may be lost, so immediately copy them to another computer.
The NBS file will be located on the robot in the
recordings/role_name/folder. The name will correspond to the time and date it was created, according to the robot's clock. Make note of the name. In our example, this may be
Copy the file across. On the destination computer, run:scp nubots@<address>:recordings/rolename/file_name path/to/destination
For our example, you could run:# Replace <address> with the robot's IP addressscp nubots@<address>:recordings/test/sensor/18072020T082100.nbs .
The file will be copied from the robot to the folder you ran the command in. The
scpcommand is like the regular
cpcommand, but it copies to or from another computer over SSH. The first path in the command points to the file we are copying and the second path points to where we want to copy that file to.
NBS files are binary and not easily human-readable. There are tools in the NUbots codebase to convert NBS files to alternative formats that are easier to understand.
These tools are grouped under the
./b nbs command in the NUbots codebase, and can be executed by running:
./b nbs <tool_name> <nbs_file_name> <arguments>
Generates statistics about the messages in an NBS file. This includes a count of each message type in the file, and how many messages there are in total.
# Example./b nbs stats path/to/file.nbs
|Uses the message's |
Converts the data in an NBS file to JSON, and prints the output. To save the output to a file, append
> file.json to the command.
# Example./b nbs json path/to/file.nbs > file.json
Extracts JPEG image files from
CompressedImage messages in an NBS file. It will also write each image's metadata (
Hcw and lens information) to a JSON file next to the extracted image.
# Example./b nbs extract_images path/to/file.nbs --output path/to/output-folder/
|Specifies the folder to save the images to.||The current working directory|
Extracts images from
CompressedImage messages in an NBS file, and combines them into an MP4 video.
# Example./b nbs video path/to/file.nbs --output path/to/output-folder/
|The folder to save the video to.||The current working directory|
|The quality to save the video in.||30M|
|The encoder to use when encoding the video.||h264_nvenc|
Calculates camera intrinsics using an NBS recording of asymmetric circles grid patterns.
|The path to the directory containing the camera configuration files.||Required|
|The number of rows in the asymmetric circles grid.||4|
|The number of columns in the asymmetric circles grid.||11|
|The distance between rows/cols in the grid in meters.||0.04|
|Does not change the intrinsic configuration values.||-|
|Does not change the extrinsic configuration values.||-|
Trains a foot down network using sensor data from the legs.
# Example./b nbs footdown path/to/data-folder/
Messages recorded to an NBS file can be played back to the system. When an NBS file is played back, the messages in it are emitted into the system at the same (relative) times they were recorded.
Following on from the previous example, lets now run
test/sensor.role with recorded sensor data to analyse the results more closely without using the robot.
- Remove any modules that emit the message/s you recorded. In this example, remove the SensorFilter module, since it emits
Sensors. To do so, remove
input::SensorFilterfrom the role.
support::logging::DataPlaybackto the role.
- Add in the path to the NBS file in
DataPlayback.yaml. If the file is in the
recordingsdirectory, then just add in the file name.
- Add in the messages you are playing back in
DataPlayback.yaml, in the same way it was done in
DataLogging.yaml. In the example, change
- Recompile and run the role you are playing back the data with. In the example, rerun
./b build, then run
./b run test/sensorto locally run the role.
- For this example, it makes sense to view the robot in NUsight. Find out how to set up NUsight on the NUbook Getting Started page.
Playing back recorded data is useful when:
- Working remotely, where the data was previously recorded or recorded by another team member
- Testing a system with the same input data
- Not wanting to run the real robot continuously, where it is possible to use the same data
- Viewing the Visual Mesh in NUsight, due to bandwidth issues in sending it over the network