Depth Calibration
Calibrate stereo camera depth perception
ToF Calibration
Calibrate for ToF extrinsics
Depth Calibration
Camera calibration is a process of determining the intrinsic, extrinsic, and distortion parameters of a camera. These are required for a camera to map points in the 3D world to 2D points in an image, and vice versa, to undistort images, and to determine depth from stereo images.From these parameters, the camera also calculates the rectification matrices used to rectify the images from the stereo camera pair, allowing the StereoDepth node to perform stereo disparity calculation and depth estimation.All OAK cameras (except Modular line) are calibrated before shipment, and it's not required to recalibrate them. For the OAK FFC camera modules, it is necessary to perform camera calibration after mounting the cameras in the desired configuration.
./calibrate.py --help which will print out all of the calibration options.Prerequisites
If you don't yet have thedepthai repository on your computer, you need to clone it and install the requirements:Command Line
1git clone https://github.com/luxonis/depthai.git
2cd depthai
3git submodule update --init --recursive
4python3 install_requirements.pyPrepare Charuco Board
We recommend displaying the charuco board on a TV or a large flat monitor. Larger screens are better, as they allow more charuco markers to be visible in the image, typically improving calibration accuracy. Depending on the screen size, we suggest displaying the following charuco board in full-screen:If you have a different screen size, round down to the nearest size.Display the Charuco Board
When displaying the charuco board, markers and squares should be sharp and clearly visible. Keep in mind:- Avoid too bright or dim screens.
- Avoid direct bright lights or sun on the screen.
- Display the charuco board in full-screen.
Note the size of the squares on the printed charuco board. This is the SQUARE_SIZE_IN_CM argument for the calibration script.
Calibration
Please select the appropriate calibration procedure for your device:Compact devices
Modular devices
| Argument | Argument alias | Argument Description |
|---|---|---|
-s | --squareSizeCm | Measured square size of the printed charuco board in centimeters |
-brd | --board | Name of the camera (from depthai-boards, not case-sensitive), or path to a custom .json board config |
-nx | --squaresX | Number of squares in X direction. SquaresX is specified in Prepare Charuco Board, depending on your screen size. |
-ny | --squaresY | Number of squares in Y direction. SquaresY is specified in Prepare Charuco Board, depending on your screen size. |
-cm | --cameraMode | Camera mode, either perspective (default) or fisheye. |
-mdmp | --minDetectedMarkersPercent | Minimum percentage of detected markers in a frame, to consider the frame valid. Default is 0.5 (50%). If you want to be more strict, you can increase this value, but it can cause longer time to get enough valid frames. |
-ep | --maxEpipolarError | Maximum epipolar error in pixels, to consider the frame valid. Default is 0.7. If you want to be more strict, you can decrease this value. |
Command Line
1python3 calibrate.py --helpCamera Positioning During Calibration
We suggest capturing the calibration from different angles and distances, as it will assist the calibration algorithm in finding the best possible calibration.1. Close to the screen: Ensure the calibration board covers almost the entire field of view (FOV). Take 5 images to cover the entire FOV of the camera:- Front view, with the calibration board in the middle of the FOV.
- Without moving the camera, rotate to align the camera FOV with the calibration board edges. Take 4 images, one for each edge of the screen.
- Front view, with the calibration board in the middle of the FOV.
- Similar to the close to the screen position, rotate the camera to align the FOV with the calibration board edges.
- Frontal view, with the calibration board in the middle of the FOV.
- Take 4 images aligning the camera FOV with all 4 edges, similar to the close and middle distance positions.
- Also, take 4 images aligning with the corners of the screen.
Running the Processing Stage
After capturing images, we can run the processing stage of the calibration. This is done by pressings key. The script will show epiplolar lines for each image, and you should check if they are aligned correctly. Once all images are checked, the calibration result (if successful) will be flashed on to the device EEPROM. Each captured image is saved in the dataset folder, so you can re-run the calibration process on its own, without having to capture the images again.If you want to re-run the calibration process on the captured images, use the -m process argument:Command Line
1python3 calibrate.py -s [SQUARE_SIZE_IN_CM] --board [BOARD] -nx [squaresX] -ny [squaresY] -m processresources/ folder and can be used later for testing/debugging purposes. You can also load/flash this local calibration file to the device - see example for more details.Test Depth
For testing depth quality, use DepthAI Viewer, which can be installed by following the Getting Started instructions.Troubleshooting
- If calibration fails with error:
High reprojection error!, the usual cause is a misconfigured board config, often due to incorrect specified HFOV of the used camera module. - If despite successful calibration, the depth is still incorrect, perhaps your left and right cameras are swapped. Retry the calibration with changed board config, or swap the board sockets the cameras are plugged into.
ToF Calibration
The Time-of-Flight (ToF) calibration is essential for aligning the ToF sensor with other cameras in the system. This procedure focuses on obtaining extrinsic parameters, crucial for ensuring coordinated operation among different cameras.Note that this calibration does not enhance the depth accuracy, as that aspect is managed by the device's firmware.
Calibration Procedure
If you have already installed the DepthAI repository, update it for TOF calibration with the following commands:Command Line
1git checkout new_tof_calib
2git submodule update --init --recursiveCommand Line
1python3 .\install_requirements.pycalibrate.py with the appropriate parameters for the Charuco board you are using. For example:Command Line
1python3 calibrate.py -db -nx 12 -ny 9 -c 1 -cd 0 -s 6 -ms 4.7 -brd OAK-D-SR-POE-db: Indicates the default board, meaning you are using Charuco markers.-nx: Number of Charuco markers in the x direction.-ny: Number of Charuco markers in the y direction.-c: Number of pictures taken each time the polygon is displayed (optional).-cd: Countdown time before a picture is taken in seconds (optional).-s: Size of the square around the Charuco marker in centimeters.-ms: Size of the markers in centimeters.-brd: Board of the device (in this case, OAK-D-SR-POE).
division by zero or Failed to detect markers in the image dataset/rgb/rgb_p3_10.png, delete the picture with poor detection of Charuco boards in all camera folders, then run the same code again with the added parameter -m process. This will initiate only the processing stage, so you won't have to retake pictures of the board.Command Line
1python3 calibrate.py -db -nx 12 -ny 9 -c 1 -cd 0 -s 6 -ms 4.7 -brd OAK-D-SR-POE -m process