A stereo camera is an arrangement composed of two or more image sensors with fixed positioning between each other. No surprise the most popular stereo system is the one made up of two cameras. Its configuration tries to mimic the biological eye with the main purpose of retrieving the depth info.
The success of depth estimation using the triangulation technique depends on two factors:
the definition of system parameters
the correct association of corresponding points between two images
The system parameters can be calculated from the calibration method. The most common one is made by capturing several chessboard images. For triangulation, it is necessary to know the focal length 𝑓 of the cameras and the baseline 𝑏 (distance between the cameras' projections centers 𝐶). Regarding points association, several techniques are available, but it is still a widely studied problem.
As you may know, the purpose of this post is to demonstrate the triangulation method. For that, we will visualize this process from a two-camera stereoscopic system. Figure 1, together with equations 1.1 to 1.6, illustrates one way of interpreting the triangulation process through two images obtained from different points of view.
Looking for figure 1, 𝐶𝐿 and 𝐶𝑅 indicate the center of the coordinate system of the cameras. The point 𝑃(𝑋, 𝑍) is plotted as a function of the left camera coordinate system; 𝑏 represents the distance between the left and right camera coordinate system; 𝑢𝐿 and 𝑢𝑅 are the projection of point 𝑃 onto the left and right image plane, respectively; 𝑓 is the distance from 𝐶𝐿 (or 𝐶𝑅) to its orthogonal projection onto the respective image plane.
Figure 1: Point 𝑃 is projected onto two different viewpoints. Source: author [1].
Given the highlighted elements and the green and blue dashed triangles in Figure 1, it is possible to conclude equations 1.1 and 1.2 from the trigonometric relationships in the right triangle.
Equation 1.3. is found by isolating and equating 𝑋 in equations 1.1 and 1.2.
Another relation between 𝑡𝑎𝑛(𝜃) and 𝑡𝑎𝑛(𝛼) is found considering 𝑢𝑅 and 𝑢𝐿, as shown in Equations 1.4 and 1.5 (triangles highlighted in light pink, Figure 1). Consider the origin of the image plane coordinate system is located on the perpendicular projection of 𝐶𝐿 (or 𝐶𝑅) onto the plane (illustrated in the image opposite). However, keep in mind the origin in an image plane is commonly defined in the upper left corner.
Finally, substituting into Equation 1.3 the respective values of 𝑡𝑎𝑛(𝜃) and 𝑡𝑎𝑛(𝛼) of equations 1.4 and 1.5, we reach Equation 1.6.
Nice to mention that the difference 𝑢𝐿 - 𝑢𝑅 is called disparity. You can check this concept and others in Stereo Camera: Recovering the Scene's Depth to learn more about this challenge in computer vision! See you. :)
[1] MELO, Mirella Santos Pessoa de. Mapeamento de região navegável a partir de um sistema SLAM e segmentação de imagem. 2021. Dissertação de Mestrado. Universidade Federal de Pernambuco.