stereo-calibration | : camera : : camera : Stereo camera calibration | Camera library

This is a problem concerning stereo calibration and rectification using openCV (vers. 4.5.1.48) and Python (vers. 3.8.5).

I have two cameras placed on the same axis as shown on the image below:

The left (upper) camera is taking pictures with 640x480 resolution, while the right (lower) camera is taking pictures with 320x240 resolution. The goal is to find an object on the right image (320x240) and crop out the same object on the left image (640x480). In other words; To transfer the rectangle that makes up the object in the right image, to the left image. This idea is sketched below.

A red object is found on the right image and I need to transfer it's location to left image and crop it out. The objects is placed on a flat plane 30cm from the camera lenses. In other words; The distance (depth) from the two cameras lenses to the flat plane is constant (30cm).

This main question is about how transfer a location from one image to another, when two cameras are placed side by side, when the images are of different resolutions and when the depth is (fairly) constant. It's not a question about finding objects.

To solve this problem, as far as I know, stereo calibration must be used, and I have found the following articles/code, among other things:

• https://github.com/bvnayak/stereo_calibration/blob/master/camera_calibrate.py
• https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_calib3d/py_calibration/py_calibration.html
• https://python.plainenglish.io/the-depth-i-stereo-calibration-and-rectification-24da7b0fb1e0

Below are an example of a calibration pattern that I used:

I have 25 photos of the calibration pattern with the left and right camera. The pattern is 5x9 and the square sizes is 40x40 mm.

Based on my knowledge, I have written the following code: