homography | python library for 2d homographies

I am trying to check if this image:

is contained inside images like this one:

I am using feature detection (SURF) and homography because template matching is not scale invariant. Sadly all the keypoints, except a few, are all in the wrong positions. Should I maybe trying template matching by scaling multiple times the image? If so, what would be the best approach to try and scale the image?

Code:

With the method explained in CV - Extract differences between two images we can identify the differences between two aligned images.

How to do this with OpenCV when the camera angle (point of view) and the lighting condition are slightly different?

The code from How to match and align two images using SURF features (Python OpenCV )? helps to rotate / align the two images but as the result of the perspective transform ("homography") is not perfect, the "difference" algorithm will not work well here.

As an example, how to get only the green sticker (= the difference) from these 2 photos?

I am doing an image stitching project using OpenCV. Now I have the homography H1 between img1 and img2, and the homography H2 between img2 and img3. Now I need to compute the homography between img1 and img3, simply multiply H1*H2 is not working.

Are any ideas to calculate the new homography between img1 and img3?

I a following this tutorial on image alignment via openCV. There was no part with face detection, so I added it by myself.

I am quite intrigued by the idea of a homography and try to get it to work at a minimal example with python and OpenCV. Yet, my tests do not pass and I am not quite sure why. I pass in a set of corresponding points into the findHomography function according to This and then multiply the homography matrix to receive my new point.

so the idea behind it is to find the planar coordinate transformation and then transform the points with

X' = H@X

where X' are the new coordinates and X are the coordinates in the new coordinate frame.

Here is some minimal code example:

This is a problem concerning stereo calibration and rectification using openCV (vers. 4.5.1.48) and Python (vers. 3.8.5).

I have two cameras placed on the same axis as shown on the image below:

The left (upper) camera is taking pictures with 640x480 resolution, while the right (lower) camera is taking pictures with 320x240 resolution. The goal is to find an object on the right image (320x240) and crop out the same object on the left image (640x480). In other words; To transfer the rectangle that makes up the object in the right image, to the left image. This idea is sketched below.

A red object is found on the right image and I need to transfer it's location to left image and crop it out. The objects is placed on a flat plane 30cm from the camera lenses. In other words; The distance (depth) from the two cameras lenses to the flat plane is constant (30cm).

This main question is about how transfer a location from one image to another, when two cameras are placed side by side, when the images are of different resolutions and when the depth is (fairly) constant. It's not a question about finding objects.

To solve this problem, as far as I know, stereo calibration must be used, and I have found the following articles/code, among other things:

• https://github.com/bvnayak/stereo_calibration/blob/master/camera_calibrate.py
• https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_calib3d/py_calibration/py_calibration.html
• https://python.plainenglish.io/the-depth-i-stereo-calibration-and-rectification-24da7b0fb1e0

Below are an example of a calibration pattern that I used:

I have 25 photos of the calibration pattern with the left and right camera. The pattern is 5x9 and the square sizes is 40x40 mm.

Based on my knowledge, I have written the following code:

Part of my dissertation involves rectifying a section of an image that is not in perfect frontal perspective. I can't add images because of my low score, but it's essentially this python code (with images).

I've already built the program and it works, but I don't understand the math behind it. I've looked into matrix image rectification and homography, but there is a huge skill gap to the point I can't understand any of it. I'm not sure where to start learning the math. I know the basics of matrices and that's about it. I want to get to the point where I can code the matrix transformation function myself.

In an nutshell, what do I need to "study" to understand the math?

Thank you.

Currently, I'm working on image stitching of aerial footage. I'm using the dataset, get from OrchardDataset. First of all, thanks to some great answers on stackoverflow, especially the answer from @alkasm (Here and Here). But I having an issue, as you can see below at Gap within the stitched image section.

I used the H21, H31, H41, etc to wrap the images. The stitched image using H21 is excellent, but when wrap the img3 to current stitched image using H31, result shown terrible alignment between img3 and current stitched image. As the more images I wrap, the gap gets bigger and the images totally not well aligned.

Does the brillant stackoverflow community have an ideas on how can I solve this problem?

These are the steps I use to stitch the images:

1. Extract the frame every second from the footage and undistort the image to get rid of fish-eye effect using the provided camera calibration matrix.
2. Compute the SIFT feature descriptors. Set up macther using FLANN kd-tree and find matches between the images. Find the Homography (H21, H32, H43 and etc, where H21 refer to the homography which warps imag2 into coordinates of img1)
3. Compose the homography with the previous homographies to get net homography using the method suggested in Here. (Compute H31, H41, H51, etc)
4. Wrap the images using the answer provided in Here.

Gap within the stitched image:

I'm using the first 10 images get from OrchardDataSet.

Stitched Image with Gaps

Here's portion of my script:

main.py

ref_img is the first frame (img1). AdjHomoSet contain the images to be wraped (img2, img3, img4, etc). AccHomoSet contain the net homography (H31, H41, H51, etc)

In page 116 of Multiple View Geometry, the graphs compare DLT error (solid circle), Gold Standard/reprojection error (dash square), theoretical error (dash diamond) methods of estimating a homography between figure a and the original square chess board. Figure b shows that when you use more point correspondences to do such an estimation, there is a higher residual error (see figure below). This doesn't make sense to me intuitively, shouldn't more point correspondences result in a better estimation?