OpenPano | Automatic Panorama Stitching From Scratch | Computer Vision library

OpenPano fails at stitching text because it cannot retrieve enough feature points (or keypoints) to do the stitching process.

Text stitching doesn't need a matching method that is robust to rotations but only to translations. OpenCV conveniently offers such a function. It is called : Template Matching.

The solution I will develop is based on this OpenCV's feature.

I will now explain the main steps of my solution (for further details, please have a look at the code provided bellow).

In order to match two consecutive images (done in the matchImages function, see code bellow):

1. We create a template image by taking 45% (H_templ_ratio) of the first image as depicted bellow:

This step is done in my code by the function genTemplate.

1. We add black margins to the second image (where we want to find the template). This step is necessary if the text is not aligned in the input images (this is the case on these sample images though). Here is what the image looks like after the margin process. As you can see, the margins are only needed bellow and above the image:

The template image could theoretically be found anywhere in this margined image. This process is done in the addBlackMargins function.

1. We apply a canny filter on both the template image and the image where we want to find it (done inside the Mat2Edges function). This will add robustness to the matching process. Here is an example:

1. We match the template with the image using matchTemplate and we retrieve the best match location with the minMaxLoc function.

This step consists in calculating the size of the final matrix where we will stitch all the images together. This is particularly needed if all the input images don't have the same height.

This step is done inside the calcFinalImgSize function. I won't get into to much details here because even though it looks a bit complex (for me at least), this is only simple maths (additions, subtractions, multiplications). Take a pen and paper if you want to understand the formulas.

Once we have the match locations for each input images, we only have to do simple maths to copy the input images in the right spot of the final image. Again, I recommend you to check the code for implementation details (see stitchImages function).

Here is the result with your input images:

As you can see, the result is not "pixel perfect" but it should be good enough for OCR.

And here is another result with input images of different heights:

My program is written in Python and uses cv2 (OpenCV) and numpy modules. However it can be ported (easily) in other languages such as C++, Java and C#.