SurfaceMapperGUI | simple projection mapping interface using Processing | 3D Animation library

I want to warp subsections of an image to project it on a nonuniform surface. Ultimately I want to warp an image as seen HERE, kinda like was is done in HERE from THIS project.

My problem is when I apply the transformations to each subsection of the image, things just do not line up

This is my process by which I achieve the transformations and then stitch (crop and paste them together on the final image.

1. Get a list of all the points
2. Create a quadrilaterals Region of Interest (ROI) from set of 4 points

3. Those 4 points are used to Transform the image with the corresponding original 4 points. This is done using my function called perspective_transform()

   a. I take the 2 set of 4 points and pass them in to M = cv2.getPerspectiveTransform(corners, newCorners)

   b. Then I call: warped = cv2.warpPerspective(roi, M, (width, height))

4. After getting the new warped image I use mask’s to stich everything together based on the ROI it was associated with:

   a. This is done by the function quadr_croped()

   Initialization of screen to Get raw pixels from the screen, save it to a Numpy array ...