emberfire | officially supported adapter for using Firebase | Database library

I am writing an image viewer JS component and i'm currently stuck on the calculation for the zooming in. I figured out the zooming in or out on one point (instead of just the center) using css transforms. I'm not using transform-origin because there will be multiple points of zooming in or out, and the position must reflect that. However, that's exactly where i'm stuck. The algorithm i'm using doesn't calculate the offset correctly after trying to zoom in (or out) when the mouse has actually moved from the initial position and I can't for the life of me figure out what's missing in the equation.

The goal is whenever the mouse is moved to another position, the whole image should scale with that position as the origin, meaning the image should scale but the point the mouse was over should remain in its place. For reference, look at this JS component. Load an image, zoom in somewhere, move your mouse to another position and zoom in again.

Here's a pen that demonstrates the issue in my code: https://codepen.io/Emberfire/pen/jOWrVKB

I'd really apreciate it if someone has a clue as to what i'm missing :)

Here's the HTML