java-tips | Java 语言编程技巧、最佳实践 | Build Tool library

You've almost got it. You're going to need a good value for Z in the unproject function though.

What you are trying to do is take the position of the cursor and multiply by a matrix to give a point in "3d space". Your matrices are likely 4x4 or 4x3, so you need a 4 component vector. (x,y,z,w)

When you draw your map, the existing point is multiplied by 1 or more matrices including the projection matrix. ( e.g. -25.0f,25.0f,0.0f,1.0f - actually a 3d point). When this is multiplied by all matrices, the GPU essentially gets back a value in normalised device coordinates (NDC) (between -1 and 1 in all axes) for that vertex.

To do the opposite and unproject you'll need to have a valid/good value for Z. The reason is that in NDC everything that is drawn is in -1,1 on all axes, to get everything in (further away things are squashed a bit). This is how you get flickering and weirdness if you have a huge > 100000 zFar distance for example, it still has to fit into -1,1.

The best way to do this is to use the depth buffer, by capturing the depth value it'll give you a good approxomation of the z coordinate in NDC, which you can pass to the unproject call.

The reason why 0.945 is the sweet spot is probably dependent on how far the camera is from your map or vice versa. It's usually the case that the depth buffer has much more detail closer to the near plane than the far - it's not linear.

http://www.opengl-tutorial.org/beginners-tutorials/tutorial-3-matrices/ has a good visual near the bottom of the page, and is a good resource for intro to matrices in general:

You can see the distortion caused by moving to NDC. This is required for viewing from a perspactive viewpoint, but you need to take it into consideration when you transform backward too.

Colour picking as mentioned is also viable for picking, but will still require some work. Because you have a single object, you'll have to render each texel of the image with a different colour, output that to a separate colour buffer, check to see what colour is on the buffer and somehow relate that to a point in space. It could probably be done though, but I'd say colour picking is more suited to multiple objects.

From what I've read - the depth buffer one might be more suitable for you as it's one object, and the depth buffer will give you a Z coordinate for every point you click on. It could still be on your far plane, but it will still give you a value.

Alternatively, as suggested by @elect use an orthographic projection.