Thanks for your code.  I ran the program you wrote and saw that the cube was placed wherever my mouse was clicked.  I'm not sure if this solves my problem or not because I still don't understand it.  Where's the z=1 that the reproject2D3D assume fit in?  I don't see any reference to z=1 in the code.

I was able to accomplish what I wanted using the algorithm I described earlier (iteratively solving for the inverse of the project3D2D).

Thank you for your kind efforts, and thanks for being so quick to respond to the forum questions.

I put a video describing my question here:

http://www.youtube.com/watch?v=9IcNArtiG_c

I don't know (or understand) where my square is in camera space.  I know that it is at z=0 in world space.  And I never change it (I only move the camera).

in the video, when you see the image moving, I'm only moving the camera position and orientation (not moving the axes which remain fixed in the z=0 plane in world space)

Note:  what I want is the inverse of the the project3D2D(x,y,z=0) function.  It maps a point (x,y,z=0) in the world space to the screen - for any camera position/orientation.  I want the inverse of this.

Sorry for the confusing description of my problem.

Tom

My object is a square in the z=0 plane.  (as described in previous post).  My camera is at some arbitrary position (cx,cy,cz).

When I look at this square on the screen, it looks like a diamond shape (in general).

As I move the mouse around on the screen, I want to know where I am in the rectangle.  ie. I want the mapping from 2D mouse coordinates to the 3D world coordinates at z=0.

Since I understand project3D2D method (but don't  understand your reproject2D3D) I am able to accomplish what I want with the following algorithm:

1)  initialize my "guess" of world coordinates to be:  worldGuess=(0,0,0)
2)  calculate the screen position of my "guess" using project3D2D(worldGuess)
3)  calculate the error in screen position between my "guess" and my actual mouse position
3a) if error is less than 1 pixel then I'm done
4)  calculate the gradients in the screen position for "world delta" vectors (0.001,0,0) and (0,0.001,0)  (ie. figure out how my screen coordinates change for a small movement in the world coordinates)
5)  refine my guess based on the gradients and the error
6)  goto 2)

ie. I used Newton's method to find the solution
ie. successive approximation

This works, but is obviously not optimal (because it iterates).

I think the reproject2D3D should be able to do it somehow.  I just don't understand how to do it.

Thanks for the quick reply.  I'm still confused.  In my situation it is possible that there isn't even any objects at z=0 yet I still want to know what the coordinate is.  (ie. the calcMinDistance wouldn't work because there is no object getting hit by the ray).

Are you saying that the reproject2D3DWS method returns a vector who's direction and length is from camera origin to the mouse position (assuming the mouse position is at z=1)?  If this is true, maybe somehow I could use this information to calculation where the mouse is at z=0?

One thing I tried was to multiply the mouse coordinates by 2.  I think I need to do this, but it didn't solve my problem.

For whatever reason, the project3D2D works fine.  Given a point in 3D, I can figure out where it is on the 2D screen. 

I was thinking I could do the mapping from 2D to 3D myself (knowing the mapping of the origin, a point at 1,0,0 and a point at 0,1,0), but the perspective rendering make it difficult.

Let me try to better explain what I'm trying to do:

Suppose I have a rectangle in the z=0 plane with the 4 points:  (0,0,0),(1,0,0),(1,1,0),(0,1,0).  (in world space)
I could be viewing this rectangle from any direction.  (example, put the camera at position (1,1,1) pointing to the origin)

When the mouse moves over this rectangle, I want to know the coordinates within the rectangle.  (ie. map a point in on the 2D screen to a point in the 3D world space at z=0).

I'm not trying to "pick" anything.  I just want to get the world coordinates (at plane z=0) from the mouse coordinates.  Seems like the method I need to use would be Interact2D.reproject2D3DWS, however it doesn't give me ability to choose z=0.  (it assumes z=1).  Even with the assumption of z=1, the result seems incorrect.

From other examples in the forums it seems that the result of reproject2D3DWS gives a "direction" and not a "position".

Here's my code:

    public void mouseMoved(MouseEvent e) {
        double x=currentMouseX=e.getX();
        double y=currentMouseY=e.getY();
        System.out.printf("Mouse screen: %d %d\n",currentMouseX,currentMouseY);
        System.out.println("Mouse position: "+Interact2D.reproject2D3DWS(world.getCamera(), buffer, currentMouseX, currentMouseY).toString());
}

It just gives me nonsense.

Other information:
I adjusted my camera position to be at (6,6,6) with camera orientation pointing to the origin, and camera "up" being the vector normal to the camera orientation such that "z" is up.

Tom