Gyroscope driven camera + touch correction - reproject2D3D not corresponding

[moyosa]

I'm building a 360 camera in a 3d world. Now the gyroscope is updating the camera, so it's like an AR kind of camera.

Now I want to set an offset on the x and y axis. When I start the 3d world in landscape. For example left and right is working.
But when I rotate the device to portrait, x and y should be reversed. But reproject2D3D is still giving me the same results.
So now to scroll left and right, you need to scroll up and down (makes sense?)

Seems like unproject/reproject is not working for me here.

Code Select Expand

SimpleVector startLocation = new SimpleVector(Interact2D.reproject2D3D(world.getCamera(), fb, (int)mPreviousX2, (int) mPreviousY2)).normalize();
                SimpleVector currentLocation = new SimpleVector(Interact2D.reproject2D3D(world.getCamera(), fb, (int) me.getX(), (int) me.getY())).normalize();

double alpha = Math.acos(currentLocation.y) - Math.acos(startLocation.y);
double beta = Math.atan2(currentLocation.z, currentLocation.x) - Math.atan2(startLocation.z, startLocation.x);


double tRotX = rotX - MathUtil.toDegrees((float)beta);
rotX = tRotX%360;


PS I've got this working in iOS, so should be good.

[EgonOlsen]

Have you actually tested if the values returned by me.getX() and me.getY() are the ones that you expect?

[moyosa]

Yes I did, and they remain as if in the same orientation landscape. Because i locked it in landscape.

But my camera is always leveled horizontally. So when I go from landscape to portrait my camera adjusts it's never tilted or upside down.

1 would expect that unprojecting the coordinates it would give me different results.?

[EgonOlsen]

I'm confused now...can you make a drawing or something? I'm not sure what is locked and what rotates in your case...

[moyosa]

My activity/view is locked to landscape. My gyroscope controls the camera. So it's like a viewport. when you move your device around you, you can look around the 3dworld.

But I want to be able to also control the camera with touch so I need an offset. Normally you can use the x and y of the touch point. But because you can also rotate the device to portrait for example, you cannot rely on the x-y touch point. So I need to unproject the 2 d coordinates to 3d, to give me the the right x and y in the view and not the ones of the real world.

Hope that makes more sense?

Thanks!

[EgonOlsen]

Kind of, but i'm still not sure what the actual problem is. reproject2D3D will give you the coordinates in camera space. Maybe you want them in world space instead ( http://www.jpct.net/jpct-ae/doc/com/threed/jpct/Interact2D.html#reproject2D3DWS(com.threed.jpct.Camera, com.threed.jpct.FrameBuffer, int, int) )?

[moyosa]

Now that seems to be working, But I have like a stereoscopic view, and using new FrameBuffer(width/2, height); with first com.threed.jpct.Config.viewportOffsetX = 0.0f; and then com.threed.jpct.Config.viewportOffsetX = 1.0f;

So I can draw the same scene but using a left and right eye.

For example in portrait, left en right is working fine, but in landscape it only calculates on the right view, so when I move over the left eye the offset seems to be slowing down.

SimpleVector startLocation = new SimpleVector(Interact2D.reproject2D3DWS(world.getCamera(), fb, (int) mPreviousX2, (int) mPreviousY2, 10.f));
                    SimpleVector currentLocation = new SimpleVector(Interact2D.reproject2D3DWS(world.getCamera(), fb, (int) me.getX(), (int) me.getY(), 10.f));

I've tried using fb = new FrameBuffer(width, height); for the full use of the view. But it doesn't seem to work.

[EgonOlsen]

What?... Sorry, but i don't really understand...do you have a screen shot of what's actually working and what's not?

[moyosa]

Haha yeah sorry, will do. Just wondering is there a way to use gluUnProject

All I need is the modelview and projection.

[moyosa]

Probably some unnecessary steps, but gives me the result I need

Matrix temp;
                    temp = world.getCamera().getProjectionMatrix(touchfb).cloneMatrix();
                    temp.transformToGL();
                    float [] projection = temp.getDump();
                    temp =world.getCamera().getBack().cloneMatrix();
                    temp.transformToGL();
                    float [] modelView = temp.getDump();
                    GLU.gluUnProject(mPreviousX2, viewport[3]-mPreviousY2, 1, modelView, 0, projection, 0, viewport, 0, startcoords, 0);
                    GLU.gluUnProject(me.getX(), viewport[3]-me.getY(), 1, modelView, 0, projection, 0,viewport, 0, currentcoords, 0);

[EgonOlsen]

...whatever these results are... I'm glad that you've solved it. I bet that there is a solution without falling back to gl calls, but as long as it works, as is well...