Jpct-AE Vuforia integration Portrait mode

[MSL]

HI..
I was able to integrate jpct and vuforia following the wiki article. Everything works fine in landscape mode. However, in portrait mode the model is not positioned properly(model not sticking to marker) also model moves when the camera is moved.
if camera moves UP, model moves RIGHT
if camera moves DOWN, model moves LEFT
if camera moves LEFT, model moves UP
if camera moves RIGHT, model moves DOWN

In many posts people talk about this problem but no one has provided proper solution for this or may be I am missing something.
Can anyone help me by pointing out the changes that is required to be done to the wiki article on Integration so that portrait mode can be handled?

[EgonOlsen]

Judging from some threads here and on Stackoverflow, people have got this working...just nobody posted the actual solution...

Have you tried to rotate the matrix +-90° around Z in protrait mode? AFAIK, Vuforia always *thinks* in landscape mode, so the matrices are in landscape mode. So it should help to rotate them and maybe switch the fov-settings as well (in cause you haven't already).

[MSL]

Hi.. Thanks for your reply.. Yes..I did try rotating the modelViewMatrix by 90 along Z-axis.
if (isActivityInPortraitMode){
        SampleUtils::rotatePoseMatrix (90.0f, 0, 0, 1.0f, &modelViewMatrix.data[0]);
        }
        SampleUtils::rotatePoseMatrix(180.0f, 1.0f, 0, 0,
                       &modelViewMatrix.data[0]);
Theoritically, this should work but it seems, this solution is not working. I am unable to determine where its going wrong.

This is code from Rajawali3D integration with Vuforia.. This is working properly in both Landscape and portrait mode:
      if (isActivityInPortraitMode)
         Utils::rotatePoseMatrix(90.0f, 0, 1.0f, 0,
               &modelViewMatrix.data[0]);
      Utils::rotatePoseMatrix(-90.0f, 1.0f, 0, 0, &modelViewMatrix.data[0]);
There is different logic in Rajawali. I have attached cpp and java file of Rajawali in case you want more info:
Too sad 2 people on Vuforia forum asked me to move to Rajawali bcz of this problem .. but I want to use jpct. Once this problem is resolved I will make a complete project on Integration for everyone use

[attachment deleted by admin]

[EgonOlsen]

Switching the engine because of some issue that's not even part of the engine seems a little strange to me. Anyway...i think that the problem might be related to the order of the rotations. Keep in mind that "first z, then x" isn't the same as "first x, then z". I would try something like

Code Select Expand


SampleUtils::rotatePoseMatrix(180.0f, 1.0f, 0, 0, &modelViewMatrix.data[0]);
if (isActivityInPortraitMode){
        SampleUtils::rotatePoseMatrix (-90.0f, 0, 0, 1.0f, &modelViewMatrix.data[0]);
}


instead. And if that doesn't work (most likely it won't, because it's just some brain fart of mine), you can either work it out on paper or, which is what i would do, try different combinations of angles and axes. It might help to keep the two branches separate IMHO, i.e. have one set of rotations for landscape and another one for portrait. That way, you can fiddle around with one without changing the other.

[Wolf17]

  I really do not have any idea about the Vuforia stuff . But a look into the wiki page about integration of jpct with vuforia , I got this thing -

Code Select Expand

public void updateCamera() {
Matrix m = new Matrix();
m.setDump(modelViewMat);
        cam.setBack(m);
}

  I am not sure , but maybe you can experiment tweaking the values of the matrix "m" before applying it to the camera .

Code Select Expand

public void updateCamera() {
Matrix m = new Matrix();
        m.setDump(modelViewMat);
        //your adjustments for " m" here
        cam.setBack(m);
}

(Sorry if I am posting something wrong ! And please correct me if anybody feels so ! )

[MSL]

Hi all,

Finally I was able to solve this issue... I got the answer in this post in Vuforia forum
https://developer.vuforia.com/forum/android/integrating-jpct-ae-vuforia

I am posting the code here so that anyone facing the same issue can use this:

JNI code:

Code Select Expand


-------------------------------------------------------------------------------------------------------------------------------
JNIEXPORT void JNICALL
Java_com_qualcomm_QCARSamples_ImageTargets_ImageTargetsRenderer_renderFrame(JNIEnv *env, jobject obj)
{
// Get the class out of Activity
jclass activityClass = env->GetObjectClass(obj);

// Get the method from above class
jmethodID cameraPositionMethod = env->GetMethodID(activityClass, "cameraPositionMatrix", "([F)V");

    // Clear color and depth buffer
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    // Get the state from QCAR and mark the beginning of a rendering section
    QCAR::State state = QCAR::Renderer::getInstance().begin();
   
    // Explicitly render the Video Background
    QCAR::Renderer::getInstance().drawVideoBackground();
       
    jfloatArray cameraViewArray = env->NewFloatArray(16);

    // Did we find any trackables this frame?
    for(int tIdx = 0; tIdx < state.getNumTrackableResults(); tIdx++)
    {
        // Get the trackable:
        const QCAR::TrackableResult* result = state.getTrackableResult(tIdx);
        const QCAR::Trackable& trackable = result->getTrackable();
        QCAR::Matrix44F modelViewMatrix = QCAR::Tool::convertPose2GLMatrix(result->getPose());

        QCAR::Matrix44F inverseMV = SampleMath::Matrix44FInverse(modelViewMatrix);
        QCAR::Matrix44F invTranspMV = SampleMath::Matrix44FTranspose(inverseMV);

            env->SetFloatArrayRegion (cameraViewArray, 0, 16, invTranspMV.data);
            env->CallVoidMethod (obj, cameraPositionMethod, cameraViewArray);
}
-------------------------------------------------------------------------------------------------------------------------------

Java Code:
public void cameraPositionMatrix(float matrix[])
    {
    mCameraViewMat = matrix;
   
    if(mCameraViewMat!=null){
   
            Configuration config = mActivity.getResources().getConfiguration();

            switch (config.orientation)
            {
            case Configuration.ORIENTATION_PORTRAIT:
        mCam_Pos_x = mCameraViewMat[12];
        mCam_Pos_y = mCameraViewMat[13];
        mCam_Pos_z = mCameraViewMat[14];
       
        mCam_Dir_x = mCameraViewMat[8];
        mCam_Dir_y = mCameraViewMat[9];
        mCam_Dir_z = mCameraViewMat[10];
       
        mCam_Up_x = -mCameraViewMat[0];
        mCam_Up_y = -mCameraViewMat[1];
        mCam_Up_z = -mCameraViewMat[2];

                break;

            case Configuration.ORIENTATION_LANDSCAPE:
    mCam_Pos_x = mCameraViewMat[12];
    mCam_Pos_y = mCameraViewMat[13];
    mCam_Pos_z = mCameraViewMat[14];
   
    mCam_Dir_x = mCameraViewMat[8];
    mCam_Dir_y = mCameraViewMat[9];
    mCam_Dir_z = mCameraViewMat[10];
   
    mCam_Up_x = -mCameraViewMat[4];
    mCam_Up_y = -mCameraViewMat[5];
    mCam_Up_z = -mCameraViewMat[6];

                break;
            case Configuration.ORIENTATION_UNDEFINED:
            default:
                break;
            }

    mCameraDirection.set(mCam_Dir_x, mCam_Dir_y, mCam_Dir_z);
    mCameraUp.set(mCam_Up_x, mCam_Up_y, mCam_Up_z);
    }
    }

    public void updateCamera()
    {
    if(mCameraViewMat!=null){
mCam.setPosition(mCam_Pos_x, mCam_Pos_y, mCam_Pos_z);
mCam.setOrientation(mCameraDirection, mCameraUp);
    }
    mCam.setFOV(mFov);
    mCam.setYFOV(mFovY);
    }

    public void onDrawFrame(GL10 gl)
    {
        if (!mIsActive)
            return;

        // Update render view (projection matrix and viewport) if needed:
        mActivity.updateRenderView();

        // Call our native function to render content
        renderFrame();
        updateCamera();
        mWorld.renderScene(mfb);
        mWorld.draw(mfb);
        mfb.display();
    }
-------------------------------------------------------------------------------------------------------------------------------


May be in the coming week I will try to make a Android project and will put it in gitHub.
Thanks everyone

[EgonOlsen]

Could you add that finding to the wiki page? That would be great. I can create you an account...just let me know.

[MSL]

ya sure... we can add it to wiki so that it will cover Portrait mode changes also. However, I have the solution using the 2nd method (i.e when we obtain the camera position, direction) not for 1st method(i.e., rotating the pose matrix). So I am not sure if some one is using 1st method then how can portrait mode be handled.

[EgonOlsen]

That doesn't matter IMHO. One working solution is better than none. I'll create an account later with the email address that you have used to register here.

[EgonOlsen]

Have you received the wiki account?

[MSL]

HI..
yes.. I did receive it .. Thanks
I will update the article tonight if possible or tomorrow.