Vuforia + jPCT-AE

[kelmer]

I know there are a few posts regarding this integration already, but they are all of a point much more advanced that the one I am

I am trying to integrate the basic ImageTargets from Vuforia with JPCT-AE's simple HelloWorld demo, but I am facing some problems.

First I started from Vuforia's code, using their extension of GlSurfaceView. Then on ImageRendered I pasted the code from JPCT's hello world, like this:

Code Select Expand

public ImageTargetsRenderer(ImageTargets activity) {

this.mActivity = activity;

world = new World();
world.setAmbientLight(20, 20, 20);
sun = new Light(world);
sun.setIntensity(250, 250, 250);

// Create a texture out of the icon...:-)
Texture texture = new Texture(BitmapHelper.rescale(BitmapHelper.convert(mActivity.getResources().getDrawable(R.drawable.ic_launcher)), 64, 64));
TextureManager.getInstance().addTexture("texture", texture);

cube = Primitives.getCube(10);
cube.calcTextureWrapSpherical();
cube.setTexture("texture");
cube.strip();
cube.build();

world.addObject(cube);

Camera cam = world.getCamera();
cam.moveCamera(Camera.CAMERA_MOVEOUT, 50);
cam.lookAt(cube.getTransformedCenter());

SimpleVector sv = new SimpleVector();
sv.set(cube.getTransformedCenter());
sv.y -= 100;
sv.z -= 100;
sun.setPosition(sv);
MemoryHelper.compact();
}

/** Called when the surface is created or recreated. */
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
DebugLog.LOGD("GLRenderer::onSurfaceCreated");

// Call native function to initialize rendering:
initRendering();

// Call QCAR function to (re)initialize rendering after first use
// or after OpenGL ES context was lost (e.g. after onPause/onResume):
QCAR.onSurfaceCreated();
}

/** Called when the surface changed size. */
public void onSurfaceChanged(GL10 gl, int width, int height) {
DebugLog.LOGD("GLRenderer::onSurfaceChanged");

// Call native function to update rendering when render surface
// parameters have changed:
updateRendering(width, height);

// Call QCAR function to handle render surface size changes:
QCAR.onSurfaceChanged(width, height);

if (fb != null) {
fb.dispose();
}
fb = new FrameBuffer(gl, width, height);

}

/** The native render function. */
public native void renderFrame();

/** Called to draw the current frame. */
public void onDrawFrame(GL10 gl) {

fb.clear(back);
world.renderScene(fb);
world.draw(fb);
fb.display();

if (!mIsActive)
return;

// Update render view (projection matrix and viewport) if needed:
mActivity.updateRenderView();
// Call our native function to render content
renderFrame();

}
:
But when I launch my app I get an ArrayIndexOutOfBounds error on the world.renderScene method. If I comment vuforia's mGlView.init() code:

Code Select Expand

public void init(int flags, boolean translucent, int depth, int stencil) {
// By default GLSurfaceView tries to find a surface that is as close
// as possible to a 16-bit RGB frame buffer with a 16-bit depth buffer.
// This function can override the default values and set custom values.

// Extract OpenGL ES version from flags
mUseOpenGLES2 = (flags & QCAR.GL_20) != 0;

// By default, GLSurfaceView() creates a RGB_565 opaque surface.
// If we want a translucent one, we should change the surface's
// format here, using PixelFormat.TRANSLUCENT for GL Surfaces
// is interpreted as any 32-bit surface with alpha by SurfaceFlinger.

DebugLog.LOGI("Using OpenGL ES " + (mUseOpenGLES2 ? "2.0" : "1.x"));
DebugLog.LOGI("Using " + (translucent ? "translucent" : "opaque") + " GLView, depth buffer size: " + depth + ", stencil size: " + stencil);

// If required set translucent format to allow camera image to
// show through in the background
if (translucent) {
this.getHolder().setFormat(PixelFormat.TRANSLUCENT);
}
// Setup the context factory for 1.x / 2.0 rendering
setEGLContextFactory(new ContextFactory());

// We need to choose an EGLConfig that matches the format of
// our surface exactly. This is going to be done in our
// custom config chooser. See ConfigChooser class definition
// below.
setEGLConfigChooser(translucent ? new ConfigChooser(8, 8, 8, 8, depth, stencil) : new ConfigChooser(5, 6, 5, 0, depth, stencil));

}

/** Creates OpenGL contexts. */
private static class ContextFactory implements GLSurfaceView.EGLContextFactory {
private static int EGL_CONTEXT_CLIENT_VERSION = 0x3098;

public EGLContext createContext(EGL10 egl, EGLDisplay display, EGLConfig eglConfig) {
EGLContext context;
if (mUseOpenGLES2) {
DebugLog.LOGI("Creating OpenGL ES 2.0 context");
checkEglError("Before eglCreateContext", egl);
int[] attrib_list_gl20 = { EGL_CONTEXT_CLIENT_VERSION, 2, EGL10.EGL_NONE };
context = egl.eglCreateContext(display, eglConfig, EGL10.EGL_NO_CONTEXT, attrib_list_gl20);
} else {
DebugLog.LOGI("Creating OpenGL ES 1.x context");
checkEglError("Before eglCreateContext", egl);
int[] attrib_list_gl1x = { EGL_CONTEXT_CLIENT_VERSION, 1, EGL10.EGL_NONE };
context = egl.eglCreateContext(display, eglConfig, EGL10.EGL_NO_CONTEXT, attrib_list_gl1x);
}

checkEglError("After eglCreateContext", egl);
return context;
}

public void destroyContext(EGL10 egl, EGLDisplay display, EGLContext context) {
egl.eglDestroyContext(display, context);
}
}

Then my app runs but I get "called unimplemented OpenGL ES API" errors constnatly.

Am I approaching anything badly here?

[EgonOlsen]

You can't expect gl to run if you don't initialize it. Might be that if you omit that initialization that the gl context is null, which makes jPCT-AE think that you want to use GLES 2.0, which will then fail because it hasn't been initialized. About the exception you are getting: It's always helpful to post the actual stack trace. Errors of that kind usually happen if another thread than the rendering thread fiddles around with the world in parallel to rendering, but i'm not sure where this should happen in this case.

[kelmer]

I do initialize it, like this:

Code Select Expand

mGlView = new QCARSampleGLView(this);

getWindow().setFormat(PixelFormat.TRANSLUCENT);

mRenderer = new ImageTargetsRenderer(this);
mGlView.setRenderer(mRenderer);


the init() method of QCARSampleGlView just sets the config.

Anyway here's the stacktrace:

Code Select Expand

01-17 17:40:02.776: I/jPCT-AE(24147): OpenGL version:    OpenGL ES 2.0 build 1.8@905891
01-17 17:40:02.776: I/jPCT-AE(24147): OpenGL renderer initialized (using 0 texture stages)
01-17 17:40:02.776: E/libEGL(24147): called unimplemented OpenGL ES API
01-17 17:40:02.776: E/libEGL(24147): called unimplemented OpenGL ES API
01-17 17:40:02.776: E/libEGL(24147): called unimplemented OpenGL ES API
01-17 17:40:02.776: I/jPCT-AE(24147): [ 1358440802791 ] - WARNING: State: 0/0/0/0/0/0/0
01-17 17:40:02.831: W/dalvikvm(24147): threadid=13: thread exiting with uncaught exception (group=0x40e57930)
01-17 17:40:02.839: E/AndroidRuntime(24147): FATAL EXCEPTION: GLThread 46650
01-17 17:40:02.839: E/AndroidRuntime(24147): java.lang.ArrayIndexOutOfBoundsException: length=0; index=0
01-17 17:40:02.839: E/AndroidRuntime(24147): at com.threed.jpct.CompiledInstance._fill(CompiledInstance.java:1206)
01-17 17:40:02.839: E/AndroidRuntime(24147): at com.threed.jpct.CompiledInstance.fill(CompiledInstance.java:746)
01-17 17:40:02.839: E/AndroidRuntime(24147): at com.threed.jpct.Object3DCompiler.compile(Object3DCompiler.java:148)
01-17 17:40:02.839: E/AndroidRuntime(24147): at com.threed.jpct.World.compile(World.java:1951)
01-17 17:40:02.839: E/AndroidRuntime(24147): at com.threed.jpct.World.renderScene(World.java:1046)
01-17 17:40:02.839: E/AndroidRuntime(24147): at com.qualcomm.QCARSamples.ImageTargets.ImageTargetsRenderer.onDrawFrame(ImageTargetsRenderer.java:134)
01-17 17:40:02.839: E/AndroidRuntime(24147): at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1516)
01-17 17:40:02.839: E/AndroidRuntime(24147): at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1240)

[EgonOlsen]

You are using GLES 2.0. You have to use the constructor in FrameBuffer for it (the one without the gl context). At least the code that you posted above uses the wrong one.

[kelmer]

Yes, that was it. Thanks!

Is it possible to make the background transparent? Using a RGBA color when calling clear() does not do it.

[EgonOlsen]

I think i dimly remember some transparent/opaque setting in either Activity or GLSurfaceView...

[kelmer]

Yes, that worked

[kelmer]

just one more question, I read here: http://www.jpct.net/forum2/index.php?topic=3005.0

that the modelview matrix needs some transformations before applying it in JPCT-AE.

I just took the matrix and directly applied it to the camera like this:

Code Select Expand


com.threed.jpct.Matrix _cameraMatrix = new com.threed.jpct.Matrix();;
SimpleVector _cameraPosition = new SimpleVector();

_camer4aPosition.set(modelViewMat[3],modelViewMat[7],modelViewMat[11]);
modelViewMat[3] = modelViewMat[7] = modelViewMat[11] = 0;

_cameraMatrix.setDump(modelViewMat);
//_cameraMatrix = _cameraMatrix.invert();

cam.setBack(_cameraMatrix);
cam.setPosition(_cameraPosition);


And seems to be working fine. Maybe I am missing something here? Do I actually need to transform the matrix in any way?

[EgonOlsen]

I would have expected that you have to do some conversions to transform between the different coordinate systems, but if it works this way, i see no need to bother with it unless it becomes a problem.

[kelmer]

I've got a little question. I perfected the integration with QCAR's engine, but I am faced with a doubt.

If I create a 30 units big cube and don't move it at all, it looks like the cube lies "beneath" the marker, so if I rotate the camera around it you can see it rotate as though it was inside the marker instead of above it. Obviously this is difficult to see (and probably to understand without seein it), but to clear things a bit it is as though the marker is positioned at (0, 0, 0) facing Z positive direction, and the cube lies at (0, 0, 0) but extends towards Z negative direction (or maybe it is the other way around).

For the cube I can just move it 30 points towards the positive z direction and I'm done, but with models I wouldn't know how much to move them so they lie perfectly above the marker. How can I solve this?

Does this all make sense to you? I'm not very clear as to how JPCT positions objects, or what is the difference between center, origin and transformed center...

[EgonOlsen]

Maybe you can post one or two screen shots? I've no idea what exactly you are talking about...

[kelmer]

Okay, ionstead of a screenshot (which would make it even harder to understand) I am posting a scheme.

This is my scene from a sideview:



[attachment deleted by admin]

[EgonOlsen]

This depends on the coordinates of your model in object space. You can try to create a generic solution by looking at the bounding box (can be obtained from an Object3D's mesh after build() and translating your model accordingly.

[kelmer]

Okay, I checked with a plane and it's more like I'm showing in this attachment.

I understand your answer, but given that I'm using a cube (Primitives.getCube()), does this mean the cube is expanded from the origin in all directions? It's not created, like, towards positive x,y, and z axis?

[attachment deleted by admin]

[kelmer]

Here's another pic.

[attachment deleted by admin]