assign a GL texture to an Object3D

[raft]

solved the problem turned out to be a threading issue, a race condition between the camera thread and the GL thread.

To me, everything looks fine now.

yes, this setup is working. either with  "* vertexColor" or not.

camera image set as texture to the plane. seems as this is a NPOT texture, no need for UV adjustment.


should I do anything special to cleanup this texture? how does TextureManager.flush() and freeMemory() effect this?

thx:)

[raft]

sky maze's trailer playing on the plane


edit: updated the screenshot with a non-debug one

[EgonOlsen]

Have you tried to apply it to something more interesting, like a sphere or some animated mesh?

[raft]

no, but it should work. i dont see any reason for the opposite.

possibly because of quad cores, it has almost no effect on FPS

[raft]

if you or someone else has an interesting idea, i'm more than ready to listen. we'll possibly make a demo depending on all this AR stuff but i'm not sure about the scenario yet.

maybe open the detected marker (indeed this is called markerless i suppose) as a door to inside of screen into a tunnel to give some sense of 3d? or even a creature climbing up that tunnel?

[raft]

Skyrim360's panoramic video playing in MediaPlayer and assigned as a texture to an inverted sphere. then that sphere is rendered via jPCT on top of Google cardboard api. the results are below. by wearing a Google cardboard like glasses, one can interactively watch the movie and look around in the movie.

as it's the video gives a feeling of 3d. it will be even better if the video was recorded stereoscopically, ie: for both eyes separately.

some scene:


looking down:

[EgonOlsen]

Interesting. Is there anything special to do when combining jPCT and the card board API?

[Irony]

Hey raft,
I am trying to achieve something similiar to you: streaming mediaplayer video to a texture. It seems I am stuck exactly here:

argh you're right. texture was never initialized.

Can you give me a hint how you are doing the initialization now?
I create the texture object, set up the frameavailable listener, etc., but all I see is some random texture instead of the video ouput.
The playback itself works, as I can view the video in fullscreen, but it's not visible on JPCT's texture.

[raft]

Interesting. Is there anything special to do when combining jPCT and the card board API?

no actually. it's pretty straightforward. cardboard API gives you eye information and wants you to render for it. it also takes care of distorting the view for rounded sides. the code for rendering each eye is something like:

Code Select Expand

Matrix camBack = world.getCamera().getBack();
camBack.setIdentity();

tempTransform.setDump(eye.getEyeView());
tempTransform.transformToGL();
camBack.matMul(tempTransform);

world.getCamera().setBack(camBack);

frameBuffer.clear(RGBColor.BLACK);

world.renderScene(frameBuffer);
world.draw(frameBuffer);

frameBuffer.display();


in its own sample, eye also has a float[16] getPerspective method which takes near and far plane distances as parameters. i dont use it. I guess jPCT handles it itself for perspective distortion, right? (although i'm not sure about this)

[raft]

Can you give me a hint how you are doing the initialization now?
I create the texture object, set up the frameavailable listener, etc., but all I see is some random texture instead of the video ouput.
The playback itself works, as I can view the video in fullscreen, but it's not visible on JPCT's texture.

you should call TextureRender.surfaceCreated method at onSurfaceChanged method or similar. you can find a link to TextureRender class in earlier posts of this thread.

these are in my onSurfaceChanged method: (indeed externalTexture is created in onCreate but that doesnt matter that much)

Code Select Expand


textureRenderer.surfaceCreated();

surfaceTexture = new SurfaceTexture(textureRenderer.getTextureId());

Surface surface = new Surface(surfaceTexture);
mediaPlayer.setSurface(surface);
//surface.release();

Texture externalTexture = new Texture(32, 32);
externalTexture.setExternalId(textureRenderer.getTextureId(), GLES11Ext.GL_TEXTURE_EXTERNAL_OES);

TextureManager.getInstance().addTexture("video_texture", externalTexture);

surfaceTexture.setOnFrameAvailableListener(new SurfaceTexture.OnFrameAvailableListener() {

@Override
public void onFrameAvailable(SurfaceTexture surfaceTexture) {
synchronized(someLock) {
            frameAvailable = true;
}
}
});


than in draw method:

Code Select Expand

synchronized(someLock) {
    if (frameAvailable) {
        int error = GLES20.glGetError();
if (error != 0) {
    Logger.w("gl error before updateTexImage" + error + ": " + GLU.gluErrorString(error));
}
surfaceTexture.updateTexImage();
frameAvailable = false;
//            System.out.println("updateTexImage");
        }
}


note the synchronization above is very important. because onFrameAvailable is not called again if you dont consume the last frame by calling surfaceTexture.updateTexImage()

[Irony]

Thank you! At first glance, I am doing all of this already. It's probably some tiny bit of code elsewhere that's not right... *putting on glasses, brewing some more coffee*

[raft]

You can download a working sample here

[Irony]

Thank you so much, I will try it out as soon as I find the time.

[EgonOlsen]

Nice! The video doesn't play on every run on my Nexus 4, but don't see any suspicious log output either. However, going back and selecting it again usually fixes the problem. Is this zip file here to stay? If so, do you mind if i link to in the wiki as an example?

[raft]

sure np.

dont have any idea why video doesnt start sometimes.