[Tips] Android, augmented reality 3D with JPCT + Camera.

[pritom057]

Is it possible Android Camera and JPCT-AE as a render that overlays the camera???
I don't think so.When I initialize

World world = new World();

it render a black screen itself...
I haven't seen any function function in World class that I can change it black color to make it transparent.
There is function in FrameBuffer Class
public void blit(int[] src,....... ,boolean transparent)

by this I can have the transparent background....But the problem I am facing that is how can i get the first parameter of this function the current renderer texture??

[EgonOlsen]

You can simply clear the framebuffer with a color that has an alpha value assigned (http://www.jpct.net/jpct-ae/download/alpha/doc/com/threed/jpct/FrameBuffer.html#clear(com.threed.jpct.RGBColor)). That will give you the transparent background if everything else is setup correctly.

[pritom057]

EgonOlsen Thanks for your reply...
But still I am not able to make transparent background
I will be very glad if you fix my problem.....

Into the onDrawFrame function I have Written the following code

fb.clear();
world.renderScene(fb);
world.draw(fb);
fb.blit(fb.getPixels(), this.widht, this.height, 0, 0, this.widht, this.height, this.widht, this.height, true);
blitNumber(lfps, 5, 5);
fb.display();

still I am not able to find any solution...

My total code is

Code Select Expand

package com.HelloAndroid;


import java.security.PublicKey;

import javax.microedition.khronos.egl.EGL10;
import javax.microedition.khronos.egl.EGLConfig;
import javax.microedition.khronos.egl.EGLDisplay;
import javax.microedition.khronos.opengles.GL10;

import android.app.Activity;
import android.content.res.Resources;
import android.graphics.Color;
import android.graphics.PixelFormat;
import android.opengl.GLSurfaceView;
import android.os.Bundle;
import android.util.FloatMath;
import android.util.Log;
import android.view.KeyEvent;
import android.view.MotionEvent;

import com.threed.jpct.Camera;
import com.threed.jpct.Config;
import com.threed.jpct.FrameBuffer;
import com.threed.jpct.GenericVertexController;
import com.threed.jpct.Interact2D;
import com.threed.jpct.Light;
import com.threed.jpct.Loader;
import com.threed.jpct.Logger;
import com.threed.jpct.Object3D;
import com.threed.jpct.Primitives;
//import com.threed.jpct.R;
import com.threed.jpct.RGBColor;
import com.threed.jpct.SimpleVector;
import com.threed.jpct.Texture;
import com.threed.jpct.TextureManager;
import com.threed.jpct.World;

/**
* A simple demo. This shows more how to use jPCT-AE than it shows how to write
* a proper application for Android, because i have no idea how to do this. This
* thing is more or less a hack to get you started...
*
* @author EgonOlsen
*
*/
public class HelloAndroid extends Activity {

private GLSurfaceView mGLView;
private MyRenderer renderer;
private FrameBuffer fb = null;
private World world = null;
private int move = 0;
private float turn = 0;
private boolean paused = false;
public Object3D testObj;

private float touchTurn = 0;
private float touchTurnUp = 0;

private float xpos = -1;
private float ypos = -1;

protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
mGLView = new GLSurfaceView(this);

mGLView.setEGLConfigChooser(8, 8, 8, 8, 16, 0);

renderer = new MyRenderer();
mGLView.setRenderer(renderer);
mGLView.getHolder().setFormat(PixelFormat.TRANSLUCENT);
setContentView(mGLView);
}

@Override
protected void onPause() {
paused = true;
super.onPause();
mGLView.onPause();
}

@Override
protected void onResume() {
paused = false;
super.onResume();
mGLView.onResume();
}

protected void onStop() {
renderer.stop();
super.onStop();
}

public boolean onTouchEvent(MotionEvent me) {

if (me.getAction() == MotionEvent.ACTION_DOWN) {
xpos = me.getX();
ypos = me.getY();
//testObj = renderer.PickObj(xpos, ypos);
//Log.d("hello", testObj.getName());
return true;
}

if (me.getAction() == MotionEvent.ACTION_UP) {
xpos = -1;
ypos = -1;
touchTurn = 0;
touchTurnUp = 0;
return true;
}

if (me.getAction() == MotionEvent.ACTION_MOVE) {
float xd = me.getX() - xpos;
float yd = me.getY() - ypos;

xpos = me.getX();
ypos = me.getY();

touchTurn = xd / 100f;
touchTurnUp = yd / 100f;
return true;
}
return super.onTouchEvent(me);
}

public boolean onKeyDown(int keyCode, KeyEvent msg) {

if (keyCode == KeyEvent.KEYCODE_W) {
move = 2;
return true;
}

if (keyCode == KeyEvent.KEYCODE_S) {
move = -2;
return true;
}

if (keyCode == KeyEvent.KEYCODE_D) {
turn = 0.05f;
return true;
}

if (keyCode == KeyEvent.KEYCODE_A) {
turn = -0.05f;
return true;
}

return super.onKeyDown(keyCode, msg);
}

public boolean onKeyUp(int keyCode, KeyEvent msg) {
if (keyCode == KeyEvent.KEYCODE_W) {
move = 0;
return true;
}

if (keyCode == KeyEvent.KEYCODE_S) {
move = 0;
return true;
}

if (keyCode == KeyEvent.KEYCODE_D) {
turn = 0;
return true;
}

if (keyCode == KeyEvent.KEYCODE_A) {
turn = 0;
return true;
}

return super.onKeyUp(keyCode, msg);
}

protected boolean isFullscreenOpaque() {
return true;
}

class MyRenderer implements GLSurfaceView.Renderer {

private Object3D plane = null;
private Object3D tree2 = null;
private Object3D tree1 = null;
private Object3D grass = null;
private Texture font = null;

private int fps = 0;
private int lfps = 0;

private long time = System.currentTimeMillis();

private Light sun = null;
private Object3D rock = null;

private boolean stop = false;

private float ind;

private boolean deSer = false;
private int height;
private int widht;
RGBColor transc = new RGBColor(0, 0, 999);

public MyRenderer() {
Config.maxPolysVisible = 5000;
Config.farPlane = 1500;

}

public void stop() {
stop = true;
if (fb != null) {
fb.dispose();
fb = null;
}
}

public void onSurfaceChanged(GL10 gl, int w, int h) {
if (fb != null) {
fb.dispose();
}
fb = new FrameBuffer(gl, w, h);
this.widht = w;
this.height = h;
}

public void onSurfaceCreated(GL10 gl, EGLConfig config) {
TextureManager.getInstance().flush();
world = new World();
Resources res = getResources();

TextureManager tm = TextureManager.getInstance();
Texture grass2 = new Texture(res.openRawResource(R.raw.grassy));
Texture leaves = new Texture(res.openRawResource(R.raw.tree2y));
Texture leaves2 = new Texture(res.openRawResource(R.raw.tree3y));
Texture rocky = new Texture(res.openRawResource(R.raw.rocky));

Texture planetex = new Texture(res.openRawResource(R.raw.planetex));

font = new Texture(res.openRawResource(R.raw.numbers));

tm.addTexture("grass2", grass2);
tm.addTexture("leaves", leaves);
tm.addTexture("leaves2", leaves2);
tm.addTexture("rock", rocky);
tm.addTexture("grassy", planetex);


// Use the normal loaders...
plane = Primitives.getPlane(20, 30);
grass = Loader.load3DS(res.openRawResource(R.raw.grass), 5)[0];
rock = Loader.load3DS(res.openRawResource(R.raw.rock), 15f)[0];
tree1 = Loader.load3DS(res.openRawResource(R.raw.tree2), 5)[0];
tree2 = Loader.load3DS(res.openRawResource(R.raw.tree3), 5)[0];

plane.setTexture("grassy");
rock.setTexture("rock");
grass.setTexture("grass2");
tree1.setTexture("leaves");
tree2.setTexture("leaves2");
tree1.setName("HelloOBJ");
///testing collution
tree1.setCollisionMode(Object3D.COLLISION_CHECK_OTHERS);


plane.getMesh().setVertexController(new Mod(), false);
plane.getMesh().applyVertexController();
plane.getMesh().removeVertexController();



grass.translate(-45, -17, -50);
grass.rotateZ((float) Math.PI);
rock.translate(0, 0, -90);
rock.rotateX(-(float) Math.PI / 2);
tree1.translate(-50, -92, -50);
tree1.rotateZ((float) Math.PI);
tree2.translate(60, -95, 10);
tree2.rotateZ((float) Math.PI);
plane.rotateX((float) Math.PI / 2f);

plane.setName("plane");
tree1.setName("tree1");
tree2.setName("tree2");
grass.setName("grass");
rock.setName("rock");

world.addObject(plane);
//world.addObject(tree1);
//world.addObject(tree2);
//world.addObject(grass);
//world.addObject(rock);

RGBColor dark = new RGBColor(100, 100, 100);

grass.setTransparency(10);
tree1.setTransparency(0);
tree2.setTransparency(0);

tree1.setAdditionalColor(dark);
tree2.setAdditionalColor(dark);
grass.setAdditionalColor(dark);

world.setAmbientLight(20, 20, 20);
world.buildAllObjects();

sun = new Light(world);

Camera cam = world.getCamera();
cam.moveCamera(Camera.CAMERA_MOVEOUT, 250);
cam.moveCamera(Camera.CAMERA_MOVEUP, 100);
cam.lookAt(plane.getTransformedCenter());

cam.setFOV(1.5f);
sun.setIntensity(250, 250, 250);
SimpleVector sv = new SimpleVector();
sv.set(plane.getTransformedCenter());
sv.y -= 300;
sv.x -= 100;
sv.z += 200;
sun.setPosition(sv);
}

public void onDrawFrame(GL10 gl) {
try {
if (!stop) {
if (paused) {
Thread.sleep(500);
} else {
Camera cam = world.getCamera();
if (turn != 0) {
world.getCamera().rotateY(-turn);
}
if (touchTurn != 0) {
world.getCamera().rotateY(touchTurn);
touchTurn = 0;
}

if (touchTurnUp != 0) {
world.getCamera().rotateX(touchTurnUp);
touchTurnUp = 0;
}

if (move != 0) {
world.getCamera().moveCamera(cam.getDirection(), move);
}
fb.clear();
world.renderScene(fb);
world.draw(fb);
fb.blit(fb.getPixels(), this.widht, this.height, 0, 0, this.widht, this.height, this.widht, this.height, true);
blitNumber(lfps, 5, 5);
fb.display();
sun.rotate(new SimpleVector(0, 0.05f, 0), plane.getTransformedCenter());

if (System.currentTimeMillis() - time >= 1000) {
lfps = (fps + lfps) >> 1;
fps = 0;
time = System.currentTimeMillis();
}
fps++;
ind += 0.02f;
if (ind > 1) {
ind -= 1;
}
}
} else {
if (fb != null) {
fb.dispose();
fb = null;
}
}
} catch (Exception e) {
Logger.log("Drawing thread terminated!", Logger.MESSAGE);
}

}

private class Mod extends GenericVertexController {
private static final long serialVersionUID = 1L;

public void apply() {
SimpleVector[] s = getSourceMesh();
SimpleVector[] d = getDestinationMesh();
for (int i = 0; i < s.length; i++) {
d[i].z = s[i].z - (10f * (FloatMath.sin(s[i].x / 50f) + FloatMath.cos(s[i].y / 50f)));
d[i].x = s[i].x;
d[i].y = s[i].y;
}
}
}

private void blitNumber(int number, int x, int y) {
if (font != null) {
String sNum = Integer.toString(number);
for (int i = 0; i < sNum.length(); i++) {
char cNum = sNum.charAt(i);
int iNum = cNum - 48;
fb.blit(font, iNum * 5, 0, x, y, 5, 9, true);
x += 5;
}
}
}

public Object3D PickObj(float x, float y){
    SimpleVector position = new SimpleVector(Interact2D.reproject2D3D(world.getCamera(), fb, (int) x,(int) y));
    Object[] result = world.calcMinDistanceAndObject3D(world.getCamera().getPosition(), position, 10000F);
    return (Object3D)result[1];
}
}
}


[EgonOlsen]

It might help if you actually do what i had written: Clear with alpha(!) and remove that blit-call.

[pritom057]

Sir
Actually I am no getting you....Would you please explain how can i Clear with alpha(!)

Do u mean this??

Code Select Expand

RGBColor transp = new RGBColor(0,0,0,0);
//fb.clear();
fb.clear(transp);
world.renderScene(fb);
world.draw(fb);
//fb.blit(fb.getPixels(), this.widht, this.height, 0, 0, this.widht, this.height, this.widht, this.height, true);
blitNumber(lfps, 5, 5);
fb.display();

[EgonOlsen]

Yes, something like that. I've never used this myself, so i'm not sure what else you have to do (Android-side) to get it working, but on the jPCT-side, this should be all that is required. And it works: http://www.jpct.net/forum2/index.php/topic,1542.60.html

[pritom057]

EgonOlsen
Thanks for your reply sir......
But I haven't got any idea how to do this..
i am feeling frustrating...can u please help me out.

[EgonOlsen]

As said, i never did this myself, so i'm not a great help here. But this thread is all about doing what you want to do, so i guess the answer lies somewhere within here. I suggest to take the code and xml-snippets that dl zerocool posted at the beginning of the thread, make an Activity from that alone and see what happens. Your code seems to be a mix of my example and zerocools stuff and i assume that you are simply missing some important step...but it's easier to find that if you start simple and clean IMHO. For example, you are still doing this:

Code Select Expand


protected boolean isFullscreenOpaque() {
return true;
}


....which is fine for my example, but obviously makes no sense when trying to create a view that isn't opaque. I don't know if this is the root cause of the problem though.

Edit: BTW, is your mail-addr really ..@gmal.com (not gmail.com?). I keep getting bounces from my server regarding this address. Please correct it, if it's wrong. Thanx.

[pritom057]

Hi sir I have got another problem.....
I am not able to rotate camera with the android sensor orientation.....
I have done this

Code Select Expand

    SensorManager.getRotationMatrix(RTmp, I, grav, mag);
Rt=RTmp;

         tempR = world.getCamera().getBack();
tempR.setRow(0, Rt[0], Rt[1], Rt[2],0);
tempR.setRow(1, Rt[3], Rt[4], Rt[5],0);
tempR.setRow(2, Rt[6], Rt[7], Rt[8],0);
tempR.setRow(3, 0, 0, 0,1);
world.getCamera().setBack(tempR);


Can you please give me some idea how to do this??
Thanks in advance.............

[EgonOlsen]

No idea what's actually in that rotation matrix from the sensor. It might help if you do some test rotations and post the content of the resulting matrix. Most likely thisis either a row/column major issue or caused by differences in the coordinate system. What's the actual result if you execute your code?

[pritom057]

sir it shows only a black screen

[Darkflame]

pritom057 - is that my code your using? What variables did you feed into RTmp, I, grav and mag?

In the end I solved my problem with the following code;

Code Select Expand


switch (s_ev.sensor.getType()) {
       case Sensor.TYPE_ACCELEROMETER:
System.arraycopy(s_ev.values, 0, mGravs, 0, 3);
break;
       case Sensor.TYPE_MAGNETIC_FIELD:
        System.arraycopy(s_ev.values, 0, mGeoMags, 0, 3);
        break;
       default:
        return;
       }
       

       
       if (SensorManager.getRotationMatrix(mRotationM, null, mGravs, mGeoMags)){
//              Rotate to the camera's line of view (Y axis along the camera's axis)
               SensorManager.remapCoordinateSystem(mRotationM, SensorManager.AXIS_X, SensorManager.AXIS_Z, mRemapedRotationM);
               SensorManager.getOrientation(mRemapedRotationM, mOrientation);
               
               SimpleVector cameraVector = new SimpleVector();
               cameraVector.x = mOrientation[1];
               cameraVector.y = mOrientation[2];
               cameraVector.z = mOrientation[0];
               
               myworld.setCameraOrientation(cameraVector);
       }
     

setCameraOrientation() leads too;

Code Select Expand


public void setCameraOrientation(SimpleVector xyzAngles)
{


Camera worldcam = world.getCamera();

worldcam.getBack().setIdentity();

float Z = xyzAngles.z;
float Y = xyzAngles.y;
float X = xyzAngles.x;

worldcam.rotateCameraAxis(new SimpleVector(0,1,0), -Z);
worldcam.rotateCameraAxis(new SimpleVector(1,0,0), X);
worldcam.rotateCameraAxis(new SimpleVector(0,0,1), -Y);

}

I still think the setBack() method should be possible, and quicker too, but this way at least works.

[pritom057]

Thanks Darkflame

Ya I am using your code...And now it's working as u mentioned in last post...
But I am also not able to use setBack() function.
thanks....

[gman]

Thanks a lot for your initiated ideas, specially for dl.zerocool and all jPCT team. Great !!!
I'm able to overlay the 3D object on the camera now. Quite fast on my HTC wildfire 2.1.

I have tried many 3D engine. jPCT is the fastest among other engine, rendering and loading.

Nice jPCT engine
Nice team developer.

Many Thanks.

[tuanphong]

Thanks a lot for your initiated ideas, specially for dl.zerocool and all jPCT team. Great !!!
I'm able to overlay the 3D object on the camera now. Quite fast on my HTC wildfire 2.1.

I have tried many 3D engine. jPCT is the fastest among other engine, rendering and loading.

Nice jPCT engine
Nice team developer.

Many Thanks.

Have you tried libGDX framework ?
Is it fast than jPCT ?