using sensor to rotate camera

[mgrabul]

Can some one post a simple code (example) on using phone sensors to rotate jpct camera.
I tried reading the topics on the forum  but the code is not complete.I think that a coomplite code will be interesting for all the users of the forum. 
and becouse I am new to android I will realy aprriciate a simple code.

[nimo]

Hi,
here is the complete code described in my post (I just omitted the package declaration and the imports).
It's a merge between the JPCT Demo and the code found in the article I mentioned.
Because of the merging, you'll find a lot of not used code. When you'll clean it, keep in consideration that only MODUS=1, 2, 3 and 4 work (MODUS=2 is the preferred). Remember also to set the screen orientation to landscape, as described in my post.

How to use the code:
1 - Download and open in Eclipse the JPCT Demo (make it working before to continue)
2 - Substitute Demo.java with the code below
3 - Set the screen orientation = landscape in the manifest file
4 - Runnit on your device
5 - ...move around your phone and enjoy 

Code Select Expand


/**
* A simple demo. This shows more how to use jPCT-AE than it shows how to write
* a proper application for Android, because i have no idea how to do this. This
* thing is more or less a hack to get you started...
*
* @author EgonOlsen
*
* Modified in order to move the camera according the sensors
*
*/
public class Demo extends Activity implements SensorEventListener {

private static final boolean TRY_TRANSPOSED_VERSION = false;

private static int MODUS = 2;

private GLSurfaceView mGLView;
private MyRenderer renderer = null;
private FrameBuffer fb = null;
private World world = null;
private boolean paused = false;

private SensorManager mSensorManager;
private float[] rotationMatrix = new float[9];
private float[] accelGData = new float[3];
private float[] bufferedAccelGData = new float[3];
private float[] magnetData = new float[3];
private float[] bufferedMagnetData = new float[3];
private float[] orientationData = new float[3];
private float[] resultingAngles = new float[3];

private int mCount;

final static float rad2deg = (float) (180.0f / Math.PI);

private boolean landscape;

protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE);

mGLView = new GLSurfaceView(getApplication());

mGLView.setEGLConfigChooser(new GLSurfaceView.EGLConfigChooser() {
public EGLConfig chooseConfig(EGL10 egl, EGLDisplay display) {
// Ensure that we get a 16bit framebuffer. Otherwise, we'll fall
// back to Pixelflinger on some device (read: Samsung I7500)
int[] attributes = new int[] { EGL10.EGL_DEPTH_SIZE, 16,
EGL10.EGL_NONE };
EGLConfig[] configs = new EGLConfig[1];
int[] result = new int[1];
egl.eglChooseConfig(display, attributes, configs, 1, result);
return configs[0];
}
});

renderer = new MyRenderer();
mGLView.setRenderer(renderer);
setContentView(mGLView);
}

@Override
protected void onPause() {
paused = true;
super.onPause();
mGLView.onPause();
}

@Override
protected void onResume() {
paused = false;
super.onResume();
mGLView.onResume();

if (((WindowManager) getSystemService(WINDOW_SERVICE))
.getDefaultDisplay().getOrientation() == 1) {
landscape = true;
} else {
landscape = false;
}

mSensorManager.registerListener(this,
mSensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER),
SensorManager.SENSOR_DELAY_GAME);
mSensorManager.registerListener(this,
mSensorManager.getDefaultSensor(Sensor.TYPE_MAGNETIC_FIELD),
SensorManager.SENSOR_DELAY_GAME);
mSensorManager.registerListener(this,
mSensorManager.getDefaultSensor(Sensor.TYPE_ORIENTATION),
SensorManager.SENSOR_DELAY_GAME);
}

protected void onStop() {
renderer.stop();
super.onStop();
}

protected boolean isFullscreenOpaque() {
return true;
}

class MyRenderer implements GLSurfaceView.Renderer {

private Object3D plane = null;
private Object3D tree2 = null;
private Object3D tree1 = null;
private Object3D grass = null;
private Texture font = null;

private int fps = 0;
private int lfps = 0;

private long time = System.currentTimeMillis();

private Light sun = null;
private Object3D rock = null;

private boolean stop = false;

private float ind;

private boolean deSer = true;

public MyRenderer() {
Config.maxPolysVisible = 5000;
Config.farPlane = 1500;
}

public void stop() {
stop = true;
if (fb != null) {
fb.dispose();
fb = null;
}
}

public void onSurfaceChanged(GL10 gl, int w, int h) {
if (fb != null) {
fb.dispose();
}
fb = new FrameBuffer(gl, w, h);
}

public void onSurfaceCreated(GL10 gl, EGLConfig config) {
TextureManager.getInstance().flush();
world = new World();
Resources res = getResources();

TextureManager tm = TextureManager.getInstance();
Texture grass2 = new Texture(res.openRawResource(R.raw.grassy));
Texture leaves = new Texture(res.openRawResource(R.raw.tree2y));
Texture leaves2 = new Texture(res.openRawResource(R.raw.tree3y));
Texture rocky = new Texture(res.openRawResource(R.raw.rocky));

Texture planetex = new Texture(res.openRawResource(R.raw.planetex));

font = new Texture(res.openRawResource(R.raw.numbers));

tm.addTexture("grass2", grass2);
tm.addTexture("leaves", leaves);
tm.addTexture("leaves2", leaves2);
tm.addTexture("rock", rocky);
tm.addTexture("grassy", planetex);

if (!deSer) {
// Use the normal loaders...
plane = Primitives.getPlane(20, 30);
grass = Loader.load3DS(res.openRawResource(R.raw.grass), 5)[0];
rock = Loader.load3DS(res.openRawResource(R.raw.rock), 15f)[0];
tree1 = Loader.load3DS(res.openRawResource(R.raw.tree2), 5)[0];
tree2 = Loader.load3DS(res.openRawResource(R.raw.tree3), 5)[0];

plane.setTexture("grassy");
rock.setTexture("rock");
grass.setTexture("grass2");
tree1.setTexture("leaves");
tree2.setTexture("leaves2");

plane.getMesh().setVertexController(new Mod(), false);
plane.getMesh().applyVertexController();
plane.getMesh().removeVertexController();
} else {
// Load the serialized version instead...
plane = Loader.loadSerializedObject(res
.openRawResource(R.raw.serplane));
rock = Loader.loadSerializedObject(res
.openRawResource(R.raw.serrock));
tree1 = Loader.loadSerializedObject(res
.openRawResource(R.raw.sertree1));
tree2 = Loader.loadSerializedObject(res
.openRawResource(R.raw.sertree2));
grass = Loader.loadSerializedObject(res
.openRawResource(R.raw.sergrass));
}

grass.translate(-45, -17, -50);
grass.rotateZ((float) Math.PI);
rock.translate(0, 0, -90);
rock.rotateX(-(float) Math.PI / 2);
tree1.translate(-50, -92, -50);
tree1.rotateZ((float) Math.PI);
tree2.translate(60, -95, 10);
tree2.rotateZ((float) Math.PI);
plane.rotateX((float) Math.PI / 2f);

plane.setName("plane");
tree1.setName("tree1");
tree2.setName("tree2");
grass.setName("grass");
rock.setName("rock");

world.addObject(plane);
world.addObject(tree1);
world.addObject(tree2);
world.addObject(grass);
world.addObject(rock);

RGBColor dark = new RGBColor(100, 100, 100);

grass.setTransparency(10);
tree1.setTransparency(0);
tree2.setTransparency(0);

tree1.setAdditionalColor(dark);
tree2.setAdditionalColor(dark);
grass.setAdditionalColor(dark);

world.setAmbientLight(200, 200, 200);
world.buildAllObjects();

sun = new Light(world);

Camera cam = world.getCamera();
cam.moveCamera(Camera.CAMERA_MOVEOUT, 250);
cam.moveCamera(Camera.CAMERA_MOVEUP, 100);
cam.lookAt(plane.getTransformedCenter());

cam.setFOV(1.5f);
sun.setIntensity(250, 250, 250);

SimpleVector sv = new SimpleVector();
sv.set(plane.getTransformedCenter());
sv.y -= 300;
sv.x -= 100;
sv.z += 200;
sun.setPosition(sv);
}

private void copyMatrix(float[] src, com.threed.jpct.Matrix dest,
boolean traspose) {
if (!traspose) {
dest.setRow(0, src[0], src[1], src[2], 0);   
dest.setRow(1, src[3], src[4], src[5], 0);   
dest.setRow(2, src[6], src[7], src[8], 0);   
dest.setRow(3, 0f, 0f, 0f, 1f);
} else {
dest.setRow(0, src[0], src[3], src[6], 0);
dest.setRow(1, src[1], src[4], src[7], 0);
dest.setRow(2, src[2], src[5], src[8], 0);
dest.setRow(3, 0f, 0f, 0f, 1f);

}
}

public void onDrawFrame(GL10 gl) {

try {
if (!stop) {
if (paused) {
Thread.sleep(500);
} else {
Camera cam = world.getCamera();

if ((MODUS == 1) || (MODUS == 2) || (MODUS == 3)
|| (MODUS == 4)) {
boolean traspose = false;
if (landscape) {
// in landscape mode first remap the
// rotationMatrix before using
// it with glMultMatrixf:
float[] result = new float[9];
SensorManager.remapCoordinateSystem(
rotationMatrix, SensorManager.AXIS_MINUS_Y,
SensorManager.AXIS_MINUS_X, result);
com.threed.jpct.Matrix mResult = new com.threed.jpct.Matrix();
copyMatrix(result, mResult, traspose);
//mResult.rotateZ((float)Math.PI / 2);
cam.setBack(mResult);
// gl.glMultMatrixf(result, 0);
} else {
com.threed.jpct.Matrix mResult = new com.threed.jpct.Matrix();
copyMatrix(rotationMatrix, mResult, traspose);
cam.setBack(mResult);
}
} else {
// in all other modes do the rotation by hand
// the order y x z is important!
// gl.glRotatef(resultingAngles[2], 0, 1, 0);
// gl.glRotatef(resultingAngles[1], 1, 0, 0);
// gl.glRotatef(resultingAngles[0], 0, 0, 1);
cam.getBack().setIdentity();
cam.rotateCameraX((float) Math.PI / 2);

cam.rotateCameraY(resultingAngles[0]);
cam.rotateCameraX(resultingAngles[2]);
cam.rotateCameraZ(resultingAngles[1]);
}

// move the axis to simulate augmented behaviour:
// gl.glTranslatef(0, 2, 0);

fb.clear();
world.renderScene(fb);
world.draw(fb);
blitNumber(lfps, 5, 5);

fb.display();

sun.rotate(new SimpleVector(0, 0.05f, 0),
plane.getTransformedCenter());

if (System.currentTimeMillis() - time >= 1000) {
lfps = (fps + lfps) >> 1;
fps = 0;
time = System.currentTimeMillis();
}
fps++;
ind += 0.02f;
if (ind > 1) {
ind -= 1;
}
}
} else {
if (fb != null) {
fb.dispose();
fb = null;
}
}
} catch (Exception e) {
Logger.log("Drawing thread terminated!", Logger.MESSAGE);
}
}

private class Mod extends GenericVertexController {
private static final long serialVersionUID = 1L;

public void apply() {
SimpleVector[] s = getSourceMesh();
SimpleVector[] d = getDestinationMesh();
for (int i = 0; i < s.length; i++) {
d[i].z = s[i].z
- (10f * (FloatMath.sin(s[i].x / 50f) + FloatMath
.cos(s[i].y / 50f)));
d[i].x = s[i].x;
d[i].y = s[i].y;
}
}
}

private void blitNumber(int number, int x, int y) {
if (font != null) {
String sNum = Integer.toString(number);
for (int i = 0; i < sNum.length(); i++) {
char cNum = sNum.charAt(i);
int iNum = cNum - 48;
fb.blit(font, iNum * 5, 0, x, y, 5, 9,
FrameBuffer.TRANSPARENT_BLITTING);
x += 5;
}
}
}
}

public void onAccuracyChanged(Sensor sensor, int accuracy) {
}

public void onSensorChanged(SensorEvent event) {

// load the new values:
loadNewSensorData(event);

if (MODUS == 1) {
SensorManager.getRotationMatrix(rotationMatrix, null, accelGData,
magnetData);
}

if (MODUS == 2) {
rootMeanSquareBuffer(bufferedAccelGData, accelGData);
rootMeanSquareBuffer(bufferedMagnetData, magnetData);
SensorManager.getRotationMatrix(rotationMatrix, null,
bufferedAccelGData, bufferedMagnetData);
}

if (MODUS == 3) {
rootMeanSquareBuffer(bufferedMagnetData, magnetData);
SensorManager.getRotationMatrix(rotationMatrix, null, accelGData,
bufferedMagnetData);
}

if (MODUS == 4) {
rootMeanSquareBuffer(bufferedAccelGData, accelGData);
SensorManager.getRotationMatrix(rotationMatrix, null,
bufferedAccelGData, magnetData);
}

if (MODUS == 5) {
// this mode uses the sensor data recieved from the orientation
// sensor
resultingAngles = orientationData.clone();
if ((-90 > resultingAngles[1]) || (resultingAngles[1] > 90)) {
resultingAngles[1] = orientationData[0];
resultingAngles[2] = orientationData[1];
resultingAngles[0] = orientationData[2];
}
}

if (MODUS == 6) {
rootMeanSquareBuffer(bufferedAccelGData, accelGData);
rootMeanSquareBuffer(bufferedMagnetData, magnetData);
SensorManager.getRotationMatrix(rotationMatrix, null,
bufferedAccelGData, bufferedMagnetData);
final float[] anglesInRadians = new float[3];
SensorManager.getOrientation(rotationMatrix, anglesInRadians);
// TODO check for landscape mode
resultingAngles[0] = anglesInRadians[0]; // * rad2deg;
resultingAngles[1] = anglesInRadians[1]; // * rad2deg;
resultingAngles[2] = anglesInRadians[2]; // * -rad2deg;
}

if (MODUS == 7) {
SensorManager.getRotationMatrix(rotationMatrix, null, accelGData,
magnetData);

rotationMatrix = transpose(rotationMatrix);
/*
* this assumes that the rotation matrices are multiplied in x y z
* order Rx*Ry*Rz
*/

resultingAngles[2] = (float) (Math.asin(rotationMatrix[2]));
final float cosB = (float) Math.cos(resultingAngles[2]);
resultingAngles[2] = resultingAngles[2] * rad2deg;
resultingAngles[0] = -(float) (Math.acos(rotationMatrix[0] / cosB))
* rad2deg;
resultingAngles[1] = (float) (Math.acos(rotationMatrix[10] / cosB))
* rad2deg;
}

if (MODUS == 8) {
SensorManager.getRotationMatrix(rotationMatrix, null, accelGData,
magnetData);
rotationMatrix = transpose(rotationMatrix);
/*
* this assumes that the rotation matrices are multiplied in z y x
*/

resultingAngles[2] = (float) (Math.asin(-rotationMatrix[8]));
final float cosB = (float) Math.cos(resultingAngles[2]);
resultingAngles[2] = resultingAngles[2] * rad2deg;
resultingAngles[1] = (float) (Math.acos(rotationMatrix[9] / cosB))
* rad2deg;
resultingAngles[0] = (float) (Math.asin(rotationMatrix[4] / cosB))
* rad2deg;
}

if (MODUS == 9) {
SensorManager.getRotationMatrix(rotationMatrix, null, accelGData,
magnetData);
rotationMatrix = transpose(rotationMatrix);
/*
* this assumes that the rotation matrices are multiplied in z x y
*
* note z axis looks good at this one
*/

resultingAngles[1] = (float) (Math.asin(rotationMatrix[9]));
final float minusCosA = -(float) Math.cos(resultingAngles[1]);
resultingAngles[1] = resultingAngles[1] * rad2deg;
resultingAngles[2] = (float) (Math.asin(rotationMatrix[8]
/ minusCosA))
* rad2deg;
resultingAngles[0] = (float) (Math.asin(rotationMatrix[1]
/ minusCosA))
* rad2deg;
}

if (MODUS == 10) {
SensorManager.getRotationMatrix(rotationMatrix, null, accelGData,
magnetData);
rotationMatrix = transpose(rotationMatrix);
/*
* this assumes that the rotation matrices are multiplied in y x z
*/

resultingAngles[1] = (float) (Math.asin(-rotationMatrix[6]));
final float cosA = (float) Math.cos(resultingAngles[1]);
resultingAngles[1] = resultingAngles[1] * rad2deg;
resultingAngles[2] = (float) (Math.asin(rotationMatrix[2] / cosA))
* rad2deg;
resultingAngles[0] = (float) (Math.acos(rotationMatrix[5] / cosA))
* rad2deg;
}

if (MODUS == 11) {
SensorManager.getRotationMatrix(rotationMatrix, null, accelGData,
magnetData);
rotationMatrix = transpose(rotationMatrix);
/*
* this assumes that the rotation matrices are multiplied in y z x
*/

resultingAngles[0] = (float) (Math.asin(rotationMatrix[4]));
final float cosC = (float) Math.cos(resultingAngles[0]);
resultingAngles[0] = resultingAngles[0] * rad2deg;
resultingAngles[2] = (float) (Math.acos(rotationMatrix[0] / cosC))
* rad2deg;
resultingAngles[1] = (float) (Math.acos(rotationMatrix[5] / cosC))
* rad2deg;
}

if (MODUS == 12) {
SensorManager.getRotationMatrix(rotationMatrix, null, accelGData,
magnetData);
rotationMatrix = transpose(rotationMatrix);
/*
* this assumes that the rotation matrices are multiplied in x z y
*/

resultingAngles[0] = (float) (Math.asin(-rotationMatrix[1]));
final float cosC = (float) Math.cos(resultingAngles[0]);
resultingAngles[0] = resultingAngles[0] * rad2deg;
resultingAngles[2] = (float) (Math.acos(rotationMatrix[0] / cosC))
* rad2deg;
resultingAngles[1] = (float) (Math.acos(rotationMatrix[5] / cosC))
* rad2deg;
}
logOutput();
}

/**
* transposes the matrix because it was transposted (inverted, but here its
* the same, because its a rotation matrix) to be used for opengl
*
* @param source
* @return
*/
private float[] transpose(float[] source) {
final float[] result = source.clone();
if (TRY_TRANSPOSED_VERSION) {
result[1] = source[4];
result[2] = source[8];
result[4] = source[1];
result[6] = source[9];
result[8] = source[2];
result[9] = source[6];
}
// the other values in the matrix are not relevant for rotations
return result;
}

private void rootMeanSquareBuffer(float[] target, float[] values) {

final float amplification = 200.0f;
float buffer = 20.0f;

target[0] += amplification;
target[1] += amplification;
target[2] += amplification;
values[0] += amplification;
values[1] += amplification;
values[2] += amplification;

target[0] = (float) (Math
.sqrt((target[0] * target[0] * buffer + values[0] * values[0])
/ (1 + buffer)));
target[1] = (float) (Math
.sqrt((target[1] * target[1] * buffer + values[1] * values[1])
/ (1 + buffer)));
target[2] = (float) (Math
.sqrt((target[2] * target[2] * buffer + values[2] * values[2])
/ (1 + buffer)));

target[0] -= amplification;
target[1] -= amplification;
target[2] -= amplification;
values[0] -= amplification;
values[1] -= amplification;
values[2] -= amplification;
}

private void loadNewSensorData(SensorEvent event) {
final int type = event.sensor.getType();
if (type == Sensor.TYPE_ACCELEROMETER) {
accelGData = event.values.clone();
}
if (type == Sensor.TYPE_MAGNETIC_FIELD) {
magnetData = event.values.clone();
}
if (type == Sensor.TYPE_ORIENTATION) {
orientationData = event.values.clone();
}
}

private void logOutput() {
if (mCount++ > 30) {
mCount = 0;
Log.d("Compass", "yaw0: " + (int) (resultingAngles[0])
+ "  pitch1: " + (int) (resultingAngles[1]) + "  roll2: "
+ (int) (resultingAngles[2]));
}
}

}


[mgrabul]

Thanks for the code. I made my own code using the example you posted earlier but i still heven quite figure out the oreintation and the plasement of the objects in the 3d space. I hope after I study this code things will be easier to unrerstand:)

[rodplamo]

Good morning,

First, i'm sorry for my bad english, i hope all people of this forum understand me.

I'm trying to create an augmented reality browser, and this code are very useful (thanks nimo). But, like mgrabul said, i still don't understand how get the orientation of the camera, and (this is more important yet) how link real coordenates with virtual camera coordinates.

I'm not newbie in Android platform, but this (AR) is more complicated than other proyects that i've done. I don't know if someone can help me, because also i'm newbie in this forum, but i will be grateful if someone do it.

Regards.