Object3D.rayIntersectsAABB question

[voronwe13]

I'm working on an android application where I need to be able to select objects on screen, and I'm having issues with the collision detection code in Object3D.  This is a 2D image viewing/editing application, so everything is pretty simple shapes, mostly just squares.   My first thought for selecting an item was to use Object3D.rayIntersectsAABB, with the coordinates of the touch point translated to world space as the origin (say [25, 25, -10], a simple vector pointing down towards the plane of the images and objects ([0,0,1]).   The problem is that no matter where I touch, it thinks the ray intersects.   I've confirmed the AABB (using this example, it could be [0,5,0,5,-1,-1]), and it looks accurate to the object.   It seems like it's using the AABB as infinite planes instead of just a box.  If I used the direction vector [0,0,-1], it never intersects, and if I use a direction vector of [0,1,0], it intersects if I touch anywhere other than one side of the box, even though the z value of the touch point should make it not intersect at all.   Is this how the AABB is supposed to work, or am I missing something?

I've also tried using calcMinDistance for checking if it intersects, and this works with some objects, but I have an object that uses a VertexController to change the mesh, and no matter what I've tried, it only detects a hit if I touch somewhere in the object at its original size/shape before any changes from the VertexController.  Is there a way to get it to do hit detection based on the updated mesh?

Thanks for all your help, this forum has been a lifesaver on many occasions!

[EgonOlsen]

rayIntersectsAABB works in object space...might that be the reason, why it didn't work out?

About the calcMinDistance()-thing...i would expect it to work on a modified mesh as well. Can you create a simple test case that shows the problem?

[voronwe13]

No, it doesn't appear to be related to the fact that it's in object space.    From the testing I've done so far, it appears the size and positioning of the AABB is correct.   What appears to be happening is that it's first just doing a ray-plane intersection test for each of the sides of the box, but then just returning the result of that test, rather than checking if the intersection is inside the box part of the plane or outside of it.

For the calcMinDistance(), I'm still narrowing down the issue.  So far it's only happening to the objects I have that use the vertex controller, but after further testing, it doesn't seem to matter how big or small the original object was.  It's only detecting a hit right near one specific vertex of the object, no matter how big or small the object or where that vertex is.   I'll update here if I figure out what is going on.

[voronwe13]

To get a little more specific on what's happening with calcMinDistance(), I have a vertex controller that allows for resizing rectangular polygon, but it also keeps the vertices in the same positions relative to each other, so that the UV coordinates stay the same.  I do this because the texture contains text that I want always oriented the same way so it's readable.   This also guarantees that I always have the front of the polygon facing the camera.

calcMinDistance() appears to only detect a collision inside a box approximately 10x10 coming from the upper left corner of the polygon, regardless of the actual size or shape of the polygon (they normally have width and height >1000, and are not square).   It also doesn't seem to matter what the original size or shape of the polygon was, so I don't know what's happening.   

I'm still testing things, so I'll update if I figure out what is going on.

[voronwe13]

Okay, I've narrowed it down more... I had another graphic object that didn't have any problems with me touching it, so I did some comparisons... it turns out that it also had the same problem, but I didn't see it because I treated it differently...   

What appears to be happening is all polygons have a 16x16 square hit zone that is scaled with the polygon, and then clipped by the polygon.   If the initial size of the polygon is bigger than 16x16, anything that was outside of that 16x16 before scaling does not register a hit.   

The object that wasn't having a problem was initially 4x16 and then scaled to an appropriate size for where it was going.   When I changed its initial dimensions to 8x32, it would only detect a hit in the upper half of the polygon, no matter the scale.

The object I am having a problem with doesn't use scale to resize it, since I need 2D scaling and the scale method in Object3D only takes a scalar value instead of a vector.  So instead I use a vertex controller to resize it.   Also, I'm using a polyline to outline the rectangle, and that only uses world coordinates, so I just reuse the world coordinates from the polyline as the new vertex coordinates of the polygon (although I adjust to make sure the corners always stay in the same relative positions, regardless of how they are for the polyline).   This means that the object coordinates (since that's what the vertex controller manipulates) are actually the same as the world coordinates.

I hope this clears up what is happening.   I'll have to come up with a way around this limitation if I still want to have arbitrarily resizable polygons for my text boxes.

[EgonOlsen]

I'm not sure what's going on here. I'm not aware of any issues with any of these methods (and i'm using them a lot internally) and there's certainly no 16*16 limit to anything. The distance calculation if solely based on geometry and and on some grid or anything like that. I'll make myself a test case and we'll see...

[EgonOlsen]

Ok, so here's some hacky test case that tests both methods. You can touch anywhere to trigger a distance calculation at that point and you can drag left/right to rescale the object by using an IVertexController. I fail to see the problem, everything works as expected. Please note the offset that i'm applying to the touch coordinates to compensate for the status- and the title bar.

Code Select Expand


package com.example.scaletest;

import javax.microedition.khronos.egl.EGLConfig;
import javax.microedition.khronos.opengles.GL10;

import android.app.Activity;
import android.opengl.GLSurfaceView;
import android.os.Bundle;
import android.view.MotionEvent;
import android.view.Window;

import com.threed.jpct.Camera;
import com.threed.jpct.FrameBuffer;
import com.threed.jpct.GenericVertexController;
import com.threed.jpct.Interact2D;
import com.threed.jpct.Light;
import com.threed.jpct.Logger;
import com.threed.jpct.Object3D;
import com.threed.jpct.Primitives;
import com.threed.jpct.RGBColor;
import com.threed.jpct.SimpleVector;
import com.threed.jpct.Texture;
import com.threed.jpct.TextureInfo;
import com.threed.jpct.TextureManager;
import com.threed.jpct.World;
import com.threed.jpct.util.BitmapHelper;

/**
*
* @author EgonOlsen
*
*/
public class ScaleActivity extends Activity {

private GLSurfaceView mGLView;
private MyRenderer renderer = null;
private FrameBuffer fb = null;
private World world = null;
private RGBColor back = new RGBColor(50, 50, 100);

private float touchScale = 0;
private boolean clicked = false;

private float xpos = -1;
private float ypos = -1;

private Object3D cube = null;
private Light sun = null;
private MyVertexController mvc = new MyVertexController();

protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
mGLView = new GLSurfaceView(getApplication());
mGLView.setEGLContextClientVersion(2);

renderer = new MyRenderer();
mGLView.setRenderer(renderer);
setContentView(mGLView);
}

@Override
protected void onPause() {
super.onPause();
mGLView.onPause();
System.exit(0);
}

@Override
protected void onResume() {
super.onResume();
mGLView.onResume();
}

@Override
protected void onStop() {
super.onStop();
}

public boolean onTouchEvent(MotionEvent me) {

if (me.getAction() == MotionEvent.ACTION_DOWN) {
xpos = me.getX();
ypos = me.getY();
clicked = true;
return true;
}

if (me.getAction() == MotionEvent.ACTION_UP) {
xpos = -1;
ypos = -1;
touchScale = 0;
return true;
}

if (me.getAction() == MotionEvent.ACTION_MOVE) {
float xd = me.getX() - xpos;

xpos = me.getX();
ypos = me.getY();

touchScale = xd;
if (touchScale != 0) {
if (touchScale < 0) {
touchScale = 0.99f;
} else {
touchScale = 1.01f;
}
}
return true;
}

return super.onTouchEvent(me);
}

protected boolean isFullscreenOpaque() {
return true;
}

class MyRenderer implements GLSurfaceView.Renderer {

public MyRenderer() {
//
}

public void onSurfaceChanged(GL10 gl, int w, int h) {
if (fb != null) {
fb.dispose();
}
fb = new FrameBuffer(w, h);

world = new World();
world.setAmbientLight(20, 20, 20);

sun = new Light(world);
sun.setIntensity(250, 250, 250);

// Create a texture out of the icon...:-)
Texture texture = new Texture(BitmapHelper.rescale(BitmapHelper.convert(getResources().getDrawable(R.drawable.ic_launcher)), 64, 64));
TextureManager.getInstance().addTexture("texture", texture);
TextureInfo ti = new TextureInfo(TextureManager.getInstance().getTextureID("texture"));

cube = Primitives.getCube(10);
cube.calcTextureWrapSpherical();
cube.setTexture(ti);
cube.compile(true);
cube.build();
cube.getMesh().setVertexController(mvc, true);

world.addObject(cube);

Camera cam = world.getCamera();
cam.moveCamera(Camera.CAMERA_MOVEOUT, 50);

SimpleVector sv = new SimpleVector();
sv.set(cube.getTransformedCenter());
sv.y -= 100;
sv.z -= 100;
sun.setPosition(sv);
}

public void onSurfaceCreated(GL10 gl, EGLConfig config) {
}

public void onDrawFrame(GL10 gl) {
if (touchScale != 0) {
mvc.setScale(touchScale);
cube.getMesh().applyVertexController();
cube.touch();
touchScale = 0;
}
if (clicked) {
clicked = false;

// Calculate minimal distance
int yOffset = getWindow().findViewById(Window.ID_ANDROID_CONTENT).getTop();
SimpleVector touchPoint = Interact2D.reproject2D3DWS(world.getCamera(), fb, (int) xpos, (int) ypos - yOffset);
float min = cube.calcMinDistance(world.getCamera().getPosition(), touchPoint, 1000);
if (min != Object3D.COLLISION_NONE) {
Logger.log("Distance: " + min);
} else {
Logger.log("Missed!");
}

// Calculate minimal distance to AABB
min = cube.rayIntersectsAABB(world.getCamera().getPosition(), touchPoint);
if (min != Object3D.RAY_MISSES_BOX) {
Logger.log("Distance to AABB: " + min);
} else {
Logger.log("Missed AABB!");
}
}

fb.clear(back);
world.renderScene(fb);
world.draw(fb);
fb.display();
}
}

private static class MyVertexController extends GenericVertexController {

private static final long serialVersionUID = 1L;
private float scale;

public void setScale(float scale) {
this.scale = scale;
}

@Override
public void apply() {
SimpleVector[] source = this.getSourceMesh();
SimpleVector[] dest = this.getDestinationMesh();

for (int i = 0; i < source.length; i++) {
SimpleVector s = source[i];
s.scalarMul(scale);
dest[i].set(s);
}
}
}
}



[voronwe13]

Okay, I took your test case and modified it a bit to closer match what I'm doing.  I'm currently unable to replicate the calcMinDistance issue with this, but it is showing the AABB issue.   The main difference with what you did and what I'm doing is that I'm converting the touch to a concrete position inside the world and checking a ray from there, rather than just a ray straight from the camera position.  This should be the same as checking the collision between two objects in the world, regardless of the camera.

I'll have to leave it for today, but here's the code I have right now.  Maybe this will help find the issue with AABB.   I'll try to figure out the calcMinDistance issue tomorrow.

Code Select Expand

package com.example.scaletest;
import javax.microedition.khronos.egl.EGLConfig;
import javax.microedition.khronos.opengles.GL10;

import android.app.Activity;
import android.opengl.GLSurfaceView;
import android.os.Bundle;
import android.view.MotionEvent;
import android.view.Window;

import com.threed.jpct.Camera;
import com.threed.jpct.Config;
import com.threed.jpct.FrameBuffer;
import com.threed.jpct.GenericVertexController;
import com.threed.jpct.Interact2D;
import com.threed.jpct.Light;
import com.threed.jpct.Logger;
import com.threed.jpct.Object3D;
import com.threed.jpct.Primitives;
import com.threed.jpct.RGBColor;
import com.threed.jpct.SimpleVector;
import com.threed.jpct.Texture;
import com.threed.jpct.TextureInfo;
import com.threed.jpct.TextureManager;
import com.threed.jpct.World;
import com.threed.jpct.util.BitmapHelper;

/**
*
* @author EgonOlsen
* @edited voronwe13
*/
public class ScaleActivity extends Activity {

private GLSurfaceView mGLView;
private MyRenderer renderer = null;
private FrameBuffer fb = null;
private World world = null;
private RGBColor back = new RGBColor(50, 50, 100);

private float touchScale = 0;
private boolean clicked = false;

private float xpos = -1;
private float ypos = -1;

private Object3D square = null;
private Light sun = null;
//private MyVertexController mvc = new MyVertexController();
private static SimpleVector ul = new SimpleVector();
private static SimpleVector ur = new SimpleVector();
private static SimpleVector bl = new SimpleVector();
private static SimpleVector br = new SimpleVector();
private static final SimpleVector downvec = new SimpleVector(0,0,1);
private static final float objwidth = 700, objheight = 700;
private static float fov, viewwidth, viewheight;

protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
mGLView = new GLSurfaceView(getApplication());
mGLView.setEGLContextClientVersion(2);

renderer = new MyRenderer();
mGLView.setRenderer(renderer);
setContentView(mGLView);
}

@Override
protected void onPause() {
super.onPause();
mGLView.onPause();
System.exit(0);
}

@Override
protected void onResume() {
super.onResume();
mGLView.onResume();
}

@Override
protected void onStop() {
super.onStop();
}

public boolean onTouchEvent(MotionEvent me) {

if (me.getAction() == MotionEvent.ACTION_DOWN) {
xpos = me.getX();
ypos = me.getY();
clicked = true;
return true;
}

if (me.getAction() == MotionEvent.ACTION_UP) {
xpos = -1;
ypos = -1;
touchScale = 0;
return true;
}

if (me.getAction() == MotionEvent.ACTION_MOVE) {
float xd = me.getX() - xpos;

xpos = me.getX();
ypos = me.getY();

touchScale = xd;
if (touchScale != 0) {
if (touchScale < 0) {
touchScale = 0.99f;
} else {
touchScale = 1.01f;
}
}
return true;
}

return super.onTouchEvent(me);
}

protected boolean isFullscreenOpaque() {
return true;
}

class MyRenderer implements GLSurfaceView.Renderer {

public MyRenderer() {
//
}

public void onSurfaceChanged(GL10 gl, int w, int h) {
if (fb != null) {
fb.dispose();
}
fb = new FrameBuffer(w, h);
viewwidth = w;
viewheight = h;
world = new World();
world.setAmbientLight(200, 200, 200);
Config.farPlane = 10000;


// Create a texture out of the icon...:-)
Texture texture = new Texture(BitmapHelper.rescale(BitmapHelper.convert(getResources().getDrawable(R.drawable.ic_launcher)), 64, 64));
TextureManager.getInstance().addTexture("texture", texture);
TextureInfo ti = new TextureInfo(TextureManager.getInstance().getTextureID("texture"));

square = createTile(0,0,objwidth,objheight);
square.calcTextureWrapSpherical();
square.setTexture(ti);
square.compile(true);
square.build();
//square.getMesh().setVertexController(mvc, true);

world.addObject(square);

Camera cam = world.getCamera();
fov = cam.getFOV();
cam.moveCamera(Camera.CAMERA_MOVEOUT, 3*objwidth/fov);
cam.moveCamera(Camera.CAMERA_MOVEDOWN, objheight/2);
cam.moveCamera(Camera.CAMERA_MOVERIGHT, objwidth/2);

}

public void onSurfaceCreated(GL10 gl, EGLConfig config) {
}

public void onDrawFrame(GL10 gl) {
if (touchScale != 0) {
//mvc.setScale(touchScale);
//square.getMesh().applyVertexController();
//square.touch();
touchScale = 0;
}
if (clicked) {
clicked = false;

// Calculate minimal distance
int yOffset = getWindow().findViewById(Window.ID_ANDROID_CONTENT).getTop();
SimpleVector touchpos = new SimpleVector();
convertPosition(xpos, ypos, -10, touchpos);
Logger.log("touch position"+touchpos+", direction: "+downvec.toString());
float min = square.calcMinDistance(touchpos, downvec, 10000);
if (min != Object3D.COLLISION_NONE) {
Logger.log("Distance: " + min);
} else {
Logger.log("Missed!");
}

// Calculate minimal distance to AABB
min = square.rayIntersectsAABB(touchpos, downvec);
if (min != Object3D.RAY_MISSES_BOX) {
Logger.log("Distance to AABB: " + min);
} else {
Logger.log("Missed AABB!");
}
}

fb.clear(back);
world.renderScene(fb);
world.draw(fb);
fb.display();
}
}


//creates a tile of a specific size and location
public static Object3D createTile(float left, float top, float right, float bottom){
Object3D tileobj = new Object3D(2);
float offset = 0;

float u0 = 0;
float u1 = 1;
float v0 = 0;
float v1 = 1;
ul.set(left, top, 0);
ur.set(right, top, 0);
bl.set(left, bottom, 0);
br.set(right, bottom, 0);
tileobj.addTriangle(ul, u0, v0, bl, u0, v1, br, u1, v1);
tileobj.addTriangle(ul, u0, v0, br, u1, v1, ur, u1, v0);
return tileobj;
}

//converts a position from screen dimensions to world space, assuming a camera looking
//down the positive z-axis with camera oriented with -y (or (0,-1,0)) as up (the default orientation
//for jpct's camera).
public void convertPosition(float startx, float starty, float z, SimpleVector tofill) {
SimpleVector campos = world.getCamera().getPosition();
float adjfactor = (z-campos.z)*fov/viewwidth;
tofill.set((startx-viewwidth/2)*adjfactor + campos.x, (starty-viewheight/2)*adjfactor + campos.y, z);
}

// private static class MyVertexController extends GenericVertexController {
//
// private static final long serialVersionUID = 1L;
// private float scale;
//
// public void setScale(float scale) {
// this.scale = scale;
// }
//
// @Override
// public void apply() {
// SimpleVector[] source = this.getSourceMesh();
// SimpleVector[] dest = this.getDestinationMesh();
//
// for (int i = 0; i < source.length; i++) {
// SimpleVector s = source[i];
// s.scalarMul(scale);
// dest[i].set(s);
// }
// }
// }
}

[EgonOlsen]

I haven't tried your modified test case yet and can't do so before tomorrow evening, but is there any chance that your converted touch point lies inside of the AABB? If that's the case, then you'll get this intersection-with-plane-effect that you described.

[voronwe13]

No, because if the point were inside the AABB, it would intersect no matter what direction the ray was going.  In this case, if the ray is pointing directly away from the box, it doesn't intersect.   You can check this by changing the down vector to a side vector, or an up vector.

[EgonOlsen]

I'm not really sure about your convert method...even if it does work (i haven't checked), it's valid only for a special case. But let's assume that it works as indented...you can't cast a ray down the z-axis to pick your object properly. I understand where this idea comes from and it seems fine at first glance, but it doesn't take into account that what you see on the screen is just a projection from 3d into 2d and what's linear in 3d isn't in 2d. I don't know how to explain that any better...
Just use the same method as i did and as its explained in the wiki and you have almost pixel perfect picking no matter how the objects and the camera are transformed.

[voronwe13]

I'm curious why casting down the z-axis doesn't work... assume this had nothing to do with the camera and picking, shouldn't a ray cast down the z-axis from (10,10,-10) miss a rectangle from (0,0,0) top-left to (7,7,0) bottom-right?

[EgonOlsen]

Yes, it should. And it certainly will. But that's not the situation here.

[EgonOlsen]

However, i can confirm that the AABB-test creates a strange result in this case even when using desktop jPCT. I'll look into, but i suggest not to use this anyway. Just use calcMinDistance instead, which should work fine.

[voronwe13]

The only thing I can think of is that your AABB calculation is using camera space instead of world or object space, but I really don't know.   My convert position gives actual world space position for the given (x,y) screen position at the world z position given, with specific assumptions about the camera (I left out some code that also accounts for camera rotation on the z, which is all I need for my purposes).   Also, the world space values you'll get with the code I posted will be a little off in the Y.  I forgot to include the screen offset from the android window title.

calcMinDistance is what I would prefer to use for the picking, but I still haven't figured out why it's only hitting a small portion of my polygons...   And it doesn't seem like it's a transformation problem, because it's hitting on the polygon completely accurately if the original polygon before transformation is anything smaller than 16x16.   I just need to figure out how to reproduce this problem in the test case so you can see what I'm talking about.