i tried pose estimation for my augmeted reality.  i used solvepnp opencv
this is the code
TargetPoint = new MatOfPoint3f();
TargetPoint.alloc(4);
TargetPoint.put(0, 0, 0, 0, 0);
TargetPoint.put(1, 0, 4, 0, 0);
TargetPoint.put(2, 0, 4, -4, 0);
TargetPoint.put(3, 0, 0, -4, 0);

imagePointsMat = new MatOfPoint2f();
imagePointsMat.alloc(4);

index1 = point.getCenter();
if (index1 != false){
index1x = point.getCenterx()*2.4f; index1y = point.getCentery()*2.4f;
imagePointsMat.put(0, 0, index1x-960, index1y-540);
}
thumb1 = point.getThumb1();
if (thumb1 != false){
thumb1x = point.getThumb1x()*2.4f; thumb1y = point.getThumb1y()*2.4f;
imagePointsMat.put(1, 0, thumb1x-960, thumb1y-540);
}
middle1 = point.getMiddle1();
if (middle1 != false){
middle1x = point.getMiddle1x()*2.4f; middle1y = point.getMiddle1y()*2.4f;
imagePointsMat.put(2, 0, middle1x-960, middle1y-540);
}
ring = point.getLittle1();
if (ring != false){
ringx = point.getLittle1x()*2.4f; ringy = point.getLittle1y()*2.4f;
imagePointsMat.put(3, 0, ringx-960, ringy-540);
}

Mat cameraMatrix = Mat.eye(3, 3, CvType.CV_32F);
cameraMatrix.put(0, 0, 800);
    cameraMatrix.put(1, 1, 800);
    cameraMatrix.put(0, 2, 800/2);
    cameraMatrix.put(1, 2, 450/2);
    cameraMatrix.put(2, 2, 1);
   
MatOfDouble distCoeffs = new MatOfDouble();
distCoeffs.alloc(5);
distCoeffs.put(1, 0, -0.04372335597872734);
distCoeffs.put(2, 0, 1.203663110733032);
distCoeffs.put(3, 0, 0);
distCoeffs.put(4, 0, 0);
distCoeffs.put(5, 0, -4.552499294281006);

Mat rvec = new Mat(3,1,CvType.CV_32FC1);
    Mat tvec = new Mat(3,1,CvType.CV_32FC1);
   
    Calib3d.solvePnP(TargetPoint, imagePointsMat, cameraMatrix, distCoeffs, rvec, tvec);

Mat R = new Mat(3,3,CvType.CV_32FC1);                       
    Calib3d.Rodrigues(rvec, R);

Mat m44 =  new Mat(4, 4, CvType.CV_32FC1);

        for (int i=0; i<3; i++)
         for (int j=0; j<3; j++)
        m44.put(i, j, R.get(i, j));

        for(int k=0;k<3;k++)
         m44.put(k, 3, tvec.get(k, 0));
       
        Matrix result = new Matrix();
        result.set(0, 0, (float) m44.get(0, 0)[0]);
        result.set(0, 1, (float) m44.get(0, 1)[0]);
        result.set(0, 2, (float) m44.get(0, 2)[0]);
        result.set(0, 3, (float) m44.get(0, 3)[0]);
        result.set(1, 0, (float) m44.get(1, 0)[0]);
        result.set(1, 1, (float) m44.get(1, 1)[0]);
        result.set(1, 2, (float) m44.get(1, 2)[0]);
        result.set(1, 3, (float) m44.get(1, 3)[0]);
        result.set(2, 0, (float) m44.get(2, 0)[0]);
        result.set(2, 1, (float) m44.get(2, 1)[0]);
        result.set(2, 2, (float) m44.get(2, 2)[0]);
        result.set(2, 3, (float) m44.get(2, 3)[0]);
        result.set(3, 0, (float) m44.get(3, 0)[0]);
        result.set(3, 1, (float) m44.get(3, 1)[0]);
        result.set(3, 2, (float) m44.get(3, 2)[0]);
        result.set(3, 3, (float) m44.get(3, 3)[0]);
       
return result;

and i rotate my model using
object.setRotationMatrix(result); but the model rotate all over place

and i tried to rotate camera using
camera.setBack(result); i got result like object rotate.

rotation gone wild all over screen. and blinking when rotate.

is there anyone know where my problem?

what should i do if i want to export multiple mesh ogre to bones?

"jmeOgre2Bones -out bone.bones -in mes1.xml mesh2.xml" like this giving error number of vertices differ

my model in 1 blender file

you procedural animation saved me. my finger now can move.

can i select a mesh from animated group? example i want to select my fingertip. can i select fingertip mesh?

what mat to fill in transformJointOnPivot(Joint joint, Matrix mat)? i only know simplevector to translate

thank you. i already can move my finger to my desire location. Can but parent mesh not follow the object rotation. how can i calculate the rotation to make the parent mesh like follow the child? and is the skeleton helper can move the joint and follow by the parent joint?

So how i can move the joint? Ex: i want to move my joint to (240,200,10) this is my marker position.

   Joint pointing  = currentPose.getSkeleton().getJoint(15);
final Joint parentJoint = currentPose.getSkeleton().getJoint(pointing.getParentIndex());
final Matrix global= new Matrix(pointing.getBindPose());
global.translate(x,y,z);
global.matMul(parentJoint.getInverseBindPose());
currentPose.getLocal(pointing.getIndex()).setTo(global);
currentPose.updateTransforms();
skeletonDebugger.update(currentPose);

global.translate giving different position from object3d.translate()

is matrix translate different form object 3d translate?
when i used matrix translate and object3d translate with same value, giving different output.
can i cast matrix to object 3d? because i want to make augmented reality, that my object translate where my marker position.

in the picture my marker is the white circle. if i use matrix translate, i cant move to the white circle.
but if i use .obj file with object3d translate , i can move to white circle.

any idea?

i already solved the model . And now i can tranlate and rotate the skeleton with this code.


Joint pointing  = currentPose.getSkeleton().getJoint(15);
final Joint parentJoint = currentPose.getSkeleton().getJoint(pointing.getParentIndex());
final Matrix global= new Matrix(pointing.getBindPose());
global.translate(10,10,10)
global.matMul(parentJoint.getInverseBindPose());
currentPose.getLocal(pointing.getIndex()).setTo(global);
currentPose.updateTransforms();
skeletonDebugger.update(currentPose);


the joint at index 15 stretch. but my mesh not. i want the parent joint following, like seymour neck in ProceduralAnimationSample.
how i can do like that? i don't know where the mistake, in the program or my model.

thank you. i followed ProseduralAnimationSample.java.
but my skeleton  and my mesh model not in same position.
my skeleton a bit lower form my mesh model.
in blender my model and my armature is okay.

i have bones file. i want to select 1 part of my file to rotate.
how i can get access to my bones joint?
how can  i cast animated group to animated 3d?