View source for Hybrid GPU Shader Animations for Bones

== Overview ==
While working on an Android project, I (alias Redman) found that jPCT+Bones for skeletal animations were becoming too CPU intensive on the UI thread.  When adding twenty animated models on screen, I was noticing a considerable drop in the frame-rate.  So I developed a hybrid solution of offloading calculations to the GPU for my project, which I will share with you now.

I have added notes and personal recommendations.  This architecture may not be perfect for your project, so please alter and use to fit your needs.

Comments and suggestions are welcome as I will be adding a FAQ to the bottom of the page.

== Requirements ==
<ul>
<li>JPCT (requires GL ES 2+ as it uses GLSL Shaders)</li>
<li>Bones</li>
</ul>

== How It Works ==
This implementation works by offloading all of the vertex calculations to the GPU.  The bone matrix transforms for the poses are still performed on the CPU side.

'''Gains:''' Speed.  You will harness the GPU which is setup to handle floating point calculations quickly and is dedicated to such tasks.  As an example with my project: having 20 animated models dropped the frame-rate to 8-9 fps on my device.  Using this method, it went back up to the 60-62 fps cap.

'''Losses:''' Software knowledge of the transformed pose.  Since the meshes of your models will not be animated on the software-side, it will have no knowledge of its transformed pose.  The polygons will be in the initial state, which will not give you an accurate bounding-box, ray-trace collisions, etc... of the current animated mesh pose as it will not actually be transformed on the software-side.

== The Code ==
This code is free, open-source code and you are welcome to use it as you see fit.


'''GPUAnimated3D Class:'''
<pre>
package yournamespacegoeshere;

import yournamespacegoeshere.GPUAnimated3DShader;
import com.threed.jpct.GLSLShader;
import com.threed.jpct.IRenderHook;
import com.threed.jpct.Object3D;

import raft.jpct.bones.Animated3D;
import raft.jpct.bones.BonesNamespaceUtils;
import raft.jpct.bones.SkinClip;

/**
 * Created by Dougie on 6/21/16.
 */
public class GPUAnimated3D extends Animated3D implements IRenderHook {
    public GPUAnimated3D(Animated3D object) {
        super(object);
        setSkeletonPose(new SkeletonPose(getSkeleton()));
        BonesNamespaceUtils.setSkinAttributes(object);
        setRenderHook(this);
    }

    public void animateSkin(float index, int sequence) {
        if(getSkinClipSequence() != null) {
            if(sequence == 0) {
                BonesNamespaceUtils.animate(this, index*getSkinClipSequence().getTime(), getSkeletonPose());
            } else {
                SkinClip clip = getSkinClipSequence().getClip(sequence - 1);
                clip.applyTo(index * clip.getTime(), getSkeletonPose());
            }
            getSkeletonPose().updateTransforms();
        }
    }
    public void animatePose(float index, int sequence, float weight) {
        BonesNamespaceUtils.animatePoseDontApply(this, index, sequence, weight);
    }

    @Override
    public void beforeRendering(int i) { ; }
    @Override
    public void afterRendering(int i) { ; }
    @Override
    public void setCurrentObject3D(Object3D object3D) { ; }
    @Override
    public void setCurrentShader(GLSLShader glslShader) {
        if(glslShader!=null && glslShader instanceof GPUAnimated3DShader) {
            ((GPUAnimated3DShader)glslShader).updateBeforeRenderingObject(this);
        }
    }
    @Override
    public void setTransparency(float v) { ; }
    @Override
    public void onDispose() { ; }
    @Override
    public boolean repeatRendering() { return false; }
}
</pre>


'''GPUAnimated3DShader Class:'''
<pre>
package yournamespacegoeshere;

import android.opengl.GLES20;

import com.threed.jpct.GLSLShader;
import com.threed.jpct.Matrix;

import raft.jpct.bones.Animated3D;
import raft.jpct.bones.BonesNamespaceUtils;

/**
 * Created by Dougie on 6/21/16.
 */
public class GPUAnimated3DShader extends GLSLShader {
    int skinWeightsHandle = -1;
    int jointIndicesHandle = -1;

    public GPUAnimated3DShader(String vertexShaderSource, String fragmentShaderSource) {
        super(vertexShaderSource, fragmentShaderSource);

        skinWeightsHandle = GLES20.glGetAttribLocation(getProgram(), BonesNamespaceUtils.ATTR_SKIN_WEIGHTS);
        jointIndicesHandle = GLES20.glGetAttribLocation(getProgram(), BonesNamespaceUtils.ATTR_JOINT_INDICES);
    }

    public void updateBeforeRenderingObject(Animated3D pObject) {
        if(pObject!=null) {
            Matrix[] skelPose = BonesNamespaceUtils.getSkeletonPosePallete(pObject.getSkeletonPose());
            if(skelPose!=null && skelPose.length>0) {
                setUniform(BonesNamespaceUtils.UNIFORM_SKEL_POSE_SIZE, skelPose.length);
                setUniform(BonesNamespaceUtils.UNIFORM_SKEL_POSE, skelPose);
            } else {
                setUniform(BonesNamespaceUtils.UNIFORM_SKEL_POSE_SIZE, 0);
            }
        }
    }
}
</pre>


'''JPCTNamespaceUtils Class:'''

This class must be in the package 'com.threed.jpct'.  This helper class is setup specifically to be able to access namespace restricted variables/methods.
<pre>
package com.threed.jpct;

/**
 * Created by Dougie on 6/18/16.
 */
public class JPCTNamespaceUtils {
    public static boolean VertexAttributesNameIs(VertexAttributes pVertAttrs, String pName) {
        if(pVertAttrs!=null && pName!=null && pVertAttrs.name!=null && pVertAttrs.name.equals(pName))
            return true;
        return false;
    }
}
</pre>


'''BonesNamespaceUtils Class:'''

This class must be in the package 'raft.jpct.bones'.  This helper class is setup specifically to be able to access namespace restricted variables/methods.
<pre>
package raft.jpct.bones;

import com.threed.jpct.Matrix;
import com.threed.jpct.JPCTNamespaceUtils;
import com.threed.jpct.VertexAttributes;

/**
 * Created by Dougie on 6/17/16.
 */
public class BonesNamespaceUtils {
    public static final String ATTR_SKIN_WEIGHTS = "skinWeights";
    public static final String ATTR_JOINT_INDICES = "jointIndices";
    public static final String UNIFORM_SKEL_POSE = "skelPose";
    public static final String UNIFORM_SKEL_POSE_SIZE = "skelPoseSize";

    public static void animate(Animated3D pAnimated3D, float pTime, SkeletonPose pPose) {
        if(pAnimated3D!=null && pAnimated3D.getSkinClipSequence()!=null) {
            pAnimated3D.getSkinClipSequence().animate(pTime, pPose);
        }
    }
    public static void animatePoseDontApply(Animated3D pAnimated3D, float index, int sequence, float weight) {
        if(pAnimated3D!=null) {
            pAnimated3D.animatePoseDontApply(index, sequence, weight);
        }
    }
    public static Matrix[] getSkeletonPosePallete(SkeletonPose pPose) {
        if(pPose!=null) {
            return pPose.palette;
        }
        return null;
    }
    public static void setSkinAttributes(Animated3D pAnimated3D) {
        int i, j, k, len;
        if(pAnimated3D != null && pAnimated3D.getMesh()!=null) {
            boolean hasWeights = false, hasJointIndices = false;
            VertexAttributes[] vertAttrs = pAnimated3D.getMesh().getVertexAttributes();
            if(vertAttrs!=null) {
                //Loop backwards as they should be the last added ones if shared mesh
                for(i=vertAttrs.length-1;i>=0;--i) {
                    if(JPCTNamespaceUtils.VertexAttributesNameIs(vertAttrs[i], ATTR_SKIN_WEIGHTS)) {
                        hasWeights = true;
                    } else if(JPCTNamespaceUtils.VertexAttributesNameIs(vertAttrs[i], ATTR_JOINT_INDICES)) {
                        hasJointIndices = true;
                    }
                    if(hasWeights && hasJointIndices) {
                        break;
                    }
                }
            }

            if(!hasWeights) {
                float[][] weights = pAnimated3D.skin.weights;
                if(weights != null && weights.length > 0) {
                    len = weights[0].length;
                    float[] weightsArr = new float[pAnimated3D.getMesh().getUniqueVertexCount() * len];
                    for (j = 0; j < weights.length; ++j) {
                        for (k = 0; k < len; ++k) {
                            weightsArr[j * len + k] = weights[j][k];
                        }
                    }
                    if (weights.length < pAnimated3D.getMesh().getUniqueVertexCount()) {
                        for (j = weights.length; j < pAnimated3D.getMesh().getUniqueVertexCount(); ++j) {
                            for (k = 0; k < len; ++k) {
                                weightsArr[j * len + k] = 0f;
                            }
                        }
                    }
                    pAnimated3D.getMesh().addVertexAttributes(new VertexAttributes(ATTR_SKIN_WEIGHTS, weightsArr, len));
                }
            }
            if(!hasJointIndices) {
                short[][] jointIndices = pAnimated3D.skin.jointIndices;
                if (jointIndices != null && jointIndices.length > 0) {
                    len = jointIndices[0].length;
                    float[] jointIndicesArr = new float[pAnimated3D.getMesh().getUniqueVertexCount() * len];
                    for (j = 0; j < jointIndices.length; ++j) {
                        for (k = 0; k < len; ++k) {
                            jointIndicesArr[j * len + k] = jointIndices[j][k];
                        }
                    }
                    if (jointIndices.length < pAnimated3D.getMesh().getUniqueVertexCount()) {
                        for (j = jointIndices.length; j < pAnimated3D.getMesh().getUniqueVertexCount(); ++j) {
                            for (k = 0; k < len; ++k) {
                                jointIndicesArr[j * len + k] = 0f;
                            }
                        }
                    }
                    pAnimated3D.getMesh().addVertexAttributes(new VertexAttributes(ATTR_JOINT_INDICES, jointIndicesArr, len));
                }
            }
        }
    }
}
</pre>


'''Vertex Shader Snippet:'''

This just contains the necessary snippets of code to add to your Vertex Shader to have it work.
<pre>
...
uniform mat4 modelViewProjectionMatrix;
...
uniform mat4 skelPose[50];
uniform int skelPoseSize;

attribute vec4 position;
attribute vec3 normal;
...
attribute vec4 skinWeights;
attribute vec4 jointIndices;

void main(void)
{
	vec4 animPosition = vec4(0.0,0.0,0.0, position[3]);
	vec3 animNormal = vec3(0.0,0.0,0.0);
	if(skelPoseSize>0) {
		float weight;
		for(int j=0; j<4; ++j) {
			weight = skinWeights[j];
			if(weight != 0.0) {
				mat4 boneMat = skelPose[int(floor(jointIndices[j]+0.5))];
				animPosition.xyz += vec3(position.x*boneMat[0][0] + position.y*boneMat[1][0] + position.z*boneMat[2][0] + boneMat[3][0],
										position.x*boneMat[0][1] + position.y*boneMat[1][1] + position.z*boneMat[2][1] + boneMat[3][1],
										position.x*boneMat[0][2] + position.y*boneMat[1][2] + position.z*boneMat[2][2] + boneMat[3][2])*weight;
				animNormal += vec3(normal.x*boneMat[0][0] + normal.y*boneMat[1][0] + normal.z*boneMat[2][0],
								  normal.x*boneMat[0][1] + normal.y*boneMat[1][1] + normal.z*boneMat[2][1],
								  normal.x*boneMat[0][2] + normal.y*boneMat[1][2] + normal.z*boneMat[2][2])*weight;
			}
		}
		animPosition.yz *= -1.0;
		animNormal.yz *= -1.0;
	} else {
		animPosition = position;
		animNormal = normal;
	}

	...

	gl_Position = modelViewProjectionMatrix * animPosition;
}
</pre>

== How to Use the Code ==
<ul>
<li>First import/load your animated model using Bones (See Bones documentation on how to load a model).</li>
<li>Create a new GPUAnimated3D() passing in the Animated3D object of your loaded model as the parameter of the constructor.
'''''Please note, this will clone the Animated3D object passed in, reusing/sharing the mesh data.  This was setup this way for my project, as I keep one default instance loaded in memory (not added to the world) for spawning GPUAnimated3D objects.  This methodology may not fit your project.'''''
<li>Create a GPUAnimated3DShader in the same means you would a normal GLSLShader.  You must use a Vertex Shader Source implementing the Vertex Shader Snippet above for it to work.  I recommend only creating one instance of the GPUAnimated3DShader, if possible, and using the same shader for all animated objects (You do not need a unique shader per object).</li>
<li>Add the GPUAnimated3DShader to your GPUAnimated3D by calling the setShader() method passing in your GPUAnimated3DShader as the parameter.
<li>Call your Bones animateSkin() as you normally would on an Animated3D on your GPUAnimated3D.
</ul>


That's it (unless you haven't added your GPUAnimated3D to the World :Þ )!  The GPU will now handle the per vertex skeletal animation calculations.


== FAQ ==
'''''To be written'''''