Hybrid GPU Shader Animations for Bones
Overview
While working on an Android project, I (alias Redman) found that jPCT+Bones for skeletal animations were becoming too CPU intensive on the UI thread. When adding twenty animated models on screen, I was noticing a considerable drop in the frame-rate. So I developed a hybrid solution of offloading calculations to the GPU for my project, which I will share with you now.
I have added notes and personal recommendations. This architecture may not be perfect for your project, so please alter and use to fit your needs.
Comments and suggestions are welcome as I will be adding a FAQ to the bottom of the page.
Requirements
- JPCT (requires GL ES 2+ as it uses GLSL Shaders)
- Bones
How It Works
This implementation works by offloading all of the vertex calculations to the GPU. The bone matrix transforms for the poses are still performed on the CPU side.
Gains: Speed. You will harness the GPU which is setup to handle floating point calculations quickly and is dedicated to such tasks. As an example with my project: having 20 animated models dropped the frame-rate to 8-9 fps on my device. Using this method, it went back up to the 60-62 fps cap.
Losses: Software knowledge of the transformed pose. Since the meshes of your models will not be animated on the software-side, it will have no knowledge of its transformed pose. The polygons will be in the initial state, which will not give you an accurate bounding-box, ray-trace collisions, etc... of the current animated mesh pose as it will not actually be transformed on the software-side.
The Code
This code is free, open-source code and you are welcome to use it as you see fit.
GPUAnimated3D Class:
package yournamespacegoeshere;
import yournamespacegoeshere.GPUAnimated3DShader;
import com.threed.jpct.GLSLShader;
import com.threed.jpct.IRenderHook;
import com.threed.jpct.Object3D;
import raft.jpct.bones.Animated3D;
import raft.jpct.bones.BonesNamespaceUtils;
import raft.jpct.bones.SkinClip;
/**
* Created by Dougie on 6/21/16.
*/
public class GPUAnimated3D extends Animated3D implements IRenderHook {
public GPUAnimated3D(Animated3D object) {
super(object);
setSkeletonPose(new SkeletonPose(getSkeleton()));
BonesNamespaceUtils.setSkinAttributes(object);
setRenderHook(this);
}
public void animateSkin(float index, int sequence) {
if(getSkinClipSequence() != null) {
if(sequence == 0) {
BonesNamespaceUtils.animate(this, index*getSkinClipSequence().getTime(), getSkeletonPose());
} else {
SkinClip clip = getSkinClipSequence().getClip(sequence - 1);
clip.applyTo(index * clip.getTime(), getSkeletonPose());
}
getSkeletonPose().updateTransforms();
}
}
public void animatePose(float index, int sequence, float weight) {
BonesNamespaceUtils.animatePoseDontApply(this, index, sequence, weight);
}
@Override
public void beforeRendering(int i) { ; }
@Override
public void afterRendering(int i) { ; }
@Override
public void setCurrentObject3D(Object3D object3D) { ; }
@Override
public void setCurrentShader(GLSLShader glslShader) {
if(glslShader!=null && glslShader instanceof GPUAnimated3DShader) {
((GPUAnimated3DShader)glslShader).updateBeforeRenderingObject(this);
}
}
@Override
public void setTransparency(float v) { ; }
@Override
public void onDispose() { ; }
@Override
public boolean repeatRendering() { return false; }
}
GPUAnimated3DShader Class:
package yournamespacegoeshere;
import android.opengl.GLES20;
import com.threed.jpct.GLSLShader;
import com.threed.jpct.Matrix;
import raft.jpct.bones.Animated3D;
import raft.jpct.bones.BonesNamespaceUtils;
/**
* Created by Dougie on 6/21/16.
*/
public class GPUAnimated3DShader extends GLSLShader {
int skinWeightsHandle = -1;
int jointIndicesHandle = -1;
public GPUAnimated3DShader(String vertexShaderSource, String fragmentShaderSource) {
super(vertexShaderSource, fragmentShaderSource);
skinWeightsHandle = GLES20.glGetAttribLocation(getProgram(), BonesNamespaceUtils.ATTR_SKIN_WEIGHTS);
jointIndicesHandle = GLES20.glGetAttribLocation(getProgram(), BonesNamespaceUtils.ATTR_JOINT_INDICES);
}
public void updateBeforeRenderingObject(Animated3D pObject) {
if(pObject!=null) {
Matrix[] skelPose = BonesNamespaceUtils.getSkeletonPosePallete(pObject.getSkeletonPose());
if(skelPose!=null && skelPose.length>0) {
setUniform(BonesNamespaceUtils.UNIFORM_SKEL_POSE_SIZE, skelPose.length);
setUniform(BonesNamespaceUtils.UNIFORM_SKEL_POSE, skelPose);
} else {
setUniform(BonesNamespaceUtils.UNIFORM_SKEL_POSE_SIZE, 0);
}
}
}
}
JPCTNamespaceUtils Class:
This class must be in the package 'com.threed.jpct'. This helper class is setup specifically to be able to access namespace restricted variables/methods.
package com.threed.jpct;
/**
* Created by Dougie on 6/18/16.
*/
public class JPCTNamespaceUtils {
public static boolean VertexAttributesNameIs(VertexAttributes pVertAttrs, String pName) {
if(pVertAttrs!=null && pName!=null && pVertAttrs.name!=null && pVertAttrs.name.equals(pName))
return true;
return false;
}
}
BonesNamespaceUtils Class:
This class must be in the package 'raft.jpct.bones'. This helper class is setup specifically to be able to access namespace restricted variables/methods.
package raft.jpct.bones;
import com.threed.jpct.Matrix;
import com.threed.jpct.JPCTNamespaceUtils;
import com.threed.jpct.VertexAttributes;
/**
* Created by Dougie on 6/17/16.
*/
public class BonesNamespaceUtils {
public static final String ATTR_SKIN_WEIGHTS = "skinWeights";
public static final String ATTR_JOINT_INDICES = "jointIndices";
public static final String UNIFORM_SKEL_POSE = "skelPose";
public static final String UNIFORM_SKEL_POSE_SIZE = "skelPoseSize";
public static void animate(Animated3D pAnimated3D, float pTime, SkeletonPose pPose) {
if(pAnimated3D!=null && pAnimated3D.getSkinClipSequence()!=null) {
pAnimated3D.getSkinClipSequence().animate(pTime, pPose);
}
}
public static void animatePoseDontApply(Animated3D pAnimated3D, float index, int sequence, float weight) {
if(pAnimated3D!=null) {
pAnimated3D.animatePoseDontApply(index, sequence, weight);
}
}
public static Matrix[] getSkeletonPosePallete(SkeletonPose pPose) {
if(pPose!=null) {
return pPose.palette;
}
return null;
}
public static void setSkinAttributes(Animated3D pAnimated3D) {
int i, j, k, len;
if(pAnimated3D != null && pAnimated3D.getMesh()!=null) {
boolean hasWeights = false, hasJointIndices = false;
VertexAttributes[] vertAttrs = pAnimated3D.getMesh().getVertexAttributes();
if(vertAttrs!=null) {
//Loop backwards as they should be the last added ones if shared mesh
for(i=vertAttrs.length-1;i>=0;--i) {
if(JPCTNamespaceUtils.VertexAttributesNameIs(vertAttrs[i], ATTR_SKIN_WEIGHTS)) {
hasWeights = true;
} else if(JPCTNamespaceUtils.VertexAttributesNameIs(vertAttrs[i], ATTR_JOINT_INDICES)) {
hasJointIndices = true;
}
if(hasWeights && hasJointIndices) {
break;
}
}
}
if(!hasWeights) {
float[][] weights = pAnimated3D.skin.weights;
if(weights != null && weights.length > 0) {
len = weights[0].length;
float[] weightsArr = new float[pAnimated3D.getMesh().getUniqueVertexCount() * len];
for (j = 0; j < weights.length; ++j) {
for (k = 0; k < len; ++k) {
weightsArr[j * len + k] = weights[j][k];
}
}
if (weights.length < pAnimated3D.getMesh().getUniqueVertexCount()) {
for (j = weights.length; j < pAnimated3D.getMesh().getUniqueVertexCount(); ++j) {
for (k = 0; k < len; ++k) {
weightsArr[j * len + k] = 0f;
}
}
}
pAnimated3D.getMesh().addVertexAttributes(new VertexAttributes(ATTR_SKIN_WEIGHTS, weightsArr, len));
}
}
if(!hasJointIndices) {
short[][] jointIndices = pAnimated3D.skin.jointIndices;
if (jointIndices != null && jointIndices.length > 0) {
len = jointIndices[0].length;
float[] jointIndicesArr = new float[pAnimated3D.getMesh().getUniqueVertexCount() * len];
for (j = 0; j < jointIndices.length; ++j) {
for (k = 0; k < len; ++k) {
jointIndicesArr[j * len + k] = jointIndices[j][k];
}
}
if (jointIndices.length < pAnimated3D.getMesh().getUniqueVertexCount()) {
for (j = jointIndices.length; j < pAnimated3D.getMesh().getUniqueVertexCount(); ++j) {
for (k = 0; k < len; ++k) {
jointIndicesArr[j * len + k] = 0f;
}
}
}
pAnimated3D.getMesh().addVertexAttributes(new VertexAttributes(ATTR_JOINT_INDICES, jointIndicesArr, len));
}
}
}
}
}
Vertex Shader Snippet:
This just contains the necessary snippets of code to add to your Vertex Shader to have it work.
...
uniform mat4 modelViewProjectionMatrix;
...
uniform mat4 skelPose[50];
uniform int skelPoseSize;
attribute vec4 position;
attribute vec3 normal;
...
attribute vec4 skinWeights;
attribute vec4 jointIndices;
void main(void)
{
vec4 animPosition = vec4(0.0,0.0,0.0, position[3]);
vec3 animNormal = vec3(0.0,0.0,0.0);
if(skelPoseSize>0) {
float weight;
for(int j=0; j<4; ++j) {
weight = skinWeights[j];
if(weight != 0.0) {
mat4 boneMat = skelPose[int(floor(jointIndices[j]+0.5))];
animPosition.xyz += vec3(position.x*boneMat[0][0] + position.y*boneMat[1][0] + position.z*boneMat[2][0] + boneMat[3][0],
position.x*boneMat[0][1] + position.y*boneMat[1][1] + position.z*boneMat[2][1] + boneMat[3][1],
position.x*boneMat[0][2] + position.y*boneMat[1][2] + position.z*boneMat[2][2] + boneMat[3][2])*weight;
animNormal += vec3(normal.x*boneMat[0][0] + normal.y*boneMat[1][0] + normal.z*boneMat[2][0],
normal.x*boneMat[0][1] + normal.y*boneMat[1][1] + normal.z*boneMat[2][1],
normal.x*boneMat[0][2] + normal.y*boneMat[1][2] + normal.z*boneMat[2][2])*weight;
}
}
animPosition.yz *= -1.0;
animNormal.yz *= -1.0;
} else {
animPosition = position;
animNormal = normal;
}
...
gl_Position = modelViewProjectionMatrix * animPosition;
}
How to Use the Code
- First import/load your animated model using Bones (See Bones documentation on how to load a model).
- Create a new GPUAnimated3D() passing in the Animated3D object of your loaded model as the parameter of the constructor. Please note, this will clone the Animated3D object passed in, reusing/sharing the mesh data. This was setup this way for my project, as I keep one default instance loaded in memory (not added to the world) for spawning GPUAnimated3D objects. This methodology may not fit your project.
- Create a GPUAnimated3DShader in the same means you would a normal GLSLShader. You must use a Vertex Shader Source implementing the Vertex Shader Snippet above for it to work. I recommend only creating one instance of the GPUAnimated3DShader, if possible, and using the same shader for all animated objects (You do not need a unique shader per object).
- Add the GPUAnimated3DShader to your GPUAnimated3D by calling the setShader() method passing in your GPUAnimated3DShader as the parameter.
- Call your Bones animateSkin() as you normally would on an Animated3D on your GPUAnimated3D.
That's it (unless you haven't added your GPUAnimated3D to the World :Þ )! The GPU will now handle the per vertex skeletal animation calculations.
FAQ
To be written
