Transparency lost after deserialization

[Promyclon]

Hi,

I used the DeSerializer to speed-up loading of the model I had in OBJ:

Code Select Expand


class SerializeEngine {
private Object3D[] objects;
private String sourceModelFileName;
private String sourceMaterialFileName;

SerializeEngine(final String obj, final String mtl) {
sourceModelFileName = obj;
sourceMaterialFileName = mtl;
}

int loadObj() throws IOException {
try (InputStream modelFile = new FileInputStream(sourceModelFileName);
InputStream materialsFile = new FileInputStream(sourceMaterialFileName)) {
objects = Loader.loadOBJ(modelFile, materialsFile, 1.0f);
for (Object3D o : objects)
o.build();
}
return objects.length;
}

String serialize() throws FileNotFoundException, IOException {
if (null == objects) throw new NullPointerException("Object not loaded");
if (0 == objects.length) throw new IndexOutOfBoundsException("Object is empty");
int l = sourceModelFileName.lastIndexOf('.');
String destModelFileName = ((l <= 0) ? sourceModelFileName : sourceModelFileName.substring(0, l)) + ".j3d";
try (OutputStream dest = new FileOutputStream(destModelFileName)) {
(new DeSerializer()).serializeArray(objects, dest, true);
}
return destModelFileName;
}
}

Then, instead of slow loading from .obj:

Code Select Expand

elements = Loader.loadOBJ(modelStream, materialStream, 1.0f);

I loaded the model in Android from a serialized form:

Code Select Expand

elements = Loader.loadSerializedObjectArray(modelStream);

and got the model loaded in application in jus about one second. The dispalyed model looked fine except for transparent elements. They lost their transparency. Did I do something wrong or the transparency isn't written to/read from the serialized form? I was looking at the jPCT-AE DeSerializer's code and couldn't find any code related with transparency but It's hard to believe this information is just lost.

[EgonOlsen]

No, that information isn't a part of the serialized data. The idea of the serialized format is to handle the actual data the defines the object, not the attributes of an object.

[Promyclon]

When I load object florm .obj, the attributes are taken from .mtl, assigned to objects by material names when the parser finds o or g lines. The material name is 'forgotten' but material's attributes are kept in object. When serializing some of them, like texture names are exported but some are not. If the object is not bound to a material anymore, how can I retrieve it's attributes?

The point is, I have a nice looking model in .obj & .mtl and I want it look nicely when I load it, faster, from a serialized form. And I cannot rewrite code whenever I change a model. For now I have retrieved from .mtl transparency data like that:

Code Select Expand


Map<String, Float> transMap = new TreeMap<>(); // <MaterialName, d attribute>
transMap.put("glass", 0.34f);
transMap.put("windscreen", 0.44f);
transMap.put("lights", 0.52f);


Only because I split earlier all objects to face groups by material, adding material to group names (because transparency is assigned by object an I had multimaterial objects) I could strip the object name from the original name and _jPCTnn suffix and reassign transparency after importing serialized model.

Code Select Expand


for (Object3D element : elements) {
  String elementMaterial = extractMaterialName(element.getName());
  Float transparency = transMap.get(elementMaterial);
  if (null != transparency) element.setTransparency((int) (100f * transparency)); // is this formula correct?
}


This, however, is an workaround that I cannot use in production because each model would need some implementation.

Of course, I can add all needed material attributes to object names (is there a limitation on object length?) but if we went to binary serialization to avoid text parsing overhead, it's not the direction we'd like to go.

[EgonOlsen]

I guess I didn't consider it to be important. I had to make a cut somewhere and transparency didn't make it into the format. I'll look into it. Should be easy to add it.

[Promyclon]

Thank you, Egon! Material data is important and so small, compared to the geometry, that adding it cannot harm

[EgonOlsen]

Please try these jars:

http://jpct.de/download/beta/jpct.jar
http://jpct.de/download/beta/jpct_ae.jar

They should add support for transparency to the (De-)Serializer as well as a new Texture constructor that leaves the stream open if wanted.

[Promyclon]

Hi Egon,

I appreciate so quick update. I've used new jpct.jar (429 037 bytes) in my Java transcoder and got serialized (.j3d) file just one byte bigger, having stored 166 object in this file.
I copied jpcr_ae.jar (388 667 bytes) into my Android project, invalidated Android Studio caches, rebuilt a project and tried to load my new .j3d file having the following message:

08-22 06:26:29.204 4388-4411/gov.nasa.iss E/jPCT-AE: [ 1503383189204 ] - ERROR: Can't deserialize object: float[] array expected (410)!

So, something went wrong and I have now only programatically generated surface in my scene.

[EgonOlsen]

I'll look into it..

[Promyclon]

Further test showed that the new jpct_ae.jar successfully loads the old .j3d.

I've also checked the new constructor

Code Select Expand

Texture(InputStream is, boolean useAlpha, boolean closeStream)

After reading the first texture the ZipInputStream was closed. I've found that the chain of passing closeStream parameter is broken here:

Code Select Expand


private void loadTexture(InputStream is, Bitmap img, boolean alpha, boolean closeStream) {
        this.isLoaded = false;
        Logger.log("Loading Texture...", 2);

        try {
            Bitmap image = img;
            boolean recycleMe = false;
            if(img == null) {
                image = BitmapHelper.loadImage(is); // closeStream defaults to true there :(
                recycleMe = true;
            }


And, finally, my mistake: new .j3d has 16 366 092 bytes while the old is 16 364 764 bytes long, that's 1328 bytes more in new file (I took 1kB for 1B), 8 bytes more per each object.

[EgonOlsen]

Oops...I've obviously forgotten to take the parameter into account. I'll fix that later.

[EgonOlsen]

Please re-download and try again...

[Promyclon]

I've re-exported the model, removed workarounds, launched the app and the scene still looks good. Thanks Egon, your betas work!