Low performance when using uncompiled objects

[EgonOlsen]

If it happens only when an object comes into view for the first time that uses a mesh that has never been used on any other object before, then it might be the buffer filling for the compiled data that gets transfered to the GPU. jPCT-AE has a method to do this beforehand (http://www.jpct.net/jpct-ae/doc/com/threed/jpct/World.html#compileAllObjects()), but desktop jPCT hasn't. I guess I didn't see it as an issue on the desktop. I think that I could add it to the desktop version at least when not using the AWTGLRenderer (you aren't using that, are you?) but I want to be sure that it's actually the issue. If it is, there should be a log message like

Subobject of object 12/blah compiled to...

each time the pause happens. Is there?

[Jakes]

I've compiled a sample log from an execution and attached it to the post. (I had to zip it so it could pass the file size limit)
Well, I don't know if its that issue or not, but I'm seeing a lot of these:

    Subobject of object 7225/object7227 compiled to indexed data using 24/9 vertices in 0ms!
   Object 7225/object7227 compiled to 1 subobjects in 0ms!

do you think it could be this?

I think that I could add it to the desktop version at least when not using the AWTGLRenderer (you aren't using that, are you?)

It is in fact written on the log "Using LWJGL's AWTGLCanvas", so I guess it is in fact being used, but does it have anything to do with this issue?

I'm using the FrameBuffer.enableGLCanvasRenderer() to get the renderer I need to use Hardware accelerated rendering.

Note: to help tracking what's happening, I've added information to let you know what's the operation being performed before some blocks, in this format:

"LOG: --------------------[Media] is STARTING----------------"
"LOG: --------------------[Media] is ENDING----------------"

this would be a block of execution, for instance.

[Jakes]

I've done some tests, and it seems that it could be something like what you said.
Every time a new object comes into play and becomes visible inside the camera frustum, a new line of log will appear with the next content followed by some dragging and freezing of the simulation:

   Object 'object1579' shares compiled data with object 'object15'

and the simulation keeps running fine afterwards, until new objects are created in front of the camera while writing more of these lines to the console.

[EgonOlsen]

Ok, I'll go and see if I can port the compileAll method from jPCT-AE or if there was a reason not to other than my false assumption that it isn't needed...

[Jakes]

Many thanks for your time!

[EgonOlsen]

I did something. It's called compileAllObjects(<FrameBuffer>) and can be found in World. It doesn't seem to cause a crash, so that's a good start. I'm not sure how much it helps though. There might still be the uploading of the actual data to GPU left at rendertime, but everything else should be done if you call this method. The GL renderer had be enabled when doing so or otherwise, the method will do nothing. Please give it a try: https://jpct.de/download/net/jpctapi_132b1.zip

[Jakes]

So I tested it out yesterday, and apparently 2 results happened:

- I tested it after the GL context is created (meaning a FrameBuffer) and nothing much changed, perhaps due to the GL data being sent through the pipeline for the first time as you said before.
- I tested it the first time an iteration happens (onProgress) to guarantee that there is in fact an existent GL context, but the result is mostly the same, sometimes faster but its non-deterministic, which I think might have to do with the loading of the GL VBOs into the GPU.

now my question is: How come we can't pre-load all these beforehand on a loading moment, so that those mid simulation loadings are avoided altogether?
Other frameworks allow for a preloading of all data (Textures, animations and 3Dobject models), no? Because no other game or simulation, I ran before, ever done that while in the middle of a running scene.

Isn't there any way to pass all the VBO data into the GPU synchronously beforehand and receive an "ok" when its done so I can progress further on loading the next ones before I can start the simulation itself? (Something like loading a texture, where you send the pixeldata to the GPU and it responds back with ok or not ok, and you proceed to load the next one?)

[Jakes]

I've gathered some information while browsing the web and came across something similar:
  https://community.khronos.org/t/opengl-glbufferdata-cause-stuttering-while-loading-terrain-lod/76292/11

And something to try to solve the asyncronous problem:
  https://www.gamedev.net/forums/topic/677398-opengl-check-if-vbo-upload-is-complete/

but from what I can tell, this seem to be a known problem, and hard to deal with.

My idea is to try to load each object individually in front of the camera, behind an overlay that conceals it so that it can be loaded beforehand in order to avoid any future stuttering of the simulation. Do you whink this will solve the issue itself?

[EgonOlsen]

It might be possible to do the VBO transfer beforehand as well. I'll try to look into it this weekend. The reason why this doesn't happen, is because all GL related operations have to take place in the GL thread (which owns the context). Depending on the renderer, this can either be applications main thread or the awt event dispatch thread of Java or a normal Java thread. To ensure that it always happens in the correct thread, some operations are postponed until jPCT can be sure to be in the right place. I never saw these little hickups as a problem myself (didn't notice them at all), so I kept it that way.
Have you tried to disable VBO support in Config (http://www.jpct.net/doc/com/threed/jpct/Config.html#glUseVBO)? In that case, the transfer doesn't happen and these hickups should go away in case they are really caused by this.

[EgonOlsen]

Try this one. It should also prefill the VBO: https://jpct.de/download/net/jpct.jar

[Jakes]

So I tried to set Config.glUseVBO to false, and did a lot of different combinations and cases, and still got the same results, perhaps my graphics card always tends to use VBO? Some say that the latest cards only have support for VBO and Display Lists are becoming deprecated.

I also tried with uncompiled objects (with and without textures, with more or less objects) but got to the same result.

Try this one. It should also prefill the VBO: https://jpct.de/download/net/jpct.jar

Thanks, I'll give it a go when I get home later on.

Btw, no need to call World.compileAllObjects(FrameBuffer) or any other method on this change?

[EgonOlsen]

No, you need to call that method anyway. It now does the prefilling in addition to what it did before.

[EgonOlsen]

So I tried to set Config.glUseVBO to false, and did a lot of different combinations and cases, and still got the same results, perhaps my graphics card always tends to use VBO? Some say that the latest cards only have support for VBO and Display Lists are becoming deprecated.

I guess the driver could do something like that, yes. After all, OpenGL is a client/server model (with the client being the application and the server being the driver/GPU) and you can't be sure which data resides on the client or gets copied over to the server in the process.

[Jakes]

So, I've run some tests with a lot of different cases, and it seems that everything seemed to be fine with the structures.

I ran some detailed analysis, and I haven't checked any differences whatsoever while using the new Jars, the problem persisted, so I took it a little deeper.

And I found that:
- New tests (with and without the new Jars) indicate that without textures the issue is non-existent.

So I tried to use the TextureManager.preWarm(FrameBuffer) (which I was already using before) in different moments and events, but to no success.
So I was reading the Wiki, and it says: "This method isn't guaranteed to do anything, if the renderer doesn't offer any pre-warming." -> Could it be this? How can I check this?

So, in conclusion it seems that the issue only exists when using big-textured compiled objects, otherwise, it runs smoothly regardless of the numer of vertices/faces or count. The staggering only happens every time a new texture comes to play stuck on an object.

PS: I'm trying to arrange a different setup/machine to try to test out the different scenarios to check if the issues are Mesh, VBO, Display List, Texture or Graphics card related.

[EgonOlsen]

The OpenGL renderer does support it, so it should actually do something. How large are these textures and which graphics card are you using for your tests?