Thanks for the update, this will really benefit everyone who is using shadows with large shadow maps.

I found that your changes made a big difference with large shadow maps – RAM was cut from 1.177GB to 404MB for a shadow map setting of 8192. For small maps, i.e. 1024, I did not see a major difference.

Scenario:
-   Using Config.glUseVBO=true;
-   Removed some of the higher poly models for this test to speed up profiling.
-   Use HW mode with AA.

Test results:

Case 1: Set the max shadow map to 1024 the memory
- Before (Jpct jar prior to memory utilization enhancement):

- Before (Jpct jar prior to memory utilization enhancement):
Memory: 370MB with a spike to 490MB during the first 20 paint frames.
CPU: Spike to 83 percent at startup, stabilizes at 16 percent once running.

- After (Jpct jar after to memory utilization enhancement):
memory: 370MB with a spike to 480MB during the first 20 paint frames.
Cpu: Spike to 82 percent at startup, stabilizes at 16 percent once running.


Case 2: Set the max shadow map to 8192 the memory.

- Before (Jpct jar prior to memory utilization enhancement):
Memory: 1.177GB with a spike to 1.53GB during the first 20 paint frames.
CPU: Spike to 83 percent at startup, stabilizes at 45 percent once running.

- After (Jpct jar after to memory utilization enhancement):
Memory: 404MB with a spike to 570GB during the first 20 paint frames.
CPU: Spike to 83 percent at startup, stabilizes at 45 percent once running.

Case 3: No shadows
Memory: 320MB with a spike to 470MB during the first 5 paint frames.
CPU: Spike to 80 percent at startup, stabilizes at 15 percent once running for several paint() frames.

Again, thanks for the update

Thank you for the quick response and all the suggestions. My results are below. I noted your suggestions, and then the results.

I commented out:
<code>
jpctWorld.renderScene( frameBuffer );
jpctWorld.draw( frameBuffer );
</code>
and still have the same issue. But I'll keep this commented out when the shadow helper is enabled since you mentioned this is done in the shadow helper (thanks for the suggestion).

ATTEMPT 1) Suggestion - Try to lower batch size from 4000 to 2000 to 1000 to...and see if that helps.
2000 gives the same issue.

Result:
I tried 2000, 1000, 500, 100. The lower the value the less the crash happens, especially when the number is set to 100. At 100 I have not been able to cause the crash (yet).

Other observations (when batch size was set to 100).
 I noticed that when I set the max shadow map to 8192 the memory of my java process is 1.6GB of RAM at startup, and then eventually after a minute falls down to 910MB, a huge number. When I disable the shadows completely this drops to 380MB. That is a lot of additional memory to display shadows. Is this typical?

 I noticed that the memory of my java process is 760MB of RAM when I set the max shadow map to 1024 instead of 8192 at startup, and drops to 680MB after a minute.



ATTEMPT 2) Suggestion -  compile(...) with preferDisplayLists set to false. This will be slower but it would be interesting to see if it actually works.
Results: I set the batch size to 10000 to force a consistent JVM crash before applying this change (so I can see if this change alone fixes the issue).

After making this change I did not see the crash (even when I set the batch size to 100). However, like you said, there was a hit in performance. My FPS dropped significantly, to the point my application is too slow.

ATTEMPT 3) Suggestion - Set Config.glUseFBO to false at application startup to see if that helps. Will be slower and limited in shadow map resolution, but again it would be interesting to see if it works that way.
Results: I set the batch size to 10000 to force a consistent JVM crash before applying this change (so I can see if this change alone fixes the issue).

When I made the change I got the same JVM crash consistently.


ATTEMPT 4) Suggestion - Try to synchronize your render loop to the object returned by FrameBuffer.getLock();

I grabbed the lock in the render thread, and ran all the framebuffer calls in a synchronized block as suggested. I really try to avoid doing this, since often it kills performance.

Result: Unfortunately, synchronizing all the frame buffer calls does not work and the exception is the same. The crash still happens after waiting for all the framebuffer code is finished. Here is the output:

<code>
AWT-EventQueue-1 - EVWorld - FPS: 0
AWT-EventQueue-1 - EVWorld - paint() - Entering synchronized block.
AWT-EventQueue-1 - EVWorld - paint() - In synchronized block.
AWT-EventQueue-1 - EVWorld - paint() - Left synchronized block ...
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000010815ea9, pid=10328, tid=9472
# ...
</code>


<code>
j  org.lwjgl.opengl.GL11.nglDrawElements(IIILjava/nio/Buffer;IJ)V+0
j  org.lwjgl.opengl.GL11.glDrawElements(ILjava/nio/IntBuffer;)V+37
j  com.threed.jpct.CompiledInstance.compileToDL()V+139
j  com.threed.jpct.GLBase.compileDLs()V+57
j  com.threed.jpct.AWTGLRenderer.drawVertexArray(Lcom/threed/jpct/AWTDisplayList;I)V+9
j  com.threed.jpct.AWTJPCTCanvas.paintGL()V+149
j  org.lwjgl.opengl.AWTGLCanvas.paint(Ljava/awt/Graphics;)V+165
</code>

ATTEMPT 5) Suggestion - Try to render the first frame without using shadow mapping, i.e. by using the normal renderScene...draw...display-sequence instead of the ShadowHelper.

There are two ways I tried this. First, I still create a new shadowHelper, but I delay calling updateShadowMap() and drawScene() on the shadow helper until after 5 canvas paints. This fails, because the JVM crash is not caused by updateShadowMap() and drawScene() – in fact if I comment them out the crash still happens. So long as a create a new shadow mapper the crash will happen.

The second way is to wait 5 paint cycles before creating a new shadowHelper. The problem with this approach is once the regular rendering sequence is done, and then you later try to create a new shadowHelper, the program freezes indefinitely when trying to create a new Shadow Helper.

ATTEMPT 6) If nothing else helps, you might want to give this jar a try: http://www.jpct.net/download/beta/jpct.jar It ports jPCT-AE's VBO support back to jPCT. You can try to use them instead of display lists by setting Config.glUseVBO=true; before creating the frame buffer.

Result:
The JVM crash did not happen when I set Config.glUseVBO=true.


Conclusion:
Attempt 2 slows things down too much. Attempt 1 forces me to use a small batch size.  Attempt 6 works, does not seem to restrict the batch size, and I don't see a major impact on performance. Attempt 6 adds a little time to startup (to compile VBOs), but this is minor.

Now I have two more questions
- Are there any things I should be aware of when using VBOs?
- Would you kindly suggest ways of reducing memory when using shadows (other than reducing the max shadow map size)? Even for small shadow size values, my memory jumps an additional 100 percent when shadows are enabled.

Thanks again.

Hi,

When I enable shadows, using the shadowhelper, around 60 percent of the time I get a JVM fatal error that causes the application to crash. The other 40 percent the shadows are displayed as expected (my avatar casts a shadow on the floor). Below is the error, the stack trace, and snippets of my code. The error is related to the buffer when openGl is drawing the elements. The crash happened outside the Java Virtual Machine in native code.


#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000000000fc55ea9, pid=2340, tid=8428
#
# JRE version: 6.0_20-b02
# Java VM: Java HotSpot(TM) 64-Bit Server VM (16.3-b01 mixed mode windows-amd64 )
# Problematic frame:
# C  [atio6axx.dll+0x6a5ea9]
#


And the trace:


Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  org.lwjgl.opengl.GL11.nglDrawElements(IIILjava/nio/Buffer;IJ)V+0
j  org.lwjgl.opengl.GL11.glDrawElements(ILjava/nio/IntBuffer;)V+37
j  com.threed.jpct.CompiledInstance.compileToDL()V+139
j  com.threed.jpct.GLBase.compileDLs()V+57
j  com.threed.jpct.AWTGLRenderer.drawVertexArray(Lcom/threed/jpct/AWTDisplayList;I)V+9
j  com.threed.jpct.AWTJPCTCanvas.paintGL()V+149
j  org.lwjgl.opengl.AWTGLCanvas.paint(Ljava/awt/Graphics;)V+165
j  org.lwjgl.opengl.AWTGLCanvas.update(Ljava/awt/Graphics;)V+2


As seen in the above stack, the error occurs when jPCT calls glDrawElements on GL11. I see a few forums that discuss this issue, such as: http://www.gamedev.net/community/forums/topic.asp?topic_id=463211 and http://jmonkeyengine.org/groups/graphics/forum/topic/trouble-with-dynamically-built-geometry/

However, since no one else has posted this error, I'm assuming that I must be doing something wrong. However, the steps are straight forward so I am hoping that this requires a simple change.


Framebuffer mode: FrameBuffer.SAMPLINGMODE_GL_AA_4X
and the width and height is set at runtime based on the browser size calculations.

Also, since this is running in HW mode, I compile the objects as follows:


objs.compile(true,  true, true, true, 4000);


My code snippets:

Initialization code for shadow helper (runs only once):


projector = new Projector();
projector.setFOV(0.5f);
projector.setYFOV(0.5f);

shadowHelper = new ShadowHelper(jpctWorld, frameBuffer, projector, 2048);
shadowHelper.setCullingMode(false);
shadowHelper.setAmbientLight(new Color(80, 80, 80));
shadowHelper.setLightMode(true);      
shadowHelper.setBorder(1);      
shadowHelper.addCaster(avatar);      
shadowHelper.addReceiver(floor);


Then, in the game loop, after the initialization has completed:


frameBuffer.clear();

projector.lookAt(avatar.getTransformedCenter());
projector.setPosition( new SimpleVector(10, -20, 0) );
shadowHelper.updateShadowMap();
shadowHelper.drawScene();

//Code to render scene to buffer, draw to the buffer, etc...
jpctWorld.renderScene( frameBuffer );
jpctWorld.draw( frameBuffer );
frameBuffer.update();
frameBuffer.displayGLOnly();
hardwareOpenGLWorldCanvas.repaint();


Note: when I set the indexed mode to false when compiling objects that cast shadows, that is:

objs.compile(true,  true, true, false, 4000);

then I get a similar error, but the  glDrawArrays is called instead of glDrawElements.


j  org.lwjgl.opengl.GL11.nglDrawArrays(IIIJ)V+0
j  org.lwjgl.opengl.GL11.glDrawArrays(III)V+20
j  com.threed.jpct.CompiledInstance.compileToDL()V+151


I've tried decreasing the shadow buffer to 256 and still the same error. I've looked at other code samples and I don't see any major differences. I don't see a similar issue on the forum, except one where updating the graphics driver solved the problem. Any suggestions of what I can try would be greatly appreciated (except for update your graphics drivers, because I have done that).  Perhaps the way I am using jPCT to specify the Open GL buffer is incorrect? BTW, my card is an ATI Mobility Radeon HD 4650.

Thanks!

Thanks for the response and the explanation.

As for applet security, once I signed the jpct jar file in addition to my own Jars the reflection operation was permitted and the rollback happened as I was hoping (to revert from AA mode to normal HW mode). I suppose the reason that opengl was permitted and reflection was not is because opengl was run under a security context that was initiated from my Jar which is signed and thus trusted. Also, LWJGL jars come pre-signed and I guess for development this works fine (for prod all jars should be signed with the same cert). However, the reflection call must have been running under a different security context that was not trusted, but signing the jpct jar file established the trust that was required.

As for the callback, I no longer need it for this particular scenario since it is working as desired. That being said, I think if you did support a callback for other cases where exceptions occur that would be a nice addition. If you wanted to implement a callback I suppose one way is to use injection similar to the way Spring does. In this case, the client application would inject its own exception handler into jPCT that would implement an interface that jPCT understands (i.e. the interface could have an "exceptionOccured" if you want to be consistent or "handleException" method). When an exeption occurs, jPCT could test whether the client assigned (injected) a value to the handleException instance, and, if it did, then jPCT would call the method on that instance. Right now I see jPCT already has a mode that lets the client specify what should happen on an error (such as continue on or abort) which is nice, but allowing the client to handle exceptions the way they want (such as to display a more informative error web page, etc.) would be helpful. This way if an exception that can't be handled by jPCT occurs, the client will be able to display an error page (or do whatever else) that makes sense in their situation. Otherwise the applet might just go blank if jPCT aborts which is not desirable.

Just a suggestion, I know you probably have bigger fish to fry.

And, thanks again for your help.

Thank you for getting back to me in a timely manner and for offering to include the original exception in the next release. And of course, thanks for this great engine!

Regarding the comment about using reflection, based on your response, I'm assuming that reflection is required, hence I'll have to consider this fact in the restrictive world the applet runs in.

I was hoping to by default use the mode FrameBuffer.SAMPLINGMODE_GL_AA_4X, and, if there is an exception raised, then I would do an internal retry with the mode RENDER_MODE_HARDWARE (and if that fails, resort to the software mode). The problem is that I have no way to intercept the exception and add my own handling routine or try to use a different framebuffer. In other words, I can't extend the hardware canvas since it is returned internally by jPCT via enableGLCanvasRenderer(); Also, I can't intercept the internal jPCT Logger. And in most cases the exception occurs outside my thread stack - i.e.:

at com.threed.jpct.AWTJPCTCanvas.exceptionOccurred(Unknown Source)
at org.lwjgl.opengl.AWTGLCanvas.paint(AWTGLCanvas.java:319)
at org.lwjgl.opengl.AWTGLCanvas.update(AWTGLCanvas.java:334)
at sun.awt.RepaintArea.updateComponent(Unknown Source)
at sun.awt.RepaintArea.paint(Unknown Source)

If you can think of any way that I can get an opportunity to capture an exception that is thrown within the above stack (as an example), without using reflection or anything else that will cause the applet to raise security exceptions, that would be much appreciated. For example, if I could handle the exception, I could present a customized error page instead of a blank applet.

If there is no way to intercept the exception, then unfortunately the choice is that I go with RENDER_MODE_HARDWARE since I need to support both higher end machines and machines with sub-par graphics cards. I would really like to support AA for those cards that can handle it and revert to using RENDER_MODE_HARDWARE if the card can't.

Thanks again for your assistance. 

Dave

Hi,

I noticed that the jPCT exception handler, specifically in the "exceptionOccurred" handler of AWTJPCTCanvas, is triggering a "java.lang.reflect.ReflectPermission suppressAccessChecks" exception when a LWJGLException exception is thrown since I am running an applet in an environment that does not have the reflection permission set.

The problem is that the jPCT "exceptionOccurred" handler is eating the original LWJGLException, and hence the only exception I see is the secondary ReflectPermission (i.e. the only message I see is: ERROR: Unable to recover: access denied (java.lang.reflect.ReflectPermission suppressAccessChecks). Perhaps this handler is trying to do something with the LWJGL exception, and is using reflection in one way or another.

Is there any way to avoid using reflection in the exceptionOccurred method of AWTJPCTCanvas for those environments where reflection is not permitted?

If this is not possible, can you please print the stack trace of the original LWJGLException so that I know what the root cause is? Note: that the case is when I see the jpct message: "Unable to recover: " -- you may print out the LWJGLException in other cases but without the source I can't say for certain.

By the way, I did find the root cause by doing thread dumps, but this was a hit and miss approach. The issue is that my graphics driver was throwing a opengl.WindowsPeerInfo.nChoosePixelFormat related error since I was using a HW framebuffer mode that is not supported by the default drivers.

Thank you! 
Dave