3D Sound System

[paulscode]

After running some better tests with the MIDI stuff, it is starting to look less appealing.  The MIDI events often take up a noticable amount of time to run, so the program hangs at random points when MIDI playback is running on the main thread.  I may be able to fix this by running the MIDI on a seperate thread.  For me, this destroys the appeal of MIDI alltogether.  I guess since I put a great deal of effort into coding the thing, instead of just deleting it, I will probably go ahead and add in a MIDI playback thread.  It will only be spawned if MIDI is played, so it will not be an issue if MIDI is not being used by a program.  At this poing, I doubt I'll be using MIDI in my game anyway.  I'll just convert any MIDI music to OGG format and stream it.  I'll go ahead and leave the functionality in there just in case someone else wishes to use MIDI.

As for .mp3 format, there does not appear to be any non-GPL source code out there, so I am not adding support for the .mp3 format at this time.  Eventually I may come back and write my own .mp3 decoder, but that is REALLY low-priority (lot of work for very little gain).  It is easy enough to find file format converters anyway, and .ogg seems to be a well supported compressed audio format.

I've figured out how to manipulate source volume and master volume in Javasound (pretty easy, actually), but I am having trouble doing the same for OpenAL.  The problem is that to manually alter the "gain" for a source, it prevents me from using OpenAL's built-in rolloff attenuation model.  I could bypass the built-in rolloff attenuation stuff and use my own rolloff attenuation code that I wrote for Javasound, but that is something I would like to avoid if at all possible.

I'm not sure if it would be better to have the ability to change volume or to just leave SoundSystem the way it is.  Does anyone have any thoughts on this?

[EgonOlsen]

It would be nice to have the option to adjust the volume, but personally, i can easily live without that.

[paulscode]

Good news!  I figured out how to alter a source's volume in OpenAL without having to bypass the built-in rolloff attenuation stuff.  I have finished implementing master volume and per-source volume for both libraries.  Usage is simple.  There are two new methods available in the SoundSystem class.  Both take a float value for volume, in the range 0.0f - 1.0f.

Code Select Expand

setVolume( String sourcename, float value );
setMasterVolume( float value );

I think I have finished adding new stuff to SoundSystem, so now I'll get back to fixing the SoundManager problem.

[paulscode]

I solved the stop() bug in SoundManager.  Now there seems to be a logic error somewhere in the source management code.  If the source count stays less than 50-75, it seems to work great.  As the number becomes larger, though, it seems like the wrong sources are playing (i.e. not necessarily the closest ones).  I will try my best to solve this problem tomorrow if possible, so I can post the next release of SoundManager (and SoundSystem).

I also discovered today that the coordinate system for jPCT and the one for SoundSystem are backwards as far as the z-axis is concerned.  This has been my first oportunity (since I fixed the panning problem from the old version of SoundManager) to bind a Source to an Object3D and both visually and audibly compare the two.  The problem exists because OpenAL thinks -z means "forward" and +z means "reverse" (when +y is "up" and +x is "right").  jPCT, on the other hand, thinks z means "forward" and -z means "reverse" (to me, this makes more sense, which is probably why the panning code I wrote for Javasound was originally backwards from OpenAL).

There are two ways I can compensate for this problem.  The first way would be to change SoundSystem so that it reverses all z coordinates supplied for methods like "setPosition()" "setListenerOrientation()", etc., thus making SoundSystem behave like jPCT.  The second way would be to leave SoundSystem alone, and instead reverse the z coordinates in the SoundManager tick() method after it reads the coordinates from the Camera and Object3D's.  Any thoughts on which way sounds better?

[EgonOlsen]

I would go for the second way. +y up and -z into the screen is more common than what jPCT does, albeit i find +z into the screen more logical, which is why i choosed that one. What does that mean for the setListenerPosition()-method in SoundSystem? Do i have to have to do something like

Code Select Expand


soundSys.setListenerPosition(pos.x, -pos.y, -pos.z);


to be correct? Currently, i'm simply doing

Code Select Expand


soundSys.setListenerPosition(pos.x, pos.y, pos.z);


which seems to work for me, but my y doesn't change anyway. Z may be reversed then, but it's impossible to hear that on stereo speakers only.

[paulscode]

+y up and -z into the screen is more common than what jPCT does

jPCT and OpenAL have been my first experience with 3D programming, so I wasn't aware of that.  I agree that +z to the screen seems more logical, but if the other way is more common, I should probably leave SoundSystem as it is, and just change the SoundManager tick() method where SoundSystem Listener and Source's follow jPCT Camera and Object3D's.

What does that mean for the setListenerPosition()-method in SoundSystem? Do i have to have to do something like

Code Select Expand


soundSys.setListenerPosition(pos.x, -pos.y, -pos.z);


which seems to work for me, but my y doesn't change anyway. Z may be reversed then, but it's impossible to hear that on stereo speakers only.

I hadn't considered that the y-coordinates may be reversed as well.  I was just reversing the z-coordinates.  I know FOR SURE that +y is "up" in OpenAL, but I have not used jPCT enough to notice what the up-directon is there (none my demo applets have manipulated the listener y-coordinates).  Just to make sure, is "up" -y in jPCT?  If so, then you would be correct in reversing all y-coordinates as well.  If the y-coordinates were backwards, my guess is it would be quite noticable in a game where the Camera/Listener had free motion through 3-D space (although I have never tested such a scenerio).

In addition to setting the listener's position as you indicated above, you should also set the listener's orientation as well (at least once), or the listener will be facing the wrong direction and be upside-down by default.  Hmm... come to think of it, that is probably what you meant when you said "Z may be reversed then".

ANYWAY, to reverse and flip the default listener orientation use this:

Code Select Expand


soundSys.setListenerOrientation( 0, 0, -1, 0, -1, 0 );   // look toward -z, up toward -y


Or you can match the Listener orientation to the Camera (or to any normalized jPCT SimpleVector's for that matter) using something like this:

Code Select Expand


SimpleVector direction = camera.getDirection();
SimpleVector up = camera.getUpVector();
soundSys.setListenerOrientation( direction.x, -direction.y, -direction.z, up.x, -up.y, -up.z );  // both y and z coordinates reversed



Also, don't forget to reverse the y and z coordinates when creating new sources (and when moving them):

Code Select Expand


SimpleVector pos = myObject3D.getTransformedCenter();
soundSys.quickPlay( "explosion.wav", false, pos.x, -pos.y, -pos.z );


[EgonOlsen]

Just to make sure, is "up" -y in jPCT? 

Yes, it is. That's the drawback of having +z going into the screen.

BTW: I accidently edited your post instead of replying...i hope that i've rebuild it as close as possible...sorry...

[paulscode]

I thought I would explain what you'd experience when trying to use jPCT coordinates in SoundSystem without reversing the z-direction (just in case anyone is having trouble visualizing what I'm talking about).  I drew a simple diagram representing an Object3D moving in the +x direction (viewed from above):



As you can see from this diagram, the jPCT Camera would "see" the object moving to the right, while the SoundSystem Listener would "hear" an attached source moving to the left, because it would be "listening" from the opposite direction.

[paulscode]

BTW: I accidently edited your post instead of replying...i hope that i've rebuild it as close as possible...sorry...

hehe, np.

[paulscode]

Sound System, Bug Fixes and Additions

JAR:
http://www.paulscode.com/libs/SoundSystem/14SEP2008/SoundSystem.jar

Source Code:
http://www.paulscode.com/source/SoundSystem/14SEP2008/SoundSystemSource.zip

JavaDoc:
LINK NO LONGER EXISTS

Test Applet:
LINK NO LONGER EXISTS

Resources JAR: (only required when playing MIDI from an applet)
http://www.paulscode.com/libs/SoundSystem/SoundSystemResources.jar

Rather than waiting until I finish fixing SoundManager, I decided to go ahead and post the current version of SoundSystem, since there have been a couple of important bug fixes:

1) The main bug fix is that I was not calling each source's "positionChanged()" method when the listener moved or changed orientation, so "distanceFromListener" was not being updated.  There was no noticable problem in OpenAL when using rolloff-attenuation (because the gain is set internally by OpenAL in that case).  The bug was apparent in linear attenuation under OpenAL and in both attenuation models under JavaSound.  In JavaSound, this bug also caused panning to not update when the listener orientation changed.

2) The minor bug fix is that I had the default listener orientation set to (0, 0, 1, 0, 1, 0) instead of (0, 0, -1, 0, 1, 0), because I did not realize that OpenAL considered "-z" to be "forward" and "z" to be "reverse".  So in other words, the listener by default was facing the wrong direction (toward the player) instead of facing forward.  This explains why Egon was able to use jPCT coordinates for sources in his "Nameless Bomberman Clone" game without noticing any problems (although technically coordinates were backwards and upside down, but you can't hear the difference between that and facing the correct direction, when playing on normal stereo speakers).

Another major update in this version is the ability to change each source's volume, and there is also a master volume which affects all sources.  I think this is an important addition to the library, since most games have configuration menus which allows the player to manually change the volume, turn down or mute the music, etc.

-- EDIT --
One more new thing in this version is the ability to play back MIDI.  Playback of other sources is a bit choppy when MIDI is playing while the JavaSound library is being used, but simultaneous MIDI and other sources both play smoothly under OpenAL.  I have no intention of working on MIDI any time soon, so it is available "as is" for now.

-- EDIT #2 --
Another bug that is fixed in this version is the regular expression for SoundSystemConfig.PREFIX_URL.  Before, I had accidentaly typed in an extra [tT] (i.e. using the expression would search for "htttp://" instead of "http://").

[EgonOlsen]

I've dropped the new sources into my game...everything works fine! 

[paulscode]

SoundManager, Version 2

JAR:
http://www.paulscode.com/libs/SoundSystem/15SEP2008/SoundManager.jar

JavaDoc:
LINK NO LONGER EXISTS

Source Code:
http://www.paulscode.com/source/SoundSystem/15SEP2008/SoundManagerSource.zip

Demo Applet:
http://www.paulscode.com/demos/SoundSystem/15SEP2008/

Demo Applet Source Code:
http://www.paulscode.com/demos/SoundSystem/15SEP2008/HelicopterSource.zip

I finally fixed the source management bug I was having, and everything appears to be working ok now.  Initial tests show that source management is still rather resource-intensive.  I will continue to try and streamline the source management process, but I can't promise any big optimization breakthroughs.

The demo applet listed above is just a modified version of the Helicopter demo applet from before.  I changed it that you can see the different source management models at work.  What it does is make the balls red when they have an active (non-culled) source attached.  Balls with culled sources attached appear grey.

After finishing SoundManager and weighing the pros and cons between it and SoundSystem alone, I have decided not to use SoundManager in my game.  I decided to go ahead and release SoundManager in case anyone else finds it usefull, though, and I will work to fix any bugs that people find.  For my game, though, I'm going to just make a jPCT version of SoundSystem which has SoundManager's two "bind" methods for attaching the listener to the camera and sources to object3D's and the tick() method to call in the main game loop.  The way SoundSystem handles sources is more than sufficient, in my opinion.  This new jPCT version of SoundSystem will have an additional "quickPlay" method which takes an Object3D as an argument (automatically binding the source to the Object3D).  I'll post the JAR and source code when I finish it.

[paulscode]

SoundSystemJPCT

JAR:
http://www.paulscode.com/libs/SoundSystem/15SEP2008/SoundSystemJPCT.jar

JavaDoc:
LINK NO LONGER EXISTS

Source Code:
http://www.paulscode.com/source/SoundSystem/15SEP2008/SoundSystemJPCTSource.zip

Here is the jPCT-friendly version of SoundSystem.  Instead of overwriting the SoundSystem class, I decided to just extend it.  That way, any updates and bug fixes in SoundSystem will carry over to SoundSystemJPCT.  I added in a whole bunch of methods for creating sources, quickPlay, moving sources, listener orientation, etc.  These new methods take SimpleVector arguments.  To make things easy for interfacing with jPCT, all methods which take SimpleVectors as arguments assume the SimpleVectors are using the jPCT coordinate system, and they convert them to the OpenAL coordinate system automatically.  The JavaDoc goes into more detail about that.

Next thing I am going to work on is some user-friendly "getting started" tutorials.  I am suspect SoundSystem looks really complicated when you are staring at the JavaDoc or source code, so I think a guide will let people know how simple SoundSystem really is (especially compared to any other 3D sound library).

[EgonOlsen]

FYI: Your SoundSystem works fine with LWJGL 2.0rc2...just in case you haven't tried it yet.

[EgonOlsen]

Is there a way to determine if SoundSystem has been initialized correctly? I have a problem with LWJGL2.0rc2 on one machine and OpenAL doesn't initialize (not your fault, i think...must be some flaw in rc2) correctly. But the SoundSystem seems to discard this information except that it prints out the stack trace, but that doesn't allow me to react to this problem. If i could, i would simply fall back to JavaSound instead. Why don't you propagate the LWJGLException (or maybe a simple RuntimeException) up to the constructor of SoundSystem, so that i can catch it!?