3D Sound System

[paulscode]

I'm back from vacation, and I'll get back to finishing this project soon.  I have a couple other non-jPCT projects I'm working on as well.

[paulscode]

I spent some time going back through the code.  It is always a challenge to come back to a project after not looking at it for a while.  I came across something I had wanted to change a while back.  I had put it on a back burner to work on other things, and then just forgot about it.  It is related to loading a .ogg file into a non-streaming source.  I had originally used the code from the .ogg streamer, and was just loading in a single chunk of data of a pre-determined size.  Anything larger required the user to change a variable (MAX_OGG_SIZE or something), which seems really hackish to me.  I decided to work on this problem for a while, to get back into the swing of things.  I want it to read in the entire file in seperate chunks of data, and then combine them into a single sound buffer.  I haven't got this working yet, so I'll do some more work on it today.  Also, messing around with the .ogg related code seems to have broken .ogg streaming   I'm not getting any exceptions or errors - streaming sources are just not playing.  I'll have to track that problem down next.

[EgonOlsen]

Hurry up...i need sound!  Good to see that you are back on track.

[paulscode]

Ok, I got non-streaming .oggs being completely loaded rather than just the first chunk.  This was actually a memory-saving fix.  Before, every (non-streaming) .ogg sound took up exactly the same amount of memory.  Now they now use up only as much memory as needed to load them.  I decided to keep in the maxOggSize configuration varriable there just to prevent a virtually infinite loop in a case where an .ogg was insanely stupid-huge or if there was some strange problem during load time that prevented OggDecoder from reaching the end of the file.  By default I set the max .ogg size at 256 MB (I don't think anyone will be trying to load anything that ginormous - should be streaming that sucker anyway!), and the user is able to change that number if so desired.  As you might expect, when loading large .ogg sounds, the application will hang while waiting for it to load.  This can of course be prevented by streaming large sounds instead of pre-loading them.

I have not fixed the "streaming-sources not playing" problem yet.  I can't seem to track it down.  I am sure it is some sort of logic error, I just have to find it.

[paulscode]

Man, this is getting rediculous!  I seriously cannot figure out what's wrong.  I have put in println messages to see where it is breaking down, and everything seems fine up to the point where it goes to read in data.  Then it gets into an infinite loop of saying "Reading a chunk of data", but nothing plays.  This is really strange -  I mean, streaming was working fine before - I don't know what I did to break it.  Unfortunately, I didn't keep a recent backup of the code before I messed around with it, and I have been looking at this for a week now without success.  I am thinking that the project has become so large I am having a lot of trouble just swimming through the code.  I have decided to start stripping things down to the bare bones and hopefully I will be able to find the source of the problem.  I'll start by cutting out all the Javasound code, since I know there are other problems in that part, and focus on getting the OpenAL part working again.  Of course I will keep a backup so I can put everything back together after I fix this darn streaming (or should I say not streaming) problem.

[paulscode]

Problem solved!  I had to tear the project apart, but I finally found the problem!  I was not preloading a block of data before starting the stream loop (yeh - something really simple).  I must have accidentally deleted the line that called this method when I was removing comments or debug messages.  A single line of code can really kill things sometimes!

Anyway, I am hopefully back on track now.  I just need to plug all the Javasound code back in, and fix the problems I was having with streaming oggs in Javasound.  It is finally looking like there may be a light at the end of the tunnel!

[EgonOlsen]

Go, go, go...

[paulscode]

I am nearly complete.  All the Javasound stuff is back in place, and I have streaming working as well.  The only thing left I need to figure out is how to reserve all the available channels and reuse them (like I am doing in the OpenAL part).  Javasound is different enough from OpenAL that it is not immediately obvious to me how to acomplish this.  I have some ideas - I just need to make some test programs to see what works.

[paulscode]

I found a method that seems to work.

The main problem I was having was trying to use "Clip" for nonstreaming sources and "SourceDataLine" for streaming sources.  Theoretically, this would be the best way, but these two classes do not work interchangably, and managing both types was getting to be a headache.  The only way I can get around this is to use SourceDataLine for every source -- in other words "stream" everything.  For a "nonstreaming" source, all the data will be queued to play at once, whereas in a "streaming" source, buffers are used to queue the data as usual.  This may not be the ideal method, but it works.

Another problem I was having was that when instantiating a SourceDataLine, it takes "format" as a parameter (which, for my purposes, is unknown until the user actually wants to play a source).

A third problem is that I read online that there can be any number of SourceDataLines created, but trying to "open()" more than 32 simultaneously results in an exception.  I haven't actually tested this, but I've read the same thing from more than one website.  It seems reasonable that the maximum number of SourceDataLines would be constant since the Javasound "Mixer" is "software-rendered", from what I understand.  I plan to test to make sure the number 32 can be trusted here even if other programs with sound are running.  Worse case scenario, it may turn out that, as with OpenAL, I'll just have to live with the uncertainty of how many sources can be played simultaniously.  Either way, I will write my code to "catch" any exceptions that happen due to running out of voices.

So my solution works like this: I create a mixer when Javasound initializes, and I create a list of 32 SourceDataLine varriables (they are all "null" to start with).  "Loading" a sound file stores its format and byte data for later use, but it does nothing to the SourceDataLines.  Then, whenever a source is to be played, a SourceDataLine is instantiated with the correct format, set to the correct gain and pan information, and supplied with byte data for whatever sound is to be played.  If the end of the list is reached and no SourceDataLines are empty, the program loops back to the first SourceDataLine, stops and closes it, reinstantiates it with the new format, changes the gain and pan, and supplies it with the new source data.

Preliminary tests show this method to be working, so I am now in the process of moving the code over to my SoundSystem library.  There will also be some minor tweaking for things like "priority sources" and the like.  I will let everyone know how this goes.  I am nearly ready to run some final tests and release the SoundSystem library (finally!)


--EDIT--
Oh, BTW, I do not have a working formula for calculating pan in the Javasound library.  Basically I have vectors for souce position and listener position, and normal vectors for look-at point and up direction.  From that I need to calculate a float between -1 (left speaker) and 1 (right speaker).  My math skills are far to poor to handle a problem like this, so I have been looking for a formula online that someone else has written (so far without luck).  In the mean time, Javasound will just not have panning 3D sources.  BTW, if any of you are more "mathematically endowed" than me, suggestions would be greatly appreciated.

--EDIT 2--
I found someone to help me out with the panning formula.  The psudocode they gave me is:

Code Select Expand


vec3 side = cross(listener.up, listener.look_at);
side.normalize();
float x = dot(source.position - listener.position, side);
float z = dot(source.position - listener.position, listener.look_at);

float angle = atan2(x, z);

// One way you could compute the panning value from the angle:
float pan = sin(angle);

I am going to try and implement this to see if it works (crossing my fingers).

[paulscode]

YES!  SoundSystem is finally finished!  Everything is in place and seems to be working with no problems.  Just to be safe, I am going to run some more extensive tests tomorrow.  I also need to write up some documentation for using the library.  I will try and post an initial release tomorrow afternoon.

[EgonOlsen]

Good news. I guess this means that i can now start to search for some cool sounds for my game...

[paulscode]

SoundSystem Library, Alpha Release

I was called in to work today, so I didn't get around to the testing that I planned to do.  However, I decided to release the library today, anyway, in case anybody wants to start getting familiar with it (I can't guarantee there are no bugs).

I am going to start writing up some documentation, and I'll try and post that by tomorrow evening.  Then I'll begin running some serious tests and fixing bugs.  I'll post updates every couple of days as things get fixed, until I am comfortable that the library is stable.  The I'll start working on the SoundManager class that uses this new library.


The JAR file:
http://www.paulscode.com/libs/SoundSystem/10AUG2008/SoundSystem.jar

The source code:
http://www.paulscode.com/source/SoundSystem/10AUG2008/SoundSystemSource.zip


Since there is no documentation, here are a couple of quick things to get you started:

Create the SoundSystem:

Code Select Expand

SoundSystem mySoundSystem = new SoundSystem( SoundSystemConfig.SOUND_LIBRARY_OPENAL );

Create and Play 3D sources on the fly:

Code Select Expand

mySoundSystem.quickPlay( "tada.wav", true, 0, 0, 0 );  // filename, loop, position

Switch between libraries on the fly:

Code Select Expand

mySoundSystem.switchLibrary( SoundSystemConfig.SOUND_LIBRARY_JAVASOUND );

The methods for creating new sources, streaming, moving sources, moving the listener, etc. are all similar to the ones in the SoundManager class from before, with the exception that now there is a "priority" boolean for making priority sources (ones that are not overriden by new sources when channels run out).  And one more thing: all the configuration stuff has been put together in the SoundSystemConfig class, so if you are having trouble finding something, it is probably there.  It has synchronized "get" and "put" interface methods for accessing the various settings.

Have fun!

[EgonOlsen]

What am i supposed to use for playing event sounds like explosions, shoots...that kind of thing? Does quickPlay() make sense for this or does it load the soundy every time i call it or creates some other overhead that should be avoided? Or is using newSource() a better idea? Can i start playing one source multiple times at a time (like when many bombs explode at the same time)?

[paulscode]

What am i supposed to use for playing event sounds like explosions, shoots...that kind of thing? Does quickPlay() make sense for this or does it load the soundy every time i call it or creates some other overhead that should be avoided? Or is using newSource() a better idea? Can i start playing one source multiple times at a time (like when many bombs explode at the same time)?

quickPlay() is exactly the method you want to use for explosions, etc.  I actually designed it specifically for your game after reading how you wanted to use the sound library.  A sound effect (explosion for example) may be preloaded with the method "loadSound()", or you can simply run "quickPlay()", and it will load the sound for you the first time it encounters it.  The sound file is loaded only once and reused, and you can have as many source as you like playing a sound.


On another topic, I have had a chance to use SoundSystem a little and so far I have identified the following issues:

1) The OpenAL library takes several seconds to initialize.  Not a huge problem since you don't usually initialize it very often, but still anoying.  I believe the time is being spent reserving the audio channels.  I haven't thought of any way around this, but I will look into it further.

2) The Rolloff attenuation model for JavaSound is WAY different than the one for OpenAL.  Sounds become silent VERY quickly as they move away from the listener.  I am going to tweak the formula a bit more to try and improve this.

3) Pausing does not work correctly for either OpenAL or JavaSound (different problems for each library).  In OpenAL pausing only works for non-streaming sources.  In JavaSound, there is some issue with pausing and rewinding normal sources, and catching a pause event when streaming a source.

4) OGG sounds played in JavaSound are not playing at the correct sample rate (they are too slow).  I hadn't noticed it before due to the test .ogg file I was using of breaking glass, which sounds fine at a slower sample rate.  Things like background music are greatly affected by this problem, so it is currently my highest priority on the list of bugs to fix.

These are the only issues I have noticed, but I haven't done much testing as of yet.  Let me know if you notice any other problems while using the library.

I am working on a JavaDoc for the SoundSystem library, but it is going pretty slow.  I'll let you know when I get it finished.  I also want to write an tutorial-style guide for newcommers to use.

[paulscode]

I thought I would clarify the quickPlay() method some more, in case there was any confusion about what it actually does.

Basically, the method creates a temporary source which is removed after it has finished playing.  It can be made into a permanant source by calling the setActive() method using the source name returned by the quickPlay() method.  The setActive() method could also be used to have a source automatically removed after it finishes playing.

Now if you are concerned about the overhead involved with "removing" and "recreating" sources, do not worry.  It all comes down to terminology.  As far as the underlying library is concerned, sources are not actually being deleted - there are 32 permanant sources the entire time which never go away (I call them "channels").  When a temporary "source" is removed, all that is really being removed is the peripheral information: a 3D vector, two Strings, and some floats.  When a new "source" is created, new peripheral information is created, and as far as the underlying library is concerned, one of those 32 "channels" merely changed its position.  Bytes from the correct sound file are fed into whichever source is supposed to play that sound, so there is no extra overhead involved in switching between different sound effects on a single channel or playing a single sound effect on multiple channels.

BTW, I have been working on the JavaDoc for this part, and I noticed that I never implemented the "removing of temporary sources" code.  I corrected this (a very simple fix), and it will be working properly in the next release.  In the mean time, all sources act like permanant sources.  This is a minor bug, and not likely to affect things much, since the overhead for each "source" is quite small.

I hope this clears up some confusion about how things work in the background.  I am about halfway finished with the JavaDoc -hopefully I will have it completed by the weekend.