3D Sound System

[EgonOlsen]

Isn't it possible to use the old infrastructure, where not all the sources were streaming ones and just let the user reserve some as streaming sources if he/she wants to?

[paulscode]

Unfortunately, all Java sources in the old infrastructure were technically streaming sources, and had seperate threads to stream them.  Also, there was a problem with OpenAL streams (they could not be replayed after playing once), a problem which I never solved.

I could make this a fallback option if I am ultimately unable to get the single source-stream thread idea to work.  I would have to change non-streaming Java sources over to "Clips", and just using "SourceDataLines" for the reserved streaming channels.  I would have to solve the OpenAL bug, of course.  I am sure there is a solution, I'd just have to spend some more time on it.

[paulscode]

On another topic, I've decided to refracter a number of class names, varriable names, globals, methods, etc. in this project, because practically everything is prefixed with the word "Sound" (which makes it really hard to find stuff sometimes).  Everyone knows this is a sound library, so this naming convention was needlessly redundant in the first place.

There will be a small change felt by the end user:  I removed the word "SOUND_" from the global identifiers used to switch between libraries:
SoundSystemConfig.SOUND_LIBRARY_NOSOUND, SoundSystemConfig.SOUND_LIBRARY_OPENAL, SoundSystemConfig.SOUND_LIBRARY_JAVASOUND, and SoundSystemConfig.SOUND_LIBRARY_TEMPLATE

It shouldn't be a serious change, since you are most likely only using these globals in a couple of places in your program.  I know I said I wouldn't change the interface at all, but I really had to this one time ...

[EgonOlsen]

I've written a simple wrapper around your sound system, so a change in the naming convention only affects one class in the game...no problem at all...

[EgonOlsen]

A short question: Is it possible to determine if the system is still playing something (i'm still using the first released version)? The thing is, that i'm using a click-sound in the GUI. This sound is also played when the user clicks "quit". On the way to quitting, the SoundSystem is cleaned up. If this happens while it's still processing the click-sound, the best thing that happens is an exception, the worst thing is a complete system stall. cleanUp() should handle this case IMHO but as long as it doesn't: Is there a way to determine this situation? Currently, i'm waiting 300ms before calling cleanUp(). That may fix it, but you never know...

[paulscode]

A short question: Is it possible to determine if the system is still playing something (i'm still using the first released version)? The thing is, that i'm using a click-sound in the GUI. This sound is also played when the user clicks "quit". On the way to quitting, the SoundSystem is cleaned up. If this happens while it's still processing the click-sound, the best thing that happens is an exception, the worst thing is a complete system stall. cleanUp() should handle this case IMHO

I will definitately address that problem before the next release.  Thanks for pointing it out!

but as long as it doesn't: Is there a way to determine this situation? Currently, i'm waiting 300ms before calling cleanUp(). That may fix it, but you never know...

DOH!  I forgot to put a "playing()" method into the SoundSystem class to check a source's state!!   I looked at the code, and there isn't any way to check a source's state from outside the SoundSystem class (the soundLibrary varriable is private).  The only thing you can do is recompile the source code adding the following method to the SoundSystem class:

Code Select Expand

public synchronized boolean playing( String sourcename )
{
    if( soundLibrary == null )
        return false;

    SourceData src = soundLibrary.getSources().get( sourcename );

    if( src == null )
        return false;

    return src.playing();
}

Only requirements for to recompile are to include the lwjgl.jar and lwjgl_util.jar libraries.  The above method will be in the next release.  Sorry I forgot to put that in there.  What an oversight! 

-- EDIT --
I also just thought, in case you couldn't figure this out for your self , if you are using quickPlay to play the click sound, you will need to get the sourcename for it.  You could do something like this before shutting down your application:

Code Select Expand

String sourcename = mySoundSystem.quickPlay( "click.wav" );   // wherever you are playing the click

// ...  in the shutdown location:

while( mySoundSystem.playing( sourcename ) )
{
    try{Thread.sleep(100);}catch( Exception e ){}
}
mySoundSystem.cleanUp();
// The End

--EDIT #2--
I just realized you are talking about anything playing at all.  That would also be a good method to add.  You could use something like this (Again, you'll have to add this method to the SoundSystem class):

Code Select Expand

public synchronized boolean somethingPlaying()
{
    if( soundLibrary == null )
        return false;

    HashMap<String, SourceData> sourceMap = soundLibrary.getSources();
    if( sourceMap == null )
        return false;

    Set<String> keys = sourceMap.keySet();
    Iterator<String> iter = keys.iterator();       
    String sourcename;
    SourceData source;
       
    while( iter.hasNext() )
    {
        sourcename = iter.next();
        source = sourceMap.get( sourcename );
        if( source != null )
            if( source.playing() )
                return true;
    }

    return false;
}
I will also add a method like this to the SoundSystem for the next release (not certain about the name, though )

[EgonOlsen]

Thank you. The sources are a part of the project anyway, so i have no problem to modify and recompile them.

[paulscode]

Isn't it possible to use the old infrastructure, where not all the sources were streaming ones and just let the user reserve some as streaming sources if he/she wants to?

I would have to change non-streaming Java sources over to "Clips", and just using "SourceDataLines" for the reserved streaming channels.  I would have to solve the OpenAL bug, of course.

After more consideration, I've decided to use Egon's suggestion of reserving a certain number of streaming channels, making all the other ones non-streaming.  The reason I changed my mind on this is that streaming is quite an involved process, and I want my library to be as non cpu-intensive as possible.  The benefit of using non-streaming sources is that you simply hand all of the pre-loaded data off to the sound library to play.  Streaming a source, on the other hand, is done in a busy loop which reads data in from a file input stream (or from a buffer if it is a "normal" source), keeps checking how much data has been processed, and feeds the data to the channel in pieces.

I want to use a hybrid of Egon's suggestion and my single stream-thread concept.  This will hopefully solve the OpenAL multi-thread bug I mentioned earlier.  One potential issue I am aware of is that in OpenAL, non-streaming sources can be any size, but I believe there is a maximum size you can use for a Javasound Clip.  I'll need to figure out what that value is and change the default settings in SoundSystemConfig to match that.  I'll be sure to mention this Javasound limitation in the documentation as well.  The result of this limitation would mean that if a sound effect is too long, the end of it would be cut off.  Hopefully the maximum Clip size is something reasonable.
-- EDIT --
On second thought, I will check for maximum Clip size violations in the LibraryJavaSound class rather than changing SoundSystemConfig default settings.  That way I won't be placing a needless limitation on the OpenAL side.
-- END EDIT --

By default I will have the number of streaming channels set at four.  You will be able to change this number using a method like SoundSystemConfig.setMaximumStreamingSources( int ).  Four should be more than enough streaming channels.  If you think about it, in all likelyhood you would never need to have more than a couple of streaming sources playing at the same time in a game - for things like background music, characters speaking, or rendered cut-scenes.  Most other sound effects are going to be short enough to not need them streamed.

This is a fairly large change to the code and will require some extensive debugging.  I'm hoping to spend a lot of time this weekend working on it.  Hopefully I will be able to post the next release in a couple of days, assuming I don't run into any major problems.

[paulscode]

I believe there is a maximum size you can use for a Javasound Clip.  I'll need to figure out what that value is and change the default settings in SoundSystemConfig to match that.  I'll be sure to mention this Javasound limitation in the documentation as well.  The result of this limitation would mean that if a sound effect is too long, the end of it would be cut off.  Hopefully the maximum Clip size is something reasonable.
-- EDIT --
On second thought, I will check for maximum Clip size violations in the LibraryJavaSound class rather than changing SoundSystemConfig default settings.  That way I won't be placing a needless limitation on the OpenAL side.
-- END EDIT --

After looking into this a little further, I am no longer sure there is a limit to how large a Clip can be.  The reason I originally thought so is that I remembered coming across the following line of code when I was looking at raft's Karga Sound SoundBank class:

Code Select Expand

MAX_SAFE_CLIP_LENGTH = 2097151;

A quick search online, however, has not shown this to be a limitation of Javasound.  Most likely this is just the Karga Sound equivalent to:

Code Select Expand

SoundSystemConfig.setMaxFileSize( 2097151 );

I will run some tests with really huge Clips to see if it causes any problems.  If not, this will not be an issue after all.

[paulscode]

I finished coding the new infrastructure today (it was a bit more involved than I had anticipated).  I decided to run a simple test and...  no sound.  hehe, I love it when that happens.  Oh well, at least the main part of the work is finished.  I'll start debugging tomorrow 

[paulscode]

After looking into this a little further, I am no longer sure there is a limit to how large a Clip can be.
...
I will run some tests with really huge Clips to see if it causes any problems.  If not, this will not be an issue after all.

Ok, turns out there is a maximum clip size.  Unfortunately, the solution is a little more complicated than just trimming down the size of the buffer, as you can see from the console output:

UNTRIMMED BUFFER:

Code Select Expand


buffer length: 5950208
Error in class ChannelJavaSound
    Unable to attach buffer to clip in method 'attachBuffer'
java.lang.IllegalArgumentException: Requested value 176400.0 exceeds allowable maximum value 48000.0.


TRIMMED BUFFER:

Code Select Expand


buffer length: 48000
Error in class ChannelJavaSound
    Unable to attach buffer to clip in method 'attachBuffer'
java.lang.IllegalArgumentException: Requested value 176400.0 exceeds allowable maximum value 48000.0.


As you can see, output from the exception is the same regardless of the buffer size, so my guess is it is getting its information from the AudioFormat varriable.  You may also notice the "Requested value 176400.0" is not the same as the untrimmed buffer size 5950208.

-- EDIT --
After looking more closely at the AudioFormat varriable, I discovered that 176400 is the value of myAudioFormat.getFrameRate().  It is not related to the buffer length (or a maximum Clip size).  This was completely unanticipated, as you can imagine, since these files work fine on SourceDataLines.  I did some searching online, and found that the maximum framerate value for a mixer framerate control is 48000 (as the above output indicated).  Strange thing is, I just now ran a quick test and the file can be opened in the exact same place in the code using the same AudioFormat information without problems when I use a SourceDataLine instead of a Clip.  I am not sure why I am able to use SourceDataLine to load these files and not Clip.  Unfortunately, this seems to be happening with every single .ogg file I have, so this problem absolutely must be solved, even if it means doing a complete 180 and going back to streaming all Javasound sources on SourceDataLines.  I'll tinker around some more to see if I can get them to load into a Clip and let you know what I find out.

[paulscode]

I discovered the "source" of my problem

Turns out it was related to the sample-rate hack I added in for slow ogg files (where I was multiplying the sample-rate by 4).  Basically, there are two constructors for AudioFormat, a long one which lets you specify frame rate and frame size, and a short one which calculates them for you based on sample rate.  Since I previously multiplied the sample rate by 4, this in turn resulted in the frame rate being calculated 4 times too large as well.  I switched to the long constructor to specify all values, and this seems to have fixed both the sample-rate problem and the frame-rate problem.  I was even able to remove that X4 hack!

So at least the problem is solved for Javasound, I haven't had a chance to see if sample-rates are also working in OpenAL (there are other OpenAL-specific bugs preventing me from testing this at the moment).

And the final story on maximum clip size:  YES there is one!  It is probably 2097151 as indicated in raft's Karga Sound code, however clip sizes seem to be limited to powers of 2 (I'm getting out of bounds exceptions otherwise).  Since 2097152 is one byte too large, I am forced to set the maximum clip size in Javasound to 1048576.  This translates to about 5-10 seconds of playback depending on the samplerate and mono or stereo.  It seems reasonably long enough for most game sound effects.  Anything longer would have to be streamed on one of the 4 stream channels.

--EDIT--
One more thing I noticed during all this is that apparently Clips, unlike SourceDataLines, will persist after you shut down your application unless their close() method is called.  What this means is if you run your program several times never closing the Clips, you will eventually run out of soundcard voices and have to reboot!  This is just one more reason why the SoundSystem cleanup() method must be called before exiting your program.

[paulscode]

OpenAL non-streaming sources are working now.  I had zero problems with any of my sample ogg files, and everything plays at the correct sample rate.  I found one wav file (the gear sound from my first sample applet) which plays too slowly in OpenAL, but plays correctly in JavaSound.  Also, there was one ogg file (the "broken glass" sound) which will not load in Javasound due to the frame rate being too high, but it plays fine in OpenAL.

That makes two problems out of 30 total test files.  For now, I think that is acceptable.  I suspect that if a particular sound doesn't work, running it through a tool to reduce its sample rate will probably make it usable (i.e. lower quality sounds are more likely to work than super high-quality stereo sounds).  If it turns out that this becomes a big issue to a lot of people, I could put a hack into the file reader which would check the frame rate, and if it is too large, divide that by two and then double the frame size.  Another thing I might do is write an application which lets you try out a sound effect on both OpenAL and Javasound, streaming and non-streaming.  That way you could check if a sound is going work before using it.

[EgonOlsen]

I had some wav-files that didn't load either...mono, low sample rate ones. I don't know why, but i don't care. I just picked similar ones that worked fine.

[paulscode]

I had some wav-files that didn't load either...mono, low sample rate ones. I don't know why, but i don't care. I just picked similar ones that worked fine.

Well I do know that in the first release of SoundSystem, I was loading sound files four different ways, in four different places.  The place your files are probably being loaded is when pre-loading wav files to use in OpenAL non-streaming sources, in SoundLibraryOpenAL.loadSound().  This location used org.lwjgl.util.WaveData to load the wav file.

To make things more consistant, all files are now loaded from a single class, called FileReader.  While this should keep things more consistant, hopefully it won't end up making any of the files unusable that you are currently using.  I hate it when one thing breaks while I am trying to improve something else.  If that were to happen, I'll have you send the problem file(s) to me and I'll take a look to see if I can figure out why they don't work.

One thing I could do if WaveData turns out to be more compatible than the method FileReader is currently using, I may be able to change FileReader to use WavData instead to load wav files.  For now, I like the current method I'm using, because both formats are initially placed into an AudioInputStream / AudioFormat context, and reading data later doesn't require me to keep checking what format I am loading from.