3D Sound System

[EgonOlsen]

For the record: I'm just using quickPlay, nothing else.

[paulscode]

For the record: I'm just using quickPlay, nothing else.

Awesome, thanks.

1) Use WaveData for all sources (would use more memory for streaming sources and would require large files to be pre-loaded to avoid noticable lag the first time they were played)

2) Use WaveData only for normal sources and keep the current AudioInputStream infrastructure for streaming sources (less compatible, some files sound distorted in OpenAL but not in JavaSound).

After giving it some thought, I believe I will go with the first option for now.  I intend to come back and look at the compatibility issues I am having with the AudioInputStream method, but considering all the problems it currently causes, I just don't think it is smart to use it until I can figure out how to make it work reliably.  After looking at the JavaDoc for the WaveData class, it looks like I can make it work for both .wav and .ogg files (WavaData.create() method can take an input stream as an argument, so I should be able to pass it the OggInputStream).  Using WaveData should fix the first problem you are having and hopefully that 'Invalid Value' exception as well.  It will solve some of the compatibility issues I have been having.  It should also eliminate the locking up while streaming multiple .oggs (since files will always be read in one at a time).  I am going to add a new method to SoundSystem called "unloadSound()".  This will allow memory from a previously loaded file to be freed up.

That being said, I have not yet found the reason for the JavaSound stop() bug, but when I finish the WaveData stuff, I will take a closer look into that problem as well.

[paulscode]

I've finished switching over to the new file loading method, and all my tests so far have been very good.  The switch has corrected nearly every compatibility issue.  Every one of my test files load without a hitch, including the three files from Egon's game.  All files play correctly in both JavaSound and OpenAL, except for one .ogg file (the famous broken_glass.ogg).  That file still can not be attached to non-streaming sources in JavaSound (due to the frame rate being too high, as before).  Like before, it can be streamed in JavaSound, and it works fine in OpenAL.  This is still a much higher success rate than ever, so I am feeling really good about file compatibility in SoundSystem now.

I will now begin looking into that stop() error.  Hopefully, I will be able to re-create the bug and find out what is causing it.

-- EDIT --
I forgot to mention, the WaveData class is really geared more toward lwjgl and OpenAL, so I ended up downloading the source for that class and modifying it to be more genaric.  Also, I want to eventually make a JOAL version of SoundSystem for some of my friends who are not using lwjgl, so by using my own version of WaveData, SoundSystem is not totally dependant on the lwgjl library (I will easily be able to replace the three classes LibraryOpenAL, ChannelOpenAL, and SourceOpenAL with JOAL versions).

[paulscode]

I created a test applet that has allowed me to replicate the JavaSound stop() bug:
LINK NO LONGER EXISTS

The interesting thing about it is when the error occurs, only normal sources are affected, while the streaming source still works fine.  This is really looking like a bug in my channel-selector code, but I have not found what is causing the problem as of yet.  I will look into this some more tomorrow.  At least I can recreate the problem, which is hopefully the first step to solving it.

-- EDIT --
I forgot to mention, the problem can be easily replicated by simply starting and stopping the "special source" several times.  The number of times it takes to replicate the problem seems to vary, so I am thinking perhaps the problem is not in the channel-selector code as I mentioned above.  In that case, I would expect the problem to always occur on the same number.  This is quite odd.

-- EDIT #2 -- I have discovered that after a while the sources start getting cut off during playback.  Initially multiple sources can play simultaneously just great, but then it seems like the previous source is cut off when the same sound effect (i.e. explosion.wav) is played on a new source.  I wonder if this is related to the first problem, as it seems to happen just before the stop() error message gets printed and normal sources stop working.  Seems like the sound card is running out of voices or something.

-- EDIT #3 --
I have found an "access control" exception that happens occasionally when shutting down or switching libraries while multiple sources are playing.  I will look into solving this problem.

-- EDIT #4 --
Last update for today.  I noticed that the stop() problem happens on OpenAL as well, not just JavaSound, which is very significant.  I also discovered that the source-cut-off problem only seems to happen in JavaSound, and not in OpenAL, which may indicate that the two problems are not related after all.  Hmm.

[paulscode]

Last update for today.  I noticed that the stop() problem happens on OpenAL as well, not just JavaSound, which is very significant.  I also discovered that the source-cut-off problem only seems to happen in JavaSound, and not in OpenAL, which may indicate that the two problems are not related after all.

I take that back.  The cut-off bug does in fact happen in OpenAL as well, it just seems to take longer.  I still have not found the source of these problems, although I am almost positive it has something to do with how I am managing the channels.

[paulscode]

   I found the bug!  It was a simple typo where I mixed up a varriable for the streaming channel list instead of the correct one for the normal channel list.  Fixing this appears to have corrected both the null stop() exception and the source cut-off bug, however I have only run a couple of tests so far.  I also discovered what is causing the control access exception I mentioned, and I have that problem fixed as well.  I will post another release this evening after I run a few more extreme tests to make sure the library is now stable.

[paulscode]

Sound System Stable Release

JAR:
http://www.paulscode.com/libs/SoundSystem/03SEP2008/SoundSystem.jar

JavaDoc:
LINK NO LONGER EXISTS

Source Code:
http://www.paulscode.com/source/SoundSystem/03SEP2008/SoundSystemSource.zip


I also uploaded the test applet again using the new-and-improved Sound System:
LINK NO LONGER EXISTS

Let me know whether or not you have any problems, and sorry it took so long for me to get this thing working like it should

[fireside]

The example applet seemed to work well in both switch modes for me.  Good job. 

[EgonOlsen]

Ok, i'll see if i manage to try this version today and report back my findings...

[zammbi]

Nice one, demo works all good for me. I will surely be using this for my game when I add sound.

[EgonOlsen]

I've tried OpenAL and JavaSound on Vista and JavaSound on Linux (OpenAL is broken on that Linux anyway)...everything worked fine. The only difference between the two that i found is, that the explosion sound sounds quite different, but i can live with that. 

[paulscode]

I've tried OpenAL and JavaSound on Vista and JavaSound on Linux (OpenAL is broken on that Linux anyway)...everything worked fine. The only difference between the two that i found is, that the explosion sound sounds quite different, but i can live with that. 

Yes, I noticed that about the explosion, too.  Not really sure what causes that difference (it sounds crisper on OpenAL to me, like the difference between stereo and mono).  I'm glad to hear everything is working fine, though.

I have started working on the new SoundManager.  It really shouldn't take very long to complete, since SoundSystem already takes care of most of the stuff SoundManager was doing before.  All SoundManager will need to handle is source culling/activating based on distance from listener, attaching the listener to the Camera, and attaching sources to Object3D's.  All of that code is already written, so it should be a simple matter of copy and paste, and then debugging.

[paulscode]

I've been making progress on SoundManager.  I created the methods in SoundManager for binding sources to Object3Ds and binding the listener to the camera, as well as the tick() method, and everything appears to be working well.

The SoundManager class extends the SoundSystem class, since most of its methods directly interface with the SoundSystem anyway.  There is a new SoundManagerConfig class for interfacing SoundManager-specific settings like Management Model and Cull-Distance, while all the general settings will still be interfaced using the SoundSystemConfig class.

Rather than creating a whole new source management thread, I decided to extend the CommandThread class, to have it handle source management.  The SoundSystem init() method is extended in SoundManager so that it creates a SoundManagerThread in place of a CommandThread.  It makes sense for this thread to be in charge of source management anyway, since it is already the thread responsible for executing commands to create, play, cull, or activate sources.

Unfortunately, source management is not going to be as copy-and-paste as I originally thought.  I forgot that there was a bug which I never got around to fixing (the "boing" problem I explained back on page 4 of this topic).  I also want to improve the speed of source management by maintaining a list of actively playing sources amd culled looping sources, so that I only need to sort that shorter list rather than the entire list of sources every time distances change.

[paulscode]

I finished the initial coding for the new SoundManager, and for the most part it works really well.  The bindListener( Camera ) and bindSource( Object3D ) methods seem to work, and both management models appear to be working the way they are supposed to.

There is one bug that I am trying to track down.  The behavior for this bug is exactly like the stop() problem that SoundSystem used to have.  It seems to usually occur when trying to stop a source after the list of actively playing sources fills up and gets sorted.  I haven't had a chance to track the problem down to a more specific sequence of commands, though, so sorting may be completely unrelated.  Just to be sure the problem isn't on the SoundSystem side, I ran some more tests on SoundSystem for cases where the channels become full and sources are forced to stop to make room for other sources.  I was not able to recreate the error that way, so I am fairly certain the problem is isolated to SoundManager's culling and activating of sources.  This does narrow down my search a little, since I can assume the bug is located somewhere in the culling/activating code found in classes SoundManager, SoundSystem, Library, or Source, or in the logic I am using for the two source-management models.

[paulscode]

I am taking a break from SoundManager to add a couple of "wish list" items to SoundSystem.

First of all, I have added support for MIDI to SoundSystem.  Everything is stable, and there do not seem to be any problems.  OpenAL does not have an interface for MIDI, so all MIDI playback must be done through Javasound.  Since MIDI playback is done completely seperate from the Javasound Mixer, there are no conflicts when switching between libraries while using MIDI (i.e., you can play MIDI when either OpenAL or Javasound is active, it doesn't matter).  Typically, MIDI plays through a seperate chipset on the soundcard (although a Windows software synthesizer is used for some low-end soundcards), so it shouldn't use up any of the soundcard voices used by other sources.  The drawback is that only one MIDI file can be played at once, so what I had to do is create a new channel type specifically for MIDI playback.  It is instantiated in the base Library class, so all extended libraries play the MIDI source through the same route.

I have SoundSystem taking care of all the messy details, so creating and playing a source using a .mid file is no different than when using a .wav or .ogg file, except that the file can not be pre-loaded.  It makes no difference whether you specify the MIDI source as "Streaming" or "Normal", because MIDI playback involves things like "sound banks", "pressures", and "sequences", which is completely unlike other sound types (i.e., there is no "sound byte" data or stream-buffers involved).  Only one MIDI source can exist at a time, so if multiple sources are created from .mid files, the newest MIDI source will automatically remove any previous one, to avoid conflicts.  3D position has no effect on the MIDI source, so MIDI is really only suitable for background music.

A couple of things about MIDI in Java.  MIDI will not play from within an applet unless a "Soundbank" (.gm) file is specified.  Also, I do not believe MIDI will work in an application if a soundbank is not installed.  I know a sound bank is installed along with versions of the JDK, but of course you can not assume that your players will always have JDK installed on their machines.  What I did is make SoundSystem smart enough to check if a sound bank exists, and if not it loads a default one automatically.  I compiled a JAR called SoundSystemResources.jar (about 500KB), which contains the default soundbank file in package "paulscode.sound.resources".  If your applet or application uses MIDI, you will want to link with that JAR at runtime (or compile that package and file in your own JAR).  It is not necessary to link with SoundSystemResources.jar if your program does not use MIDI, or if you somehow know the end user already has a soundbank installed on their machine.

Also, if you prefer to use your own soundbank file rather than the one in SoundSystemResources.jar, you can use a method called SoundSystemConfig.setDefaultMidiSoundBank( String ).  The string parameter is a package path (such as "pauscode/sound/resources/soundbank.gm"), or an online URL path (beginning with "http://").

One more thing about MIDI.  I would eventually like to add my own software synthesizer to SoundSystem so that MIDI files could be loaded into an AudioInputStream context.  That way you could play as many .mid files as you like simultaneously, and the 3D functions would work the same as any other source.  Unfortunately, I have not been able to find any useful source code (other than GPL stuff), and it is simply too large a project for me to tackle myself at this time.  Perhaps in the future.

I am looking into adding support for .mp3 files, but I will only add this if I can find freely usable source code, and that it is not GPL-licensed source code.  Two reasons:
1) I do not want SoundSystem to be dependant on an external JAR.
2) GPL stuff can not be used in commercial applications ("free and open" does not mean "no-strings-attached").

Another thing I am looking into is manual volume manipulation, especially for the MIDI source (MIDI files tend to be quite loud, I've noticed).  I haven't really decided the best way to do this yet.  I would kind of like to make volume changes on a per-source basis as well as a "master volume".  I am not sure how difficult that is likely to be (the code will of course be different between Javasound and OpenAL).