3D Sound System

[raft]

i guess you need to read javadocs of these methods

* Object.wait()
* Object.notify()
* Object.notifyAll()

concurrency chapter of java tutorial may also help

r a f t

[paulscode]

concurrency chapter of java tutorial may also help

Thanks a lot - very good tutorial.

[paulscode]

The new thread infrastructure is in place, and I have done some extensive debugging before putting in the actual sound parts.  I did some conflict tests, and confirmed that without the "synchronized" methods, there were a lot of things that could go wrong in rare instances.  With things synchronized it runs somewhat slower, but has zero conflicts.  I have begun adding in the actual OpenAL stuff (I should have it finished this evening), then I am going to run more tests and do some comparisons between the two management models.  Some differences I have already discovered about them are:

1) The source culling method (the one used in the flying boxes demo) ensures that the closest sounds will play.  The drawback is that if there are not enough channels available, it cuts off other sounds before they finish in order to play the closer ones.  Since sounds always begin playing from the beginning (even when they were culled then reactivated), in cases where there are many looping sources of the same sound effect (like in the boxes demo), you hear the beginning of the sound a lot.  It's something to keep in mind when using this management model, but I still think this is going to be the best model to use in most cases.

2) The channel monitoring method (the new one) ensures that every time a sound starts, it plays from beginning to end, unless it moves beyond the cull distance (cull distance can be set to a really large number to turn this culling feature off).  The drawback is that closer sounds may not play if there are no channels open for them to play on.  Rather than just playing queued sources in just any old order, I've tried to make this management model somewhat intelligent.  It sorts the queued sources by distance before trying to play them, so the channels fill up with the closest sounds, but this isn't a perfect solution.  For example, you could have like 40 sources, some of them quite far away, which all get queued to play, filling up all available channels.  A second later, a really close source could get queued to play, but end up not playing because there are no channels open.  That's an important thing to keep in mind when using this management model.

If nothing goes wrong, I am planning to release the new SoundManager this weekend, along with a nice demo applet, and the source code as always.

[paulscode]

Things are coming along nicely.  All the OpenAL code is back in place, and the three threads are working together beautifully so far.  I haven't done any really intensive tests just yet, because I am working out a logic error related to sources starting out initially culled.  After I get it fixed, then I will start on a test applet.  I haven't decide what I want it to be yet

[paulscode]

New version of SoundManager out today!
http://www.paulscode.com/source/SoundManager/13APR2008/SoundManager.jar
Source Code:
http://www.paulscode.com/source/SoundManager/13APR2008/SoundManagerSource.zip

I also made a demo applet:
http://www.paulscode.com/source/SoundManager/13APR2008/Helicopter.html
Applet Source Code:
http://www.paulscode.com/source/SoundManager/13APR2008/HelicopterSource.zip

This has been a major overhaul of the class - I started from scratch and rebuilt it from the ground up.  Some new features about this version:

    All OpenAL calls are all done from a single thread.  This required a complicated Command Buffer infrastructure.

    Sorting and source management are now all done on a seperate thread.

    There are two management models to choose from:  active culling (plays closest sounds and culls further away sounds), and channel monitoring (plays closest sounds, only when channels are open - doesn't cull sources).

    Priority sources (will not be culled, example: background music).


In order to keep SoundManager simple to use, the actual methods and the way the class is used, have not changed very much from the previous version.  A couple of minor things have changed:

    I changed the method setPos() to setPosition(), just to be consistant in my naming conventions.  I believe I made a couple of other minor method name changes, but I don't recall which ones.  If something in your program breaks though, it shouldn't be very difficult to find the correct method name, as the changes are very minor.

    The basic commands like play, stop, rewind, etc. no longer return booleans for success or failure.  This is because now they merely queue a command which gets executed on a seperate thread.

    There are new constructors for newSource() and newStreamingSource().  They are the same as the previous constructors, plus a new first arguement being a boolean.  Setting this argument to "true" makes the new source a "priority source" (doesn't get culled).  All the old constructors still exist, so if priority is not specified, SoundManager assumes a new source is not a "priorty source".

    There is a new varriable the user can set:  MANAGEMENT_MODEL.  Possible values are MANAGEMENT_ACTIVE_CULLING and MANAGEMENT_CHANNEL_MONITORING.  MANAGEMENT_MODEL can be changed dynamically at any time in your program.

    There are some new methods that give you access to the OpenAL components like source buffer identifiers and the like.  This is just in case anyone wants to do OpenAL stuff not currently supported by SoundManager, like doplar effect, directional sources, etc.

I plan to write a new guide to fully explain how to use the class, including all the new features.

Because there have been such extensive changes, there are bound to be problems.  I have run tests and everything seems stable, but the best way to find all the bugs is for people to start using the class.  If you have any problems, error messages, exceptions, etc. let me know, and I will look into it.

[fireside]

Interesting applet.  Nice work on the sound manager. 

[EgonOlsen]

I'm going to use this in "nameless Bomberman clone", when it's time to add sound to it. Would it be possible to add a kind of fallback mode that uses Javasound instead? It doesn't have to adjust volume or position, just play some sound in case that OpenAL doesn't work. My experience from Paradroidz is, that OpenAL isn't that reliable on some platforms, while Javasound always worked (after setting the mixing to software...). For me, this can be a very simple fallback. Something like:

Code Select Expand

import java.io.*;
import javax.sound.sampled.*;

public class StrippedSound {

   private static Mixer mixer;
   private byte[] data;
   private AudioFormat format;
   private DataLine.Info dataLineInfo;
   private boolean ok=false;
   private Clip currentLoopClip=null;
   private Clip lastClip=null;
   private boolean mute=false;

   static {
      Mixer.Info[] mixers=AudioSystem.getMixerInfo();
      for (int i=0; i<mixers.length; i++) {
         if ("Java Sound Audio Engine".equals(mixers[i].getName())) {
            mixer=AudioSystem.getMixer(mixers[i]);
         }
      }
   }

   public static void main(String[] args) {
     StrippedSound s1=new StrippedSound(new File("sound1.wav"));
     StrippedSound s2=new StrippedSound(new File("sound2.wav"));
     StrippedSound s3=new StrippedSound(new File("sound3.wav"));
     
     for (int i=0; i<50; i++) {
       double rnd=Math.random();
       if (rnd<0.3) {
         s1.play();
       }
       else {
         if (rnd < 0.6) {
           s2.play();
         }
         else {
           if (rnd < 1) {
             s3.play();
           }
         }
       }
       
       try {
         Thread.sleep(200);
       } catch(Exception e) {
         //egal
       }
     }
     
     while (s1.isRunning()||s2.isRunning()||s3.isRunning()) {
       // Warten, bis auch der letzte fertig ist...
       Thread.yield();
     }
     
     System.exit(0);
   }

   public StrippedSound(File sound) {
      try {
         System.out.println("Loading sound from file");
         AudioInputStream soundStream=AudioSystem.getAudioInputStream(sound);
         format=soundStream.getFormat();

         int len=(int) (format.getFrameSize()*soundStream.getFrameLength());
         data=new byte[len];
         int l=0;
         int lp=0;
         do {
            l=soundStream.read(data, lp, len);
            lp+=l;
         } while (l!=-1);

         dataLineInfo=new DataLine.Info(Clip.class, format);
         if (!AudioSystem.isLineSupported(dataLineInfo)) {
            System.out.println("Audio-Line not supported!");
         }
      } catch (Exception e) {
         System.out.println("Error loading sound: "+e);
      }
   }

   public void setMute(boolean is) {
      mute=is;
   }

   public boolean playedFine() {
      return ok;
   }

   public boolean isRunning() {
      return lastClip!=null&&lastClip.isRunning();
   }

   public void stop() {
      if (lastClip!=null&&lastClip.isRunning()) {
         lastClip.stop();
         lastClip=null;
      }
   }

   public void play() {
      if (mute) { return; }
      Clip soundClip=getClip();
      if (soundClip!=null) {
         soundClip.start();
      }
   }

   public void endLoop() {
      if (currentLoopClip!=null) {
         currentLoopClip.loop(0);
         currentLoopClip=null;
      }
   }

   public void loop() {
      if (mute) { return; }
      if (currentLoopClip==null) {
         currentLoopClip=getClip();
         if (currentLoopClip!=null) {
            currentLoopClip.loop(Clip.LOOP_CONTINUOUSLY);
         }
      }
   }

   private Clip getClip() {
      Clip soundClip;
      try {
         if (mixer!=null) {
            soundClip=(Clip) mixer.getLine(dataLineInfo);
         } else {
            soundClip=(Clip) AudioSystem.getLine(dataLineInfo);
         }
         soundClip.open(format, data, 0, data.length);
      } catch (LineUnavailableException e) {
         ok=false;
         return null;
      }
      ok=true;
      lastClip=soundClip;
      return soundClip;
   }
}

...just with the API of the bigger OpenAL solution, so that i don't have to wrap the sound stuff myself to support both.

[paulscode]

I'm going to use this in "nameless Bomberman clone", when it's time to add sound to it. Would it be possible to add a kind of fallback mode that uses Javasound instead?

Yes, that is a good idea.  I'll look into adding Javasound into SoundManager so that you can use the same interface for either sound system.  I'll also do some research to see if there is a way to detect when OpenAL is not working properly.  At the very least, I could have SoundManager automatically switch to Javasound if an exception gets thrown during AL initialization.  It would also be good to allow the user to switch modes dynamically.

I'll start looking into these ideas.

[paulscode]

I have begun looking into Javasound and have download a bunch of sourcecode I am going to dig through.  Specifically I am looking for ways to do the things that SoundManager already does: Multiple Sources; Pause, Play, Rewind, and Stop; normal playback of WAV and OGG sounds; and streaming playback of OGG sounds.  I'm also going to look for a function for changing source volume - if I can find that, then Javasound could potentially support Linear Attenuation for a simi-3D sound system.  If I get really ambitious, I might even write my own Logrithmic (Rolloff) Attenuation algorithm.  That way the only difference between the OpenAL and Javasound systems would be the "true 3D" left/right fading for mono sounds that OpenAL supports -- something that, at least on my soundcard, doesn't seem to work very well anyway 

I'm going to start by writing test programs to learn the skills needed to use Javasound.  After that, I will begin incorperating the code into SoundManager.  The later task will not be trivial, mind you.  I hadn't originally planned to use multiple sound libraries when I designed the class, so there will have to be some significant structural changes.

While I'm restructuring the class I also plan to address any other bugs that are discovered.  I'm hoping a few people will give me feedback on the helicopter demo applet, or even better on any of their own projects they are using the SoundManager class in.  One potential bug I noticed yesterday is that often a ball will bounch quite close, but I don't seem to hear a nice "Boing" when it happens.  I'm not sure if this is a logic problem specific to that applet (using Math.sin() always gives me a headache), or if it is a systematic problem with the SoundManager.  Thinking back, I did do a ton of testing with looping sources because of the complexity involved with managing them.  As a result of this special attention, looping sources seem to work quite well now (the flying cubes applet works beautifully with the latest version of SoundManager).  However, I may not have done enough testing for the simple non-looping non-streaming sources, so I will go back and look at those types of sources more closely.  Could be a delay issue (between when the command is queued and when it actually gets executed), could be a channel issue (didn't cull sources correctly before trying to play), or could be a logic issue (told too many sources to play, filled up channels, didn't cull the right ones).  Hard to say at this point, but I am sure to find the cause if I dig into it a little further.

--- UPDATE ---
    I found the cause for this "boing" problem.  It is due to some flawed logic on my part, related to the fact that the command-processing thread runs seperate from the source-management thread.  What's happening is whenever several balls hit the ground at the same time, commands are queued to play more sources than there are available channels.  Because there are not enough channels, some of the sources do not play.
    When using the active-culling management model, the source-management thread goes through the list and culls the far-away sounds, freeing up channels.  At this point, if these were looping sources, source-management would then reactivate the closest sources.  Problem is, these are not looping sources, and SoundManager does not reactivate non-looping sources (to avoid sound effects being repeated if they happen to be located at a position where they alternate back and forth between culled and active).  So basically, if OpenAL was not able to play them before, they simply do not play at all.
    Something similar happens when using the channel-monitoring management model.  As before, the channels get filled up with a bunch of sources (not necessarily the closest ones).  Now, if these were looping sources, the management thread would play the closest ones as soon as channels became free.  But since they are non-looping sources, SoundManager does not play them when channels open up (to avoid a potential time-delay between when the sound was supposed to play, and when it actually does).  So like in the active-culling management model, if OpenAL was not able to play the source before, it does not play at all.
    Ok, so all this sounds rather complicated.  Well, actually this is an easy fix.  Instead of having the command processor actually playing a source, I'll just have it set a "toPlay" boolean for the source.  Then the source-management thread can actually play the sources after making a decision which sources should be played.  Only thing I'll have to make sure of is that the entire process from queueing a command to play, and when the sound actually comes out of the speakers, happens fast enough.  There's nothing worse than firing your lazer cannon, then hearing the blast half a second later (brings to mind the scene from Star Wars, Attack of the Clones, where Jango Fett was setting off seismic charges in the asteroind field)

--- END UPDATE ---

I'll keep you posted on my progress.  Development on the SoundManager may slow down just a bit, because starting this week I am going to begin working on some other components for my game, but I will try and manage my time so SoundManager is completed in a timely manner.

Thanks for all the great help and tips you guys have been giving me.  Relatively small community here, but the jPCT forums are my favorite by far.

[JavaMan]

I tried out the applet. Works fine for me. Sound is clear and not broken or anything. One weird thing is that when I press escape Firefox crashes. 

Relatively small community here, but the jPCT forums are my favorite by far

I agree, the jpct forums are active. I also use the lejos robotics forums and things over there aren't even half of active as things here.

[paulscode]

I tried out the applet. Works fine for me. Sound is clear and not broken or anything. One weird thing is that when I press escape Firefox crashes. 

Oh, yes, I know what is causing that.  When you press esc, it cleans up memory and stops rendering.  Problem is, I don't know the Java command for actually closing the browser window from an applet  (I know, I am such a noob)

-- EDIT --
Oh, and I didn't call shutdown method anyway.  oops!  Let me fix that real quick!

[JavaMan]

I am using the applet in one tab and I am in the jpct forums in another tab so I'm not sure if thats it. I don't no maybe.

Oh, excuse my ignorance, but what is a noob?

[paulscode]

OK ... So NOW the applet actually DOES clear up memory when you press Esc .  I still don't know how to close the browser from the applet, though.

Oh, excuse my ignorance, but what is a noob?

Gamer's jargon, short for "newbie".  In normal slang, it means something like "green" or "wet behind the ears".  Basically someone who is inexperienced at something.

[JavaMan]

OK ... So NOW the applet actually DOES clear up memory when you press Esc Cheesy.  I still don't know how to close the browser from the applet, though.

I don't know anything about applets, so I can't answer that. The applet stops running when I press esc, and I can go and surf on the other tab, but when I close the applets tab firefox crashes. I know ff has some problems so maybe thats it. Anyway the soundmanager works on this machine.

Gamer's jargon, short for "newbie".  In normal slang, it means something like "green" or "wet behind the ears".  Basically someone who is inexperienced at something.

Ah, ok thanks.

[paulscode]

The applet stops running when I press esc, and I can go and surf on the other tab, but when I close the applets tab firefox crashes. I know ff has some problems so maybe thats it.

I tried to recreate the problem on my main PC.  I tried multiple tabs in Internet Explorer, Firefox, and Netscape Navigator.  None of them crashed for me when closing the applet tab, whether the applet was stopped or running.  I'll see if I have any problems on my other test machines.  Can't really say what the problem is.  Do you only have the problem after stopping the applet with Esc, or does it also happen if you close the tab while the applet is running?