3D Sound System

[fireside]

That's a good start.  Even if there is a slight delay on one of the other systems, It might be something to live with since a vast majority use windows. 

[paulscode]

Err, Ok I am having more difficulty than expected finding what software synthesizers are common on other operating systems.  For example, I can find a ton of synthesizers that are available for Linux, but nothing indicating if any of them are part of a normal install.  Similar situation for Mac.  I think what I'll do is go on several different forums and post a link to an applet that outputs the available MIDI devices.  I'll have people running Non-Microsoft Windows operating systems tell me what output they get, and then maybe I can figure out what (if any) software synthesizers are commonly installed.

[paulscode]

Here is a link to that MIDI Device Discovery applet:

http://www.paulscode.com/source/MIDI/MIDIDeviceDiscovery/

If anyone here is running a non-Microsoft Windows operating system, could you please run the applet and copy the output from the Java Console?  Be sure to mention what operating system and version you are running.

--Interesting note:  Microsoft changed the name of their synthesizer slightly in Vista (they removed the "SW").  Kind of odd since it has had the same name since Windows 3.1, as far as I can remember.

[EgonOlsen]

This is the output when running it on an EEEPC 701 4G with the default Linux installation (some Xandros IIRC).

Code Select Expand


2 MIDI Devices:
    Real Time Sequencer
        *this is a Sequencer instance
    Java Sound Synthesizer
        *this is a Synthesizer instance


[paulscode]

Man, I keep seeing "Real Time Sequencer" and "Java Sound Synthesizer" over and over again for various Linux versions as well as for Mac OS.  So far the only different Synthesizer I have seen is on a Kubuntu 8.04 (64 bit) with Konqueror, which has the Gervill synthesizer (that's the GPL-licensed open-source Java software synthesizer I was talking about a few posts back.  It may be a standard-install on Kubuntu - I'll have to see a few more examples, though to know).

So it looks like practically all non-Microsoft OS's are going to have to use the default Synthesizer and Sequencer, and therefore will probably still going to have to deal with the delay problem when playing MIDI and sound effects simultaneously under JavaSound.

Maybe I could figure out a way to distribute Gervill seperately (since it is open-source and all) for folks who are not using a Microsoft OS.  I'll look into this (first I will have to see if using it corrects the delay problem).

[paulscode]

Ok, another question.  I can think of several ways to distribute Gervill.  Which way sounds best?

Option #1:  Make a stand-alone Java application that the user would have to run before playing his/her game.  Advantages:  Would make meeting the GPL-license requirement easy (just distribute the stand-alone application's source code and license document along with it).  Disadvantages:  Would require the user to actually think.

Option #2:  Distribute the application from Option #1 along with the game, and have it launched automatically by the game if a software synthesizer wasn't found on the user's computer.  Advantages:  Same as Option #1.  Also, would be completely transparent (user doesn't have to think).  Disadvantages:  Would only be an option for Applications.

Option #3:  Make a seperate applet that would be added to the page via Javascript if a software synthesizer wasn't found on the user's computer.  Advantages:  (none, but would would be required to cover Applets if Option #2 were chosen)  Disadvantages:  Applets only.  Also, would require the game website to have a link for downloading the seperate applet's source code and GPL license-agreement.

Option #4:  Link with a seperate JAR library that contains Gervill, which would be used if a software synthesizer wasn't found on the user's computer.  Advantages:  Simplest option to implement.  Disadvantages:  Would increase the memory footprint (could be an issue for applets).  Also, would require making the Gervill JAR library's source code and GPL license-agreement available (links on the game website for applets, or bundled with applications).

Did I mention that I hate GPL-licensed stuff?

Anyway.. I tend to lean toward Option #4, but I thought I would get some feedback on this before deciding.

[fireside]

This is side tracking, but this appears to be a mixer problem to me.  The reason it works with another synth is that it isn't using the sound api for both effects and music anymore.  There is a mixer for java 5.  I was just wondering if you explored that possibility at all.  I don't know if it's automatic or what.

http://java.sun.com/j2se/1.5.0/docs/api/javax/sound/sampled/Mixer.html

[paulscode]

Yes, I am using a mixer for the sound effects.  How would I link a MIDI device up to the Mixer, though?  I don't see any "getTransmitter" or "getReceiver" methods (or any MIDI-related methods) for the Mixer or Line classes.  I don't see "getMixer" or "setMixer" methods (or any Mixer or Line related methods) for the MIDIDevice, Sequencer, or Synthesizer classes either.  The Mixer appears to be only used for mixing Lines, which are instances either of DataLine (SourceDataLine, TargetDataLine, or Clip) or Port (compact disks, headphones, audio input jacks, audio output jacks, microphones, or speakers).

I'll do some more searching on this though, to see if MIDI can somehow be routed through a mixer.  At the moment, the only way I know how to do it is by converting the MIDI sequence into an AudioInputStream context (which is what Gervill does).

[fireside]

OK.  This whole problem is just so mystifying to me.  I mean, we're just trying to play some music and sound effects at the same time.  Surely that must have occurred to them. 
I guess either way, then.  Apple might have one, so it might be just linux and I would think it wouldn't be much of a download.

[paulscode]

I'm afraid there isn't one on apple either, here is an example:

-- Opera - Java Console --

Java vendor: Apple Inc.
Java version: 1.5.0_16

type 'h' for help

--
Sun Feb 01 14:52:57 EST 2009 JEP creating applet MIDIDeviceDiscovery (http://www.paulscode.com/source/MIDI/MIDIDeviceDiscovery/)
2 MIDI Devices:
Real Time Sequencer
*this is a Sequencer instance
Java Sound Synthesizer
*this is a Synthesizer instance

I agree that this does seem like a Mixer issue, and you would think they would have thought of that when they created the Java Sound Synthesizer.  I just need to keep playing around with this thing to get it working.

[paulscode]

I think I may have solved the delay problem (using "Java Sound Synthesizer")!  I noticed that if I select the Java Sound Synthesizer it returns "null" when I call 'getDefaultSoundbank()'.  I recalled that I added in the following code, because applets (and applications as well if JDK is not installed) were completely silent (I originally got this solution from a forum somewhere)

Code Select Expand

        if( !( sequencer instanceof Synthesizer ) )
        {
            try
            {
                    synthesizer = MidiSystem.getSynthesizer();
                    synthesizer.open();
                    Soundbank soundbank = synthesizer.getDefaultSoundbank();
                    if( soundbank == null )
                    {
                          URL defaultSoundBankFile = getClass().getClassLoader()
                              .getResource(
                                        SoundSystemConfig.getDefaultMidiSoundBank() );

                          Soundbank defaultSoundBank =
                                      MidiSystem.getSoundbank( defaultSoundBankFile );
                          synthesizer.loadAllInstruments( defaultSoundBank );
                    }

                    Receiver receiver = synthesizer.getReceiver();
                    Transmitter transmitter = sequencer.getTransmitter();
                    transmitter.setReceiver( receiver );
                }
...

The thought occurred to me, that perhaps the instruments are really there, built into the Java Sound Synthesizer and simply not retrievablty with getDefaultSoundbank() - If that were the case, it would mean that I am loading a whole new set of instruments on top of the ones already there, and confusing JavaSound.  I removed the sound bank loading code (leaving in the transmitter/receiver linkage), and ran the applet.  It works beautifully now, with no delay problem (at least on my main computer - I still need to verify this on my test machines).

So moral of the story:  Don't always trust what you read on the forums  .  I think the reason this took so long to solve, is there was I thought I had corrected another problem, I wasn't getting any error messages or exceptions, and I didn't notice the delay problem until sometime later after I made the change.

[paulscode]

Verified on my test machines - problem solved!  I will make the changes to SoundSystem and post an update later.

An interesting side-note:  If there is a MIDI music keyboard hooked up to the computer, JavaSoundSynthesizer will play MIDI through it - I little bit anoying, but something I can live with (I suspect Java is actually playing the MIDI through the Microsoft MIDI Mapper - I had a similar bug several years ago with a MIDI program called "Cakewalk", which I solved by changing the Microsoft MIDI Mapper setting for "default MIDI device" or something).  Most players are not going to have MIDI keyboards hooked up, anyway.

[fireside]

Cool.  Glad you finally got it worked out.

[paulscode]

Sound System MIDI Fixes

JARS:

Sound System jPCT  (Download Source Code)

Sound System  (Download Source Code)

Sound System Small  (Download Source Code)


Other:

Demo Applet

JavaDoc

PDF Tutorial


What's new?

- Corrected the delay problem that was happening when playing sound effects with the Java Sound library while MIDI was playing.  This fix also eliminated the need for SoundSystemResources.jar, which significantly reduces the memory footprint required to play MIDI in applets.
- Removed the MidiThread class (a work-around that is no longer needed).
- Added support for MIDI into the small version of sound system.
- Made a few changes to the PDF tutorial document.

[fireside]

Does the soundsystemsmall have any dependencies?  Does it play ogg?
The demo applet isn't working for me for some reason.  Never mind, I needed to click in the window to get focus.