MIDI.js: Playing Audio in the Browser with Javascript

“Everything old is new again.”

        –Peter Allen

The evolution of the web browser is in many ways the repackaging and reintroduction of technologies that have existed on the desktop for decades. Consider WebGL replacing OpenGL with the help of the <canvas> tag and HTML5 video replacing ActionScript/Flash with the <video> element.

At the well-run Science Hack Day SF I was able to dive into these new browser technologies along with Jade and Rich to build Symphony of Satellites, an app that generates music based on the trajectories of satellites currently overhead. Using data calculated live from NORAD, musical notes ride and set as satellites appear and disappear over the horizon. The velocity of the satellite, its elevation, and other aspects of its trajectory determine the instrument, pitch, and rhythm of notes generated by that satellite and the visualization on the page. The promo video for the app is worth watching.

While transmigrated browser technologies often initially lack the versatility and raw power offered by their desktop brethren (mostly due to the sluggishness of JavaScript and the sheer number of development hours spent creating and optimizing native code for desktops environments like Mac OS, Windows, and Linux), it was a real joy and a testament to HTML5’s promise of better support for the future of online multimedia to play with MIDI.js, a framework that has brought a veteran technology to the browser with the help of the HTML5 <audio> element, the Web Audio API, and the Web MIDI API.

MIDI

Musical Instrument Digital Interface has influenced, if not generated, much of the music we hear today. Announced to the public by Robert Moog himself, the protocol has withstood the test of time (and a number of revisions) to become the de facto standard by which electronic musical instruments and computers communicate. MIDI, at its core, describes methods to select an instrument and play a note at a certain volume for a given length of time. There is a huge body of knowledge surrounding MIDI making it a logical choice for inclusion in the browser.

The MIDI.js project unites a number of libraries to reproduce the MIDI standard using JavaScript in a way that works on most browsers (including mobile). Imagine you wanted to synthesize a Xylophone playing a middle C at maximum volume for half a second. Using MIDI.js the code would look as follows.

MIDI.loadPlugin({
    soundfontUrl: "/path/to/soundfonts/",
    instrument: "xylophone",
    callback: function () {
        var channel = 0, // MIDI allows for 16 channels, 0-15
            // the xylophone is represented as instrument 13 in General MIDI.
            instrument = 13,
            // middle C (C4) according to General MIDI
            note = 60,
            velocity = 127, // how hard the note hits, from 0-127
            delay = 0.5; // how long to hold the note, in seconds
        // play the note
        MIDI.programChange(0, instrument); // Load xylophone into Channel 0
        MIDI.noteOn(0, note, velocity) // Play middle C on Channel 0
        MIDI.noteOff(0, note, delay) // Release the middle C after 0.5 seconds
    }
});

To understand how MIDI.js generates the tone you need to understand SoundFonts (.sf2), a file format containing note samples for one or more instruments. SoundFonts are plentiful on the internet and many are available for free. While the browser cannot play the SoundFont files directly, MIDI.js comes packaged with two pre-made instruments and a soundfont-generator that converts the .sf2 files into .wav and .ogg formats suitable for playback in the browser. However, the script is a little difficult to get up and running and requires installation of Ruby, Node.js, FluidSynth, and Lame (a hurdle too high for most budding web musicians). I have created a package with all 128 General MIDI instruments that is freely available under a Creative Commons 3.0 license.

Using these instruments and the MIDI.js library you can leverage the full power of MIDI in the browser. MIDI (.mid) files can be played easily as well.

MIDI.Player.loadFile("file.mid", callback);

For an introduction to electronic music theory check out Michael Hewitt’s Music Theory for Computer Musicians. For those interested in scales I’ve built a pocket reference in JavaScript.

There are a bounty of great applications leveraging MIDI and the Web Audio API in the browser. Check out my favorites Plink, ToneCraft, and Piano.