My friend Judah and I are going to collaborate on an album of 30 30 second songs, and will be releasing it August 1st, 2009.
Archive for the ‘Sound’ Category
In my last post, Plotting a Sound Wave in Flash AS3 I detailed a method to use when displaying audio data. The method itself works great, but due to Flash’s frame-based code execution and event processing the user looses input capabilities while the flash player chugs through millions of numbers adding, rounding and comparing. In order to make displaying an audio waveform easier on both the programmer and the user I wrote a class that analyzes a Sound object progressively, and dispatches a special event containing the analyzed data. The class will construct a left and right channel Vector, each containing one data point [a number between 0 and 1] for a given number of windows, between two positions in the sound. The left and right position are measured in samples and two types of analyzation are offered. Here is a demo of the class in action:
The calculated data can be reached incrementally through the WaveformEvent object which is dispatched every frame, or at the end of all analyzation in the Waveform object’s leftChannel and rightChannel properties. The details are listed in the documentation below.
And here is a Flex version (made in windows): // Thanks dem!
Sources (Flex version)
I’ve always been really into wave editors. I used to make songs in Amadeus by piecing together samples from other songs. Tedious but very rewarding. In a post I made not a long time ago I detailed a little Theremin project which included some wave data visualization. In this post I”ll be going further into detail about plotting sound data.
Digital audio in it’s rawest form [PCM wave data] is a long list of numbers from 1 to -1, which represent the sound’s amplitude. Another way of thinking about this is that each number represents your speaker’s distance away from it’s rest position. At 1 the speaker is fully extended, blowing out your ears and scaring your cats, while at -1 it is fully retracted, blowing out your ears and scaring your cats. To make a meaningful visual out of this we just set up a graph where time is plotted on the horizontal axis and amplitude on the vertical. So at a really high resolution, that might look like this:
Okay, maybe it would look like that if you were living in the 80′s. Or if you were really into oscilloscopes. In reality with most popular songs being two to five minutes long, we’d be looking at HUGE graphs. One three minute song sampled at 44.1kHz/s comes out to be eight million samples per channel. Per channel. Since most modern music is in stereo, we’re looking at two graphs now. So how do we compress this data and view it in a meaningful way? We cheat a little. We kinda scrap the whole graph/function thing. Well kinda. Let’s say you have a window 1000 pixels wide and a sound 3 minutes long. We have to compress enough samples together in order to represent them using each pixel [about 7,938 samples per pixel]. Averaging doesn’t work because with values oscillating between -1 and 1 the mean is usually zero. We could take one sample every so often to represent an entire chunk and plot that, but that’s just resampling at a much lower resolution, which results in aliasing and all sorts of bogus stuff. Take this video for instance:
Just instead of helicopter blades not moving, it’ll be your data points. No, instead what we do is we scan every single sample and pick out the largest and the smallest numbers from the chunk, which in our case means running through every 7,938 samples and picking the biggest and smallest ones. Then we plot them on top of each other, maybe with a line connecting them. Flash gets a little slow working with lists and arrays this big, but I’m sure we’ll figure out some neat tricks to get this stuff working fast. That said, here’s a little demo. If you can figure out the song I’ll give you a million bucks. [Let it chug for a while, Flash is a slow beast]:
ps – don’t cheat, guess the song before you download the source, goofball.
Yesterday I started fooling around with FP10′s sound generation capabilities and I got the idea to code a little Theremin. It came out pretty good, but was bland in aesthetics. I decided to try my hand at coding a visualization algorithm similar to the type of visualization one would see while using Amadeus or Digital Performer, etc. I wanted to plot the wave form on the stage. Taking a copy of the sound buffer and selecting equally spaced samples I plotted them on the bitmapData of a Bitmap object and drew lines between data points. This is what it looks like:
The mouse position in x controls the frequency of a sine wave [from 220Hz to 440Hz or A3 to A4], while the y position controls volume. There’s some tearing that occurs when modulating volume, I haven’t figured that out how to fix that yet. As you move your mouse the wave that is generated is plotted on the stage. The pink dots represent sampled data points [which correspond to speaker movement] while the purple lines are approximations of the in-betweens [just lines drawn from point to point]. After playing with that for a while I wrote a class that matches note names to frequencies and drew markers indicating where on the x-axis of the stage one would have to place the cursor to generate a given note between A3 and A4. This is just a small example of the possibilities of sound in flash, and one way to interact with it. I’m looking forward to building on the classes I made for this example. I’ve included my entire TextMate project and source below [which will be the foundation of many more experiments], you can chop it up and use it however you want, although I didn’t copy any license to it, but trust me I won’t sue.