Meet the Musical Clairvoyant Who Finds Ghosts In Your MP3s

This sound technician makes music from the bits cut out during compression, and it's mad spooky.

|
Mar 18 2015, 11:24am

Remember when the above face of Richard D. James appeared during Aphex Twin’s “[Equation]” if you listened to it through a Spectograph? Or that moment during Radiohead’s In Rainbows where the backing vocals whisper the album’s title at a specific time to indicate the “golden ratio” moment of the record? Well, these are just two examples of how music can often have a buttload of very secretive hidden layers.

These days though, in our rush to listen to all music everywhere at all times, we often sacrifice these layers by listening to the most readily available streams or downloads, which are usually relatively crappy formats like MP3, AAC, or whatever the hell Grooveshark uses, which can sometimes sound like the recording of a song being through a coke can in a garden shed.

Often, we’re losing out on a significant amount of what the artist intended, because when the original analog music is converted to one of these formats, certain layers of sound are lost in the digital compression. Translation: there’s a lots of bits to your favourite albums that you may have never even heard.

Exploring this, is the Ghost in the MP3 project by doctoral music student Ryan Maguire from the University of Virginia’s Center for Computer Music. He investigates these lost layers of sound, what they sound like when rescued, and then tries to make new music with them. For an example in his study, he took the layers of sound lost to compression from the acapella song “Tom’s Diner” by Suzanne Vega, which was also the template song used by Karlheinz Brandenburg, the pioneer of the MP3, to test whether the compression of MP3s worked. You can hear the track he made from those bits below.

I decided to get in contact with Ryan, because the idea of me not hearing my favorite albums properly scares the shit out of me. And I also decided to quiz him on why compression matters so much, and whether we should all be rushing out to buy one of Neil Young’s Ponos.

A track made by Ryan using the sound lost via compression of "Tom's Diner" by Suzanne Vega

Noisey: Hi Ryan! So what fascinated you about MP3s specifically?
Ryan Maguire: Well, the thing about MP3 is that unlike previous technologies it stores music measured towards a human auditory perception, what we hear. They reduce file sizes dramatically by erasing information that most people won't hear. This is done by focusing on human perception very closely and erasing sound that should be just beyond the limits of our perception. Of course when this is implemented in real life, the ideal scenario is never quite reached, but the fact that we all listen to MP3s is a testament to how close it comes.

Continues below

So you want to find these lost layers and make something out of them?
The end goal for me is to create music, so I had to find a way to take these lost ideas, which are very interesting conceptually, and derive interesting sounds from them. I would compare an uncompressed audio file with a compressed MP3, and figure out what the difference was between the two. That difference would become the "ghost" eventually. To get there I had to analyze these audio files and compare the results.

So, how did you compare your results?
There are two primary ways of looking at digital audio on a computer- as a waveform and as a spectrum. Both of these representations have the same information overall but they reveal different aspects of the sound. Once I started comparing spectrums, I was able to look at the lost audio more closely and isolate certain parts of it.

To simplify this with an extreme example, here's Ryan's uncompressed recording of white, pink and brown noise:

And here is the same recording compressed to the lowest form of MP3 (8kbps). Do you hear that?

Did you notice the difference in sound right away?
I did have a small eureka moment when I finished rendering the first file and finally got to hear it. It was so ghostly and grainy, I immediately realised that I'd be able to compose something interesting with this material. So, I decided to work with that song "Tom's Diner" as a starting point. I got pretty deeply involved with analyzing the song structure and the lyrics, and looking for musical and poetic ways to play with how this song sits in our collective consciousness. I wanted it to sort of dance around the melody, flickering between being present and absent.

Why did you pick "Tom's Diner"?
When the MP3 was being developed in the early 90s in Germany, the engineers needed music to test their algorithm with. One of those engineers, Karlheinz Brandenburg, heard “Tom's Diner” on the radio and realised that this very delicate vocal acapella track would be difficult to encode convincingly as an mp3. So, it became the litmus test for the algorithm: if “Tom's Diner” didn't sound good they went back to the drawing board. Because of this, Tom's Diner is known as the "Mother of the MP3," and it represents a best case scenario for mp3 encoding. It seemed like a logical starting point for my project.

What did you think of the sounds or ghosts that were left from the song after the conversion?
I thought the sounds were really beautiful and kind of spooky. Vega's voice was transformed into this almost unintelligible whisper. So I knew I could make something as soon as I heard it. A problem with a lot of electroacoustic music experimentation today is that composers will just make demonstrations of their new technology or technique, instead of actually making music. So, I was very conscious of not wanting this to just be a demo—that would've been the easy way out. I could've just released the "ghost" file unedited and called it a day, but I wanted to try to really make music out of this material.

What a ghost in an MP3 might actually look like, if it wasn't just an expression.

What's your personal opinion on MP3 and the potential loss of sound?
I think there are better alternatives, such as FLAC, ALAC and even AAC. Nonetheless, I listen to MP3s myself on a daily basis while commuting. The disappointing thing is when people don't realise that there are higher fidelity ways of experiencing recorded music. Recording engineers often lament the fact that most listeners will never have the chance to truly hear what they have created in the studio. It extends beyond just MP3s to cheap preamplifiers and earbud headphones, and even to general cultural attitudes about sound.

Do you think the resurgence in vinyl or Neil Young's Pono are the only ways to go?
No, I don't think so, but I hope that at some point we won't need to use such heavy compression for mobile audio, and that quality will beat out quantity. A funny thing about vinyl in particular is that digital audio often has a wider dynamic range, less noise, a flatter frequency response curve, and even a broader frequency range than an inexpensive record player. But if you want to have a really immersive listening experience, then by all means you should invest in something high quality. People spend so much money on huge HDTVs but folks that love music are ridiculed for investing in a Hi-Fi system, as if wanting to listen to something other than MP3s is a pretentious thing to do.

Yeah, I should probably stop doing that. What does the future store for MP3 ghosts then?
I'm currently working on more songs with this technique, with the goal of releasing an album of this material in the near future. I am very interested in continuing to explore the boundary between what is perceptible and what isn't, and using sound as a means to do so.