The App that Changes the Sound of Music According to the Colors Around You

If, like everyone in the world, you’ve ever listened to your new favorite song on an endless loop, it can start to get a bit boring. The drums begin to sound the same, the melody no longer creates that funny feeling in your stomach, and soon enough, it’ll be relegated to the bottom of your iTunes library, banished for five years until it’s bearable enough to listen to again. However, there may soon be a solution to this problem.

An innovative and interactive musical app called “Sadly By Your Side”, aims to offer a new musical experience with each listen (building upon a concept that Gwilym Gold established). The app allows you to remix the Sadly By Your Side album, which was already released by Davide Cairo on the Fabrica arts label, who collaborated with the making of the app. Essentially, it will respond to the colors in any given environment that the phone’s camera is placed in. For instance, an environment with mostly black-toned colors will alter tempo, while a red or blue background would vary the tone of the piece. I couldn’t wait to try out every song in a rainbow-colored room, but first I called up Angleo, who is behind the app, to find out a little more.

Noisey: What was the initial idea for “Sadly By Your Side”?

Angleo: It started as part of a bigger project called objectify Mp3. It’s a project we started in Fabrica and the song grew from there. I have a computer science background, mostly in interaction design, and I’ve developed a few music apps already.

How did you feel about it initially, Davide?

Davide: I’ve worked on a lot of installation. I wanted to distribute the record in a new way. Fabrica’s not a record label, we wanted to try out new things. It was like doing a record for the second time.

How did the project develop, was it a collaboration?

A: The album was already done before the app. I thought about windows media player and other ways of expressing music. The iPhone is not just an MP3 player, it can do a lot more, it can add to the listening experience. I thought about how I could expand the surroundings of a person’s listening experience by going with the colors. I wanted the music to change with the listener’s environment.

D: Music that was constantly adapting really excited me.

In layman's terms, can you explain how it actually works?

I used this program called Open Frameworks for the installation and Core Audio which is used for Logic as well. So the camera sees your environment and then takes it apart pixel by pixel, working out the colors. These correspond to a formula which adds or subtracts from the volume, delay, or reverb.

How many variations could this produce?

A: It’s potentially infinite; I could work it out though.

What other applications could it have?

A: Someone has asked me to do a poetry version of the idea, the words reacting to the environment. It just needs to create a meaningful music experience. The experience should be fully sensory and you should be filled with the music.

D: When I first tried out the app, months after finishing the record, I felt like I was experiencing it again for the first time in so many different ways. I wanted to push the limits by going into different places in Venice and I discovered so many new things. It’s definitely made me change the way I compose. The media we make music on always helps us to change the music.

The future sounds bright guys, thanks!

Follow Dan on Twitter: @KeenDang