Back in the late 1990th and the early 2000th there was an artistic movement going on named VJs, who were basically the visual complement to what a DJ is doing with music. Similar to DJs, VJs were reusing existing bits and pieces and often played their videos alongside a DJ. Though computers weren’t as powerful back in the day there were some pretty amazing visualisations done. And as a DJ is changing the set and songs depending on the audience so did the visual artist, making every performance an unique experience. This required a pretty intense collaboration and interaction between both Jays and plenty of bootleg work before hand. Naturally there were only very little artists available and capable of doing that and more and more live-audio-visualisations tools were used.

But anyone, who’s ever experienced a live VJ knows, those automatic visuals are empty and boring. The same was and is true for major light equipment and other visual installations and tools (like fire) on and around stage: though available almost everywhere now, they add only very little as they are used randomly without any “feeling” for the music: By hooking to the baseline or the pitch, you might be able to show the rhythm (partly) but not the melody or intention. Like when a DJ is building up their sound to a climax, so the rhythm, the pitch or the melody grows towards that and then goes for a big splash. Our tools, hooked to the current live audio aren’t capable of comprehending that. And that is why most visualisations feel empty and disconnected.

Which is also why more and more DJs take back the control of those add-ons. And not only sometimes create a big choreography of these tools, including complete videos of multiple hours perfectly synced to the pitches. With the downside of them being static and not able to adapt dynamically to the audience any more. But with the vast improvements on audio analysis and information we are able to extract not only live but even before-hand about audio, we can guess melodies, moods and genres pretty accurately. And we could adapt the visualisation in real-time, our computers are powerful enough.

Just imaging the Windows Media Player color circle not only vibrating on the pitch and the bass and changing color randomly but also growing on intensity, switching colors and form to adapt to the mood and melody by pre-processing the coming audio and extracting context and anticipate what will happen next. It is time we stop understanding visualisations as demos for powerful graphic cards and give it some real meaning. Let’s build algorithms with feel for the music.

Thanks to Osvaldo Gago for releasing this amazing photo under Creative Commons BY-SA 2.0.