Stephan Bernsee's Blog

Stephan M. Bernsee talks about developing the Prosoniq morph plug in

by Reinhard G. Benedikt November 1st, 2003, translated by Eva Remowa

How did you come to develop the morph plug in?
Well, I began thinking about such an application shortly after I visited Peter Gabriel at Realworld studios where I was introducing the Neuron synthesizer. They needed a way to seamlessly morph from one sound to another for a television production they were working on. It turned out that the Neuron was capable of doing this, but it is certainly not a good idea to purchase a Mercedes when you just need an easy way to get from point A to point B. So I figured that many people might need such a tool and so we began tossing around the idea of doing a real time morphing plug in. As it turns out, the Prosoniq morph has the potential to be one of our most successful products.

Peter Gabriel with Stephan Bernsee demoing the Hartmann Neuron at Realworld Studios in Bath, UK

What technical problems did you have to solve?
Quite a few. The concept of morphing actually requires looking at the whole sound before it can determine how the morph should sound like exactly. For a realtime application, this restriction is of course unacceptable, since you would prefer not having to analyze your sound files before you can start twiddling knobs. So we had to make our morph algorithm derive a model of the sound that can be continuously updated as more of the sound arrives. The whole process includes a pattern recognition, comparable to a speech recognition in complexity, so we had to find a way to speed it up so it doesn’t consume all available CPU power. It’s still quite demanding, but no more than, say, a good reverb.

From what I’ve read, you originally developed the morphing algorithm on the Atari ST. What is different when developing on todays platforms?
Everything! <g> Most importantly, in the Atari days we had no way to make anything work in realtime when it comes to DSP. We had far less developed tools and no multi-tasking environment, we could hardly debug our code without rebooting the machine in 10 minute intervals. Still, the Atari and the C-64 are probably the only two computers that were used far beyond what their designers intended them to do. Today, developers use maybe about 75% of the potential of the capabilities of a system or processor, since you have to get your products to the market and have less time to really understand what lies beneath. Also, today’s systems are far more complex, of course. The Atari TOS operating system wasn’t really updated for about 6 years. Today, hardly a month goes by without a system update and you usually see a new major system revision every year.

I’ve heard that the Symbolic Sound Kyma also does audio morphing. How does it compare?
Actually, you will find quite a few systems and software packages that claim to do some kind of morphing. Of course, there is no real definition of the morphing process that would indicate how someone has to do it to make it “right”. My definition of morphing comes from the video world, where morphing was first invented and employed by ILM (Industrial Light and Magic) in “Terminator”, if I’m not mistaken. Morphing, in that context means that an object in a scene slowly changes its shape to become another object. The analogous process in the audio world would therefore be an instrument that slowly changes its shape to become another instrument. All morphing processes I’ve seen – except ours – do some kind of spectral manipulation that has nothing to do with the shape of the original instrument. The concept of “shape” requires understanding the nature of the sound that is being processed – I therefore tend to think of them as some elaborate mathematical tools with musical effects, but not really “musical” tools. The problem with these tools is that they inevitably require the whole sound to be analyzed prior to manipulation, and that they tend to produce non-musical results in the intermediate stages of the morphing. Our morphing process is based on an inherently “musical” approach and does therefore not have these restrictions.

You’ve written articles about using the Fast Fourier Transform to achieve musical effects, you have held lectures about it and now you’re telling me that it is an inherently non-musical tool?
Yes and no <g>. If you think of sound, the concept of “frequency” immediately comes to mind. It is therefore understandable that the Fourier Transform seems like the ideal mathematical tool for signal processing in the musical context, since it also has a “frequency” axis. To an extent, this is so. For processes like fast convolution and correlation it is a powerful time-saver with regard to CPU cycles. However, like a screwdriver is a powerful tool for screws but quite useless for nuts and bolts, the Fourier Transform is not a powerful tool for all applications. It is, above all, a non-parametric transform, which means in plain English that it gives a damn about the data it analyzes. It’s like a powerful filter bank, a mathematical tool that does something similar to what a prism does with light. You “gain” nothing about the underlying musical context from it, you just convert your signal into a different representation. It’s still up to you to come up with some intelligent way to derive meaningful information from that. And it has its drawbacks. As I have discussed in the various tutorials on dspdimension.com, operations on the Fourier Transform of a signal can introduce characteristic artifacts which are objectionable in most cases, for similar reasons why you prefer a plain audio CD to a MP3 file when you want quality. Like distortion and aliasing in the “time domain”, warbling and ringing are the characteristic by-products of manipulation a signal in the Fourier transform domain. So, the Fourier transform is just a plain converter, the “musicality” of your algorithm can make use of it, but it has to be musical in the first place – the Fourier transform is not.

So, I guess you’re not using a Fourier transform based morphing. Why, and what are you doing differently?
Our morphing is achieved using an adaptive transform, which means that the transform itself depends on the underlying signal. This allows for adjusting the transform resolution and its basic properties according to the actual signal, not according to a fixed preference. Clearly, this allows for more musical manipulations with less artifacts – thinking of our metaphor from above, it would be like having the whole toolbox available and not just a screwdriver. Of course, like carrying the toolbox means more weight than carrying a single screwdriver we need to rely on powerful processors to make these kinds of algorithms a reality.

Talking about the future – how would you describe the impact of increasing CPU power on your work, and what can we expect in the future?
The impact is quite profound. Things that have not been possible are now slowly becoming feasible on an average desktop computer. We are working with artificial neural networks which are structures that require massive parallel computation. Recent advances in the area of vector unit computation have made them possible to be faster than the playback speed, ie. realtime. Many more will be possible in the next months and years.

One last personal question if you don’t mind: I’ve learned from an osxAudio comment that you’ve had the nickname “morph” when you were working at the UCMG, before Prosoniq was founded as individual company. Is that true?
<laughs> Yes. At that time we had another guy working there named “Nicky Sprenger”. Since my last name at that time, before my marriage, was also “Sprenger”, people kept confusing us, ultimately asking whether we are related and other obvious and unnerving questions. We’re not related, this name match was a mere coincidence which is kindof funny since Sprenger is not really a popular name in Germany. Since I was working on morphing algorithms for our sonicWORX software at that time everyone started calling me “morph” to avoid confusion. At some point there even was a story about the race “morph” which were in constant battle with the “mutants” and I had to come up with stories to answer these types of silly questions that were no better than those about the Sprengers. After the film “matrix”, everyone was suddenly calling him- and herself “morph” and “morpheus”, so this name has lost its charm and everybody reverted to calling me “Stephan” again… of course, Nicky is also no longer working here so this is no problem <g>.

Stephan, thanks for your time!