Audio Analogies

Ian Simon         Sumit Basu         David Salesin         Maneesh Agrawala

Abstract

Given a MIDI score and an audio recording of an example piece of monophonic music, our method synthesizes audio to correspond with a new MIDI score. The algorithm we use is based on concatenative synthesis, commonly used for generating speech. Two versions of our algorithm are explored, one in which individual notes from the example piece are concatenated, and one in which pairs of adjacent notes from the example piece are concatenated. We examine the range of example pieces and target scores for which each version of our algorithm yields good results. Our underlying framework remains general enough to be applicable to other problems, such as rendering a stylized version of the target score, and other types of sound analogies.

Paper

Ian Simon, Sumit Basu, David Salesin, and Maneesh Agrawala. Audio Analogies: Creating new music from an existing performance by concatenative synthesis. In ICMC, 2005.

Results

In our first set of results, we used some trumpet and guitar lesson recordings as the example data. For test scores, we chose a few short, (hopefully) recognizable pieces, as well as some of the other example scores. This set of results was obtained using an algorithm which concatenates single notes from the example piece to form the result.

Trumpet Lesson (1)

A (MIDI)
A' (WAVE)

Blues for Alice B (input MIDI) B' (output WAVE)
Trumpet Lesson (2) B (input MIDI) B' (output WAVE)

Trumpet Lesson (2)

A (MIDI)
A' (WAVE)

Ornithology B (input MIDI) B' (output WAVE)
Guitar Lesson B (input MIDI) B' (output WAVE)

Guitar Lesson

A (MIDI)
A' (WAVE)

Blues for Alice B (input MIDI) B' (output WAVE)
Waltz of the Flowers B (input MIDI) B' (output WAVE)

The next set of results uses a second algorithm, which concatenates pairs of adjacent example notes, and blends the overlapping notes.

Trumpet Lesson (1)

A (MIDI)
A' (WAVE)

Blues for Alice B (input MIDI) B' (output WAVE)
Hava Nagila B (input MIDI) B' (output WAVE)

Guitar Lesson

A (MIDI)
A' (WAVE)

Blues for Alice B (input MIDI) B' (output WAVE)
Ornithology B (input MIDI) B' (output WAVE)

The following case is interesting, since it gives us a taste of what can happen when we allow the algorithm to stray from the target score.

Take Five

A (MIDI)
A' (WAVE)

Happy Birthday B (input MIDI) B' (output WAVE)