We synchronize background music with an animation by changing the timing of both, an approach which minimizes the damage to either. Starting from a MIDI file and motion data, feature points are extracted from both sources, paired, and then synchronized using dynamic programming to time-scale the music and to timewarp the motion. We also introduce the music graph, a directed graph which encapsulates connections between many short music sequences. By traversing a music graph we can generate large amounts of new background music, in which we expect to find a sequence which matches the motion better than the original music.

paper: Hyun-Chul Lee and In-Kwon Lee, "Automatic Synchronization of Background Music and Motion in Computer Animation," to appear in Eurographics 2005 proceedings, also in Computer Graphics Forum.

video: wmv (18MB)