Developing real-time musical improvisation systems involves a number of challenges from data collection through to performance. There is currently a lack of datasets that capture improvised music performances with symbolic detail. Research models for capturing subtle performance detail can be very large and complex, but they also need to be fast enough for real-time interaction.
Data was collected from musicians who are skilled at improvising using a Roland TD-50 electronic drum kit and melodic instruments, such as saxophone and clarinet. Information including the precise position the drums were hit and hit velocity was recorded in order to capture realistic performance traits in detail. A neural network was then trained to predict a measure of drum performance, with a control of how the model output is sampled to vary predictability of performances in real-time.
The Musically Attentive Neural Drum Improviser (MANDI) is now being trained on a new world-first large-scale (50 hour) data-set of variable tempo improvised drum duets using a new model architecture.
The research is ongoing.