As music AI systems become more interactive, a major challenge is for both human and machine to better understand each other. When human musicians improvise together, communication is a natural and intuitive process, as performers make use of non-musical cues and communication to keep things flowing. But when your improvisational partner isn’t human, this natural communication breaks down. How do you know what the AI is “thinking” or “feeling” if its just a black box?
In this research we investigated how musicians and artificially intelligent music improvisors can keep a performance flowing through non-musical communication. We trained a neural network drummer to improvise with human musicians and explored bi-directional communication between human and machine performers.
The AI improvisor communicated its confidence in the performance using a simple animated emoticon. When the AI lacks confidence it avoids eye contact and frowns. When confident it looks at the human performer and smiles. Sustained confidence causes the emoticon to glow. We communicated the human performer’s state to the AI by measuring skin conductivity during the performance. Prior research had demonstrated that skin conductivity can be used as a proxy for being in musical flow states.