AI DJ Project is a live performance featuring an Artificial Intelligence (AI) DJ playing alongside a human DJ. Utilizing various deep neural networks, the software (AI DJ) selects vinyl records and mixes songs. Playing alternately, each DJ selects one song at a time, embodying a dialogue between the human and AI through music. DJ-ing “Back to Back” serves as a critical investigation into the unique relationship between humans and machines.

The system of AI DJ consists of the following three features:

1. Music selection We trained three different neural networks for inferring genres, musical instruments and drum machines used in the track from spectrogram images. AI DJ “listens” to what human DJ plays and extracts auditory features using those networks. The extracted features are compared with those of all tracks in our pre-selected record box, so that the system can select the closest one, which presumably has similar musical tone/mood.
2. Beatmatching It is also a task for AI DJ to control the pitch (speed) of the turntable to match the beat. We used “reinforcement learning” (RL) to teach the model how to speed up/down, nudge/pull the turntable to align downbeats through trials and errors. For this purpose, we built an OSC-compatible custom turntable and robot fingers to manipulate.
3. Crowd-reading A good DJ should pay attention to the energy of the audience. We utilize a deep learning-based motion tracking technique to quantify how much people in the audience dance to the music AI plays for future music selection.

We have performed several times in different locations in Japan and Europe. AI’s slight unpredictability always brought amusing tension into the performance and gave new ideas to human DJs on what/how to play music as a DJ. AI is not a replacement for the human DJ. Instead, it is a partner that can think and play alongside its human counterpart, bringing forth a wider perspective of our relationship to contemporary technologies.

Shoya Dozono (JP)Shoya Dozono (JP), born in 1988. Designer/Programmer. After graduating from the Faculty of Design, Tokyo Zokei University, Dozono completed his Master’s degree at the Institute of Advanced Media Arts and Sciences(IAMAS). He joined Qosmo in September 2016. Since 2013, Dozono has worked as a visual programmer for Hiroaki Umeda’s dance and audio-visual projects.

Nao Tokui (JP)Nao Tokui (JP), born in 1976, serves as the CEO of Qosmo Inc. He is also a media artist and a DJ. Tokui received his PhD from the Department of Electrical Engineering and Information Systems (EEIS), Graduate School of Engineering, The University of Tokyo. After pursuing his research and creative interest as a visiting research fellow at Sony Computer Science Laboratories in Paris, Tokui founded Qosmo in 2009. His recent works include the production of a music video of a song by Brian Eno, using AI.

Credits:
Concept/Machine Learning: Nao Tokui
Visualization: Shoya Dozono
Project management: Miyu Hosoi (Qosmo)
Assistance: Yuma Kajihara (Qosmo), Robin Jungers (Qosmo)
Robot: TASKO, inc.
Customized turntable for AI: Mitsuhito Ando (YCAM)
Production support: YCAM InterLab