Hamamatsu, Japan, January 31 — Yamaha releases footage of their project to have a machine play music, based on the movements of a dancer. They claim to involve an AI into this process. They managed to get Kaiji Moriyama to be the dancer and had the Berlin Philharmonic Orchestra Scharoun Ensemble as musical support.
The concert originally took place in Tokyo on November 22, 2017, and was named “Mai Hi Ten Yu”‘. The event was sponsored by Tokyo University of the Arts and Tokyo University of the Arts COI. You can watch the video of this footage below the article.
MIDI as foundation to translate movement into music
While the concept of using motion as input to trigger MIDI notes and play sound on virtual instruments, isn’t new, Yamaha has managed to develop their solution into a great state, beyond anything I’ve heard or seen before. MIDI is a digital protocol that transmits signals for musical notes and other configuration of the instrument. The acronym stands for Musical Instrument Digital Interface and has been first developed in the early 80s to create a standard between companies for music production devices.
The simplest and most traditional way for MIDI input is a musical keyboard but ever since the 80s creative people have been working on alternative ways to translate anything possible into new ways of playing music. If you’re really in for experimenting, you can even play music on bananas.
Technologies involved
Yamaha states that their systems adopted an AI which identifies a dancer’s motion in real-time based on the input of four types of sensors on the body of the dancer. Another system consists of a database that allows the mapping of data with particular sounds over MIDI. The new thing here is that the Yamaha AI creates melodies instantly rather than only playing notes on an instrument. In this case, they have been using the Yamaha Disklavier to play the music based on the input.
How all that really resembles to an artificial intelligence is unclear and open to interpretation.
Thoughts from the crew and from the artists
Kaiji Moriyama, the dancer who performed this concert, states, “As the AI system processed and translated my dance movements into music, I felt reawakened to the incredible power of human beings to sense thought and action.” As the representative of the Berlin Philharmonic Scharoun Ensemble, Peter Riegelbauer, adds, “It has been very impressive for us to play with such a wonderful dancer, and we are extremely grateful to have had this great opportunity.”
Composer and Vice President of the Tokyo University of the Arts, Isao Matsushita, expressed, “Art must change in tandem with the changing times. This time, many persons were involved in creating this performance, including a dancer and engineers. This style of creating musical compositions is challenging, but these initiatives have opened up new possibilities and given us even more powerful inspiration than we have experienced in the past.”
Before the closing, Motoichi Tamura, General Manager of Development Department No. 1, Research & Development Division from Yamaha Corporation, wanted to conclude, “At Yamaha, we believe that AI will become a bridge between human beings and musical instruments. We are continuing our AI development activities to enable human beings to have freer and more direct expression through musical instruments.”
YouTube: Yamaha Artificial Intelligence (AI) Transforms a Dancer into a Pianist – Full version
Photo credit: Ayane Shindo / Yamaha
Source: Yamaha press release / hat tip to Steve Dent from Engadget
