Tools: P5.js, ML5.js

Date: 2021

Music Credits: Looperman

Openprocessing Link: oppr.org/s/DmKRFHXz

This is a music-making tool for multiple people to create live music performances together. The camera in the computer is used to detect body parts and the tool transforms the parts into balls designated with various instruments. A player can then use their body parts to move the instrument ball on the screen and interact with the music’s wave form. (the ball’s color and size will change, and the assigned instrument will be played when they hit the amplitude, *frequency). When there are multiple players, one would be using a MIDI controller/keyboard to add the various drums’ rhythm, others would each be assigned an instrument, hence making music together by moving and dancing their bodies.

The original prompt from YouTube is to create a system that could play music from my computer using a physical custom interface. During my design iterations, I thought about different ways that could boost/promote interaction and extract human’s emotion and communication, and an idea came to my mind that the visual could also be a means for musical input. Hence this design is brought forth: by detecting the movements of a designated body part of the users and transforming it into the meduim that interacts with the sound track, users could bump the waves and create new music.

I adopted the drum as my basic background music, with additional drum kits controlled by one user with the keyboard. The idea came from hearing my own heartbeat in the dead of night. It is so ever-present that we fail to notice it, yet fuels our every movement. While I focused on my heartbeat, I soon became aware of my body vibrating in accordance with it: bodily movement likewise is ever-present. All these movements reminded me of the beats of a drum. It may be the reason why its sound is encouraging for people to get their body moving.