Tinguely sculpture
This is an idea for an application that generates MIDI notes from moving machine parts. Not real machines however, but software simulations. Cogs, wheels and gears that rhythmically rotate and drive each other and while doing so transmit their rotation, torque and most importantly their collisions as MIDI notes and CC messages.
I had the idea when I was at the Stedelijk Museum and saw an artwork by Jean Tinguely (see below). It’s a machine-like construction of fragile looking metalwire cogwheels. Although it could clearly function and even has a small electric motor at the lower left corner, a short electric wire ends with an unattached connector.
Jean Tinguely, Elément Détaché III, 1954. Collection Stedelijk Museum Amsterdam, c/o Pictoright Amsterdam, 2016
The machine probably wouldn’t last long if was turned on. The wires are so thin. I imagined how the parts would move if it was set in motion. I realised that I would probably never see it turned on in reality, but that something like this could be simulated in software with a physics engine.
Machine sounds
I’ve seen other machine sculptures by Jean Tinguely over the years. Many of the larger ones are still in operation. It’s fascinating to see the large and complex but useless machines work and move endlessly without ever reaching a goal. But equally impressive is the sound. The screetching noise of metal sliding over metal and the percussive sounds of rusty iron parts hitting each other.
A simulation of a Tinguely machine should always include sound as well as motion.
Box2D and the Web MIDI API
This post shows a web application that uses Box2D to create the machine, and the Web MIDI API to send MIDI out of the application to connected hardware or other software that can receive MIDI messages to generate sound. A sampler, synthesizer or drum machine for example.
Box2D is a ‘2D rigid body physics engine’. It simulates how physical objects would behave and interact as if they had mass, elasticity and friction, and were subject to forces like gravity. If you see a wall of blocks tumbling down realistically in a game, that’s probably calculated by a physics engine.
The Web MIDI API is a new browser standard that provides access to MIDI devices within web browsers. That means a web application can detect MIDI ports and send or receive MIDI messages from Javascript to connected MIDI devices. It’s not yet available in all browsers. Check CanIUse.com to see which ones currently support it.
Have al look at the source files of the app in the repository on GitHub.
The application
The application is shown here, loaded in an iframe. It doesn’t resize responsively while the rest of this site does, so on smaller screens you might only see part of it. Click here to open the application in a new window.
This is only a first proof of concept, so there are no complex systems of gears and cogs. Only four rotating rectangles, two of which are motorized. To keep the animation going.
- Select an output if you have any MIDI equipment to try the app out yourself.
- The Reset button resets the four rectangles to their start position. This is useful because they usually get stuck after some time. It also randomly changes the length of each rectangle somewhat, so the animations will be different each time.
- The Animate checkbox pauses the animation.
To have a quick look at the application in action I made a short screengrab video as a demonstration:
What the app does
The app sends MIDI note and pitch bend data to the selected output.
A note is sent each time two objects collide, and its pitch is always 60 (C3). The note’s velocity depends on the force with which the two objects collide. The note’s channel depends on which two rectangles touch:
- Channel 1 if the left and center objects collide
- Channel 2 if it’s the center and right top
- Channel 3 for center and right bottom
- Channel 4 for right top and right bottom
Pitch bend is sent continuously. It’s directly translated from each of the rectangles’ rotation. Each rectangle sends on one of the first four MIDI channels.
Because the rectangles don’t rotate freely but bounce off each other in unpredictable ways, the pitch bend value goes up and down following an interesting and random path.
This is the the pitch bend data of a demo song recorded in Ableton Live:
MIDI pitch bend data from the Box2D app recorded in Ableton Live.
The results
When I had a working prototype the first thing of course was to try and make some music.
I connected the app to Ableton Live through the IAC Bus (on Mac OS X) and made a demo song.
It’s obvious this music is completely random. There are absolutely no patterns to be found. It’s more of an ambient soundscape than a typical song. But I like it. The sounds that pitch up and down and the white noise fading in and out remind me of tuning an old radio in between stations where you hear beacons, test signals and bits of morse code through the noise.