physical computing, coding

Design Brief:

Create a system that allows you to play music (and possibly video) from your computer using a physical custom interface of your own design and creation. Will you make it for your own personal use or more general, usable by a larger group of people? Is it mean for solo performance or multiple people? Will it be specific to a genre of music? How do you select the parameters that you can manipulate?


I began this project with some experiments in creating a single-channel controller using serial communication. I was really curious about the functionality of pressure sensors, so I used one for this project. I started by setting up my Huzzah board with a pressure sensor, with a resistor and connection to analog input A0.

Then, I wrote the Arduino code to set the sensor as an input and activate serial communication. I used the serial monitor to check that the connection was working. I wanted to write a code that changed the background color and size/color of a circle based on the pressure sensor input. I started with the code for the circle, which used radius to map the serial values to the size of the circle. The circle size change was successful, but I couldn’t get the background color to change. As a workaround, I drew a rectangle the size of the canvas and changed the color using a similar mapping method to the circle.

This method wasn’t ideal, so after some research and tweaking the code I was able to get the background color to change without a rectangle. I set three new variables for the hue, saturation, and brightness of the background, added default values to each, and mapped each variable to the serial value. Then, I used those variables for the background color.

I still wanted to change the color of the circle, so I created variables for the circle hue, saturation, and brightness. This method didn’t work, and I could’t figure out exactly why. The workaround that I was able to figure out was to use the background HSB variables as the circle fill and use division to differentiate the background and circle colors.

After experimenting with serial communication, I started thinking about what kind of instrument to make. Physical computing feels very magical to me, and I wanted to make an instrument that reflected that magic. I brainstormed different ideas using light sensors, and I ultimately decided on building a theremin because of its ethereal quality. I started by setting up my breadboard and ESP32 to use a light sensor on one side and a potentiometer on the other. Then, I set up a basic code in processing for serial communication. I wanted to make sure the values were being read correctly, so I printed and split them into an array.

Once I had the basics setup, I decided to work on the encasement for the theremin. Because the light sensor had to be in a specific place, I glued the breadboard to the bottom of a box, cut out a window for the light sensor and a hole for the potentiometer, and used clear plastic to protect the window. I used some simple printed paper for the outer box, and cut a hole for the cable to go out of the back.

Back in Processing, I used serial communication to map the knob and light sensor inputs to values that I wanted to use for amplitude and frequency. Then, I imported the sound library and introduced a sine wave.

I used a new function called thereminPlay to assign frequency to lightSensor and amplitude to knob. I had some trouble getting this to work, so I renamed the function thereminParam for clarity and moved the function to be below draw(). I couldn’t get any sound, even with these changes.

After the sound not working, I realized that I had set the mapped values of frequency too low for human hearing, which is why I wasn’t getting any audio. Once I remapped the frequency, I heard the wave and decided to change it to a triangle wave, because the sound was more fun. Then, I added the thereminParam() function into the draw function, and added a circle that changed size based on the knob input and the background changed based on the light sensor input.

After some feedback that the knob and light sensor were difficult to reach, I decided to add a hand rest to direct users on how to play the theremin.


When I decided to make the theremin, I wanted everyone to be able to use it. I tried to reflect this through the usage of a hand rest and potentiometer, as well as through the wrapped packaging. Though it can only be used by one person at a time, I would like to make a version in the future that uses pressure and bend sensors within different objects so that multiple people can play the instrument at the same time. There’s no specific genre to the theremin, besides maybe experimental, which is part of what makes it so much fun. I selected the parameters to manipulate (amplitude and frequency) based on the design of real theremins, which have one rod for amplitude and one for frequency. Though my theremin isn’t as sensitive as a real theremin, I hope to be able to add more sensors and parts to my theremin to mimic a real theremin better.