Dropnotes

Programmed water droplets sampler + sequencer

The initial idea for the project was basically our interest in how we can design music composition interface easier and intuitive. Digital Audio Workstation (DAW) software is very complex to use, and sets a very high bar for new users. We ran a role-playing brainstorming session, playing the possible interaction with some tangible artifacts at hand (such as tissue box, scissors, cap of a pet bottle, etc.) on a table - and came to the idea of representing the whole composition process in series of tangible interactions related to water, such as a funnel, droplets, water dyes, etc. We designed recording interaction with a metaphor of infusion: putting a funnel to a water bottle (each bottle contains colored water) and sample sound in it. Those colored water represents sampled sound - they can be used as a note on the glass table. When users are placing droplets - the glass table shows the grid (radius = pitch, angle = sequencer timing) so user can compose rhythm and melody.

We used Mac Mini as our machine to run a few programs on - 1) image processing to label / recognize droplets on the glass table, 2) projection mapping on to the glass table to render UI / animations, 3) recording input from the water bottle stand and 4) sound processing for audio out. Any image i/o was in Processing, and the audio i/o was in Pure Data with Arduino USB device. A few notes about each program: 1) and 2) - Image i/o program runs both labeling of water droplets and projecting mapping to project animations. 3) The funnel object has a magnet on its tip, and the water bottle can switch on/off the recording per channel distinguished with magnetic sensors. 4) Runs a sequencer with pitch control - it shifts the pitch on a pentatonic scale so any pitch combination sounds musically intentional.

We published several research papers about the project. - "DropNotes: A Music Composition Interface Utilizing the Combination of Affordances of Tangible Objects", Proceedings of the 10th International Conference on Advances in Computer Entertainment (2013) (equally contributed 1st author). - “DropNotes: A Music Composition Interface Utilizing the Combination of Affordances of Tangible Objects”, Inter College Computer Music Concert 2013 (2013).