Soundpainting
Dariusz Jackowski; Francho Melendez
CeTA; University of Wroclaw;
Description
The relationship between color and sound have been largely explored during history. The aim of this project was to create a tool that could be useful for musical exploration, as well as a something approachable for kids, fun, and with application in music education and self-expression.
We designed two iterations of our interactive systems for music composition using a canvas painting paradigm, where a 32 inches touch screen functions as canvas, and a tablet as palette. We develop two different applications that allow the user to compose music as he paints, by interpreting colors, and positions in the canvas as notes and pitch. The systems allow for different modes and synthesizers, control time, looping and even sonify images. We tested the system both for music education for kids, as for artistic interpretation, recovering feedback from artists and iterated towards a useful compositing tool.
System Description
The system consists of a computer, a 32 inches screen, where infrared touching sensors have been installed, a small tablet, and a brush. We developed two different painting applications with different expression capabilities working under the same hardware.
Interface
The first application uses a coarse grid of 70 by 30 cells. The vertical position of the cell will determine its tone while the horizontal position determines time, very much like a pentagram. The grid is played from left to right, either once, or in a loop. We also allow the users to expand the cells horizontally to create larger compositions. As the user paints, the system also produces the sound for the given vertical position and color.
The Palette
We developed a mobile application that runs on the tablet and emulates a painter’s color palette. We originally designed the palette as a set of circles with primary colors and a mixing circle that where the painter could create new color. However, we found that most users would simply take use the primary colours and would not mix them, simply for convenience. It was important to give the user the option to save colours so he could reuse them afterwards.