This project extends the older Touch Table with multi-touch capability. Building on our previous experiences with Tangible Acoustic Interfaces, and in particular of combining acoustic techniques with computer vision techniques, we developed a technology capable of transforming virtually any surface into a giant multi-touch input interface. The MUTE technology (MUlti-Touch Everywhere) is based on the same approach as the virtual keyboards found on the market (watching fingers intercepting a plane of infrared light), but we have adapted this method to be compatible with larger surfaces than the usual keyboard size, and combined it with acoustic onset detection in order to get precise timing information. In addition to fingers, our system can detect oblong objects striking the surface, like sticks and mallets, and it also measures the intensity of taps or impacts, allowing to perform the interface both with percussive and touch gestures. Our system has a precision of about 3 mm, which makes possible to draw virtual buttons on the surface and use them as menu commands or toolbars (see picture). In combination with the Surface Editor, any surface can be transformed into a freely reconfigurable multi-touch musical controller!

The MUTE technology features low-latency finger tracking, which is an essential aspect for making music. We could bring the latency of the system down to 10ms, thanks to the use of high-speed cameras and a custom tracking application, the Surface Tracker. This open source application is available in the Download section.