Infrared vs. ultrasonic finger detection on a virtual piano keyboard

An immaterial digital keyboard is presented, aiming at testing possibilities to substitute physical with augmented piano keys during the performance. The main effort has been made around the realization of an accurate meanwhile fast detection of the hands movement. To achieve this goal we have tested low-cost infrared as well as ultrasonic capture devices, whose current pros and cons are presented in either cases. Multimodal feedback has been realized by filming the hands’ action with the rear camera of a consumer’s tablet PC, and then projecting this action on its screen; furthermore this projection has been layered over the image of a piano keyboard reacting to the hands’ action. Especially in connection with a Leap Motion system in charge of doing the infrared-based detection, and other light hardware for the sonic and vibrotactile rendering of the feedback, the proposed prototype promises potential application as an inexpensive mobile music interface transforming a normal table in an augmented reality scenario, where a pianist can perform simple musical tasks by relying on reasonably accurate and realistic feedback even in absence of a hardware keyboard.