An immaterial digital keyboard is presented, aiming at testing possibilities to substitute physical with augmented piano keys during the performance. The main effort has been made around the realization of an accurate meanwhile fast detection of the hands movement. To achieve this goal we have tested low-cost infrared as well as ultrasonic capture devices, whose current pros and cons are presented in either cases. Multimodal feedback has been realized by filming the hands’ action with the rear camera of a consumer’s tablet PC, and then projecting this action on its screen; furthermore this projection has been layered over the image of a piano keyboard reacting to the hands’ action. Especially in connection with a Leap Motion system in charge of doing the infrared-based detection, and other light hardware for the sonic and vibrotactile rendering of the feedback, the proposed prototype promises potential application as an inexpensive mobile music interface transforming a normal table in an augmented reality scenario, where a pianist can perform simple musical tasks by relying on reasonably accurate and realistic feedback even in absence of a hardware keyboard.

Infrared vs. ultrasonic finger detection on a virtual piano keyboard

De Pra, Yuri;FONTANA, Federico;
2014-01-01

Abstract

An immaterial digital keyboard is presented, aiming at testing possibilities to substitute physical with augmented piano keys during the performance. The main effort has been made around the realization of an accurate meanwhile fast detection of the hands movement. To achieve this goal we have tested low-cost infrared as well as ultrasonic capture devices, whose current pros and cons are presented in either cases. Multimodal feedback has been realized by filming the hands’ action with the rear camera of a consumer’s tablet PC, and then projecting this action on its screen; furthermore this projection has been layered over the image of a piano keyboard reacting to the hands’ action. Especially in connection with a Leap Motion system in charge of doing the infrared-based detection, and other light hardware for the sonic and vibrotactile rendering of the feedback, the proposed prototype promises potential application as an inexpensive mobile music interface transforming a normal table in an augmented reality scenario, where a pianist can perform simple musical tasks by relying on reasonably accurate and realistic feedback even in absence of a hardware keyboard.
978-960466137-4
File in questo prodotto:
File Dimensione Formato  
AR_SMC.pdf

accesso aperto

Descrizione: Articolo principale
Tipologia: Documento in Pre-print
Licenza: Creative commons
Dimensione 1.14 MB
Formato Adobe PDF
1.14 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11390/1069785
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 6
  • ???jsp.display-item.citation.isi??? ND
social impact