Auditory, Visual And Somatosensory Localization Of Piano Tones: A Preliminary Study

The paper presents an experiment in which subjects had to localize headphone-reproduced binaural piano tones while in front of a grand Disklavier instrument. Three experimental conditions were designed: when the fallboard was closed the localization was auditory only; when the fallboard was open the localization was auditory and visual, since the Disklavier’s actuated key could be seen moving down while the corresponding note was produced; when the listener actively played the note the localization was auditory, visual and somatosensory. In all conditions the tones were reproduced using binaural recordings previously acquired on the same instrument. Such tones were presented either transparently or by reversing the channels. Thirteen subjects participated in the experiment. Results suggest that if auditory localization associates the tone to the corresponding key, then the visual and somatosensory feedback refine the localization. Conversely, if auditory localization is confused then the visual and somatosensory channels cannot improve it. Further experimentation is needed to explain these results in relation with i) possible activation of the auditory precedence effect at least for some notes, and ii) potential locking of the sound source position that visual and/or somatosensory cues might cause when subjects observe a key moving down, or depress it.