Multi-sensory design, physical computing and experimental validation of virtual buttons and knobs for professional appliances

Touch screens have become the most popular computer interface in our daily life. Although they respond to finger contact, most of them are able to provide only a visual representation of standard physical controls such as buttons, sliders, and knobs. Indeed, tactile and auditory feedback is neutralized by a touch screen into an interaction with a flat surface. Among other issues that occur during this interaction, the lack of multimodal sensory cues inevitably unbalances the cognitive load toward vision. This thesis targets the design, realization and validation of user interfaces for the professional appliances domain. In such a specific working environment, populated by multiple and cooperative human activities that frequently and sometimes unexpectedly trigger attendance by the operator, the use of touch screens is generally problematic because often the visual attention must be switched from a machine interface to an emerging task, without suspending any activity in between. In this context, new designs and implementations providing rich auditory and tactile information can contribute to more reliable and robust interactions capable of restoring, partially or absolutely, the multisensory essence of physical controllers hence improving effectiveness and safety of the working environment. To such an end, this thesis discusses three interface prototypes reproducing the multimodal feedback of two fundamental physical controllers: buttons and knobs. The proposed research starts from the peculiar interaction with professional appliances, and aims at 1) designing and 2) prototyping audio-haptic interfaces that combine innovative haptic and force actuators, as well as 3) evaluating such interfaces through rigorous experiments involving human participants. The first part of the thesis deals with surface haptics --- a research field studying the reproduction of tactile effects on touch surfaces. This part provides a series of experiments on auditory and tactile surface perception, together with the development of a mock up aimed at testing and improving the reliability and robustness of interaction with professional appliances controlled by touch screen interfaces. The second part of the thesis deals with rotary controllers with haptic feedback. As opposed to touch screens, knob controllers do not require visual attention, thus allowing a user to perform multiple actions simultaneously. In this part, the design of two innovative knob controllers is presented, each providing specific haptic features: while the first device essentially consists of a technology improvement in the form of a programmable haptic knob exposing a low-cost force resistance technology, the second device, called "Non-a-knob'', affords a new interaction primitive in which rotation gestures are operated over a motionless cylinder. Along with its machine learning-based sensing algorithm, a validation experiment is presented aiming at studying objective and subjective parameters of the user interaction. Finally, the third part of the thesis shows some hardware and software tools that were developed in the context of this research.