Stress Detection with Physiological Sensors for Evaluating Interactive Systems and Building Relaxation Training Applications

Affective computing denotes computing that relates to, arises from or influences emotions. Affective computing systems (ACSs) must be able to quantify users' affective state, which influences the system response accordingly: when it is integrated into a computer application, the ACS output is employed to change the application state with a main goal of altering users' emotions. An ACS integrated into a VE–based training application (i.e., an application that, with the use of immersive virtual environments, allows the acquisition of knowledge, skills and competencies through direct or indirect experience) can modulate the intensity of stressful stimuli to increase trainees' habituation to stressful situations and the ability to maintain a high level of performance. Psychologists are studying the nature of emotions since the 19th century, without reaching an universally accepted definition of what are emotions and how emotions are generated. However, over a century of research shows that emotions and bodily functions are related. It is not surprising, therefore, that many studies in the affective computing literature employ physiological data such as cardiovascular, muscular, and brain activity to detect users' emotions. Other instruments could be used to assess users' affective state, such as questionnaires and scales. However, they cannot be administered to participants without interrupting the experiment (and thus potentially affecting the participants' emotions); furthermore, the intrinsic ambiguity in describing emotion through written words could undermine the reliability of such instruments. Despite all the research work, the design of an ACS remains a very difficult task, for two reasons. First, there are still issues that have a critical impact on the development and evaluation of ACSs: for example, instances of unique one-to-one relationship between a physiological measure and a psychological state, which are ideal for ACSs, are very rare in the psychophysiological literature relative to one-to-many, many-to-one, and many-to-many relationships. Second, it is not trivial to appropriately adjust the application state to influence users’ emotions in the desired way. Emotions can be experienced for a variety of reasons: a malfunctioning controller, a difficulty level perceived as too high, an increase of the room temperature, etc. In those cases, even if the ACS has correctly detected the user’s affective state, it may not be able to identify the reason for the high stress level, and thus may incorrectly modify the application state. This thesis highlighted the importance of physiology for different research topics other than affective computing, illustrating how physiological signals can be successfully employed to evaluate the ergonomics of novel interaction devices and techniques. Physiology was also useful for conducting the evaluation of a violent video game that gave more prominence to the effects on UX with respect to classic studies on video game violence. This thesis also showed that physiology can be used to build and evaluate applications for relaxation training. To be able to influence users' emotions on the basis of their bodily reactions, we then developed an ACS for automatic stress detection that builds upon the current state of the art in affective computing. The proposed ACS, called ACME, can read in real–time users' physiological signals, and determine their level of stress. ACME was tested on a game designed to support relaxation training that employs realistic 3D graphics to provide players with embodied feedback. Affective computing, however, needs to further progress. As shown in this thesis, a possible solution to improve the detection of stress and anxiety could be the use of signals that are successfully employed in other research fields, such as eye–blink startle response (EBSR). Finally, this thesis proposed an improvement over the traditional comparison methodology employed in ACSs evaluations in affective computing studies. For the evaluation of the proposed game, we added a placebo condition in which user’s stress level, unbeknown to him/her, is determined pseudo–randomly instead of taking into account his/her physiological sensor readings. The results of the study of our thesis show that without the placebo condition the effects of biofeedback based on ACME and on a second algorithm would have instead appeared similar.