Electrodermal activity as a measure of emotions in media accessibility research: methodological considerations

Anna Matamala, Olga Soler-Vilageliu, Gonzalo Iturregui-Gallardo, Universitat Autònoma de Barcelona
Anna Jankowska, Universitat Autònoma de Barcelona and Jagiellonian University in Kraków
Jorge-Luis Méndez-Ulrich, Anna Serrano Ratera, Universitat Autònoma de Barcelona

ABSTRACT

This article proposes electrodermal activity (EDA) as a new objective measure for experimental studies in media accessibility. It first presents a theoretical framework in which the concept of emotion and its categorisation are presented. It then explains how EDA can be used to measure emotional reaction: the article reports on experimental design, participant selection, stimuli preparation, data collection devices, experimental procedure and data analysis. It also discusses briefly how EDA can be combined with other measures. Overall, the article provides a general methodological framework for the implementation of electrodermal activity as a measure of emotions in media accessibility research.

Keywords

Media accessibility, reception studies, electrodermal activity, psychophysiology, emotions.

1. Introduction

Reception research is taking a leading role in audiovisual translation (AVT) and media accessibility (MA), and various methodological approaches are being adopted to investigate audience reactions. Researchers are moving from text-based or speculative studies towards experiments grounded in objective data obtained through diverse devices (Rojo et al. 2014; Szarkowska et al. 2016; Chmiel et al. 2017). This paper focuses on a relatively new measure for AVT and MA, which is used as an indirect indicator of emotional arousal: electrodermal activity (EDA).

In section 2, we set up the theoretical framework, briefly revising research on accessibility services, defining what an emotion is, and introducing the concept of EDA and its measurement. Section 3 proposes the methodology for designing studies using EDA. We report on design, participants selection, stimuli preparation, devices for data collection, and provide some indications on the experimental procedure and data analysis. In section 4 we discuss other measures that can be combined with EDA. The last section summarises the concepts presented in the paper and draws some conclusions.

This article has been written on the basis of the experience gained through the NEA project (pagines.uab.cat/nea), in which EDA has been used to test the effect of different audio subtitling voicing strategies on emotional activation.

2. Some key concepts in user reception, emotions and EDA

Not only is the methodology changing, so is the object of study. The research focus has moved from the study of preferences and comprehension towards user experience (UX), which aims to elicit the way the user interacts and reacts when exposed to certain stimuli. In audio described content, for instance, UX has been studied in terms of presence (Fryer 2013; Walczak and Fryer 2018) and emotion (Fryer and Freeman 2014; Ramos 2015).

These research developments require new methodological approaches: testing the emotion arousal of the user through objective measures such as psychophysiological response is an example of a new method to evaluate UX.

2.1. What is an emotion and how to measure it?

In this section we give a brief introduction to the concept of emotion and its categorisation. This theoretical background will allow us to build a rationale for experimental purposes using a psychophysiological approach to the assessment of emotions.

2.1.1. Defining emotion and emotional response

One of the most cited definitions of emotion is Kleinginna and Kleinginna’s (1981: 355):

Emotion is a complex set of interactions among subjective and objective factors, mediated by neural hormonal systems, which can (a) give rise to affective experiences such as feelings of arousal, pleasure/displeasure; (b) generate cognitive processes such as emotionally relevant perceptual effects, appraisals, labelling processes; (c) activate widespread physiological adjustments to the arousing conditions; and (d) lead to behaviour that is often, but not always, expressive, goal-directed and adaptive.

This definition points out the multiplicity of aspects involved in emotion and refers to four different possible components in emotions: (1) the subjective experience, (2) the cognitive processes related to it, (3) the physiological responses, and (4) the consequent behaviour. It is important to point out that an evaluation of the experience and cognitive processes related to the emotion will always be subjective, i.e. only the person that experiences this emotion will be able to evaluate it and rate it. However, the physiological adjustments that an emotion generates can be measured using different bodily signals, such as EDA, HR, or brain waves, which provide objective indexes for evaluation. Likewise, the presence of a consequent behaviour is also observable and can be measured.

At a physiological level, an emotional response consists of three types of components: behavioural, autonomic, and hormonal. These components are controlled by separate neural systems. It has been shown that single neurons in various nuclei of the amygdala become active when emotionally relevant stimuli are presented (Jacobs and McGinty 1972). The emotional information arrives to the amygdala from all regions of the neocortex synapsing in the lateral nucleus, travels through the basal nucleus and from there gets to the central nucleus.

The central nucleus of the amygdala is considered the most important region of the brain for the expression of the emotional reaction. Its activation in the face of threatening stimuli has been proved, as well as the absence or diminution of emotional response due to inactivation or lesion (Philips and Le Doux 1992). The central nucleus projects to regions of the hypothalamus, midbrain, pons, and medulla, which are responsible for the expression of the various components of emotional responses.

The hypothalamus regulates the autonomic circuits to recruit appropriate physiological responses for specific emotions, with different patterns of activation characterising different situations or experiences and their associated emotional states. For example, facing a stressful experience or living a threatening situation triggers feelings of fear, anger, or sadness, with their characteristic pattern of autonomic manifestations. On the contrary, when we are in a context of joy and cheerfulness, physiologic responses follow a pattern associated with happiness. These characteristic responses may involve changes in the activity of the visceral autonomic motor systems and may result in increases or decreases in the heart rate, cutaneous blood flow, piloerection, sweating and gastrointestinal motility. These are the result of the complementary and even antagonistic coordination between the sympathetic and parasympathetic responses, integrated by the central autonomic network. Physiological responses are usually activated by basic emotional stimuli like threats or predators, but complex and idiosyncratic stimuli mediated by the forebrain such as films, books or music can also lead to autonomic activations and arouse a sympathetic response.

Given the relationship between the emotions and the amygdala and, consequently, with the autonomic nervous system (ANS), psychophysiology measures depending on the ANS will be good measures of the physiological component of emotions. As the focus on this paper is on EDA as an objective measure for emotion, a taxonomy of emotions will be introduced in the next section.

2.1.2. The categorisation of emotions

There are two main approaches to the classification of emotions, namely the discrete model of emotions and the dimensional model of emotions. The former proposes that each emotion consists of a unique set of experience, physiology and associated behaviour (Izard 1977; Ekman 1984, 1992), implying that there are a few basic emotions that can be easily recognised. Yet, the identification of these basic emotions has proved rather controversial. Based mainly on facial expressions, Ekman (1984) proposed six basic emotions: happiness, disgust, anger, fear, surprise and sadness. To these, Plutchik (1994) proposed a model adding trust and anticipation. Other authors suggest that basic emotions can combine to generate complex emotions, examples of which are shame, embarrassment, guilt or pride. These emotions involve the knowledge of cultural rules and appropriate behaviour and are called consequently self-conscious emotions (Lewis 1995).

The dimensional model (Russell 1983; Lang 1988; Mauss and Robinson 2009), on the other hand, describes emotions according to different psychological components, mainly valence or polarity and physical activation or arousal. Valence or polarity is the dimension related to the experience of the emotion that can go from positive (pleasant emotion) to negative (unpleasant emotion). Physical activation or arousal refers to the physiological reaction that accompanies the emotional experience and can go from low to high activation. According to this model, sadness could be described as negative in terms of valence and low in terms of physical activation, whereas fear could also be rated as negative but high on physical activation. The dimensional model allows for the representation of different nuances within some discrete emotion. For example, happiness has a positive valence, but it can convey different levels of activation found in the definition of current words related to the realm of happiness, e.g. from excitement to joy.

Other authors have proposed new dimensions, such as control over the emotion (Lang 1988; Fontaine et al. 2007) or approach and avoidance (Mauss and Robinson 2009). We have control over the emotion when it does not modify substantially our physical or mental state, while other emotions can be overwhelming and give us the impression that they prevail upon us. This is a dimension easy to rate by users and it is included in subjective measures of emotion (see description of SAM in section 4). The approach and avoidance dimension, which refers to the desire to approach the source of the emotion or rather to shy from it, is hard to evaluate, because, in terms of valence, the approach can be either positive (elation) or negative (fear), and avoidance can also be appraised as positive (relief) or negative (depression) (Carver 2006). Due to this multiplicity of interpretations, it is difficult to take this dimension into account for subjective measures.

The discrete and the dimensional models of emotions are not in conflict, and some authors use a combination of both (Harmon-Jones et al. 2017), since the basic discrete emotions can be described according to the dimensions, as illustrated above in the cases of sadness, happiness and fear.

When it comes to measuring emotions, different approaches can be combined: subjective ones like questionnaires and objective ones mainly based on the physiological reaction to emotions (see section 4). One of the most popular psychophysiological measurements in the study of emotions is EDA, as it enables a direct link between skin reaction and the ANS activity. In addition, the devices required for its measurement are non-invasive and relatively easy to use, as discussed in the following sections.

2.2. EDA and its measurement

EDA is a measure of electrical activity on the surface of the skin or originating from the skin. It has been used for more than a century in psychological studies and it is still one of the easiest and most reliable psychophysiological measures to apply (Bailey 2017) when assessing physiological activation in the context of emotional induction. The relationship between skin conductance and emotions was established at the end of the 19th century and was used in research shortly afterwards (Landis and Hunt 1935).

A long history of research into the active and passive electrical properties of the skin has resulted in a plethora of terms, such as skin conductance, galvanic skin response (GSR), electrodermal response (EDR), psychogalvanic reflex (PGR), skin conductance response (SCR), sympathetic skin response (SSR) and skin conductance level (SCL), though the terminology has been recently standardised under the term EDA (Boucsein 2012).

To measure EDA, the skin’s response to the passage of a small electric current is registered. The ease with which the current flows between two points on the skin can indirectly indicate the arousal or activation of the ANS. The conductivity of the skin surface depends on how much sweat the body produces. When a person is tense or emotional, the sweat glands become more active due to the activation of the parasympathetic branch of the ANS, increasing skin moisture. This allows the electric current to flow more readily, which reflects the emotional activation state.

The most widely accepted model of skin conductance is the sweat circuit model proposed by Edelberg (1972). According to this model, short and instantaneous changes in skin conductance, known as phasic changes, occur when the sweat ducts in the epidermis fill. Skin conductance recovers to tonic levels, that is, the steady level when the person is non-altered, when the moisture is deposited on the skin or reabsorbed by the sweat glands. In Edelberg’s model, the sweat ducts act as variable resistors because their resistance lowers, and conductance increases, as they fill with sweat. The amplitude of the change in conductance depends on the amount of sweat delivered to the ducts and on the number of sweat glands that are activated (Das and Roy 2016). Sweat gland activation is a simple physiological survival mechanism, controlled by the brain via the sympathetic division of the ANS.

Although the electrical change alone does not identify which specific emotion is being elicited, it has been observed that some patterns correlate with different basic emotions such as happiness, anger or sadness (Kreibig et al. 2007). The activation and valence of these emotions correlate with the drop and recovery of skin conductance. The relationship between the activity of sweat glands and the ANS and its effects on the electrical conductance of the skin EDA is a good measurement of emotional arousal.

3. Experimental methodology

In this section we introduce a general methodological framework for using EDA measures to assess emotional activation of users when exposed to audiovisual stimuli and, more specifically, to programmes that have been translated or made accessible. Previous studies have measured EDA to assess users’ sensitivity to emotional video scenes (Brumbaugh et al. 2013), to compare emotional involvement in 2D and 3D movies (Rooney et al. 2012), and to contrast gender differences in the emotional reaction to pleasant and unpleasant movies (Codispoti et al. 2008), to name just a few.

Within the field of AVT and MA, to the best of our knowledge, EDA has only been experimented with by Fryer (2013), who used it together with self-reporting instruments to investigate audience reaction to different types of audio descriptions in clips eliciting three targeted emotions: amusement, fear and sadness. In the following sections we suggest methodological guidelines to conduct experiments, taking into consideration experimental design, participant selection and recruitment, emotion selection and stimuli preparation, and data collection and analysis.

3.1. Design of experiments using EDA

The objective of EDA measurement is to compare the fluctuations between the tonic levels of this parameter, in absence of stimulation, with the phasic levels of this parameter, induced by different types of emotional stimuli. EDA is sensitive to individual differences (e.g. trait anxiety) and contextual variables, such as the physiological arousal of the participants induced by everyday situations prior to the experiment. Therefore, the advisable experimental design for this type of assessment is a within-subjects design in which participants are exposed to all experimental conditions. This allows for observing the differential effect of the various types of stimulation compared to a baseline.

The baseline consists in reducing the vital signs to a stable minimum and should be recorded for a duration of 2 to 4 minutes (Braithwaite et al. 2015). Prior to that, it is recommended that participants go through a brief relaxation period to standardise their emotional state, which can be induced with relaxing stimuli or exercises such as watching neutral images or listening to certain sounds. To that end, the International Affective Picture System (Lang 1997) and the International Affective Digital Sounds (Bradley and Lang 1999) are widely used resources. Another highly recommended technique is the diaphragmatic breathing relaxation (Jerath et al. 2015).

It is recommended that when exposed to emotional stimuli a brief recovery period of 2 to 4 minutes of duration be left between them. This will extinguish the emotional response produced by the previous stimulus, before the presentation of a new one. During this time, the subject should be required to sit comfortably and relax, doing a brief task of diaphragmatic breathing, for instance, which could be accompanied by relaxing visual or auditory stimulation.

3.2. Participants and ethical approach to testing

Research on MA often relies on users of accessibility services, who are a minority in the global population. For this reason, recruitment of participants for experiments often requires the help of user associations and it should be borne in mind that, according to Orero et al. (2018: 110), “sample sizes lower than 25 per group are unlikely to yield statistical power,” and this could also be applied to EDA research.

The European Code of Conduct for Research Integrity (ALLEA 2017) recommends adherence to four main ethical principles, namely, reliability, honesty, respect, and accountability. The experimental protocol and procedures must be approved by the ethics committee of the institution where the research is to be conducted. Special care must be taken to make sure users are clearly informed of the purpose of the experiment and the procedure, as well as the duration of the experimental sessions, and that they give their informed consent. In the case of users with disabilities, alternative means of communication must be provided, if necessary, such as a version of the informed consent form in sign language, a Braille version, an easy-to-read version or an oral recording. Users must know that their data will be confidential, and that it will not be possible to connect their biometrics to their name. They should be informed whether their data will be preserved for future analyses or destroyed within a specified period. The approach taken may vary according to the specific experiment, although many funding agencies currently require open access to data. In any case, explicit consent must always be obtained from all participants.

The health and safety of participants must also be duly considered. Although the sensors for EDA are not invasive (Figure 1) and do not pose any risks, some participants may feel uncomfortable because the researcher must touch their hand to place the sensor on the fingers.

The researcher must guarantee a comfortable environment and provide all the necessary information, allowing participants to see and touch the devices, an aspect which is especially relevant for persons with sight loss. All participants should be informed that they have the right to withdraw from the experiment without giving any further explanation.

3.3 Stimuli selection and preparation

Any study with EDA should first define the emotions to be analysed as well as the stimuli to be used, as discussed below.

3.3.1. Selecting emotions

In section 2.1, it has been shown that emotions are linked to certain responses of the organism. When considering the selection of emotions to be tested, some scholars have argued that negative emotions are easier to induce than positive ones, as they result in body reactions with higher arousal (Taylor 1991; Westermann et al. 1996; Uhrig et al. 2016). According to Kreibig (2010), the emotions most commonly used in research are fear, sadness and anger.

Both anger and fear are considered to have similar characteristics when it comes to their position across dimensions and their psychophysiological values, as they both give low values on the valence or polarity dimension and induce high arousal (Lang 1995). In terms of physiology, some authors have noted that these two emotions are barely distinguishable (Russell 1980; Watson and Tellegen 1985), while others find both emotions have a similar pattern involving an increase in EDA and in other measures such as HR (Kreibig 2007). As for sadness, it is considered to be also negative in its dimensional values but with low arousal – or lower than anger and fear (Lang 1995). Its associated physiological response is characterised by an overall withdrawal of the sympathetic system, thus implying a decrease in EDA (Kreibig 2007).

To sum up, different emotions may result in different physiological reactions, but negative emotions are more suitable than others for experimental research.

3.3.2. Selecting clips

When deciding on suitable video extracts, two strategies are possible. The first strategy consists in using material from the few existing validated stimuli databases, e.g. the Emotional Movie Database (Carvalho et al. 2012), the Annotated Creative Commons Emotional Database (Baveye et al. 2015) or the FilmStim (Schaefer et al. 2010).

When sourcing suitable material from existing databases is not possible, the second strategy consists in selecting certain stimuli and carrying out a validation process. Validation is important to avoid choosing the clips according to researchers’ subjective impression of the type and intensity of the emotion. Validation tests should be completed by a large number of participants, at least 100, and one of the easiest ways to obtain a significant amount of responses is by carrying out an online survey. There are a number of free (Google Forms) and paid-for (Survey Monkey) tools that can be used for this purpose and allow videos with the required copyright to be embedded. There is also a variety of instruments that can be used for the validation of clips, such as the Differential Emotions Scale (Izard et al. 1974), the Positive and Negative Affect Schedule (Schaefer et al. 2010) or the Self-Assessment Manikin (SAM) (Bradley and Lang 1994). All these instruments include the self-reporting of emotional arousal by participants.

The number of clips used in the study will depend on the conditions tested in each experiment. Length of the clips is not usually an issue when triggering emotions, as they can be elicited in a very short time-span. A clear example is a standard television advertisement. Although research in AVT currently favours longer audiovisual excerpts to increase ecological validity, self-contained clips of between 13 seconds and 10 minutes have proven to be enough to elicit emotional activation in EDA research (Kreibig et al. 2007; Rottenberg et al. 2007; Rooney et al. 2012).

An important aspect to consider when selecting stimuli is to provide all participants with the same experimental conditions. For instance, to ensure that actors, settings, camerawork, colour, brightness, sound and music editing are similar, extracting the stimuli from a single source should be prioritised. Additionally, and depending on the nature of the study, other factors may need to be controlled. For example, when researching the influence of translation (subtitling, audio subtitling, voiceover), the researcher may want to select clips in which the original language is unknown to the participants. When working with persons with sight or hearing loss, the researcher may want to remove the visual or audio component to eliminate potential variables that can distort the outcome of the study. This manipulation of variables can have an impact on the ecological validity of the experiments. A choice must be made depending on the ultimate aim of each study and related hypotheses.  

Finally, an important issue when working with audiovisual stimuli is copyright. There are currently no clear guidelines regarding the use of copyrighted material in research, and legislation varies from country to country. Some argue that, based on the copyright exemption known in many countries as ‘right of quotation’, short excerpts of copyrighted material could be used in tests when access is restricted to researchers and participants. However, making the material open to the public (for instance, through an online survey) is much more problematic. In those cases, it is advisable to obtain permission from the copyright holder for both exploitation of the clips and provision of AVT and MA services as they are derivative work. In any case, as noted at the beginning of this section, there are some open access video repositories that can be used.

3.4. Psychophysiological devices 

Many devices are available on the market to carry out EDA research, including the two devices used by our research team: CAPTIV Neurolab, manufactured by TEA ERGO, and its corresponding software (v1.5), and Biosignalsplux Explorer. Both systems include hardware and software. Of the freeware available, one of the most popular is PsychoPy.

The two recording devices allow researchers to capture the phasic (transient) changes of EDA with respect to its basal activity, as well as the tonic responses induced by long-lasting stimuli (video fragments), which are measured in micro siemens (μs). In both cases, the four components of the electrodermal response are: latency (elapsed time between the start of the stimulation and the physiological response), elevation time (speed of the response until reaching its peak), its intensity, and average recovery time, which is the time that elapses until the response level returns to the middle of the response peak.

3.5. Procedural aspects of experimenting with EDA

When preparing a test with EDA, many contextual and procedural variables must be controlled. This can be achieved by standardising experimental conditions such as location, temperature and light conditions as well as image resolution, audio quality and volume. The experiment should be conducted in a closed and isolated room with artificial light and air conditioning, as these will guarantee stable conditions. Artificial light guarantees a more stable environment equal to all participants. It is also important to ensure that the distance between the screen and the participants remains constant. Temperature is particularly relevant since it impacts on sweating. According to Boucsein (1992), a room temperature of 23ºC is adequate. Regarding the audio features of the experiment, using the same headphones or an audio amplifier to keep a constant volume may help to reduce external noise and ensure that all participants are exposed to the same sound characteristics.

Finally, additional elements to control prior to the test are that none of the subjects has any cardiovascular condition, has recently experienced any kind of stressful situation (Miró et al. 2002), or has ingested any substance that may alter their physiological reactions, such as caffeine (Mikalsen et al. 2001) or alcohol (Naitoh 1972). It is therefore advisable to ask participants to avoid coffee or alcohol consumption in the four hours prior to the experiment (Sanz et al. 2006).

3.5.1. Data collection

The data acquisition procedure and its analysis vary depending on the device being used and on the type of measures to be gathered. In general, when working with EDA, the recorded reactions can be analysed as phasic or tonic (Braithwaite et al. 2015). While phasic reactions are immediate and generate a fast activation, tonic reactions involve slower activation. When working with short, intense stimuli such as images or sounds that last for a few seconds at most, or specific brief fragments of a longer stimulus (such as 4-5 second scenes in a video), the recording should be analysed as phasic. If the stimuli last for longer than a few seconds (such as a video or music) and the aim is to identify the general reaction to those stimuli, the subsequent analysis should be carried out based on a tonic response (Boucsein 2012).

Before the experiment, all sensors must be tested to ensure their proper functioning. Skin conductance is recorded using two electrodes. Recordings are typically taken from locations on the palm of the hands, with several acceptable placements. As shown in Figure 1, the most common electrode placements are the eminences of the palm, and the volar surface of the medial or distal phalanges of the fingers (Dawson et al. 2007).

Figure 1. Recommended placement of the EDA electrodes

The location on the skin to which the electrode is attached should not be cleaned with alcohol or abrasive substances (ibid.). Since moisture facilitates the flow of the electric current, humidification with a saline solution is advised. During the experiment, the subject should be relaxed and keep as still as possible since any movement will be registered and affect the recordings. Two key elements to consider when configuring the EDA record are the gain and the acquisition frequency. The latter can be defined as the number of samples recorded per second and it is considered adequate between 20 and 30 Hz (Boucsein et al. 2012). Gain measures the increase in power or amplitude of a signal and it is recommended that a signal gain of between 5 and 10 micromhos / cm is established (Braithwaite et al. 2015). These parameters are automatically set up by the system.

3.5.2. Data analysis

When the aim is to observe the reactivity of the EDA caused by a phasic emotional stimulation, such as an image or a short sound, the statistical processing of the signal usually consists of the calculation of the difference between the peak of conductance produced by the stimulation and its baseline level before the stimulus (Norris et al. 2007). However, when evaluating the emotional experience induced by tonic stimuli, given the amount of data to process and the presence of several conductance peaks that could distort the mean values, it is necessary to use other procedures to obtain the average response in each measurement phase, such as the calculation of the integral of the electrodermal activity of each period. The integral can then be compared to the obtained baseline (Bach et al. 2010). Some psychophysiological devices such as Captiv include their own integrated software for data analysis, offering an easy interface for research. As highlighted by Boucsein et al. (2012), difficulties might arise from data quality caused by a number of factors such as technical failure of the instruments (mechanical pressure on the electrodes, loose electrodes, electrode deterioration, connectivity, flow gel and its reaction with the skin), participant’s behaviour (body movement, speech, irregular breathing) or external influences (ambient noise, light, temperature). Before proceeding to data analysis, it is recommended to review the record visually, so that portions containing artefacts can be excluded from analysis (Boucsein, 2012), and to filter the data through software that detect artefacts. Artefact removal process, even if semi-automatic, is a very time-consuming task and it requires knowledge of removal criteria, e.g. morphology of the EDA signal.

4. Combining EDA with other measures

Psychophysiological measures can be combined with other data to triangulate results. In theoretical approaches to emotions, the idea that different measures should converge for the same emotional state in emotional experience, physiology and behaviour, is commonplace but studies showing relevant correlations of measures are still scarce (Mauss and Robinson 2009). However, since there is no perfect solution, if more measures are used and they are better tailored to the needs of the experimental context and stimuli, a study has the potential of yielding more informative results (Larsen and Prizmic-Larsen 2006). Such combinations include both objective and subjective measures and three examples are provided in the following pages to illustrate possible combinations of methods.

EDA has been combined with HR measurements to assess emotional sensitivity to music and paintings (Baumgartner et al. 2006; Jäncke 2006), to evaluate children’s emotional sensitivity (Sohn et al. 2001), and to assess emotional reactivity in coma patients (Daltrozzo et al. 2010). They have been also combined in various studies with different audiovisual stimuli, such as immersive environments (Egan et al. 2016), cinema (Rooney et al. 2012) and audio described and audio subtitled contents (Fryer 2013; Iturregui-Gallardo et al. 2018). The combination of EDA and HR is a common one, as the tools are relatively accessible and non-invasive for the participant. Furthermore, Kettunen et al. (1998) have demonstrated that there is synchronisation between EDA and HR and that both measures correlate with self-reported measures. It is important to remember that there is no gold standard for the measurement of emotional arousal (Mauss and Robinson 2009: 14). Therefore, the more measures used for obtaining data, and the better these suit the characteristics of the study, the more is to be learnt from user reactions (Larsen and Prizmic-Larsen 2006).

EDA can also be combined with facial electromyography (fEMG), a technique used to measure muscle activity. It is believed to be a sound method for measuring emotional response (Van Boxtel 2010) and has been used to measure emotional reaction to visual stimuli (Bhandari et al. 2017). EDA and fEMG are often combined in psychophysiological experimental design (Li et al. 2016; Lindsey et al. 2011). This is because while EDA allows the measurement of the strength of the participants’ reaction to a given stimulus and some EDA patterns seem to correlate with different emotions, fEMG enables better determination of  which emotion they are feeling (Thompson et al. 2016). fEMG has previously been used on visually impaired participants (Tosello et al. 2008), but there is no evidence of it being used in AVT or MA, except for some informal observations of facial expressions in a pilot study by O’Hagan (2016).

EDA can also be combined with questionnaires and interviews to tease out more information from participants. One of the possible options is the SAM questionnaire (Bradley and Lang 1994), which uses a vivid pictorial representation and is one of the gold standard tools to assess emotional elicitation, with a high number of citations since its creation in the 1980s (Betella and Verschure 2016). The SAM questionnaire allows for the measurement of emotional valence and arousal in a simple and fast way and has been validated in previous research.

In media accessibility research, EDA has been combined with other psychophysiological measures such as HR and questionnaires measuring presence (Fryer, 2013). When using self-reporting instruments with participants with specific needs, these should be taken into account and adapted if necessary, as has been done by scholars like Betella and Verschure (2016) to make them more operative with modern technology. Another good example is the adaptation at the Universitat Autònoma de Barcelona of the SAM questionnaire into a simplified and tactile form, the T-SAM, shown in Figure 2, (Iturregui-Gallardo and Méndez-Ulrich forthcoming), and discussed on https://www.youtube.com/​watch?​v=mcA_s4sVpaA&t=2s:

Figure 2. T-SAM (Tactile-SAM)

The use of the SAM questionnaire is recommended because it is an efficient tool for the assessment of the main dimensions of emotions while being short and simple. It has also been proven to minimise the possible bias provoked by culture or subjective interpretation of the item, because it is made up of non-semantic scales (Bradley et al. 2001a; Bradley et al. 2001b; McManis et al. 2001).

5. Conclusions

AVT and MA researchers are more and more interested in experimental approaches to research, in which objective data obtained through different tools contribute to the better understanding of user experiences. Deploying new perspectives and using new tools might be challenging, but taking a multidisciplinary approach and collaborating with other specialists can also be rewarding. In this regard, AVT and MA researchers have now some considerable experience using instruments such as eye-trackers, but other biometric sensors such as EDA are seldom used, despite their potential. EDA is affordable, non-invasive, quick, and provides objective data on emotional activation. Its use does not come without challenges, though. Conducting research with EDA requires strong interdisciplinary collaboration as analyses can be complex. The experimental setting must be carefully controlled (room temperature, previous stressful situations and alcohol/caffeine consumption), and the stimuli accurately chosen.

 

This article has discussed the main issues to be borne in mind when using EDA measures, in an attempt to offer orientation for research design and to highlight the main methodological challenges, possibilities and limitations that this research perspective can pose. In this regard, the potential of EDA to measure emotional activation in users, often in combination with other tools, has been highlighted. Creators of audiovisual materials want the audience to understand their content, engage with it and feel certain emotions and, in this respect, EDA can assess in an objective manner whether users feel certain emotions when faced with different transfer modes, access services and translation strategies. These results can then be triangulated with other subjective data in order to gain a more precise picture of the effect on users.

Acknowledgements

This research is part of the NEA (Nuevos Enfoques sobre Accesibilidad / New Approaches to Accessibility) project (FFI2015-64028-P, MINECO/FEDER, UE). It is also linked to TransMedia Catalonia, a research group funded by the Catalan government (2017SGR113).

References

• ALLEA (2017). The European Code of Conduct for Research Integrity. Berlin: All European Academies.
• Bach, Dominik, Karl Friston and Raymond Dolan (2010). “Analytic measures for quantification of arousal from spontaneous skin conductance fluctuations.” International Journal of Psychophysiology 76(1), 52-55.
• Bayley, Rachel (2017). “Electrodermal Activity (EDA).” The International Encyclopedia of Communication Research Methods. John Wiley and Sons, https://doi.org/10.1002/​9781118901731.iecrm0079.
• Baumgartner, Thomas, Michaela Esslen and Lutz Jäncke (2006). “From emotion perception to emotion experience: Emotions evoked by pictures and classical music.” International Journal of Psychophysiology 60(1), 34-43.
• Baveye, Yoann, Emmanuel Dellandréa, Christel Chamaret and Liming Chen (2015). “LIRIS-ACCEDE: A video database for affective content analysis.” IEEE Transactions on Affective Computing 6(1), 43-55.
• Betella, Alberto and Paul Verschure (2016). “The affective slider: A digital self-assessment scale for the measurement of human emotions.” PLOS ONE 11(2).
• Bhandari, Upasna, Neben Tillmann, Klarissa Chang and Wen Yong Chua (2017). “Effects of interface design factors on affective responses and quality evaluations in mobile applications.” Computers in Human Behavior 72(July), 525-534.
• Boucsein, Wolfram (1992). Electrodermal Activity. New York: Plenum Press.
• Boucsein, Wolfram (2012). Electrodermal Activity. 2nd Edition. New York: Springer Science & Business Media.
• Boucsein, Wolfram, Don Fowles, Sverre Grimnes, Gershon Ben-Shakhar, Walton Roth, Michael Dawson and Diane Filion (2012). “Publication recommendations for electrodermal measurements.” Psychophysiology 49(8), 1017-1034.
• Bradley, Margaret and Peter Lang (1994). “Measuring emotion: The self-assessment semantic differential manikin.” Journal of Behavior Therapy and Experimental Psychiatry 25(1), 49-59.
• Bradley, Margaret and Peter Lang (1999). International Affective Digitized Sounds (IADS): Stimuli, Instruction Manual and Affective Ratings. Technical Report B-2. Gainesville: University of Florida.
• Bradley, Margaret, Maurizio Codispoti, Dean Sabatinelli and Peter Lang (2001a). “Emotion and motivation I: Defensive and appetitive reactions in picture processing.” Emotion 1(3), 276–298.
• Bradley, Margaret, Maurizio Codispoti, Bruce Cuthbert and Peter Lang (2001b). “Emotion and motivation II: Sex differences in picture processing.” Emotion 1(3), 300-319.
• Braithwaite, Jason, Derrick Watson, Robert Jones and Mickey Rowe (2015). A Guide for Analyzing Electrodermal Activity (EDA) & Skin Conductance Responses (SCRs) for Psychological Experiments (Revised version: 2.0). Birmingham: Behavioural Brain Sciences Centre.
• Brumbaugh, Claudia Chloe, Ravi Kothuri, Carl Marci, Caleb Siefert and Donald Pfaff (2013). “Physiological correlates of the big 5: Autonomic responses to video presentations.” Applied Psychophysiology and Biofeedback 38(4), 293-301.
• Carvalho, Sandra, Jorge Leite, Santiago Galdo-Álvarez and Oscar F. Gonçalves (2012). “The emotional movie database (EMDB): A self-report and psychophysiological study.” Applied Psychophysiology and Biofeedback 37(4), 279-294.
• Carver, Charles (2006). “Approach, avoidance, and the self-regulation of affect and action.” Motivation and Emotion 30(2), 105-110.
• Chmiel, Agnieszka, Agnieszka Lijewska, Agnieszka Szarkowska and Łukasz Dutka (2017). “Paraphrasing in respeaking – comparing linguistic competence of interpreters, translators and bilinguals.” Perspectives: Studies in Translation Theory and Practice 26(5), 1-20.
• Codispoti, Maurizio, Paola Surcinelli and Bruno Baldaro (2008). “Watching emotional movies: Affective reactions and gender differences.” International Journal of Psychophysiology 69(2), 90-95.
• Daltrozzo, Jérôme, Norma Wioland, Véronique Mutschler, Philippe Lutun, Bartholomeus Calon, Alain Meyer, Albert Jaeger, Thierry Pottecher and Boris Kotchoubey (2010). “Emotional electrodermal response in coma and other low-responsive patients.” Neuroscience Letters 475(1), 44-47.
• Das, Pratik and Sukumar Roy (2016). “Designing of a low cost GSR acquisition system and estimation of stress level from the acquired signal.” International Journal for Research in Applied Science & Engineering Technology 4(10), 131-136.
• Dawson, Michael, Anne Schell and Diane Filion (2007). “The electrodermal system.” John T. Cacioppo, Louis G. Tassinary and Gary G. Bernston (eds) Handbook of Psychophysiologyi. 3rd Edition. Cambridge: Cambridge University Press, 159-181.
• Edelberg, Robert (1972). “Electrodermal recovery rate, goal‐orientation, and aversion.” Psychophysiology 9(5), 512-520.
• Egan, Darragh, Sean Brennan, John Barrett, Yuansong Qiao, Christian Timmerer and Niall Murray (2016). “An evaluation of heart rate and electrodermal activity as an objective QoE evaluation method for immersive virtual reality environments.” 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX), Lisbon, 6-8 June, 1-6.
• Ekman, Paul (1984). “Expression and the nature of emotion.” Klaus Rainer Scherer and Paul Ekman (eds). Approaches to Emotion. Hillsdale: Lawrence Erlbaum Associates, 319-344.
• Ekman, Paul (1992). “Are there basic emotions?” Psychological Review 99(3), 550-553.
• Fontaine, Johnny, Klaus Reiner Scherer, Etienne Roesch and Phoebe Ellsworth (2007). “The world of emotions is not two-dimensional.” Psychological Science 18(12), 1050-1057.
• Fryer, Louise (2013). Putting It into Words: The Impact of Visual Impairment on Perception, Experience and Presence. PhD Thesis. London: University of London.
• Fryer, Louise and Jonathan Freeman (2014). “Can you feel what I’m saying? The impact of verbal information on emotion elicitation and presence in people with a visual impairment.” Proceedings of the International Society for Presence Research 2014, Vienna, 17-19 March, 99-107.
• Harmon-Jones, Eddie, Cindy Harmon-Jones and Elizabeth Summerell (2017). “On the importance of both dimensional and discrete models of emotion.” Behavioral Sciences 7(4), 1-16.
• Iturregui-Gallardo, Gonzalo, Anna Matamala and Olga Soler-Vilageliu (2018). “Audio subtitling: Measuring emotional arousal in users through psychophysiological measures.” Paper delivered at Languages & the Media,Berlin, 3-5 October. https://ddd.uab.cat/record/196323?ln=ca (consulted 2.02.2019).
• Iturregui-Gallardo, Gonzalo and Jorge Luis Méndez-Ulrich (forthcoming). “Towards the creation of a tactile version of the Self-Assessment Manikin (T-SAM) for the emotional assessment of visually impaired people.” International Journal of Disability, Development and Education.
• Izard, Carroll E. (1977). Human Emotions. New York: Plenum.
• Izard, Carroll E. (1979). Differential Emotions Scale. Unpublished manuscript. University of Delaware.
• Jacobs, Barry and Dennis McGinty (1972). “Participation of the amygdala in complex stimulus recognition and behavioral inhibition: Evidence from unit studies.” Brain Research 36(2), 431-436.
• Jerath, Ravinder, Molly Crawford, Vernon Barnes and Kyler Harden (2015). “Self-regulation of breathing as a primary treatment for anxiety.” Applied Psychophysiology and Biofeedback 40(2), 107-115.
• Kettunen, Joni, Niklas Ravaja, Petri Näätänen, Pertti Keskivaara and Liisa Keltikangas-Järvinen (1998). “The synchronization of electrodermal activity and heart rate and its relationship to energetic arousal: A time series approach.” Biological Psychology 48(3), 209-225.
• Kleinginna, Paul and Anne Kleinginna (1981). “A categorized list of emotion definitions, with suggestions for a consensual definition.” Motivation and Emotion 5(4), 345-379.
• Kreibig, Sylvia (2010). “Autonomic nervous system activity in emotion: A review.” Biological Psychology 84(3), 394–421.
• Kreibig, Sylvia, Frank Wilhelm, Walton Roth and James Gross (2007). “Cardiovascular, electrodermal, and respiratory response patterns to fear-and sadness-inducing films.” Psychophysiology 44(5), 787-806.
• Krejtz, Izabela, Agnieszka Szarkowska and Krzysztof Krejtz (2013). “The effects of shot changes on eye movements in subtitling.” Journal of Eye Movement Research 6(5), 1–12.
• Kruger, Jan-Louis and Faans Steyn (2014). “Subtitles and eye tracking: Reading and performance.” Reading Research Quarterly 49(1), 105–120.
• Kruger, Jan-Louis, María Teresa Soto-Sanfiel, Stephen Doherty and Ronny Ibrahim (2016). “Towards a cognitive audiovisual translatology.” Ricardo Muñoz Martín (ed.). Reembedding Translation Process Research. Amsterdam: John Benjamins, 171-194.
• Landis, Carney and William Hunt (1935). “The conscious correlates of the galvanic skin response.” Journal of Experimental Psychology 18(5), 505–529.
• Lang, Peter (1988). “What are the data of emotion?” Vernon Hamilton, Gordon H. Bower and Nico H. Frijda (eds), Cognitive Perspectives on Emotion and Motivation. Dordrecht: Springer, 173-191.
• Lang, Peter (1995). “The emotion probe: Studies of motivation and attention.” American Psychologist 50(5), 372–385.
• Larsen, Randy J. and Zvjezdana Prizmic-Larsen (2006). “Measuring emotions: Implications of a multimethod perspective.” Michael Eid and Ed Diener (eds). Handbook of Multimethod Measurement in Psychology. Washington: American Psychological Association, 337-351.
• Lewis, Michael (1995). “Self-conscious emotions.” American Scientist 83(1), 68-78.
• Li, Shanshi, Gabby Walters, Jan Packer and Noel Scott (2016). “Using skin conductance and facial electromyography to measure emotional responses to tourism advertising.” Current Issues in Tourism 21(15), 1761-1783.
• Lindsey, Kathryn, Kelly Rohan, Kathryn Roecklein and Jennifer Mahon (2011). “Surface facial electromyography, skin conductance, and self-reported emotional responses to light- and season-relevant stimuli in seasonal affective disorder.” Journal of Affective Disorders 133(1-2), 311-319.
• Mauss, Iris and Michael Robinson (2009). “Measures of emotion: A review.” Cognition & Emotion 23(2), 209-237.
• McManis, Mark, Margaret Bradley, Keith Berg, Bruce Cuthbert and Peter Lang (2001). “Emotional reactions in children:Verbal, physiological, and behavioral responses to affective pictures.” Psychophysiology 38(2), 222-231.
• Mikalsen, Anita, Bard Bertelsen and Magne Flaten (2001). “Effects of caffeine, caffeine-associated stimuli, and caffeine-related information on physiological and psychological arousal.” Psychopharmacology 157(4), 373-380.
• Miró, Elena M., Carmen Cano‐Lozano and Gualberto Buela‐Casal (2002). “Electrodermal activity during total sleep deprivation and its relationship with other activation and performance measures.” Journal of Sleep Research 11(2), 105-112.
• Naitoh, Paul (1972). “The effect of alcohol on the autonomic nervous system of humans: Psychophysiological approach.” Benjamin Kissin and Henri Begleiter (eds). The Biology of Alcoholism. Boston: Springer, 367-433.
• Norris, Catherine, Jeff Larsen and John Cacioppo (2007). “Neuroticism is associated with larger and more prolonged electrodermal responses to emotionally evocative pictures.” Psychophysiology 44(5), 823-826.
• O’Hagan, Minako (2016). “Game localisation as emotion engineering methodological exploration.” Qi Zhang and Minako O’Hagan (eds). Conflict and Communication: A Changing Asia in a Globalising World. New York: Nova, 123-144.
• Orero, Pilar and Anna Vilaró (2012). “Eye tracking analysis of minor details in films for audio description.” MOnTI, Monographs in Translation and Interpreting 4, 295-319.
• Orero, Pilar, Stephen Doherty, Jan-Louis Kruger, Anna Matamala, Jan Pedersen, Elisa Perego, Pablo Romero-Fresco, Sara Rovira-Esteva, Olga Soler-Vilageliu and Agnieszka Szarkowska (2018). “Conducting experimental research in audiovisual translation (AVT): A position paper.” The Journal of Specialised Translation 30, 105-126.
• Phillips, Russell and Joseph LeDoux (1992). “Differential contribution of amygdala and hippocampus to cued and contextual fear conditioning.” Behavioral Neuroscience 106(2), 274-285.
• Plutchik, Robert (1994). The Psychology and Biology of Emotion. New York: Harper Collins.
• Ramos, Marina (2015). “The emotional experience of films: Does audio description make a difference?” The Translator 21(1), 68–94.
• Rojo, Ana, Marina Ramos and Javier Valenzuela (2014). “The emotional impact of translation: A heart rate study.” Journal of Pragmatics 71, 31-44.
• Rooney, Brendan, Ciarán Benson and Eilis Hennessy (2012). “The apparent reality of movies and emotional arousal: A study using physiological and self-report measures.” Poetics 40(5), 405-422.
• Rottenberg, Jonathan, Rebecca D. Ray and James J. Gross (2007). “Emotion elicitation using films.” James A. Coan and John J. B. Allen (eds) Handbook of Emotion Elicitation and Assessment. New York: Oxford University Press, 9-28.
• Russell, James (1980). “A circumplex model of affect.” Journal of Personality and Social Psychology 39(6), 1161-1178.
• Russell, James (1983). “Pancultural aspects of the human conceptual organization of emotions.” Journal of Personality and Social Psychology 45(6), 1281-1288.
• Sanz, Antonio, Francisco Villamarín, María Álvarez-Moleiro and Joaquín Limonero (2006). “Microanalysis of the relationship between goal self-efficacy and cardiovascular reactivity. A test for the moderating role of incentive value and the mediating role of anxiety.” International Journal of Psychophysiology 62(1), 66-76.
• Schaefer, Alexandre, Frédéric Nils, Xavier Sanchez and Pierre Philippot (2010). “Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers.” Cognition & Emotion 24(7), 1153-1172.
• Szarkowska, Agnieszka, Izabela Krejtz, Zuzanna Kłyszejko and Anna Wieczorek (2011). “Verbatim, standard, or edited? Reading patterns of different captioning styles among deaf, hard of hearing, and hearing viewers.” American Annals of the Deaf 156(4), 363-378.
• Szarkowska, Agnieszka, Krzysztof Krejtz, Lukasz Dutka and Olga Pilipczuk (2016). “Cognitive load in intralingual and interlingual respeaking – a preliminary study.” Poznań Studies in Contemporary Linguistics 52(2), 209-233.
• Sohn, Jin Hun, Estate Sokhadze and Shigeki Watanuki (2001). “Electrodermal and cardiovascular manifestations of emotions in children.” Journal of Physiological Anthropology and Applied Human Science 20(2), 55-64.
• Taylor, Shelley (1991). “Asymmetrical effects of positive and negative events: The mobilization-minimization hypothesis.” Psychological Bulletin 110(1), 67–85.
• Thompson, Dominic, Ian Mackenzie, Hartmut Leuthold and Ruth Filik (2016). “Emotional responses to irony and emoticons in written language: Evidence from EDA and facial EMG.” Psychophysiology 53(7), 1054-1062.
• Tosello, Darcy de Oliveira, Matthias Vitti, Selma Siéssere, Carla Santos, Edson Donizetti Verri, Jaime Eduardo Hallak and Simone Regalo (2008). “Effect of congenital blindness on EMG activity of the facial muscles.” Electromyography Clinical Neurophysiology 48(2), 117-124.
• Uhrig, Meike K., Nadine Trautmann, Ulf Baumgärtner, Rolf-Detlef Treede, Florian Henrich, Wolfgang Hiller and Susanne Marschall (2016). “Emotion elicitation: A comparison of pictures and films.” Frontiers in Psychology 7, 1-12.
• van Boxtel, Anton (2010). “Facial EMG as a tool for inferring affective states.” Proceedings of the Measuring Behavior. Noldus Information Technology Wageningen, Eindhoven, 24–27 August, 104-108.
• Walczak, Agnieszka and Louise Fryer (2017). “Creative description: The impact of audio description style on presence in visually impaired audiences.” British Journal of Visual Impairment 35(1), 6-17.
• Walczak, Agnieszka and Louise Fryer (2018). “Vocal delivery of audio description by genre: measuring users’ presence.” Perspectives: Studies in Translation Theory and Practice 26(1), 69-83.
• Watson, David and Auke Tellegen (1985). “Toward a consensual structure of mood.” Psychological Bulletin 98(2), 219-235.
• Westermann, Rainer, Kordelia Spies, Günter Stahl and Friedrich Hesse (1996). “Relative effectiveness and validity of mood induction procedures: A meta-analysis.” European Journal of Social Psychology 26(4), 557-580.

Biographies

Anna Matamala, associate professor at Universitat Autònoma de Barcelona, is the main researcher of TransMedia Catalonia and actively participates in media accessibility projects.

More information: gent.uab.cat/amatamala

Email: anna.matamala@uab.cat

 

 

 

 

Olga Soler-Vilageliu, associate professor at Universitat Autònoma de Barcelona. With a background in language processing, her research has focussed on online measuring of handwriting in preschool and school children, and she participates in international networks working on literacy (COST IS1401). Within TransMedia Catalonia, she is currently setting up experiments on different accessibility services.

Email: olga.soler@uab.cat

 

 

Gonzalo Iturregui-Gallardo, PhD researcher, holds an MA in Translation Studies from Leicester University, His current research is on audio subtitling and its impact on end users. He has worked as a translator specialising in audiovisual and legal texts and as an audio describer for opera and theatre.

Email: gonzalo.iturregui@uab.cat

 

 

Anna Jankowska, PhD, assistant researcher and lecturer at the Chair for Translation Studies and Intercultural Communication, Jagiellonian University, Krakow, Poland. Post-doctoral researcher, 2016-2019, at TransMedia Catalonia, Universitat Autònoma de Barcelona. Co-founder and president of the Seventh Sense Foundation that provides access services.

More information: www.annajankowska.eu

Email: anna.jankowska@uab.cat

 

 

>Jorge Luis Méndez-Ulrich, PhD in Health Psychology and MSc in Research in Health Psychology. He is a member of the Research Group on Stress and Health and assistant professor at the Department of Basic, Developmental and Educational Psychology at the Faculty of Psychology, Universitat Autònoma de Barcelona.

Email: jorgeluis.mendez@uab.cat

 

 

 

Anna Serrano has a BA in Psychology and an MA in Neuroscience by the Universitat Autònoma de Barcelona. She joined TransMedia Catalonia for her internship and is working on the psychological and experimental part of various projects related to audio description, audio subtitling and the emotional response from the viewers.

Email: anna.serranora@e-campus.uab.cat