From Brows to Trust - Evaluating Embodied Conversational Agents

2. About Brows (p.193-194)

In a seminal paper, Ekman (1979) describes the role of eyebrow movements as emotional and conversational signals. Sometimes the distinction between these two kinds of signals is di.cult to make (for instance because both often occur during conversation). Still clear di.erences between the two exist: conversational signals typically do not occur when a person believes (s)he is unobserved, while emotional signals do. Moreover, emotional but not conversational signals are believed to be universal.

While the use of eyebrows as emotional signals has been addressed in many studies (already in Darwin (1872)), the conversational use is still relatively understudied and most of the work that has been done in this area is based on intuitions and impressionistic observations. This is surprising, since eyebrow movements are according to Ekman (1979:183) "probably among the most frequent facial actions employed as conversational signals". Various authors have suggested that eyebrow movements can be used to emphasize important pieces of information (see e.g., Birdwhistell (1970); Eibl-Eibesfeldt (1972); Condon (1976); Ekman (1979)). Ekman observes that eyebrows can play this accentuation role in two di.erent ways: they can function as a baton (in the terminology of Efron (1941)), which may be used to accentuate a particular word as it is spoken, or they can function as an underliner (in Ekman’s own terminology), where the emphasis stretches out over more than one word.

It is well-known that speakers may use auditory speech signals to emphasize words as well. For instance, speakers of Germanic languages (such as Dutch, English and German) can use pitch accents to indicate the information status of words: accents tend to distinguish information that is in focus (since it is new or contrastive) from information which is given from the prior discourse context (see e.g., Chafe (1974); Terken (1984); Hirschberg (1993)). That both eyebrow movements and pitch accents can be used to signal focus, suggests that there is close correspondence between the two. This correspondence has indeed been noted by Morgan (1953) and Bolinger (1985:202.).

The latter formulated his Metaphor of Up and Down which implies, among other things, that when the pitch rises or falls, eyebrows tend to follow the same pattern. As an illustration of this metaphor, it is instructive to try and utter a two-word phrase, say "blue square," with a pitch accent (and no corresponding eyebrow movement) on the word "blue" and an eyebrow movement (but no pitch accent) on the word "square".

This implies that eyebrow movements often co-occur with pitch accents. It is important to realize that the opposite is not the case. Ekman (1979:184): "There are many occasions when people mark emphasis in their speech without either a baton or an underliner." People do more with their pitch than with their eyebrows, as the reader can easily verify by looking at an arbitrary speaker. If not all emphasized words are accompanied by an eyebrow movement, which words are? This is still an open question. Ekman (1979:184) is "not optimistic about being able to predict when a baton or underliner will be used and when emphasis will be carried just by voice, although perhaps there might be some weak relationship with overall involvement in what is said."