People Projects Publications Software Data Policy Press Videos FAQ Vintage

Affective Communication

Affective communication is communicating with someone (or something) either with or about affect. A crying child, and a parent comforting that child, are both engaged in affective communication. An angry customer complaining to a customer service representative, and that representative trying to clear up the problem are both also engaged in affective communication. We communicate through affective channels naturally every day. Indeed, most of us are experts in expressing, recognizing and dealing with emotions. However, affective communication that involves computers represents a vast but largely untapped research area. What role can computers play in affective communication? How can they assist us in putting emotional channels back into "lossy" communications technologies, such as email and online chat? How can computer technology support us in getting to know our own bodies, and our own emotions? What role, if any, can computers play in helping manage frustration, especially frustration that arises from using technology?

We are beginning to investigate several key aspects of Affective Communication as it relates to computers. Affective communication may involve giving computers the ability to Recognize Emotional Expressions as a step toward interpreting what the user might be feeling. However, the focus in this area is on communication that involves emotional expression. Expressions of emotion can be communicated to others without an intermediate step of recognition; they can simply be "transduced" into a form that is able to be digitally transmitted and re-presented at another location. Several devices are being investigated for facilitating this, under the name of Affective Mediation -- using computers to help communicate emotions to other people through various media. Affective Mediation is a primary research focus of Jocelyn Scheirer. It can potentially be used to enable improved "Human-to-Human Communication". We also are interested in use of affective technology to help people see themselves better ("Human-to-Self [Reflexive] Communication"). It is also sometimes appropriate for technology, e.g., a software agent, to have a means of ("Expressing emotion") to users or to other agents or devices. This can be the case whether or not the technology actually "has" any mechanisms of emotion. A number of research projects investigating affective communication are underway, and may be found at the bottom of this page.

Affective Mediation

Technology supporting machine-mediated communication continues to grow and improve, but much of it still remains impoverished with respect to emotional expression. While much of the current research in the Affective Computing group focuses on sensing and understanding the emotional state of the user or the development of affective interfaces, research in Affective Mediation explores ways to increase the "affective bandwidth" of computer-mediated communication through the use of graphical visualization. By graphical visualization, we mean the representation of emotional information in an easy-to-understand, computer graphics format. Currently the focus is on physiological information, but it may also include behavioral information (such as if someone is typing louder or faster than usual.) Building on traditional representations of physiological signals -- continuously updating line graphs -- one approach is to represent the user's physiology in three-dimensional, real-time computer graphics, and to provide unique, innovative, and unobtrusive ways to collect the data. This research focuses on using displays and devices in ways that will help humans to communicate both with themselves and with one another in affect-enhanced ways.

Human-to-Human Communication

From email to full-body videoconferencing, virtual communication is growing rapidly in availablity and complexity. Although this richness of communication options improves our ability to converse with others who are far away or not available at the precise moment that we are, the sense that something is missing continues to plague users of current methodologies. Affective Communication seeks to provide new devices and tools for supplementing person-to-person communications media. Specifically, through the use of graphical displays viewable by any or all members of a mediated conversation, we hope to provide an augmented experience of affective expression which supplements but also challenges traditional computer-mediated communication.

Human-to-Self (Reflexive) Communication

Digitized representation of affective responses creates possibilities for our relationship to our own bodies and affective response patterns. Affective communication with oneself - reflexive communication - explores the exciting possibilities of giving people access to their own physiological patterns in ways previously unavailable, or available only to medical and research personnel with special, complex, or expensive equipment. Our graphical approach creates new technologies with the express goal of allowing the user to gain information and insight about his or her own responses. To accomplish this, we represent physiological data in a visualizable and intuitive manner.

Computer expression of emotion

This work represents a controversial area of human-computer interaction, in part since attributing emotions and emotional understanding to machines has been identified as a philosophical problem: what does it mean for a machine to express emotions that it doesn't feel? What does it mean for humans to feel "empathized with" by machines that are simply unable to really "feel" what a person is going through? Currently, few computer systems have been designed specifically to interact on an emotional level. Eliza, Clark Elliot's Affective Reasoner, and interactive pets such as the Tamagochi are all examples of systems built for affective human-computer communication. When the Tamagochi "cries" for attention, this is an example of computer expression of emotion. Another example is the smile that Macintosh users are greeted with, indicating that "all is well" with the boot disk. If there is a problem with the boot disk, the machine displays the "sad Mac".

Humans are experts at interpreting facial expressions and tones of voice, and making accurate inferences about others' internal states from these clues. Controversy rages over anthropomorphism: Should we leverage this expertise in the service of computer interface design, since attributing human characteristics to machines often means setting unrealistic and unfulfillable expectations about the machine's capabilities? Show a human face, expect human capabilities that far outstrip the machine?

Yet the fact remains that faces have been used effectively in media to represent a wide variety of internal states. And with careful design, we regard emotional expression via face and sound as a potentially effective means of communicating a wide array of information to computer users. As systems become more capable of emotional communication with users, we see systems needing more and more sophisticated emotionally-expressive capability.

Research projects in Affective Communication


[mit][media lab + Room E15-419 + 20 Ames Street + Cambridge, MA 02139]