Applications for Affective Computing

Once the affective computing system has sensed the user's biosignals and recognized the patterns inherent in the signals, the system's Understanding module will assimilate the data into its model of the user's emotional experience. The system is now capable of communicating useful information about the user to applications that can use such data. What do affective computing applications look like? How would they differ from current software applications?

Perhaps the most fundamental application of affective computing will be to inform next-generation human interfaces that are able to recognize, and respond to, the emotional states of their users. Users who are becoming frustrated or annoyed with using a product would "send out signals" to the computer, at which point the application might respond in a variety of ways -- ideally in ways that the user would see as "intuitive". For example, a computer piano tutor might change its pace and presentation based upon naturally expressed signals that the user is interested, bored, confused, or frustrated. A video retrieval system might help identify not just scenes having a particular actor or setting, but scenes having a particular emotional content: fast-forward to the "most exciting" scenes. Your wearable computer could pay attention to things that increase your stress, so that it might help you learn better strategies to boost your immune system when needed, practicing preventive medicine.
Beyond this quantum leap in the ability of software applications to respond with greater sensitivity to the user, the advent of affective computing will immediately lend itself to a host of applications, a number of which are described below.

Current projects

AffQuake: AffQuake is an attempt to incorporate signals that relate to a player's affect into ID Software's Quake II in a way that alters game play. Several modifications have been made that cause the player's avatar within Quake to alter its behaviors depending upon one of these signals. In StartleQuake, when a player becomes startled, his or her avatar also becomes startled and jumps back. Quake changes the size of the player's avatar in relation to the user's response as well, representing player excitement by average skin conductivity level, and growing the avatar's size when this level is high.
Affective Jewelry and Accessories: Wearable jewelry and other clothing designs with embedded sensors for sensing physiological changes associated with emotions.
Affective Tangibles: People naturally express frustration through the use of their motor skills. The purpose of the Affective Tangibles project is to develop physical objects that can be grasped, squeezed, thrown, or otherwise manipulated via a nat ural display of affect. Current tangibles include a squeeze mouse, affective pinwheels that are mapped to skin conductivity, and a voodoo doll that can be shaken to express frustration. All of these physical manipulations are converted to bits via a new i nterface we have designed for facilitating such forms of communication.
Affective Touchables: Physical objects that sense affective parameters through being held or touched, and communicate the emotions abstractly through sight, sound, or haptic changes
Affective Tutor—the Learning Companion: We are interested in constructing an agent that senses affective states like boredom, anxiety, and engagement, and adjusts its response to the user in accord with the user's state. This would be aimed at learning situations where the agent acts as a kind of mentor, occasionally supporting the user in his or her otherwise self-propelled exploration. It also provides an opportunity for learning about the role of human emotions expressed during a learning situation.
The Galvactivator: A wearable device which maps your skin conductivity to a glowing red LED. Set the baseline, and then use this as a device to learn about and communicate your body's response.
Interface Tailor : The Interface Tailor is an agent that attempts to adapt the system in response to affective feedback. Frustration is being used as a fitness function to select between a wide variety of different system behaviors. Currently, the Microsoft Office Assistant (or Paperclip) is one example interface that is being made more adaptive. Ultimately the project seeks to provide a generalized framework for making all software more tailor-able.
Learning and pattern recognition: Computers can potentially learn patterns of behavior (physiological and otherwise) that depend on the user (personality, goals, preferences, etc.) and on his or her situation (work, commute, too much caffeine, etc.). We conduct basic research in Bayesian learning theory, pattern recognition, and other aspects of modeling that form the crucial underpinnings of our efforts to build robust, flexible, and adaptive systems.
Mood Interfaces: This project explores graphical interfaces in which physiological signals drive the visual display, allowing users or their conversational partners to engage in a computer-mediated dialogue of emotional expression, viewing a graphical representation of expressions of their current physical/emotional state.
Robotic Computer: A personal computer with an active, robotic, monitor, which expresses and responds to social-emotional cues.

Want to email us? For answers to frequently asked questions and for our email address, please visit the FAQ and email page. Thanks!