Research on Sensing Human Affect
Sensors are an important part of an Affective Computing System
because they provide information about the wearer's physical state or
behavior. They can gather data in a continuous way without
having to interrupt the user. The emphasis on this page is on
describing physiological sensors; however, we are interested in many
kinds of new sensors that might be useful in recognizing affective
cues. Several projects in sensing affect are listed below.
A Prototype Physiological Sensing System had been developed for use on
a UNIX based operating system. The system shown below includes a
Galvanic Skin Response (GSR) Sensor, a Blood Volume Pulse (BVP)
sensor, a Respiration sensor and an Electromyogram (EMG). This system
was adopted for the following reasons: The sensors are available
through a commercial manufacturer who has obtained FDA approval for
use of these sensors on human subjects; the system is lightweight and
portable and relatively robust to changes in user position; the system
is entirely on the user which helps maintain the privacy of the
wearer's data. Each of the sensors is explained below and a sample of
the data taken from each sensor is shown for clarity.
prototypical sensing system shown here is a combination of four sensors: The
Blood Volume Pulse (BVP) sensor, the Galvanic Skin Response (GSR)
sensor, the Electromyogram (EMG) sensor, and the respiration
sensor. Each of these sensors is described in detail below. The
sensors all attach to the Thought Technology ProComp Encoder
Unit, a device that receives the signals and translates them into
digital form. The output from the ProComp unit then ports to a
computer for processing and recognition.
The Galvanic Skin Response (GSR) Sensor
Galvanic Skin Response is a measure of the skin's conductance between
two electrodes. Electrodes are small metal plates that apply a safe, imperceptibly
tiny voltage across the skin. The electrodes are typically attached to the
subject's fingers or toes using electrode cuffs (as shown on the left electrode
in the diagram) or to any part of the body using a silver-Chloride electrode
patch such as that shown on the EMG. To measure the resistance, a small
voltage is applied to the skin and the skin's current conduction is measured.
Skin conductance is considered to be a function of the sweat gland activity
and the skin's pore size. An individual's baseline skin conductance will
vary for many reasons, including gender, diet, skin type and situation. Sweat
gland activity is controlled in part by the sympathetic nervous system.
When a subject is startled or experiences anxiety, there will be a fast
increase in the skin's conductance (a period of seconds) due to increased
activity in the sweat glands (unless the glands are saturated with sweat.)
After a startle, the skin's conductance will decrease naturally due to reabsorption.
There is a saturation to the effect: when the duct of the sweat gland fills
there is no longer a possibility of further increasing skin conductance.
Excess sweat pours out of the duct. Sweat gland activity increases the skin's
capacity to conduct the current passing through it and changes in the skin
conductance reflect changes in the level of arousal in the sympathetic nervous
A graph of Galvanic Skin Response (GSR) skin conductance over a 27-minute
period during an experiment. Increased GSR indicates a heightened sympathetic
nervous system arousal.
The Blood Volume Pulse Sensor
The Blood Volume pulse sensor uses photoplethysmography to detect the
blood pressure in the extremities. Photoplethysmography is a process of
applying a light source and measuring the light reflected by the skin. At
each contraction of the heart, blood is forced through the peripheral vessels,
producing engorgement of the vessels under the light source--thereby modifying
the amount of light to the photosensor. The resulting pressure waveform
A graph of Blood Volume Pulse (BVP) sensor output showing a waveform
of human heart beats, measured over a 27-minute period during an experiment.
Note in this signal how the envelope (the overall shape of the waveform)
of the BVP "pinches" as the subject is startled in this diagram,
at around the 16-18 minute period.
Three typical heart beats, as measured in the fingertips by the Blood
Volume Pulse sensor.
Since vasomotor activity (activity which controls the size of the blood
vessels) is controlled by the sympathetic nervous system, the BVP measurements
can display changes in sympathetic arousal. An increase in the BVP amplitude
indicates decreased sympathetic arousal and greater blood flow to the fingertips.
The Respiration Sensor
The respiration sensor can be placed either over the sternum for
thoracic monitoring or over the diaphram for diaphragmatic monitoring. In
all experiments so far we have used diaphragmatic monitoring. The sensor
consists mainly of a large velcro belt which extends around the chest cavity
and a small elastic which stretches as the subject's chest cavity expands.
The amount of stretch in the elastic is measured as a voltage change and
recorded. From the waveform, the depth the subject's breath and the subject's
rate of respiration can be learned.
A typical respiration waveform reading over approximately 27 minutes.
The Electromyogram (EMG) Sensor
The electromyographic sensors measure the electromyographic activity
of the muscle (the electrical activity produced by a muscle when it is being
contracted), amplify the signal and send it to the encoder. In the encoder,
a band pass filter is applied to the signal. For all our experiments, the
sensor has used the 0-400 microvolt range and the 20-500 Hz filter, which
is the most commonly used position.
An electromyogram of jaw clenching during an experiment.
Projects in Sensing Affective Signals: