Emerging Trends in The Social and Behavioral Sciences

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

Media

Part of Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

Title

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

extracted text

Homuncular Flexibility: The Human
Ability to Inhabit Nonhuman Avatars
ANDREA STEVENSON WON, JEREMY N. BAILENSON, and JARON LANIER

Abstract
This essay seeks to explicate an unorthodox idea that spans psychology, neuroscience, psychology, philosophy, and computer science called homuncular flexibility
(HF). HF posits that the homunculus—the part of the cortex that maps movement
and sensing of body parts—is capable of adapting to novel bodies, in particular
bodies that have extra appendages or appendages capable of atypical movements.
Evidence demonstrates neural plasticity in nature; for example, amputees experience cortical shifting such that their face receives extra attention in the brain after
a limb is amputated. However, experiments such as the rubber hand illusion, in
which people respond to rubber hands placed near their arms as if they were their
actual hands, demonstrated that a person’s sense of their body can be adjusted to
include external objects. The recent advent of virtual reality technology, which can
track physical human motions and display them on avatars, allows for the wholly
new human experience of inhabiting distinctly nonhuman bodies. HF is a paradigm
in which physical motions are transformed by remapping degrees of freedom from
tracked movements onto an avatar. For example, if a human being were to inhabit
an avatar of a lobster, controlling two of the eight lobster arms would be simply be a
function of tracking the two physical arms and directly mapping those movements
onto the avatar’s first two arms. However, in order to control the remaining six
arms, degrees of freedom that are orthogonal to the movements of the first two
arms need to be measured and remapped. In this essay, we discuss advancements in
neuroscience, psychology, and computer science that relate to HF. We review some
preliminary studies that demonstrate how humans accommodate novel bodies.
Finally, we discuss theoretical implications and practical applications relating to HF.

HISTORY OF HOMUNCULAR FLEXIBILITY
Accounts of the history of experimentation are usually paced by examples
of phenomenon coming to light because of improved instrumentation.
homuncular flexibility (HF) is a rare example of the opposite; it was first
noticed because of poor instrumentation.
The 1980s saw the first attempts to implement networked immersive
virtual worlds (deemed “virtual reality”) at VPL Research. To be networked,
Emerging Trends in the Social and Behavioral Sciences. Edited by Robert Scott and Stephen Kosslyn.
© 2015 John Wiley & Sons, Inc. ISBN 978-1-118-90077-2.

1

2

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

users would experience being present in a shared virtual setting and would
be able to see representations of each other. Thus, it was necessary to create
three-dimensional avatars of each user. One of the strengths of the early
development system for virtual reality at VPL Research was that it supported
extremely rapid prototyping, with revisions appearing instantly while a
subject was “inside” a virtual world experiment. The calibration of early
full-body avatars presented a challenge; it was difficult to design a suit so
that sensors would remain in precisely the same locations on the user’s body
with extended use. In the context of rapid experimentation with mappings
and heuristics, it came to pass that nonrealistic, whole-body avatars were
occasionally activated. These would usually result in a complete breakdown
of usability. As an example, if an avatar’s head were made to jut out of the
side of the hip, the world would appear rotated awkwardly to the user, who
would immediately become disoriented, unable to perform any task. In the
course of exploring avatar designs, researchers occasionally came upon an
unusual avatar design that preserved usability despite being nonrealistic or
even bizarre.
The first one occurred during the process of creating an immersive city and
harbor planning tool, a collaboration between Jaron Lanier at VPL Research
and Tom Furness and others at the HITLab of the University of Washington. One of the scientists was inhabiting an avatar—a worker at the docks in
Seattle—when his arm was made very large, perhaps the size of a very large
crane. This most likely occurred because a designer entered extra zeros in a
scale factor in the software that measured movement. What was remarkable
was that the scientist was able to pick up vehicles and other objects in a large
scene at great distance with a highly distorted arm, and was able to do so
with accuracy and no apparent loss of usability.
This unexpected observation, and others similar to it, motivated an informal study of “weird avatars that were still usable.” A number of increasingly
strange, but usable, nonhuman avatars were created and tested between
approximately 1989 and 1991. Most of these resembled mammals, in terms
of overall structure and inventory of limbs.
A number of principles emerged via informal pilot testing. For example, it
appeared that the relative scaling of parts of the body was surprisingly plastic. The attachment points of limbs could be changed so long as the angle of
attachment did not change too much. However, the ultimate strange avatars
departed from the mammalian plan entirely. Most of these did not preserve
usability. However, there was one that bears mention. One colleague, Ann
Lasko, had seen a postcard of people in lobster suits at a festival, and created
a lobster avatar and set about programming a body map for it. As the lobster
body includes more limbs than a person, there were not enough parameters measured by the body suit to drive the lobster avatar in a one-to-one

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

3

map. Therefore, the scientist mapped the degrees of freedom of the body suit
to the greater number of degrees of freedom of the lobster. The initial mappings were not usable, but over time, as the algorithms to control the extra
limbs evolved, some mappings emerged that succeeded. Over time, and with
practice, humans were able to slowly learn to control the lobster. The biologist Jim Bower, when visiting Lanier’s lab during this period, commented
that the range of usable nonhuman avatars might be related to the phylogenetic tree. The human brain might be expected to find body plans that had
occurred in the history of its own descent to be more usable than other body
plans, though of course this would not include the lobster. Nonetheless, the
intriguing question remains of what makes certain nonhuman avatars usable
while others are not.
In this essay, we will first give a rough description of the brain structures
that might support flexibility of virtual embodiment. Then, we review some
landmark findings related to brain plasticity that provide foundations for
the current work on remapping physical bodies onto avatar bodies. Next,
we describe recent work in virtual reality that tests the ability of people to
accommodate various avatar bodies. Finally, we suggest two frameworks for
organizing this emerging area of research—one based on the overall function
of the transformation—for example, restoring function compared to inventing function, and the other based on the mode of the transformation.
THE HOMUNCULUS AND EXPERIMENTS IN NEURAL PLASTICITY
The “homunculus” in the term HF refers to motor and somatosensory representations of the human body in the brain. Both the primary motor cortex
and the primary somatosensory cortex contain areas where the innervation
arising from different parts of the body roughly maps to corresponding cortical regions. These areas form Penfield’s “homunculus” (Penfield & Boldrey,
1937). As with many brain structures, the left side of the body is represented
in the right cerebral hemisphere, and vice versa. The primary motor cortex,
which relates to muscle movement, is located on the frontal side of the central
gyrus, whereas the primary sensory cortex, which is involved in the perceptions of touch sensation, opposes it. A schematic approximating the relationships of these two structures is shown in Figure 1.
Areas of the body with greater innervation are represented by more space
in the primary motor cortex than body parts with less innervation. Thus,
the hands, which have complex movements/muscular innervations, take up
more space than the leg. This mapping is not currently considered to be
one-to-one; it has been proposed that rather than specific cortical regions
mapping to specific muscles, the motor cortex controls the motions made by
these body regions (Barinaga, 1995). However, it does provide for a general

4

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

Cross-section of right hemisphere
Somatosensory cortex
Hand

Tongue

Post-central gyrus

Motor cortex
Hand

Tongue

Pre-central gyrus

Figure 1 Homunculi of the somatosensory and motor cortices, showing the
differences in size mapping for the hand and tongue in the somatosensory and
motor cortices, reflecting their comparative differences in innervation.

relationship between brain regions and the movements of different regions of
the body. The primary somatosensory cortex is similarly arranged, such that
parts of the body with greater sensitivity, such as the lips, fingers, and thumb,
take up more space than, for example, the back. In contrast to the motor cortex, the hands are less dominant in the sensory cortex, whereas the sensitive
(but immobile) teeth and gums are well present here but not in the motor
cortex.
The mappings of the somatosensory and motor cortex may be altered by
injury. For example, in the case of an amputated arm, cortical mapping may
shift such that areas that previously innervated the hand may shift to another
part of the body (Roricht, Meyer, Niehaus, & Brandt, 1999). This cortical plasticity is hypothesized to be one source of phantom limb pain.
Phantom limb pain is the phenomenon in which an amputee continues
to have sensation, sometimes very painful, perceived as originating in the
absent limb long after the trauma has healed. This persistent pain has been
proposed to result from cortical remapping. In 1996, Ramachandran and
Rogers-Ramachandran proposed that altering the visual input that patients
received could help in remapping the motor cortex and thus alleviate
phantom limb pain. To do this, they designed a box with a mirror in the
midline. When patients placed their uninjured limb in such a box, it created

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

5

the illusion of two normally moving limbs. This appeared to relieve some
patients’ pain, perhaps because the pathological cortical map was restored
by the visual input of a plausible limb in the location of the amputated limb.
Similarly, Lotze, Flor, Grodd, Larbig, and Birbaumer (2001) have proposed
that the use of a myoelectric prosthesis providing similar visual feedback
may prevent cortical reorganization and the accompanying phantom limb
pain. Therapies building on mirror visual therapy have also been used
to treat complex regional pain syndrome (CRPS), a condition with some
similarities to phantom limb pain (Sato et al., 2010).
Visual feedback combined with somatosensory input has also been proposed to change a person’s understanding of his or her body. Stroking a
prosthesis as a substitute for a missing limb has been proposed as a therapy
for phantom limb pain (Ramachandran & Rogers-Ramachandran, 1996)
and CRPS (Schmalzl, Ragnö, & Ehrsson, 2013). Creating the illusion of
touch has also been suggested for pain syndromes in the face that would
not respond well to movement-based mirror therapies (Ramachandran
& Altschuler, 2009; Won & Collins, 2012). These therapies build upon a
phenomenon of body schema transfer often linked to phantom limb pain,
known as the rubber hand illusion (Botvinick & Cohen, 1998). In this illusion,
a participant’s hand is placed underneath a cloth or a table on which a
rubber arm is laid. The participant’s real hand and the rubber hand are then
stroked at the same time with a soft brush. Gradually, the patient feels a
sense of increasing ownership toward the rubber hand, and he or she will
flinch if the rubber hand is suddenly hit with a mallet. IJsselsteijn, de Kort,
and Haans (2006) replicated the illusion in the physical world, in virtual
reality, and in mixed reality, finding effects across conditions, although the
illusion was strongest in the unmediated condition. A similar experiment
using touch on an elongated rubber arm used magnetoencephalography to
demonstrate changes in the somatosensory cortex. In this experiment, the
cortical representations of the little finger and thumb moved closer together
during the stroking portion of the illusion compared to both the control and
the resting states. In addition, subjective report of the feeling of an elongated
arm correlated significantly with a change in the cortical distances (Schaefer,
Flor, Heinze, & Rotte, 2007).
These cases of remapping hint at how malleable an individual’s sense
of their body may be. In order to consider how humans can identify with
entirely new bodies, we will now discuss more radical examples of avatar
alteration. We propose that given the proper visual/sensory or visual/motor
input, users can learn to control, and possibly identify with, bodies radically
different than their own. In the following section, we will discuss how
virtual reality leverages its unique abilities to track and render motion to
create the illusion of embodying a new persona.

6

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

RELATED RESEARCH ON VIRTUAL AVATARS
In this section, we describe the technology required to create avatars, representation of people in virtual reality. We then discuss research that examines
the relationship between oneself and one’s avatar, in particular, research that
begins to test the process and effect of HF.
Virtual reality is implemented as a function of three distinct processes:
tracking, rendering, and display. Consider the first commercially successful
video game called Pong. A player had a virtual experience that featured a
table-tennis like game, and controlled a virtual racquet. In order to do so,
his hand was “tracked”—a device measured the rotation of the players’ hand
via a mechanical device that rolled from side to side. Then, the screen was
rendered with each new movement—if the roller detected the hand moved
to the left, then the racquet was redrawn in a digital scene to be a few pixels
over. Finally, the newly rendered scene was displayed via sights (an array of
pixels changed colors to show the scene) and sounds (speakers produced
sounds to accompany the action on the screen). This process—tracks the
person’s movements, changes a digital scene accordingly, and updates the
senses to receive this new scene—repeats itself at a high frequency such that
the transitions are seamless.
In virtual reality, people are represented by avatars, digital representations
that are controlled by the actions of a human being in real time (for an explication of the concept, see Bailenson & Blascovich, 2004). A number of scholars
have examined the process and the implications of occupying bodies different from the physical self. In an early explication of this concept, Frank Biocca
(1997) coins a term known as self presence, the degree to which one perceives
one’s avatar as his or her actual self. He provided a framework discussing the
mismatch between one’s avatar and one’s physical self, and posited the psychological effects these mismatches might have on those who utilize avatars.
A recent chapter by Ratan (2012) provides a thorough description of this concept. The idea that the self can be present in a mediated body is key to the
concept of HF in virtual reality.
About a decade after Biocca’s first discussion of this notion, Nick Yee and
colleagues began a research program that tested this notion (see Blascovich
& Bailenson, 2011, Chapter 7, for a review of these studies). The so-called
Proteus effect (Yee & Bailenson, 2007) states that “an individual’s behavior
conforms to their digital self-representation independent of how others perceive them” (p. 271). In these cases, a person can be lead to identify with an
avatar simply by seeing the mirror image of that avatar move in conjunction
with the users own movements. Given the familiarity of seeing one’s own
movements reflected in a mirror, this combination of mirror and motor input
appears to be sufficient to create an identification with the avatar body, even

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

7

when the avatar’s body is different than one’s physical body—for example, a
different race or gender. This effect is lasting enough that participant’s behavior is found to conform to the stereotypes of the avatar, even after the user
exits virtual reality, and is typically measured by behavioral dependent variables. For example, those with taller avatars behave more confidently than
those with shorter avatars (Yee & Bailenson, 2007), college students who use
senior citizen avatars are more likely to put money in a savings account than
those wear age appropriate ones (Hershfield et al., 2011), men who embody
female avatars behave in a more nurturing manner than men who embody
male avatars (Yee, Ducheneaut, Yao, & Nelson, 2011), and people who see
their own avatars exercising are more likely to exercise than those who do
not (Fox & Bailenson, 2009). In sum, the mismatch between the physical self
and the virtual self, proposed initially by Biocca, has both psychological and
behavioral implications.
At around the same time that Yee and colleagues were studying the
implications of spending time in bodies different than one’s physical self,
Ehrsson (2007) and colleagues began to study the process of transferring
one’s sense of self to virtual bodies. By utilizing techniques of synchronous
touch, that is, when a user feels tactile feedback on their body that corresponds to the same visual feedback on an external body that is typically
displayed via video, these scholars shed light on the neural mechanisms
behind the process of self presence. Slater and colleagues have thoroughly
extended this “body transfer” paradigm to test the process of inhabiting
various avatars that are fundamentally nonhuman. For example, in one
study, Slater, Spanlang, Sanchez-Vives, and Blanke (2010) compared the
effects of simultaneous touch, first person perspective, and coordinated
head movements in generating a feeling of transfer from male participants
to a female avatar in virtual reality, and a corresponding physiological
response when the body of that avatar was suddenly attacked. Participants
also reported a subjective feeling of body transfer despite the fact that the
gender was swapped. In another study (Kilteni, Normand, Sanchez-Vives,
& Slater, 2012), participants were led to identify with a body with one
arm rendered as much longer than a normal human arm. Identification
was created using a combination of somatosensory and visual input, and
identification was confirmed using physiological reaction measures, and
the extent to which participants flinched in real life, when the “long” arm
was threatened. This confirms the finding described earlier by Schaefer
et al. (2007) who demonstrated shifts in the somatosensory cortex when the
rubber hand illusion was duplicated using an unusually long hand.

8

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

EMPIRICAL INVESTIGATIONS OF HOMUNCULAR FLEXIBILITY
More recent research is providing direct tests of the ability of people to control
bodies with expanded function, that is, ones that allow people to implement
motor skills that humans have never been able to attempt previously. For
example, Steptoe, Steed, and Slater (2013) demonstrated that people were
able to accommodate avatars that had tails that were controlled by hip movements, and felt distressed when those tails were threatened.
The three authors of this essay have also collaborated on empirical research
examining how humans adapt to controlling avatars with different limb
structures (Won, Lee, Bailenson, & Lanier, in press). In two studies, we
manipulated one’s ability to control novel bodies, and examined how
experimental participants adapted to the new bodies.
The first study involved switching the tracking data of virtual arms and
legs, such that virtual arms adapted features typically associated with
physical legs, and vice versa. There were three conditions: normal, switched
source, and switched range. Figure 2 demonstrates the normal condition.
In this condition, when a participant arm moves her physical arms and
legs, they are tracked and the avatar’s limbs move accordingly, with similar
ranges.
The switched source condition is depicted in Figure 3. In this condition,
when the participant moves her physical legs, her avatar moves its virtual
arms. Similarly, when the participant moves her physical arms, her avatar
moves its virtual legs.
Figure 4 demonstrates the switched range condition. Here, the participant’s arms and legs move the appropriate avatar limbs, but the range of the
avatar’s limb movement is either expanded (in the case of the virtual legs,
which now have the range of real-life arms) or contracted (in the case of
the virtual arms, which now have the range of real-life legs, and do not rise

Figure 2 Normal tracking showing the participant’s tracked movements (black
figure) and the resulting rendering of the avatar (gray figure).

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

Avatar

9

Participant

Figure 3 The switched source condition showing the participant’s tracked
movements (black figure) and the resulting rendering of the avatar (gray figure).

Avatar

Participant

Figure 4 The switched range condition showing the participant’s tracked
movements (black figure) and the resulting rendering of the avatar (gray figure).

above the shoulder). In other words, the avatar gets to move its legs with
more range than humans do, and its arms with less range than humans do.
In our study, the participants were tasked with popping balloons for about
10 min. The balloons appeared in random spots in X y Z space and were
reachable by either the virtual hands or feet. We were able to track the number of balloons they could pop with specific limbs, as well as the overall

10

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

movement of the four limbs. Participants used their physical arms to pop balloons in the normal condition, as their arms offered a greater range to reach
the randomly appearing balloons and humans are also more accustomed to
using their arms to manipulate the physical environment. In the switched
source condition, they used their physical legs (which were then powering
the avatar’s arms) more often than in the normal condition. However, this
did not establish whether or not participants were motivated by the greater
range reached by the avatar’s arms, or whether participants simply preferred
the naturalness of seeing one’s virtual arms perform the task in the virtual
environment. Thus, we implemented the switched range condition, which
demonstrated that participants would use their physical legs more than their
physical arms when offered the utility of a leg that could sweep across the
range of the balloon space. In addition to the performance data, anecdotally,
subjects would at first be disoriented but would only take a few minutes to
accommodate to the new body structures.
In the second, within-subjects study, participants were provided with a
third arm, which emanated from the middle of their chests, and extended
about 1.3 m forward. The participants could rotate their left wrist to control
the x position of their third arm, and could rotate their right wrist to control
the y position of their third arm. The wrist rotations were orthogonal to the
position of the left and right arms, so the control scheme did not interfere
with the ability of the natural arms to perform a task. In this study, participants had to touch cubes floating in space. Some of the cubes were within
arm’s length of the participants, whereas others were approximately. 5 m further away in front of them. In the third arm condition, the close cubes were
reachable by the natural arms, and the far cubes were reachable by the third
arm without walking. In the normal condition, participants had to step forward to touch the most distant cube with one of their two hands. A trial was
defined by completing a task of hitting two close cubes and one far cube. Participants took <5 min, on average, to master using the third arm. After 5 min,
the scores in the third arm condition were routinely higher than those in the
normal condition (Figure 5).
AN ORGANIZING FRAMEWORK
There have been a number of attempts by scholars to provide an overall
framework for thinking about this research on embodiment as it relates to
neural plasticity. For example, Clark (2007) provides a novel framework to
think about augmenting the self as it relates to tool use and neural plasticity.
In their 2012 paper, Haans and IJsselsteijn (2012) discuss how tools and other
artifacts may be incorporated into different levels of embodiment, and what
this implies for the experience of telepresence, including in more novel avatar

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

11

Figure 5 The schematic shows the method of controlling the “third limb” (shown
in gray extending out of the participant’s chest). Rotating the right wrist moved the
participant’s third limb up and down, whereas rotating the right wrist moved the
avatar’s third limb from side to side. Thus, rotating both arms allowed the
participant to control the third limb while also hitting targets with the left and right
arms.

bodies. In this essay, we do not propose an overarching theory to explain the
process, but rather suggest an organizing framework to help provide a common language.
First, one can consider the changes in body schema from a functional
standpoint. The first function is restoring the body. For example, the research
on how the homunculus adjusts to the presence of a prosthetic limb would
fit this category. The second is replacing the body with one that has equal
human function. For example, scholars who study the process and implications of “gender-bending,” that is, avatar gender that does not match
physical gender, fit this parameter. The third is reinventing [term adapted
from Clark (2007)] the body, by providing bodies, or at least body parts,
which are decidedly nonhuman, but yet can be controlled by a human user,
for example, having a third arm.
However, one can also sort these experiences by the mode in which
the user’s sense of embodiment is transformed. Changes within a single
channel of interaction, such as adding extra limbs that are controlled by
different degrees of freedom from the participant’s physical body, would
be considered a type of ipsimodal remapping. Another type of transformation
moves between different modes. For example, if spatial information that
would normally be perceived visually is presented audibly, as in aids for
the visually impaired, that would be a case of sensory substitution. A third
type of transformation would remap tracked input that the user may not
consciously experiences (such as heart rate or EEG information) into renderings that the user, or another, may easily observe and use. For example,
an avatar might change colors to match his or her emotion, or move more
quickly when the user is agitated. These would be cases of para-synthetic
expression.

12

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

CONCLUSION
The notion of HF is one that has been percolating for decades, but recent
technological advancements are bringing the construct to the forefront of
research that spans computer science, neuroscience, psychology, and biology.
As research continues to push the boundary on what it means to be human
for the purpose of basic science, it is also important to consider the practical
implications of this work.
Consider video games in the year 2013. The Microsoft Kinect, the fastest
selling consumer electronic product in the US history as determined by Guinness, has completely transformed the notion of tracking. Instead of holding
onto a physical device, the Kinect uses the body itself as an input device. In
other words, it unobtrusively—that is, the user does not have to wear any
special markers or hold any devices—tracks body movements, by emitting
infrared light that then hits the user, reflects back to the device, and can figure
out the position of various limbs by creating a depth map of the body. Instead
of a single joystick to control a video game, the user now has many degrees of
freedom to control a virtual experience, for example, hand, arm, leg, and head
movements. Of course, researchers studying virtual reality have been tracking body movements for decades (see Blascovich & Bailenson, 2011, Chapter
3, for a review), but for the first time ever, it is possible to do it cheaply and
unobtrusively.
One area to apply this work is in the idea of empathy. Research has
demonstrated that avatars have a unique ability to allow one to “walk
a mile” in someone else’s shoes, be allowing them to literally become
someone of another race, gender, or social category. Work by Ahn, Le, and
Bailenson (2013) placed humans with normal vision in avatars with visual
impairments, such that the humans were able to viscerally experience a
disability. Compared to control conditions, for example, imagining one
had the disability, the avatar experience caused experimental subjects to
volunteer more of their own time to help those with disabilities, and their
attitude change toward the disabled remained changed a day after the
virtual treatment. Similarly, this notion of body transfer should be able
to allow people to gain empathy for different species. The plight of an
endangered animal will likely feel more personal and salient when a person
experiences a day in the life in that animal’s body.
Another obvious application of this framework is to increase human function, to literally have eyes on the back of one’s head and to be in multiple
places at once, via an expanded body. There are many other possibilities
for productivity, education, entertainment, and travel. On the other hand, it
is critical to keep a skeptical eye on these applications, and to monitor the
academic research closely as we build these applications for society at large.

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

13

Clifford Nass and colleagues (Ophir, Nass, & Wagner, 2009; Pea et al., 2012)
have demonstrated that, while alluring, multitasking—specifically using
more than one form of media at once—is robustly counterproductive, and
possibly harmful to cognitive function. It remains to be seen if hidden costs
from occupying novel bodies with “extra” functions, especially over time,
arise.
REFERENCES
Ahn, S. J., Le, A. M. T., & Bailenson, J. N. (2013). The effect of embodied experiences on self-other merging, attitude, and helping behavior. Media Psychology,
16(1), 7–38.
Bailenson, J. N., & Blascovich, J. (2004). Avatars. In Encyclopedia of human-computer
interaction. Great Barrington, MA: Berkshire Publishing Group.
Barinaga, M. (1995). Remapping the motor cortex. Science, 268(5218), 1696–1698.
Biocca, F. (1997). The cyborg’s dilemma: Progressive embodiment in virtual environments. Journal of Computer-Mediated Communication, 3(2).
Blascovich, J., & Bailenson, J. N. (2011). Infinite reality—avatars, eternal life, new worlds,
and the dawn of the virtual revolution. New York, NY: William Morrow.
Botvinick, M., & Cohen, J. (1998). Rubber hands’ “feel” touch that eyes see. Nature,
391(6669), 756.
Clark, A. (2007). Re-inventing ourselves: The plasticity of embodiment, sensing, and
mind. Journal of Medicine and Philosophy, 32, 263–282.
Ehrsson, H. H. (2007). The experimental induction of out-of-body experiences. Science, 317, 1048. doi:10.1126/science.1097011
Fox, J., & Bailenson, J. N. (2009). Virtual self-modeling: The effects of vicarious reinforcement and identification on exercise behaviors. Media Psychology, 12, 1–25.
Haans, A., & IJsselsteijn, W. A. (2012). Embodiment and telepresence: Toward a comprehensive theoretical framework. Interacting with Computers, 24(4), 211–218.
Hershfield, H. E., Goldstein, D. G., Sharpe, W. F., Fox, J., Yeykelis, L., Carstensen, L.
L., & Bailenson, J. N. (2011). Increasing saving behavior through age-progressed
renderings of the future self. Journal of Marketing Research, 48, S23–S37.
IJsselsteijn, W. A., de Kort, Y. A., & Haans, A. (2006). Is this my hand I see before me?
The rubber hand illusion. In reality, virtual reality, and mixed reality. Presence:
Teleoper. Virtual Environment, 15(4), 455–464.
Kilteni, K., Normand, J.-M., Sanchez-Vives, M. V., & Slater, M. (2012). Extending body
space in immersive virtual reality: A very long arm illusion. PLoS One, 7(7), e40867.
doi:10.1371/journal.pone.0040867
Lotze, M., Flor, H., Grodd, W., Larbig, W., & Birbaumer, N. (2001). Phantom movements and pain. An fMRI study in upper limb amputees. Brain, 124(11), 2268–2277.
Ophir, E., Nass, C., & Wagner, A. (2009). Cognitive control in media multitaskers.
Proceedings of the National Academy of Sciences, 106(33), 15583–15587.
Pea, R., Nass, C., Meheula, L., Rance, M., Kumar, A., Bamford, H., … & Zhou,
M. (2012). Media use, face-to-face communication, media multitasking and social
wellness among 8–12 year old girls. Development Psychology, 48(2), 327–336.

14

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

Penfield, W., & Boldrey, E. (1937). Somatic motor and sensory representation in the
cerebral cortex of man as studied by electrical stimulation. Brain, 60, 389–443.
Ramachandran, V. S., & Altschuler, E. L. (2009). The use of visual feedback, in particular mirror visual feedback, in restoring brain function. Brain, 132(7), 1693–1710.
Ramachandran, V. S., & Rogers-Ramachandran, D. (1996). Synaesthesia in phantom
limbs induced with mirrors. Proceedings of the Royal Society of London, Series B: Biological Sciences, 263(1369), 377–386.
Ratan, R. A. (2012). Self-presence, explicated: Body, emotion, and identity extension
into the virtual self. In R. Luppicini (Ed.), Handbook of research on technoself . IGI
Global: New York, NY.
Roricht, S., Meyer, B. U., Niehaus, L., & Brandt, S. A. (1999). Long-term reorganization of motor cortex outputs after arm amputation. Neurology, 53, 106–111.
Sato, K., Fukumori, S., Matsusaki, T., Maruo, T., Ishikawa, S., Nishie, H., ... Morita,
K. (2010). Nonimmersive virtual reality mirror visual feedback therapy and its
application for the treatment of complex regional pain syndrome: an open-label
pilot study. Pain Medicine, 11(4), 622–629.
Schaefer, M., Flor, H., Heinze, H. J., & Rotte, M. (2007). Morphing the body: illusory
feeling of an elongated arm affects somatosensory homunculus. NeuroImage, 36(3),
700–705.
Schmalzl, L., Ragnö, C., & Ehrsson, H. H. (2013). An alternative to traditional mirror
therapy: Illusory touch can reduce phantom pain when illusory movement does
not. The Clinical Journal of Pain, 29(10), e10–e18.
Slater, M., Spanlang, B., Sanchez-Vives, M. V., & Blanke, O. (2010). First person experience of body transfer in virtual reality. PLoS One, 5(5), e10564.
Steptoe, W., Steed, A., & Slater, M. (2013). Human tails: Ownership and control of
extended humanoid avatars. Visualization and Computer Graphics, IEEE Transactions
on, 19(4), 583–590.
Won, A. S., & Collins, T. A. (2012). Non-immersive, virtual reality mirror visual
feedback for treatment of persistent idiopathic facial pain. Pain Medicine, 13(9),
1257–1258.
Won, A. S., Lee, J. D., Bailenson, J. N., & Lanier, J. (in press). Homuncular flexibility
in virtual reality. Journal of Computer-Mediated Communication.
Yee, N., & Bailenson, J. (2007). The Proteus effect: The effect of transformed
self-representation on behavior. Human Communication Research, 33(3), 271–290.
Yee, N., Ducheneaut, N., Yao, M., & Nelson, L. (2011). Do men heal more when in
drag? Conflicting identity cues between user and avatar. Proceedings of CHI 2011,
pp. 773–776.

ANDREA STEVENSON WON SHORT BIOGRAPHY
Andrea Stevenson Won received her MS in Biomedical Visualization from
the University of Illinois at Chicago in 2005. She is a PhD candidate in the
Department of Communication at Stanford University. Her research interests

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

15

include mediated self-representation and tracking, assessing, and manipulating body movements to affect outcomes. She is a member of the International
Communication Association. Contact her at aswon@stanford.edu.
JEREMY N. BAILENSON SHORT BIOGRAPHY
Jeremy N. Bailenson is founding director of Stanford University’s Virtual
Human Interaction Lab, an associate professor in the Department of Communication at Stanford, and a senior fellow at the Woods Institute for the
Environment. He earned a BA cum laude from the University of Michigan
in 1994 and a PhD in cognitive psychology from Northwestern University in
1999. After receiving his doctorate, he spent 4 years at the Research Center
for Virtual Environments and Behavior at the University of California, Santa
Barbara as a post-doctoral fellow and then an assistant research professor.
Bailenson’s main area of interest is the phenomenon of digital human
representation, especially in the context of immersive virtual reality. He
explores the manner in which people are able to represent themselves when
the physical constraints of body and veridically rendered behaviors are
removed. Furthermore, he designs and studies virtual reality systems that
allow physically remote individuals to meet in virtual space, and explores
the manner in which these systems change the nature of verbal and nonverbal interactions. In particular, he explores how virtual reality can change the
way people think about education, environmental behavior, and health.
His findings have been published in over 90 academic papers in the fields
of communication, computer science, education, law, political science, and
psychology. His work has been consistently funded by the National Science
Foundation for over a decade, and he also receives grants from various Silicon Valley and international corporations. He consults regularly for government agencies including the US Army and Air Force, the Department of
Defense, the Department of Energy, the National Research Council, and the
National Institute of Health on policy issues surrounding virtual reality.
His book Infinite Reality, coauthored with Jim Blascovich, was recently
quoted by the Supreme Court outlining the effects of immersive media.
JARON LANIER SHORT BIOGRAPHY
Jaron Lanier is a member of Microsoft Research. He either coined or popularized the term virtual reality and in the early 1980s founded VPL Research,
the first company to sell VR products. He led the team that developed the
first implementations of multi-person virtual worlds using head-mounted
displays, for both local and wide area networks, as well as the first “avatars,”
or representations of users within such systems. While at VPL, he and his colleagues developed the first implementations of virtual reality applications in

16

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

surgical simulation, vehicle interior prototyping, virtual sets for television
production, and assorted other areas.
Harvard’s Kennedy School of Government chose Lanier’s book “Who Owns
the Future?” for 2014s Goldsmith Award. He received a Lifetime Career
Award from the IEEE in 2009 for contributions to virtual reality, and was
the recipient of CMU’s Watson award in 2001. He has received honorary
doctorates from the New Jersey Institute of Technology and Franklin and
Marshall College. He has also been affiliated with USC, UC Berkeley, NYU,
Columbia, UPenn, and Internet2. He has written best-selling books, such
as “You Are Not a Gadget.” He is also a composer of “classical” music (with
commissions from the St. Paul Chamber Orchestra and others) and plays
a large number of instruments (having performed with Philip Glass, Yoko
Ono, George Clinton, Ornette Coleman, and many others.)



Homuncular Flexibility: The Human
Ability to Inhabit Nonhuman Avatars
ANDREA STEVENSON WON, JEREMY N. BAILENSON, and JARON LANIER

Abstract
This essay seeks to explicate an unorthodox idea that spans psychology, neuroscience, psychology, philosophy, and computer science called homuncular flexibility
(HF). HF posits that the homunculus—the part of the cortex that maps movement
and sensing of body parts—is capable of adapting to novel bodies, in particular
bodies that have extra appendages or appendages capable of atypical movements.
Evidence demonstrates neural plasticity in nature; for example, amputees experience cortical shifting such that their face receives extra attention in the brain after
a limb is amputated. However, experiments such as the rubber hand illusion, in
which people respond to rubber hands placed near their arms as if they were their
actual hands, demonstrated that a person’s sense of their body can be adjusted to
include external objects. The recent advent of virtual reality technology, which can
track physical human motions and display them on avatars, allows for the wholly
new human experience of inhabiting distinctly nonhuman bodies. HF is a paradigm
in which physical motions are transformed by remapping degrees of freedom from
tracked movements onto an avatar. For example, if a human being were to inhabit
an avatar of a lobster, controlling two of the eight lobster arms would be simply be a
function of tracking the two physical arms and directly mapping those movements
onto the avatar’s first two arms. However, in order to control the remaining six
arms, degrees of freedom that are orthogonal to the movements of the first two
arms need to be measured and remapped. In this essay, we discuss advancements in
neuroscience, psychology, and computer science that relate to HF. We review some
preliminary studies that demonstrate how humans accommodate novel bodies.
Finally, we discuss theoretical implications and practical applications relating to HF.

HISTORY OF HOMUNCULAR FLEXIBILITY
Accounts of the history of experimentation are usually paced by examples
of phenomenon coming to light because of improved instrumentation.
homuncular flexibility (HF) is a rare example of the opposite; it was first
noticed because of poor instrumentation.
The 1980s saw the first attempts to implement networked immersive
virtual worlds (deemed “virtual reality”) at VPL Research. To be networked,
Emerging Trends in the Social and Behavioral Sciences. Edited by Robert Scott and Stephen Kosslyn.
© 2015 John Wiley & Sons, Inc. ISBN 978-1-118-90077-2.

1

2

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

users would experience being present in a shared virtual setting and would
be able to see representations of each other. Thus, it was necessary to create
three-dimensional avatars of each user. One of the strengths of the early
development system for virtual reality at VPL Research was that it supported
extremely rapid prototyping, with revisions appearing instantly while a
subject was “inside” a virtual world experiment. The calibration of early
full-body avatars presented a challenge; it was difficult to design a suit so
that sensors would remain in precisely the same locations on the user’s body
with extended use. In the context of rapid experimentation with mappings
and heuristics, it came to pass that nonrealistic, whole-body avatars were
occasionally activated. These would usually result in a complete breakdown
of usability. As an example, if an avatar’s head were made to jut out of the
side of the hip, the world would appear rotated awkwardly to the user, who
would immediately become disoriented, unable to perform any task. In the
course of exploring avatar designs, researchers occasionally came upon an
unusual avatar design that preserved usability despite being nonrealistic or
even bizarre.
The first one occurred during the process of creating an immersive city and
harbor planning tool, a collaboration between Jaron Lanier at VPL Research
and Tom Furness and others at the HITLab of the University of Washington. One of the scientists was inhabiting an avatar—a worker at the docks in
Seattle—when his arm was made very large, perhaps the size of a very large
crane. This most likely occurred because a designer entered extra zeros in a
scale factor in the software that measured movement. What was remarkable
was that the scientist was able to pick up vehicles and other objects in a large
scene at great distance with a highly distorted arm, and was able to do so
with accuracy and no apparent loss of usability.
This unexpected observation, and others similar to it, motivated an informal study of “weird avatars that were still usable.” A number of increasingly
strange, but usable, nonhuman avatars were created and tested between
approximately 1989 and 1991. Most of these resembled mammals, in terms
of overall structure and inventory of limbs.
A number of principles emerged via informal pilot testing. For example, it
appeared that the relative scaling of parts of the body was surprisingly plastic. The attachment points of limbs could be changed so long as the angle of
attachment did not change too much. However, the ultimate strange avatars
departed from the mammalian plan entirely. Most of these did not preserve
usability. However, there was one that bears mention. One colleague, Ann
Lasko, had seen a postcard of people in lobster suits at a festival, and created
a lobster avatar and set about programming a body map for it. As the lobster
body includes more limbs than a person, there were not enough parameters measured by the body suit to drive the lobster avatar in a one-to-one

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

3

map. Therefore, the scientist mapped the degrees of freedom of the body suit
to the greater number of degrees of freedom of the lobster. The initial mappings were not usable, but over time, as the algorithms to control the extra
limbs evolved, some mappings emerged that succeeded. Over time, and with
practice, humans were able to slowly learn to control the lobster. The biologist Jim Bower, when visiting Lanier’s lab during this period, commented
that the range of usable nonhuman avatars might be related to the phylogenetic tree. The human brain might be expected to find body plans that had
occurred in the history of its own descent to be more usable than other body
plans, though of course this would not include the lobster. Nonetheless, the
intriguing question remains of what makes certain nonhuman avatars usable
while others are not.
In this essay, we will first give a rough description of the brain structures
that might support flexibility of virtual embodiment. Then, we review some
landmark findings related to brain plasticity that provide foundations for
the current work on remapping physical bodies onto avatar bodies. Next,
we describe recent work in virtual reality that tests the ability of people to
accommodate various avatar bodies. Finally, we suggest two frameworks for
organizing this emerging area of research—one based on the overall function
of the transformation—for example, restoring function compared to inventing function, and the other based on the mode of the transformation.
THE HOMUNCULUS AND EXPERIMENTS IN NEURAL PLASTICITY
The “homunculus” in the term HF refers to motor and somatosensory representations of the human body in the brain. Both the primary motor cortex
and the primary somatosensory cortex contain areas where the innervation
arising from different parts of the body roughly maps to corresponding cortical regions. These areas form Penfield’s “homunculus” (Penfield & Boldrey,
1937). As with many brain structures, the left side of the body is represented
in the right cerebral hemisphere, and vice versa. The primary motor cortex,
which relates to muscle movement, is located on the frontal side of the central
gyrus, whereas the primary sensory cortex, which is involved in the perceptions of touch sensation, opposes it. A schematic approximating the relationships of these two structures is shown in Figure 1.
Areas of the body with greater innervation are represented by more space
in the primary motor cortex than body parts with less innervation. Thus,
the hands, which have complex movements/muscular innervations, take up
more space than the leg. This mapping is not currently considered to be
one-to-one; it has been proposed that rather than specific cortical regions
mapping to specific muscles, the motor cortex controls the motions made by
these body regions (Barinaga, 1995). However, it does provide for a general

4

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

Cross-section of right hemisphere
Somatosensory cortex
Hand

Tongue

Post-central gyrus

Motor cortex
Hand

Tongue

Pre-central gyrus

Figure 1 Homunculi of the somatosensory and motor cortices, showing the
differences in size mapping for the hand and tongue in the somatosensory and
motor cortices, reflecting their comparative differences in innervation.

relationship between brain regions and the movements of different regions of
the body. The primary somatosensory cortex is similarly arranged, such that
parts of the body with greater sensitivity, such as the lips, fingers, and thumb,
take up more space than, for example, the back. In contrast to the motor cortex, the hands are less dominant in the sensory cortex, whereas the sensitive
(but immobile) teeth and gums are well present here but not in the motor
cortex.
The mappings of the somatosensory and motor cortex may be altered by
injury. For example, in the case of an amputated arm, cortical mapping may
shift such that areas that previously innervated the hand may shift to another
part of the body (Roricht, Meyer, Niehaus, & Brandt, 1999). This cortical plasticity is hypothesized to be one source of phantom limb pain.
Phantom limb pain is the phenomenon in which an amputee continues
to have sensation, sometimes very painful, perceived as originating in the
absent limb long after the trauma has healed. This persistent pain has been
proposed to result from cortical remapping. In 1996, Ramachandran and
Rogers-Ramachandran proposed that altering the visual input that patients
received could help in remapping the motor cortex and thus alleviate
phantom limb pain. To do this, they designed a box with a mirror in the
midline. When patients placed their uninjured limb in such a box, it created

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

5

the illusion of two normally moving limbs. This appeared to relieve some
patients’ pain, perhaps because the pathological cortical map was restored
by the visual input of a plausible limb in the location of the amputated limb.
Similarly, Lotze, Flor, Grodd, Larbig, and Birbaumer (2001) have proposed
that the use of a myoelectric prosthesis providing similar visual feedback
may prevent cortical reorganization and the accompanying phantom limb
pain. Therapies building on mirror visual therapy have also been used
to treat complex regional pain syndrome (CRPS), a condition with some
similarities to phantom limb pain (Sato et al., 2010).
Visual feedback combined with somatosensory input has also been proposed to change a person’s understanding of his or her body. Stroking a
prosthesis as a substitute for a missing limb has been proposed as a therapy
for phantom limb pain (Ramachandran & Rogers-Ramachandran, 1996)
and CRPS (Schmalzl, Ragnö, & Ehrsson, 2013). Creating the illusion of
touch has also been suggested for pain syndromes in the face that would
not respond well to movement-based mirror therapies (Ramachandran
& Altschuler, 2009; Won & Collins, 2012). These therapies build upon a
phenomenon of body schema transfer often linked to phantom limb pain,
known as the rubber hand illusion (Botvinick & Cohen, 1998). In this illusion,
a participant’s hand is placed underneath a cloth or a table on which a
rubber arm is laid. The participant’s real hand and the rubber hand are then
stroked at the same time with a soft brush. Gradually, the patient feels a
sense of increasing ownership toward the rubber hand, and he or she will
flinch if the rubber hand is suddenly hit with a mallet. IJsselsteijn, de Kort,
and Haans (2006) replicated the illusion in the physical world, in virtual
reality, and in mixed reality, finding effects across conditions, although the
illusion was strongest in the unmediated condition. A similar experiment
using touch on an elongated rubber arm used magnetoencephalography to
demonstrate changes in the somatosensory cortex. In this experiment, the
cortical representations of the little finger and thumb moved closer together
during the stroking portion of the illusion compared to both the control and
the resting states. In addition, subjective report of the feeling of an elongated
arm correlated significantly with a change in the cortical distances (Schaefer,
Flor, Heinze, & Rotte, 2007).
These cases of remapping hint at how malleable an individual’s sense
of their body may be. In order to consider how humans can identify with
entirely new bodies, we will now discuss more radical examples of avatar
alteration. We propose that given the proper visual/sensory or visual/motor
input, users can learn to control, and possibly identify with, bodies radically
different than their own. In the following section, we will discuss how
virtual reality leverages its unique abilities to track and render motion to
create the illusion of embodying a new persona.

6

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

RELATED RESEARCH ON VIRTUAL AVATARS
In this section, we describe the technology required to create avatars, representation of people in virtual reality. We then discuss research that examines
the relationship between oneself and one’s avatar, in particular, research that
begins to test the process and effect of HF.
Virtual reality is implemented as a function of three distinct processes:
tracking, rendering, and display. Consider the first commercially successful
video game called Pong. A player had a virtual experience that featured a
table-tennis like game, and controlled a virtual racquet. In order to do so,
his hand was “tracked”—a device measured the rotation of the players’ hand
via a mechanical device that rolled from side to side. Then, the screen was
rendered with each new movement—if the roller detected the hand moved
to the left, then the racquet was redrawn in a digital scene to be a few pixels
over. Finally, the newly rendered scene was displayed via sights (an array of
pixels changed colors to show the scene) and sounds (speakers produced
sounds to accompany the action on the screen). This process—tracks the
person’s movements, changes a digital scene accordingly, and updates the
senses to receive this new scene—repeats itself at a high frequency such that
the transitions are seamless.
In virtual reality, people are represented by avatars, digital representations
that are controlled by the actions of a human being in real time (for an explication of the concept, see Bailenson & Blascovich, 2004). A number of scholars
have examined the process and the implications of occupying bodies different from the physical self. In an early explication of this concept, Frank Biocca
(1997) coins a term known as self presence, the degree to which one perceives
one’s avatar as his or her actual self. He provided a framework discussing the
mismatch between one’s avatar and one’s physical self, and posited the psychological effects these mismatches might have on those who utilize avatars.
A recent chapter by Ratan (2012) provides a thorough description of this concept. The idea that the self can be present in a mediated body is key to the
concept of HF in virtual reality.
About a decade after Biocca’s first discussion of this notion, Nick Yee and
colleagues began a research program that tested this notion (see Blascovich
& Bailenson, 2011, Chapter 7, for a review of these studies). The so-called
Proteus effect (Yee & Bailenson, 2007) states that “an individual’s behavior
conforms to their digital self-representation independent of how others perceive them” (p. 271). In these cases, a person can be lead to identify with an
avatar simply by seeing the mirror image of that avatar move in conjunction
with the users own movements. Given the familiarity of seeing one’s own
movements reflected in a mirror, this combination of mirror and motor input
appears to be sufficient to create an identification with the avatar body, even

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

7

when the avatar’s body is different than one’s physical body—for example, a
different race or gender. This effect is lasting enough that participant’s behavior is found to conform to the stereotypes of the avatar, even after the user
exits virtual reality, and is typically measured by behavioral dependent variables. For example, those with taller avatars behave more confidently than
those with shorter avatars (Yee & Bailenson, 2007), college students who use
senior citizen avatars are more likely to put money in a savings account than
those wear age appropriate ones (Hershfield et al., 2011), men who embody
female avatars behave in a more nurturing manner than men who embody
male avatars (Yee, Ducheneaut, Yao, & Nelson, 2011), and people who see
their own avatars exercising are more likely to exercise than those who do
not (Fox & Bailenson, 2009). In sum, the mismatch between the physical self
and the virtual self, proposed initially by Biocca, has both psychological and
behavioral implications.
At around the same time that Yee and colleagues were studying the
implications of spending time in bodies different than one’s physical self,
Ehrsson (2007) and colleagues began to study the process of transferring
one’s sense of self to virtual bodies. By utilizing techniques of synchronous
touch, that is, when a user feels tactile feedback on their body that corresponds to the same visual feedback on an external body that is typically
displayed via video, these scholars shed light on the neural mechanisms
behind the process of self presence. Slater and colleagues have thoroughly
extended this “body transfer” paradigm to test the process of inhabiting
various avatars that are fundamentally nonhuman. For example, in one
study, Slater, Spanlang, Sanchez-Vives, and Blanke (2010) compared the
effects of simultaneous touch, first person perspective, and coordinated
head movements in generating a feeling of transfer from male participants
to a female avatar in virtual reality, and a corresponding physiological
response when the body of that avatar was suddenly attacked. Participants
also reported a subjective feeling of body transfer despite the fact that the
gender was swapped. In another study (Kilteni, Normand, Sanchez-Vives,
& Slater, 2012), participants were led to identify with a body with one
arm rendered as much longer than a normal human arm. Identification
was created using a combination of somatosensory and visual input, and
identification was confirmed using physiological reaction measures, and
the extent to which participants flinched in real life, when the “long” arm
was threatened. This confirms the finding described earlier by Schaefer
et al. (2007) who demonstrated shifts in the somatosensory cortex when the
rubber hand illusion was duplicated using an unusually long hand.

8

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

EMPIRICAL INVESTIGATIONS OF HOMUNCULAR FLEXIBILITY
More recent research is providing direct tests of the ability of people to control
bodies with expanded function, that is, ones that allow people to implement
motor skills that humans have never been able to attempt previously. For
example, Steptoe, Steed, and Slater (2013) demonstrated that people were
able to accommodate avatars that had tails that were controlled by hip movements, and felt distressed when those tails were threatened.
The three authors of this essay have also collaborated on empirical research
examining how humans adapt to controlling avatars with different limb
structures (Won, Lee, Bailenson, & Lanier, in press). In two studies, we
manipulated one’s ability to control novel bodies, and examined how
experimental participants adapted to the new bodies.
The first study involved switching the tracking data of virtual arms and
legs, such that virtual arms adapted features typically associated with
physical legs, and vice versa. There were three conditions: normal, switched
source, and switched range. Figure 2 demonstrates the normal condition.
In this condition, when a participant arm moves her physical arms and
legs, they are tracked and the avatar’s limbs move accordingly, with similar
ranges.
The switched source condition is depicted in Figure 3. In this condition,
when the participant moves her physical legs, her avatar moves its virtual
arms. Similarly, when the participant moves her physical arms, her avatar
moves its virtual legs.
Figure 4 demonstrates the switched range condition. Here, the participant’s arms and legs move the appropriate avatar limbs, but the range of the
avatar’s limb movement is either expanded (in the case of the virtual legs,
which now have the range of real-life arms) or contracted (in the case of
the virtual arms, which now have the range of real-life legs, and do not rise

Figure 2 Normal tracking showing the participant’s tracked movements (black
figure) and the resulting rendering of the avatar (gray figure).

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

Avatar

9

Participant

Figure 3 The switched source condition showing the participant’s tracked
movements (black figure) and the resulting rendering of the avatar (gray figure).

Avatar

Participant

Figure 4 The switched range condition showing the participant’s tracked
movements (black figure) and the resulting rendering of the avatar (gray figure).

above the shoulder). In other words, the avatar gets to move its legs with
more range than humans do, and its arms with less range than humans do.
In our study, the participants were tasked with popping balloons for about
10 min. The balloons appeared in random spots in X y Z space and were
reachable by either the virtual hands or feet. We were able to track the number of balloons they could pop with specific limbs, as well as the overall

10

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

movement of the four limbs. Participants used their physical arms to pop balloons in the normal condition, as their arms offered a greater range to reach
the randomly appearing balloons and humans are also more accustomed to
using their arms to manipulate the physical environment. In the switched
source condition, they used their physical legs (which were then powering
the avatar’s arms) more often than in the normal condition. However, this
did not establish whether or not participants were motivated by the greater
range reached by the avatar’s arms, or whether participants simply preferred
the naturalness of seeing one’s virtual arms perform the task in the virtual
environment. Thus, we implemented the switched range condition, which
demonstrated that participants would use their physical legs more than their
physical arms when offered the utility of a leg that could sweep across the
range of the balloon space. In addition to the performance data, anecdotally,
subjects would at first be disoriented but would only take a few minutes to
accommodate to the new body structures.
In the second, within-subjects study, participants were provided with a
third arm, which emanated from the middle of their chests, and extended
about 1.3 m forward. The participants could rotate their left wrist to control
the x position of their third arm, and could rotate their right wrist to control
the y position of their third arm. The wrist rotations were orthogonal to the
position of the left and right arms, so the control scheme did not interfere
with the ability of the natural arms to perform a task. In this study, participants had to touch cubes floating in space. Some of the cubes were within
arm’s length of the participants, whereas others were approximately. 5 m further away in front of them. In the third arm condition, the close cubes were
reachable by the natural arms, and the far cubes were reachable by the third
arm without walking. In the normal condition, participants had to step forward to touch the most distant cube with one of their two hands. A trial was
defined by completing a task of hitting two close cubes and one far cube. Participants took <5 min, on average, to master using the third arm. After 5 min,
the scores in the third arm condition were routinely higher than those in the
normal condition (Figure 5).
AN ORGANIZING FRAMEWORK
There have been a number of attempts by scholars to provide an overall
framework for thinking about this research on embodiment as it relates to
neural plasticity. For example, Clark (2007) provides a novel framework to
think about augmenting the self as it relates to tool use and neural plasticity.
In their 2012 paper, Haans and IJsselsteijn (2012) discuss how tools and other
artifacts may be incorporated into different levels of embodiment, and what
this implies for the experience of telepresence, including in more novel avatar

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

11

Figure 5 The schematic shows the method of controlling the “third limb” (shown
in gray extending out of the participant’s chest). Rotating the right wrist moved the
participant’s third limb up and down, whereas rotating the right wrist moved the
avatar’s third limb from side to side. Thus, rotating both arms allowed the
participant to control the third limb while also hitting targets with the left and right
arms.

bodies. In this essay, we do not propose an overarching theory to explain the
process, but rather suggest an organizing framework to help provide a common language.
First, one can consider the changes in body schema from a functional
standpoint. The first function is restoring the body. For example, the research
on how the homunculus adjusts to the presence of a prosthetic limb would
fit this category. The second is replacing the body with one that has equal
human function. For example, scholars who study the process and implications of “gender-bending,” that is, avatar gender that does not match
physical gender, fit this parameter. The third is reinventing [term adapted
from Clark (2007)] the body, by providing bodies, or at least body parts,
which are decidedly nonhuman, but yet can be controlled by a human user,
for example, having a third arm.
However, one can also sort these experiences by the mode in which
the user’s sense of embodiment is transformed. Changes within a single
channel of interaction, such as adding extra limbs that are controlled by
different degrees of freedom from the participant’s physical body, would
be considered a type of ipsimodal remapping. Another type of transformation
moves between different modes. For example, if spatial information that
would normally be perceived visually is presented audibly, as in aids for
the visually impaired, that would be a case of sensory substitution. A third
type of transformation would remap tracked input that the user may not
consciously experiences (such as heart rate or EEG information) into renderings that the user, or another, may easily observe and use. For example,
an avatar might change colors to match his or her emotion, or move more
quickly when the user is agitated. These would be cases of para-synthetic
expression.

12

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

CONCLUSION
The notion of HF is one that has been percolating for decades, but recent
technological advancements are bringing the construct to the forefront of
research that spans computer science, neuroscience, psychology, and biology.
As research continues to push the boundary on what it means to be human
for the purpose of basic science, it is also important to consider the practical
implications of this work.
Consider video games in the year 2013. The Microsoft Kinect, the fastest
selling consumer electronic product in the US history as determined by Guinness, has completely transformed the notion of tracking. Instead of holding
onto a physical device, the Kinect uses the body itself as an input device. In
other words, it unobtrusively—that is, the user does not have to wear any
special markers or hold any devices—tracks body movements, by emitting
infrared light that then hits the user, reflects back to the device, and can figure
out the position of various limbs by creating a depth map of the body. Instead
of a single joystick to control a video game, the user now has many degrees of
freedom to control a virtual experience, for example, hand, arm, leg, and head
movements. Of course, researchers studying virtual reality have been tracking body movements for decades (see Blascovich & Bailenson, 2011, Chapter
3, for a review), but for the first time ever, it is possible to do it cheaply and
unobtrusively.
One area to apply this work is in the idea of empathy. Research has
demonstrated that avatars have a unique ability to allow one to “walk
a mile” in someone else’s shoes, be allowing them to literally become
someone of another race, gender, or social category. Work by Ahn, Le, and
Bailenson (2013) placed humans with normal vision in avatars with visual
impairments, such that the humans were able to viscerally experience a
disability. Compared to control conditions, for example, imagining one
had the disability, the avatar experience caused experimental subjects to
volunteer more of their own time to help those with disabilities, and their
attitude change toward the disabled remained changed a day after the
virtual treatment. Similarly, this notion of body transfer should be able
to allow people to gain empathy for different species. The plight of an
endangered animal will likely feel more personal and salient when a person
experiences a day in the life in that animal’s body.
Another obvious application of this framework is to increase human function, to literally have eyes on the back of one’s head and to be in multiple
places at once, via an expanded body. There are many other possibilities
for productivity, education, entertainment, and travel. On the other hand, it
is critical to keep a skeptical eye on these applications, and to monitor the
academic research closely as we build these applications for society at large.

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

13

Clifford Nass and colleagues (Ophir, Nass, & Wagner, 2009; Pea et al., 2012)
have demonstrated that, while alluring, multitasking—specifically using
more than one form of media at once—is robustly counterproductive, and
possibly harmful to cognitive function. It remains to be seen if hidden costs
from occupying novel bodies with “extra” functions, especially over time,
arise.
REFERENCES
Ahn, S. J., Le, A. M. T., & Bailenson, J. N. (2013). The effect of embodied experiences on self-other merging, attitude, and helping behavior. Media Psychology,
16(1), 7–38.
Bailenson, J. N., & Blascovich, J. (2004). Avatars. In Encyclopedia of human-computer
interaction. Great Barrington, MA: Berkshire Publishing Group.
Barinaga, M. (1995). Remapping the motor cortex. Science, 268(5218), 1696–1698.
Biocca, F. (1997). The cyborg’s dilemma: Progressive embodiment in virtual environments. Journal of Computer-Mediated Communication, 3(2).
Blascovich, J., & Bailenson, J. N. (2011). Infinite reality—avatars, eternal life, new worlds,
and the dawn of the virtual revolution. New York, NY: William Morrow.
Botvinick, M., & Cohen, J. (1998). Rubber hands’ “feel” touch that eyes see. Nature,
391(6669), 756.
Clark, A. (2007). Re-inventing ourselves: The plasticity of embodiment, sensing, and
mind. Journal of Medicine and Philosophy, 32, 263–282.
Ehrsson, H. H. (2007). The experimental induction of out-of-body experiences. Science, 317, 1048. doi:10.1126/science.1097011
Fox, J., & Bailenson, J. N. (2009). Virtual self-modeling: The effects of vicarious reinforcement and identification on exercise behaviors. Media Psychology, 12, 1–25.
Haans, A., & IJsselsteijn, W. A. (2012). Embodiment and telepresence: Toward a comprehensive theoretical framework. Interacting with Computers, 24(4), 211–218.
Hershfield, H. E., Goldstein, D. G., Sharpe, W. F., Fox, J., Yeykelis, L., Carstensen, L.
L., & Bailenson, J. N. (2011). Increasing saving behavior through age-progressed
renderings of the future self. Journal of Marketing Research, 48, S23–S37.
IJsselsteijn, W. A., de Kort, Y. A., & Haans, A. (2006). Is this my hand I see before me?
The rubber hand illusion. In reality, virtual reality, and mixed reality. Presence:
Teleoper. Virtual Environment, 15(4), 455–464.
Kilteni, K., Normand, J.-M., Sanchez-Vives, M. V., & Slater, M. (2012). Extending body
space in immersive virtual reality: A very long arm illusion. PLoS One, 7(7), e40867.
doi:10.1371/journal.pone.0040867
Lotze, M., Flor, H., Grodd, W., Larbig, W., & Birbaumer, N. (2001). Phantom movements and pain. An fMRI study in upper limb amputees. Brain, 124(11), 2268–2277.
Ophir, E., Nass, C., & Wagner, A. (2009). Cognitive control in media multitaskers.
Proceedings of the National Academy of Sciences, 106(33), 15583–15587.
Pea, R., Nass, C., Meheula, L., Rance, M., Kumar, A., Bamford, H., … & Zhou,
M. (2012). Media use, face-to-face communication, media multitasking and social
wellness among 8–12 year old girls. Development Psychology, 48(2), 327–336.

14

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

Penfield, W., & Boldrey, E. (1937). Somatic motor and sensory representation in the
cerebral cortex of man as studied by electrical stimulation. Brain, 60, 389–443.
Ramachandran, V. S., & Altschuler, E. L. (2009). The use of visual feedback, in particular mirror visual feedback, in restoring brain function. Brain, 132(7), 1693–1710.
Ramachandran, V. S., & Rogers-Ramachandran, D. (1996). Synaesthesia in phantom
limbs induced with mirrors. Proceedings of the Royal Society of London, Series B: Biological Sciences, 263(1369), 377–386.
Ratan, R. A. (2012). Self-presence, explicated: Body, emotion, and identity extension
into the virtual self. In R. Luppicini (Ed.), Handbook of research on technoself . IGI
Global: New York, NY.
Roricht, S., Meyer, B. U., Niehaus, L., & Brandt, S. A. (1999). Long-term reorganization of motor cortex outputs after arm amputation. Neurology, 53, 106–111.
Sato, K., Fukumori, S., Matsusaki, T., Maruo, T., Ishikawa, S., Nishie, H., ... Morita,
K. (2010). Nonimmersive virtual reality mirror visual feedback therapy and its
application for the treatment of complex regional pain syndrome: an open-label
pilot study. Pain Medicine, 11(4), 622–629.
Schaefer, M., Flor, H., Heinze, H. J., & Rotte, M. (2007). Morphing the body: illusory
feeling of an elongated arm affects somatosensory homunculus. NeuroImage, 36(3),
700–705.
Schmalzl, L., Ragnö, C., & Ehrsson, H. H. (2013). An alternative to traditional mirror
therapy: Illusory touch can reduce phantom pain when illusory movement does
not. The Clinical Journal of Pain, 29(10), e10–e18.
Slater, M., Spanlang, B., Sanchez-Vives, M. V., & Blanke, O. (2010). First person experience of body transfer in virtual reality. PLoS One, 5(5), e10564.
Steptoe, W., Steed, A., & Slater, M. (2013). Human tails: Ownership and control of
extended humanoid avatars. Visualization and Computer Graphics, IEEE Transactions
on, 19(4), 583–590.
Won, A. S., & Collins, T. A. (2012). Non-immersive, virtual reality mirror visual
feedback for treatment of persistent idiopathic facial pain. Pain Medicine, 13(9),
1257–1258.
Won, A. S., Lee, J. D., Bailenson, J. N., & Lanier, J. (in press). Homuncular flexibility
in virtual reality. Journal of Computer-Mediated Communication.
Yee, N., & Bailenson, J. (2007). The Proteus effect: The effect of transformed
self-representation on behavior. Human Communication Research, 33(3), 271–290.
Yee, N., Ducheneaut, N., Yao, M., & Nelson, L. (2011). Do men heal more when in
drag? Conflicting identity cues between user and avatar. Proceedings of CHI 2011,
pp. 773–776.

ANDREA STEVENSON WON SHORT BIOGRAPHY
Andrea Stevenson Won received her MS in Biomedical Visualization from
the University of Illinois at Chicago in 2005. She is a PhD candidate in the
Department of Communication at Stanford University. Her research interests

Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars

15

include mediated self-representation and tracking, assessing, and manipulating body movements to affect outcomes. She is a member of the International
Communication Association. Contact her at aswon@stanford.edu.
JEREMY N. BAILENSON SHORT BIOGRAPHY
Jeremy N. Bailenson is founding director of Stanford University’s Virtual
Human Interaction Lab, an associate professor in the Department of Communication at Stanford, and a senior fellow at the Woods Institute for the
Environment. He earned a BA cum laude from the University of Michigan
in 1994 and a PhD in cognitive psychology from Northwestern University in
1999. After receiving his doctorate, he spent 4 years at the Research Center
for Virtual Environments and Behavior at the University of California, Santa
Barbara as a post-doctoral fellow and then an assistant research professor.
Bailenson’s main area of interest is the phenomenon of digital human
representation, especially in the context of immersive virtual reality. He
explores the manner in which people are able to represent themselves when
the physical constraints of body and veridically rendered behaviors are
removed. Furthermore, he designs and studies virtual reality systems that
allow physically remote individuals to meet in virtual space, and explores
the manner in which these systems change the nature of verbal and nonverbal interactions. In particular, he explores how virtual reality can change the
way people think about education, environmental behavior, and health.
His findings have been published in over 90 academic papers in the fields
of communication, computer science, education, law, political science, and
psychology. His work has been consistently funded by the National Science
Foundation for over a decade, and he also receives grants from various Silicon Valley and international corporations. He consults regularly for government agencies including the US Army and Air Force, the Department of
Defense, the Department of Energy, the National Research Council, and the
National Institute of Health on policy issues surrounding virtual reality.
His book Infinite Reality, coauthored with Jim Blascovich, was recently
quoted by the Supreme Court outlining the effects of immersive media.
JARON LANIER SHORT BIOGRAPHY
Jaron Lanier is a member of Microsoft Research. He either coined or popularized the term virtual reality and in the early 1980s founded VPL Research,
the first company to sell VR products. He led the team that developed the
first implementations of multi-person virtual worlds using head-mounted
displays, for both local and wide area networks, as well as the first “avatars,”
or representations of users within such systems. While at VPL, he and his colleagues developed the first implementations of virtual reality applications in

16

EMERGING TRENDS IN THE SOCIAL AND BEHAVIORAL SCIENCES

surgical simulation, vehicle interior prototyping, virtual sets for television
production, and assorted other areas.
Harvard’s Kennedy School of Government chose Lanier’s book “Who Owns
the Future?” for 2014s Goldsmith Award. He received a Lifetime Career
Award from the IEEE in 2009 for contributions to virtual reality, and was
the recipient of CMU’s Watson award in 2001. He has received honorary
doctorates from the New Jersey Institute of Technology and Franklin and
Marshall College. He has also been affiliated with USC, UC Berkeley, NYU,
Columbia, UPenn, and Internet2. He has written best-selling books, such
as “You Are Not a Gadget.” He is also a composer of “classical” music (with
commissions from the St. Paul Chamber Orchestra and others) and plays
a large number of instruments (having performed with Philip Glass, Yoko
Ono, George Clinton, Ornette Coleman, and many others.)