Steuer- Defining virtual reality: dimensions determining telepresence

Steuer, J. (1992). Defining virtual reality: dimensions determining telepresence. Journal of Communication: Autumn 1992; 42(4), 73- 93.
Nice article, seems to be an early contribution to the now growing field of “presence” research, and also, there is a mention of the cognitive load issue towards the end of the article which may be an effect of multimedia that is high in “telepresence”.

Jonathan Steuer is writing in the early 1990’s, but even then, is making the argument to redefine virtual reality in terms of its experience rather than in terms of specialized technology, as it was until then defined. So rather than in terms of the technological hardware of computers, displays, motion-sensing gloves and so on, Steuer defines virtual reality in terms of the experience of “tele-presence”. Drawing from an earlier definition of “presence” as “the sense of being in an environment” (Steuer citing Gibson, 1979, p. 4) and connecting presence to “the phenomenon of distal attribution or externalization, which refer to the referencing of our perceptions to an external space beyond the limits of the sensory organs themselves” (Steuer citing Loomis, 1992). Steuer then proposes the term telepresence to distinguish the mediated aspect of the experience of presence in a virtual environment, “telepresence refers to the mediated perception of an environment” (76). However, researchers have used the term presence and telepresence sometimes interchangeably, and sometimes to denote distinct cases. The distinction seems understandable, but Steuer insists on using telepresence as the broader term that incorporates all cases.

Now Steuer feels free to define the term, virtual reality (VR): “A virtual reality is defined as a real or simulated environment in which a perceiver experiences telepresence” (76-7). Again, Steuer is trying to make a broader case for the terms so that various technologies can be included, not just the specialized technologies typically associated with VR. Examples included: the experience of using the telephone for the first time, the experience of listening to live recordings of music, the experience of video game play.

Two factors of the medium determine telepresence
Citing Durlak (1987), Steuer writes that “Face-to-face interaction with other humans is used as a model for all interactive communication” (79-80). VR environments need to engage the same processes invoked in the perception (mental and sensory) of real-world environments in order to re-create a sense of telepresence. The medium needs to convey these perceptual processes, and so factors that influence the sense of telepresence include two factors of vividness (sensory richness) and interactivity (ability to act on the environment).
More precisely, vividness is the “the representational richness of a mediated environment as defined by its formal features; that is, the way in which an environment present information to the senses” (81). Vividness is made up of “sensory breadth, which refers to the number of sensory dimensions simultaneously presented, and sensory depth, which refers to the resolution within each of these perceptual channels” (81). And here, redundancy of sensory information presented over multiple channels serves moreover to enhance vividness. Newer technologies such as hi-def TV for example provide greater depth of sensory information. And Steuer looks forward to a day when media systems are capable of representations that are so close to the real thing that the distinction between representation and reality becomes blurred.

And in fact, although we are not there yet, the blurring has already started over the synapse of experienced worlds- Lemke’s notion of “traversals” seems useful here to appreciate the kind of blurring that happens as people become immersed in moving back and forth between virtual and physical realities- SecondLife comes to mind- the traversals are the boundary over which the distinction between representation and reality have become blurred… perhaps?

Interactivity “refers to the degree to which users of a medium can influence the form or content of the mediated environment” (80) and can be further understood in terms of degrees of speed, range and mapping. Speed has to do with how responsive the system is to the user’s actions. Range refers to how many possibilities for manipulation there are in the mediated environment, so intensity (loudness, brightness, etc.), spatial organization (where objects appear, etc.), temporal ordering, and so on. Thus, by these definitions, a book is not that interactive. Mapping refers to Donald Norman’s term, and how closely actions taken on the mediated environment are mapped to corresponding “natural” actions in the human physical environment. Actions in the mediated environment that map closer to real world counterparts contribute to a sense of telepresence.

Constraints on telepresence
Of course, the experience of presence/telepresence is internal to individuals and thus constrained by the particularity of individual sensibilities and situations that individuals are located in. Steuer gives examples of how one sensory modality might be more important in some situations than in others, and yet depending on the individual also, some sensory modalities may be more important than others for the experience to be meaningful and comprehensible. The variabilities of situation and individual will always constrain the degree of telepresence experienced through the given media.

Implications for designing educational media
Steuer injects a paragraph towards the end of the article for another constraint or concern (though he doesn’t name it as such here) which has bearing on the use of immersive technologies for educational purposes, and it is essentially the concern for cognitive load: it is that although the dimensions of vividness and interactivity seem to contribute to the sense of presence, sometimes the vividness may overwhelm the ability to engage in more “reflective” cognitive processing. Hmmm…

Rieber- Discovery learning, Referential processing, Multimedia explanations, Simulations

Rieber et al. look at computer-based simulations, and how to promote referential processing through supplementing the simulations with brief multimedia explanations. Their findings support the usefulness of brief multimedia explanations coupled with appropriate representational feedback in the simulation to facilitate more referential processing and deeper learning. I find the following concepts from this article to be especially important to keep in mind for the design of any simulation-based educational media:

1) The cognitive processing called for through computer-based simulations, especially when they are “video game-like”, is primarily VISUAL (see Dual Coding Theory)

2) The necessity for timely feedback to facilitate reflective thinking.

3) The affordances of computer-based simulations to engage the learner in experiential, discovery-oriented, “interactive” learning.

4) The importance of the form of representation (in the design of the interface) for promoting engagement and or reflection.

The notion of referential processing comes from Pavio’s “Dual Coding Theory” (DCT). DCT proposes two processing systems in the brain for visual and verbal information. Within and between both systems, or “channels”, there are different levels of processing- representational, associational, and referential. Representational processing takes place in each respective channel depending on the representational nature of the information (visual or verbal), and associational processing also takes place in each channel (again in relation to the representation of information), but referential processing moves between both channels and functions to relate coding between the two channels. When information is processed in this way within and between both channels, there is a greater chance for retrieval and usage of the information.

An implication of DCT is thus that the way information is represented then will greatly affect the processing of that information; and for this reason, the representation of information becomes an important issue in the design of educational materials. Computer-based simulations can be used in education, and their interactive affordances can provide ways for students to learn more difficult conceptual material through experiential learning scenarios, they enable the student to engage in “discovery learning.” Computer-based simulations may be especially effective for learing complex systems like physics by letting a student focus first on conceptual understanding and approach the material through experiences rather than explanations. However, simulations that are purely exploratory may not provide enough support for referential processing as the representational nature of simulations is overwhelmingly VISUAL. Thus, this is where Rieber et al. write that “the “video game-like” quality of the simulation may have interfered with referential processing by only promoting processing in the visual system and discouraging processing in the verbal system” (309). Being immersed in the overwhelmingly visual realm of the simulation seems to disallow students from the kind of reflection needed for referential processing.

Also, another important concept is the importance of FEEDBACK:

“One of the most important considerations in a simulation’s interface design is how to provide meaningful feedback to the user… Given the way computers can represent feedback in a simulation, research is needed to ensure that design decisions are made based on the psychological needs of the individual user and not simply on what the computer is capable of doing” (308).

Rieber’s study proposed to supplement a simulation (on physics concepts) with five short and embedded explanations. The explanations themselves were then provided to the student either as graphical feedback or textual feedback. The feedback (whether graphical or textual) was provided as brief explanations of the scientific principle being modeled after the student had an oppportunity to interact with the simulation, but before they mastered the content. Findings indicated that students had the best scores for explicit learning (which implies better referential processing) when using the simulations embedded with graphical feedback.

Wittrock- Generative Learning Processes of the Brain

Wittrock presents a model for generative learning that reflects neural research into how knowledge is generated in the brain. Neural research has been showing how the brain generates knowledge by building it, in other words, it is not just an information processing unit. The brain operates through processes of generating meaningful relations “among concepts and between knowledge and experience” (531). And accordingly, Wittrock’s model informs teaching, which “becomes the process of leading learners to use their generative processes to construct meanings and plans of action” (531).

Wittrock’s model for generative learning goes from the neural research on generative learning processes in the brain to propose a corollary and functional model for generating learning processes in classrooms. And it is quite interesting to go from the neural research to forming a hypothesis about how people learn in the physical & social world, and then to testing the hypothesis. I’ve wondered myself how to go from my interest in the neuroscience of “embodied cognition” to forming a research hypothesis that applies the theory to educational contexts.

However, I think I’m missing something? Other than the brain research that informs it, I found the idea that the process of “generating knowledge” rather than just receiving it, to not be a radical model for how people learn. This sounds more like a scientific approach to proving something that most people (teachers) already know? But I’m simplifying. The research based on the model of generative learning has been constructed to test whether learning through constructing relationships (e.g. analogies, using graphs, constructing summaries) with the content resulted in greater comprehension than without. Not surprisingly, in most of the studies, the groups that engaged in constructing relationships between and among concepts in the content had greater learning gains.

Perhaps Wittrock does provide more than just this general hypothesis. He also clarifies and foregrounds the importance of what he calls four major processes that are involved in generating learning: Attention, Motivation, Knowledge & preconceptions, and Generation. And it does seem that more effort is still needed to dispel the view of learning as “information processing”- and move the dominant view of learning forward into an understanding of its dynamic, distributed, and emergent nature.

I also liked the explanations of relevant brain research. Wittrock describes Luria’s three functional units of the brain in detail and relates them to an educational context-

Luria’s first functional unit of the brain involves arousal and attention and “is influenced by the cortex and by conceptually driven behavior “, in the context of learning, this means that the attention and arousal within the learner is influenced by the plans and intentions of the learner.

The second unit functions to receive, analyze and store information from all of the senses. The verbal & spatial, propositional & appositional, and analytic brain mechanisms for learning and understanding information function here.

Luria’s third functional unit in the frontal lobes functions as a generative processor & integrator of the brain’s generative functions- it engages in the planning, organizing and regulating of cognition & behavior.

the one reference to how sensory stimuli are actually worked on, modified by the brain even before it is processed in the specific sensory areas of the cortex- and doesn’t this seem to go against the model proposed by Norman of cognitive processing happening through 3 different levels (sensory/visceral, behavioral, and reflective).

Schneiderman & Plaisant—Designing the User Interface, Chapt. 2

An exhaustive overview of guidelines, principles and theories for designing interfaces. Within this overview, the set of theories that focus on “context of use” and draw upon their relation to newer and more ubiquitous technologies are especially interesting for me.

There are 3 levels of consideration- 1) guidelines for practice, standards and concrete principles for application that have been developed through experience; 2) middle level principles, more widely applicable, fundamental and enduring principles that govern the design ; 3) high level theories, those that are descriptive of principles of design, and those that are predictive, providing rules and measures for outcomes.

Examples of Guidelines-
• To promote accessibility (provide text element for every non-text element, etc.)
• To organize the display (high level objectives to organize the display include, consistency of formats, fonts, colors, and so on; minimum memory load on user; compatibility of data input with data display; flexibility for user control of display, etc.)
• For getting the user’s attention (ex., intensity- use 2 levels, reserving the high intensity for important points; color- use up to 4 standard colors, etc.)
• For facilitating data entry (ex., consistency of data entry transactions; minimal memory load on users, etc.)

Middle Level principles- More fundamental principles but for which there may be many solutions. Looking here especially at the principle of universal usability and how to recognize the diversity among users.

Some middle level considerations:
• Accommodating user skill levels
• Task analysis (task objects & actions, maybe even schematized for different users)
• Interaction/Manipulation style- this can include 1. Direct manipulation, 2. Form Fill-in, 3. Menu selection, 4. Command languages, 5. “natural” languages

• And some middle levelprinciples: THE 8 GOLDEN RULES OF INTERFACE DESIGN-

1. Consistency (strive for consistency)
2. Provide informative Feedback for actions
3. Strive for universal usability
4. Dialog boxes to yield Closure
5. Prevent Errors
6. Allow for Reversability of actions
7. Support Internal Locus of Control
8. Reduce short term Memory Load

Theories-
Some theories are descriptive & explanatory, helping to develop consistent terminologies for objects and actions, while some theories are predictive, enabling designers to improve the capacity and functionality of designs. Another way to group theories is according to “motor task activities, perceptual activities or cognitive aspects”. There are different ways also to group theories. With descriptive theories, there have been systems that analyze the domain by separating concepts into “levels of analysis”…

Widget level theory and Pattern Languages
However, a more interesting approach is the “widget level”, that analyzes use and performance of “widgets” or interface tools. An interface analyzed as a set of organized widgets starts to yield some data and the perforance of future widgets can be predicted and tested. This process of repeated modeling and testing of organizations by widgets then starts to yield some patterns of usage, similar to Christopher Alexander’s idea of a “pattern language” in architecture (1977).

Context of Use Theories
Another group of related approaches, that build upon the idea that an interface can only be meaningful in terms of the patterns of its use, are the “context of use” theories that take more deeply into account the way humans are situated in physical and social environments and how this situatedness affects their interactions, their patterns of use. These ideas gave rise to renewed attention to Computer Human Interaction issues, and a seminal book was perhaps Suchman’s Plans and Situated Action (1987).
With context of use theories, the epistemological orientation is towards distributed cognition. Also users become even more important to the design process as the breakdowns they experience in using the interface are a source of insight about the design. Thus the attention to design and analysis of technology shifts away from the scientific setting into the “turbulence of actual usage” (95) in the context of the social environment.

Interestingly, these context of use theories are especially relevant to newer interface technologies such as physical/ubiquitous computing and embedded devices. “Physical space became an important notion for those who began to think more about ubiquitous, pervasive, and embedded devices. However, they sought to shift attention from place to space, implying that the social/psychological space had to be considered in addition to the physical place (Dourish, 2002). These notions are likely to become still more important as varied sensors become more common.” (94).

Betrancourt—Animation and Interactivity Principles in Multimedia Learning

An entirely useful article both analyzing the use of animations in educational contexts, and providing principles around how (design principles) and when (cognitive implications) to use animations in education.

Introduction-  “Animations” are now commonly used in multimedia-based education, and yet Betrancourt argues that animations are often applied without a clear sense of how they faciliate learning.  In this article Betrancourt provides guidelines for three main ways of addressing the central question of  “When and how should animation be used to improve learning?” (288).  These are: 1) Animation can be used to support the visualization and mental representation process, 2) to produce a cognitive conflict, and 3) to enable learners to explore a phenomenon.

1) Animation to support visualization and mental representation—This is not so different from how graphics are used but because of animation’s dynamic nature, dynamic phenomena can be visualized and represented more easily.

2) Producing cognitive conflict—Animation can offer a view of phenomena that go against conventional conceptions and thus help learners to not only view alternatives, but also the animation helps learners to view and discuss their reactions, thus making their conceptions more explicit.

3) Enable learners to explore phenomena—Because of the possibility of interactivity, an animation can help a learner to explore  phenomena through a discovery-learning approach.

The Animation & Interactivity Principles
First though, Betrancourt separates the definition of “animation” from its capacity for “interaction” in order to understand their connection- because later, she shows how interactivity is one of the key determinants of whether an animation is effective for learning or not.  Betrancourt accordingly identifies two aspects (principles) of animation: 1) its structural definition (animation principle) and 2) its affordance for interactivity (the interactivity principle).

Animation Principle: “computer animation refers to any application which generates a series of frames, so that each frame appears as an alteration of the previous one, and where the sequence of frames is determined by the designer or the user” (288).

Interactivity Principle: “Whereas control is the capacity of the learner to act upon the pace and direction of the succession of frames, interactivity is defined as the capacity to act on what will appear in the next frame by actions and parameters” (288).

What has been found?
Animation may not always make much difference over static pictures in facilitating learning, even when representing dynamic phenomena. What makes a difference is the Interactivity principle. Interactivity may overcome certain perceptual and conceptual obstacles that come up for many in attending to animations in that interactivity allows for user control.  Being able to control the animation (mainly the pace) improved the perception about the material being more enjoyable, AND enable learners to manage their cognitive resources (attention and processing) thus helping to manage the perceptual and conceptual overload.

There may be many other factors determining the effectiveness and limitations of animation and interactivity. Betrancourt surveys some of these, such as the factor of the form of representations (the conveyance of different forms of representational, symbolic, and verbal information), and also the issue of prior knowledge and different visuo-spatial abilities among learners. Animations may also have 3 functions (Schnotz, 2003) with regards to building of mental models in the learner, these are the enabling, facilitating or inhibiting functions.

Implications for Instructional Design- When should animations be used?
1)    When the concept or phenomenon depicted is dynamic.
2)    When learners are novices in a domain so they can’t form a mental model (enabling function), or when the cognitive load they are faced with is high (facilitating function).

Principles to Design With
1)    Apprehension principle- external features should be clear and directly perceived, without any cosmetic features not germane to the content
2)    Congruence principle- changes in the animation need not be realistic, they should be congruent with changes in the conceptual model rather than the phenomenon itself
3)    Interactivity principle- information is better comprehended if learner has control over pace and flow of animation
4)    Attention guiding principle- signal the perceptually salient features of the display (through verbal commentary or signs such as arrows)
5)    Flexibility principle- animation should be available flexibly according to need of learner and information in animation should be clearly described to avoid redundancy between static and animated visual material (??? Don’t understand this principle?)

Dan Saffer—Designing for Interaction, Chapt.s 4 & 5

Chapt.4- Design Research
Design research embodies a collection of methods that enable designers to understand the users and their environments in order to better develop products and applications for them. Moreover, unless the designer is an “intuitive genius”, there is no way for a designer to appreciate or know about so many diverse groups of people, environments and content areas- the only way is to do some research to draw upon the user’s perspective and knowledge.

Some common methods for design research have been drawn from various field such as marketing, theater, anthropology, sociology, and science research. The methodologies can be seen to involve 3 components: 1) Go to where the users are, don’t ask them to come to your lab as you want to be in their natural environment, 2) Talk with them, and then 3) Record your observations, document, write it down.

Some notes: the research should be conducted in an ethical manner that respects privacy, time and effort of the research participants. Research doesn’t have to be a long drawn out process, but designers should observe a representative group of users (between 10 to 40 persons).

Guidelines for Observation:
Look for Patterns- of behavior, responses, explanations, stories, uses. (For a phenomenon to be a pattern, should be seen at least 3 times.)

The Methods:
Observation (Fly on the wall, shadowing, contextual inquiry)
Interviews, talking with users through:
• Directed storytelling,
• The unfocus group,
• Role playing,
• Desk, purse, briefcase tour
Activities, have users engage in an activity:
• Collaging,
• Modeling
• Drawing their experiences
Self-reporting:
• Journals,
• Beeper studies,
• Photo/video journals

Brainstorming

At this point, designers should have all the research and problem definition documents close at hand and in view for reference and inspiration. Then the design group may start through some exercises as warm-ups. Then brainstorming itself during which try to generate as many ideas as possible.

Chapter 6: The Craft of Interaction Design
This chapter gives an overview of how designers work through models and documentation to communicate their vision and understanding of the project, usually at this point, to the client.

Some common methods of “representing research data” are through:
• Flow Models to show a process or problem
• Diagrams to show relationship sets and connections
• Personas to give a sense of how the product might be used by a typical user
• Scenarios to imagine design concepts in use
• Sketches and Models to visualize concepts and ideas that are still being formed
• Storyboards to help illustrate the product or service in use
• Task Analyses- list of all tasks that users will be engaged in
• Task Flow show the logical connections among tasks and steps taken, to show HOW a user accomplishes what s/he wants to with the product or service (flow chart format)
• Use Cases- another way to imagine how a set of users will be using the product
• Mood Boards- collage like display that illustrates the mood, emotional landscape of the product
• Wireframes- documents that show structure, information hierarchy, functionality, and content- the wireframe also conveys the rough form of a product, its navigation controls, content, functionality.
• Prototypes- tangible, to-scale models