eScience Lectures Notes : Introduction to Virtual Reality


Slide 1 : Virtual Reality

Virtual Reality

Introduction

First Examples

First Examples (2)

First Examples (3)

First Examples (4)

History : terminology and popularisation

Prehistory

What it is not

Proposition for a technical definition

Purpose

Applications....

Application Examples 1 : IAr

Application Examples 2: IAi

Main Scientific Issues

Information Transfert within the Human Body

Five senses... but they were 7... or more ?

Anthropocentric User-Diagram

Human Behavior in a virtual world (3 Levels)

First Level of Immersion / Interactivity

Sensory Interface

Motor Interface

Sensorimotor Interface

Second Level : Mental Immersion / Interactivity of the user

Sensorimotor Schema

Abstraction level

On the Virtual World Side

Second Level : Mental Immersion / Interactivity of the user with comments

Design of VR Project

Second Level Graphe

Complete (with Third Level) Technocentric Designer Diagram

 

Final evaluation of a virtual world

Glossary


Source of information for this chapter

Le traité de la réalité virtuelle
By Philippe Fuchs, Guillaume Moreau, Jean-Paul Papin
Préfacé par Alain Berthoz, Professeur au Collège de France
http://www.ensmp.fr/Fr/Services/PressesENSMP/Collections/ScMathInfor/Livres/TraiteRealiteVirtuelle.html

INTRODUCING VIRTUAL ENVIRONMENTS / NCSA : http://archive.ncsa.uiuc.edu/Cyberia/VETopLevels/VR.Overview.html

Virtual Reality: A Short Introduction by K.-P. Beier : http://www-vrl.umich.edu/intro/index.html


Slide 2 : Introduction

Introduction to Virtual Reality

First let's have a look at a little video.

For the display device presentation, have a look at the Computer Graphics matching presentation

What is VR ?

Some idea...

"The experience of venturing inside a computer-generated "virtual" world, of immersion in data, is what happens in "virtual reality" or "VR." Through various display devices, data are transformed into 3D images you not only see but also hear or even touch."

"Virtual Environments: New Ways to Perceive and Explore Data..."

"In immersive VR, the user becomes fully immersed in an artificial, three-dimensional world that is completely generated by a computer."

"An artificial environment created by computers, in which people can immerse themselves and feel that this artificial reality really does exist." (Whatis.com, USA).

Various names : Virtual Worlds, Virtual Environments, Immersive VR...

Cyberspace. A consensual hallucination experienced daily by billions legitimate operators, in every nation, by children being thaught mathematical concepts... A graphic representation of data abstracted from the banks of every computer in the human system.
Unthinkable complexity. Lines of light ranged in the nonspace of the mind, clusters and constellations of data. Like city lights, receding...

Neuromancer by William Gibson

 


Slide 3 : First Examples

First Examples

See at a different scale, at a different time...

National Center for Supercomputing Applications (NCSA)

Become smaller than an atom or larger than the universe. Journey back in time and space to the birth of galaxies, watch ripples of gravity as black holes collide or travel through the human bloodstream. Build a city of cathedrals, mosques, and pyramids or listen to the sounds of chaos. All of these are possible within virtual environments.

 


Slide 4 : First Examples (2)

First Examples (2)

From abstract data visualisation to data exploration and to interaction with data

Human Interface Technology Laboratory

Computer visualization is reshaping how scientists evaluate and explore their data and promises to transform how students learn. Within the past four decades, dramatic advances in graphic display and high-end computing have made it possible to transform billions of bits of data into interactive, three-dimensional images that you can manipulate in real time. Here, all the display and feedback devices that make this possible are collectively termed "Virtual Environments. "

 


Slide 5 : First Examples

First Examples (3)

To Build or to Sell ?

Detroit Midfield Terminal Project

 

 


Slide 6 : First Examples 4

First Examples (4) : Training

Ecole des Mines de Paris / SNCF

 

 


Slide 7 : History

History : terminology and popularisation

The term 'Virtual Reality' (VR) was initially coined by Jaron Lanier, founder of VPL Research (1989). Other related terms include 'Artificial Reality' (Myron Krueger, 1970s), 'Cyberspace' (William Gibson, 1984), and, more recently, 'Virtual Worlds' and 'Virtual Environments' (1990s).

THE LAWNMOWER MAN : 1992


* * * STARRING: Jeff Fahey, Pierce Brosnan, Jenny Wright, Mark Bringleson, Geoffrey Lewis, Jeremy Slate, Dean Norris
1992, 105 Minutes, Directed by: Brett Leonard

Oh dear. Virtual reality finally hits the big screen - but not exactly in the way sci-fi fans or virtual reality boffins would have liked it. VR boffins complained that Lawnmower Man created unrealistic expectations amongst the public of what to expect from their "LSD of the 1990s" (as once 'Sixties drug guru Timothy Leary dubbed it - talk about creating unrealistic expectations!). Sci-fi fans complained that the movie was mostly cack.
They had it right: the plot, a mishmash of horror and the latest in technological buzzwords, doesn't quite make the grade. Effects wise, this film deserves all the accolades it can get. The computer graphics are spectacular and this movie is the first time cyberspace really makes it to the big screen. However, one cannot help but leave the cinema feeling cheated, feeling that all those incredible graphics might have been better served by a better screenplay - perhaps William Gibson's Neuromancer...

 

JOHNNY MNEMONIC : 1995 ( From William Gibson : "Neuromancer" : 1984)


* * * STARRING: Keanu Reeves, Dolph Lundgren, Takeshi, Ice-T, Dina Meyer, Udo Kier, Denis Akiyama, Henry Rollins, Tracy Tweed, Don Francks
1995, 98 Minutes, Directed by: Robert Longo

Cyberpunk godfather William Gibson's work finally makes it to the big screen. Unfortunately it is too late and one wonders what would have happened had his Hugo award-winning Neuromancer been made into a movie shortly after its publication. The sad truth is that we have already had buzzwords such as virtual reality and cyberspace explained in (albeit lesser) films such as

A far more general and openminded Timeline

http://www.artmuseum.net/w2vr/overture/looking.html

The same, put in images, and in a still more open context : The cybernetics

http://www.artmuseum.net/w2vr/timeline/timeline.html

Summary

 


Slide 8 : Prehistory

Prehistory

 


Slide 9 : What it is NOT

What it is NOT

When 'Virtual Reality' is used in a confusing and misleading manner...

Usual image :

What the media usually show : a person with a visiocasque and a dataglove...


From : The Virtual Reality Homebrewer's Handbook by Robin Hollands


Photo by W. Fisher / S. Fisher
VIEW (Virtual Interface Environmental Workstation)
Courtesy of Scott Fisher, NASA-Ames Research Center

The Lawnmower Man

Synopsis:
A slow-witted teenager develops a super intellect when he becomes a guinea pig for a scientist experimenting with virtual reality.

1992, 105 Minutes, Directed by: Brett Leonard

JOHNNY MNEMONIC


* * STARRING: Keanu Reeves, Dolph Lundgren, Takeshi, Ice-T, Dina Meyer, Udo Kier, Denis Akiyama, Henry Rollins, Tracy Tweed, Don Francks

1995, 98 Minutes, Directed by: Robert Longo

 

The book :

Neuromancer

By the Cyberpunk godfather William Gibson

It could be, but it is not "only that", nor "always that"


Head-Mounted Display (HMD)

 


Slide 10 : TechnicalDefinition

Proposition (Philippe Fuchs) for a

Technical definition

The man/user : observer and actor (meaning he has a action on...)

Something to explore, fictitious or not

User action, Navigation; RT : the user perception

The user should perceive the delai between its action (motor) and the sensory response from the system. The max delai is about 100 ms (depends on wich sens)(25 fps <=> 40ms)

Feeling of being somewhere

The "natural" is what you have learnt in the real life... Even if we once achieve to reach the same level of complexity (and some think it is not possible), one may wonder if we really need to reach the perfection to achieve effiency in regards to the expected result.

Input and Output, sensory-motor interface, Metaphores

To make a difference from the set "screen, keyboard and mouse" which match with the Human Computer Interface field...

A Behavioral Interface is an apparatus that involve a human behavior, natural and without (or with short) learning period.

All the senses may be taken into account, not all are needed for all the applications

NB : No "Why" (Answer : the Applications)

Other definitions often mixt purpose, what it is and what is is for (applications). Moreover, one should not define VR by the tools it uses (Head mounted display, dataglove...)

Not these all 4 points will be perfectly done in all VR application, but the 4 should appear if we want tobe able to talk about a VR project.


Slide 11 : Purpose

Purpose of Virtual Reality

To allow one or more participants to get involved into sensorimotor activities and therefore mental ones within an artificial world which is

Which aspect will depend on the application. The hiher degree of realism is not always what should be targeted. Less realism but better (intuitif and/or interaction

NB1 : This is not the answer to the question what is the use of it.

NB2 : Fictitious does not mean that this would be only for game (teaching/training situation).

NB3 : In Augmented Reality the artificial world is what is added.

 

The Purpose of VR is to allow one or several people to do sensorimotor, and thus mental, experiments in an artificial world, which is either imaginary, or a simulation of some aspects of the real world.


Slide 12 : Applications

Applications

The Why Question, what sort of things we want to do with VR

Let's propose a taxonomy

VR is of no interest at all if you strictly and exactly reproduce each aspect of the real world.

You modify at least one of these 3 main aspects

Time

Place

"Realistic vs Fictif" Artificial World and InterAction

NB


Slide 13 : Application examples 1 : IAr

Application Examples 1 : IAr

IAr.T0.L0 : Virtual Activity

Training or fun. Game, sport, surgery, army

IAr.T0.L-> : Virtual Transfer

Tourism

IAr.T0.L~ : Virtual Microscope or Telescope

Science

IAr.T+.L0 : Virtual Design

Architecture, Fashion, Product Design

IAr.T-.L-> : Virtual Event

Archeolgy, Police Investigation

 


Slide 14 : Application examples 2 : IAr

Application Examples 2: IAi

IAi.T0.L0 : Virtual Creation

Art, Database visualisation...

IAi.T0.L-> : Fictif navigation

Structure (Directory Tree) visualisation. FSN !!!

IAr.T0.LU : Televirtuality

IVR, where your being is replaced by an avatar

 


Slide 15 : Main Scientific Issues

Main Scientific Issues

To allow one or more participants to get involved into sensorimotor activities and therefore mental ones within an artificial world which is

 


Slide 16 : Information Transfert within the Human Body

Information Transfert within the Human Body


Slide 17 : 5, 6, 7 ... senses

Five senses... but they were 7... or more ?

hearing

sound

ears,body


The ability to hear; the auditory faculty; SYN. audition, auditory sense, sense of hearing, auditory modality.
ouie

sight

image

eyes

The ability to see; the faculty of vision; SYN. vision, visual sense, visual modality. vue

touch

surface / temperature

skin

The faculty of touch; SYN. sense of touch, skin senses, touch modality, cutaneous senses. toucher

smell

odour

nose

The faculty of smell; SYN. sense of smell, olfaction, olfactory modality. odorat

taste

savour, flavour

tongue/nose

The faculty of taste; SYN. gustation, sense of taste, gustatory modality. gout

kinesthesia

position, movement, muscular tensions

muscles

The perception of body position and movement and muscular tension etc; SYN: kinaestesia, feeling of movement kinesthesie

proprioception

balance, acceleration, position, location, orientation, movement of the body

ear

The ability to sense the position and location and orientation and movement of the body and its parts. proprioception

 

 


Slide 18 : Anthropocentric User-Diagram

Anthropocentric User-Diagram

Or "how we perceive the world and how I am in the world"

Or "I feel therefore I act" ;-)

Immersion

 

 

Input

Interactivity

 

 

Output

 

 


Slide 19 : Human Behavior in a virtual world

Human Behavior in a virtual world

...should follow the same principle than in a real world :

Immersion and Interactivity

3 level of Analysis


Slide 20 : First Level

First Level of Immersion / Interactivity

Low level, Physical level

Because the computer is physically connected to the human body by its senses and its motor responses.

The physical devices, Which interface to use ?

You have to know that not only Sight interface exist (HMD...), but lots of other,l you should have an idea of all the others

What Hardware (interface, computer) ?

What Software (Driver, Toolkit, Programming Language...) ?

TECHNOCENTRIC DESIGNER DIAGRAM

NB : By now (and still in the future), human characteristics are higher than what the technical VR device may provide to the user...

Notion of Transparency : with (touch) or without (sight) material medium

Target : try to minimise the delay for the feedback loop : "Action / Perception"

Are the metrology characteristics in adequation with the psychophysics characteristics ?

(example of the Head Mounted Display)

 


Slide 21 : Sensory Interface

Sensory Interface


Slide 22 : Motor Interface

Motor Interface


Slide 23 : Sensorimotor Interface

Sensorimotor Interface

Usually, force feedback interface (kinesthesia sens)

Receptor inside the muscles

Another example : eye driven pointer


Slide 24 : Second Level of Immersion / Interactivity

Second Level : Mental Immersion / Interactivity of the user

Triadic diagram for the behavioral interfacing


Slide 25 : Sensorimotor Schema

Sensorimotor Schema

What is natural seems simple and is has already been learnt

For Piaget, a schema is a mental structure which enables to repeat the same action or to apply it to new actions.

(a behavioural brick...)

Piaget : specialist in Child learning behaviour, the birth of the intelligence 1979

Imagine that you say to somebody that if he wants to stop his computer, he first has to clic on a start button... that is not natural at all, it is counterintuitive !

There is in it some parameter that describe a general situation where you may want to use that schema

Ex : When you grab an object, the same scheme may be used in different situation

The behavioural interface is a mixed entity :


Slide 26 : Different levels of abstraction

Different levels of abstraction

In VR Schema are often slightly modified

Ex : when you grab and move around a position tracker instead of an object and you see the object on the screen, not in your hand.

Schema by substitution

when you hear a sound instead of touching or when a color tell you that it is hot.

Metaphore

When color tell you that it is dangerous...

When when you show where you want to go (The superman Metaphor)

When you say to the user, to get that, you should do that, because it is as if you were in such another situation

Driving a car could either schema, a schema by subsitution (you say : "drive me to the bigest house") or a metaphore ("use the keyboard to steer and change the speed and you will move as if driving a car")

When using what ?

The best is of course the Schema (Natural immersion), but it is not always possible or of good quality, and often the other solutions are then better (mor efficient)

When a metaphore become a schema...

From the desktop to the web crawling ("clic on the blue text to get" or ... to "get the page") and to the move around...


Slide 27 : On the Virtual World Side

On the Virtual World Side

BSA = Behavorial Software Assistance

When the virtual world guess what you are trying to do and help you to do it

When you play a card game and the software put the card exactly at the right position, even if you put it 1 mm away.

Example : icon alignment

VBP = Virtual Behavioural Primitive

Everything that you will ever do in a VR is a combination of one of the following actions / VBP


Slide 28 : Second Level of Immersion / Interactivity

Second Level : Mental Immersion / Interactivity of the user

Triadic diagram for the behavioral interfacing

 


Slide 29 : Second Level Graphe

Second Level Graphe

BPVR, VBP = navigation, object handling, visual observation, locomotion, orientation, etc.

BSA = Behavorial Software Assistance

 


Slide 30 : Technocentric Designer Diagram

Technocentric Designer Diagram


Slide 31 : Final evaluation of a virtual world

Final evaluation of a virtual world

Have you had the feeling that you were completly immerse inside another world ?

VS.

Did the user behave the same way (or with enough efficiency) than in the real world, regarding what we where looking at ?

(Move, Act is different from behave : result centric)


Slide 32 : Glossary

Glossary

French English Comment (Most of the time, "Ultralingua dictionary" definition)
Interface Interface

+. (Computer science) Hardware and associated circuitry that links one device with another (especially a computer and a hard disk drive or other peripherals);
+. (Computer science) A program that controls a display for the user (usually on a computer monitor) and that allows the user to interact with the system;

In VR field, it is a bit more complex, and it is one of the subject of this lecture to define it.

A tool that allows a person to interact with a computer. For example, a mouse is an interface device that allows you to put information into a computer. Virtual reality includes interface devices such as head-mounted displays that transmit sensations of the artificial world, as well as transmitting information into the computer.

Interfacage Interfacing The fact to set up a interface
Sensorial Sensory Involving or derived from the senses; "sensory experience"; "sensory channels"; SYN. sensorial.
Sensorimoteur Sensorimotor Of or relating to the sensory and motor co-ordination of an organism or to the controlling nerves.
Motrice Motor Involving or relating to movements of the muscles; "motor co-ordination"; "a motor reflex"; "motor paralysis."
Comportemental Behavioural Of or relating to behaviour; "behavioural sciences";
Comportement Behaviour The action or reaction of something (as a machine or substance) under specified circumstances; "the behaviour of small particles can be studied in experiments";
Teleoperation Teleoperation, Remote Control Remote control of robot manipulators
Simulation Simulation. A computer model of a real phenomenon or system. The system is described by a set of mathematical formulae or models in a computer program. Running the computer program shows how the system works and, by changing variables, it is possible to make predictions about how the real system will change. When there are many variables, simulation is often the only way to reasonably predict an outcome.
Interaction Interaction A mutual or reciprocal action; interacting.
Interactivite Interactivity The quality of that which is interactive.
Effecteur Effector
1. A nerve fiber that terminates on a muscle or gland and stimulates contraction or secretion.
2. An organ that becomes active in response to stimulation.
metrologie metrology The science of, or a system of, weights and measures; also, a treatise on the subject.
psychophysique psychophysics The branch of psychology concerned with quantitative relations between physical stimuli and their psychological effects.
scheme schema An internal representation of the world; an organisation of concepts and actions that can be revised by new information about the world.

artefact

artifact

A man-made object;