Suchen und Finden
The Engineering of Mixed Reality Systems
von: Emmanuel Dubois, Philip Gray, Laurence Nigay
Springer-Verlag, 2009
ISBN: 9781848827332 , 450 Seiten
Format: PDF
Kopierschutz: Wasserzeichen
Preis: 213,99 EUR
Contents
6
Contributors
9
1 Introduction
14
1.1 Mixed Reality Systems: A Booming Domain
14
1.1.1 Variety of Mixed Reality Systems
14
1.1.2 Variety of Application Domains
15
1.2 Mixed Reality Engineering
15
1.2.1 Interaction Design
15
1.2.1.1 Elements of Design
16
1.2.1.2 Specific Design Issues
16
1.2.1.3 Structuring the Design
17
1.2.2 Software Design and Implementation
17
1.2.2.1 Technical Solutions and Interaction Techniques
17
1.2.2.2 Platform: Prototyping, Development and Authoring Tools
17
1.2.2.3 Life Cycle
18
1.2.3 Application of Mixed Reality
18
Part I Interaction Design
20
2 An Integrating Framework for Mixed Systems
21
2.1 Introduction
21
2.2 Illustrative Examples
23
2.3 Integrating Framework for Describing and Classifying Mixed Systems
25
2.3.1 Modeling of a Mixed Object
25
2.3.2 Mixed Object: Intrinsic Characterization
26
2.3.2.1 Characteristics of the Linking Modalities (Devices and Languages)
26
2.3.2.2 Characteristics of the Physical Properties
27
2.3.2.3 Characteristics of the Digital Properties
29
2.3.3 Modeling Mixed Interaction: Putting Mixed Objects into Interaction Context
31
2.3.4 Mixed Object: Extrinsic Characterization
33
2.3.4.1 Characteristics of the Roles
33
2.3.4.2 Characteristics of the Physical Properties
33
2.3.4.3 Characteristics of the Digital Properties
35
2.4 Integrating Framework for Designing Mixed Systems: The Case of Roam
36
2.4.1 Extrinsic Design
38
2.4.2 Intrinsic Design
40
2.5 Conclusion
41
References
42
3 A Holistic Approach to Design and Evaluation of Mixed Reality Systems
44
3.1 Introduction
44
3.2 Related Work
45
3.2.1 Mixed Reality Systems
46
3.2.2 Usefulness
47
3.2.3 Technology in Context
47
3.3 User Involvement in the Development Process
50
3.3.1 The Method Used in the Case Studies
51
3.3.2 The Design Process Used in the Case Studies
51
3.4 The First Case Study An Instructional Task
53
3.4.1 Equipment Used in the Study
54
3.4.2 The User Task
54
3.4.3 Participants and Procedure
55
3.4.4 Results of the Study
56
3.5 The Second Case Study A Collaborative MR Application
57
3.5.1 Equipment Used in the Study
58
3.5.2 The User Task
58
3.5.3 Participants and Procedure
59
3.5.4 Results of the Study
60
3.6 Discussion
62
3.7 Conclusions and Future Direction
63
References
64
4 Embedded Mixed Reality Environments
67
4.1 Introduction
67
4.2 Three Embedded Mixed Reality Environments
69
4.2.1 The MackRoom: Co-visiting an Exhibition from Different Physical Locations
69
4.2.1.1 MackRoom in Use and Experience
71
4.2.2 Mixed Reality Architecture: Flexible Audio-Visual Connections Between Distributed Offices
72
4.2.2.1 MRA in Use and Experience
74
4.2.3 Uncle Roy All Around You: A Mobile Mixed Reality Performance
74
4.2.3.1 URAAY in Use and Experience
76
4.3 Designing for Embeddedness
77
4.3.1 Creating Space for Interaction
77
4.3.1.1 Setting Up the Physical Interaction Space
77
4.3.1.2 Extent of Physical Interaction Space
78
4.3.2 Asymmetries in the Interface Between Digital and Physical Environments
79
4.3.2.1 User Representations
80
4.3.2.2 Spatial Mapping
80
4.3.2.3 Content Mapping
81
4.3.3 Social Interaction in Embedded Mixed Reality Environment
81
4.3.3.1 Role Taking
82
4.3.3.2 Social Rules and Norms
83
4.4 Reflection
84
4.5 Conclusions
86
References
86
5 The Semantic Environment: Heuristics for a Cross-Context HumanInformation Interaction Model
89
5.1 Introduction
89
5.2 A Holistic Framework
90
5.3 From Information Retrieval to HumanInformation Interaction
91
5.4 Resilience
92
5.4.1 Scenario: The Resilient Supermarket
93
5.5 Place
95
5.5.1 Hansel and Gretel or Getting Lost in the Woods
96
5.5.2 Berry-Picking
96
5.5.3 Information Scent
97
5.5.4 Scenario: Sense of Place in the Supermarket
98
5.6 Choice
101
5.6.1 Hick's Law
102
5.6.2 Reducing the Load: Organize and Cluster, Focus, and Magnify
102
5.6.3 Scenario: Choice in the Supermarket
103
5.7 Correlation
103
5.7.1 Scenario: Correlation in the Supermarket
104
5.8 The Semantic Supermarket
105
5.9 Conclusions: Toward a Cross-Context HumanInformation Interaction Model
106
References
107
6 Tangible Interaction in Mixed Reality Systems
110
6.1 Introduction
110
6.2 State of the Art of Tangible User Interface Models
112
6.2.1 The Seminal Tangible Interaction Model
112
6.2.2 The Extended Tangible Interaction Model
113
6.3 Designing Tangible Interaction Techniques in MR Environments
114
6.3.1 Categorizations of Tangible User Interfaces
114
6.3.2 A Multidisciplinary and Participatory Approach
115
6.3.3 Taking into Account the Skills of Users
115
6.3.4 The Design Process
116
6.4 Case Studies
117
6.4.1 A Tangible User Interface for 3D CAD Parts Assembly: ESKUA
117
6.4.2 A Tangible Tabletop for Geoscience: GeoTUI
118
6.4.3 A Tangible User Interface for the Virtual Reassembly of Fractured Archaeological Objects: ArcheoTUI
120
6.4.4 Illustration of the Design Approach on Case Studies
121
6.5 User Studies in the Workplace: Feedback
122
6.5.1 Evaluation: Setup, Metrics, Analysis
122
6.5.1.1 ESKUA
122
6.5.1.2 GeoTUI
123
6.5.1.3 ArcheoTUI
124
6.5.2 Lessons Learnt from the User Studies
124
6.5.2.1 Recommendations Derived from Our User Studies on Tangible Interaction
125
6.5.2.2 Some Questions as a Guide
126
6.6 Conclusion: The Benefits of Tangible Interaction in Mixed Reality Systems
126
References
127
7 Designing a Mixed Reality Intergenerational EntertainmentSystem
130
7.1 Introduction
130
7.2 Related Work
132
7.3 Design Methodology
133
7.3.1 Problem Identification
134
7.3.2 Problem Exploration
135
7.3.3 Design Goals
136
7.4 Design Requirements and Ideas Generation
137
7.4.1 Resources and Time Constraints
137
7.4.2 User Needs
137
7.4.3 Context of Use
138
7.4.4 Design Ideas Generation
138
7.5 Prototype Iterations and System Description
139
7.5.1 Prototype Iterations
139
7.5.2 Current System Architecture
139
7.5.3 Game Play
140
7.6 Intergenerational Player Study
143
7.6.1 Introduction
143
7.6.2 Methods
143
7.6.3 Physical Interface Design Issues
144
7.6.4 Physicality Issues of the Virtual and Physical Player Roles
146
7.6.5 Focus Group Session with Older Players
147
7.7 Conclusion
148
References
149
8 Auditory-Induced Presence in Mixed Reality Environments and Related Technology
151
8.1 Audio in Mixed Realities
151
8.2 Presence and Auditory Displays
154
8.3 Spatial Sound Rendering and Presentation Technologies
155
8.3.1 Multichannel Loudspeaker Reproduction
155
8.3.2 Headphone Reproduction
156
8.3.3 Presentation Systems -- Design Considerations
158
8.3.4 Virtual Acoustics Synthesis and Optimization
158
8.4 Auditory Presence in Mediated Environments: Previous Findings
159
8.4.1 Presence and the Auditory Background
160
8.4.2 Spatial Properties
161
8.4.3 Sound Quality and Sound Content
162
8.4.4 Consistency Across and Within Modalities
163
8.5 Example Scenario: The MR Museum of Music History
164
8.5.1 Displays and Interaction Devices
165
8.5.2 Exhibition Displays
165
8.6 Discussion
167
References
168
9 An Exploration of Exertion in Mixed Reality Systems via the Table Tennis for Three Game
172
9.1 Introduction
173
9.2 Related Work
174
9.3 Table Tennis for Three
176
9.3.1 The Table Tennis for Three Experience
177
9.4 Design of Table Tennis for Three
179
9.4.1 Choice of Tangible Equipment
179
9.4.1.1 Supporting Bodily Skill Training
179
9.4.1.2 Utilizing Existing Sport Advances
179
9.4.1.3 Uncertainty of the Real World
179
9.4.1.4 Supporting Proprioception and Force-Feedback
181
9.4.1.5 Avoiding Complex Equipment Such as Head-Mounted Displays
181
9.4.2 Implementation
182
9.4.2.1 Impact Detection Mechanism
182
9.4.2.2 Videoconferencing
184
9.4.2.3 Gameplay
184
9.5 Feedback from Users
185
9.6 Future Work
186
9.7 Discussion and Conclusions
186
References
188
10 Developing Mixed Interactive Systems: A Model-Based Process for Generating and Managing Design Solutions
190
10.1 Introduction
190
10.1.1 Existing MIS Development Support
191
10.1.2 Objective and Goal
192
10.1.3 A Case Study
193
10.2 Articulating MIS Task Analysis and Mixed Interaction Design
194
10.2.1 Presentation of the Two Selected Models: K-MAD and ASUR
194
10.2.1.1 Task Analysis with K-MAD
194
10.2.1.2 Mixed Interaction Design with ASUR
195
10.2.2 Articulation Between K-MAD and ASUR
195
10.2.2.1 Task to Mixed Interaction Model Transformations
195
10.2.2.2 Applying the Task to Mixed Interaction Model Transformations on the Case Study
196
10.2.3 Designer and ASUR Refinement
198
10.2.4 Advantages and Limits of the Transformation Process
200
10.3 Articulating Mixed Interaction Design with MIS Implementation
201
10.3.1 MIS Architecture Requirements
201
10.3.1.1 Modifiability, Portability and Development Efficiency
201
10.3.1.2 ASUR-IL Metamodel
202
10.3.1.3 Model-Driven Engineering Tools
203
10.3.2 ASUR to ASUR-IL Transformation Principles
203
10.3.2.1 Mixed Interaction to Software Architecture Model Transformation
203
10.3.2.2 Applying the Mixed Interaction to Software Architecture Model Transformation on the Case Study
204
10.3.3 From a Software Architecture Model for MIS to mplementation
206
10.3.4 Limits and Interests of These Articulations
207
10.4 Outcomes of the Design Process in an Iterative Development Context
208
10.4.1 K-MAD Level
209
10.4.2 ASUR Level
209
10.4.3 ASUR-IL Level
210
10.4.4 WComp Level
211
10.5 Conclusions and Perspectives
211
References
213
Part II Software Design and Implementation
216
11 Designing Outdoor Mixed Reality Hardware Systems
217
11.1 Introduction
217
11.2 Previous Work on Outdoor MR
220
11.3 The Tinmith System
221
11.4 Hardware for Outdoor MR Systems
222
11.4.1 Head-Mounted Electronics
224
11.4.2 Main Enclosure
226
11.4.3 Batteries
227
11.5 Input Devices
227
11.5.1 Pinch Gloves
228
11.5.2 Button Box
228
11.5.3 Additional Input Devices
229
11.6 Wearable Mixed Reality System Design
230
11.6.1 Manufacturing Techniques
230
11.6.2 Belt vs. Backpack
231
11.6.3 Electrical and Magnetic Interference
232
11.7 System Management
232
11.7.1 Power Management
232
11.7.2 Configuration Selection
233
11.7.3 Input Management
234
11.7.4 External Display
234
11.8 Conclusion
236
References
236
12 Multimodal Excitatory Interfaces with Automatic Content Classification
238
12.1 Motivation
238
12.2 Background Review
240
12.3 Inertial Sensing
240
12.4 Object Dynamics
242
12.4.1 Accelerometer Mapping
242
12.4.2 Friction and Stiction
243
12.4.3 Springs
243
12.4.4 Impacts
244
12.5 Message Transformation
244
12.5.1 PPM Language Model
244
12.5.1.1 Potential Enhancements
247
12.5.1.2 Test Model Classes
247
12.5.1.3 Certainty Filtering
249
12.5.2 Exploration
249
12.5.2.1 Identity Sieving
249
12.5.2.2 Time-Sequenced ''Rain''
250
12.6 Auditory and Vibrotactile Display
250
12.6.1 Vibrotactile Events
251
12.6.2 Audio Synthesis
252
12.6.2.1 Sample Banks
252
12.6.2.2 Audio Transformations
253
12.7 Further-Work Active Selection
254
12.8 Conclusions
254
References
254
13 Management of Tracking for Mixed and AugmentedReality Systems
256
13.1 Motivation
256
13.1.1 Requirements
257
13.1.2 Related Work
258
13.1.3 The Ubitrack and trackman Approach
258
13.2 The Ubitrack Framework
259
13.2.1 Spatial Relationship Graphs
259
13.2.1.1 Use of Cycles for Sensor Calibration and Object Registration
260
13.2.1.2 Edge Characteristics
260
13.2.2 Data Flow Networks
261
13.2.3 Spatial Relationship Patterns
261
13.2.3.1 Basic Concept
261
13.2.3.2 Synchronization Issues
263
13.2.3.3 Pattern Categories
263
13.2.4 SRG Design Activities
265
13.3 trackman: Interactive Modeling of Spatial Relationships
265
13.3.1 System Architecture
265
13.3.2 Graphical Layout
266
13.3.3 Interactive SRG Generation
267
13.3.4 Interactive Deduction of Spatial Relationships
267
13.3.5 More Modeling Functionality
268
13.3.6 Ordering of Design Activities
269
13.4 Advanced Interactive Modeling Concepts
270
13.4.1 Semi-automatic Modeling
270
13.4.2 Meta Patterns
271
13.5 Tools to Analyze and to Interact with Data Flows
272
13.5.1 Tools for Calibration and Registration
272
13.5.2 Tools for Online Analysis of Tracking Environments
274
13.6 Application Examples
275
13.7 Conclusion
276
References
277
14 Authoring Immersive Mixed Reality Experiences
279
14.1 Introduction
279
14.1.1 Definitions and Assumptions
280
14.1.1.1 Mixed Reality
280
14.1.1.2 Immersion
280
14.1.1.3 Absolute vs. Relative Coordinate Systems
280
14.2 Background: Mixed Reality Environments in the Arts
281
14.2.1 Motivation for Using Mixed Reality in the Arts
281
14.2.2 Examples of Mixed Reality in the Arts
282
14.2.2.1 Example: Markerless Magic Books
282
14.2.2.2 Mixed Reality as a Presentation Medium
282
14.2.2.3 Crossing Borders: Interactive Cinema
282
14.3 Related Work: Authoring Tools
283
14.4 Authoring Content for Mixed Reality Environments
284
14.4.1 Engineering and Authoring Platform: VGE
284
14.4.1.1 Overall Architecture
285
14.4.1.2 Perspectives
285
14.4.1.3 Sensors and Algorithms
286
14.4.2 Designing the Real World
286
14.4.2.1 Geometry and Visual Appearance
287
14.4.2.2 Lighting
287
14.4.3 Mixing Virtual Images
287
14.4.3.1 Test Setup
288
14.4.4 Directing the User Experience
290
14.4.4.1 Prototyping Tool: Interactive Table
291
14.4.5 Case Study: Exercise in Immersion 4
292
14.5 Conclusions
293
References
294
15 Fiia: A Model-Based Approach to Engineering Collaborative Augmented Reality
296
15.1 Introduction
296
15.2 Example: Collaborative Game Prototyping with Raptor
297
15.3 Related Work
301
15.3.1 Modeling Collaborative Augmented Reality
301
15.3.2 Toolkits for Collaborative AR
301
15.4 Fiia Notation
302
15.4.1 Notation for Collaborative AR
303
15.4.1.1 Adapters
304
15.4.1.2 Data Sharing
305
15.4.2 Scenario-Based Design
306
15.4.3 Mapping Fiia Diagrams to Code
307
15.4.4 Summing Up the Fiia Notation
308
15.5 The Fiia.Net Toolkit
308
15.5.1 Conceptual Framework
308
15.5.2 Distribution Architecture
309
15.5.3 Adapters
311
15.6 Implementing Fiia
312
15.7 Experience
313
15.8 Conclusion
314
References
314
16 A Software Engineering Method for the Design of Mixed Reality Systems
316
16.1 Introduction
316
16.2 Extending an SE Method for Mixed Reality Systems
318
16.2.1 Extending Symphony for the Design of Mixed Reality Systems
318
16.2.2 Case Study
320
16.3 The Functional Branch
321
16.3.1 Introduction
321
16.3.2 Initiating the Development
321
16.3.3 Conceptual Specifications of Requirements
322
16.3.4 Organizational and Interaction-Oriented Specification of Requirements
323
16.3.5 Analysis
327
16.3.6 Main Points Discussed
329
16.4 The Technical Branch
329
16.4.1 Description of the Applicative Architecture
330
16.4.2 Description of the Technical Architecture
331
16.4.3 Main Points Discussed
332
16.5 The Junction of the Functional and Technical Branches
333
16.5.1 Design
333
16.5.2 Main Points Discussed
335
16.6 Conclusions and Future Work
335
References
336
Part III Applications of Mixed Reality
338
17 Enhancing Health-Care Services with Mixed Reality Systems
339
17.1 Health Care and Mixed Reality Systems
339
17.1.1 Augmented and Mixed Reality
340
17.1.2 Usability Evaluation Techniques
341
17.1.3 Security Aspects
342
17.1.4 Work Structure
342
17.2 Overview of the Development Approach
342
17.2.1 Process Evaluation of the Health-Care Service
343
17.2.2 Evaluation of the Existing Information Systems
343
17.2.3 Identification of Decision Paths and Actions That Can Benefit from Mixed Reality Systems
344
17.2.4 Implementation of the Mixed Reality System
344
17.3 System Design and Implementation
344
17.3.1 Design of the System
345
17.3.2 ASUR Model of the System
347
17.3.2.1 Real Objects (Components R)
347
17.3.2.2 Person as User (Component U)
348
17.3.2.3 Adapters (Components A)
348
17.3.2.4 Systems (Components S)
349
17.3.2.5 Relationships Between the ASUR Components of the System
349
17.3.3 Addressing Critical Aspects of Mixed Reality Systems for Health-Care Services
349
17.3.3.1 Context Awareness
350
17.3.3.2 Timeliness and High Assurance
350
17.3.3.3 Fault Tolerance
351
17.3.3.4 Interoperability
351
17.3.4 Addressing Software Design Requirements
352
17.3.4.1 Distributed and Cooperating Services
352
17.3.4.2 Security and Privacy
353
17.3.4.3 Lookup and Discovery
353
17.3.4.4 Performance and Availability
354
17.3.5 Technology Environment and Architectural Approach
354
17.4 Conclusion and Outlook
356
References
356
18 The eXperience Induction Machine: A New Paradigm for Mixed-Reality Interaction Design and Psychological Experimentation
359
18.1 Introduction
359
18.1.1 Mixed-Reality Installations and Spaces
361
18.1.2 Why Build Such Spaces? Epistemological Rationale
363
18.1.3 Mixed and Virtual Reality as a Tool in Psychological Research
365
18.1.4 Challenges of Using Mixed and Virtual Realities in Psychological Research
367
18.2 The eXperience Induction Machine
369
18.2.1 System Architecture
369
18.2.1.1 Design Principles
369
18.2.1.2 Interfaces to Sensors and Effectors
370
18.3 XIM as a Platform for Psychological Experimentation
372
18.3.1 The Persistent Virtual Community
372
18.3.2 A Space Explains Itself: The ''Autodemo''
373
18.3.3 Cooperation and Competition: Playing Football in Mixed Reality
375
18.4 Conclusion and Outlook
377
References
378
19 MyCoach: In Situ User Evaluation of a Virtual and Physical Coach for Running
382
19.1 Introduction
382
19.1.1 Virtual Trainer/Coach
383
19.2 MyCoach
384
19.3 User Experiment
386
19.3.1 Runners
386
19.3.2 Measurement
387
19.4 Results
388
19.4.1 Pre-trial Results: Running and Training Habits
388
19.4.2 During Trial Results: Use of the MyCoach System
390
19.5 Usage Data
391
19.6 Netnography
392
19.6.1 Post-trial: Evaluation of MyCoach
393
19.7 Conclusions
396
19.8 Further Development of MyCoach
396
References
397
20 The RoboCup Mixed Reality League - A Case Study
399
20.1 Introduction
399
20.2 Hardware Architecture
403
20.2.1 Micro-Robots
403
20.2.1.1 Battery Charger
406
20.2.1.2 Infrared Transmitter
407
20.2.1.3 Firmware Uploading Interface Board
407
20.2.2 Augmented Reality Display
408
20.2.3 Tracking Camera
408
20.2.4 Computer
409
20.3 Software Architecture
409
20.3.1 Vision-Tracking Module
411
20.3.2 Application Modules
411
20.3.3 Graphics Module
413
20.3.4 Robot Control Module
413
20.3.5 Agents
415
20.4 Experience
415
20.4.1 Development Process
415
20.4.2 Soccer System
415
20.4.3 Racing Application
416
20.4.4 Future Developments
417
20.5 Summary and Conclusions
417
References
417
21 Mixed-Reality Prototypes to Support Early Creative Design
419
21.1 Introduction
419
21.2 Profession-Centered Methodology and User-Centered Design
420
21.3 Context and Needs
423
21.3.1 Architectural Design
423
21.3.2 Sketch-Based Preliminary Design
425
21.3.3 Distant Collaborative Design
426
21.3.4 Why Mixed Reality Should Be a Good Way of Responding to These Needs
427
21.4 Technological Solutions
428
21.4.1 Introduction
428
21.4.2 The Virtual Desktop
428
21.4.3 EsQUIsE
431
21.4.3.1 Introduction
431
21.4.3.2 The Entry Module
431
21.4.3.3 The Interpretation Module
432
21.4.3.4 The Evaluation Module
432
21.4.4 SketSha
434
21.5 Evaluations
436
21.5.1 Usability
437
21.5.2 Sketches
437
21.5.3 Immersion
438
21.5.4 Design Process
440
21.6 Characterization of These Mixed-Reality Systems
441
21.7 Discussions
442
References
443
Index
446
Alle Preise verstehen sich inklusive der gesetzlichen MwSt.