Facilitating the Presence of Users and 3D Models by the
Augmented Round Table
Erik Granum, Thomas B. Moeslund and Moritz
Störring,
Computer Vision and Media Technology, Aalborg
University, Denmark
Email: {eg,tbm,mst}@cvmt.dk
and
Wolfgang Broll and Michael Wittkaemper
Collaborative Virtual and Augmented
Environments Department
Fraunhofer Institute for Applied Information Technology
Sankt Augustin, Germany
Email: {Wolfgang.Broll,
Michael.Wittkaemper}@fit.fraunhofer.de
Abstract: Round table meetings are regularly
used between professionals to review development, to design new approaches and
to make decisions. While these roundtable meetings naturally facilitate the collective
presence of users they often lack a sophisticated presence of the actual object
discussed.
In this
paper we present the Augmented Round Table for architectural design. We use Virtual
Reality technologies to augment the users’ common workspace by virtual 3D
objects and providing real world interfaces to them – thus making them present.
We suggest that this may also be seen as augmenting the sense of presence at
the building site with the collective presence of people at the round table.
Keywords: Double presence, tangible user
interfaces, collaborative virtual environments, Augmented Reality
1 Introduction
The successful use of virtual reality (VR)
and augmented reality (AR) technologies often assumes a feeling of presence –
of being there – on part of the user. This is not necessarily easy to achieve
and it may depend on many different factors. When operating, e.g. with a “pure”
virtual world, a range of difficulties need to be taken into account, but at
least a focus for the type of presence aimed at may be defined as a reference
for relevant design solutions/decisions.
However, in some application cases the real
world and a virtual world are mixed and/or required to be perceived
simultaneously and this may assume presence in both worlds at the same time. A
conflict as to the role of presence in such cases seems apparent, and questions
could be whether this in itself is prohibitive for successful operation, or –
in the other “extreme” – whether it is possible for these two requirements for
presence to somehow support and/or reinforce one another.
In this paper we will show why round table
meetings are an important area for presence, introduce our Augmented Round
Table and present our approach to support multiple users.
The authors are technology developers and
scientists, and the paper will present application contexts including a
specific example, where such “double presence” is an inborn characteristic. The
purpose is thus to present context for related questions for discussions at the
Presence workshop, hoping that the outcome could inspire and improve the scope
for such applications and related technological research.
The system described may be demonstrated at
the conference, and video clips will be shown during the talk.
2
Roundtable Meetings and
Enabling Technology
Roundtable
meetings are regularly used in professional life e.g. for creative
brainstorming, decision making, or planning. Engineers may use them to design
products, interdisciplinary groups to create new product ideas, and medical
doctors to plan a complex surgery. Common to all these meetings is that the
participants are sitting together, seeing each other, and communicating
verbally and through hand gestures and facial expressions. Furthermore,
documents and objects are often on the table to support the discussion.
A
profession that frequently uses roundtable meetings is architecture,
particularly for design review meetings where a project is reviewed by senior
architects and their assistants. These meetings are extremely collaborative and
inventive to find new ideas or solve problems. They are often starting with
simple sketches (hand drawings), improving over several stages of 2D plans and
3D models, getting more and more complex, finally leading to very complex CAD
models and highly sophisticated (real) 3D models. However, currently progress
is made between meetings and the meetings themselves are only to review the current state of the work and make decisions.
Since this is a highly iterative process, it is often very time-consuming. Architects
consider the possibility of changing and touching the sketches, plans and models as an
important part of inspiration during a design review meeting.
From a certain
design phase 2D drawings are not enough because it is difficult to imagine from
a 2D drawing how the 3D object will look, particularly for the customer, but
also for the architect and other experts involved. It is necessary to look at the
model from different viewpoints, e.g. to see where shadows will appear.
Therefore, even though time consuming and expensive, 3D models are currently built
from wood or polystyrene. A 3D model, although not 100% realistic and scaled
with respect to the real building, enables the crucial sensation of ‘being
there’.
3 The Augmented Round Table
The ARTHUR
system (Augmented Round Table for Architecture and Urban Planning) bridges the
gap between real and virtual worlds by enhancing the users’ current working
environment with virtual 3D objects. Our developments focus on providing an
intuitive environment, which supports natural interaction with virtual objects
while sustaining existing communication and interaction mechanisms. Real world
objects are used as tangible interfaces (Ishi, 1997)
together with hand gestures to augment the social situation in a meeting and
make 3D environments attractive even to non-experts. The ARTHUR system contains
new types of user-friendly see-through displays, non-intrusive object tracking
mechanisms and intuitive user interface mechanisms within a location
independent multi-user real-time augmented reality environment. The ARTHUR
system addresses a wide area of possible collaborative applications with focus
on architecture and urban planning.
|
Figure 1: Real world items as tangible interfaces
supporting the sensation of presence of 3D objects |
The goal of
the project is to develop an intuitive augmented reality environment supporting
common round table meetings. Existing approaches such as BUILT-IT (Rauterberg, et al, 1997) use separate projections screens, or such as MagicMeeting (Regenbrecht, et al, 2002) limit direct user communication
due to video augmentation. In our approach virtual 3D objects are projected
into the common working environment of the users by semi-transparent wearable
stereoscopic head mounted displays (HMDs[1]).
Thus round table meetings are enhanced by virtual 3D objects. In contrast to
other approaches such as MARE (Grasset, et al, 2002)
we focus on natural interactions using unobtrusive AR based input mechanisms.
The main focus of the ARTHUR system is the development of new intuitive
interaction mechanisms. One approach is the use of real world items to realize
tangible and intuitive interfaces for the manipulation of 3D objects (see
fig. 1). This presumes a flexible and sophisticated object tracking
mechanism. The ARTHUR system therefore applies a tracking mechanisms based on
computer vision. While similar interface approaches have been presented earlier
(Billinghurst, et al, 2000), our approach aims to
support meetings involving several people.
The
developed AR environment thus provides multi-user capabilities to guarantee
individual but consistent views among all participants. While in general the
participants see and interact with the same virtual objects, personal menus and
individual additional information can be provided to each user.
As part of
the ARTHUR system a new type of a high-resolution see-through head mounted
display has been developed. Beside viewing quality (resolution, brightness,
etc.), ergonomic design issues guaranteeing a comfortable use were realized.
Another very important feature, essential for efficient collaboration, is the
ability to see other participants’ eyes during a session – usually not possible
with other types of displays.
Computer
vision techniques using head mounted and fixed cameras are used to track the
movements of real world items (placeholder objects (PHO), wand) and to recognize hand
gestures. Due to the computer vision based approach users can interact without
any disturbing cables or sensors connected to their interface elements.
Tangible
interfaces, wands (5DOF pointer), and hand gesture recognition are used for the
realization of intuitive user interaction mechanisms. Users may grab a real
world placeholder object, associate the item with a virtual object and thereby
create a direct manipulation interface (see menu item selector in figure 1).
Users may also select and manipulate the shape of virtual objects by a wand or
use hand gesture input to navigate in pop-up menus, select items and execute
actions.
The system
allows us to easily integrate its visualization and interaction capabilities
with existing professional or special purpose software. Showcase integrations
of the system with a solar gain simulation program and a
commercial CAD software has been proved.
4 Supporting Multiple Users
In order to
facilitate the sensation of viewing and manipulating virtual 3D objects within
common round table meetings, the most important aspect is to support a
consistent view of all users (see fig. 2). In general, this problem is
similar to providing consistent views in shared or distributed virtual
environments. In an AR environment however, users not only share the virtual
objects, but also their real world items and environment. Thus it is crucial to
the acceptance of the system to provide a consistent view among all local
participants, even upon changes to the scene. Additionally the problem is
intensified due to the fact that only a pretty good registration between
virtual objects and their real environment will be acceptable for the users.
Finally, real world PHO or other information gathered from the users’ locations
have to be distributed among all participants and applied to the individual
visualization in real time to match the current location of the PHO and
associated virtual objects for each user.
|
Figure
2: Viewing and manipulating
shared virtual objects in a common round table meeting |
In our
approach we use a completely distributed system. Most components such as 3D
stereo visualization or head mounted camera tracking are performed locally for
each user. In order to make the overall approach fast (keeping latency down) we
use a peer-to-peer approach for the distribution of changes to the virtual
scene. The distribution mechanism is based on CORBA. We use the same mechanisms
to distribute other information such as tracking data or computer vision
tracking information among all participants. Each participant then has a more
or less autonomous sub-system on his or her local computer. This makes the
overall approach rather independent of differences in the performance of
individual users’ computers (whether based on the overall processor and
graphics accelerator or the current load).
Beside
support for sharing virtual objects, the system additionally provides
mechanisms for individual local 3D visualizations. This especially applies to
virtual helper objects such as menus, which will not be shown to other users.
This mechanism can also be used to test and prepare virtual items in a
“private” workspace area.
5 Discussion
The ARTHUR
system under development is tested at regular intervals for usability by the
application partners of the project. The tests are primarily focused around the
control of virtual objects via PHOs and the use of pointers to creation and
manipulation of forms. The tests of the ARTHUR system is carried out by having
test subjects go through pre-defined storyboards and having them elaborate
their actions by “thinking aloud”. We have hereby obtained knowledge on
especially the design of the user interface but also on technical aspects, e.g.,
how big the fields of view in the cameras and HMDs
should be. In figure 1 and 2 examples of user tests can be seen. In figure 1 several
PHOs are used to control virtual beams in a virtual building. In figure 2 the
shape of a virtual building is controlled by the use of a 5DOF pointer.
We have not, however, made any tests yet that
have been formulated around the concept of presence. Inspiration to look at the
project objectives from this perspective has come rather recently (Riva, et al,
2003).
First, let
us recapitulate the ambitions regarding the two potentially competing settings
for presence.
·
Setting 1: Real world social situation. A number of people, say 4, are at a
roundtable meeting and should feel present in this social situation, where also
non-verbal interpersonal communication should be possible, i.e. body language
like gestures, facial expressions etc.
·
Setting 2: A virtual world visualized in the shared workspace between the meeting
participants (at the middle of the round table).
Setting 2 is achieved using HMDs which by traditionally design directly obstruct the
intentions of setting 1. Hence the criteria for design of the HMDs are that they should be “look-through”, cover/hide as little as possible of the face of the
bearer, and obscure as little as possible of the field of view of the bearer
himself. Various other things like interface facilities may potentially be
hindrances for the feeling of presence in either or both of the two settings. However,
our question in this presentation is: If and how the simultaneous feeling of
presence in the two settings is possible.
From our usability test we may in
retrospect find indications that this is possible in a sense. In terms of how
this could work we would like to project the following on the basis of just
common sense interpretations.
In its outset the project argued that the
real world setting at the round table would be augmented with a virtual world.
We find, however, that when it all works, we may have a situation that is more appropriately described
with opposite roles of the two worlds. I.e. presence at the building site
(virtual world) is augmented with a “social feeling” of being there together.
The social feeling of presence established in the real world is transferred to
the collective feeling of being together at the virtual building site. Hence,
we may have an example of augmented virtuallity
rather than augmented reality.
6
Conclusion
In this paper we presented the ARTHUR
system which is based on the concept of the Augmented Round Table. ARTHUR
provides a new interface for collaborative design and review in architecture
and urban planning. The interface relies on unobtrusive input mechanisms as
well as natural and intuitive user interactions. We showed how this approach
can be used to enhance collaboration between multiple users without limiting
their verbal and non verbal communication facilities. In our future work we
will enhance this interface by additional multi-modal facilities and evaluate
the approach by further user studies.
Regarding
presence there is not evidence to warrant conclusions.
We may, however, summarise our speculations to suggest a yes-answer to the question of the
possibility of simultaneous presence in different worlds. A way in which we
propose this is possible is when the presence in the one world can be
functionally embedded in the other. In the case discussed, the social and
collective aspects are important, and one may wonder if this is a necessary
condition or just facilitating the simultaneous presence.
The presented speculations on presence have
made us aware that for development of sophisticated interfaces, one often may
use formulation in terms of presence as well as insight from presence research
to structure important priorities in design.
References
Billinghurst, M., S. Campbell, D. Hendrickson, W. Chinthammit,
I. Poupyrev, K. Takahashi, H. Kato (2000), Magic Book: Exploring
Transitions in Collaborative AR Interfaces, SIGGRAPH 2000 Emerging
Technologies Proposal, ACM SIGGRAPH 2000.
Grasset, R., and Gascuel,
J.-D. (2002), MARE:Multiuser Augmented Reality
Environment on table setup. SIGGRAPH 2002, Conference Abstracts and
Applications, Computer Graphics Annual Conference Series, p. 213.
Ishii, H., Ullmer, B. (1997), Tangible Bits: Towards Seamless
Interfaces between people, Bits and Atoms. CHI’97, Atlanta,
Georgia 1997.
Rauterberg,
M., Fjeld, M., Krueger, H., Bichsel,
M., Leonhardt, U. & Meier, M. (1997), “BUILD-IT:
a video-based interaction technique of a planning tool for construction and
design”, in H. Miyamoto, S. Saito, M. Kajiyama &
N. Koizumi (eds.) Proceedings of Work With Display Units--WWDU'97, pp. 175-176.
Regenbrecht,
H.T., Wagner, M.T, and Baratoff, G. (2002) „MagicMeeting: A Collaborative Tangible Augmented Reality
System”. Virtual Reality (2002) 6:151-166.
Riva, G., Davide, F., and Ijsselsteijn,
W.A, (eds.) (2003), “Being There: Concepts, Effects and Measurements of User
Presence in Synthetic Environments”, IOS Press, Amsterdam, The
Netherlands.