John and I each gave a presentation of the ideas, projects and the technology when visiting the research lab “Limsi” in Paris. I introduced our visions and projects and John talked about the technological set-up and the generic system architecture.
Visions, projects and system architecture:
Christian Jacquemin is one of the researchers working at Le Limsi. He arranged our visit to Limsi and set up everything for us to see and experience the cutting-edge Limsi Lab. Christian is working together with Experience Lab and we look forward to his visit to ExLab by the turn of 2012/2013.
Limsi (Laboratoire d’Informatique pour la Mécanique et les Sciences de l’Ingénieur) is a research laboratory with a personnel of 120 permanent researchers and 60 ph.d students. The Lab covers a wide disciplinary spectrum and research themes: from thermodynamics to cognition, encompassing fluid mechanics, energetics, acoustics and voice synthesis, spoken language and text processing, vision, virtual reality…
We visited the team working with Augmented and Virtual Reality and Audio Interfaces. Three research projects are undertaken 1) The NAVIG project: This project aims at providing an augmented audio rendering to blind people to help them in their daily life but without preventing them from perceiving their normal audio environment. 2) The SMART-I² is a spatialized audio-visual rendering system of high-quality. This project aims at increasing intelligibility and immersion sensation in audio-visual interfaces for virtual reality and 3) Audio-visual Renderings for Multimedia Navigation: This axe is looking for ways to increase big database exploration interfaces by adding them 3D sound and by transposing to them metaphors coming from the visualization field.
A 3D holographic audio lab.
A studio aimed at 3D audio and sound design and studies is located In one of the corners of the huge laboratory with a virtual reality space at the centre. Brian Katz is one of the researchers affiliated with this studio and he joined us on our tour around the Lab.
Marc Rébillat showed us around the virtual reality lab.
Marc showed us and he explained the construction of the many facilities of the Lab. In the virtual reality space, he introduced a virtual and visual installation to teach students about relativity theory by showing and experiencing the changes that time and space undergo when traveling by the speed of light.
Relativity theory installation.
Virtual humans and emotions.
With a board game we played against a humanoid avatar with feelings – or at least so it seemed. Whenever the avatar was successful his expression gradually changed to show happiness and success awareness and, of course, also the sadness and acidity when the human player appeared successful. Matthieu is the post.doc who introduced his work with this integration of an emotional model and the humanoid avatar.
Finally, in the virtual reality space I sensed what it is like when huge humanoid avatars stare at you wherever you go. It was a scary sensation.
Thanks to Christian, Marc, Michelle, Brian, and Matthieu for a great experience.
John and I will visit Christian Jacquemin in Paris on Monday the 19th of November. He is working with the research lab “Limsi” and the purpose of our visit is to explore the possibilities for further collaboration between ExLab and Limsi.
Christian helped us reconstruct the sea and waves in the Experience Cylinder project Havhingsten in collaboration with the Viking ship museum in Roskilde, DK.
| v/ Specialestuderende Morten Brandrup, Mads H. Jørgensen, Steffen Thorlund |  |
|
| With Experience Lab as the physical foundation for creating experiences, we are working with the Unity 3D graphics engine to create an engaging immersive gaming experience. We aim to develop a game that utilizes each of the available technologies that Experience Lab consists of, and to use these technologies to maximize the immersion of the gaming experience.Up until now, we’ve used the computer game Unreal Tournament(1), and special software developed by Public VR(2) called CaveUT, to make a quick and easy demonstration of some of the capabilities of ExperienceLab’s cylindrical screen.
What we yet have to figure out in Unity is how to display a single game on multiple screens in order to be surrounded by the game world. Secondly we will make use of gesture tracking, to use body movements to control the game. As of today computer games are still solo experiences in terms of viewpoint. Each player in a multiplayer game interacts from his or hers own screen. In Experience Lab we have the option to place two or more players in the same game environment and cooperate on the same screen. 1 Unreal Tournament is trademark of Epic Games, Inc. 2 Is a non-profit organization dedicated to research in virtual reality for education. Our contact: Dr. Jeffrey Jacobson, Ph.D., Director PublicVR, jeff@publicvr.org,http://publicvr.org/html/about.html |
| v/ Forskningsassistent Torben Kjærgaard | |
|
| In the project “You are our centre” Torben explores the multi sensory potentials of the Experience Cylinder. The vision of the project is to create a platform for experiments with interactive art.
Med ‘You are our centre’ projektet undersøges cylinderens muligheder for formidling af sanselige oplevelser. Samtidigt undersøges det hvorledes brugerinteraktion med et kunstværk bidrager til selve den kunstneriske oplevelse.‘You are our centre’ installationen vil ligeledes fungere som et pilotprojekt til inspiration for kommende tværfaglige projekter i ExLab. Et mål kan derfor være at generere projektsamarbejde på tværs af fag og institutter, eksempelvis imellem fag på Kommunikation, Performance Design og Datalogi, til empirisk undersøgelse af brugeroplevelser. Scenarie: Beskuer går ind i en helt mørkt cylinder. Der høres kun en svag hvisken fra skiftende retninger. Når beskuer er tracket, høres tydeligere hvisken på skift fra alle retninger: ”Pisst, kom herover…”. ”Nej, kom herover…” etc. Når beskuer går frem mod én af de seks videoer, kommer personen (i videoen) frem fra mørket og starter sin handling. Brugeroplevelsen: Beskueren er tænkt til at spille en aktiv rolle i installationen. Konkret ved at kunne igangsætte handlingerne ved at gå frem mod en handling. Men tillige på et mere abstrakt plan hvor beskueren bliver tiltalt direkte og betragtet af personerne. Beskueren bliver dermed fokuspunkt for personerne og selve installationen, uden at det bliver tydeligt hvorfor. Beskuerens ydere og indre reaktioner på at blive tiltalt, betragtet og hvisket om, bliver dermed installationens egentlige kerne. |
| v/ Adjunkt Sanne Krogh Groth (performance-design) | |
|
In the project “Interactive sound,” Sanne will experiment with sound as a source of experience, perception and aesthetics. There is an increasing interest in research on different audio cultures and sound studios. The vision of the project is to research directional sound and the potentials of panning audio resources thus to produce research based knowledge that will support a further development of interactive auditive forms of communication.
I multisensoriske events og æstetiske værker spiller lyden i kraft af dens allestedsnærværende egenskab en uomgængelig rolle. Lyd organiserer os socialt, den får os til at reagere kropsligt, den hjælper os med at navigere i både rum og tid, og den kan åbne vores hukommelse for både bevidste og ubevidste erindringer. Forskningen i lyd og lydkulturer har gennem det seneste årti nydt stigende opmærksomhed i forskningsmiljøer internationalt såvel som herhjemme, hvor nye områder som auditiv kultur og ‘sound studies’ er ved at finde fodfæste i akademiske miljøer, på kunstskoler og på erhvervsorienterede uddannelser. I praksis fremstår lyden dog stadig som et underprioriteret område, der sjældent behandles med den analytiske præcision og sensitive nænsomhed, der er nødvendig, for at få den til at fremstå som en integreret del af helheden. Experience Lab danner med mulighederne for Kinect og panorering af lydkilder en oplagt ramme til forskning indenfor dette felt, hvor der i forlængelse af den eksisterende forskning på området, kan udvikles nye auditive interaktive formidlingsformer. Nærværende forskningsprojekt tager udgangspunkt i undersøgelse af lyds performative egenskaber og muligheder. Som udgangspunkt er der altså ikke tale om en genrebaseret undersøgelse, men om lydens kommunikative egenskaber mere generelt: Centrale spørgsmål vil være:
Resultaterne fra de indledende undersøgelser vil siden blive videreudviklet og implementeret i konkrete projekter såsom:
|
| v/ Professor John Patrick Gallagher (datalogi) | ||
| In the Google Street View project, John’s vision is to use the Experience Cylinder to generate an immersive experience in two versions. One version creates a window-like experience controlled by the user rather than by pre-defined paths as currently defined by Google Street View. And, 3D models of places and buildings is a second version. In this, the Experience Cylinder will allow users to walk around immersive 360 degrees 3D models.
I teknologier såsom Google Street View er panorama-billeder syet sammen for at skabe en 360o udsigt over en scene. Med en konventionel computerskærm kan en bruger navigere gennem scenen langs en foruddefineret rute (ruten, hvor fotografierne blev taget), ved hjælp af musen og tastaturet. På ethvert tidspunkt kan brugeren “kigge rundt” ved hjælp af piletasterne for at ændre visningens retning. Computerskærmen fungerer som et vindue til verden, som brugeren kan flytte ved hjælp af mus og tastatur. Projektet går ud på at overføre Street View-oplevelse til oplevelsescylinderen. Version 1. En version af projektet er baseret på panoramiske fotografier ligesom i Street View. Således vil oplevelsescylinderen vise, på ethvert tidspunkt, en 360o panoramaudsigt fra den aktuelle position i liv-størrelsen, snarere end et enkelt vindue på scenen. Dette åbner mulighed for en langt bedre oplevelse. Da den fulde 360o panoramaudsigt ville være synlig hele tiden på den store cirkulære skærm, kan brugeren blot vende sit hoved for at se forskellige dele af det. Bevægelser gennem verden opnås enten ved at spore brugeren ved brug af Kinect trackeren eller ved at inkorporere en anden form for styring, såsom en joystick monteret på en drejelig stol for at skabe et indtryk af “kørsel” gennem scenen. Version 2. I stedet for at overføre panoramiske fotografier til cylinderen, er en anden mulighed at anvende 3D modeller. Et sådant projekt kan bygge videre på ideer i den 3D model af Roskilde Universitet, som er i øjeblikket under udvikling i oplevelsescylinderen. Det ville være dyrere at bygge men giver følgende fordele: (i) man kan gå alle steder, ikke kun på den foruddefineret rute, og (ii) objekter i verden opfattes som mere solide og realistiske. Dog ville det være sværere at skabe en realistisk baggrund i modellen, mens objekter såsom træer er vanskellige at modellere. Version 2 er derfor mere egnet til visse modeller end andre, muligvis indendørs modeller. |
| v/ Nikolai Møbius, Nikolas Padfield, Morten Brandrup, Mads H. Jørgensen, Steffen Thorlund, John Gallagher, Adrian, Torben Kjærgård, Sisse Siggaard Jensen, Dixi Louise Strand. |
The first prototype of the Experience Cylinder was developed in close collaboration with the Viking museum in Roskilde and the content was a large corpus of data from the expedition to and fro Dublin with the large viking ship Havhingsten. The next step is to bring further the vision of developing a platform architecture – a second prototype – to open to multiple projects, semiotic resources, narratives etc to be implemented in the immersive 360 degrees environment. To serve these purposes improvements are made with regard to: the quality of tracking, simultaneous tracking of more than one user, further development of behaviours to guide the interplay between user agency, interface and the code of the basic architecture, user interaction with different data layers, transfer from the Cylinder to mobile tablets and improvement of visual quality and the jointing of the 360 degrees visualization.
Som udløber af Havhingstprojektet og samarbejdet med Vikingeskibsmuseet er oplevelsescylinderen nu under udvikling til at blive en åben generisk arkitektur, der er en avanceret men brugervenlig teknologisk tværmedial platform, som kan anvendes inden for mange forskellige områder og med mange forskellige formål.
Det indebærer blandt andet en forbedring af flere tekniske og funktionsmæssige aspekter af den eksisterende prototype, fx:
Der er også brug for en mere intuitiv brugergrænseflade med et sprog, så andre end programmører kan bruge og udnytte oplevelsescylinderens potentiale for immersive oplevelser og formidling af mange lag af historisk og nutidig viden via lyd, fotos, video og 3D visualiseringer. Vi arbejder på at den forbedrede udgave af den generiske platform er færdig ved årsskiftet 2012/13.
Samarbejdspartnere
Udviklingsvirksomheden GeekPhysical, kunstnerkollektivet Illutron og forskningslaboratoriet LMSI-CRNS, Frankrig.
v/ Steffen Thorlund, Mads Stengelshøj Jørgensen, Morten Brandrup.
In this project a 3D Unity model of Roskilde University has been implemented to be visualized in the Experience Cylinder. This 3D model allows us to move around every building and corner of the place in an immersed 3D version.
The results from this project will help us exemplify how rich immersive and intelligent spaces can open new horizons, potentials and possibilities for teaching and learning spaces with regard to: visualization, simulation and customization.
