WWSB: What Would Scoble Blog?: Total immersion with WorldWide Telescope and Oculus Rift

  • Article

When Robert Scoble saw a demo of WorldWide Telescope under embargo he was so moved that he posted his now famous "Microsoft Researchers make me cry" blog post. If he saw what we have been doing lately, hooking WorldWide Telescopes Eclipse release up to Oculus Rift, he might post "Microsoft Researchers make me sick". Robert did not do a lot of crying during the original WWT demo, and he probably would not be doing any real heaving now if he saw it, but many astronauts get at least a little queasy their first time in space, and I am sure he would have fun with the headline.

Over the last couple of weeks the folks in my office have been getting mind-blowing demos of the total immersion that comes with putting on a Oculus Rift head mounted VR goggles and being transported into space. I usually start people off orbiting with the international space station. They look around them and see all the modules, solar panels and radiator panels surrounding them, then look down at their feet and see the Earth slowly moving down below them as they zoom 17,200 mile per hour orbiting the Earth once every 95 minutes. Grab and XBOX 360 controller and now you are flying around the space station. They we fly down and soar thru Yosemite Valley, speed time up and watch the sun move across the sky, the light dims as night falls and the sun sets to a brilliant red atmosphere and the stars come out. Turning your head north you see the big dipper, and pointing from the end of the cusp we follow the line to Polaris, the north star. I have videos I captured of colleagues giggling, saying "Wow!" or "Oh my gosh!" over and over again. Most agree it was the single most amazing thing they have every experienced.

By this time you may have completely forgotten that you are not actually in Yosemite Valley or orbiting the ISS, but wearing an  1280x800 HDMI 7" LCD panel with a combination of Gyro, Accelerometer and compass mounted on you head. The technology does such a good job at fooling you because the head tracking is so fluid. To make this illusion successful the sensors are read about 500 times a second and the orientation of your head is tracked quickly and accurately, while WorldWide Telescopes new Eclipse DirectX11 is rendering frames at the LCD limit of 60 updates a second. The combination of low latency sensor reading, and high-performance rendering mean the feedback to your brain is near instantaneous and you are fooled into thinking you are really there.

While 3d and Stereo environments have been considered "immersive", they come nothing close to this experience. The WWT/Rift integration does use stereo, but with a twist. Many of the pixels are used to fill your peripheral vision to a point where you can't see them all. They use a lens to give you a very high perceived field-of-view. While this sacrifices some of the resolution in the center of the field, large field of view that allows your eyes to gaze left and right and see more of the scene, even before you head moves. When you add the head tracking you can't ever "get out" of the image. An IMAX has a impressive field of view, a full dome planetarium even more, but the moment you turn you see walls and seats and the projectors behind you. In the WWT/Oculus experience you can't ever turn far enough around to get out of the scene: it becomes your reality. People have talked about virtual reality for years, but no product has every come this close to achieving it.

The Oculus may have been designed to gamers, but with WWT it is already showing its chops for Research and Science Visualization. I loaded data from years of earthquakes and flew under the Earth to analyze the patterns, seeing the distribution of quakes along the tectonic plates. Then I brought in 30 years of rainfall data for the entire US, and flew over the "Terrain" of rain fall stacked over the ground in a grid of a half-million 3d bar-charts. You get a real understanding with visualization of this data, and with the Oculus you can really focus on the data without distraction. The scenarios are endless and your get more out of every data set you view.

Just when you think it could not get any cooler, I added Kinect to the mix. Now I am flying around using my hands and arms, manipulating objects, giving voice commands all without losing the illusion of being there. If you have the space to work with the Kinect lets your really lets you naturally interact with the virtual environment. Over the summer we are going to do more development of this combination, both for standing and sitting scenarios.

The impact to education could also be amazing. Imagine kids with the closest planetarium several hundred miles away, sitting down in a class-room with a headset on, immersing themselves in a WorldWide Telescope tour teaching them the astronomy (or science) lesson of the day. Exploring the Earth, Mars, the moon or the sky and getting contextual answers to their questions. All at a price that almost any school can afford. You could even have virtual planetariums where a whole group of students could put headsets on while their instructor virtually flying around the "dome" showing them the sky.

For researchers, educators and science enthusiasts the combination of WWT and the Oculus rift is a real game changer, and far beyond the gaming it was designed for.

About now you are probably wondering how to get your hands on all of this. Well around July 1st the WWT Eclipse Beta will be on our website, with built-in Oculus Rift support, so if you were lucky enough to get one thru the Kickstarter then you can be up and running right away. Getting a Rift device will be more of a challenge. They are selling $300 development Kits right now, and delivery is quoted as August. We are still developing the integration of Kinect in WWT so that part is purely a demo for now, but we hope to make it public later this year. Their are so many new scenarios that this will enable, what we are releasing now is just the beginning. The sky is not the limit!

Jonathan Fay

PS. Thanks so much to George Djorgovski from Cal Tech who loaned us his Rift to make this work possible

Related Links