Kinect-enabled VR puts users in space with Earthlight


The following blog was guest-authored by Russell Grain, a development lead at Opaque Multimedia, a Melbourne (Australia)-based digital design studio specializing in the application of video game technologies in novel domains.

Earthlight is a first-person exploration game where the players step into the shoes of an astronaut on the International Space Station (ISS). There, some 431 kilometers (about 268 miles) above the Earth, they look down on our planet from the comfort of their own spacesuit. Featuring the most realistic depiction yet of the ISS in an interactive virtual reality (VR) setting, Earthlight demonstrates the limits of what is visually achievable in consumer-oriented VR experiences.

 Opaque Multimedia’s Earthlight game enables players to explore the International Space Station in an interactive VR setting, thanks to the latest Kinect sensor.
Opaque Multimedia’s Earthlight game enables players to explore the International Space Station in an
interactive VR setting, thanks to the latest Kinect sensor.

Our team at Opaque Multimedia developed Earthlight as a technical demo for our Kinect 4 Unreal plug-in, which exposes all the functionality of the latest Kinect sensor in Unreal Engine 4. Our goal was to create something visceral that demonstrated the power of Kinect as an input device—to show that Kinect could enable an experience that couldn’t be achieved with anything else.

Players explore the ISS from a truly first-person perspective, in which the movement of their head translates directly into the viewpoint of a space suit-clad astronaut. To complete this experience, players interact with the environment entirely through a Kinect 4 Unreal powered avateering solution, pushing and pulling themselves along the surface of the ISS as they navigate a network of handles and scaffolds to reach the top of the communications array.

Everyone behaves somewhat differently when presented with the Earth hanging below them. Some race straight to the top of the ISS, wanting to propel themselves to their goal as fast as possible. Others are taken with the details of the ISS’s machinery, and some simply relax and stare at the Earth. On average, players take about four minutes to ascend to the top of the station’s communications array.

 By using Kinect, Earthlight enables players to explore the ISS without disruptions to the immersive VR experience that a keyboard, mouse, or gamepad interface would create.
By using Kinect, Earthlight enables players to explore the ISS without disruptions to the immersive VR
experience that a keyboard, mouse, or gamepad interface would create.

As well as being a fantastic tool for building immersion in a virtual game world, Kinect is uniquely positioned to help solve some user interface challenges unique to the VR experience: you can’t see a keyboard, mouse, or gamepad while wearing any current generation virtual reality device. By using Kinect, not only can we overcome these issues, we also increase the depth of the experience.

The enhanced experience offers a compelling new use case for the fantastic body-tracking capabilities of the Kinect for Windows v2 sensor and SDK 2.0: to provide natural and intuitive input to virtual reality games. The latest sensor’s huge increase in fidelity makes it possible to track the precise movement of the arms. Moreover, the Kinect-enabled interface is so intuitive that, despite the lack of haptic feedback, users still adopt the unique gait and arm movements of a weightless astronaut. They are so immersed in the experience that they seem to forget all about the existence of gravity.

Earthlight has enjoyed a fantastic reception everywhere it’s been shown—from the initial demonstrations at GDC 2015, to the Microsoft Build conference, the Silicone Valley Virtual Reality meetup, and the recent appearance at We.Speak.Code. At each event, there was barely any reprieve from the constant lines of people waiting to try out the experience.

We estimate more than 2,000 people have experienced Earthlight, and we’ve been thrilled with their reactions. When asked afterwards what they thought of Earthlight, the almost universal response was “amazing.” We look forward to engendering further amazement as we push VR boundaries with Kinect 4 Unreal.

Russell Grain, Kinect 4 Unreal Lead, Opaque Multimedia

Key links

Comments (0)

Skip to main content