Introduction to Windows Mixed Reality

Article
09/08/2017

Windows Mixed Reality

Learn to build mixed reality experiences for HoloLens and immersive headsets https://developer.microsoft.com/en-us/windows/mixed-reality

The Hardware

*All prices are subject to change and based on current published figures

As you can see each device has a very similar look and feel based on Microsoft’s base specification, providing:

1,440 x 1,440 resolution in each eyepiece
90Hz refresh rate
Adjustable headband (some also include special headbands which are anti-bacterial, a nice touch when multiple people use it)
A Flip Up visor design, which makes it easy to switch to the real world without removing the headset
USB 3 & HDMI connectors

Each manufacturer has taken its own view and design with the specs to give a unique feel and weight to each.

The Experience

The one main thing that sets the Microsoft Mixed Reality setup different to every other high-powered VR style experience (excluding the likes of Cardboard / mobile VR) is that it requires absolutely no external setup, no additional devices or sensors, it is literally just plug and go.

This is all provided by the two front motion sensors and a collection of other sensors built in to the visor which track all movement within the headset and project it’s view outwards. It knows which way you are facing, what altitude you are at (standing, sitting and even lying down) as well as the angle of your head. In my own testing, this is extremely accurate, even if you “shake your head”. The tracking system is also not bothered by the light levels in the room, so whether it’s dark or extremely bright (I’ve personally tested both) it makes no difference. The only environment I’ve not tested in is outside, as the PC cords don’t reach that far.

The way the inside-out sensors work (as opposed to the camera/sensor outside-in setup where you are “watched from outside”), is that the camera sensors track the headset’s movement by watching the room movement and then translating that in to direction. This is paired with another set of other sensors, such as a gyroscope and accelerometer, to both validate this input and also give relative force / angle measurements. This simply results in the headset itself knowing where it’s moving, instead of other systems tracking you externally. The HoloLens takes this further by actually mapping the environment it see’s, which the Mixed Reality (Immersive headset) is based upon. Currently developers don’t yet have access to this “view” from the sensors, so we can’t embellish the experience. But here’s hoping it is exposed later which will make for some even more fun projects.

The Motion Controllers

Each controller has a touch thumbstick as well as a traditional thumbstick and the usual plethora of additional buttons to cater for most VR experiences.Unlike Vive, Oculus and PSVR however, these controllers require no additional sensors, devices, light up beacons or cameras, they simply use the existing sensors built right in to the headset. The headset detects the position / rotation of the controllers and the inputs are fed back to the PC through a bluetooth connection, no wires needed.

Hardware Requirements

One of the biggest things that has put off a lot of consumers from the VR space is the cost. The PC’s and Graphic card requirements of most systems has been considerable and well out of the reach of all but the prosumer / elite gamer.

Note, to run the Mixed Reality Headsets, you will have to update your PC’s operating system to at least the Windows 10 Fall Creators Update. Available now in preview (insider program) and due for full release in October 2017.

With the Windows 10 Mixed Reality setup, things are different. Sure, if you want the Uber high end experience, you will need a PC to meet that level but for the everyday consumer who just wants to experiment and play, there is now a much more comfortable zone which meets the means of the average gaming PC.

* For Windows PCs with integrated graphics hardware, please note the following:

Windows Mixed Reality immersive headsets will only run at a maximum of 60Hz.

Windows Mixed Reality feature support has only been tested for, and confirmed to run on, the Intel HD Graphics (620 or higher) platform at this time.

AMD integrated graphics platforms have not yet been tested or confirmed by Microsoft.

AMD mobile CPUs have not yet been confirmed by Microsoft.

Testing the setup process

After you have unboxed all your toys and the boxes are strewn all over the living room floor, the first task any new Mixed Reality consumer is going to face is “how to plug all this stuff in anyway”. Sure, the headset looks cool while you are wearing it and you can look kind of space age while walking around. Eventually though you will want to actually turn the thing on.

Updates

First task is to ensure the PC is up to date, no doubt there will be sufficient notice in the documentation for the headset, to ensure that you have upgraded/updated your PC to the latest and greatest Windows 10 build, namely the Windows 10 Fall Creators Update (still no word at the time of writing if it’ll be called the “Autumn Creators update” or if we’ll have to live with the US Naming as well). So once your PC is updated, it’s time to plug it in.

Side Note, for the purposes of this test I have installed the Creators update using the “Windows Insider Program”, which gives every person the chance to test out the newest updates from Microsoft before they hit the shelves. From October, this won’t be necessary, unless you want even newer builds/versions than the Creators Update.

Plugging In

The final installation guide that is released with the production headsets will likely recommended to launch the “Mixed Reality Portal” before plugging in the Headset but in true consumer simulation testing, I plugged it in before I even tried. Thankfully whether you plug the headset in first or launch the portal, it makes no difference, so all bases are covered there.

Once the device is recognised and drivers are installed, you are ready to get started. Although at this point there isn’t much to see.

Starting up the Mixed Reality Portal

When you start the “Mixed Reality Portal” application for the first time (this new app is installed by default with the Creators Update), the first step will test your PC, your headset and then get everything else installed and ready:

Starting from the Top-Left and working right, let’s walk through the setup process:

The First screen you see is as informative as you would expect, giving you some much needed information for what to expect from this arduous process (just kidding).
Next, we have the critical System test. If your PC doesn’t come up to scratch, this will tell you where you need to focus. Whether it’s just newer drivers, replace your graphics card or grab the wallet for a newer PC.
Now the fun begins, connect your headset to the PC, plugging in the Video (HDMI) and Driver (USB3.0).
Once your headset is detected, you’ll get a personalised walk-through your gear.
And a detailed view of all the ports, including the sensors, headband and where to find the in-built headset jack (no trailing headset wires!) – Just be sure to use one with a mic for the full experience.
If you have purchased the extra controllers, you will then be prompted to pair them with the PC. Don’t worry if you haven’t, you can still use either keyboard and mouse or an Xbox One controller (If you have the original Xbox One Controllers, you will need the Xbox Wireless receiver in your PC, if you have the newer Xbox One S controllers with the 3.5mm headphone jack in the bottom, they can use bluetooth, so the adaptor isn’t required)
Next, we come to the all-important choice, do you want to sit or walk. The headset will work in either mode and it’s up to you which you will start with. Don’t worry, you can always switch later.
If you chose to walk, you’ll need to setup your initial boundary. Here the device uses its sensors and your legs, to scan your room to work out where you can and can’t walk.
First job is to setup your center. The process recommends this is at your PC but it isn’t essential. For your first go though, hold your headset in your hands and face your PC as instructed and Click “Center”.
Now you will get a little demo about tracing your environment, this simply involves walking around your room, keeping the headset facing the computer (or your center) as far as you physically can (including how far the cable will let you go), when you return to the center it will automatically finish.
In this shot, you can see the effect of the trace in my room setup, a little smaller than the demo.
Once complete, the setup recommends jumping for joy, although I would recommend you put the headset down first.
Now that you’re are all setup, all that is left to download the rest of the Mixed Reality setup, including the VR house that you begin with for experimenting (or kicking back and watching movies on the largest screen EVER!)
Now you are ready, Don your headset and start exploring.

As you can see, the setup experience has been well thought through and without the need for extra camera’s and sensors, the process is just as simple and plug-in, walk round your room and you are done. For the basic seated experience (e.g. space sim, driving, rollercoaster, etc) it’s even easier.

First experiences

As soon as you have the Headset up and running and attached to your head, you are entered in to the default Mixed Reality environment, that being your 3D virtual house. This includes such locations as:

A veranda with an excellent view
A library like area with shelves and some art
A visitor area with a globe, pictures and applications running on the wall
Finally, my favourite room (where I spend most of my time), the 100 ft cinema room.

There’s a lot to see and you start to get a feel for how the basic Mixed Reality controls work, which are pretty much the same as any other VR experience.

Managing the setup

If you know Windows 10, you will already be familiar with the “Settings” application, which is simply accessed by clicking on the start button and then clicking on the COG. It’s nice to see Microsoft are keeping trend with this when it comes to Mixed Reality, so the config is all under its own banner in the Settings application. No faffing and searching for hidden options.

It’s also nice to see that this option is only available once you have completed the “Mixed Reality Portal” setup and is removed when the Mixed Reality extensions are uninstalled (yes you can remove them). Behind this are a small selection of options to customise your setup, including:

Audio and Speech Setup

Includes options to:

Force the PC to switch to using the Headset’s audio port by default
Force the PC to switch to using the Headset’s microphone audio port by default
Is speech recognition (aka Cortana) enabled in Mixed reality
(Allows you to control your environment and interact with it, might not be good in games)

Plus, it includes some feedback and help options (actually, these are every screen so they are always to hand)

Environment Controls

This will reset any boundary or scans of your environment from the PC.
Useful if you travel and need to re-setup the experience.

It also includes options to “Reset Your Home” VR environment.
(Like when your kids fill it with animals and delete all your stuff)

Headset display options

Through this screen, you can alter your perceived experience, this includes options to:

Increase or decrease the Visual Quality pushed to the headset
Useful if you PC is struggling or you want to see if it can perform at higher settings
The Default is “Automatic” where the Mixed Reality setup will manage it for you
Calibration
If you are regularly having issues focusing on the display, you can tweak the focal distance the eyepiece uses to display visuals
This can help especially if you use it without your glasses or have regular trouble.
Usually tightening the headset band. There is enough space to also use it with your glasses on.

Uninstall

If by some chance you’ve had enough, are returning the headset or (more likely) transferring to another PC, you can remove all the Mixed Reality config from the PC

It is also useful if you had an issue in the install / setup and just want to reset the entire process and start again.

Now VR truly has a chance of being great with it so readily accessible.