Step 4–Augmented Reality, Windows 8, and Cloud Computing–How to implement with real code (Implementing the Windows 8 Client)

Article
11/06/2012

Title	Description	Link
Augmented Reality applications need a Web Service Back-End. Here is a 90-Day No obligation, totally free offer to use Windows Azure as your web service for Windows 8 Clients.	You get: Compute / 750 small compute hours per month, Web sites / 10 web sites, Mobile services / 10 mobile services, Relational database / 1 SQL database, SQL reporting / 100 hours per month, Storage / 35GB with 50,000,000 storage transactions, Bandwidth / unlimited inbound & 25GB outbound, CDN / 20GB outbound with 500,000 transactions, Cache / 128MB, Service bus / 1,500 relay hours and 500,000 messages	https://www.microsoft.com/click/services /Redirect2.ashx?CR_CC=200114759
Step 0: What we will build. Augmented Reality, Windows 8, and Cloud Computing–How to implement with real code	High level introduction to our finished application.	https://blogs.msdn.com/b/brunoterkaly/archive/2012 /11/05/step-0-what-we-will-build- augmented-reality-windows-8-and-cloud-computing-how-to-implement-with-real-code.aspx#
Step 1–Augmented Reality, Windows 8, and Cloud Computing–How to implement with real code	Introduction. What is Augmented Reality	https://blogs.msdn.com/b/brunoterkaly/archive/2012/10/29/ step-1-augmented-reality-windows-8-and-cloud-computing- how-to-implement-with-real-code.aspx#
Step 2–Augmented Reality, Windows 8, and Cloud Computing–How to implement with real code	Building the first part of our Azure back-end.	https://blogs.msdn.com/b/brunoterkaly/archive/2012/10/30/ step-2-augmented-reality-windows-8-and- cloud-computing-how-to-implement-with-real-code.aspx#
Step 3–Augmented Reality, Windows 8, and Cloud Computing–How to implement with real code (Implementing the Cloud Back-End)	This post provides all source code and explanation for the Azure back-end. This is the back-end for the augmented reality Windows 8 Client.	https://blogs.msdn.com/b/brunoterkaly/archive/2012/11/05 /step-3-augmented-reality-windows-8-and-cloud-computing-how- to-implement-with-real-code-implementing-the-cloud-back-end.aspx#
Step 4–Augmented Reality, Windows 8, and Cloud Computing–How to implement with real code ...	This post provides all source code and explanation for the Windows 8 Client (Augmented Reality Demo). The augmented reality Windows 8 Client calls into the Azure back-end described in step 3 above.	https://blogs.msdn.com/b/brunoterkaly/archive/2012/11/06/ step-4-augmented-reality-windows-8-and-cloud-computing-how-to-implement-with-real-code-implementing-the- windows-8-client.aspx#
Source Code - Web Service Back End	This is the Windows Azure Project that Windows 8 Clients call into	https://sdrv.ms/Qoqb1J
Source Code - Windows 8 Client	This is the Augmented Reality Windows 8 Client	https://sdrv.ms/T38uBC

This is a Windows 8 Project

It is running on a Windows 8 OS and was built with Visual Studio 2012
This post will build this application from scratch
It will communicate with a cloud back-end that we created in the last post
The cloud back end is a Windows Azure Cloud Project
- It was deployed to a data center

Client Tier - Windows 8

This post relies on the last post, which was the Front-End Web Services Tier.
It was a Windows Azure Cloud Application (with a web role of type ASP.NET Web
API)
The Windows 8 Augmented Reality Client will not run unless the cloud back-end
exists
- The cloud back-end could be running in a MS data center
- Or it could be running as a Visual Studio 2012 Project using the Windows
  Azure Compute Emulator

Creating a Windows 8 Application

From Visual Studio, select File/New Project/Windows Store Application
Select Blank Application

The default project : Modifying the app manifest file

You are looking a Solution Explorer
- It is available from the VIEW menu
Right mouse click on Package.appmanifest and choose VIEW DESIGNER. This will
allow us to change the capabilities. For example, we want to allow internet
connectivity.

The Application Manifest : Enabling Capabilities

I selected the options you see above.
- Internet, Microphone, Pictures Library Access, Videos Library Access, Webcam
When you run your Windows 8 application, you still need to confirm the use of
your webcam and microphone

The main interface : MainPage.xaml

You can also use EXPRESSION BLEND to edit your graphical interface
Visual Studio 2012 also has a designer, available from the VIEW MENU.

Understanding the XAML

The user interface is just a bunch of XAML controls
It is just a big XML file, similar to an Android user interface
StackPanels and Grids make up most of the interface
Buttons, TextBlocks, Images, Rectangles, Canvases, Radio Buttons are nested
inside of StackPanels and Grids
There is also a CapturePanel for displaying a live video feed from the web cam
This screen shows one of the StackPanels
- It contains nested StackPanels
- It contains 2 buttons and the Radio Buttons for the GPS Simulator
  - The GPS Simulator lets you choose San Francisco, Reno, Denver, or Death
    Valley
  - My system does not have a built-in simulator

Understanding the XAML

The Radio buttons are stacked within a StackPanel

Understanding the XAML

The augmented reality section is simply a grid with borders
The grid contains TextBlocks where we will put the data that we get from calling
our Azure cloud back-end, cloud service
- Populating the TextBlocks with location and weather information is done
  in code-behind (MainPage.xaml.cs)

MainPage.xaml

The ordering of controls is critical.
It allows us to overlay data on top of the live video stream from the web cam.
If you notice, many of the TextBlocks appear AFTER the CaptureElement control.
This controls the z-order of controls.

Line	Comment
35-49	The two buttons. One button is for turning on the webcam and the other for taking a photo. I've illustrated photo taking in a previous post. Contains a reference to an event method (ConfigureCamera_Click)
50-91	The user interface for the GPS simulator. A TextBlock and a bunch of Radio Buttons. All 4 radio buttons have an event procedure because they end up calling into the Azure Web Service for a new GPS location.
98	The canvas to display the augmented reality live video stream from the web cam
103-105	The control that displays the augmented reality live video stream
143, 169, 196, 224, 250	The TextBlocks that will display the augmented reality data coming from the Azure back-end cloud application.
257	Used to store the latest weather satellite image

MainPage.xaml.cs

12 34 56 78 910 1112 1314 1516 1718 1920 2122 2324 2526 2728 2930 3132 3334 3536 3738 3940 4142 4344 4546 4748 4950 5152 5354 5556 5758 5960 6162 6364 6566 6768 6970 7172 7374 7576 7778 7980 8182 8384 8586 8788 8990 9192 9394 9596 9798 99100 101102 103104 105106 107108 109110 111112 113114 115116 117118 119120 121122 123124 125126 127128 129130 131132 133134 135136 137138 139140 141142 143144 145146 147148 149150 151152 153154 155156 157158 159160 161162 163164 165166 167168 169170 171172 173174 175176 177178 179180 181182 183184 185186 187188 189190 191192 193194 195196 197198 199200 201202 203204 205206 207208 209210 211212 213214 215216 217218 219220 221222 223224 225226 227228 229230 231232 233234 235236 237238 239240 241242 243244 245246 247248 249250 251252 253254 255256 257258 259260 261262 263264 265266 267268 269270 271272 273274 275276 277278 279280 281282 283284 285286 287288 289290 291292 293294 295296 297298 299300 301302 303304 305306 307308 309310 311312 313314 315316 317318 319320 321322 323324 325326 327328 329330 331332 333334 335336 337338 339340 341342 343344 345346 347348 349350 351352 353354 355356 357358 359360 361362 363364 365366 367368 369370 371372 373374 375376 377378 379380 381382 383384 385386 387388 389390 391392 393394 395396 397398 399400 401402 403404 405406 407408 409410 411412

using System; using System.Collections.Generic; using System.IO; using System.Linq; using Windows.Foundation; using Windows.Foundation.Collections; using Windows.UI.Xaml; using Windows.UI.Xaml.Controls; using Windows.UI.Xaml.Controls.Primitives; using Windows.UI.Xaml.Data; using Windows.UI.Xaml.Input; using Windows.UI.Xaml.Media; using Windows.UI.Xaml.Navigation; // Some extra using statements for my code using Windows.Devices.Enumeration; using Windows.Media.Capture; using Windows.Media.MediaProperties; using System.Net.Http; using Windows.Data.Json; using System.Collections.ObjectModel; using System.ComponentModel; namespace Windows8AugmentedRealityClient{ public sealed partial class MainPage : Page { private Windows.Media.Capture.MediaCapture m_mediaCaptureMgr; // Our GPS simulator GPSSimulator _gps_simulator; // This object get populated with data from calling into // our Windows Azure Web Service, which was created in the // previous blog private LocationInfo loc_info = new LocationInfo(); public MainPage() { this.InitializeComponent(); // previewElement1 is our view port to see the streaming // video content from the web cam previewElement1.Source = null; // Make sure it is visible previewCanvas1.Visibility = Windows.UI.Xaml.Visibility.Visible; // Initialize our GPS Simulator _gps_simulator = new GPSSimulator(); _gps_simulator.City1 = "San Francisco"; _gps_simulator.Gps1 = "37.788345,-122.404679"; _gps_simulator.IsChecked1 = true; _gps_simulator.City2 = "Reno"; _gps_simulator.Gps2 = "39.545353,-119.815631"; _gps_simulator.IsChecked2 = false; _gps_simulator.City3 = "Denver"; _gps_simulator.Gps3 = "39.752073,-104.979968"; _gps_simulator.IsChecked3 = false; _gps_simulator.City4 = "Death Valley"; _gps_simulator.Gps4 = "36.619937,-117.10415"; _gps_simulator.IsChecked4 = false; // Trap some change events (for debugging purposes) _gps_simulator.PropertyChanged += new System.ComponentModel.PropertyChangedEventHandler(gpsChanged); // Bind our GPS Simulator to the Windows 8 form's DataContext this.DataContext = _gps_simulator; } private void gpsChanged(object sender, PropertyChangedEventArgs e) { string s = "The property:'" + e.PropertyName + "' was changed"; } // Ignored for this demo protected override void OnNavigatedTo(NavigationEventArgs e) { } // Hit when user hits the "Configure" button // This method initializes the web cam to display the live // streaming video content internal async void ConfigureCamera_Click(Object sender, Windows.UI.Xaml.RoutedEventArgs e) { // Initialize Media Capture API try { // Prepare Media Capture m_mediaCaptureMgr = new Windows.Media.Capture.MediaCapture(); var _captureSettings = new Windows.Media.Capture.MediaCaptureInitializationSettings(); DeviceInformationCollection deviceInfos = await DeviceInformation.FindAllAsync(Windows.Devices.Enumeration.DeviceClass.VideoCapture); _captureSettings = new MediaCaptureInitializationSettings(); _captureSettings.StreamingCaptureMode = StreamingCaptureMode.Video; _captureSettings.PhotoCaptureSource = PhotoCaptureSource.VideoPreview; if (deviceInfos.Count > 0) { _captureSettings.VideoDeviceId = deviceInfos[0].Id; } await m_mediaCaptureMgr.InitializeAsync(_captureSettings); } catch (Exception exception) { // Ignore errors for now string s = exception.Message; } // Start the Preview try { previewElement1.Source = m_mediaCaptureMgr; await m_mediaCaptureMgr.StartPreviewAsync(); } catch (Exception exception) { previewElement1.Source = null; } } // This method performs the call into our Azure Web Service // fromt the previous post. This method will pass GPS parameters // to the cloud service and receive back JSON data that will // need to be parsed. private async System.Threading.Tasks.Task CallLocationWebService(string gps) { // Call into the emulator. This assumes you are running the // cloud project from the last post in the backgruond string _location = "https://127.0.0.1:81/api/values?location={0}"; // You can use the line below once you deploy your cloud // application to the cloud (a MS data center) //string _location = "https://locationwebservice.cloudapp.net/api/values?location={0}"; // Now make the aynchronous call. We need to pass the GPS // parameters here to the _location string mentioned above. using (HttpClient clientlocation = new HttpClient()) using (var response = await clientlocation.GetAsync(string.Format(_location, gps))) { if (response.IsSuccessStatusCode) { string webresponse = await response.Content.ReadAsStringAsync(); // Parse the string into a JSONObject var parsedResults = JsonObject.Parse(webresponse); IJsonValue val; // Extract data embedded in JSONObject. // Assign to controls in user interface if (parsedResults.TryGetValue("latitude", out val)) loc_info.latitude = val.GetString(); if (parsedResults.TryGetValue("longitude", out val)) loc_info.longitude = val.GetString(); if (parsedResults.TryGetValue("bus_and_neighborhood", out val)) loc_info.bus_and_neighborhood = val.GetString(); if (parsedResults.TryGetValue("elevation", out val)) loc_info.elevation = val.GetString(); if (parsedResults.TryGetValue("bus_and_neighborhood", out val)) loc_info.bus_and_neighborhood = val.GetString(); if (parsedResults.TryGetValue("max_temp", out val)) loc_info.max_temp = val.GetString(); if (parsedResults.TryGetValue("min_temp", out val)) loc_info.min_temp = val.GetString(); this.bus_and_neighborhood.Text = loc_info.bus_and_neighborhood; this.elevation.Text = loc_info.elevation; this.latlong.Text = loc_info.latitude + "/" + loc_info.longitude; this.max_temp.Text = loc_info.max_temp; this.min_temp.Text = loc_info.min_temp; } } } // Checkbox events // Called when user clicks a radio button on the GPS Simulator private async void radSF_Checked(object sender, RoutedEventArgs e) { string s = _gps_simulator.Gps1; await CallLocationWebService(s); } private async void radReno_Checked(object sender, RoutedEventArgs e) { string s = _gps_simulator.Gps2; await CallLocationWebService(s); } private async void radDenver_Checked(object sender, RoutedEventArgs e) { string s = _gps_simulator.Gps3; await CallLocationWebService(s); } private async void radDeathValley_Checked(object sender, RoutedEventArgs e) { string s = _gps_simulator.Gps4; await CallLocationWebService(s); } // A helper method. Not currently used but was valuable during // debugging my controls. public IEnumerable<T> HelperFindMatch<T>(DependencyObject depObj) where T : DependencyObject { System.Diagnostics.Debugger.Break(); if (depObj != null) { for (int i = 0; i < VisualTreeHelper.GetChildrenCount(depObj); i++) { DependencyObject child = VisualTreeHelper.GetChild(depObj, i); if (child != null && child is T) { yield return (T)child; } foreach (T childOfChild in HelperFindMatch<T>(child)) { yield return childOfChild; } } } } // A helper method. Not currently used but was valuable during // debugging my controls. public T HelperFindVisualChildByName<T>(DependencyObject parent, string name) where T : DependencyObject { for (int i = 0; i < VisualTreeHelper.GetChildrenCount(parent); i++) { var child = VisualTreeHelper.GetChild(parent, i); string controlName = child.GetValue(Control.NameProperty) as string; if (controlName.IndexOf("rad") != -1 && controlName != name) { return child as T; } else { T result = HelperFindVisualChildByName<T>(child, name); if (result != null) return result; } } return null; } } // This class represents the data populated by the Windows Azure // Web Service public class LocationInfo { // Data we will lookup public string latitude { get; set; } public string longitude { get; set; } public string elevation { get; set; } public string bus_and_neighborhood { get; set; } public string max_temp { get; set; } public string min_temp { get; set; } } //Implement INotifiyPropertyChanged interface to subscribe for //property change notifications public class GPSSimulator : INotifyPropertyChanged { private string _gps1; private string _city1; private bool _ischecked1; private string _gps2; private string _city2; private bool _ischecked2; private string _gps3; private string _city3; private bool _ischecked3; private string _gps4; private string _city4; private bool _ischecked4; public string Gps1 { get { return _gps1; } set { _gps1 = value; RaisePropertyChanged("Gps1"); } } public string City1 { get { return _city1; } set { _city1 = value; RaisePropertyChanged("City1"); } } public bool IsChecked1 { get { return _ischecked1; } set { _ischecked1 = value; RaisePropertyChanged("IsChecked1"); } } public string Gps2 { get { return _gps2; } set { _gps2 = value; RaisePropertyChanged("Gps2"); } } public string City2 { get { return _city2; } set { _city2 = value; RaisePropertyChanged("City2"); } } public bool IsChecked2 { get { return _ischecked2; } set { _ischecked2 = value; RaisePropertyChanged("IsChecked2"); } } public string Gps3 { get { return _gps3; } set { _gps3 = value; RaisePropertyChanged("Gps3"); } } public string City3 { get { return _city3; } set { _city3 = value; RaisePropertyChanged("City3"); } } public bool IsChecked3 { get { return _ischecked3; } set { _ischecked3 = value; RaisePropertyChanged("IsChecked3"); } } public string Gps4 { get { return _gps4; } set { _gps4 = value; RaisePropertyChanged("Gps4"); } } public string City4 { get { return _city4; } set { _city4 = value; RaisePropertyChanged("City4"); } } public bool IsChecked4 { get { return _ischecked4; } set { _ischecked4 = value; RaisePropertyChanged("IsChecked4"); } } public event PropertyChangedEventHandler PropertyChanged; protected void RaisePropertyChanged(string name) { if (PropertyChanged != null) { PropertyChanged(this, new PropertyChangedEventArgs(name)); } } } }

Thanks for reading

Your assignment is to go to https://programmableweb.com and do your own augmented reality and data mash-ups.Don't forget to send it back to me so I can blog about it Smile

Step 4–Augmented Reality, Windows 8, and Cloud Computing–How to implement with real code (Implementing the Windows 8 Client)

Additional resources