Windows Mobile Development – 6.1.4 – Scraping the Weather

This demo will build things from scratch. I will post the final source at the end of this article. It is meant to get you up and running. Getting the internet portion to work is kind of tricky.

 

Here is what we are trying to build:

 

Yakutsk is supposedly the coldest place on earth, at least city wise.

 

Setting up For Windows Mobile Development

 

There are several installations you'll need to get started. This setup assumes Vista as your OS. Sorry XP users. If anyone wants to get this to work on XP, I will post the steps.

 

Software	Version and Location
Visual Studio 2008 Professional	You can download the trial varsion
Visual Studio 2008 Service Pack 1	Includes .NET framework 3.5 sp1
ASP.NET AJAX Control Toolkit	AjaxControlToolkit-Framework3.5SP1-NoSource.zip
Mobile 6 SDK Refresh Download	https://www.microsoft.com/downloads/details.aspx?FamilyId=06111A3A-A651-4745-88EF-3D48091A390B&displaylang=en
Mobile 6.1.4 Emulator Images	https://www.microsoft.com/downloads/details.aspx?FamilyId=1A7A6B52-F89E-4354-84CE-5D19C204498A&displaylang=en
Virtual PC 2007 – SP 1 May not be needed, depending on how you want the emulator to get network access.
Windows Mobile Device Center May not be needed, depending on how you want the emulator to get network access.

 

Creating the Project

 

File, New Project. Choose your project name and directory. I ran Visual Studio as administrator, just for the fun of it. I like giving myself maximum permissions.

 

Choose SmartClient application.

 

 

Choose Windows Mobile 6 Professional SDK

 

 

You should end up with a default project looking like this:

 

 

It is time to start adding controls to the form. We want to add only two controls (listbox and a web browser control).

 

But first get the toolbox ready.

 

 

Drag a ListBox control over. While in design mode, right mouse click on the listbox and select properties.

 

 

 

Choose Properties.

The list box needs to hold a list of cities. Click on the ellipses.

Click this link [ list of cities ] to get the list of cities below.

 

 

Your form should look like this:

The user will click on the city to view the weather there.

 

In Design mode (meaning you are not running), double click on the list box.

 

You want to end up with something like this:

 

Here is the code snippet [ click here ]

 

 

The tricky stuff – trying to get the code to run in the emulator. There are two approaches and I'm taking approach #2.

I avoid the headaches of Mobile Device Center, doing the "Cradle" thing, Virtual PC. I avoid a proxy all together. Perhaps that is the only way on YOUR system. But for me, less trouble using method #2.

 

 

 

 

I am at home so verify your wireless network settings.

 

 

On your desktop machine, start Internet Explorer. Choose Tools, Options, Connections. Then click on "LAN settings."

 

 

I have no need for a proxy at home, so just leave alone in my case.

 

Now we need to change some settings in the emulator phone. Select Settings.

Select Network Cards.

 

 

Choose The Internet. Also choose NE2000 Compatible Ethernet Driver.

 

 

Verify that you got a correct IP address from DHCP.

 

 

Now type in IPCONFIG /all at a DOS prompt to determine your DNS Server.

 

 

Type in the DNS address. In my case it was 68.87.76.178

 

On your system, it will probably be a different number.

 

 

Test your connection.

 

 

 

Running our sample

 

Select Debug, Start Debugging from Visual Studio

 

Choose the emulator that you previously selected.

 

Click on a city to test

 

 

Now it's time to actually start the screen scraping process.

 

Add a new class

 

Call the new class MyWeatherData

 

 

What the project looks like. Now we need to start adding code. We will start with the constructor.

 

 

First, make the class public.

 

 

You will need to add 3 sections. See the figure below for each of the 3 sections.

Section 1:

Add the appropriate "using" statements.

Section 2:

Add private data. Also add one public variable. The private data elements will be used as we parse the web page.

Section 3:

Add a constructor. Notice the constructor takes two arguments. When the user selects a city from the listbox, we will pass the appropriate city code and zip code to the constructor. Next, a URL will be constructed to retrieve the appropriate web page.

Notice GetData() still needs to be coded up.

 

To get all the source code above, follow the link to MyWeatherData Step 2.

 

Implementing GetData()

Here is where we do most of the grunt work.

 

 

Here is a breakdown for methods in the class:

 

Notice that all the Get??() functions call FindSubstrings(). Here is what the GetCurrentTemp() function looks like. The other Get??() functions are very similar. I could use some more refactoring.

 

 

The next step is to understanding the FindSubstrings() function. It uses Regexp(). We use the method "Matches()" to actually extract the weather data we are interested in.

 

 

To get all the source for the MyWeatherData class, go to MyWeatherData Step 3.

 

Finally, the form code needs some tweaking. Specifically, when the user chooses a city we need to exercise the MyWeatherData object appropriately.

Add a web browser control.

 

To get the form code, simply download the whole project. The whole project – BrunoForecast.zip

 

Happy weather reporting! To add more cities, simply go to the CNN site and see what parameters are needed.

 

Feel free to make improvements and email them back to me. I will update my code base and give you full credit !

 

BTW, anybody been here? Yakutsk sounds cool. Dumb pun intended.

 

 

Keywords Used in this Code:

 

Regex regex = new Regex()

Match oM = regex.Match(rawHtml);

SortedList uniqueMatches = new SortedList();

Match[] retArray = null;

Regex RE = new Regex(matchPattern, RegexOptions.Multiline);

MatchCollection theMatches = RE.Matches(source);

uniqueMatches.ContainsKey(theMatches[counter].Value)

StringBuilder sb = new StringBuilder();

foreach (Match m in x1)