Windows CE Tools BSP bringup project: x86 emulator BSP bringup for Windows CE 6.0
(Posted By: Craig Friesen)
Hi, this is Matt Johnson, Jalen He, Nachiket Acharya, Joel Clark and Craig Friesen of the Platform Builder Tools Test Team. We've been given two weeks from our normal duties of writing and running tests against the CE tools to actually sit down and try to use our tools, read the CE documents and see if we can make our way through some sort of BSP bring-up project.
At this point, we're not completely sure which path we want to take, but we know that we want to get some sort of x86 emulator BSP to run Windows CE 6.0. We have several options for this and we'll be taking the next couple days to consider options and choose which path we're going to take. We could:
1. Take the Device Emulator that ships with VS 2005 and Windows CE 6.0 and try to convert it from emulating ARM code to executing x86 instructions. Theoretically, this should be a fairly easy path since we wouldn't need to translate op codes between two different CPU architectures, but just run op codes that are already native to our desktop machines.
2. We could try to take the old x86 emulator that shipped in older versions of PB and see what it takes to get a BSP for a newer release of Windows CE running on that.
3. We could take the Virtual PC product, see if we can clone and tweak the Windows CE 6.0 CEPC BSP, by adding the needed drivers for that, which would include, but might not be limited to, a DC21x4 NIC driver.
4. We could take one of the open source/public domain emulators (for ARM or x86 or even some other CPU architecture) and convert that to run x86 op codes and get drivers to work on Windows CE 6.0.
So, for now our team's going to focus on 1) ramping up on BSP creation/porting (by both upgrading from 5.0 and by cloning from other 6.0 BSPs) and 2) taking a look at what various emulators have to offer.
--------
Reading lots of CE documentation https://msdn2.microsoft.com/en-us/library/ms376742.aspx.
One happy surprise (not sure why I was surprised, but I am) is how easy it was to find the sections on bringing up and porting BSPs. Platform Builder doesn't have a silver bullet "BSP upgrade wizard" -- I suspect it's too complex of a problem for a wizard, but I'm happy to see that our docs team has some good docs on how to do this by hand. Reading through this, I see a few minor problems, which we'll open bugs for, but so far they seem to do a reasonable job guiding the embedded developer through the steps.
In the meantime, Matt is slugging away at seeing what it will take to get the Virtual PC to boot as a CEPC. It sounds like converting the core of some of the other available emulators is just biting off a bit too much to chew in the limited time that we've got for our project, so we'll go with this option for now and maybe revisit the other options if we hit road blocks or find that we have some extra time to explore other ways of doing this.
But while it might be an easier path, going with the VPC doesn't solve all our problems. Unfortunately, where CE swerved to the right at one point and dropped DC21x4 support, the VPC team swerved to the left and only uses this chip set in their network card emulation. Hopefully if Matt can dig up the old DC21x4 drivers (4.2? 5.0?) and clone them for our BSP we might be able to patch together a boot loader and get this to work!
---------
Great news, with some help from Doug Cook (famous on newsgroups for answering all sorts of tools questions) they've managed to find the old driver source and are working to get the thing into our VPC CEPC BSP.
---------
In the meantime, Nachiket and Joel are busy at work getting a VPC CEPC BSP into Yamazaki. Nachiket's going the (hopefully) easier of the two routes, cloning the CE 6.0 CEPC BSP, while Joel's got a little more of an uphill battle trying to upgrade from the CE 5.0 x86 emulator BSP. Apparently there are significant changes to the driver model for 6.0, so I'm not sure how much this is going to impact Joel's path. Juggs Ravalia did a talk on the new driver model not to long ago, if you're interested: https://channel9.msdn.com/Showpost.aspx?postid=233119
I'm really curious if Joel's going to be able to port all those drivers in our small window of time. I think Joel is also proving why we don't simply have just a PB "upgrade BSP" wizard, it's not a trivial task that could be easily automated away.
---------
Well, after talking to Joel today, I'm not so sure of my confidence in the documents. It looks like the docs give more of an overview than a real step by step guide how to do the BSP upgrade.
---------
I just dropped by Jalen's office and saw a CE 6.0 desktop running on the VPC. It's not really our CE-VPC BSP, really, but it's still pretty cool. It's really just a 6.0 CEPC image that he loaded into VPC using a CEPC boot disk. Still, good to know that there's nothing hidden in the VPC that blocks running a straight CEPC image.
One thing I should mention here is that while the VPC is certainly capable of running the image, without networking support in the bootloader/CEPC boot disk, he couldn't have gotten the image loaded into the VPC to begin with. I guess Matt and Doug got some form of the network driver to work in the bootloader.
---------
Today I sat in with Jalen, to see how we're getting the DC21x4 driver into our VPC CEPC BSP.
It turns out that we actually need to get 2 CE21x4 drivers working. One for KITL to use to send debug spew, but since it halts the kernel while it does this, it doesn't interact with the OS. The other is a driver that the OS can interact with for usual networking support. Matt's working on the KITL level driver, Jalen on the one that will actually go into the BSP as part of the OS. I should also mention that the driver Matt's working on is the one that is being used in the boot loader. I was under the impression that we were using the same driver in the OS as the bootloader, but this wouldn't make sense. The one in the OS would have to play nicely within the CE OS and have the OAL to interact with, the one used for KITL and in the bootloader either has no CE OS running or no CE OS loaded, so it has to be a different driver.
Actually, Here's an explanation from Doug that will more clearly explain this.
[Doug Cook] For download and KITL, there is no scheduler and almost no OS services are available. You make what is called an "EDBG" driver. The EDBG driver is a LIB that is statically complied into eboot.exe (download) and kitl.dll (KITL). It is the simplest driver that could possibly work. When it needs to wait for something, it just loops, wasting CPU time until it has waited long enough. While this is tiny and simple, and works even when the OS is stopped at a breakpoint, it is very inefficient.
[Doug Cook] For normal ("Product") ethernet, you have a full OS and kernel. You make what is called an "NDIS" driver. The NDIS driver is a DLL that is dynamically loaded as needed by the OS when the OS detects a compatible network card. It is a full-featured driver, including various parameters to tune the driver for different hardware, buffer sizes, etc. It works in coordination with the scheduler for things like timing and memory mapping. For example, when it needs to wait for something, it puts its thread to sleep or waits for an event or waits for an interrupt, so other threads can do their work while the NDIS driver is waiting. While this is very efficient and performs well, it is complicated and doesn't work when the OS is stopped at a breakpoint.
Thanks Doug!
The first step for Jalen was pretty straightforward -- wrapping the driver with a catalog item, using the catalog editor to create a new catalog item for it. We decided to just hack it into the MS catalog file for network cards, though I think the best practice here would have been to add it to the catalog file for our newly cloned BSP.
Again, here's Doug with some guidance:
[Doug Cook] Depending on where you build the driver and how it is supported, it would either go in the network card catalog file (if we build in public\common\oak\drivers and support the driver on any hardware) or in a BSP's catalog file (if we build in platform\<bsp>\... and support the driver only on that BSP). I put my NDIS driver under platform\cepc\src\drivers\VirtualPC\ndis_dc21x4.
We should probably call out here that for external customers, the cloning route is the best practice, not trying to change the Microsoft catalog file for network cards.
The harder part is tweaking the build system logic to say that unlike the CEPC BSP, we want our driver, not the NC2000 driver, to be included in images using our BSP. This is mostly a matter of hacking into batch files that contain this logic under our platform folder. We made a few steps of progress, I'll let Jalen keep working on it and check back later.
While we're finding a few quirks with the catalog editor and catalog items view (we'll make sure to log those bugs), it doesn’t really seem to be getting in our way, and while the catalog editor isn't designed to directly edit the files where the catalog item dependency logic lives, they give us some good hints where to look, particularly using the dependency viewer, or using the context menu item for finding out why a particular catalog item was included in the OS Design even though it wasn't user selected. Just right click on a feature that's been included in OS Design as a dependency (the green square icon, not the checkmark icon) and seeing what other item(s) brought that in.
---------
I checked in with Joel and he seems to be making some good progress on the upgrade path. Some of the docs seem a little sketchy, but I think that what's probably going on is that the BSP upgrade documentation focuses more on the overall directory/file layout structure, but doesn't give you all the nuts and bolts details about how to change your sources/dirs files or debug build issues in general while your migrating your components to that new structure. But that's more general CE build knowledge than BSP migration specific, so I'm curious if we'd have a different view if we had more CE build system knowledge under our belts.
---------
I was talking to another group here who is upgrading a different BSP and was surprised to hear that they also didn't need to change any driver source files. I was really expecting that we'd have to rewrite drivers to get them to work with the new driver model, but it sounds like it's usually just moving source files around and update sources/dirs files to match, not actually rewriting lots of C code. The other team tells me the didn't need to touch any C code to get their BSP migrated.
---------
OK, looks like Nachiket and Jalen have gotten the old source for the driver refactored to meet the 6.0 model, are able to get it to build and are now playing around with bib files and build logic to get the driver included in the final image. It looks like we managed to get the driver in on the first attempt, but unfortunately, it was added to the wrong bib file (common.bib) and was forcing the driver to go into any CE image. But we soon figured that out and are now using the platform bib file so that it only, properly, gets included in images using our BSP.
Here's Jalen's clarification of what this really means:
[Jalen]: There are three levels of including a module into a image.
Level one: include the module by modifying the common.bib file. This way applies to all CE images.
Level two: include the module by modifying the platform.bib. This way applies to only the specific BSP images.
Level three: include the module by modifying the project.bib of the particular OSDesign project. This way applies THIS particular project.
The best practice for cloning a new BSP and if you want to the new module only to be included in the new BSP, you may want to use Level two so that the new driver will only show up in the new BSP but not the other BSP’s to reduce the confusion.
Matt's busy working on getting KITL to work. Once we get network connectivity and KITL support, we're on our feet in terms of having a working image that we can run, connect to and debug.
---------
I think we've learned a good lesson over the last week. There's been a few times that we've gotten stuck trying to use the documentation, but we've found that a good alternative path to try when that happens is to look at other already existing BSPs. For example, Joel figured out a few problems by looking at the 5.0 CEPC BSP and comparing it to the 6.0 CEPC BSP. For issues in cloning, Nachiket and Jalen have found that looking at other 6.0 BSPs are implemented have gotten them past a few points where they've been stuck cloning/porting components.
Here's Jalen again with some more helpful information:
[Jalen]: The driver team spent most of first week figuring out how the system worked by understanding things we used every day, not just used for granted, but understand it how and why it worked this way. I tried to manage to get the image built with the driver included but was unable to debug it and thought we were blocked by the ongoing debugging work. By Monday, we finally figured out another way without having to waiting for the debugging work through Ethernet. You are right. Through serial. I didn’t know where we got this idea but we all thought the debugging was not working in any way but it’s not true.
[Jalen]: Here is how to debug VPC through serial cable:
1. Have two XP machines. On Machine One: install Virtual PC and have the bootme disk on the floppy drive. On machine Two: Install VS2005 and PB6.0.
2. Configure:
On Machine one: Start the Virtual PC and mapping the Port1 of the Virtual PC to the real serial Port1 on the XP machine.
On Machine two: open the Connectivity dialog and configure the download as “Ethernet”, Transport as “Serial” and Debugger as “KdStub”.
Connect the serial ports between Machine one and Machine two.
3. Now build an image on Machine Two.
4. Start the Virtual PC. It is booted from the bootme disk and bootme packet is being sent.
5. On Machine two, find the newly booted machine (Machine one) and download the image. Now you can start debugging.
[Jalen]: Our original hope for migrating the dc21x4 driver from CE4.0 was in the hope that it should work without modifying much code. After spending several days debugging the migrated driver through a slow serial cable, I found that the driver got loaded first and unloaded right away after failing this function:
NdisReadPciSlotInformation(
MiniportAdapterHandle,
Adapter->SlotNumber,
PCI_CFID_OFFSET,
&DC21X4PciConfiguration,
sizeof(ULONG)
);
[Jalen]: It tries to read a PCI slot that has the matching information for this driver and it couldn’t find any. I figured that migrating a device driver from CE4.0 to CE6.0 Virtual PC might not be as easy as what we thought before. After talking to Doug Cook, who wrote his first device driver at 6th grade and have been working on CE for quite a long time. Here is the problem he found in his own words:
[Doug]: “We started with the NDIS driver code from the Macallan Emulator BSP. We replaced instances of the deprecated "NdisStallExecution" function with the more efficient "NdisMSleep" function. We updated the registration code to work with Windows CE's PCI plug-and-play support, removing legacy non-PnP code. This allows any number of cards to be automatically detected and initialized via PCI enumeration. We also fixed the SROM read code. It was signaling a “Write Operation” instead of a “Read Operation” when reading the contents of the SROM. After fixing the read code, it was no longer necessary to comment-out the “#if 0” block surrounding the code waiting for the SROM to respond to the read command.”
[Doug]: “Oh, and we redid the interrupt handler to use giisr.dll to allow for IRQ sharing (necessary since all network cards on Virtual PC are hardwired to the same interrupt).”
[Jalen]: Device driver is a very complicated part in CE and I think if given another chance, I’d prefer devoting all our time just to bring up a driver or two, instead of spending much time on building an image and making debugger working through Ethernet. Since it requires deep understanding of how it works and what the differences are between the old driver mode (pre-CE6.0) and the new driver mode (CE6.0). But overall, this project is an excellent chance for all of us to take some time off the regular work to look at some areas that we are using but barely have time to understand, and learn.
Yeah, I’d agree with what Jalen says – device driver work is an interesting challenge, and we’re probably stretching a bit to try and ramp up and deliver something within such a small window. But I’m glad we had this opportunity to take time off from our regular tasks to dig into this. Hopefully we can do this again, and when we do, we can focus more deeply on one area.