Before even going down the path of writing a WPF gadget, it is to decide if the cost is worth it. Can you achieve the visual effect you are seeking without WPF? If the answer is no, read on and find out what you are getting yourself into if you choose to use an unsupported gadget technology.
One of the big limitations with WPF gadgets is that WPF doesn't "play nicely" with the rules of being a gadget, because it swallows mouse events and doesn't propagate them back up to the UI. As such, by default, WPF gadgets won't show the default gadget UI on right click. And, also by default, WPF gadgets won't display the gadget thumb and close icons which are displayed on mouse over. There are work arounds to these problems that I will discuss below, but you can already see that the Sidebar wasn't designed for WPF.
Another limitataion is that a WPF gadget can't get at any of the Sidebar specific APIs,. So, there is going to be some clunkiness to do things like getting the flyout window to display. Now, some of the system APIs have a .NET equivalent, although they require a certain permission level. When using loose XAML or XBAPs in an IFRAME, you will be sandboxed.
Another limitation has to do with performance and working set implications. The user is going to pay a start up cost for loading the WPF engine the first time the sidebar is instantiated, assuming that WPF hasn't been instantiated yet on the system, which is most likely the case since the Sidebar, by default, loads when a user logs in. Additionally, because of the working set required by WPF and the CLR, the working set of the sidebar itself is also going to creep up. Lastly, when the sidebar closes, either at log off or when closed by the user, it has to unload WPF and the CLR. I've seen some unsavory behavior here if your WPF has opened resources that the sidebar has trouble unloading, the result making for an icky user experience when the sidebar unloads. At this point, I haven't isolated exactly what the behaviors are that cause this problem, but I've seen it. Now, all of these problems are tractable, but they are costs that need to be known up front. You certainly don't want to write a gadget that causes problems or makes for a crummy experience.
With these problems in mind, let's look at the first technique: using IFRAMEs
All three of the gadgets pictured above are available w/ source. To install the gadgets, just double click the .gadget files.
Gadgets support the IFRAME tag, which means that you can reference loose xaml or XBAPs from within a gadget. There are a few downsides to this techniques. First, IE has to install the XBAP (or, in the case of loose XAML, the XAMLReader) into the IE application cache. There is no way to preload your XBAP and no way to override the default load UI. So, the user experience is a little funky. Second, if for some reason the IE application cache gets cleared, the XBAP will have to be reloaded the next time the gadget is run. The more likely scenario here is that the gadget's XBAP could be bumped out of the cache due to space limitataions by another XBAP getting installed. The moral of the story being that when using this technique you lose some control over the experience as a whole.
Here's a gadget that hosts a loose XAML file of a rotating 3D cube displaying different images:
To use an XBAP, you simply replace the .xaml file with an .xbap file. The main difference between an XBAP and loose XAML is functionality. You'll notice that the XBAP supports a trackball control, allowing the user to rotate the cube and scale it with the mouse wheel. This was made possible by writing C# code, as can be seen in the source.
Remember early I mentioned some workarounds for the mouse propagation problem. You can see the workaround to the mouse hover problem where I set the padding of the gadget to 5 pixels. Another technique would be to put the IFRAME in an html table. I then make the background color of the padding identical to the background color of the WPF airspace. I lose some screen real estate, but at least I get the thumb and minimize UI to show up on the gadget, although if you move the mouse really fast, it doesn't work. I haven't figured out a workaround to the righ click problem yet. (If anyone does, let me know!)
Another interesting fact about XBAP gadgets is that, if they are run from the local filesystem, they can be full trust. However, if they are deployed from a server, they would have to be partial trust unless the cert that signed the XBAP is registered in the user's trusted publishers certificate store.
(Note: thanks to the artistry of Maureen Lanza for the images.)