Since Windows CE is so componentized, it's possible that the CE image that you're using doesn't have the component that you're looking for. This is even more complicated on CE because one DLL may be built from various optional components. An example is MSXML. I can see that I have some MSXML support by seeing that MSXML3.dll is in my image. But MSXML on CE is componentized such that you can get SAX by itself, or DOM, XSLT, XQL, SAX, XMLHTTP, etc... What if any of these sub-components do I have? I don't want to have to write a test app to try out XQL, SAX, etc... and see if they fail because (1) I'm busy and (2) I may have an error in my test app, making me think I don't have the component on my device when in reality I do.
To make the process of determining what you do and don't have, first check out ceconfig.h. This header file is included in the _FLATRELEASEDIR (the directory that contains all DLL's/EXE's before merging them together for a ROM on the device). CEConfig.h (at least in recent OS versions) is also included on the device itself, in \windows. The file is just a simple list of all components that were brought in this particular build. The list of all the MSXML related #defines in my current image makes it much quicker to get a general sense of what components I have and don't.
#define MSXML3_XMLDOM 1
#define MSXML3_XMLMIME 1
#define MSXML3_XMLXSLT 1
#define MSXML3_XMLISLANDS 1
#define MSXML3_XMLXQL 1
#define MSXML3_XMLSAX 1
#define MSXML3_XMLHTTP 1
#define MSXML3_XMLERRORSTRINGS 1
#define MSXML3_XMLNETFULL 1
#define MSXML3_XMLSAX 1
I've found ceconfig.h is particularly handy for ISV's who are not developing on PocketPC's (since PocketPC already has good SDK docs and a well defined set of components that are included and not) and who need to verify that the OEM they're working with has included the right stuff in their OS.
Another trick I use to see whether a particular API is included in the OS or not is using link itself. Basically, on your desktop get a copy of the CE dll and then run 'link /dump /exports <dllname.dll>'. A few days ago I wanted to recommend to a fellow to use CoFreeUnusedLibrariesEx (to unload MSXML3 quickly, of all things) but I wasn't sure if his platform had it. No need to write a dummy test app to determine this. I just did a link /dump /exports on ole32.dll and was pleased to see that CoFreeUnusedLibrariesEx() was indeed included in his platform.
Finally note that on devices like PocketPC's with a file explorer, if you look on the device it may not show you hidden/system DLL's/EXE's. Be sure you're using a complete list. If you don't see \windows\CoreDll.dll, you can be sure that your file explorer is sys/hidden files from you.