How to trace CryptoAPI calls (2)

Hi, welcome back,

Let’s try to understand a bit better what’s going on my CryptoAPI Tracer script

Let’s take a look to one of the most important breakpoints I set on a CryptoAPI function:

bm Advapi32!CryptAcquireContextW “.printf \”\\n>>>>>>>>>>>>>>>>>>>>>>\\n\\nCryptAcquireContextW (%#x)\\n\”, @$tid;    .echo;.echo IN;    .echo pszContainer; .if(poi(@esp+8)=0) {.echo NULL} .else {du poi(@esp+8)};    .echo;.echo pszProvider; .if(poi(@esp+c)=0) {.echo NULL} .else {du poi(@esp+c)};    .echo;.echo dwProvType; .if(poi(@esp+10)=1) {.echo PROV_RSA_FULL} .elsif(poi(@esp+10)=0x18) {.echo PROV_RSA_AES} .else {.printf \”%d\\n\”, poi(@esp+10)};    .printf \”\\ndwFlags\\n%#x\\n\”, poi(@esp+14); .if((poi(@esp+14)&0x0`F0000000)=0x0`F0000000) {.echo CRYPT_VERIFYCONTEXT(0xf0000000)}; .if((poi(@esp+14)&0x0`00000008)=0x0`00000008){.echo CRYPT_NEWKEYSET(0x8)}; .if((poi(@esp+14)&0x0`00000010)=0x0`00000010) {.echo CRYPT_DELETEKEYSET(0x10)}; .if((poi(@esp+14)&0x0`00000020)=0x0`00000020) {.echo CRYPT_MACHINE_KEYSET(0x20)}; .if((poi(@esp+14)&0x0`00000040)=0x0`00000040) {.echo CRYPT_SILENT(0x40)};     bp /t @$thread poi(@esp) \”    .echo;.echo OUT;    .if(poi(@esp-14)=0) {.echo phProv;.echo NULL} .else {.echo hProv; .if(poi(poi(@esp-14))=0) {.echo NULL} .else {.printf \\\”%#x\\\\n\\\”, poi(poi(@esp-14))} };    .echo;.echo RESULT;    .if(@eax=1) {.printf \\\”CryptAcquireContextW (%#x) SUCCEEDED\\\\n\\\”, @$tid} .else {.printf \\\”CryptAcquireContextW (%#x) FAILED\\\\n\\\”, @$tid;!gle};    .echo;.echo <<<<<<<<<<<<<<<<<<<<<<;    G;\”;    G;”;

It looks a bit complex, but it’s not so much. Some of the basics where already explained here.

Imagine we have the following API declaration in C:

int __stdcall myFunction(int a, int b, int c);

If I set a breakpoint on myFunction, and because we use __stdcall calling convention (the same as Win32 API/CryptoAPI functions), the stack will look this way just when we reach the breakpoint, before we run any code on myFunction:

return address <– esp
a <– esp+4
b <– esp+8
c <– esp+c

esp” (stack pointer) register will be pointing to the address where we’ll return after finishing executing myFunction. If we want to i.e. access the second parameter of the function, we can use the address esp+8.

When we finish executing myFunction and we return from it,eax” register will contain the return value of the API and the stack will look like this:

xxxxxxxx       <– esp

Now, if we want to access one of the parameters of the API we just called (i.e. because it’s an out parameter), we have to remember that the information of the stack is not cleaned. So even if “esp” register points to “xxxxxxxx” now, the parameters of the API are still on the stack:

a              <– esp-c
b <– esp-8
c <– esp-4
xxxxxxxx <– esp

So if we want to i.e. access the first parameter of the API we can use esp-c address.


Having said this, let’s write the breakpoint above in a different way:

000 bm Advapi32!CryptAcquireContextW
001 ”
002 .printf \”\\n>>>>>>>>>>>>>>>>>>>>>>\\n\\nCryptAcquireContextW (%#x)\\n\”, @$tid;
004 .echo;
005 .echo IN;
007 .echo pszContainer;
008 .if(poi(@esp+8)=0)
009 {
010 .echo NULL
011 }
012 .else
013 {
014 du poi(@esp+8)
015 };
017 .echo;
018 .echo pszProvider;
019 .if(poi(@esp+c)=0)
020 {
021 .echo NULL
022 }
023 .else
024 {
025 du poi(@esp+c)
026 };
028 .echo;
029 .echo dwProvType;
030 .if(poi(@esp+10)=1)
031 {
032 .echo PROV_RSA_FULL
033 }
034 .elsif(poi(@esp+10)=0x18)
035 {
036 .echo PROV_RSA_AES
037 }
038 .else
039 {
040 .printf \”%d\\n\”, poi(@esp+10)
041 };
043 .printf \”\\ndwFlags\\n%#x\\n\”, poi(@esp+14);
044 .if((poi(@esp+14)&0x0`F0000000)=0x0`F0000000)
045 {
046 .echo CRYPT_VERIFYCONTEXT(0xf0000000)
047 };
048 .if((poi(@esp+14)&0x0`00000008)=0x0`00000008)
049 {
050 .echo CRYPT_NEWKEYSET(0x8)
051 };
052 .if((poi(@esp+14)&0x0`00000010)=0x0`00000010)
053 {
054 .echo CRYPT_DELETEKEYSET(0x10)
055 };
056 .if((poi(@esp+14)&0x0`00000020)=0x0`00000020)
057 {
058 .echo CRYPT_MACHINE_KEYSET(0x20)
059 };
060 .if((poi(@esp+14)&0x0`00000040)=0x0`00000040)
061 {
062 .echo CRYPT_SILENT(0x40)
063 };
065 bp /t @$thread poi(@esp)
066 \”
067 .echo;
068 .echo OUT;
070 .if(poi(@esp-14)=0)
071 {
072 .echo phProv;
073 .echo NULL
074 }
075 .else
076 {
077 .echo hProv;
078 .if(poi(poi(@esp-14))=0)
079 {
080 .echo NULL
081 }
082 .else
083 {
084 .printf \\\”%#x\\\\n\\\”, poi(poi(@esp-14))
085 }
086 };
088 .echo;
089 .echo RESULT;
091 .if(@eax=1)
092 {
093 .printf \\\”CryptAcquireContextW (%#x) SUCCEEDED\\\\n\\\”, @$tid
094 }
095 .else
096 {
097 .printf \\\”CryptAcquireContextW (%#x) FAILED\\\\n\\\”, @$tid;!gle
098 };
100 .echo;
101 .echo <<<<<<<<<<<<<<<<<<<<<<;
103 G;
104 \”;
106 G;
107 “;

000) We set a breakpoint on the UNICODE version of CryptAcquireContext API. This is the declaration of the API:

BOOL WINAPI CryptAcquireContext(__out HCRYPTPROV* phProv, __in LPCTSTR pszContainer, __in LPCTSTR pszProvider, __in DWORD dwProvType, __in DWORD dwFlags);

The following commands will be executed when the application calls this API and we reach the breakpoint:

002) We print the name of the API and the id of the thread which calls it.

005) We print the string “IN”, and after that we’ll print the values of the __in parameters of the API.

007-015) We print pszContainer parameter as a string of Unicode chars, after checking if it’s NULL or not.

018-026) We print pszProvider parameter as a string of Unicode chars, after checking if it’s NULL or not.

029-041) We print dwProvType parameter. If we know the type we print it in clear text, otherwise we just print its numeric value.

043-063) We print dwFlags parameter in Hexadecimal. If we know the values of those flags, we print them in clear text.

065) Now we want to know what happened after the API finished executing. We set a breakpoint at the return address where we will return after calling the API. If we don’t want weird things to happen on multi-threaded applications, we have to specify on which thread (“$thread” pseudo-register) we are setting that breakpoint.

106) Once we’ve set the breakpoint at 065, we can continue the execution of the application.

Now, let see what happens when we stop on breakpoint set at 065:

068) We print the string “OUT”, and after that we’ll print the values of the __out parameters of the API.

070-086) We print phProv parameter. This is a pointer to hProv. If the pointer is not NULL, we print the Hexadecimal value of hProv, after checking if this value is NULL or not.

089) We print the string “RESULT”, and after that we’ll print the return value of the API, and the error if there was one.

091-098) We show if the API succeeded or not. If not, we also show the error value that the API returned.

103) Now that we know the result of calling the API, we can continue the execution of the application and trace more APIs.



I hope this helps.



Alex (Alejandro Campos Magencio)

Comments (2)

  1. maxdm says:

    I have CAPI tracer based on detours library. I can upload it somewhere if you are intersted.

  2. alejacma says:

    Hi maxdm,

    Thank you very much for the offering. I already have such a CAPI tracer dll based on detours. A colleague of mine from the US developed it (maybe it’s the same one you have) and I have the source code and all. I used to work with it before I developed my script. But this dll wasn’t working well in all scenarios (it crashed some processes, you needed to restart the process so it could load the dll, some customers didn’t like to use it in production environments, etc.), and that is why I decided to use a debugger script instead. I’ve never used detours again. The biggest advantage of the script is that I can make any changes to it (i.e. remove traces of some API, or show a parameter in a different way) without the need of recompiling any dll. Detours solution is very cool, too, but for me it’s more limited than the script.