HOWTO: ISAPI Filter which Logs original Client IP for Load Balanced IIS Servers


Invariably, when you run IIS servers that are load-balanced or forwarded requests behind some other network device, you will find that IIS logs the IP of the network device and not the original client that made the request.


Technically, there is no standard describing how to address/fix this situation, so IIS does not have built-in support for this issue. However, there are some popular private hacks used by several network devices, and here is an ISAPI Filter that allows IIS to easily work with those devices to fix this issue and log the original client IP in IIS.


Question:


Hi,


I need some help!


I have a situtation where I am using a hardware web load balancing device (Netscaler) that changes the IP address that the client HTTP request appears to come from. It also adds an HTTP header to the request (X-Client-IP) that contains the “real” IP addres that the client HTTP request came from.


This is all well and good except that the IIS log files do not contain the right client IP address. I would like to “fix” the log file so that it contains the true client IP.


What I think I need is to write an ISAPI filter that gets the X-Client-IP from the HTTP request and then replace the HTTP IP (pszClientHostName ??) before it gets logged.


Am I on the right track? Is there good example code on how to do something like this? My solution would only need to work for IIS6.


Thanks,


Answer:


Ok, this is one of the more popular behavior requests of IIS which can be easily accomplished with a simple ISAPI Filter which extends the functionality of IIS. So, I went ahead and implemented it, and it is attached at the end of this entry. If you need help compiling it, check out my “sample code” RSS feed for more assistance. My assumption is that if you are able to copy sample code, then you are able to compile it as well as support it yourself.


Behaviorally, this ISAPI Filter is very straight forward.



  1. In SF_NOTIFY_PREPROC_HEADERS, use GetHeader() to retrieve the named request header that is supposed to contain the value of the “original” client IP.
  2. This named request header is actually configurable via an INI file directive, as shown through the GetModuleFileName() and GetPrivateProfileString() Win32 calls. It defaults to “X-Client-IP:” (trailing ‘:’ is important to GetHeader() ).
  3. If the request header exists, then allocate memory using AllocMem() and save it in pFilterContext for retrieval later on in that request’s SF_NOTIFY_LOG event.
  4. In SF_NOTIFY_LOG, if pFilterContext != NULL, then it means that the named header was found on the request and pFilterContext contains the new value, so replace pszClientHostName with this value to change what IIS logs. Otherwise, do nothing.

Here are some of the subtle decision points in this filter:



  • I chose to retrieve the request header containing the original client IP using GetHeader() in SF_NOTIFY_PREPROC_HEADERS instead of GetServerVariable() in SF_NOTIFY_LOG because of two reasons:

    1. GetHeader() is able to retrieve any valid request header (including both ‘-‘ and ‘_’ characters) on all IIS versions.
    2. GetServerVariable() is NOT able to retrieve valid request headers that contain ‘-‘ UNLESS you are running IIS6 and use the documented special HEADER_ prefix

      My goal for this sample filter is for maximum compatibility with all IIS versions. Since I lose nothing with my approach and still retain compatibility, I am choosing the win-win approach. :-)

  • The memory used in changing a IIS log entry is allocated with AllocMem() and NOT new/delete. The reason is because IIS requires this memory to stay valid until it has logged the changed value to disk. This means a stack-based buffer is insufficient because it goes out of scope as soon as SF_NOTIFY_LOG finishes, and managing the lifetime of a new/delete buffer is actually icky (on a per-connection basis, you need to delete this memory after SF_NOTIFY_LOG completes but BEFORE SF_NOTIFY_PREPROC_HEADERS fires again. I cannot think of an event to consistently catch this state). AllocMem() is nice because IIS tracks this memory for you and automatically frees it when its associated connection ends. This matches perfectly with the requirements of modifying log entries and eliminates management of that memory buffer’s lifetime. Once again, a win-win situation.
  • pfc->pFilterContext is useful to convey a value generated from one event (SF_NOTIFY_PREPROC_HEADERS) and consumed by a later event (SF_NOTIFY_LOG)

If you want to configure the name of the request header that contains the original client’s IP, put the following text into an INI file located in the same directory and named the same as the name of the ISAPI Filter DLL module (i.e. if the Filter DLL is named “LoadBalancedIP.dll”, then the text needs to be in a file named “LoadBalancedIP.ini” in that directory). You need to change the MODULE_NAME #define to match the LIBRARY directive in your .def file if you want to control the name of the ISAPI Filter DLL (I am assuming it is called LoadBalancedIP.dll). Yes, this behaves just like URLScan.

[config]
ClientHostName=New-Header_Name:

This will cause the value of the request header “New-Header_Name:” to be logged as the client IP in the IIS log file, if it exists.


To have changes in the INI file take effect, you need to cause the Filter DLL to reload (i.e. go through GetFilterVersion() again). For IIS6, it means to recycle that worker process; for prior IIS versions, it means to restart the W3SVC service.


Enjoy.


//David

#include <windows.h>
#include <httpfilt.h>

#define MAX_CLIENT_IP_SIZE 256
#define MODULE_NAME “LoadBalancedIP”

TCHAR g_szHeaderName[ MAX_PATH + 1 ];
TCHAR g_szIniPathName[ MAX_PATH + 1 ];

DWORD
OnPreprocHeaders(
IN HTTP_FILTER_CONTEXT * pfc,
IN HTTP_FILTER_PREPROC_HEADERS * pPPH
);

DWORD
OnLog(
IN HTTP_FILTER_CONTEXT * pfc,
IN HTTP_FILTER_LOG * pLog
);

BOOL
WINAPI
GetFilterVersion(
HTTP_FILTER_VERSION * pVer
)
/*++

Purpose:

Required entry point for ISAPI filters. This function
is called when the server initially loads this DLL.

Arguments:

pVer – Points to the filter version info structure

Returns:

TRUE on successful initialization
FALSE on initialization failure

–*/
{
CHAR * pCursor = NULL;

//
// Locate the config file, if it exists
//
GetModuleFileName(
GetModuleHandle( MODULE_NAME ),
g_szIniPathName,
MAX_PATH );
pCursor = strrchr( g_szIniPathName, ‘.’ );

if ( pCursor )
{
*(pCursor + 1) = ‘\0’;
}

//
// Config file is located with DLL with extension INI
//
strcat( g_szIniPathName, “ini” );

if ( !GetPrivateProfileString( “config”,
“ClientHostName”,
“X-Client-IP:”,
g_szHeaderName,
MAX_PATH,
g_szIniPathName ) )
{
SetLastError( ERROR_INVALID_PARAMETER );
return FALSE;
}

pVer->dwFilterVersion = HTTP_FILTER_REVISION;
lstrcpyn( pVer->lpszFilterDesc,
“ISAPI Filter to twiddle log entry fields based ”
“on parameterized sources (like request header)”,
SF_MAX_FILTER_DESC_LEN );
//
// Technically, I could retrieve the request header from
// SF_NOTIFY_LOG using GetServerVariable, but that loses
// flexibility because GetServerVariable cannot retrieve
// headers that include “_” (underscore) without using
// IIS6 specific syntax of HEADER_(header-name-as-is).
//
// For maximum compatibility (this filter will work from
// IIS4 on up), I am retrieving the request header from
// SF_NOTIFY_PREPROC_HEADERS using GetHeader (which can
// retrieve headers as-is) and changing the log entry
// in SF_NOTIFY_LOG.
//
// This allows me to illustrate common usage case for
// pFilterContext and pfc->AllocMem(), especially when
// modifying log fields (the memory must stay valid!!!)
//
pVer->dwFlags =
SF_NOTIFY_ORDER_HIGH |
SF_NOTIFY_PREPROC_HEADERS |
SF_NOTIFY_LOG
;

return TRUE;
}

DWORD
WINAPI
HttpFilterProc(
IN HTTP_FILTER_CONTEXT * pfc,
DWORD dwNotificationType,
LPVOID pvNotification
)
/*++

Purpose:

Required filter notification entry point. This function is called
whenever one of the events (as registered in GetFilterVersion) occurs.

Arguments:

pfc – A pointer to the filter context for this notification
NotificationType – The type of notification
pvNotification – A pointer to the notification data

Returns:

One of the following valid filter return codes:
– SF_STATUS_REQ_FINISHED
– SF_STATUS_REQ_FINISHED_KEEP_CONN
– SF_STATUS_REQ_NEXT_NOTIFICATION
– SF_STATUS_REQ_HANDLED_NOTIFICATION
– SF_STATUS_REQ_ERROR
– SF_STATUS_REQ_READ_NEXT

–*/
{
switch ( dwNotificationType )
{
case SF_NOTIFY_PREPROC_HEADERS:
return OnPreprocHeaders(
pfc,
(HTTP_FILTER_PREPROC_HEADERS *) pvNotification );
case SF_NOTIFY_LOG:
return OnLog(
pfc,
(HTTP_FILTER_LOG *) pvNotification );
}

return SF_STATUS_REQ_NEXT_NOTIFICATION;
}

DWORD
OnPreprocHeaders(
HTTP_FILTER_CONTEXT * pfc,
HTTP_FILTER_PREPROC_HEADERS * pPPH
)
{
DWORD dwRet = SF_STATUS_REQ_NEXT_NOTIFICATION;
BOOL fRet = FALSE;
CHAR * pszBuf = NULL;
DWORD cbBuf = 0;

SetLastError( NO_ERROR );

if ( pfc == NULL ||
pPPH == NULL )
{
SetLastError( ERROR_INVALID_PARAMETER );
goto Finished;
}

//
// Make sure to reset the filter context to NULL for every
// request so that if we failed to find a named header, we
// do nothing in SF_NOTIFY_LOG
//
pfc->pFilterContext = NULL;

fRet = pPPH->GetHeader( pfc, g_szHeaderName, pszBuf, &cbBuf );
if ( fRet == FALSE )
{
if ( GetLastError() == ERROR_INVALID_INDEX )
{
//
// The header was not found
// Ignore and do nothing
//
OutputDebugString( “Named header is not found. Do nothing.\n” );
SetLastError( NO_ERROR );
goto Finished;
}
else if ( GetLastError() == ERROR_INSUFFICIENT_BUFFER &&
cbBuf < MAX_CLIENT_IP_SIZE )
{
//
// The header was found and fit size requirements.
//
// Let’s allocate from IIS’s ACache memory which
// is guaranteed to live as long as the connection,
// so it is perfect for log entry modification.
//
pszBuf = (CHAR *)pfc->AllocMem( pfc, cbBuf, NULL );

if ( pszBuf == NULL )
{
SetLastError( ERROR_NOT_ENOUGH_MEMORY );
goto Finished;
}

fRet = pPPH->GetHeader( pfc, g_szHeaderName, pszBuf, &cbBuf );
if ( fRet == FALSE )
{
goto Finished;
}

OutputDebugString( “Named header value is: ” );
OutputDebugString( pszBuf );
OutputDebugString( “\n” );
}
else
{
goto Finished;
}
}

//
// At this point, pszBuf points to the value of named header
// Just save it and move on.
//
pfc->pFilterContext = pszBuf;
SetLastError( NO_ERROR );

Finished:

if ( GetLastError() != NO_ERROR )
{
OutputDebugString( “Error!\n” );
dwRet = SF_STATUS_REQ_ERROR;
}

return dwRet;
}

DWORD
OnLog(
IN HTTP_FILTER_CONTEXT * pfc,
IN HTTP_FILTER_LOG * pLog
)
{
//
// If the named header was found, set the pszClientHostName
// log field with its value
//
if ( pfc->pFilterContext != NULL )
{
pLog->pszClientHostName = (CHAR *)pfc->pFilterContext;
}

return SF_STATUS_REQ_NEXT_NOTIFICATION;
}

Comments (72)

  1. Timo Sundström says:

    Hey Dadid,

    I have a question about how to compile the project. It’s been a while since I’ve worked with VisualStudio and C.

    Should I create a Dynamic Link Library or use the ISAPI Extension wizard in Visual Studio 6?

    I tried to compile it as a dll and it seems to load properly (No error messages), but it doesn’t write anything to the log.

    Thanks in advance,

    TImo

  2. David Wang says:

    Timo – Ok, I’ve already written the code, so you really need to take it as a stretch goal and figure this part out. I assure you it works easily and builds with Visual Studio 6.0 since I wrote it that way. ๐Ÿ˜‰

    See the following for links to how to debug general ISAPI Filter installation issues, especially if you are using IIS6:

    http://blogs.msdn.com/david.wang/archive/category/3969.aspx

    //David

  3. Timo Sundström says:

    OK, I managed to compile the code and figured out how to install and debug it.

    It didn’t seem to work.

    I found a bug (atleast I think it was a bug) in the code, but I got it working anyway.

    So, thanks for the code. A great blogg

  4. David Wang says:

    Timo – hey, you can’t just "claim" there is a bug and not show any evidence… that’s not very nice to me nor future readers. I mean, your compilation/linking skills are far more suspect… ๐Ÿ˜‰

    In any case:

    No proof = no bug

    When I wrote and tested it, it worked perfectly as advertised. :-)

    //David

  5. Lee says:

    Thanks for the great code.

    Boss said "See if you can get this to work". I am the only developer in the group with VS C++ 6.0 experience (over 5 years ago since I last used it) so I am the lucky guy.

    Couple of very fundemental questions about this. What sort of project do I start with? Started off with a ISAPI Extension project and only selected Filter. Looked at the code, it looks like you have already written most of the IIS hooks. Should I just copy over all that code or do I just create a blank DLL project copy/paste & compile? Point IIS to the new dll and viola a filter?

    Or am I missing something fundemental about this that I should be embarassed about?

    thanks for the help,

  6. David Wang says:

    Ok, I went and wrote up a blog entry detailing step-by-step on how to take source code from my blog and turn it into a working ISAPI:

    http://blogs.msdn.com/david.wang/archive/2005/12/19/HOWTO_Compile_and_Use_my_ISAPI_Code_Samples.aspx

    //David

  7. Alex says:

    David,

    Great job!

    I was able to get the ISAPI filter to work and it records source IP infromation to the IIS log, however is it possible also to modify REMOTE_ADDR server variable, so source IP values can be used in the ASP or .Net applications.

    If I run Request.ServerVariables("REMOTE_ADDR") it still returns load balancer IP address.

    Is it ISAPI priority issue? Currently your ISAPI filter is listed on the very bottom in the ISAPI filter list with the priority *Unknown*

    Thanks!

  8. David Wang says:

    Alex – Server Variables are considered Read-only artifacts of request execution and cannot be modified.

    You can pass that value using the request header and have ASP/ASP.Net retrieve it as HTTP_ ServerVariables. I’ll add that as another source-code example.

    ISAPI Filter priority does not affect this… though you do remind me that I should set a filter priority, so I made the one-line fix in the source code.

    //David

  9. Timo Sundström says:

    Hey,

    First of all I’d like to thank you for your blog about "ISAPI Filter which Logs original Client IP." I learned a lot about ISAPI Filters and IIS.

    I’d also like to apologize for my comment about finding an aledged bug.

    But, I can’t seem to get the filter to work. It seems to load proparly but it doesn’t appear to write anything in the log. When I debug it, the GetHeader seems to return "ERROR_INVALID_INDEX." The header I’m trying to retrieve is "HTTP_IPREMOTEADDR:" I’ve also tried to retrieve "HEADER_HTTP_IPREMOTEADDR:" but without any luck.

    I tried to code it using MFC and creating an ISAPI Extension Wizard. Then it seemed to work fine on a Windows 2000 Server with IIS5 but not on a Windows 2003 Server with IIS6.

    I’ve also tried to modify your code so that OnPreProdHeaders anly returns "SF_STATUS_REQ_NEXT_NOTIFICATION" and use GetServerVariable in "OnLog-function." Even that seems to work on Windows 2000 but again not on Windows 2003.

    Any ideas what the problem could be?

    Perhaps you could email me your version of the .DLL, so I could try that one? I’m using MS Visual Studio 6 on a Win2k machine.

    Thank you in advance

  10. David Wang says:

    Timo – Actually, your problems are due to improper syntax.

    I presume the request header you want to retrieve is:

    IPRemoteAddr: 1.2.3.4rn

    Now, before you make any code changes, please realize that my filter already supports logging from a custom request header. As I mentioned in this blog entry, create LoadBalancedIP.ini in the same directory as the DLL (if you customized the DLL name, you *must* change the #define MODULE_NAME or else the INI file won’t work) with the following contents and after recycling the Application Pool, it should start logging IPRemoteAddr’s header value as ClientIP:

    [config]

    ClientHostName=IPRemoteAddr:

    When you use GetHeader(), the syntax to use is the exact header name with trailing colon.

    i.e. GetHeader( "IPRemoteAddr:" ).

    "HTTP_IPREMOTEADDR:" and "HEADER_HTTP_IPREMOTEADDR:" are incorrect and you have verified that. You cannot simply mix syntax between GetHeader() and GetServerVariable() and hope that it works…

    GetHeader() Documentation:

    http://msdn.microsoft.com/library/default.asp?url=/library/en-us/iissdk/html/c54bd6db-16cb-4e8e-b140-e4f937bf97a9.asp

    When you use GetServerVariable(), the syntax depends on the prefix you prepend to the header name.

    – If you use HTTP_ (i.e. GetServerVariable( "HTTP_IPREMOTEADDR" ), you follow the CGI specification to retrieve header values — you use the exact header name except "-" are substituted with "_" (i.e. If the real header name is My-Header:, you use "HTTP_MY_HEADER"). The flaw in the CGI specification is that you cannot retrieve real header names that include "_".

    – If you use HEADER_ (i.e. GetServerVariable( "HEADER_IPREMOTEADDR" ), you can retrieve any headername, including those with "-" and "_" (i.e. "HEADER_MY_HEADER" retrieves My_Header: and "HEADER_MY-HEADER" retrieves My-Header:). This syntax was introduced in IIS6. See documentation for explanation.

    GetServerVariable() Documentation:

    http://msdn.microsoft.com/library/default.asp?url=/library/en-us/iissdk/html/0f8a163e-ef8b-40de-a4a7-219207614c42.asp

    The reason that your GetServerVariable() notation worked on IIS5 for "HTTP_IPREMOTEADDR:" is because IIS5 variable name parsing is looser than IIS6. You will succeed on both IIS5 and IIS6 if you pass in syntactically correct names. If you pass in syntactically incorrect names, IIS5 *may* accept but IIS6 *will* reject it.

    FYI: I do not privately email and certainly cannot email a binary DLL.

    //David

  11. Timo says:

    Thank you!

    I feel a bit stupid for not realizing that you dont need the HTTP_ when using GetHeader().

    It works fine!!!

  12. Alex says:

    Hi,

    Thanks for the code, I’m new to ISAPI filters and it’s great to learn. I have one trouble though and I thought someone could help me out.

    In OnPreProcHeaders, after the call of GetHeader, the return is false and I fall in the "else" part of the if. I debugged the values and GetLastError == 0 (NO ERROR). Also, the cbBuf variable is equal to 0 after the call. However, I managed to make it work if I call GetHeader with a pre-sized CHAR* and a buffer size equal to that of my char array. Any idea why it’s not working with your way? Thank you very much.

  13. David Wang says:

    Alex – What IIS version and can you give me the exact RAW HTTP request you used.

    I had no problems running this code on IIS5 and IIS6, so I am interested in the IIS version as well as the request that triggers the condition. The behavior you described, if reproduced, would be a bug in IIS.

    //David

  14. Alex says:

    IIS 5.0, I got the following from IEWatch

    10:02:37.136 188 ms 11445 GET 200 text/html; charset=utf-8 http://localhost/Test/page1.aspx

    I activated the trace on-page in my web app to confirm that the header I’m reading exists and contains the good value: it does.

    Is this the info you were asking for?

    Thanks,

  15. Alex says:

    By the way, I compiled it in Visual Studio .Net 2003 using the configuration you specified for visual studio 2005 express in your other blog post.

  16. David.Wang says:

    Alex – I need the exact request you made which results in GetHeader() returning FALSE and GetLastError() returning NO_ERROR. It should look something like:

    GET /Test/page1.aspx HTTP/1.1rn

    Host: localhostrn

    X-ClientIP: 1.2.3.4rn

    Accept: */*rn

    rn

    Now, you said that you observed GetHeader() returning FALSE but GetLastError() returning NO_ERROR on IIS5 – which would be considered a bug in IIS. There is nothing wrong with the filter code, so your proposed change is a work-around for your specific issue.

    To verify, I would need:

    1. the exact request

    2. what version of IIS5 you are using

    3. What LoadBalancedIP.ini you are using

    … because I do not see what you claim on my W2KSP4 IIS5 machine.

    I need this information because without it reproduced independently of your machine, the issue basically does not exist and cannot be addressed.

    //David

  17. Alex says:

    I totally agree that so far the problem looks to be on my side and I’m far from claiming there is a bug in either IIS or your code :)

    I am on w2k advanced server, IIS 5, my LoadBalancedIP.ini contains:

    [config]

    ClientHostName=X-Forwarded-For:

    This HTTP header is forwarded by the GIsapi filter (http://www.s0nic.hostinguk.com/wiki/ow.asp?GIsapiFilter) we installed on our ISA server. But since I’m not a hardened c++ developer (I usually work in c# or vb.net in the .net framework), I do not intend to take all your time to fix my problems :)

    I’m looking for a tool to get the raw http request.

  18. Alex says:

    Here is the http request when I hit my test page

    GET /TestISA/page1.aspx HTTP/1.1

    Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-powerpoint, application/vnd.ms-excel, application/msword, application/x-shockwave-flash, */*

    Accept-Language: fr-ca,en-us;q=0.5

    Accept-Encoding: gzip, deflate

    User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322; .NET CLR 1.0.3705)

    Host: eric

    Proxy-Connection: Keep-Alive

    Obviously it doesn’t have the header since it is appended to the request headers by the ISA server.

  19. David.Wang says:

    Alex – hehe, no worries… I have used IIS5 long enough to know that the bug could be in there *somewhere*; just need a special request to get it. For example, request headers that do not have terminating rn (hex 0D 0A) have caused problems in the past.

    Thanks for telling me that there is a proxy server between the client and IIS… because that means that we need the modified request by GIsapi. Easiest way to get this is to use a Network Monitoring tool like Microsoft Network Monitor or Ethereal to capture incoming traffic right before IIS server gets it. And it’s important to post the header termination characters (that was why I put in rn explicitly) because they can affect behavior.

    //David

  20. Alex says:

    Finally got it, here is the raw GET request

    00000030 47 45 54 20 2F 54 65 73 74 49 GET./TestI

    00000040 53 41 2F 70 61 67 65 31 2E 61 73 70 78 20 48 54 SA/page1.aspx.HT

    00000050 54 50 2F 31 2E 30 0D 0A 56 69 61 3A 20 31 2E 31 TP/1.0..Via:.1.1

    00000060 20 4C 44 37 30 33 35 34 35 0D 0A 48 6F 73 74 3A .LD703545..Host:

    00000070 20 65 72 69 63 0D 0A 55 73 65 72 2D 41 67 65 6E .eric..User-Agen

    00000080 74 3A 20 4D 6F 7A 69 6C 6C 61 2F 34 2E 30 20 28 t:.Mozilla/4.0.(

    00000090 63 6F 6D 70 61 74 69 62 6C 65 3B 20 4D 53 49 45 compatible;.MSIE

    000000A0 20 36 2E 30 3B 20 57 69 6E 64 6F 77 73 20 4E 54 .6.0;.Windows.NT

    000000B0 20 35 2E 30 3B 20 2E 4E 45 54 20 43 4C 52 20 31 .5.0;..NET.CLR.1

    000000C0 2E 31 2E 34 33 32 32 3B 20 2E 4E 45 54 20 43 4C .1.4322;..NET.CL

    000000D0 52 20 31 2E 30 2E 33 37 30 35 29 0D 0A 43 6F 6F R.1.0.3705)..Coo

    000000E0 6B 69 65 3A 20 41 53 50 2E 4E 45 54 5F 53 65 73 kie:.ASP.NET_Ses

    000000F0 73 69 6F 6E 49 64 3D 6B 68 72 6D 70 30 6D 61 67 sionId=khrmp0mag

    00000100 6D 6E 72 68 6D 35 35 6B 70 67 68 67 6E 35 35 0D mnrhm55kpghgn55.

    00000110 0A 41 63 63 65 70 74 3A 20 2A 2F 2A 0D 0A 41 63 .Accept:.*/*..Ac

    00000120 63 65 70 74 2D 4C 61 6E 67 75 61 67 65 3A 20 66 cept-Language:.f

    00000130 72 2D 63 61 2C 65 6E 2D 75 73 3B 71 3D 30 2E 35 r-ca,en-us;q=0.5

    00000140 0D 0A 41 63 63 65 70 74 2D 45 6E 63 6F 64 69 6E ..Accept-Encodin

    00000150 67 3A 20 67 7A 69 70 2C 20 64 65 66 6C 61 74 65 g:.gzip,.deflate

    00000160 0D 0A 58 2D 46 4F 52 57 41 52 44 45 44 2D 46 4F ..X-FORWARDED-FO

    00000170 52 3A 20 31 37 32 2E 32 35 2E 35 32 2E 31 36 35 R:.172.25.52.165

    00000180 0D 0A 58 2D 43 45 52 54 43 4F 4F 4B 49 45 3A 20 ..X-CERTCOOKIE:.

    00000190 0D 0A 58 2D 43 45 52 54 53 45 52 49 41 4C 4E 55 ..X-CERTSERIALNU

    000001A0 4D 42 45 52 3A 20 0D 0A 58 2D 43 45 52 54 53 55 MBER:…X-CERTSU

    000001B0 42 4A 45 43 54 3A 20 0D 0A 58 2D 43 45 52 54 49 BJECT:…X-CERTI

    000001C0 53 53 55 45 52 3A 20 0D 0A 58 2D 49 53 44 45 42 SSUER:…X-ISDEB

    000001D0 55 47 3A 20 46 41 4C 53 45 0D 0A 43 6F 6E 6E 65 UG:.FALSE..Conne

    000001E0 63 74 69 6F 6E 3A 20 4B 65 65 70 2D 41 6C 69 76 ction:.Keep-Aliv

    000001F0 65 0D 0A 0D 0A e….

    I hope it displays ok :)

  21. Alex says:

    Here’s a cleaner version :)

    GET /TestISA/page1.aspx HTTP/1.0

    Via: 1.1 LD703545

    Host: eric

    User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322; .NET CLR 1.0.3705)

    Cookie: ASP.NET_SessionId=khrmp0magmnrhm55kpghgn55

    Accept: */*

    Accept-Language: fr-ca,en-us;q=0.5

    Accept-Encoding: gzip, deflate

    X-FORWARDED-FOR: 172.25.52.165

    X-CERTCOOKIE:

    X-CERTSERIALNUMBER:

    X-CERTSUBJECT:

    X-CERTISSUER:

    X-ISDEBUG: FALSE

    Connection: Keep-Alive

  22. David.Wang says:

    Alex – Thanks for the details. I don’t immediately have time but I will be looking at (and around) the repro request that you provided along with the ISAPI Filter and INI file configuration and figure out what is going on.

    //David

  23. Paul says:

    Hi,

    Thanks for the code. I’m new with the IIS application and it seems that when I tried to used your code, I still see the load balancer IP in the IIS logs.

    Currently I’m using win 2k with IIS version 5. I like to ask if this will work as well in a Cisco loadbalancer? Also, is there a temp file created where the IIS reads and filters it to the IIS logs. We are just curious how the IIS logs were been created.

    Please advise.

  24. David.Wang says:

    Paul – Honestly, I have no idea whether it works for you. It all depends on how your Cisco loadbalancer works.

    The ISAPI Filter illustrates how to modify the Client-Host log field using the value from a customizable request header name. It should be good enough to solve this problem.

    I intentionally did not study any popular load balancers to pre-bake a series of request headers such that the ISAPI Filter "automatically" works. Why? Because doing that simply trivializes me into merely providing product support with no relevance to ISAPI when the greater value is for me to illustrate how to provide a solution with ISAPI.

    In other words, if I provided such an ISAPI solution, then I would be compelled to provide support to your question of "I tried to use your code but it does not work". Clearly, I cannot do that since there is only one of me.  It is far better for me to provide information to help you figure out your own answers… so this is why I provided an ISAPI that can be configured to solve the issue… but you are responsible for figuring out and supporting that ISAPI configuration.

    You need to determine whether your Cisco loadbalancer even forwards the original client IP and if so, using what request header name… and then configure the ISAPI Filter accordingly.

    //David

  25. Mike Williams says:

    The issue is not if the code will work (in??) with a Cisco loadbalancer but if the loadbalancer can insert the original client’s IP into the headers. Then you ned to know the name of the header it creates/uses and then test. We have a Cisco Loadbalancer which is certainly capable (dependent upon firmware etc). See the Cisco documentation for your particular model and firmware.  I wonder if you have the right header name as you don’t know if it will work with your hardware.

  26. Paul says:

    Hi,

    Thanks for the info. I will try to check the documentation of our load balancer maybe we have not setup it correctly. I have no question or problem with the code. It’s a great help for us to understand ISAPI.

    Honestly, I don’t have any idea the details of our load balancer and I don’t even have access to check it.

    Thanks.  

  27. David.Wang says:

    Paul – ah… well, I’ve never worked with Cisco before but I imagine there is a documented admin port available internally that hopefully advertises the hardware’s version/etc.

    You can also use a network sniffer to pragmatically view the requests being forwarded from the load balancer to see if any particular request header looks like an added Client-IP – and if it is, then configure the ISAPI Filter accordingly.

    If you don’t see a request header… then time to check the Cisco side of things to see if your version supports it, a firmware upgrade available, etc.

    //David

  28. Richard says:

    Gentelmen,

    Netscaler Already has an ISAPI for this pupose. Please read the documentation.

    ~Richard

  29. Alex says:

    Hi David,

    Any new development regarding my issue?

    Thanks,

    Alex

  30. David.Wang says:

    Alex – hmm… I took your exact request, compiled this blog entry, and used the same INI file contents… and unfortunately, I am not seeing the issue reproduce for me.

    I looked through the hex dump, toggled compression on/off, started adding/removing spaces and other characters (I see the 0D 0A for headers in what you gave, so I don’t play with that), removed X-Forwarded-For:, … but I could not get similar behavior. I just can’t think of a way to have GetHeader() return FALSE and GetLastError() = NO_ERROR – which is what causes the filter to abort any actions and not modify the log field.

    //David

  31. Alex says:

    Hi David,

    Could I be compiling against older ISAPI headers in which the GetHeader function behaves differently? Anything I should check to make sure I’m using the right stuff?

    Thanks,

    Alex

  32. David.Wang says:

    Alex – "older" ISAPI header issues would be found at compile time, not runtime.

    There is no LIB file for ISAPI to link against, so difference in behavior comes from different runtime configuration, including IIS version, request, and IIS configuration.

    Since I’ve tried similar IIS version and request without reproducing it… could you have different list of ISAPI Filters configured – what filters at the global and website level do you have configured and in what order?

    //David

  33. Alex says:

    Hi David,

    I started a new MFC ISAPI Filter in visual studio 2003, got down to code approximatively the same logic as your filter (two calls to GetHeader to call with the exact buffer size, the first call being with a NULL pointer and a zeroed buffer size) and it works perfectly.

    I just thought I’d let you know. Thank you for the support though, it’s been appreciated :)

    Alex

  34. David Wang says:

    Question:

    I’m trying to write a Filter that handles writing a W3C-compliant log file based on a special…

  35. talha says:

    hey david..

    i m new to isapi and i found this blog quite helpful..

    i m trying to make an isapi filter which logs the ip addresses and the urlzz accessed and other details of the users which access my website having extension .CBS..

    can u help me in dat..

    i m in real need of this coz my project is on hold..

    looking fwd to an early response..

    Talha

  36. David.Wang says:

    talha – if you have a specific question about ISAPI/IIS then I would try to answer it.

    Based on your description, I believe that everything you need has already been described by my various blog entries as well as related MSDN documentation, all linked from my blog.

    I suggest you do the necessary homework to figure this out.

    //David

  37. talha says:

    david i have made a filter which redirects all the request to pages having extension .cbs..now i m having probss in the ONLOG functionzz usage.. :-(

    it wud be nice if u cud kindly put the link of the blog entries u mentioned…

  38. David.Wang says:

    talha – if you have a specific question about ISAPI/IIS or a specific design question, then I can try to answer it.

    However, since I have little real information on what you trying to do or having problems with, I have no idea what links you need.

    I suggest that you search my blog or MSDN for relevant samples and documentation. It’s all written down and publicly available..

    //David

  39. fastian says:

    i am trying to access a database through ISAPI filter.

    i am using the cdatabase thingie and i am not able to conect to teh database and add a row in the database.

    hoping to get a reply

  40. Chris Benton says:

    I got sent to this link thanks to google and fastian’s comment. I’m trying to find an IIS5 ISAPI-filter code sample to connect to a SQL server database… Can anyone help?

  41. David.Wang says:

    fastian, Chris Benton – your questions about how to connect and add a row in a database is best asked a database data-access API related community (such as ADO, ODBC, etc).

    Once you get that part worked out, you should be able to insert that code directly into an ISAPI and run it on IIS.

    The user account your ISAPI Extension/Filter runs as can be determined by reading this blog entry:

    http://blogs.msdn.com/david.wang/archive/2005/06/29/IIS_User_Identity_to_Run_Code_Part_2.aspx

    //David

  42. Mike says:

    David, thanks for this example. One thing I’ve been noticing (and correct me if I’m wrong here…) is that an ISAPI filter that hooks into the SF_NOTIFY_LOG event notification seems to make IIS 6 believe the filter is not "cache friendly" and therefore prevents items from being inserted into the http.sys cache. FilterEnableCache is definitely enabled in the metabase for the filter, but whenever I have it installed I get no "Kernel: URI Cache Hits" increments.

    So, I guess my question is what can I do to stick an X-Forwarded-For header in as the Client host without losing the http.sys cache? I’m hoping that we dont have to try to manually insert the items into the cache… Any suggestions? Thanks.

  43. David.Wang says:

    Mike – Yes, SF_NOTIFY_LOG event is considered "cache unfriendly" and turns off the kernel response cache.

    Here is the dilemma:

    1. HTTP.SYS writes the log file for a given parsed request

    2. ISAPI Filter listening for SF_NOTIFY_LOG forces HTTP.SYS to ask IIS in user mode for any changes to the log entry fields (like Client host)

    3. which means it is possible to not record the correct Client host in the cache-hit case – HTTP.SYS cannot run custom code in KERNEL MODE to determine the correct Client-host for such a forwarded request.

    Thus, you cannot keep the kernel response cache and modify the log entry.

    //David

  44. Visitor says:

    Hi all,

    I have a simular problem, the loadbalancer used is a F5 Big IP. Not one of the smallest loadbalancer manufacturers…

    They have a isapi aswell (http://devcentral.f5.com/weblogs/joe/archive/2005/09/23/1492.aspx) , however not officialy. Working great however it only works on HTTP sessions. SSL sessions do not give the HTTP_X_FORWARDED_FOR. (called in this case).

    Strangely, because i thought the ISAPI filters are applied after the request comes from the LSASS.exe, which does ‘remove’ the ssl encryption from the data packet.

    So since the Isapi does not see the packet as an SSL packet it should be fine. But i cant be for sure that Lsass.exe does not passes such information to the workerprocess /isapi.

    Anyone came accros the same issue? Kinda weird, when i think of SSL sites that should be more secure because of there content. However it’s not possible to log the client ipadresses which you realy want to have incase of a security problem / unauthorized access.

  45. David.Wang says:

    Visitor – Make sure that F5 Big IP actually forwards the original client IP on SSL requests, and if so, with what request header name.

    My ISAPI Filter allows configurable header name so it is easy to read "X-Forwarded-For" – just edit a INI config file as specified in the blog entry.

    Regarding your conjectures about how LSASS, ISAPI Filters, and SSL interact – I’ll just say up front that it all works logically and correctly; the verification is an exercise best left to the reader.

    Now, ISAPI Filter has to be configured to listen and act on SSL traffic via the SF_NOTIFY_SECURE_PORT flag. I did not set it in the sample because the point of the sample was to show how to log original client IP. I presume that interested users can come up with the necessary code change for their situation – you have the source code.

    //David

  46. Nick says:

    Visitor –

    SSL encrypts the whole HTTP request, including headers.

    Unless the Big IP itself is doing the encryption (and it can, with the proper add-on module), it will not be able to add a header to the encrypted request.  Moving SSL to the Big IP should solve the problem.

    I currently do just that to save on certificate licensing costs and it works wonderfully, X-Forwarded-For headers and all.

    -Nick

  47. Mike Ayling says:

    For F5 users who want to snag the original client ip, without using an ISAPI or sacrificing IIS kernel mode cache you might try this. Its a bit of an ugly hack, but should work just fine. Create an iRule that appends some kind of delimiter and the value of $clientip to [HTTP::header User-Agent]. Then use logparser to clean it all up.

    Havent tested this, but if you dont mind doing some log parsing then it could be a reasonable solution.

  48. Jozsef says:

    Hello,

    I have IIS and ISA installed. On ISA I have turned out the logging for web pulishing and for the filters which are related with it. For web publishing I have also selected "request appear to come from the original client" option under ISA. I have php installed on this system and when I open our web based, ssl e-mail client page, my ip appears in the log files of IIS. As soon as I open any other php or html pages on our server the logfiles contains only the ip address of the server for client ip. Unfortunately, it all happens with or without this filter. Obviously, ISA logs the correct ip addresses of the clients, but for web usage statistics I need to use IIS.

    Could you advise me something more to check?

    Thanks,

    Jozsef

  49. tophe says:

    hi all,

    I am trying to make all that work on an iis6 english on win nt 2003.

    but i can’t have the filter executed, I have alway a status of Unknow.

    I have alowed the dll in the web service extensions, and made some request on the .aspx pages but nothing happens, i cant start the filter.

    i am try to use haproxy and so to track HTTP_X_FORWARDED_FOR: .

    I have compiled the dll whith "X_FORWARDED_FOR:"

    I am doing dynamic compression to.

    any help ?

  50. Steven Hope says:

    If you are looking to use X-Forwarded-For on ISA Server take a look at http://www.winfrasoft.com/X-Forwarded-For.htm

  51. mark m says:

    Why doesn’t IIS support this directly?

    To what extent has this been tested or completely transparent to applications that might need to leverage it?

    As a hosting provider, we might have little predictability as to what applications customers might be installing. I’m trying to gauge what extent we’d have to validate each implementation and whether Microsoft would officially support us if we called in an IIS or other product case.

    Thanks.

  52. JM says:

    Thanks for great information.  We are network imbeciles.

    We are using haproxy.  

    We read the code where you can have the ClientHostName in the INI or in the code where it attempts to read the ini in GetPrivateProfileString.

    We modified the X-Client-IP: to X-Forwarded-For:

    We copied the code you have listed, and compiled it.  

    We gave permissions to everyone on the server.

    When attempting to hit the web site from an outside the network web connection the page fails to load.

    We receive an error in application event viewer states:

    Failed to load dll. The data is the error.

    Any ideas?

  53. David.Wang says:

    JM – Please read and follow instructions on how to compile and use ISAPI sample code as well as how to install and troubleshoot ISAPI Filters.

    http://blogs.msdn.com/david.wang/archive/2005/12/19/HOWTO-Compile-and-Use-my-ISAPI-Code-Samples.aspx

    http://blogs.msdn.com/david.wang/archive/2005/06/21/HOWTO_Diagnose_and_Fix_Common_ISAPI_Filter_Installation_Failures.aspx

    //David

  54. David.Wang says:

    mark m – Unfortunately, there is no standard mechanism to convey this information, so it is not possible to produce an IIS feature.

    Furthermore, this problem is really caused by the Load balancer, and I consider IIS good enough that it can be extended to resolve another product’s issue. But to have a "feature" just to deal with another product’s bug — quite unreasonable.

    //David

  55. James Richter says:

    Does this solution work with 2008/IIS7?

  56. David.Wang says:

    James – this should work with IIS7 which has the ISAPI Filter Feature support installed.

    //David

  57. ken says:

    If we want to still whitelist sites in IP Address and Domain Name Restrictions on IIS, does the client request hit that whitelist first?  or does it hit the ISAPI filter?

    and if the ISAPI filter first, will IIS IP/DN restriction see the original client IP correctly via the ISAPI filter?

    thx!

  58. Kyle says:

    First, thanks for the valuable resource.

    Any suggestions why adapting this for SM_USER, or HEADER_SM_USER, or HTTP_SM_USER would not be working, given it’s compiled and that is loaded into IIS 6 fine?

    -Kyle

  59. Kyle says:

    I see the problem,  in our environment there is no SM_USER, at least available to WebScarab. Looks like I need to find out if isapi can get at encrypted cookie values…

  60. David.Wang says:

    Kyle – ISAPI can get at any value on the HTTP request. You will have to unwind any data transformations (like encryption) and formats to obtain the data you want.

    //David

  61. David.Wang says:

    ken – ISAPI can only change the IP in the log file. It is not possible to change REMOTE_ADDR for the request.

    Thus, you will not be able to use IIS’s IP Restriction behind a load balancer since the load balancer re-issues the request and loses the original client’s IP in the TCP packet header.

    //David

  62. Apokrif says:

    David,

    I read whole post and donโ€™t understand the idea.

    If Load Balanced configured properly โ€“ itโ€™ll pass real client IP, not SNATed one.

    There are situations when you need SNAT client IP to work around routing issues, but they are quite rare.

    I admit, the code is useful to log clients behind with proxy servers (but sure if it legal to do ๐Ÿ˜‰

    And not all proxy servers insert client tracking headers ether.

    The real problem is how NOT to log health check requests from load balancer itself?

    I.e. we can detect request coming from LB easily, but what to tell IIS to prevent log from littering?

  63. Someshar Janjirala says:

    Does this solution works for SSL on web front ends. I could able to compile the code and it is working fine in QA without SSL but when I install in PROD it is not working because of SSL. Did anyone tested this code using certificate on WFE, please let me know.

  64. David.Wang says:

    Someshar – the ISAPI filter works with SSL.

    When a load balancer acts as endpoint termination for a client’s HTTP and SSL connections, it needs to transmit that IP address on the load-balanced request because without it, NO downstream web server will be able to determine nor log that IP address. All traffic will look like it came from the load balancer.

    Right now, it sounds like the problem is with WFE terminating SSL connections from the client but failing to pass onward that original client IP address in an HTTP header. Since this behavior is not specified in any RFC specification for HTTP, you have no expectation to get that original client IP address.

    You have to read documentation or contact support personel for WFE to find if it retransmits the original client IP address on the load balanced request and if so, what is the header name. You can then configure this ISAPI Filter to use it, and things will work. But, if WFE does not pass that client IP address onward, it will be impossible to log the original client IP.

    //David

  65. David.Wang says:

    Apokrif  – IIS supports Request Logging configuration down to the URL scope. It is possible to configure IIS to not log requests for the specific URLs used as health check by the load balancer.

    However, it is not possible to dynamically decide whether a request should be logged or not. If the URL is configured to be logged, it will always be logged; and vice versa.

    When it comes to log files, any analysis program should have sufficient aggregation and filtering features to make sense of a raw log file. If you expect to open up a raw log file and see exactly what you want, then that expectation is flawed. A web server can receive requests from any client for any URL with any response status. What you should do is use a tool like LogParser.exe to aggregate and filter your IIS log file so that you see what you want.

    //David

  66. Joe says:

    Will this work on IIS running in 64 Bit on 2003 x64?

    Or is there a different version required?

  67. David.Wang says:

    Joe – the filter code will work on any IIS version and any bitness. You will need to ensure that you compile the source code with a x64 target, which is possible using the Windows 2003 SDK (it comes with 64bit compilers, linkers, LIB, and header files) as well as Visual C++ Express, though I do not have the step-by-step. It should not be hard to figure it out.

    //David

  68. aihoroz says:

    Hello David, thanks for this post, is what i really need, i followed all the steps exactly as you mentioned also i followed the tutorial to compile your isapi filters examples, i uploaded the LoadBalance.dll  to my win2008 server IIS7, i added to my site, and then i checked the logs, i am still getting the LoadBalancer IP address, do you think is not working because the different server and IIS versions? what could i did wrong? how can i debug it?

    also i compiled the dll on my personal computer then i uploaded to the web server. do you think this could affect it?

    Thanks in advance

    Have a nice day!

  69. Kalpesh says:

    I have done all the necesaory things and its loaded successfully but there is no log in IIS log file "C:WINDOWSsystem32Logfiles" after requesting the aspx page. (IIS version 5.1).

    Please help…

  70. Kshitiz says:

    I have an isapi filter which registers with SF_NOTIFY_LOG. Before the filter is registered, the blank space URL gets encoded as ‘+’. After the filter is registered the space gets encoded as ‘%20’. Any idea why this would chage? I can repro this by just registering the filter and returning SF_STATUS_REQ_NEXT_NOTIFICATION. (IIS 6,W3C extended log)

  71. Mike says:

    This works for ISA server working as web publishing?

    Thanks.

  72. Manuel Andrade says:

    David,

    Where you state:

    "AllocMem() is nice because IIS tracks this memory for you and automatically frees it when its associated connection ends"

    Does this still hold even if the following line does not execute?

    pfc->pFilterContext = pszBuf;

    (this would be the case if the second call to GetHeader returns false – which I do not think is a possible condition…)