Windows Server AppFabric Memory Consumption Behavior

  • Article

The behavior of Windows Server AppFabric 1.1 memory consumption is puzzling to many people. You may observe that the DistributedCacheService.exe consumes a large amount of memory even when the total size of objects stored in the cache is really small, the memory consumption is creeping up over time and can reach twice as much as the configured host cache size.

The behavior is expected if it is AppFabric 1.1. The memory management has changed from 1.0 to 1.1. AppFabric 1.1 manages the memory as its own pools. It adds the released memory to its own free pools but would not give back to GC. That is the reason why the memory usage on the OS level can be high and keeps climbing.

You can validate no memory leak by pushing additional data into the cache when the memory usage of DistributedCacheService.exe is already high or close to the configured host cache size. If AppFabric is able to accommodate, then you know AppFabric is managing it well.

By default, the amount of memory reserved for the cache on a given server is 50% of the total RAM. The host size in MB is available caching memory on the cache host. This only accounts for items in the cache and not the memory used for the caching service itself. You can configure it lower, but configuring the amount higher is not supported

Additional info on AppFabric 1.1 memory management:

  1. When you configure a host size, this is the amount of memory used to store user objects. This memory is paged into physical memory gradually (i.e. it doesn’t do it all at bootstrap for performance reasons). So irrespective of write load on the cache and the amount of data in the cache, the entire host size worth of memory is brought into physical memory eventually. So for host size = 16 GB eventually all of it would be in working set even if you are storing only 100 MB worth of objects in the cache.
  2. For AppFabric Cache 1.1 the overheads are 1:1. Which means with host size = 16 GB the working set of the process may go close to 32 GB. This is the reason we don't recommend configuring the host size greater than half of the RAM.