(This article is focused and valid only for Azure AppFabric Caches and not for Windows Server AppFabric Caches.)
Azure AppFabric Caches come with quota limits. Quota limits are mentioned here.
Here are some of the reasons why you might be throttling on:
1. Bandwidth or Transactions: This means that you are actually in need of a bigger cache since your cache usage is pretty huge. (or need to re-architecture the app so that it uses less bandwidth & does fewer transactions)
2. Connections : This might point to a bug in your code, an erroneous setting in DataCacheConfiguration or may be just because you weren't aware of the connection quotas till now. Lets take the example of 128 MB cache. The allowed connection count is 5. For you to exceed this count, following are the possibilities:
a. DataCacheFactoryConfiguration.MaxConnectionToServer is set to value of more than 5 : In this case, whenever you instantiate a new DataCacheFactory and get a DataCache from it, you are essentially opening more than 5 connections to Azure AppFabric Cache service which would mean that you are exceeding the allowed count. Please note that it is okay to do multiple GetDefaultCache() on a single DataCacheFactory since only one instance of DataCache is created per instance of factory and GetDefaultCache() returns back the same instance each time.
Solution: Try setting the value to 1 and try again.
b. Several instances of DataCaches created from different DataCacheFactories: If in your code you are initializing several DataCacheFactories and do a GetDefaultCache() from each one of them, then each instance of DataCache you get from each factory is going to have it's own connections to the Azure cache service.
Solution: Create only a single instance DataCacheFactory (per cache endpoint) in your code and re-use it as much as possible. If you really need to have several instances of DataCacheFactories, please ensure to Dispose() them off, once they are not needed.
c. Several instances of application running with Windows Azure: It might also be a case that your application has several instances deployed and each one of them is using the same cache. In such a case you are exhausting the connections since each instance is creating connections to Azure cache service.
Solution: Upgrade your cache subscription.
d. Multiple applications / test applications sharing the same cache: Since the connection count is a server side property, if various applications use the same azure cache, each one of those applications are actually contributing to the connections count to that particular cache.
Solution: Use different caches for different apps unless apps are sharing the data.