The evading solution to large remoting messages

  • Article

Thanks to rick for the title!

We are seeing more and more people encountering:

Q322975 BUG: Passing large quantities of data in .NET Remoting calls causes an unexpected exception to occur - https://support.microsoft.com/?id=322975.

As you can tell from the title of the KB article, this is indeed a bug. Unfortunately the behavior addressed by this knowledgebase article is more of a “by design” problem than a coding error. Remoting was never designed for large messages and to my knowledge no testing was done on the types of large messages that cause the errors mentioned in the KB article. From what I have seen with remoting a “typical” remoting capsule is in the neighborhood of 16k – 32k (This is a very generous estimate).

There’s a HUGE difference between 32k and > ~5MB. Taking the development stance you can see why this behavior is by design. However, taking the implementer’s stance, we can see that when using datasets and other complex objects real world scenarios can easily exceed 32k. Developers don’t like fixing code that has an “easy” work around, and implementers don’t want to compromise their design or already coded applications. What can be done?

Well, the last time I checked both you and I lived in the real world, so here are some real world things that can be done to mitigate seeing the SocketException from Q322975.

  • The source of this exception is actually within the networking stack and is courtesy of the implementation of one or another socket library. Since I’m not familiar with that code I’ve had to speculate about a few things and can tell you this: The error is somewhat hardware bound and throwing hardware at it has mildly successful results. For example, I’d see this problem sooner and more often on a PIII 600 Mhz with 256MB Ram and lest often on a P4 3.6 Ghz with 2GB of Ram. I’m under the impression that the buffering of the incoming data is somewhat dependent on the system hardware. That means that this exception can be alleviated to some extent but not eliminated. This is kind of like taking some cold medication; the medicine suppresses the symptoms but you still have the cold…it just may or may not be there when the medicine wears off.
  • If you’ve already maxed out the hardware and want the least impact to your existing codebase possible, you can create a new remoting channel that implements some level of data compression. Think of this as zipping up all of your remoting messages. Of course this requires the minor change of modifying your remoting configuration. It might cost you time and money depending on your ability/willingness to write your own channel implementation. You can certainly save both time and money by using a 3rd party channel implementation such as the one provided by GenuineChannels.
  • A more involved solution includes actually modifying your code to a slightly larger extent. The benefit of this option however is that it’s not as involved as writing your own channel and does not involve the acquisition of a 3rd party product. Another type of compression that can be implemented is actually shrinking the remoting message by excluding that data which is not needed by the client. Please consider the following KB article: Q829740 Improving DataSet Serialization and Remoting Performance – https://support.microsoft.com/?id=829740. Simply put, you modify the serialization of types you pass via remoting in such a way that you are sending only data that is necessary on the other side of the remoting call.
  • The last option is indicated in the original KB article: chunk your data into smaller pieces. This work around involves modifying those calls that generate huge messages into several smaller pieces. I recommend that messages probably shouldn’t exceed 2 MB as I’ve never seen this exception on messages smaller than twice that size.