Request Queues

  • Article

How do I measure how busy a service is and how much work is queued up for later processing?

There's no single place where you can go to see all of the work items being processed by a service or a distributed system. For example, work may be queued up in a variety of locations.

- Network buffers

Protocol driver request queues
  • Transport channel request queues
  • Layered channel request queues
  • Dispatcher pumps
  • Service instances
  • Asynchronously initiated worker threads
  • Distributed activities

You can get a rough sense of the workload by fronting the machine with a monitoring service, such as you might see in a reactive load-balancer configuration. This gives you an approximation by inspecting the incoming requests and by examining the state of the system. The quality of the approximation depends on how much data you collect and how well your heuristic predicts reality based on that data. It turns out that some of the extremely simple heuristics for measuring work, such as the one used in round robin load-balancing, are actually hard to beat without a lot of data and sophisticated processing.

Next time: Shutting Down Service Hosts