Crawler Performance, Hosts, and Impact [Hit] Rules

As part of the crawl processing flow, the Crawl Component starts “robot” threads that orchestrate items from gathering to feeding and finally committing their final state (see this previous post on Crawl Orchestration for more details and an illustration of that flow). In this post, we focus more on the “robot” threads, Crawler impact rules,…

4