Microspeak: over-index

Some time ago, I wrote a document that discusses the design of a new component. In the document overview, I summarized how the component interacts with the host. If the host supports widgets, then the component uses the host's widget manager. If the host doesn't support widgets, then there were several paragraphs that summarized how we deal with the lack of a widget manager. The body of the document had a similar pattern: A short discussion of what we do if a widget manager is available, and a long discussion of what we do if it isn't.

A piece of feedback I got about the document was

Let's not over-index on the no-widget-manager scenario.

To over-index on an attribute means to give that attribute too much prominence in a discussion or analysis. In this case, a very large portion of the document overview was dedicated to discussion of the case where the host doesn't have a widget manager, even though in practice that would not be the common case. Somebody reading the document would get the false impression that the primary purpose of the component was to compensate for the host's lack of a widget manager!

My guess for the etymology is that it comes from database indexing. If you incorporate has-a-widget-manager into too many indices, then it makes your database look like its main purpose in life is to keep track of which items have a widget manager.

Here's another use of over-index from an internal memo:

We risk over-indexing on solving the immediate problem and failing to build reusable components.

The non-Microspeak version of that sentence would be something like "We risk being so focused on solving the immediate problem that we fail to build reusable components."

Bonus chatter: Note that there is another sense of the term over-index that appears to be standard marketing terminology. From what I can gather, it means "to be above average". Here's an example:

This demographic over-indexes in tech spending.

Bonus chatter 2: Some playing around with Google Ngrams suggests that the root may be in the field of economic policy.

When inflation indexing was resumed in October 1983, shelter-cost increases for the 15-month period from October 1980 to December 1981 were not reflected in the updated ceiling. The indexing freeze and the 15-month indexing gap were intended to compensate for what was felt to be over-indexing of the shelter deduction ceiling prior to 1981, over-indexing brought about by the inclusion of homeowners' costs in the CPI-U components that had been used to calculate adjustments to the ceiling.

United States Code Congressional and Administrative News, Volume 2, 1998.

In economic policy, a value is indexed to another value when it is defined to increase at the same rate as the other value. In the above case, the shelter deduction ceiling was supposed to be indexed to inflation. However, the way it was calculated was later believed to incorrect, resulting in a situation called over-indexing, where a change in the rate of inflation resulted in a outsized change in the shelter deduction ceiling because the effect of inflation was being double-counted.

Comments (4)
  1. Interesting guess on the origins of this idiom. From the title and first paragraph I got the impression that the explanation for its usage would relate to books or other form of longer writing, which usually have an index or table of content, and of which a disproportionately large piece would be devoted to the fringe topic. I could picture a book editor giving you this same feedback under the circumstances :)


  2. cheong00 says:

    Let’s just put “the no-widget-manager scenarios” on appendix. (That’s why we sometimes have Appendix longer than the main article.)

  3. Aged .Net Guy says:

    ISTM the widget example merely demonstrates the fact known to most developers. In a well-founded design much of the code and most of the intricacy is dealing with the inevitable edge and corner cases.

    IOW, if you find lots of intricacy in your main branch logic, that’s a bad design smell. If you find lots of intricacy in your corner cases, that just demonstrates for the umpteenth time that the world is a messy place.

    So, as cheong00 says, if you find the corner cases overwhelming an explanatory doc, that’s a sign to split the doc. Not to curtail the info about the necessarily messy corner cases. Complex corner cases are the last place you want to gloss over the gotchas. That’s true whether it’s a design doc, code comments, dev-level API docs, or end user how-to docs.

  4. ken lubar says:

    I think the origin of “over indexing” is from polling and surveying. You “over index” a group where the sample size wasn’t large enough. For example, if the population is 20% blue-eyed, and your sample was only 10% blue-eyed you might over index the blue-eyed sample to come up with the population in general. This is similar to “over sampling” where you have a larger than population in general sample size (in this example, if your survey sample included 30% blue-eyed).

    I’m sure someone who is more familiar with sampling design and surveying could give a better explanation.

Comments are closed.

Skip to main content