Recommendation and the Harry Potter Problem

Greg Linden used to work at Amazon.

From time to time he’ll recount one of his ‘early days at Amazon’ stories on his blog giving some insight into the development challenges in building the world’s largest online shop.

In his latest post, Greg talks about the “Customers who bought this also bought” feature, internally known at Amazon at the time as the ‘similarities’ feature. One of the early problems with ‘similarities’ was the ‘Harry Potter’ problem.

See, everyone and their dog bought Harry Potter:

“This kind of similarity is not very useful. If I’m looking at the book “The Psychology of Computer Programming“, telling me that customers are also interested in Harry Potter is not helpful. Recommending “Peopleware” and “The Mythical Man Month“, that is pretty helpful.”

So he was asked to fix the Harry Potter problem, which he did.

The fix not only provided a better shopping experience at Amazon, but also resulted in Jeff Bezos walking into Greg’s office, bowing on his knees and chanting ‘we’re not worthy, we’re not worthy’. How cool is that?

One interesting point to note here, maybe even a self-referential observation. And that is I found Greg’s post about recommendation systems via the personalized version of Findory which he developed, itself a recommendation system that does a good job of avoiding the Harry Potter problem.