Syndicated search engines broken - Part II

  • Article

A few days ago I grumbled at the poor state of the search engines specializing in syndicated (RSS'd or Atomized) content.

Today, Michael Marshall Patrick is enthusiastically supporting a proposed standard by Bloglines that is trying to solve an apparent problem:

"“Everything you blog goes on your permanent record!” How many times have we heard that lately? From employment to family situations, many people have been frustrated to find out that things they intended to write for a personal audience is now discoverable by anyone in the world via search engines.

From the Bloglines proposal:

"As a result, we are proposing (and have implemented) an RSS and ATOM extension that allows publishers to indicate the distribution restrictions of a feed. Setting the access restriction to 'deny' will indicate the feed should not be re-distributed. In a nutshell, the proposal"

I respectfully disagree with Michael's Marshall's view here, and a user of these services, can not support the proposal, for three reasons:

1. Keeping stuff out of participating engines wouldn't ensure leakage. As one commenter on the quoted post has already pointed out (by '007') how do you avoid the repost scenario? If you really need to sneak stuff under the radar (to avoid getting fired???), use something other than public blogsite - you will be found. Another reason: why wouldn't some service providers show up that wouldn't adhere to the rules that ensure they catch the slime? (I could imagine 'Slimesearch'...). Private networks - ok = group IM, SSL'd, groups, etc (even company email considered leaky) - but just don't use inherently public networks for this kind of stuff.

2. A common issue with search results is spam. Spammers won't use the tag. I realize this isn't a stated goal of the proposal, but worth pointing out, I think.

3. IMHO, these guys (Bloglines, Technorati, etc) should be focussed on trying to solve the precisely reverse of the 'problem' they are trying to solve here with an access:restriction' tag - they should be trying to get more complete indexes, not the other way around.

Overall, this syndicated content search space is broken. The priorities seem wrong here - I don't see this step getting us any closer to getting better services when there are other much more fundamental issues that need solving.