OPML sampling


One of the nice things about sharing our OPML is the ability to sample feeds you’ve not tried before, but this can be hard work.


Example: Pete Cashmore and Chris Anderson (editor of Wired) have made their RSS feed subscriptions available for others to import / surf and sample. The idea is that there are some feeds I like to look of and then subscribe to these myself. In Chris’ case, this is 150 separate feeds he’s tracking. So for me to sample these properly it would take me at least an hour to go through each feed, find any relevant posts (to me) and then subscribe. You’d have to sample 450 of mine if you wanted to be thorough. 1,500 if Robert Scoble’s. That’s hard manual labour.


J Wynia (the RSS uberuser) let me know this afternoon that he’s put together something that might point us in the direction of solution to all this manual work. He calls it OPML Sampling.


It is brilliant approach and nice execution (he did this as a result of insomnia!). I love it – he’s created a page that is the result of the app he’s put together. See here.


It is a page of posts, but not *just* a page of *just any* posts. The reason the page has the posts it has is his app looks at a collection of RSS feeds he has defined (in this case the Web 2.0 Workgroup’s OPML), and finds out which of the posts within the feeds are being linked to most within each feed (this is determined using Yahoo’s blog search engine via their API to figure out the ‘best’ post per feed). This way the ‘best’ post within each feed acts as your sample for each feed. And this is what you see on the page – a dymanic *best of*.


One thought: this page is worth subscribing to in its own right! It is a dynamic ‘best of’ collection of RSS feeds ‘voted’ for by other users. Any chance of an RSS feed for this page? Oh, and can I upload my OPML? This would help me figure out the ‘best of’ my feeds at any given time? Not much to ask 🙂


APIs, attention, OPML and RSS. This is an app playing in the space I like!


J Wynia has more technical details and pointers to the source code. I really hope he continues to play with it and others too, it is great work.


Tags: , , , ,

Comments (8)

  1. J Wynia says:

    I definitely intend to keep pushing this. In the light of day it’s looking pretty promising.

    What’s funny is that the first bits of this were actually pointing toward making this into an RSS feed. I’ll probably pick that direction back up in addition to the other changes I’m envisioning.

    At the moment, the biggest bottleneck (and why I currently can’t take uploads or more dynamic inputs at the moment) is the API limits actually. What I really need to do is add some serious multi-level caching. Since Yahoo only allows 5000 requests a day and lots of the other API’s not even that manu, if you start running your main OPML file through (or even mine), without more caching, it’d be really easy to blow right past that.

    Given the daily limitations of those API’s, the first thing that needs to happen to make it more usable for more flexible purposes is to make it so it’s primarily analyzing actual new content, but still re-analyzing the back catalog as well for increased link counts. In other words, while adding delicious, technorati, feedster, etc. as inputs into the weightings, making sure that the limited number of allowed requests are also used as efficiently as possible.

    Of course, if this wasn’t just something I whipped together while watching Stargate SG-1 reruns at 2:00 in the morning using public API’s, it probably could just ignore those limitations and rely on brute force.

    Oh, and Alex, if you want to send me your OPML, I’d love to use it as test data for future iterations. I prefer to have something that isn’t mine to do testing (hence the test file I chose for the prototype).

  2. Last year Dave Winer started to push the idea of Reading Lists for RSS. More recently, the idea of …

  3. Just when I thought it was safe not to blog about OPML and RSS, along comes Technorati to spoil…

  4. This is the first of a two part podcast Skype chat with Joshua Porter and I (part 1: .mp3, 43mb, 45mins)….

  5. One of my favorite bloggers, J Wynia, has decided he’s going to run for Mayor of the Minneapolis suburb…

  6. Last year Dave Winer started to push the idea of Reading Lists for RSS . More recently, the idea of Dynamic