Understanding Lists and SLE

  • Article

Hey! This is Arvind. I am a Software Design Engineer in Test in the RSS Team working on among other things, the implementation of the Simple List Extensions in IE7 and the Windows RSS Platform.

There have been a few posts on RSS in IE7 regarding the user experience, the RSS Platform, and the Simple List Extensions.  As Sean mentioned in the previous post, there are two parts to the Simple List Extensions: the ability to treat a feed as list and the ability to define customized sorting and filtering functions.

I want to go into details about the benefits of using SLE to define a list and how it impacts the user experience and the platform. This is a fairly long post, so I’ve split it into 4 parts: the ‘why’, the ‘how’, ‘what’ and ‘when’ about lists!

I - Why do we need Lists

Feeds are used in various applications today. There are news feeds that keep users updated on the latest headlines; blogs that give an insight into experiences; media feeds of photos, audio, and video; travel feeds that provide discounted fares; and commerce feeds on product availability. By subscribing, users can continually get new information and updates to existing items across all of these feeds.  However, there are cases when outdated items need to be treated differently.

Let us take a closer look by comparing two different feeds.

a) News Feeds

Every time a news site publishes new items on its feed, the RSS Platform downloads and adds these new items to the existing store of items (which posted earlier by the publisher). The user has the capability to view the recent additions as well as the items published a few days earlier since the RSS platform stores all items for this feed. News feeds and blogs, by their nature, are particularly well-suited to archiving.

b) Lowest Fares Feed from a Travel Agency

Now consider a feed from a travel agency of the latest & lowest fares available for purchase.  If you view a travel website, it displays the fares that are relevant for the moment by removing fares that have been sold out or expired and introducing new promotional fares. The website’s feed also reflects the relevant fares and remove the outdated ones. When a user is subscribed to the feed, new and updated items in the feed are added to the aggregator’s local store.

The problem is that the local store reflects a mixture of new, changed, and old items that may be outdated or unavailable. There is no easy way for the user to distinguish the available fares from the unavailable ones. Clicking on an outdated fare may lead to an error that specifies that the 'Page does not exist.’ This can confuse the user and can lead to a unusable experience.

Detailed example

Let’s say that a user is subscribed to a travel agent feed on March 18th.  The live feed is the feed published by the web server on the left, and on the right is the user’s subscribed feed that is consumed in a RSS aggregator.

By March 21st, there are new fares available on the live feed.  Also, some of the fares have expired – like the trip to Chicago.  A traditional aggregator adds the new fares but doesn’t delete the expired fares.

This means that the user now has to distinguish what is expired from the relevant items.  The problem only gets worse over time. The experience after 3 months of subscribing to the feed results in more outdated items.

For these kinds of feeds, a better experience would to see the same data in the aggregator as the feed on the web server (live feed). The aggregator should also indicate which items are new since the last time the user viewed the feed.  In order for a user to benefit from these kinds of feeds, there needs to be a way that a feed publisher can indicate to an aggregator to keep only the relevant information.

In the diagram below, we've gone back to the first update. On the right is a list feed that mirrors the feed on the web server. By subscribing to the list feed, outdated items are removed and new items are marked as ‘new.’

By having publisher express that their feed is really a list feed, users can gain value of subscribing to these feeds to automatically get new items and interact with only useful data.

II - How to create a List

The Simple List Extension’s “treatAs” feature was created for exactly this scenario – to allow feed publishers to indicate to an aggregator that only the current set is relevant to the user. 

So, how does one mark a feed as a list? Really simple! Add the <cf:treatAs>list<cf:treatAs> tag as a child element of the top-level channel (or feed) element. SLE provides a useful feature with minimal developer work – and best of all, because its defined in a namespace, if an aggregator doesn’t understand the new element, it will ignore it and continue to handle the feed as it normally does (the results will be less-than-ideal, but at least the user will see the data).

RSS 2.0 Feed as a List
  
<rss version="2.0" xmlns:cf="https://www.microsoft.com/schemas/rss/core/2005">
<channel>
<cf:treatAs>list</cf:treatAs>
<title> Lowest Available Fares today </title>

..........
..........

  </channel>
</rss>

Atom 1.0 Feed as a List

<feed xmlns="https://www.w3.org/2005/Atom"
  xmlns:cf="https://www.microsoft.com/schemas/rss/core/2005">

    <cf:treatAs>list</cf:treatAs>
<title>Lowest Available Fares today</title>
..........
..........
..........

</feed>

III. How does the RSS Platform and IE feed view behave when viewing a list?

Let us look at how the RSS Platform and the IE7 Feed Reading View deal with list feeds. 

The RSS Platform stores all of the items in a feed and on a scheduled basis, it checks the live feed (on the website) for updates. When it detects a list feed (via the presence of the treatAs element), it compares the existing items stored for the feed with the live feed items.

  • If an item in the local feed store is no longer in the live feed, the item in the feed store is removed.
  • If an item in the live feed is new, then it is added to the feed store and marked as ‘unread’ (i.e. it appears new to the user).
  • If an item in the live feed is already in the local feed store, but the content is different than the one stored locally, then the item in the feed store is updated.

The RSS Platform also preserves the order of the items exactly as it appears on the live feed.  In some list feeds like “Top 10 Movies” the order in which the items appear bear relevance.

The user can read feeds in IE7 using the IE Feed Viewer.  The reading experience for lists is virtually the same as feeds:

  • Items that are updated or new since the last published time are marked as ‘new’ using a blue color and bolded text.
  • Items that are NOT new are marked as ‘read’ using a grey color.
  • When a user views the list feed, all items are automatically marked as ‘viewed.’

The differences the user sees in a list feed that is different from a regular feed are:

  • By default, the items are sorted in order as they appear on the live feed (for ordinary feeds, the default is to sort by the date the item was published/modified). The user can reverse the sort order, or sort by any other relevant sorting.
  • By default, all items are shown (for ordinary feeds, the default is to show only unread items).

IV When to use lists

Here are some scenarios where list feeds are more appropriate than regular feeds. These applications almost always provide a relevant set of items and don’t want the user to see old/archived data.

1. Feeds that list the ‘Top N’
  
Examples:

  • The Top 10 Movies of the weekend
  • The ‘Top 10 Expensive Real Estates for the month
  • The 100 most downloaded songs of the week

Here are some examples of live Top N feeds that are using SLE today.

2. Feeds that reflect a user’s list of things (wish lists, bridal registries, todo lists, etc.).

Let’s take wishlists as an example. Imagine an item the user removed from the wish list because he already received it. If this was a regular feed and was archived, a friend who views this wish list could end up gifting the item the user already has!

This is an example of a Amazon wish list using SLE: Jeff Bezo's Wish List on Amazon.com.

You can make your own and share with your friends and family, too.  Watch the video here.

3. Feeds of available products

For example, an auction site. The feed can contain sellers’ items with a particular product match.  
  
Here's an example of an eBay search for Xbox 360.

4. Lists of open issues

The RSS Team is also using list feeds internally to improve the process of tracking active bugs in IE7 during the development process. New bugs get opened and old bugs are resolved every day, and all bugs are stored in a database. When a developer wants to know what bugs to work on, she queries the database. These queries can get slower over time as more people are doing queries.

To alleviate this issue (and to try out SLE, of course), we created a list feed that lists all the active bugs for each IE7 feature. Every day as the list of bugs changes, the feed updates itself to reflect this new list of bugs.

The traffic to the database server is substantially reduced because the database has to be contacted ONLY once to create the Bugs List as opposed to creating a connection with the database every time a user wants to find out the active list of bugs.

When developers on the team subscribe to this list feed and view it in IE7 (or any other application using the RSS Platform), they see ONLY the relevant active bugs opened against their feature. Using SLE, we have made the process of tracking active issues very simple, fast and scalable.

Thanks for reading.  I’m excited to see feeds develop and take form as list feeds.  If you have any feedback, please leave a comment.

- Arvind