Two Ideas to Cut off Duplicate Questions in Online Forums


One good piece of feedback I’ve heard about on-line communities recently is that in today’s world of easy on-line publishing. A world where everyone is now getting comfortable with on-line interaction is a world where the number of people who “shoot first” with their questions before searching has grown exponentially. We can’t change those people.


“Lurk before you leap died years ago” said several grizzled public newsgroup veterans when I asked for their feedback on technical “how to” communities.  This leads to an unbelievable amount of question duplication that puts the burden on the limited number of answerers. 


Though you may not have realized it I believe this is what has led to several question experts jumping off the NNTP ship and moving to on-line forums where they can moderate and start to control the flood of duplicate questions.  (In the last 2-3 years the number of unique answerer types has been cut by over 20 % in our public developer newsgroups). But forums aren’t enough.  Moderators and experts still can’t keep up with question volume on most sites well enough to reduce duplication.  I have several ideas on how to make moderation more efficient, but those don’t get to the root cause… the duplicate questions. 


I’ve been doing a lot of thinking about what more we can do to head off duplicate questions.  I think I’ve got an idea that strikes a good balance between forced search first and freedom to post.  Let me know what you think.


At first I was thinking of some work flow that has the user entering their new question, then doing a search, then asking them if any of the results was an answer. If a result was an answer then we ask them to cancel their post. 


The problem is that this feels like a loss even in the case where users find an answer because they may have spent 30 minutes entering their whole question.  There are also lots of complications with the work flow because you have to keep track of their question even if they open a search result thread.


So what we have to do is bring back search results dynamically on the new post UI page.  Here is the work flow I’m imagining and some pictures to illustrate it. 


1. User clicks new thread on a forum. 
2. On the new post UI we bring back a heavily cached set of FAQ (sorted by most viewed in the past X days) for that forum in a search results pane on the side of the new post UI.  (See figure 1)
   a. This should head off the biggest dupes. Imagine heading off all of the “Activation” issues in the express forums for example. 
3. The user enters a post subject
4. When the user takes the cursor out of the subject we use scripting to call a search on the subject they’ve entered in the background while they may have started entering a message body. 
5. Search results are returned in the search pane based on the subject to replace the FAQ Views. 
6. Users can open up the search results in a new window so they can check them out without having to leave the new post UI. 
7. Either users find the answer or they make the new post.  Clicking on one of the threads in the search results pane will open in a new window so the user doesn’t lose their question text. 


This should be very possible with the trendy “Ajax” scripting technique used by Outlook web access web mail, google maps, etc.  The goal really is not to interrupt the user who is posting, but gently remind him there are good search results. 


Here are a couple of ROUGH mock-ups to help explain the idea.  (Warning… really bad PM art ahead.)


Figure #1 – Raw new post UI with top forum FAQ



Figure #2 – New post UI after search completes on the subject entered



Note: In real life the three lines (posted by, post date, and forum name) would likely be better off cut to enable room for more results or more abstracts. 


Comments, questions, suggestions?

Comments (20)

  1. Vurg says:

    It still doesn’t solve the real problem. The user will still go through pages and pages of search results.

    My (wild) idea is to have some sort of in-depth categorization of postings. If there is a magical way of harvesting information from posts and organizing them into a knowledge base tree, then that would solve a big chunk of the problem. Of course, there is no magical way to do that, so the user would have to cooperate.

    Usually, a specific forum focuses on one topic (a technology, applicatication, TV show, and etc). The smaller the scope of the forum, the closer the relationship of the postings will be. Maybe users can create these sub-forums through some sort of voting process. The same meta-voting features allow posts to be moved to their respective categories.

  2. MSDNArchive says:

    I think that the search result value depends on how good the search is. Your second suggestion falls in line with some of my thinking on moderation that I haven’t posted yet.

    The FAQ view, however, based on posts I’ve seen so far works well as long as users find the right forum for the question.

  3. Tobin Titus says:

    Looks eerily like google ads *shudder*. I would imagine this suggestion would obtain the same fait — being ignored. I do think this is a great feature, but honestly it won’t solve the problem.

    The problem that you stated is that expertise is spread too thin to answer all the questions. You don’t want to make it overly difficult to ask questions, but at the same time you can shift more of the responsibility on the inquisitor before they ask the question. You have to give incentive for the person asking the question to do some due dilligence themselves before asking the question. Maybe consider making two paths for asking questions — an "question wizard" and the standard question form. The wizard would allow you to type up your question, search for suggested answers and then mark some of those suggested answers as "close" or "way off" to help guage the type of response you are looking for. Going through the wizard would classify the question as a "qualified request" or something to that effect, and the question would be posted to the top of the forum. Questions posted through the standard form (without the wizard) would be posted under all the "qualified requests".

  4. Björn Graf says:

    Sounds good. But will these kind of users really take attention to such unobtrusive search results? Maybe some search results relevance based checkbox ("Yes, I have checked the search results and even if the relevance seems high non of these answered my question") could help 🙂

    Side note: the XHR should be executed in async mode and not in sync mode – AKA as blocking – like the Whidbey online library.

    "The smaller the scope of the forum, the closer the relationship of the postings will be"

    Well, newbies already get confused by the scope of the current MSDN Forums and (cross) post into the wrong ones.

    In the end the real problem with double posters is that they want an immediate answer to their questions and the idea proposed here could be a

    first step to fulfill this demand – ya, I know this is kind of contrary to my first paragraph :]

    — b.gr

  5. MSDNArchive says:

    Are google ads really ignored? Google and partners seem to be making a ton of money off of them???

    That bring up another idea… why not.. in the notification mails people get with replies to thier post we insert "other potential answers"?

  6. Chris says:

    This is a great idea, but I think you might have to combine both your ideas – e.g. what happens to those who enter a subject such as "PLEASE HELP ME…!" etc…

    So after you enter your subject you want an updated list on the right as you say. Then (I doubt it’s possible) you’d ideally want the list on the right to alter as you entered in your question (maybe there could be a separate ‘update answers’ button instead?)

    Failing that the ‘post’ button should actually be called ‘review and post’ and it would present you with a list of possible answers, then your post, then (finally at the bottom) a ‘Post’ button – i.e. before you can post you need to scroll past a list of summaries of possible answers.

  7. The MSDN Forums team is looking for feedback regarding the future direction of their RSS Feeds. Let them…

  8. The FAQ suggestion makes sense in some kind of forums – like the example you gave of activation issues for Express. But if you inside a general language forum like C#. You’ll hardly find any relevance to the top posts and the new posts being made. Hence its a good feature for some forums.

    In fact looking at other communities. Today they try to do the same thing by having a sticky *newbie read me* post. Where the moderators will keep adding the top problems and their solutions in the same post (Wikki like).

    <Snip my personal view>

    If you look on a global scale unlike many countries like USA where users have been used to using forums/BBS etc. since a long time and abiding by its cultures. Most Asian country users are just waking up to discover online communities. It’s an experience for them to ASK help outside their known group of peers. They are not aware of how to best utilize these communities.

    </Snip>

    Hence searching for them needs to be more attractive to make them use it. And I think ts a bit of education and marketing that needs to be undertaken. I like your suggestion to make background searches – this way the user does not need to perform two steps i.e. First Search and the Post. It gets in a single step process of search, if no relevant answers found post (considering the search engine can turn up right answers).

    I think there should also be some marketing involved to name the system as *Instant Answer* or *Ask Josh* – something like that to make the user feel its faster to get an answer through search rather than post.

  9. Dave says:

    What about a keyword tagging scheme(a bit like browsing del.icio.us). Both questions and answers could be associated with keywords by users. New keywords could be associated at any point later on.

    Users would search by keyword. Those Q or A’s most moderated up or newest could appear higher up, along with further keywords to refine the search further.

    In many ways it’s solving the same problems as a pure search solution, but giving real people more control over the catagorization.

  10. MSDNArchive says:

    Chris: We have considered doing a secondary search at the post & review stage. One of the issues, however, is that most search technologies aren’t yet very efficient at finding matches with paragraphs or more worth of text in the search string. And the ones that do don’t do a very good job at it. 🙂 Why do you think the google search box only accepts a limited amount of characters? I bet that people will make progress on this in the next couple years though.

  11. MSDNArchive says:

    Saurabh: I agree with your statements, but I think that idaelly we start organizing buckets where these work more frequently. It will NEVER replace, nor do i believe it could, the manual FAQ generation process.

    I also think we could do some things to make it more discoverable like calling it "Instant Answer", moving it to the left hand side of the page so eyes are drawn to it, and potentially even forcing a double check when the user hits post.

    Dave: I like the whole tagging concept. A little birdie tells me that there will be a MSFT owned forum site that implements some tagging soon. I’ll be watching carefully to see how well tagging goes over in the forum space.

  12. AdamKinney says:

    This is an interesting idea, thanks for sharing. I think comparing the proposed subject line to exsiting posts, as a start, would be useful.

  13. Chris says:

    >> most search technologies aren’t yet very efficient at finding matches with paragraphs or more worth of text in the search string

    Interesting – the same technology could also be used:

    (a) For RSS readers that want to identify/group duplicate posts

    (b) For finding emails that cover the same topic of discussion.

    Sounds like a job for Microsoft Research!

  14. I recently posted my ideas for cutting off the duplicate questions in online web based forums.&amp;nbsp;…