Who Can You Trust?


Awhile ago, my manager and I were kicking around ideas to solve a problem in the MSDN Forums.  By default, every new thread started on the forums is tagged as a “question”.  The question is considered “unanswered” until a reply is posted to that question that is marked as an “answer”.  The answer can get marked in two ways:



  1. The original question asker can mark the reply as an answer.

  2. Any moderator on the forums can mark replies as answers.

We track our metrics on how “healthy” the forums are based on percentages of questions that have been marked as answered.  The problem Josh and I were trying to find a solution for was that the extra answer-marking step just wasn’t happening all of the time.  Often, the original question-asker was new to the forums, and didn’t understand the “Mark as Answered” button.  We have very few moderators relative to the vast size of our forums community, and they simply can’t read all of the questions.  But somebody must be reading all of those replies…


We came up with the idea that we should give many, many more people “answer marking” rights in the forums.  Why not?  After a user had answered just a few questions–5 or 10, they would have the ability to mark any reply as an answer on the forums.  We did a quick check, and it looked like that would make about 3% of our users “answer markers”.  I said:  “Why hold back at all?  Why can’t we trust even a new forum user?  Let’s just give everyone answer marking rights.”  I went and made a post in the forums and in the DDCPX Team Blog asking customers what they thought about this idea.


What was interesting was that people generally disliked the idea.  People said that replies were already marked too aggressively now–that the basic new user on the forums couldn’t be trusted to know what really was an answer.  Somebody even replied that the original question asker couldn’t even know if their question was answered enough to mark it!


The entire thread was interesting to me–who and what do you trust in an online community?  I don’t mean “trust” as in letting them watch your dog while you are on vacation, but who can you trust to not spoil the the community experience?  Should people be given the benefit of the doubt, until they mess up (the Wikipedia philosophy, where anyone can edit or create an article), or should they have to earn their stripes before they get those rights (the MSDN Forums current philosophy)?  I firmly believe the more controlling we are with those rights, the less interested newbie users are in joining the community…but other’s believe that control is a necessary evil in making sure there’s true structure to the community.


As for the forums, we added a work item to our request list to the forums team to please give answer marking rights to everyone with over 5 answers contributed.  Time will tell if we’re going to create bedlam or not… :)

Comments (5)

  1. David Hayden says:

    Joe,

    I am a new C# MVP, and until a few weeks ago I have never stepped foot in the MSDN Forums. There is a part of me that forgot all about them, a part of me that was never a fan of forums for reasons I forget, and a part of me that typically learns my knowledge mainly through books and surfing for examples on the Internet.

    That being said, when I rediscovered the MSDN forums a couple of weeks ago to look for additional ways to help the community, I have been "crazy in love" with helping people solve problems in the forums.  Answering questions not only helps the developer in question and makes me feel good, it helps both of us keep our skills and knowledge up-to-date.  Everybody wins!

    Although I realize a community run forum will never be perfect and too much control can often do more harm than good, I urge you to be very selective in who has the ability to mark questions as answers.

    In my humble, but very active couple of weeks, I have come across a number of apparent "answers" that were far from complete and sometimes inaccurate.

    I have also noticed a few individuals who just post an "answer" to anything as if they are trying to fill a daily quota rather than actually be helpful.  Their replies are often merely 2 or 3 sentences long offering no actual help, yet the number of posts next to their name is staggering in number.

    Rather than looking at the number of posts by a person, I would look more at the quality of the answers.  Are they complete?  Are they accurate?  Do they provide code samples?  Do they not only answer the question technically but offer alternatives and mention best practices?  Do they point to other posts in the forums that have answered the same question and external resources like blogs, tutorials, and MSDN documentation for additional information?

    These are the kinds of people we want to have the ability to mark questions as answers and I urge you and your team to look for people of this caliber.

    Regards,

    Dave

    MVP C#

    http://davidhayden.com/blog/dave/

    http://codebetter.com/blogs/david.hayden/

  2. Joe Morel says:

    There definitely can be a problem with people trying to "game" the system by providing curt, incomplete answers, and marking them as answered.  To help prevent this, the forums allow the original question asker, or any moderator, to go ahead and "unmark" a question.  No, not everybody uses this, so we definitely have a "quality of answer" problem, but I don’t think it’s any worse in our forums than any other technical support community.

    Outside of moderation, what would you suggest as a way to ensure the quality of an answer?  Right now, our approach is definitely black and white, but we’re always open to creative suggestions.  We have the "post is helpful" button–but the people who are voted the most helpful are typically the ones that answer the most questions.  Maybe some sort of ratio between number of answers and number of helpful votes would help?

    Thanks for your input…and if you’d like to become a moderator in the MSDN Forums, please send me an email through this blog with your forums display name, and I’ll work on getting you access.

    Joe

  3. In my last post, I talked about how difficult it was to try and decide what groups of people you can…

  4. David Hayden says:

    Joe,

    If people think less about the forum as a quick answer to a problem and more as an archive / resource for finding quality answers, the items you have in place are fine and this is a moot point.

    You may want to make it more formal to ask and answer a question.  Perhaps have additional fields requiring more information when asking a question and the ability to only have 2 open ( unanswered ) questions running at a time.

    Answers may have a specific place to put code, links to external resources, links to similar questions in the forum that already have an answer, etc.  In this way, you have a little more information to assure questions are answered well and measure good answers.  By having information broken up that way you can also present and use the information ( code and links ) in a much more useful fashion.

    Regards,

    Dave

    MVP C#

    http://davidhayden.com/blog/dave/

    http://codebetter.com/blogs/david.hayden/

  5. David Hayden says:

    I meant to say each user is only allowed to have 2 open ( unanswered ) questions running at a time per forum.  Now that I think about it that is probably unrealistic and unfriendly.

    I was trying to think of a way to get people to mark questions as answered.

    Dave