If you have taken a Microsoft beta exam and made comments on the questions at the end, you may be wondering where they go. Does anyone read these? Do they make a difference or do we discard them without a though?
Rest assured that each and every comment is read and taken into consideration.
Our rationale for beta exams has nothing to do with allowing a limited amount of people early access to the exam for free. These betas are designed to validate the questions and answers with as many people as possible within the beta time frame. The purpose is to allow those who were not involved with the exam creation and item selection phases to provide an objective view. In much the same way as software is sent out for beta testing, this allows us to have fresh eyes on the exam in the effort to catch unwanted errors.
Once the beta exam period ends, the exam scores for each and every question goes through a thorough statistical analysis indicating the performance of each an every question across all the candidates.
During this phase, each and every comment is read and evaluated and compared to the satistical analysis. A new group of subject matter experts, from outside of Microsoft, are brought in for a focus group. Some of these people are taken from the pool of beta candidates that scored very well on the exam, and some have not taken the exam yet.
They review each question along with the comments and then work with our team internally to make decisions on the items such as;
1) Keep the item if it scored well, was statistcally good and is a valid and relevant question that tests the candidate's knowledge well.
2) Fix the item if it is a good test of knowledge but may have some other issue in terms of correctness or hard to understand etc.
3) Remove the item if is a problem from the comment standpoint, scores poorly or any other reason that rates the question as poor.
So, if you wondered what happens to your comments, I hope this helps to answer that question. Knowing this, I hope that on future beta exams, you take the time to provide us with relevant, constructive feedback that can help make a difference and a better exam for all. Comments such as, "This question just plain sucks", does nobody any good. And yes, it is in quotes because it is a direct quote from an item feedback.
For those that take our beta exams, thanks for your comments and helping us create quality, relevant exams.