Exam simulations: nectar of the gods


 


Getting more questions on simulations as more of you are running in to them. We call simulations performance-based testing (PBT). The reason for this is that the acronym for simulations would just be S, and that is not in keeping with Microsoft’s policies of over acronizing as much as possible. PBT, though still fairly short, is less disappointingly succinct and has a nice rhyme-y feel to it*.

Performance-based testing (PBT) in Microsoft Certification exams

Performance-based items are exam questions in which the tester must perform a task in a simulated version of the technology, rather than answering a finite-result style question (like multiple choice or drag and drop). As you all know, PB means a brain-dump user with no experience on the technology can’t waltz in and cheat their way to a “pass,” you have to be able to get around the software hands-on to get something done.

PB is only in some exams at the moment (a few ‘new gen’ exams and a handful of Windows Server 2003 exams); there are more coming. I know the strategy team has an eye to getting PB items in to many (all?!?) key MCTS exams, that would be amazing. No exams are all PB, there will be other more traditional item types, too.

In case you have run into PB in the past, you should know that scoring for PB items in new generation exams (e.g. for MCTS exams rather than MCP exams) is different than how it has worked before. Here is a little info on what to expect. 

Scoring new generation exams with a PB component


  • Your score report will include two different sections: one for your performance-based (PB) items and one for your non-performance based (i.e. multiple choice) items
  • The PB section will likely have fewer questions than the non-PB section, though all questions (PB and non-PB) have the same value per question. In other words, a multiple choice question is worth one โ€œpointโ€; a PB task is also worth one point.
  • You must pass both sections to receive a pass on these exams.
  • Scoring of PB items is based on the end state, rather than how you got there. In other words, we do not look at your path or number of clicks; if your end result is as intended, you have passed the question.

  • EDITED OCTOBER 30, 2007: I want to clarify that, per the first bullet, your score report *will* include two different sections–at the moment it does not. The new reporting format will roll-out with Prometric (not VUE) in coming months. In the meantime, your score report will only include ONE score, and it is the LOWER of your scores on the two sections. I have heard customers concerned by a really low score on 431 when they felt they performed well. In this case, it is likely that the low score was the simulations-only score; the multiple choice section would not be reported even if it were 100%, because only the LOWER of the two scores will be reported, and BOTH SECTIONS must have a passing score to pass the exam. We know this is confusing, and that is why the team is working to provide clear score reporting for this new format as soon as possible!

*It makes me think of Bel Biv Devoe (BBD). Never trust a big butt and a smile.

Comments (54)

  1. Lukas Beeler says:

    I think PBT is a good idea, but the in the exams i’ve experienced it was rather awkward (the 70-29x ones).

    The problem is that the simulations are incomplete – while i usually launch the Services MMC using WIN-R, services.msc, ENTER, this didn’t work in the simulation. Not that it’s more difficult to hangle through the menus to find it, but it’s still a hassle.

    And then there are PBTs which behave odd when you’re doing something that was not intended to be done. I remember a PBT where i had to create some DNS records and i probably misunderstood the question and tried to create multiple records with the same name – but instead of having multiple DNS records, i just overwrote the one i’ve created previously (this works completely different on a live system). That irritated me – after rereading the question it became clear to me that i tried an approach which was different from the one the questions author used.

  2. mdalligood says:

    I am soooooo blogging this.

    Thanks, T. ๐Ÿ™‚

  3. Stan Segers says:

    Thanx for answering the question.

  4. Leonardo says:

    Yeah thats nice…exams will be more fun now.

  5. Wayne Anderson says:

    I think that this is going to go a long way towards addressing some of the ongoing community concern in the integrity of the Microsoft exams process.  

    Unfortunately it will still take some time for the reputation of the exam series to be impacted and will need to be a mainstay of the 2008 exams to see much pickup from the early adopter community.

    I would personally like to see the path of 2003 continued, albeit with more PBT.  With NT4, tests were a joke.  With 2000, tests were something of a joke.  With 2003 there was some challenge there, particularly on -291 (breadth of material) as well as -297 (apparent capriciousness of answers).  

    With the 2008 exam set I would definitely like to see challenge there and would STRONGLY encourage the MSL folks to try and be more interactive with beta participants.  Follow up a beta exam attempt with a brief survey, even if the survey is only a set of email questions for them to reply to.  

    How was the difficulty?  

    Just right?

    Too hard?

    Too Easy?

    Why?

    Do you feel that this exam covered the identified objectives (link to objeectives)?

    If you were writing this exam, is there a section you would put more emphasis?  less?

    I have always felt that taking a Microsoft beta exam, Microsoft misses out on a real opportunity to get feedback on the exam contents from professionals.  Microsoft is getting a score report and, if the candidate is actaully trying to beta the test rather that just get a free exam attempt, hopefully some comments.  And while the beta process may be a little late in the exam generation process to rebuild content, that guidance can always be applied to future development efforts.

  6. Lee... says:

    The PBT questions are very enjoyable (if you know how to complete them!), and provide a nice mental break in the exams from the multiple and single choice questions.

    I do think Lukas has a good point, in that with most system engineers, they have their own way of completing a specific task, the task actions lead to the successful completion of the task, but there are many ways, the goal (task) can be achieved. However, some of the 70-29x PBT’s do tend to railroad you in the manner of doing things. It will be interesting to see how this pans out with Srv2008, what with powershell et al, adding another way of completing a task.

    And there’s nothing wrong with a big butt and a smile! I wouldn’t say no to J-Lo ๐Ÿ˜‰

  7. Alexander Trofimov says:

    Errr… Trika, I’m wondering about "You must pass both sections"… Does this mean that PBT and MultipleChoice have independent scoring and pass mark???

    And if I get, say more than 700 points (just for example) on Multiple Choice, but less than 700 on PBT then I’ll fail?

  8. Lukas Beeler says:

    Alexander, it sure sounds like it, and this is a good idea of the simu err PBTs get better.

  9. Peter Read says:

    Alexander – sounds very much like it.  Should go a long way to ridding us of braindumpers…

  10. As a MCT, my frustration has been having students encounter questions and PBT that is not in any of the "Official" academic material. The labs should be more reflective of the PBTs that are encountered.

  11. Alexander Trofimov says:

    The good idea would be eliminate multiple-choice questions and leave PBTs only in exams =)

    I’d appreciate it.

  12. trikah@microsoft.com says:

    Alexander, yes to your question. You have to pass both sections to pass the exam.

    I think we would like to create PBT-only exams, too! but Microsoft Learning would have to get a sugar daddy to afford it; they are crazy expensive to create.

    Hold up, let me run down the hall and ask Bill G for some more green…

  13. Alexander Trofimov says:

    Ok, that’s good news.

    As for exams cost… PBT are certainly expensive… But they are extremely valuable, IMHO. Of course if we mention quality of certified specialists, not the quantity =)

    I’d prefer to pay more (twice or thrice, as you wish) for each exam only not to see "wanted sysadmin for 15 PCs with MCSE:Messaging, MCSE:Security and MCITP:Enterprise Admin" =)

  14. We are always working on ways to improve our exams, as you know. We are doing a pilot of a very cool

  15. We are always working on ways to improve our exams, as you know. We are doing a pilot of a very cool

  16. Disgruntled IT Person says:

    I did the 70-431 SQL Server Implemenataion and Maintancence Exam yesterday. I got 85% in the multiple choice bit and 35% in the simulation bit. I was not expecting a simulation part and had only prepared for a multiple chioce exam. The microsoft self paced training kit for this exam has no simulation sample exams, only multiple choice. Also the 2 websites recommended by miscrsoft for sample tests only have multiple choice exams, no simulations. Does anyone know where I can get my hands on some simluation practice tests for the 70-431 exam. Without doing these I will probably just fail the exam again.

  17. Alabi Abiola says:

    I ll be taking exam 70-431 SQL Server Implemenataion and Maintancence in a week’s time , all I prepared with is the microsoft self paced training kit. How do I get practises having simulation (PBT) tests?

      This is crucial for me!

  18. trikah@microsoft.com says:

    Dear Disgruntled and Alabi,

    Our partner SelfTest let me know they have simulations in practice tests for 70-290, -291, -293, -294 and the -431. Our other partner, MeasureUp, has simulations practice tests for 290 and 291, with more in development.

    Self TEst: http://www.selftestsoftware.com/dept.aspx?dept_id=1000

    MeasureUp: http://www.measureup.com/Site/display_article.aspx?id=955

    I hope that helps.

    Trika

  19. Disgruntled IT Person says:

    Thanks, got the self test one. only 6 questions but better than nothing. Thanks Again.

  20. Examinee says:

    Does any one have an idea how to get the latest, up-to-date exam prep for mcts 070-431 exam? Selftest has got just 6 questions but there are 12 simulation questions in the exam. Does anyone know how/where to get the exact questions, if youn like examcheats?

  21. rickj says:

    if ms sells a training kit for a ms cert why would the test not be reflective of the course matereal their selling and the exams arent free

    most jobs ive resarched havent required a ms cert so why would i pay and pay and pay for some thing thats not realy needed

  22. SMOTS says:

    I wrote Exam 70-431 and passed yesterday.

    The exam has been updated on my MCP Transcript BUT WAS NOT given the MCTS certification and the MCSA(CORE Exams written). What do I do?

  23. SMOTS says:

    I wrote Exam 70-431 and passed yesterday.

    The exam has been updated on my MCP Transcript BUT WAS NOT given the MCTS certification and the MCSA(CORE Exams written). What do I do?

  24. bali_kad says:

    I have cleared the exam 70-431 and also enjoyed it.

    Microsoft should change the pattern of all certification exams as 70-431.

  25. Bruce Waid says:

    I passed the 70-431 exam two weeks ago.  The simulations were interesting and I had heard there were some on the exam and so prepared as best as I could.  Unfortunately, the Microsoft Study Guide for the exam only covered the multiple choice questions.  It would have been nice if there had been some simulations to work with.  The problems with simulations is that there are always multiple ways to accomplish a task.  If you came up through the various Windows O/S as I did, you learned many ways to do things.  That is why I am curious on how the "final result" is scored.  To me, that means a comparison of the area(s) of the registry affected or any configuration files created.  The path taken to get there may have very little (if anything) to do with the final result.  Unfortunately, if you get set in your ways, you may not achieve the correct final results the way Microsoft wants.

  26. Please can someone let me know what the pass marks are for the multiple choice part of the exam and the practical/simulation part. I’m referring to Microsoft’s SQL Server 2005 70-431 exam here. Many thanks.

  27. trikah@microsoft.com says:

    SMOTS, have your certifications shown up on your transcript yet? If not, have you contacted your regional helpdesk? Let me know.

  28. Kathy says:

    I too am interested in the pass marks for 70-431.  I’ve been studying the multiple choice route for several weeks now and am getting ready to take the test.

  29. Shivam Guness says:

    I just cleared the exam yesterday. The simulation part of the exam was all that I had expected but it was very long about 1 hour 45 minutes. The only thing I think that should be improved is there should be a detailed report based on the exam objectives. I got just 2 lines, 1 was performance   and the other non-performance.

    Regards

    Shivam

  30. Dev says:

    I passed 070-431.  Is ther any surprises on these 070-443 and 070-444.?

  31. Toby says:

    Yes, I would also like to know if there is any PBT in 443 or 444.

    Toby

  32. icm says:

    To Those Who Passed 70-431 exam recently,

    what takes to pass the exam and what did you use to study? Please advise.  It’s important for me.

    THanks!

    icm

  33. balu says:

    i would have been glad, if the 431 live exam timer countdown clearly showed that the 2 sections are timed separately.

    e.g. Section 1 of 2 Time remaining 00:59:48

    to avoid costly and tensed surprises

  34. b says:

    Are there any sample test simulations available to practice them while preparation for the exam?

  35. Manoj Sharma says:

    i want to know about the SQL Certification and pass (%) of Exam. It also include what are the contants of that Certification.

    give me answer at my E-Mail Id : manojsh21@rediffmail.com

  36. Rajesh says:

    Its fun to get arround the skill testing and make yourself evaluation much easier. It also builds up confidence and self esteem.

  37. Peter says:

    I will be taking the 70-431 exam quite soon. Could somebody please comment on how complex the PBT Simulation questions are? Are we talking configuration tasks > 5 steps with a correct outcome to achive a score or are we talking lesser UI questions like "Create a new scheduled task called "Exams Make Me Nervous" in the SQL Server Agent"?

    Have I understood correct that the pass mark is 70% for both the Multiple Choice and the PBT sections independently?

    Supposing I do not pass on my first attempt (lets not be negative with the ‘F’ word), does every subsequent attempt at the exam require a full exam fee or is there a free retake or discount on retries?

    Would I receive any feedback (as a result of a non-pass) that would indicate whether there’s a particular area of study that I should focus on?

    In the exam does the exam software allow one to pass on a question and return to it later?

    Is the 70-431 exam really a 4 hour epic as indicated during the scheduling of the exam, or is it a bit shorter?

  38. Webster says:

    Citrix has an exam that is like 90 to 95% simulations (the 1Y0-456 exam).  They give you 3.5 hrs to take the exam.  There are probably 5 to 7 multiple choice questions and the rest are sims.

    The exam was so difficult that originally 95% of test takes failed.  Even the Citrix Certified Instructors were failing.

    There were many issues wwith the sims.  For example, say you are given the task of creating a user in AD and also creating a mailbox for the user.  You would just go to ADUC, create  anew user and check the checkbox to create the mailbox.  Only the dialog box would be there to create the mailbox but there would be no checkbox!  Makes it very hard to pass the sim question when you can’t complete the sim.

    Citrix had to go back and manually rescore every exam, rebuild the sims, completely redo the scoring engine and drop the passing grade to a very low score of, IIRC, 53%.  Even after that the failure rate was around 65%.

    It took me two times to pass that exam even after 18 yrs of Citrix experience.  The exam also cost $300 a try.

    So Microsoft, if you ever go to mainly sim type exams, I hope you listen to the beta test takers and fix issues before you make the exam available to the public.

  39. ZhangZhu says:

    I did not pass the 70-431 because of the sims did not work even though I knew how to do it. Anyone had the same experience.

  40. Fayyaz says:

    There is nothing new. IIS Exam in Windows NT Track was of the same format.

  41. niv says:

    simulation did not worked and i failed.

    what to do for this problem. will they re regsiter or will they return the money

  42. trikah@microsoft.com says:

    Hi, ZhangZhu and Niv. I’m not sure what you mean by "the simulatiosn did not work." if there was a technical problem, you can escalate that to your test center directly, or to MSLEDP@microsoft.com. If you think the exam is unfair or incorrect, you can start an exam challenge, per the information here: http://www.microsoft.com/learning/mcpexams/policies/itemchallenge.mspx

  43. I’ve heard that some of you have had problems recently when taking one of our exams with performance-based

  44. trikah@microsoft.com says:

    Hi, ZhangZhu and Niv. It sounds like you’re not the only ones to have issues, (http://blogs.msdn.com/trika/archive/2008/07/21/trouble-in-sim-city.aspx) please let me know via e-mail if you did not get rescheduled and helped appropriately by Prometric and we’ll get you sorted.

    Thansk,

    Trika

  45. Overthinker says:

    If the simulation exams now in circulation are officially dubbed Performance Based testing, and there are emulation exams in development, then are the emulations also going to be lumped under the PBT moniker or will they have their own unique (and non-succinct) acronym?

  46. Nice says:

    Is ther any performance based questions on these 070-443 and 070-444.?

  47. muhammad says:

    Am preparing for the exam but i need more guidance about the pass mark,type of question

  48. shahid says:

    070-444 exam has no simulations. it has only mcq

  49. shahid says:

    070-444 exam has no simulations. it has only mcq

  50. Robert says:

    Simulations are fine if they are accurate and reliable. Based on the 70-620 book which has a copywrite of 2007 and an errata in MS Knowledge Base pointing out over 20 mistakes in the book, I dont trust microsoft to get it right. Especially after reading the comment about Citrix tests with simms. It does not seem fair to put simms in the test and not offer practice simms in the book or at least all of the possible simms made avaialable on MS learning for a small fee or free as they are not in the outdated $60.00 MS Press Book. I have used windows since 3.11 and there is always more than one way to skin a cat.

  51. Manju says:

    Hi,

    I am going to take the 70-431 exam this week. Is there any dump for simulation questions.

    Plz help.Plz Send the dump to manju.visvanath@gmail.com

    Thanks in advance.

    Regards,

    Manju Visvanathan.

  52. @Munju says:

    Dumps damage the integrity of certification.  Their usage is unethical.  I invite you to study for the exams like the rest of us.  Here’re two links which are the beginning of your path to success:

    http://www.microsoft.com/learning/en/us/Exam.aspx?ID=70-431&locale=en-us#tab2

    http://www.microsoft.com/learning/en/us/Exam.aspx?ID=70-431&locale=en-us#tab3

    Jimmy May, MCM:SQL

  53. Ravi says:

    Hi,

    Can anyone plz tell me the exact number of simulation and multiple choice question r there for the 7-431 exam…. Plz reply…

    Thanks,

    Ravi