braindumps and whatnot


Hi, here is the recording link from last week’s anti-piracy, anti-cheat meeting with Peggy Crowley.

Comments (33)

  1. Thank you Ma’am. I missed you during the second session. Now get back to work.

    btw, the beer is chillin’.

  2. helmer@hznet.nl says:

    Thnx Trika! I wasn’t able to join the session. I’ve didn’t had the time (getting married this summer, my gf is moving in, studying,etc.etc).. ๐Ÿ™

    Isn’t the session available as a podcast? ๐Ÿ™‚

  3. Garth Jones says:

    In this Live Meeting event Peggy talked about what and how MS is trying to stop Exam cheaters and what

  4. Garth's Blog says:

    In this Live Meeting event Peggy talked about what and how MS is trying to stop Exam cheaters and what

  5. jtb says:

    brilliant!  cheaters to be "pwned" by high-tech…

    <a href="http://en.wikipedia.org/wiki/All_your_base_are_belong_to_us">"AYB"</a&gt;, indeed!!!  

    I’m glad for this particular piece of news…

  6. jtb says:

    Meanwhile, I’m wondering why I can’t remember how to add a link…  ๐Ÿ™

  7. BJ Bond says:

    Thanks for the link. This was an informative session. I especially liked the slide on the NDA questions that can be answered and the questions that canโ€™t be answered.  Iโ€™m always running into people that want to know how many questions are on the exam and how long it takes.  Itโ€™s not really relevant in my opinion, but the knowledge provides some comfort for them as they go into the exam so it was good to know this is โ€œlegalโ€.

    I do see a disconnect with one question though and am a little confused.  It  was:

    Can I post or share questions if it is not word for word?

     The provided answer is No, it would be against the NDA.

    However, isnโ€™t that what respected practice test sites such as MeasureUp and Transcender do? Are these practice test sites violating the NDA and are we wrong to use them? There must be a line between the braindump sites and the legitimate practice test companies, but Iโ€™m confused on where the line is.

    Similarly, test prep books have questions and answers that are often similar in content to the live exams. I canโ€™t imagine an author writing a book without taking an exam. Are these books against the NDA and is it wrong for people to use the books? (These types of books are published by MSPress, Sybex, ExamCram, and more. The question sounds Nineteen Eighty Four-ish, but are some publishers more acceptable than others?)

    Should one of these companies, publishers, or authors be concerned? For example, if a MCP or MCITP wrote a book that included practice test questions, should she be concerned that she may end up with a lifetime ban?

    I do understand the spirit of the statement โ€“ individuals shouldnโ€™t take an exam and then regurgitate it in the form of braindumps that others can use. However, Iโ€™m unable to determine where the line is in Microsoftโ€™s eyes and Iโ€™d hate to be on the wrong side of that line.

    BJ Bond

  8. trikah@microsoft.com says:

    Helmer, congraulatiosn on getting married. That is excellent, indeed. But what does your wife think about your gf moving in?????

    JTB, are you drunk? Sometimes, when I’m really really incredibly drunk, I can’t remember how to add a link. If that’s not it, I’m not sure what your problem is.

    BJB, hi. I know the "line" is fuzzy – I wish it were less so. I will try to summarzie the can/can not in a single post pretty soon here, but in the meantime our guidance is —-IF it is on the prep guide, talk about it. If it is not, don’t. If you aren’t sure you’re allowed to share it, don’t—– Here is what Peggy tells me "We want people to use their best judgment and not disclose information relating to the actual exam content.   Legit practice test providers and authors are not disclosing actual exam content." More on this in a quick minute.

  9. WallyTgoat says:

    Trika thanks for the link.. I was really not happy when I realized I missed the Live Meeting.  Now I feel better… (Note to self: get Doc to test me for OCD.)  Your the best!

  10. LarryWestMCSD says:

    Trika: In the new exams, the Preparation Guides have a lot of lines stating "May include, but is not limited to …."

    Would it or would it not be wrong to state that one of the things not mentioned on the prep guide but on the exam is …. ?

    What about "There were x questions on topics not mentioned explicitly in the preparation guide."?

    It would be nice if the guides didn’t have the "is not limited to" part. It makes it harder to study.

    P.S. Do you know where I can find some braindumps for that new MCITP: United States History Major? I am afraid to ask Thomas C. Tips, as he may be busy doing other stuff. ๐Ÿ™‚

  11. Liz says:

    Hey Cuz!  Just Googled you.  It was sort of hard to pick you out among the thousands of "Trika Harms zum Spreckels" out there…  anyway, BOO! [[disappears again into the ethernet]]

  12. I got the email earlier as if to mock me for missing the live session. ๐Ÿ™‚ It seems every time I schedule time to see a session live something comes up….This time it was a SharePoint Server issue…

    Will take a look at the recording tonight, thanks for the reminder. ๐Ÿ˜‰

  13. jtb says:

    No, not drunk…  haven’t been since the day I, well, that’s not important now…  but I’ve forgotten the syntax…

    I guess that’s not important either…

    I was just happy that the technology has finally caught up with the cheaters…

  14. jtb says:

    This is what I meant: [url=http://en.wikipedia.org/wiki/All_your_base_are_belong_to_us]AYB[/url]…

    Forgot about the BBCode…    must be getting ready for AARP…

  15. Shawn says:

    It was mentioned it was statistically possible for MS to tell when someone has been using braindumps – seems like a slippery slope to accuse someone and perhaps ban them because they attained a high score in a short period of time (the likely criteria) I have aced a few MS exams over the years in a short period of time simply because I know the material, I don’t want to possibly get accused of cheating in the future if I ace an exam. Although there are probably factors that would work in my favor – The fact that I have been an MCSE since 1997 (NT,2K,2K3) probably would.

    I just can’t see statistics with no other evidence grounds for saying someone cheated.

  16. Peter Read says:

    Must say I kinda agree with Shawn – I took 70-299 a few weeks back and got everything right, which might look dodgy statistically.  

    Ok so maybe the "Yeah but I didn’t get 100% on the last 15-20 so maybe it’s statistically likely for me to get that in 5% of exams" would clear me, but relying solely on statistics is going to produce false positives.  Almost guaranteed.

    That said "braindumpers" need to be combated somehow…

  17. hartplaza says:

    On all my exames, I only had once a 100%, and that exam I felt it was a waste of money doing that exam. I found it to easy, there where only 2 questions where I was in doubt between 2 answers.

    Anyway. I think there is a possibility. If you do an 50 questions exam in 25 minutes, and have a score of 100%, You may seriously doubt the result. That are 2 questions/answers a minut. If you have a lot of text in the questions/answers, then that would be inpossible to read the whole questions and the answers.

  18. Franz says:

    Thanks for the link to the recording. I was surprised to hear that new tests are on braindump sites within a day after publishing them. How’s that possible without someone from MSL or one of their vendors leaking the tests? Maybe I’m naive but shouldn’t it be the most easy thing to dry out this source of leaks?

    What I still don’t like about the dump site thingy is that although some of us are giving hints about them to Microsoft for ages nothing happens. E.g. certbase.de should be closed down real soon now ™ because they obviously have verbatim copies of test questions and answers for a lot of MS exams. Especially the 70-620 is frightening. Any German who took this test should be asked to retake it with a whole new batch of possible questions that are not stored at certbase.de just to see if they are masters of memorization or if they really know Vista.

  19. Kevin says:

    I guess I’ll have to watch the show and see what’s up, but I can say that using statistics to spot braindumpers probably isn’t as far-fetched as you think it is.  I’ve heard of people going to bootcamps where all they do is study braindumps for two weeks, and then take the exam when they feel like they’ve memorized it well enough.

    For example, let’s say that Microsoft sees a newly certified MCSE who lives in Virginia but took all 7 exams at a test center in Atlanta in the span of a week and a half.  Let’s say that the MCSE in question had perfect scores on 5 of the 7 exams, and 970+ scores on the other two.  Would you suspect them of using braindumps?  What if there were 12 other newly certified MCSEs who took the exact same 7 exams during the same period at the same location and had similar results?  Are you telling me that wouldn’t be a giant red flag waving over a giant neon arrow pointing at the word "cheater"?  Because in my mind, that’s one case where statistical analysis probably is going to hit a home run.

  20. LarryWestMCSD says:

    Unfortunately, Kevin, what would probably happen is that the MCT involved would be considered such a great teacher that he would get more gigs doing the same cr*p, and would thus get more rewards from Microsoft than the rest of us.

    I don’t know if it is possible, but one idea would be to place a "seed" in each exam in order to determine who leaked it. For example, if there is a question involving a GUID and/or even a company name or name of a person, each exam, at least in the beta and the first couple of days of release, would have it’s own values, and could be traced to the person taking it and the exam center. If the braindumps contain that person’s values, then he is probably the leaker.

    One can also seed the order of the answers. In a four-choice question, there are 24 (4 factorial) ways to order the answers. In a 40 question test, that makes 24^40 (16.1 septendecillion) different ways of ordering. That a match between a braindump and the order the questions were given to a particular person would be the same would be astronomical that the person was innocent.

    Another idea would be for Microsoft to flood the market with false braindumps, where perhaps the answer choices are the same, but the questions slightly different with different correct answers. (The answers in the braindumps would have to match the braindump’s question to avoid it obviously being a fake, but not match the real questions enough to let the person pass.) I wouldn’t buy a Rolex, even if I could afford one, for example, because there are too many fake ones out there. Since Microsoft will, unfortunately, never get rid of braindumps, reducing the validity of them would be one step in the right direction.

  21. Peter Read says:

    Larry – no there are ten ways to order a 4 answer multi-choice question’s answers. i.e. n*(max+min)/2. And for the questions there are 820 (same formula but I’m tired so arithmetic might be off…)

    even multiplying the two, if 50k people take it then you’ll end out with 6-8 false positives.  Probably a lot more when you discount "obviously wrong" answers.

  22. bwatson says:

    One of the ultimate ways to combat braindumps and cheating is to move our exams to performance-based testing (PBT) where you remotely use the actual software and network configurations to complete various scenarios.

    Until recently this was the holy grail of testing but with recent advances in virtualization technology, improved online ‘virtual labs’ and testing centers having more robust Internet access Microsoft Learning is already piloting using PBT scenarios in our certification exams.

    In this model it will be extremely challenging to ‘copy and paste’ questions as these scenarios involve setting up multiple computers and can take 20min or more to complete. Since you are using the actual software there are multiple ways to complete the scenario but we measure the ‘end state’ of the computers/networks that you have configured. As you can imagine this is an extremely complex challenge and has been under development internally for the last few years but PBT scenarios will become a core component of our exams in the near future.

    More on this to come over the next few months.  

    Sincerely,

    Ben

    Ben Watson

    Director, Microsoft Learning

  23. Peter Read says:

    Sounds great if you can do it.  I always assumed that ‘measuring the endstate’ was a fairly small comment for a horrendously difficult thing to really do.  

    But yes, ideally I guess "Set up automatic IP addressing and name resolution for a private class C network, and ensure a reverse name resolution is available" then dump someone at a server console is the way to go for "the 70-291 equivalent for Windows 7" ๐Ÿ™‚

  24. Stephen says:

    The old Novell method may also help. I can remember in my CNE 3.12 days, you would start the exam, get a couple of questions on DNS, if you answered correct, well done, but if you answered wrong, you got more and more harder questions on DNS.

    If the total exam was 60 questions and you got enough incorrect after maybe 5 or 10 you failed without seeing the complete exam,

  25. Kevin says:

    Ah…the old "adaptive testing" spectre.

    I agree, performance-based testing is the way to go.  A lot of people who I’ve talked to always seemed worried about the simulations on the MCSE exams, but I found them refreshing.  They never seemed tricky to me, and having years of experience with the product meant that I found doing the simulations easier than some of the Q&A style items.  I can see why you’d have a hard time doing an entire exam of PBT/sims simply due to the amount of time it could take, but I’d definitely like to see more of that in the exams.  I was very disappointed along those lines with the exams that I took for the MCITP:Enterprise Administrator cert.

  26. Guillaume says:

    I have gone through all the exams required for the MCITP:Enterprise admin and MCITP:Database administrator.

    Now I suppose because of the NDA I can’t mention which exams have, or dont have simulations..

    But what I can say is that I thought the MCITP:Enterprise admin was very easy, and could be braindumped through extremely fast.

    The MCITP:DBA was a little harder IMO, but could still have been braindumped through easily.

    The simulations I have seen right now are just limited ones, a few clicks and you’re done. I can’t wait to see the actual virtual-server-through-rdp simulations. Those will be awesome, and I think it would be a much better exam.

    For example you could have 4 20minute questions, that’s it. Have flexible scoring for different items in each question..

    That way, by having only 4 "situations" per exam, you could design a hundred of them, and make it very hard for braindumpers to know which questions they will get, and even harder to memorize all the steps to 100*20minutes..

  27. S.Y. Paul Lai says:

    1. adaptive testing

    2. place a "seed" in each exam

    The above methods cannot stop the dump producers.  The problem is the at the test centres.  They can just attach some video recording devices to the test computers to get all the exams contents from the test takers.  They don’t need their employees to take the exams.

    Some "more" honest test centres will distribut the dump questions to all their students and to the Internet.  (Yes, there are well known training centres who distribut special notes and deliver "Pre-Exams" for their students, but everyone in my city accept their existence…)  Some "less" honest test centres will only distribut the dump questions to their own employees for "internal use".

    No test center is trustworthy.

  28. sean mcveigh says:

    i attended the event and found the presenter to be okay.  obviously, she does a lot more legal work and lot less presenting that the technet team.  it’s a tough gig.  That being said the material was worthwhile and i hope that we are seriously getting to the point where we can provide good study materials and prep materials that are compelling and eliminate as much of the braindump stuff as possible.  

  29. Pierre says:

    hi there, very interesting comments outthere…

    I’ve been in IT for more than 10 years now, i’ve teached IT material a few years back, and i recently had to pass a bunch of exams to update my resume and obtain a raise (you now the way it is with HR).

    ok, i passed 15 certs during the last 8 months, earned first my MCITP:EST, by the way passed 70-620 without any preparation work, taking advantage of the second chance offer to discover what where the MS Exams like… Ok i had participated in beta since early ages ๐Ÿ™‚ but none the less, i seriously doubt having to cheat for this particular one, lol… then took my way up to MCSA 2K3, then TS upgrade to 2K8 during techdays (coupled with betas Vista exams – Media center, home theater related) for a 3 days 4 exams+workshops and sessions (techdays stuff,lol)… i then terminated my MCSE 2K3, then passed the 70-643 (that particular one too seemed far too easy if u ask me)… i passed this morning the 70-646 (MCITP:SE) only to find it quite easy as well  with really a lot of questions having at most 2 possible answers out of 4, couple that with a light knowledge of topic makes it easy to guess…ok it wouldn’t get you in the 800/900 lines but it keeps underevaluating the whole thing i think….

    just to say that according to statistics i have a ‘cheater’ profile… i almost clear all examns with a max of an hour and a half time and almost got 900+ on every one of them…all in 8 months… but with a 10 years history in MS and UNIX/Linux IT activity, i have to admit that the most challenging exams of all those where the 2 last ones of the 2003 MCSE cursus… I still have the MCITP:EA,due next week and that’ll do it for next raise, untill the next one!

    I never considered Certifications (MS or other) very valuable until i started working with medium to large accounts… and i have to admit that it regularly gets in the way, discussing about hiring someone over someone else, so i decided to get to my HR and obtained the deal i got: ok, you got the skills, no doubts, you drive a 10 MCSEs/MCITPs team, but you are not yourself a MCSE… be one get more cash…

    so here i am boring everyone to death with my way-too-long-ego-filling-self-acclaiming history of CERTIFIED SMART AS* (trademarked by Trika)… happy caus i’m going to get my raise by the end of the month, WITHOUT CHEATING (i would have tried it, but i have no memory whatshoever…ask my wife, i constantly forget our anniversarys ๐Ÿ™‚ )

    just my two cents about statistics…i’d just drop in a "background history check" on reported highscorers… beeing fast and accurate doesn’t mean you do everything by memory! it can also mean you do it everyday at work ๐Ÿ™‚ or just have enough common sense to get the hesitations away from obvious out of the scope answers when they come accross..

    ok, i’m out!

    Pierre ( and sorry every one for that too long a comment)

  30. jtb says:

    When is our hostess going to return to blogging?

  31. LarryWestMCSD says:

    > When is our hostess going to return to blogging?

    She’s probably gone undercover. I heard that there are a lot of braindumps going on in Maui and she’s gone to personally investigate (or, at least, that’s where Microsoft should send her next – just to be sure the rumor’s false; heck, I’d be willing to go to "investigate").

    Or maybe she’s actually studying to take the 70-620 Vista exam.

    The third possibility is that the infamous Tick broke down. Will someone please investigate that van that’s been parked by the side of the freeway for two weeks now.

  32. Pierre says:

    I guess Trika’s out for a drinking-tanning-partying-summer spree!!! her well deserved holiday for making us, geeks, all happy at least twice a week ๐Ÿ™‚

    I sure hope it’s that reason, or the 70-620 one, lol, wich would be a well deserved work in progress achievement ๐Ÿ™‚ for the learning team.

    Anyway, just a post to tell you, Trika:

    WE MISS YOU!!!!

    WE ARE WORRIED ABOUT YOUR SILENCE,

    AND WE WANT YOUR POSTS BACK ๐Ÿ™‚

    Internet is just not the same without our international booth babe ๐Ÿ™‚

    any way, enjoy your offline time…

    Pierre.

  33. trikah@microsoft.com says:

    JTB, Larry, Pierre,

    I thought you’d never ask.

    Yours from the side of the road,

    Skinner (that’s my Tick name)