Throughout our worldwide MEDC events, one of the hardest thing for us to judge is the difference between speaker ratings between the 3 regions. US, EMEA and Asia. It seems there is a disparity between the 3 and I believe some of them are due to cultural and psychological differences.
The US audience tend to be more critical and honest about giving speaker feedback and if a speaker is good, the US audience tend to reward the speaker with 9s. If the speaker is bad, the feedback can be really brutal. So on a scale of 1 - 9, most US audience will utilize almost every value within that range. Perhaps the only black hole numbers (numbers that are hardly used in speaker ratings) are 3 - 5. If you're bad below 6 you'll automatically drop to 2 or 1.
In other parts of the world, audience tend to be more reserved the further out you reach from the US. Maybe because many people are hesitant to answer 9 in feedback forms because of the word "Very Satisfied". If you take a step back and think about it, most US Audience already know that 9 means very good score, Kudos!, Great Job!. But for someone who was not brought up with English as the first language, giving a "Very Satisfied" rating means a whole lot more. Meaning they would have to be blown away totally. I might be wrong, but that is a psychologically barrier that makes the rating different worldwide. If a session is good, I'll give an 8.
So let's take Mike Hall for instance (since he has the most consistent feedback we've seen so far). His session in the Las Vegas event rated 8.33 out of a possible 9.0. Which in the US terms and according to peers, is a fantastic rating. He did not get top spot but nonetheless, an exceptional score. The 2nd session he presented had an 8.32 score (consistent).
In Europe, his score was 8.02 for a similar session and 7.84 for his next session. He also presented in Malaysia and he attained 7.57 and 7.58 session scores. I've also compared session scores between Taiwan, China, Japan and Malaysia and they're similar enough that we can lump them into one region (ASIA). So I've built a simple table to illustrate the difference between regions.
|Mike Hall 1||8.33||8.02||7.57||0.31||0.76|
|Mike Hall 2||8.32||7.84||7.58||0.48||0.74|
As you can see, the difference between regions are also consistent. Obviously there are many other moving factors that you have to take into account such as type of session delivered, appropriateness of content etc.
So this is my personal attempt at establishing a Speaker Rating Handicap. I believe that in order to compare speaker ratings across the world (in this case 3 main regions, US, EU and Asia), you can apply Regional Modifiers or Handicap especially for US speakers travelling to other countries to present or if you need to compare how good a speaker is compared to local ones. This is my guideline:
|Modifier for US to EMEA||0.40|
|Modifier for US to ASIA||0.75|
|Modifier for EMEA To ASIA||0.35|
I know I did not go very deep and used a lot of average (I suck at stats in school), but this is a simplified way of being able to determine how good a speaker is across regions. I will not use this in a competition though, unless someone can help me figure this out in a more accurate way.