Risk analysis and estimation...

I hadn't planned on writing a followup on my risk post, but one comment compelled me to write some more. I hope it isn't too preachy.

J writes:

My girlfriend is a doctor. She deals with an inordinate number of motorcyclists every day, from every walk of life. The actual level of risk appears to be very high from the evidence I've seen.

n the final statement, J is asserting that the risk is very high, and he's also asserting that he's basing his conclusion on evidence. There's also an implicit assertion that his conclusion applies to me.

I believe that is an unsupported assertion, and I'd like to discuss why.

The branch of science that deals with risk analysis is know as epidemiology. To pull a definition from a state website:

"Epidemiology is the study of the occurrence and distribution of illnesses and injuries in human populations."

If you read a research paper that says something like, “smoking increases your chance of lung cancer by 45%“, chances are that the work was done by an epidemiologist.

The reason that we do studies in the first place is that it's well known that drawing conclusions from individual experience is often problematic. The whole point of the scientific method is to approach things in an objective and repeatable manner (which doesn't always happen, but that's another subject).

Just to pick a few of the factors that could invalidate J's conclusion (which I'll restate as “Eric shouldn't ride motorcycles because my girlfriend the doctor see's lots of injured motorcyclists“) (and I should note that I'm not asserting that any of these factors are valid or invalid, just listing what are possible sources of error):

  1. The doctor may be filtering based on preconception of motorcyclists being more risky
  2. The doctor may be giving a summary that is weighted towards more injuries than actually occur
  3. The hospital may get a higher percentage of motorcycle injuries than other hospitals in the area.
  4. The motorcyclists in the region may ride more miles than in other regions.
  5. The region around the hospital may have a higher percentage of motorcyclists compared to a larger population
  6. The average motorcyclist age in the region may be younger
  7. The region may have less effective motorcycle safety training programs
  8. The region may have motorcycle dealers who focus on selling more powerful motorcycles
  9. There may be motorcycle clubs who increase the incidence of risky behavior
  10. Law enforcement may not be enforcing against risky behavior.

Because of factors such as these, anecdotal data isn't very useful to draw conclusions from (though it is useful to come up with good research topics). You need a real study that looks at motorcycle injuries, looks at the causes of those injuries, looks at the demographics of the situation, and then does some analysis to identify correlations between risk factors and injuries.

(Aside: Epidemilology is a requirement when you're looking at low-level effects, such as environmental cancer rates, danger from EMF radiation, etc. If not, you can't separate clustering due to random distribution (which is by definition not uniform) from a real effect))

So, once you've done that research, you should have a lot of information saying how much more likely motorcyclists are to be injured or killed based on a number of factors. And then you can apply those risk factors to a specific situation, and come up with an estimate on how risky an activity is compared to a different activity.

The literature isn't great in this area, and more study is needed. Many motorcycle accidents come from the actions of other drivers, but I don't know of a recent good study in that area. On single-vehicle accidents, we have the following:

This study states that on a per-mile basis, a motorcyclist is 3x as likely to be injured, and 16x as likely to be killed. But it also lists some risk factors (look to the study for all of them):

Helmet use among fatally injured motorcyclists below 50 percent

High blood alcohol levels are a major problem among motorcycle operators

Almost two thirds of the fatalities were associated with speeding as an operator contributing factor in the crash

Almost 60 percent of motorcyclist fatalities occur at night

Braking and steering maneuvers possibly contribute for almost 25 percent of the fatalities

Almost one third of the fatally injured operators did not have a proper license

I always wear a helmet. I don't drink before I ride. I rarely ride at night. I'm well trained and understand how to properly brake and steer, and I practice a bunch (or I did when I rode more). I have a proper license.

So I don't have a lot of the risk factors that the accident-involved motorcyclist has, so it's unlikely that the 3x and 16x factors apply to my risk as compared to the general population.

Of course, my risk as a car driver is also lower because of good habits, so the relative factors could be the same, or could even be worse.

So, what's my point in all of this? Well, two things.

The first is that many people make the mistake of assessing risk based upon the overall societal attitude towards a activity and/or anecdotal data (“I knew a guy who...“) rather than any factual basis. If you to to an emergency room with a leg injury, you will get a different response based on whether you say you were skiing or skydiving.

My second point is that it takes a good study to tell you how you can reduce your risk, so being better informed really pays off.