Probability underpins AI, cryptography and statistics. However, as the philosopher Bertrand Russell said, “Probability is the most important concept in modern science, especially as nobody has the slightest notion what it means.”
I teach statistics to engineers, so I know that while probability is important, it is counterintuitive.
Probability is a branch of mathematics that describes randomness. When scientists describe randomness, they’re describing chance events – like a coin flip – not strange occurrences, like a person dressed as a zebra. While scientists do not have a way to predict strange occurrences, probability does predict long-run behavior – that is, the trends that emerge from many repeated events.
We may say ‘random’ to describe strange occurrences (person dressed as zebra), but probability describes chance events (a coin flip).
Zebras in La Paz, Bolivia by EEJCC, Own Work CC A-SA 4.0; https://commons.wikimedia.org/wiki/File:Zebra_La_Paz.jpg _ , CC BY-SA
Modeling with probability
Since probability is about events, a scientist must choose which events to study. This choice defines the sample space. When flipping a coin, for example, you might define your event as the way it lands.
Coins almost always land on heads or tails. However, it’s possible – if very unlikely – for a coin to land on its side. So to create a sample space, you’d have two choices: heads and tails, or heads, tails and side. For now, ignore the side landings and use heads and tails as our sample space.
Next, you would assign probabilities to the events. Probability describes the rate of occurrence of an event and takes values between 0% and 100%. For example, a fair flip will tend to land 50% heads up and 50% tails up.
To assign probabilities, however, you need to think carefully about the scenario. What if the person flipping the coin is a cheater? There’s a sneaky technique to “wobble” the coin without flipping, controlling the outcome. Even if you can prevent cheating, real coin flips are slightly more probable to land on their starting face – so if you start the flip with the coin heads up, it’s very slightly more likely to land heads up.
In both the cheating and real flip cases, you need an appropriate sample space: starting face and other face. To have a fair flip in the real world, you’d need an additional step where you randomly – with equal probability – choose the starting face, then flip the coin.
The probabilities for different coin-flipping scenarios.
Zachary del Rosario, CC BY-SA
These assumptions add up quickly. To have a fair flip, you had to ignore side landings, assume no one is cheating, and assume the starting face is evenly random. Together, these assumptions constitute a model for the coin flip with random outcomes. Probability tells us about the long-run behavior of a random model. In the case of the coin…
