Bayes' Theorem

From ForcaWiki, the simple encyclopedia

Here is a question that trips people up: if you know how likely B is when A is true, does that also tell you how likely A is when B is true? It feels like it should, but it does not. Knowing "almost every dog barks" does not mean "almost everything that barks is a dog," since cars, seals, and people can bark too. Bayes' theorem is the exact rule mathematicians use to correctly flip a chance like that around, turning "how likely is B, given A" into "how likely is A, given B," without guessing.

A worked example: two bags of marbles

Suppose you have two bags. Bag A has 8 red marbles and 2 blue ones. Bag B has 3 red marbles and 7 blue ones. You pick one of the two bags completely at random, so each bag has a 50 percent chance, then you pull out one marble without looking. It is red. Which bag do you think you probably grabbed?

Picture doing this 100 times to see the pattern:

• About 50 times you grab Bag A, and since 8 out of 10 of its marbles are red, you would expect around 40 red marbles from those picks.
• About 50 times you grab Bag B, and since only 3 out of 10 of its marbles are red, you would expect around 15 red marbles from those picks.
• Altogether, that is about 55 red marbles out of 100 tries, and 40 of those 55 came from Bag A.

So if you drew a red marble, the chance you had actually grabbed Bag A is about 40 out of 55, which is roughly 73 percent. Notice this is very different from the 80 percent chance of "red, given Bag A" that we started with. Bayes' theorem is precisely the tool that turns that starting number into this new, flipped around answer.

The rule itself

Written out as a small formula, Bayes' theorem says:

chance of A, given B = (chance of B, given A) x (chance of A) / (chance of B)

Matching that to the marbles: the chance of A is 50 percent, the chance of "red, given Bag A" is 80 percent, and the overall chance of red, given either bag, is the 55 percent we counted. Multiply and divide those three numbers together and you land on the same 73 percent as before. The formula is just a shortcut for the careful counting we already did.

Why it matters

Bayes' theorem is the precise engine underneath Bayesian Probability, which is the broader idea of updating a guess as new evidence comes in. Anywhere someone needs to flip "how likely is this evidence, if my guess is true" into "how likely is my guess, given this evidence," Bayes' theorem is doing the work behind the scenes, whether that is a doctor reading a test result, a spam filter reading an email, or a detective reading a clue.

Fun facts

• The theorem is named after Thomas Bayes, an English minister and mathematician. He never published the idea himself. His friend Richard Price found it among his papers after he died and published it in 1763.
• A French mathematician named Pierre-Simon Laplace discovered much the same rule on his own a few decades later and used it far more widely, so some very old books actually call it Laplace's rule instead.
• People use a rough version of Bayes' theorem all the time without realizing it. If your normally reliable smoke detector suddenly beeps, you get far more worried than if a detector that beeps constantly for no reason goes off, because you are quietly weighing the new evidence against how trustworthy the detector already was.