Bayesian Probability

From ForcaWiki, the simple encyclopedia

Imagine a detective working on a case. At the very start, before finding any clues, the detective might have a rough hunch about who did it. Then a new clue turns up, a muddy footprint, say, and the detective updates their suspicion: maybe the hunch gets stronger, or maybe it points somewhere else entirely. Each new clue nudges the belief a little, without ever throwing out everything learned before.

Bayesian probability is the math version of that detective work. Instead of only asking "how likely is this to happen," the way the topic of Probability usually does with something like a coin flip, it asks "how confident should I be in this belief right now, given everything I have learned so far," and it gives you an exact way to update that confidence whenever fresh evidence shows up.

Starting guess, new evidence, updated guess

Bayesian probability always works in three steps:

1. Start with a "prior," your best guess about how likely something is before you have the new evidence.
2. Gather some evidence.
3. Combine the prior with the evidence to get a "posterior," your updated, more informed guess.

The clever part is that the posterior does not throw away the prior. It blends the old guess with the new evidence in a precise way, using a rule called Bayes' theorem. The next time more evidence arrives, today's posterior simply becomes tomorrow's prior, and the whole process repeats.

A surprising example: a positive test result

Here is a case where Bayesian thinking reveals something that catches people by surprise. Imagine a rare illness that affects only 1 out of every 100 people, and a medical test for it that is correct 90 percent of the time, for both sick and healthy people. If you test positive, how likely is it that you actually have the illness?

Picture testing 1,000 people:

• About 10 of them actually have the illness. The test correctly catches 9 of those 10.
• About 990 people do not have the illness. But the test is only 90 percent accurate, so it wrongly flags 99 of those healthy people as positive too.
• That means 108 people total test positive, but only 9 of them are actually sick.

So even with a positive result, the real chance you are sick is only about 9 out of 108, roughly 8 percent, not anywhere near 90 percent. The test did make you far more worried than before, since your chances jumped from 1 percent up to about 8 percent, but the illness started out so rare that one positive test is still not enough to be sure. A doctor would likely order a second test to update the guess again, pushing that confidence higher still.

Why this matters

Bayesian probability is not just for tricky puzzles. It is a whole way of thinking about uncertainty that updates smoothly as new evidence rolls in, instead of starting from scratch every time. It shows up in:

• Email programs that guess whether a new message is spam, based on which words tend to appear in spam messages you have seen before.
• Doctors weighing test results against how common or rare a disease already is.
• Weather forecasts that update hour by hour as new radar data comes in.
• Self driving cars and robots that constantly update their best guess about where nearby objects are as new sensor readings arrive.

Fun facts

• The idea is named after Thomas Bayes, an English minister and mathematician from the 1700s. His big idea was not even published while he was alive. A friend, Richard Price, found his notes after he died and published them in 1763.
• Some of the earliest spam filters that cleaned up email inboxes worked almost entirely on this kind of reasoning, guessing "spam" or "not spam" and updating their guess as they saw more mail.
• Bayesian thinking is really just a more careful version of something people do naturally. If a friend who is almost always right tells you a wild rumor, you probably believe it more than if an unreliable friend told you the exact same rumor, because you are quietly weighing the new evidence against what you already expect.