Probability & Uncertainty
The mathematical language of AI: how probabilistic reasoning enables machines to make decisions under uncertainty and quantify confidence in their predictions.
Learning Objectives
- Understand why AI systems must reason under uncertainty
- Apply Bayes' theorem to update beliefs given new evidence
- Interpret probability distributions as representations of uncertainty
- Distinguish between different types of uncertainty (aleatoric vs epistemic)
- Connect probabilistic reasoning to modern ML predictions
Why Uncertainty is Fundamental
Intelligent systems operate in environments where information is incomplete, noisy, or ambiguous. Probability theory provides the mathematical framework for reasoning rationally about uncertain situations and making optimal decisions despite incomplete knowledge.
Consider a medical AI analyzing an X-ray for signs of pneumonia. The image is noisy, the disease presents differently in different patients, and some conditions look similar. The AI cannot be certain of its diagnosis—and it shouldn't pretend to be.
Instead, a well-designed system outputs something like "82% probability of pneumonia." This isn't a failure—it's honest reporting of uncertainty that helps doctors make better decisions. A confident wrong answer is worse than a calibrated uncertain one.
Mathematically, probability quantifies uncertainty on a scale from 0 (impossible) to 1 (certain). For an AI system:
- P(Y | X): The probability of output Y given input X. This is what classifiers learn to estimate.
- P(X): The prior probability of seeing input X—how common different inputs are in the world.
- P(Y): The marginal probability of output Y—how common each class is overall.
Neural network classifiers typically output a probability distribution over classes using a softmax function, where outputs sum to 1 and can be interpreted as confidence levels.
A spam filter doesn't just label emails "spam" or "not spam"—it estimates P(spam | email features). This lets users set their own threshold: aggressive filtering (flag anything >30% likely) vs. conservative (only flag >90% likely).
Language models don't just predict the next word—they produce a probability distribution over all possible words. Sampling from this distribution (with varying "temperature") creates diverse, creative outputs.
Bayes' Theorem: Updating Beliefs
The Heart of Probabilistic Reasoning
Bayes' theorem tells us how to update our beliefs when we receive new evidence. It's not just a formula—it's a normative rule for rational belief revision.
Example: Medical Diagnosis
A disease affects 1% of the population. A test is 90% accurate (detects disease when present 90% of the time) with a 5% false positive rate. You test positive. What's the probability you have the disease?
Given:
- P(Disease) = 0.01 (prior: 1% have it)
- P(Positive | Disease) = 0.90 (sensitivity)
- P(Positive | No Disease) = 0.05 (false positive rate)
Calculate P(Positive):
= P(Pos | Disease) × P(Disease) + P(Pos | No Disease) × P(No Disease)
= 0.90 × 0.01 + 0.05 × 0.99 = 0.009 + 0.0495 = 0.0585
Apply Bayes:
P(Disease | Positive) = (0.90 × 0.01) / 0.0585 ≈ 15.4%
Surprising result: Despite a positive test, you probably don't have the disease! This is because false positives from the 99% healthy population outnumber true positives from the 1% sick population. The base rate matters enormously.
This example illustrates why Bayesian reasoning is crucial: naive interpretation of test results leads to wildly incorrect conclusions. AI systems trained on imbalanced data face the same challenge—the prior distribution of classes fundamentally affects what predictions are rational.
Types of Uncertainty
Not all uncertainty is the same. Understanding the source of uncertainty helps us know whether it can be reduced and how to handle it appropriately.
Aleatoric Uncertainty
Irreducible randomness in the world
This uncertainty comes from inherent randomness or variability in the process being modeled. No amount of additional data can eliminate it.
- Noise in sensor measurements
- Inherent ambiguity in language ("bank" = financial institution or riverbank?)
- Quantum effects, chaotic systems
- Two patients with identical symptoms can have different outcomes
Handling: Model the distribution of outcomes, not just point predictions. Report confidence intervals.
Epistemic Uncertainty
Uncertainty from limited knowledge
This uncertainty stems from gaps in our knowledge—things we could know but don't yet. It can be reduced with more data or better models.
- Model hasn't seen enough examples of this type
- Input is far from training distribution
- Model architecture can't capture the true pattern
- Conflicting evidence in training data
Handling: Use ensemble methods, Bayesian neural networks, or techniques that detect out-of-distribution inputs. Flag high-uncertainty cases for human review.
Why the Distinction Matters
A self-driving car should behave differently when uncertain because the road markings are faded (aleatoric—drive cautiously) versus uncertain because it's never seen this type of intersection before (epistemic—maybe stop and alert).
Probability Distributions in ML
Machine learning models don't just output answers—they (implicitly or explicitly) define probability distributions over possible outputs.
Softmax and Classification
For a classification task with K classes, neural networks typically output K numbers that get transformed through the softmax function:
This ensures outputs sum to 1 and can be interpreted as probabilities. If a model outputs [0.75, 0.20, 0.05] for three classes, it's saying "75% confident class 1, 20% class 2, 5% class 3."
Caution: Softmax outputs are often overconfident. A model might output 99% confidence while being wrong. Calibration techniques help align stated confidence with actual accuracy.
Language Models as Probability Distributions
A language model like GPT defines a probability distribution over all possible next tokens. Given context "The cat sat on the", it might assign:
- mat35%
- floor25%
- couch15%
- bed10%
- table8%
- ...7%
Text generation works by sampling from this distribution repeatedly. The "temperature" parameter controls randomness: low temperature → always pick highest probability (deterministic), high temperature → more random exploration.
Common Misconceptions
"High confidence means the model is probably right"
Neural networks are often miscalibrated—they can output 95% confidence while being wrong 30% of the time. High confidence only means the model found strong patterns; it doesn't guarantee those patterns are valid for this input.
The accurate framing: Confidence scores require calibration to be meaningful. A well-calibrated model should be correct 90% of the time when it says 90% confident. Modern techniques like temperature scaling help achieve this.
"Probabilities are subjective opinions"
While there are debates about probability interpretation, in ML, probabilities have precise operational meaning: they predict long-run frequencies. If a model is well-calibrated and says "80% chance of rain," then among all days where it said 80%, about 80% should actually have rain.
The accurate framing: Probabilities in ML are empirically verifiable predictions about frequencies. They can be tested against reality and should match observed outcomes when the model is properly calibrated.
"Uncertainty is a weakness to be minimized"
This assumes certainty is always better. But expressing appropriate uncertainty is often more valuable than false confidence. A doctor wants a model that says "I'm uncertain—get more tests" rather than one that confidently wrong-diagnoses.
The accurate framing: Good uncertainty quantification is a feature, not a bug. Systems should be uncertain when evidence is ambiguous and confident when it's clear. The goal is calibrated uncertainty—being uncertain about the right things.
Interactive Lab: Bayesian Reasoning
Explore how prior beliefs and evidence combine through Bayes' theorem. Adjust the parameters and watch how the posterior probability changes.
Scenario: Disease Testing
Configure the test parameters and see how Bayes' theorem determines the true probability.
The Math
A probability distribution shows how likely different outcomes are. Adjust parameters to see how distributions change shape.
A well-calibrated model should be right X% of the time when it predicts X% confidence. Test whether different "models" are well-calibrated.
Is this image a cat?
Results After 0 Trials
Key Takeaways
- Base rates matter: Even accurate tests can give misleading results when the prior probability is very low or very high.
- Distributions encode uncertainty: A narrow distribution means high confidence; a wide distribution means uncertainty.
- Calibration is testable: We can empirically verify whether a model's confidence scores match its actual accuracy.
Check Your Understanding
In Bayes' theorem, what does the "prior" represent?
What is the key difference between aleatoric and epistemic uncertainty?
A disease affects 0.1% of the population. A test has 99% sensitivity and 1% false positive rate. If you test positive, the probability you have the disease is approximately:
What does it mean for a classifier to be "well-calibrated"?
In language models, what role does "temperature" play during text generation?