Probability Basics

calendarDec 12, 2024 · 2 min read · probability statistics fundamentals  ·
Share on: linkedincopy

Visual guide to probability fundamentals and axioms.

Axioms of Probability

Axiom 1: Non-Negativity

$$0 \leq P(A) \leq 1 \text{ for any event } A$$

Probabilities are always between 0 (impossible) and 1 (certain).

Axiom 1: Probability Range [0, 1]0.0Impossible0.5Equally Likely1.0CertainImpossibleUnlikely (0.25)50-50 (0.5)Likely (0.75)Certain

Axiom 2: Certainty

$$P(\Omega) = 1$$

The probability of the entire sample space (all possible outcomes) is 1.

Axiom 2: Total Probability = 1Ω (Sample Space)P(Ω) = 1ABC

All events A, B, C, ... are subsets of ΩP(A) + P(B) + P(C) + ... = P(Ω) = 1

Axiom 3: Additivity

$$P(A \cup B) = P(A) + P(B) \text{ for mutually exclusive events}$$

For events that cannot occur simultaneously, probabilities add.

Note: When events don't overlap (mutually exclusive), their probabilities add directly.

Overlapping Events (General Addition Rule)

For events that can occur together:

$$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$

Why subtract? The intersection $P(A \cap B)$ is counted twice when we add $P(A) + P(B)$, so we must subtract it once to avoid double-counting.

Conditional Probability

$$ P(A|B) = \frac{P(A \cap B)}{P(B)} $$

Meaning: Probability of $A$ happening, given that $B$ has already occurred.

Visual Explanation

Intuition: Once we know $B$ occurred, $B$ becomes our new "universe". We ask: what fraction of $B$ also contains $A$?

Step-by-Step Process

Proportional Area Representation

Key Insight: Among the 60% of outcomes where $B$ occurs, the intersection $A \cap B$ represents 20% of all outcomes. Therefore:

$$ P(A|B) = \frac{\text{intersection size}}{\text{condition size}} = \frac{0.2}{0.6} = \frac{1}{3} $$

This means 33.3% of event B also contains event A.

Real-World Example

Scenario: Drawing cards from a deck

  • $P(\text{King}) = \frac{4}{52}$
  • $P(\text{King | Heart}) = \frac{1}{13}$

Given that we drew a heart (13 cards), only 1 of those 13 is a king.

Independence

Events $A$ and $B$ are independent if:

$$ P(A \cap B) = P(A) \cdot P(B) $$

Or equivalently: $P(A|B) = P(A)$

Law of Total Probability

$$ P(A) = \sum_i P(A|B_i) P(B_i) $$

Bayes' Theorem

$$ P(A|B) = \frac{P(B|A) P(A)}{P(B)} $$

Python Examples

 1import numpy as np
 2
 3# Simulate coin flips
 4flips = np.random.choice(['H', 'T'], size=1000)
 5p_heads = np.mean(flips == 'H')
 6
 7# Conditional probability
 8# P(A|B) = P(A and B) / P(B)
 9def conditional_prob(data, event_a, event_b):
10    p_b = np.mean(event_b(data))
11    p_a_and_b = np.mean(event_a(data) & event_b(data))
12    return p_a_and_b / p_b if p_b > 0 else 0

Further Reading

Related Snippets