Null Hypothesis Testing

calendarDec 12, 2024 · 3 min read · statistics hypothesis-testing null-hypothesis significance  ·
Share on: linkedincopy

Visual guide to null hypothesis testing and the scientific method of statistical inference.

What is a Null Hypothesis?

Null Hypothesis (H₀): A statement of "no effect" or "no difference" that we test against.

Alternative Hypothesis (H₁ or Hₐ): What we suspect might be true instead.

Visual Representation

Error Types

Reality →H₀ is TrueH₁ is True
Reject H₀❌ Type I Error
(False Positive)
Probability = α		✅ Correct Decision
(True Positive)
Probability = Power
Fail to Reject H₀✅ Correct Decision
(True Negative)
Probability = 1 - α		❌ Type II Error
(False Negative)
Probability = β

Key Points:

  • Type I Error (α): Rejecting H₀ when it's actually true (false alarm)
  • Type II Error (β): Failing to reject H₀ when H₁ is true (missed detection)
  • Power (1 - β): Probability of correctly detecting an effect when it exists

Types of Hypotheses

Two-Tailed Test

$$ \begin{aligned} H_0 &: \mu = \mu_0 \ H_1 &: \mu \neq \mu_0 \end{aligned} $$

Critical regions in both tails:

  • Reject H₀ if test statistic is too far from center in either direction
  • α is split between both tails (α/2 each)
  • For α = 0.05: critical values at ±1.96 standard deviations

One-Tailed Test (Right)

$$ \begin{aligned} H_0 &: \mu \leq \mu_0 \ H_1 &: \mu > \mu_0 \end{aligned} $$

One-Tailed Test (Left)

$$ \begin{aligned} H_0 &: \mu \geq \mu_0 \ H_1 &: \mu < \mu_0 \end{aligned} $$

Decision Making

See the error types table above for the complete decision matrix.

Common Examples

Example 1: Drug Testing

1H₀: Drug has no effect (μ_drug = μ_placebo)
2H₁: Drug has an effect (μ_drug ≠ μ_placebo)
3
4Type I Error: Approve ineffective drug
5Type II Error: Reject effective drug

Example 2: Quality Control

1H₀: Product meets specifications
2H₁: Product is defective
3
4Type I Error: Reject good product
5Type II Error: Accept defective product

Example 3: Criminal Trial

1H₀: Defendant is innocent
2H₁: Defendant is guilty
3
4Type I Error: Convict innocent person
5Type II Error: Acquit guilty person

Power Analysis

Statistical Power = Probability of correctly rejecting H₀ when H₁ is true

$$ \text{Power} = 1 - \beta $$

Steps in Hypothesis Testing

  1. State hypotheses: Define H₀ and H₁
  2. Choose significance level: Usually α = 0.05
  3. Select test: t-test, z-test, chi-square, etc.
  4. Calculate test statistic: From sample data
  5. Find p-value: Probability of observing data under H₀
  6. Make decision:
    • If p < α: Reject H₀
    • If p ≥ α: Fail to reject H₀
  7. Interpret: In context of original question

Key Concepts

Significance Level (α)

  • Probability of Type I error
  • Typically 0.05, 0.01, or 0.001
  • Set before collecting data

Confidence Level

$$ \text{Confidence Level} = 1 - \alpha $$

  • 95% confidence → α = 0.05
  • 99% confidence → α = 0.01

Critical Region

  • Range of test statistic values that lead to rejecting H₀
  • Determined by α and test type (one/two-tailed)

Common Pitfalls

  1. Confusing "fail to reject" with "accept"

    • We never "prove" H₀ is true
    • We only have insufficient evidence against it

  2. P-hacking

    • Testing multiple hypotheses until finding p < 0.05
    • Use multiple comparison corrections

  3. Ignoring effect size

    • Statistical significance ≠ practical importance
    • Always report effect sizes

  4. One-sided vs two-sided

    • Choose before seeing data
    • One-sided has more power but is less conservative

Further Reading

Related Snippets