Confidence Intervals

A confidence interval is a range that would capture the true parameter in a stated fraction of repeated experiments.

Definition

A 95% confidence interval is constructed by a procedure that, across many repeated samples, contains the true parameter 95% of the time. The confidence level describes the long-run reliability of the method, not the probability for any single interval already computed.

The common form

Kronos motion — parameter scan

Many intervals take the shape estimate ± (critical value) × (standard error). For a mean with known σ the critical value is a normal quantile (1.96 for 95%); with σ estimated from a small sample it is a Student-t quantile, which is larger and widens the interval.

python

import statistics as st
x = [9.8, 10.1, 9.9, 10.3, 10.0, 9.7]
n = len(x); m = st.mean(x); s = st.stdev(x)
se = s/n**0.5
t = 2.571  # t critical, 5 df, 95%
print(round(m-t*se,3), round(m+t*se,3))

The correct interpretation

It is wrong to say 'there is a 95% probability the parameter lies in this specific interval'. The parameter is fixed; the interval is random. Before sampling, the procedure has a 95% success rate; after, the interval either contains the truth or it does not. The Bayesian credible interval is the object that does carry a direct probability statement.

Width and precision

Interval width scales with the standard error, so it shrinks like 1/√n. Narrowing an interval by half requires quadrupling the data. Wide intervals honestly signal that the data cannot pin the parameter down.

Relationship to testing

A value falls outside the 95% confidence interval exactly when a two-sided test rejects it at the 5% level. Intervals and tests are two views of the same inference, but the interval also conveys the magnitude and precision that a bare p-value hides.