Copilot Confidence and Uncertainty

Copilots report calibrated uncertainty drawn from twin UQ, retrieval quality, and self-consistency — so operators know how much to trust each answer.

Uncertainty is part of the answer

A copilot answer without uncertainty is incomplete. Kronos copilots attach a calibrated confidence to every substantive claim and proposal, aggregated from three sources: the twin's own uncertainty quantification on any predicted quantity, the quality and agreement of the retrieved evidence, and the self-consistency of the copilot's reasoning across independent samples.

Sources of uncertainty

• Twin UQ: predictive bands on tau_E, margins, DEC efficiency, etc.
• Retrieval quality: rerank scores, evidence agreement, coverage
• Self-consistency: agreement across sampled reasoning traces
• Calibration state: how well the copilot's stated confidence has matched reality

confidence(claim):
  u_twin  = twin.uq(claim.predicted_quantity)     # if physical
  u_ret   = retrieval_quality(claim.sources)      # score + agreement
  u_self  = self_consistency(claim, n_samples)    # trace agreement
  c = combine(u_twin, u_ret, u_self)
  return recalibrate(c)     # via reliability curve from golden set

Calibration, not vibes

Stated confidence is only useful if it is calibrated — if answers the copilot calls 90% confident are right about 90% of the time. Calibration is measured against the golden set: predicted confidence is binned against realized correctness to build a reliability curve, and a miscalibrated copilot's confidences are recalibrated or the release is blocked. The evaluation harness reports calibration error alongside correctness.

Confidence drives behavior, not just display. Low-confidence proposals are flagged for extra scrutiny in human oversight, and below a threshold the copilot refuses or escalates rather than answering. This links uncertainty directly to safety: the copilot is most cautious exactly where it knows least.

Physical uncertainty is inherited honestly from the twin rather than manufactured. Because both machines are pre-FOAK, the twin's own uncertainty is often wide, and the copilots reflect that width rather than projecting false precision — consistent with making no net-gain claim before the machine demonstrates it. See twin confidence scoring and uncertainty quantification.