Statistics

Simple terms

Apex treats statistics as risk management for business decisions. These pages explain the methods, the limits, and the simulations used to verify them.

A confidence number is useful only if it keeps its meaning under the way people actually use the product. Apex therefore tests its statistical decision code under repeated peeking, low and high traffic, multiple variants, and safety-trigger scenarios.

Read the methodology

• How Apex statistics work explains confidence thresholds, one-sided decisions, repeated peeking, parallel experiments, Guardian, sample-ratio checks, and the 68-cell certification run.
• Audit of common Bayesian stopping rules shows what published “chance to beat” and expected-loss rules did in deterministic A/A simulations.
• Audit of bandit campaign-mode allocation measures what adaptive traffic allocation earns in 1.15 million simulated merchant campaigns, and what it does to the reported lift.

What is live today

• The live winner verdict uses a sequential method designed for repeated looks at accumulating data. New experiments default to an 80% winner threshold, configurable from 80% to 99%.
• Apex Guardian uses a stricter 99% harm threshold for client-facing safety actions.
• Revenue significance remains gated while its inference path is rebuilt and re-certified.
• Sample-ratio mismatch and other trust checks can suppress a verdict when the underlying allocation is not trustworthy.

Recent changes

• The unlinked /experiments/statistics dashboard route and its orphaned components were removed in the release at f2ac995 (owner decision 2026-08-27). This repo change deleted the UI route and related components; the methodology, certification, and audit documentation remain as the canonical public explanation of Apex's statistical approach. (See release delta: .release-delta.json.)

What the evidence covers

The Apex certification ran more than three million simulated experiments through the production decision functions. The same fixed seeds and engine checkout reproduce the measured statistical fields byte-for-byte. The public methodology page includes the complete rollup, including the surfaces that failed their budgets.