gammacert: Γ-robust bounds for strategic AI oversight

A silent monitor is not a safety proof

AI developers monitor deployed models by sending a sample of traffic to another model, and a record of zero or few flags is read as evidence that harmful behavior is rare. That inference assumes the monitored system cannot tell where oversight is weak. A capable system can: the features that set monitoring intensity are part of the context it acts on. We give confidence bounds on the number of harmful actions that stay valid when the system can see the stratification, and show how to design monitoring so that knowing the policy gives little advantage.

Certificate against strategic concentration and measured recall by stratum — Left: five tiers with the sampling rates of a published risk report. A silent record rules out about 3 harmful actions if harm follows traffic, and 244 if harm may be twice over-represented in weakly monitored traffic; equal effective coverage at the same budget certifies 8.3 for every Γ. Right: one open-weight monitor on 16 attack strata from three public benchmarks.

2.6 → 244

bound after zero detections when harm may be 2× over-represented in weak strata

8.3 for every Γ

same budget, equal effective coverage

0 to 0.92

recall of one monitor across 16 attack strata

19 vs 18,723

bounds under two signals with the same AUC of 0.55

Certificate calculator

Edit the strata, then move Γ: harmful actions may be at most Γ times over-represented in any stratum relative to its traffic share. The chart compares your design with equal effective coverage at the same monitored fraction.

Preset:

stratum	traffic share	routing π	recall r	coverage c = πr

confidence detections

–

bound on harmful actions, your design

–

same budget, equal coverage

Γ1.00

your designequal coverage, same budget

Zero detections: $U_\alpha(0;\Gamma)=\log(1/\alpha)/\mathrm{LCVaR}_{1/\Gamma}(\kappa)$ with $\kappa_h=-\log(1-c_h)$. With $d\ge1$ detections the bound is $\lambda_\alpha(d)/\mathrm{LCVaR}_{1/\Gamma}(c)$, where $\lambda_\alpha(d)$ is the exact Poisson upper limit. The same numbers come from gammacert.certificate(pi, recall, shares, gamma, detections).

The idea

A harmful unit in stratum $h$ is detected with probability $c_h=\pi_h r_h$: it must be routed to the monitor and recognized. A strategic system can move harm toward strata where $c_h$ is small. If harm may be at most $\Gamma$ times over-represented anywhere, a silent record certifies

$$U_\alpha(0;\Gamma)=\frac{\log(1/\alpha)}{\mathrm{LCVaR}_{1/\Gamma}(\kappa)},$$

the detection hazard averaged over the least-covered fraction $1/\Gamma$ of traffic. The bound needs only the monitor's own randomization, no model of the system's behavior.

Partial visibility. When the system only partly infers the policy, the certificate depends on the extreme lower tail of its posterior detection probability. AUC, guessing accuracy and every summary that is Lipschitz in total variation cannot bound that tail.

Robust design. Equalizing effective coverage is minimax optimal once Γ is large enough, and a hidden randomized floor $\pi_0\ge(1-\alpha^{1/K})/r_{\min}$ certifies at most $K$ harmful actions whatever the system knows.

Campaigns. Subsampling cannot rule out one catastrophic action unless coverage is nearly complete, but it can bound the probability that a multi-step campaign escapes interruption.

Measurements

We scored 4,058 transcripts from APPS backdoors, BashArena and SLEIGHT-Bench with seventeen open-weight monitor configurations.

Tail stress test — A model asked which attacks the monitor will miss (AUC 0.58) finds a 1% tail with recall 0.125; a probe with higher AUC finds a weaker tail.

Five designs at the same monitored fraction on measured recall bounds: risk-targeted routing certifies nothing at Γ = 5, equal coverage stays below 8.8.

Validity — 510,390 exact coverage checks: never above α, and the exact Poisson constant uses ≥ 99.95% of it.

BibTeX

@software{gammacert2026,
  title  = {gammacert: Γ-robust bounds for strategic AI oversight},
  author = {{The gammacert authors}},
  year   = {2026},
  url    = {https://github.com/ziwaa-se/gammacert}
}