Empty Intersection: Provenance Coverage Rose to 98% and Neither Verification Decision Moved
- Source
- arxiv.org
- Author
- Dong Hyeon Jeon
- Date

Context
Better records of where data came from can leave a verification system without usable evidence. In one production deployment, expanding the categories used to describe data origins increased classified coverage from 36.1% to 98.4%. Yet all 121,296 records that gained a named origin fell outside the records consulted by two verification checks. Filtering those checks by origin grade changed their results from pass to undetermined. The study traced this mismatch using a frozen database snapshot.
The distinction matters because a system can accurately label its own generated values without obtaining an independent observation. Improving coverage across a database therefore needs a separate check: does the particular decision actually receive the evidence it requires?
- On a frozen production snapshot of 194,620 rows, the paper tests two provenance defenses, a grade on every row and a single write ingress that enforces it, against the two verification decisions the snapshot supports. Neither gave either decision admissible input.
- Widening the grade vocabulary raised classified coverage from 36.1% to 98.4%, and the single ingress refused 3,070 ungraded writes, yet the rows these fixes repaired never overlapped with the 32 rows the two decisions actually read.
- The author's diagnosis: both prescriptions are stated over the whole population and never name which rows a decision will read, so coverage metrics can climb to 98% while the verdicts that matter stay blocked.
- Filtering the verification queries by provenance grade turned both decisions from pass to undetermined. A companion paper had found the routines were deciding outcomes from values the system itself had written.
articleBeyond "Made with AI": Visualizing Provenance Density to Mitigate the Transparency PenaltyQing Zhang, Yifei Huang, Juyoung Lee, Thad Starner, Jun Rekimoto
articleSilent Success: A Release Gate That Passed on Checks It Never Ran, and Eight MoreDong Hyeon Jeon
articleThe reach of a verification tool decides its value: A controlled study of verification surface, artifact quality, and cost in AI coding agentsAchint Mehta
Checking sign-in…
Loading comments…