Vibeleaderboard
← All Intel
Intel / article

Empty Intersection: Provenance Coverage Rose to 98% and Neither Verification Decision Moved

Source
arxiv.org
Author
Dong Hyeon Jeon
Date

Context

Better records of where data came from can leave a verification system without usable evidence. In one production deployment, expanding the categories used to describe data origins increased classified coverage from 36.1% to 98.4%. Yet all 121,296 records that gained a named origin fell outside the records consulted by two verification checks. Filtering those checks by origin grade changed their results from pass to undetermined. The study traced this mismatch using a frozen database snapshot.

The distinction matters because a system can accurately label its own generated values without obtaining an independent observation. Improving coverage across a database therefore needs a separate check: does the particular decision actually receive the evidence it requires?

Key takeaways · AI-distilled
  • On a frozen production snapshot of 194,620 rows, the paper tests two provenance defenses, a grade on every row and a single write ingress that enforces it, against the two verification decisions the snapshot supports. Neither gave either decision admissible input.
  • Widening the grade vocabulary raised classified coverage from 36.1% to 98.4%, and the single ingress refused 3,070 ungraded writes, yet the rows these fixes repaired never overlapped with the 32 rows the two decisions actually read.
  • The author's diagnosis: both prescriptions are stated over the whole population and never name which rows a decision will read, so coverage metrics can climb to 98% while the verdicts that matter stay blocked.
  • Filtering the verification queries by provenance grade turned both decisions from pass to undetermined. A companion paper had found the routines were deciding outcomes from values the system itself had written.
More from Dong Hyeon Jeon
Recommended reads
Comments

Checking sign-in…

Loading comments…