Why "100% caught" is a lie, and what an honest collision detector promises instead

If you are shopping for a tool that warns you when two pull requests may need coordination before merge, sooner or later you will see a vendor claim that sounds something like "catches 100% of cross-PR conflicts" or "never miss a collision again" or "guarantees a clean main."

This post is about why those claims are, with very high probability, wrong - not because the vendor is unusually dishonest, but because the math of the problem rules them out. And it is about the five honest promises you should ask for instead, so that the tool you buy can be evaluated on what it actually does.

The math: false negatives and false positives are baked in

A pre-merge cross-PR collision detector works by comparing open pull requests against the same repository context and asking: do these two changes overlap in a way that could matter when they land together?

That construction has two unavoidable error modes.

False negatives - collisions the detector misses. Every advisory pre-merge check works from an incomplete picture of the codebase. Some risks only show up at runtime, through generated code, in configuration, in shared state, or in the developer's head. Those are exactly the cases a pre-merge surface can fail to see.

A vendor that says "100% caught" is implicitly claiming their advisory signal is the codebase. It is not.

False positives - pairs flagged that wouldn't actually collide at merge time. Two PRs may both work in the same area and be flagged as a likely collision - but in fact one PR edits a comment and the other adds a helper that the first never uses. The advisory signal can say "review these together." Whether they collide at merge time is a different question, and the detector cannot fully answer it without running the tests of one PR against the other's state, which is the very thing the pre-merge surface is trying to avoid being.

So a detector has to make a trade-off. Push toward recall (catch more) and false positives go up. Push toward precision (only flag the certain ones) and false negatives go up. There is no setting that gives you 1.0 on both.

Anyone who claims otherwise is selling you the marketing slide, not the detector.

The five honest promises a buyer should ask for

A vendor who is being honest about the math above will be willing to answer five questions in plain yes/no terms. If the answer to any of them is "trust us" or a long re-direct, the tool is not ready to be on your default branch.

1. Pre-merge advisory: yes or no?

Does the signal fire while the PRs are open in parallel, before either has been queued for merge? Or does it only show up after the fact (in a post-merge report, a weekly digest, a dashboard you have to remember to visit)?

Pre-merge is the surface where the author and the reviewer can still cheaply change the PR. Post-merge is autopsy.

The honest answer is yes or no. Not "both."

2. Content-free: yes or no?

Does the tool retain source code, or does it keep only minimal content-free service metadata?

Content-free is not a flourish. It is the difference between a tool that keeps source handling narrow and a tool that turns your repository into a stored review corpus. "We don't train on your data" is not the same as content-free. "We delete after 30 days" is not the same as content-free. Content-free means source bodies are not stored or shown back as product data.

The honest answer is yes or no.

3. False-positive triage path: yes or no?

When the detector fires on a pair that turns out to be fine, what does the user do? Is there a way to mark it "not actually a collision" that the system learns from? Is there a way to silence a specific pair without silencing the whole signal?

A tool with no triage path will, over time, train its users to ignore the signal. The siren that cries wolf is worse than no siren. Honest tools acknowledge the false-positive rate is non-zero and ship a path to deal with it.

The honest answer is yes or no.

4. Unknown verdict honesty: yes or no?

When the tool genuinely cannot tell whether two PRs will collide - because the advisory signal is ambiguous, because coverage for that slice of the repo is thin, because one PR touches generated code the tool can't parse - does it say "unknown"?

Or does it default to "safe" (and silently miss) or default to "flag" (and silently inflate the false-positive rate)?

A detector that distinguishes "I checked and found nothing" from "I could not check this slice" is operating with integrity. A detector that collapses the two into one verdict is hiding its own coverage.

The honest answer is yes or no.

5. Recall and precision public: yes or no?

Will the vendor tell you how they evaluate real collisions and false alarms? If they publish recall, precision, dataset, or ground-truth numbers, do they also explain how those numbers were measured?

This is the question that separates "we have a model" from "we have a model and we're willing to be measured." Published numbers are useful, but only when the method is legible. A vendor that claims "100% caught" and cannot explain the dataset or measurement process behind it is making a marketing claim, not a measurable one.

The honest answer is yes or no.

What Veripsa actually says

Veripsa Core is the kind of tool this post is about. We surface possible collision risk between open PRs while the PRs are still in flight. The signal is a per-PR GitHub check, advisory, content-free, and bounded to the same repository under the same owner today.

What we do not say is "100% caught." We do not say "we guarantee a clean main." We do not say "we prevent collisions." Those claims would be lies under the math above.

What we say instead is closer to: we surface collision risk; you decide. The check flags pairs of open PRs that may interfere and gives an outcome-level reason. The decision to rebase, to coordinate, to merge anyway, or to ignore the signal stays with the author and the reviewer. Branch protection still decides what is required for merge. The signal is peripheral vision, not a wall.

We also publish plain-language coverage notes for a given repo: which kinds of files the advisory signal can read, and where it will return Unknown. Some repos are well-suited to advisory pre-merge checks. Some are not - opaque generated blobs, single-file scripts, or code written in a language we don't support yet. That tells the team whether the tool is a fit for their codebase before they commit to it.

That posture - advisory, content-free, honest about coverage, honest about unknowns - is the posture this post is arguing for, in general, not just for Veripsa. We do not publish a universal recall or precision number today; until we can explain a measurement method plainly, we prefer to publish boundaries and examples instead of pretending one number settles the question.

If you remember one thing

If a DevTool vendor tells you they catch 100% of cross-PR collisions, ask them which of the five questions above they will answer yes to in writing. The vendor who can answer all five is rare. The vendor who can answer all five and claim 100% recall is, with very high probability, not telling you the truth.

An honest pre-merge risk signal should be small: pre-merge, content-free, with a triage path, an unknown verdict, and measurement claims that are explained rather than inflated. Everything else is decoration.

If that posture resonates, the Veripsa fit page is at /fit - including the cases where Core does not earn its keep yet. The buyer-facing notes on what the check actually says, plan tiers, and coverage are at /docs. The data-handling posture and the content-free boundary are at /trust.