Certified PASSING on a real production Express app: 9.9% wrong-match, under our own 10% bar, on the full post-fix pool (596 judged, 90.1% correct). The number that matters here isn't just the win — it's that we caught our own regression before publishing it and re-proved the result with the same judge.
Habitica is a real, large, production Express.js codebase — the kind of app where a detector surface either survives contact with real code or it doesn't. Three rounds of blind adjudication against it, and one caught regression, get us to a number we'd put our name on.
The result
9.9% wrong-match, 90.1% correct, 8.2% real-rate — 596 findings, full post-fix pool, judged blind. That clears our own <10% bar. Express is now certified PASSING.
On the same round-3 pool, Semgrep's wrong-match was 62.7% (51 judged, 37.3% correct, 21.6% real-rate) — TrueMend at 9.9% against Semgrep at 62.7%. Semgrep's 62.7% wrong-match includes a genuine, verified 18-instance false-positive cluster on Habitica's own JSON/CSV/PDF renderers (not our bar, but disclosed for context).
| Stage | Judged | Wrong-match |
|---|---|---|
| Round 1 baseline | 872 | 36.9% |
| Round 1 (fixes + disables) | 622 | 11.6% |
| Round 2 baseline (fresh sample) | 666 | 17.4% |
| Round 2 (fixes) | 611 | 9.98% |
| Round 3, certified (fresh sample, same judge as round 2) | 596 | 9.9% |
We shipped a win, then found out it had gone stale
Round 2's 9.98% merged clean. Then an unrelated engine fix elsewhere in the
codebase — a noise-suppressor threshold correction that originated in the
FastAPI campaign — landed on main afterward and silently changed Habitica's
raw finding volume underneath the already-published number: 2,885 findings
became 8,986 on the same corpus. We don't let a published number go stale
without saying so, so round 3 re-collected and investigated rather than
re-publishing the old figure.
Two real, previously-invisible bugs surfaced, neither caused by the noise-suppressor change itself — it just changed the sample enough to expose them:
- Vue/Svelte files were getting JSX rules.
.vue/.sveltesingle-file components are bucketed under the"typescript"language for scope purposes, and atypescript→tsxalias let React/JSX-only rules run against them. One rule alone —typescript-eslint-react-no-unknown-property, which flags invalid JSX attributes — misread ordinary Vue template attributes (class=,v-html) as invalid JSX properties. 5,261 of 8,986 findings (58.5%) were this one false-positive category. - A console.log rule with no structural anchor.
express-ux-console-log-production's own strategy note says it's meant to catchconsole.logleft in Express route handlers — but nothing in the detection logic actually required that context. It fired on migration scripts, build tooling, and a client-side logging library's ownconsole.errorcalls: 840 findings, a quarter of the corrected pool.
Fixing both dropped the count from 8,986 to 2,567. A third, smaller cluster
then surfaced — a naive regex tightening attempt on the JSX rule paradoxically
made it worse (23 false positives → 49) by changing how the engine masked
string/template content before matching. The real fix added a same-line
negative pattern to exclude dotted-member-access assignments
(user.stats.class = klass) while narrowing the match scope, clearing all 23
without losing any genuine JSX detections.
Judge-divergence, caught and corrected
Round 3's first adjudication pass used a different judge model than rounds 1–2, and it came back at 23.3% wrong-match — worse than round 2's 9.98%, and on its face a regression. Before publishing that number as a loss, we ran the same 407 findings through both judges and found the new one was materially stricter on the literal correctness axis (23.8% wrong vs. 12.3% wrong on identical items, 85% agreement) — the two numbers weren't measuring the same thing. We re-judged the identical 596-finding round-3 sample with the original judge from rounds 1–2, giving the certified, directly comparable result: 9.9%, consistent with round 2's 9.98%, no regression.
What was fixed overall
46 categories tightened across rounds 1–2, plus the round-3 fixes above. Two
detectors were disabled outright rather than patched, because the underlying
claim was false, not just imprecise: css-border-zero-vs-none (border:0
and border:none render identically — there's no bug to detect) and
yaml-compose-depends-on-circular (the detector had no actual
cycle-detection logic behind its name). Two shared-engine bugs were found and
fixed along the way, benefiting every language that hits the same code paths:
Vue/Svelte SFCs parsed through a synthetic wrapper made function-too-long
fire on any SFC over ~37 lines (the whole script block counted as one
function), and src/core/regex_dispatch.py's single-window file handling
used .search() instead of .finditer(), silently dropping genuine second
matches on any file with more than one hit in a single scope window. Full
suite green throughout: 11,187 passed, 9 skipped, 0 failed.
Still open
This is one real corpus, exhaustively re-verified across three rounds — not
yet a from-scratch campaign against a second, independent Express app. Full
detail and raw evidence: fp_campaign/EXPRESS_CAMPAIGN_REPORT.md.