TrueMend vs. ESLint on TypeScript/React

Same discipline as our Rust and C# benchmarks: we don't just report the good number, we report what it took to get there — including the part where we were behind.

We published the loss

A blind-adjudicated pass earlier this campaign put TrueMend at 13.6% wrong-match against ESLint's 8.2% on real next.js source — a confirmed loss, independently judged, not self-measured. We said so. Then we went and fixed it.

Root cause, not a patch

Reading the judged verdicts against the real source turned up one dominant meta-pattern across roughly 35 of 41 wrong categories: detectors matching a lexical/keyword shape without confirming the actual AST construct, type, or semantic role they claimed to be flagging — a nesting-depth counter that mis-tracked flat else-if chains as nested, a "redundant variable" check that couldn't see past an intervening statement, an accessibility check whose free-tier detector never even ran the same logic as its (separately maintained) paid-tier counterpart.

We split the fix across a concentrated pass plus a five-worker fleet, one worker per failure cluster (nesting/coupling metrics, promise/async construct-confirmation, variable/type/string hygiene, test-isolation and security absence-claims, and the long tail of UX/lint edge cases), each required to verify against the real flagged source line — not just a synthetic fixture — before calling anything fixed. 39 of the 41 wrong categories got a real fix; the other 2 turned out, on inspection, to be cases where TrueMend's original finding was factually correct and the judge's call didn't hold up under scrutiny — we left those alone rather than force a fix that would only teach the detector to miss real positives. 142 new regression tests, full suite green throughout.

We re-verified before re-claiming anything

Before running a second blind adjudication, we checked our own fix against the real corpus, not just our own test suite: of the 108 originally-wrong findings, 104 (96.3%) were confirmed silent at the exact file:line. The remaining 4 are disclosed, not hidden — one documented residual we couldn't find a safe general fix for, and three cases (found above) where the original judge verdict was wrong.

Result — re-confirmed on a larger sample

We more than doubled our sample coverage (14.6% → 31.7% of the finding pool) and ran a fresh, independent blind adjudication. Raw score: TrueMend 3.5% wrong-match, ESLint 11.3%. But that ESLint number looked too high given nothing about ESLint had changed — so we checked, rather than took the win. 60% of ESLint's wrong findings sat on vendored, pre-bundled CommonJS build output (bundles/) that TrueMend's file classifier correctly skips as generated code; TrueMend sampled zero findings there at all. Recomputed on the same file scope both tools actually cover:

TrueMend: 3.5% wrong-match. ESLint: 5.0%. A real, confirmed, if narrow, win — not the same-day preliminary reversal we've flagged as premature before. This one went through the full independent-judge process twice.

The last residual, closed honestly

js-promise-all-fail-fast was the single dominant residual — 31% of TrueMend's remaining wrong findings, all the same shape: the rule assumed Promise.all implies a fail-fast defect needing allSettled semantics, without checking whether the surrounding code actually needs partial- failure tolerance. We tried to narrow it with a lexical signal first — per-item error handling inside the Promise.all args — and it only resolved 7 of 17 wrong findings. The other 10 were lexically identical to all 20 correct-bucket instances. A same-file, same-author minimal pair made the ceiling explicit: action-handler.ts:940 and action-handler.ts:973 are both try { await Promise.all([...]) } catch { abort(); throw } around two coupled operations — one judged correct, one wrong, with no regex-visible difference between them. Whether fail-fast or allSettled is right depends on downstream consumption semantics, which is an architectural judgment call, not something a line-scan can resolve without overfitting to individual call sites. So we disabled it by default rather than ship a detector we can't stand behind — the same call we made earlier for csharp-is-pattern-without-null and cq-ai-gratuitous-design-pattern. Confirmed against the real corpus: 0 findings for the category, all 37 evidence pairs from both adjudication rounds silent at their exact file:line. Full suite green (10,448 passed). This campaign is done.