TrueMend vs. ESLint+react on React

Excalidraw is a real, large production React+TypeScript app — 372 files in scope, tests and vendored Emscripten glue excluded. We deliberately did not reuse react-reconciler's own source as a corpus for this campaign: it's the reconciler's own implementation, not app-shaped code, and its findings skew toward generic nesting/length detectors rather than React-specific ones. Competitor is ESLint configured with eslint-plugin-react and eslint-plugin-react-hooks — bare ESLint would only re-measure the language surface already covered by our TS/JS language campaigns (§8), not the React-framework detector surface this one targets.

Round 1 — an interim number, five real bugs

An 80-of-378 interim sample (2 of 10 planned batches — stopped early once both the wrong-match rate and the bug-discovery rate made the remaining 8 low-value) put TrueMend at 37.5% after two fixes, then 28.6% after three more detectors were withdrawn from the default surface. ESLint+react held steady at 14.3% throughout — both tools were above the 10% bar on this interim read, not a clean loss for either side.

Five systemic bugs, verified by re-running the collector post-fix (800 → 722 findings): react-optional-chaining-in-render-without-fallback (53 of 800 pool findings, the single largest category) matched any a?.b anywhere in a file with no JSX-context check at all — disabled, needs real JSX-context tracking a line-scan can't do. react-usecallback-no-benefit had no negative pattern whatsoever, firing on every useCallback call including ones genuinely passed as props — disabled. react-useeffect-empty-deps-warning had a 6-line scope window against real useEffect bodies routinely 15-130+ lines long, plus a setter-detection regex that also matched setTimeout/ setInterval — disabled. react-unnecessary-fragment's ast-grep pattern used the wrong wildcard ($$$ instead of $X), matching any Fragment regardless of child count — fixed, not disabled; count dropped 7→1, the survivor a genuine true positive. Two duplicate iframe-title specs had scope_lines: 0, missing any title= attribute placed even one line below the flagged tag in real multi-line JSX — widened to 15 lines.

Round 2 — a fresh certification, not a partial sample, still fails

The round 1 "still open" note called for a full re-sample rather than trusting a 2-of-10-batch read. Round 2 delivers exactly that: a fresh crawl of the live pool (1310 TrueMend + 196 ESLint+react findings) drawn to a stratified sample and judged completely — 274 of 274 TrueMend findings, 88 of 88 ESLint+react, zero unjudged.

Tool	Judged	Correct	Wrong-match	Worth-fixing	Real-rate
TrueMend	274	69.7%	30.3%	21.2%	21.2%
ESLint+react	88	92.0%	8.0%	40.9%	40.9%

Not a like-for-like delta against the 28.6% interim figure — different sampling scheme, different pool size — but the same verdict either way: failing. ESLint+react clears the wrong-match bar cleanly but is itself under the 70% real-rate bar (40.9%) — neither tool is fully clean on this corpus's framework-noise categories, TrueMend is just far worse on both axes.

Six categories, 100% wrong, none of them touched by round 1's fixes

api-ux-no-pagination (8/8) fires on plain DOM .select() calls (searchInputRef.current?.select()), not backend list endpoints — a new instance of a token-collision bug a prior FP campaign already partially guarded against for other frameworks. react-window-event-listener-missing-cleanup (8/8) fires on listeners that already have an explicit cleanup a few lines below. cq-deeply-nested-callbacks (8/8) fires on a single shallow callback with no actual nesting. react-missing-aria-labels (7/7) — a 2-line scope window misses aria-label attributes placed 3-13 lines away in real JSX. react-conditional-rendering-anti-pattern (7/7) fires on plain single-level ternaries, not the nested-ternary/&&-chain pattern it claims to catch. typescript-sq-s6480-jsx-no-duplicate-props (5/5) fires on JSX elements with no duplicate props at all. Two smaller categories are also fully wrong: typescript-eslint-react-style-prop-object (3/3) and react-aborted-request-no-cleanup (3/3).

The five round-1 fixes verifiably held — none of the three disabled detectors appear anywhere in this round's 274-finding sample, and the one fixed detector's lone sampled instance is a genuine true positive. This gap isn't a regression; it's six categories the 80-item interim sample simply never reached.

Round 3 — nine whole-construct detectors quarantined, not deleted

All six round-2 100%-wrong categories, plus three more surfaced by a larger sample, shared one structural defect: each was checking a whole-construct property — an element's accessible name, a button's declared type, a nested ternary's real depth, a callback pyramid's actual nesting, a duplicate JSX prop, an unpaginated endpoint — that a single-line regex cannot scope, no matter how the window is widened. Root-caused against real Excalidraw source and quarantined with a data-driven quarantine_reason on each: api-ux-no-pagination, cq-deeply-nested-callbacks, typescript-sq-s6480-jsx-no-duplicate-props, typescript-eslint-react-style-prop-object, react-conditional-rendering-anti-pattern, react-missing-aria-labels, typescript-eslint-react-button-has-type, react-window-event-listener-missing-cleanup, react-aborted-request-no-cleanup.

Four of the nine — the aria-label/button-type/cleanup group — looked fixable by widening the negation window. A constructed recall test proved otherwise: no window size from 2 to 18 lines in either direction is recall-safe, because an adjacent element's own attribute suppresses a real finding on the flagged one. The current configs were already recall-lossy by construction; widening would only trade one false-positive count for a false-negative one, which we don't count as progress.

A shared engine bug surfaced mid-round: per-pattern quarantine was honored by the line-scan pool loader but not by the shared per-module spec loader React/auth/AI detectors load through — four quarantined detectors kept firing regardless, and the first round-3 measurement came in at 18.1% because of it. Fixed the loader; two other hand-rolled spec loaders still bypass quarantine on non-React surfaces, documented as a follow-up, confirmed to have zero effect on this measurement.

Re-collected (1244 TrueMend + 196 ESLint, down from round 2's 1310 by exactly the nine quarantined detectors' findings), re-sampled, and re-adjudicated blind — this round by glm-4.7, not Claude:

Tool	Judged	Correct	Wrong-match	Worth-fixing	Real-rate
TrueMend	217	89.4%	10.6%	22.1%	22.1%
ESLint+react	88	80.7%	19.3%	36.4%	36.4%

At the bar, within noise at n=217, and now ahead of ESLint on the same corpus and same judge. No category runs 100% wrong at n≥5 anymore; the residual is spread thin, led by typescript-eslint-ts-no-unsafe-assignment (3/8).

The real-rate number needs a judge-dependence caveat

Round 2 was judged by Claude; round 3 by glm-4.7. ESLint's own findings moved sharply across that judge change alone — 8.0%→19.3% wrong, 40.9%→36.4% real — on identical findings. That's the judge, not either tool, so round-2 and round-3 real-rates are not directly comparable. What is comparable: round 3's TrueMend-vs-ESLint gap (same judge, same corpus), and the wrong-match drop itself (30.3%→10.6%), since the quarantined detectors' findings are gone from the pool regardless of who's judging.

Still open

Real-rate (22.1%) still misses the >70% bar, but the same judge scores ESLint's own correct findings the same way — glm-4.7 treats most correct any-usage/magic-number/deep-nesting findings as technically-true-but-not- worth-fixing for both tools alike. That's a judge-calibration question, not evidence either detector surface is broken. A same-judge (Claude) re-measure of this exact round-3 pool, the way Express caught and resolved its own judge-divergence, would confirm whether 10.6%/22.1% holds — not yet run.