TrueMend vs. SpotBugs + Checkstyle + PMD on Java

Java's detector surface first cleared blind adjudication at 17.2% wrong-match against a three-tool bundle — SpotBugs, Checkstyle, PMD — at 60.8%, on 325 judged findings. A fleet-wide fix pass targeted the top wrong-match categories across five above-bar language surfaces at once, then a fresh, from-scratch full-pool sample re-certified each one, Java included. Here's what that re-certification actually showed.

Round 2 — a fresh, larger sample, not a re-run of round 1

Round 2 pulled a new full-pool sample from real Java corpora: 485 TrueMend findings judged, up from 325. The bundle was not re-run this round — its 60.8% is still the round-1 number. Every claim below about the bundle inherits that staleness; every claim about TrueMend is fresh.

Result

TrueMend: 18.6% wrong-match (485 judged), real-rate 22.9% — up from 5.5% before this pass. SpotBugs + Checkstyle + PMD: 60.8% (round 1, not re-measured). Wrong-match moved only slightly, 17.2% to 18.6% — within noise of a fresh, 1.5x-larger sample pulling in categories the prior pool never touched. Real-rate is the number that actually moved: it nearly quadrupled. That means the fixes below made TrueMend's Java findings four times more likely to be something an engineer would actually act on, even though the headline wrong-match figure barely shifted. Both axes matter here, and they moved independently — reading the flat wrong-match number as "no progress" would miss the real result.

What was fixed

Four concrete bugs, root-caused against real source:

• java-raw-type-usage matched JDK collection names as an unanchored substring, so project classes like SinglyLinkedList or MergeKSortedLinkedList tripped it purely for ending in LinkedList — none of them declare a type parameter, so every one of those hits was factually false. Fixed with a whole-identifier enumeration of real JDK generic types plus a same-file import-scope gate, so LinkedList list = new LinkedList<>(); still fires and SinglyLinkedList no longer does.
• A tree-sitter grammar gap in src/metrics/function_metrics.py: nesting depth for else if chains relies on the parser wrapping a nested if in an else_clause node. TS/JS grammars do that; Java, C, C#, Go, Kotlin, and Rust don't — their if_statement's else branch attaches the nested if directly, with no wrapper node. TrueMend was reading flat else if chains as escalating nesting depth on every one of those six languages, producing deep-nesting false positives. This fix is not Java-specific — it corrects nesting depth on all six affected languages at once.
• An assertion-delegate detection improvement, closing a gap where test assertions routed through a helper method weren't recognized as the assertion they delegate to.
• A line-attribution bug in src/core/regex_dispatch.py's slop-unnecessary-variable check: when a match's own regex anchors on a leading newline (an alternation like start-of-file-or-after-line-break), the windowed line-scan undercounted those leading newlines and attributed the finding to the line before the construct actually starts — caught via a regression on Treap.delete in the-algorithms-java. Also cross-language, not Java-specific.

Still open

TrueMend still beats the bundle by a wide margin even on the stale comparison — 18.6% against a 60.8% number that hasn't been re-measured — but 18.6% sits above our own 10% gold-standard bar, same as it did at 17.2%. The real-rate jump from 5.5% to 22.9% is genuine, not cosmetic: it means far more of what fires is worth an engineer's time. But 22.9% real-rate also means most findings in the pool still aren't. Four fixes touched a handful of categories on a corpus that fires 80-150+ distinct categories; the rest of that surface is unaudited. Closing the remaining wrong-match breadth, and getting the bundle re-measured on the same fresh pool, are both open.