TrueMend vs. Roslynator on C#/.NET

The first C#/.NET number we published was preliminary — "roughly at parity," disclosed as noisy. It didn't hold. Here's what a real head-to-head adjudication found instead, and what it took to earn a number worth publishing.

The preliminary pass didn't survive a fresh check

That first round, self-measured on Polly, put TrueMend at 31.8% wrong-match against Roslynator's 42.9% — encouraging, but explicitly flagged as noisy sampling, not a confirmed result. A fresh, independently-judged head-to-head on the same corpus came back at 15.6% wrong-match for TrueMend vs. 49.4% for Roslynator — a materially different baseline once cross-cutting TS/JS-era fixes had also touched the shared C# detector surface. The lesson from Rust and TypeScript held again here: don't trust a self-measured or stale number, re-verify before publishing anything.

Round 2 — six real bugs, all domain-premise mismatches

Reading the judged wrong findings against Polly's actual source turned up a consistent shape: detectors built on a premise that didn't hold for a resilience library specifically. csharp-ef-take-missing flagged Skip(1) calls doing small-literal recursive slicing as if they were Entity Framework pagination that needed a paired .Take() — Polly has no EF in it. csharp-httpclient-no-timeout fired inside documentation code paths and, worse, described HttpClient's default behavior as an "indefinite hang" — factually wrong; the runtime default is a 100-second timeout, not infinite. We fixed the detector's exclusions and its own claim. Four more fixes in the same vein. Result: 7.2% wrong-match vs. Roslynator's 37.0% — a real, confirmed win on precision. But a second number told us we weren't done: worth-fixing came back 0% for TrueMend vs. 11.1% for Roslynator. Winning on precision while missing an entire category of real bugs isn't a result to stop at.

Round 3 — the worth-fixing gap was a missing category, not a bug

Every one of Roslynator's worth-fixing findings on this corpus was the same kind of thing: RCS1232/RCS1243 XML-documentation defects — <param> descriptions swapped between adjacent parameters, a doubled word in a /// comment. TrueMend had zero detectors for that category, full stop — not a precision problem, a coverage gap. We built two: csharp-xml-doc-param-swap (adjacent <param> tags with cross-swapped descriptions) and csharp-xml-doc-duplicate-word (a repeated word inside a doc comment). Both fire on real Polly instances — 13 genuine public-API doc bugs, including three param-swaps Roslynator itself missed.

Result — confirmed via a fresh head-to-head

TrueMend: 7.7% wrong-match. Roslynator: 39.5%. Worth-fixing closed from 0% to 10.3% against Roslynator's 11.1% — now within noise of parity, not a gap. 78 TrueMend findings and 81 Roslynator findings, judged blind, same-session head-to-head — the same discipline as the Rust and TypeScript campaigns. 28 new regression tests across the three rounds, full suite green throughout (10,562 passed).

What's still disclosed, not hidden

Two of eleven RCS1232 cases are genuine misses: pure tag-order mismatches where each individual <param> description is actually correct for its own parameter name, just in the wrong position relative to the signature. Catching those needs AST-level signature-to-doc-order correlation, which a line-scan detector can't do — a real, disclosed limitation, not swept into the win.

A regression, caught and fixed same day

The 7.7% number above was published and treated as settled. It wasn't. A same-repo reverify on 2026-07-25 (csharp-reverify-20260725) measured 12.5% wrong-match on the same corpus — a real regression, not a stale number or measurement noise.

Root cause: meai-unconfigured-chat-options and three sibling Microsoft.Extensions.AI detectors were anchored on the bare method name GetResponseAsync with no file-level type gate. That fired 3/3 wrong on Polly's own Snippets/Docs/Migration.Policies.cs — a file that defines its own, unrelated GetResponseAsync returning HttpResponseMessage as part of an HTTP-to-Polly migration guide, nothing to do with IChatClient. Sibling patterns in the same spec file already guarded against exactly this failure mode with file_content_requires: "IChatClient"; these four detectors just hadn't been given the same guard. That's the gap — four detectors shipped without the type gate their own neighbors already proved was necessary.

Fixed the same day by adding the guard. Retrospective recompute of the same judged sample with the three now-impossible false positives removed: judged 80→77, wrong 10→7, wrong-match 12.5%→9.1% — back under the 10% bar, against Roslynator's 42.0% on the same comparison.

Be clear about what 9.1% is and isn't: it's a retrospective recompute on the existing judged sample, not a fresh from-scratch blind campaign. A new independent blind run against Polly (or a second C# corpus) is the gold-standard confirmation, and it hasn't happened yet.