← all writing

What TrueMend Deliberately Can't Do

Six in January, fifteen in February, thirty-five in March, CVEs traced to AI-generated code. Here's the honest version of where a static analyzer stops.

Georgia Tech runs something called the Vibe Security Radar, tracking CVEs that trace back specifically to AI-generated code. Six in January of this year, fifteen in February, thirty-five in March. That's a curve, and curves like that don't flatten on their own. Two of the named ones from that stretch: a directory traversal bug scoring 9.1 out of 10 on severity, and an input handling flaw in an SDK. Both old, well-understood bug classes, both reproduced by a model that had no reason to know better.

The instinct when you sell a code quality tool is to imply it catches everything upstream of that curve. We'd rather tell you exactly where the line is, because a tool that overclaims what it protects against is more dangerous than one that's honest about its edges.

TrueMend is static analysis over materialized text. No runtime inspection, no data-value tracing, no bug-fix synthesis, no ship or merge judgment, no reasoning about an API contract nobody wrote down. It complements a coding agent's judgment. It doesn't replace it.

What that boundary means in practice

Static analysis over materialized text means TrueMend reads the code that's actually written down and reasons about its structure. It does not run the program, does not know what value flows into a function at 2am under real load, and does not trace whether user input actually reaches a dangerous sink at runtime. That last one is the big one. Interprocedural taint analysis, the kind that traces untrusted input from source to sink across functions and files, is CodeQL and Semgrep-Pro territory. We say this plainly in our own competitive benchmark: none of the tools we measure ourselves against do it, TrueMend included.

// what a pattern matcher sees
// looks exactly like the dangerous version on the page in front of it
const filePath = path.join(UPLOAD_DIR, userInput);
// TrueMend flags this shape reliably -- it's visible on the line

// what it can't see
// whether userInput can ACTUALLY reach here with "../../etc/passwd"
// depends on validation three function calls upstream, in a
// different file, that a pattern matcher has no reachability
// model for. That's a data-flow question, not a pattern question.

TrueMend catches the shape reliably; it approximates reachability structurally rather than knowing it, which is a meaningfully different guarantee than a tool with an actual semantic database. That's also, not coincidentally, why the false-positive work in our own benchmark is so heavy on "is this eval actually reachable by an attacker, or is it a local dev tool reading its own terminal input." A pattern matcher can't tell those apart on its own. A human, or a deeper semantic engine, can.

Bug-fix synthesis is the same kind of boundary. TrueMend's deterministic tier applies a fix when the correct rewrite is provably the only reasonable one, a nesting collapse, a dead import, a formatting normalization. It does not invent a fix for a genuine logic bug, because that's not a mechanical transform, that's a judgment call about what the code was supposed to do. No ship or merge judgment either. TrueMend will tell you a function is 200 lines of tangled nesting. It will not tell you whether that's acceptable for this particular release.

Why naming the boundary is the useful part

A rising CVE curve doesn't get fixed by one tool claiming to cover the whole thing. It gets addressed by teams knowing which layer catches which failure mode, so nothing silently falls into the gap between "our static analyzer would have caught that" and "no, actually, it wouldn't have, and we assumed it would." The old bug classes, the OWASP-top-ten kind of mistake a model reproduces at scale, those are exactly where a pattern-based deterministic pass earns its keep. The reachability question, the "does this actually get exploited in production" question, that's a different tool's job, and pretending otherwise is how a team ends up with a false sense of coverage.

← all writingnext: how the engine works →