We set out to write a very different article.

We run a trust scanner for MCP servers and AI skills. We have a version archive that watches thousands of them change over time, and we went digging in it for a “the ecosystem is getting dangerous” story. We had three leads: a spike in maintainer takeovers, a rise in risky install scripts, and a growing pile of servers our scanner had flagged “do not install.”

Every one of them fell apart.

The “maintainer takeovers” were developers renaming their GitHub handles. The “risky install scripts” were patch-package and electron-builder. And the pile of blocked servers? It included Google’s official Chrome DevTools MCP, MongoDB’s official server, and goose, a widely-used coding agent — flagged as dangerous over an HTML comment in a README, a class named Keychain, and a curl | sh line inside a Dockerfile.

The danger signal we went looking for was our own scanner’s noise.

This is the write-up of what we found when we stopped and audited ourselves: 13% of everything we’d scored was hard-blocked, and once we looked closely, essentially none of it was a genuine safety threat. It’s also the write-up of how we fixed it, and the three rules we’d hand any team running a scanner that’s drowning them in alerts.

TL;DR

323Servers blocked (13% of corpus)
107Recovered on re-score
0Still block on a safety finding
64%Of findings never execute

First, what this is not

This is not “MCP servers are safe, relax.” We didn’t measure that, and it isn’t true. What we measured is our scanner’s precision — how often it flagged something harmless. That’s a different axis from how much real malware is out there. Two of the projects below stayed blocked for legitimate reasons even after the overhaul, and we’ll close on a supply-chain attack that was very real. If anything, the argument here is more vigilance, not less: a scanner that cries wolf is a security liability, because the noise buries the one alert that matters.

The three false leads

“Maintainer takeovers.” Our archive tracks a repo’s top contributors over time. When that set flips completely, it looks — in the data — exactly like an xz-style hijack: the original author gone, a new name in control. We found a cluster and got excited. Then we checked against GitHub. Every one was a developer who had renamed their account. A rename 301-redirects the old handle and rewrites the contributor attribution; in a snapshot it’s indistinguishable from a takeover. Zero of the ones we sampled were hostile.

“Install-script creep.” Next lead: a rise in packages adding preinstall/postinstall scripts — the classic supply-chain vector. We pulled them. They were patch-package, electron-builder install-app-deps, husky, and a tree-sitter build step. Standard tooling. The signature that reads “arbitrary code on install” is also the signature of every Electron app’s build.

The blocked pile. Hundreds of servers our scanner had marked “do not install.” We started reading the findings, and the pattern was immediate: curl https://sh.rustup.rs | sh inside a Dockerfile. http://127.0.0.1:9222 read as data exfiltration. The word cookies. An HTML comment in a README. None of it was in code that runs on the user’s machine. All of it matched a keyword.

The scale of it

We re-scored every server our scanner had blocked — 323 of them, about 13% of a ~2,490-server corpus — and took the whole thing apart finding by finding. Across those 323 repos, the old scanner produced 713 findings. Here is where they lived and what they were:

Put plainly: two out of every three “threat” findings on a blocked AI-skill repo were either in a file that never runs, or a benign pattern — a localhost URL, a standard installer, the word “cookies.”

The projects we were wrong about

The false positives weren’t landing on sketchy no-name packages. They were landing on the most trustworthy tools in the ecosystem — which is the tell that the method, not the targets, was broken.

We were not, to be clear, discovering that these projects had shipped something bad. We were discovering that our scanner was wrong about good software. The maintainers here are the sympathetic party. That distinction matters, and it’s the whole reason this is a confession and not an exposé.

Why it happened: scope and vocabulary

Two root causes, and they’re the two mistakes almost every pattern-based scanner makes.

It scanned the wrong files. The scanner fetched and analyzed the whole repo — Dockerfiles, CI, devcontainers, docs — as if all of it were the thing you install. But a consumer of an MCP server runs the server’s code. They don’t run your devcontainer. They don’t execute your GitHub Actions. A curl | sh in your CI is a fact about your build, not a threat to your users. Sixty-four percent of our findings were in files with no bearing on what actually executes.

It matched vocabulary, not behavior. “Contains the word cookies” is not “steals cookies.” “Has a class named Keychain” is not “reads the OS keychain.” “Contains curl … | sh” is not “downloads and runs an attacker’s payload.” The scanner keyed on the surface form of dangerous code, and dangerous code shares that surface form with an enormous amount of completely legitimate engineering.

The proof that vocabulary was the deeper problem: even after we scoped scanning to runtime files only, a handful of projects still blocked — and they were still false positives. MongoDB’s Keychain class is in runtime code. Scoping alone doesn’t save it. You have to stop matching the word.

What we changed: three rules

We rebuilt the safety scorer (it shipped as algorithm v3.4.0) around three principles. Each is portable to any scanner, and each came with a measured payoff.

1. Scope to the installed artifact.

Scan what ships and runs — source code, the files a package actually publishes, entry points, and scripts that auto-execute on install. Not the CI, deploy, and devcontainer scaffolding your users never touch. Impact: 64% of our findings lived in non-runtime files. Scoping alone removed the majority of the noise.

2. Match behavior, not vocabulary.

“Opens a path under ~/.aws/credentials” is a signal. The word cookies is not. “Pipes an unknown host to a shell” is a signal. curl sh.rustup.rs | sh is not. 127.0.0.1 is not the internet. We rewrote every check to describe an action, and allowlisted the standard-installer and localhost cases that dominate the false positives.

3. Tier findings by blast radius.

This is the one most scanners get wrong. A finding in a Dockerfile is context — it should inform a reviewer, never issue a verdict. We split every finding into three levels: Blocking (dangerous behavior in code that runs), Review (worth a human’s eye, never auto-blocks), and Note (informational — infrastructure and docs, zero weight). Collapsing “note” into “block” is the machine that manufactures alert fatigue. We stopped.

One detail that matters for trust: we did not only loosen. In the same change, we made the one genuinely dangerous case stricter — a postinstall script that auto-runs curl <unknown-host> | sh on install is now a single-finding immediate block. Precision cuts both ways. That’s the difference between calibration and just turning the sensitivity down.

The result

We re-scored all 323 blocked servers under the new engine and wrote the corrected scores to production:

The part that keeps this honest: the real threat

Here’s where the “we don’t measure malware rate” caveat stops being a hedge and becomes the point.

● Real incident — OSV MAL-2025-47604

In September 2025, postmark-mcp — a legitimate, widely-installed MCP server for sending email — was updated to version 1.0.16 with a small addition: it began silently BCC-ing every email it processed to an attacker’s server. A backdoor, in a tool people had already integrated and trusted. This isn’t our allegation to make; the malicious code carries the OSV advisory MAL-2025-47604, and the backdoor was disclosed publicly by the security firm Koi Security.

That is the real threat. And it is exactly the class our malware gate is built to hard-block: a package carrying a malware advisory on its installable version is an automatic no.

But postmark-mcp teaches the sharper lesson. By the time the advisory was published, npm had pulled every malicious version. The package is fully unpublished today — a version-scoped scan has nothing left to flag. Malware is only catchable in the narrow window before it’s removed. The durable defense isn’t the CVE lookup; it’s the behavioral signal around a package — a sudden version bump that adds a network call, a new maintainer, a changed install script. Watching for those is the harder, more valuable job. And it’s a job you cannot do if your scanner is screaming.

This is the reframe the whole exercise earns: an attack like postmark-mcp produces one genuine signal — one version bump, one new outbound connection. If your scanner is already flagging Google’s DevTools, a localhost URL, and the word “cookies,” that one real signal is invisible. It’s the 323rd red flag on a list nobody reads anymore.

We cleared 107 false alarms so that the next postmark-mcp is the only thing left blinking red.

The MCP ecosystem is not “safe” — postmark-mcp proves it isn’t. But you cannot defend it with a scanner that cries wolf. Precision is not a nicety. It is the prerequisite for the real alert being seen.

Methodology

Every server ran through the production mcpskills.io engine — the same behavior-based safety model now live as algorithm v3.4.0, documented in full at /methodology. The 323 blocked servers were re-scored against the corrected engine; findings were classified by file tier (runtime / infrastructure / docs) and by behavior. The finding distribution (713 findings across the 323) was computed against the pre-overhaul scanner so the before/after is apples-to-apples. The postmark-mcp incident is documented under OSV MAL-2025-47604 and the Koi Security disclosure; the package is unpublished from npm as of September 2025.

Companion reports: What Changed in the MCP Trust Landscape, The Trust Middle — State of MCP Server Security.

Check a server before you trust it

Run the behavior-based trust report for any MCP server or skill — and see exactly what blocks, what’s a note, and why.

Open Scanner