Promptfoo vs sandbox-runtime: the honest comparison
The short answer
Edition v2026.Q3 · pricing and status verified 2026-08-06.
Choose Promptfoo for finding the guardrail you forgot — adversarial testing of prompts, agents and tool calls, run locally and wired into CI. Choose sandbox-runtime for putting a hard filesystem and network boundary around any agent process — coding agents especially — without containers or a cloud bill.
On the elems table, Promptfoo holds seat #1 of the Guardrails element and sandbox-runtime holds #2 — this is the closest call in the category, and the honest answer depends on which trade-off you can live with.
The case for each
1PromptfooPromptfoo, Inc. → OpenAI (acquired Mar 9, 2026)
Community free (all evals, all providers, 10k red-team probes/mo, self-hosted) · Enterprise custom · On-Premise customBest for Finding the guardrail you forgot — adversarial testing of prompts, agents and tool calls, run locally and wired into CI.
Watch It tests; it does not enforce. A green promptfoo run is not a runtime control, and nothing here blocks a live request. The OpenAI acquisition also puts the category's most-used neutral scanner inside a frontier lab — the OSS pledge is a pledge, not a governance structure.
2sandbox-runtimeAnthropic (Apache-2.0, beta research preview)
Free · Apache-2.0 · no service, no accountBest for Putting a hard filesystem and network boundary around any agent process — coding agents especially — without containers or a cloud bill.
Watch Explicitly a "beta research preview": Windows support is alpha, and the API is not stable. It does nothing about content — no injection detection, no PII, no topic control. And a sandbox with an over-broad network allowlist is theatre; the config is the product.
Promptfoo vs sandbox-runtime: side by side
Edition v2026.Q3 · verified 2026-08-06.
| Promptfoo | sandbox-runtime | |
|---|---|---|
| Layer | Test / red-team | Containment |
| Open source | Yes (MIT) | Yes (Apache-2.0) |
| Entry price | Free (10k probes/mo) | Free |
| Injection detection | Attack generation, not blocking | No |
| Tool-call / action control | Yes — agent + tool plugins | Hard fs + network boundary |
| Self-host | Yes | Yes (local) |
| Model-agnostic | Yes | Yes |
| Added latency | None (offline) | Negligible |
What each side won't tell you
Promptfoo: It tests; it does not enforce. A green promptfoo run is not a runtime control, and nothing here blocks a live request. The OpenAI acquisition also puts the category's most-used neutral scanner inside a frontier lab — the OSS pledge is a pledge, not a governance structure.
sandbox-runtime: Explicitly a "beta research preview": Windows support is alpha, and the API is not stable. It does nothing about content — no injection detection, no PII, no topic control. And a sandbox with an over-broad network allowlist is theatre; the config is the product.
If it's neither
The rest of the top five: NVIDIA NeMo Guardrails (programmable rails, open source) · Amazon Bedrock Guardrails (managed rails, any model) · Lakera Guard (enterprise injection defense). The complete field — 30 more tools including the graveyard — is on the element page.
This analysis is drawn from the Guardrails element dossier — the ranked top 5, the comparison matrix, and the complete field of 30 more tools live there, with every source. Data: gd.json (CC BY 4.0).
Every claim above is dated and sourced from the elems dossiers — 1,421 tools tracked across 58 categories, verified 2026-08-06, including the 276 we found dead, renamed, acquired, or sunsetting. Rankings are editorial, never paid — the charter.
Build your stack in 5 questions →