AI & ML
The Safeguard Worked. Is the LLM System Safer?
arXiv cs.CR0 views
arXiv:2609.00519v2 Announce Type: replace Abstract: Safeguards for large language model services are typically evaluated by their attack-blocking rates on test suites. Deployment decisions also depend on how much harmful assistance attackers can still obtain from the system, which these rates do not measure. To determine what published evaluations establish about deployment safety, we develop a framework that translates reported results into the minimum and maximum harmful assistance that can remain under a specified outcome and threat model. For safeguards that block attacks through a check, we show that the most an attacker can obtain depends on three properties: whether attack paths pass through the check, its miss rate against every attacker in the threat model, and how much assistance remains after a block. We apply this framework to 85 safeguard evaluations from 52 papers. Of these, only five argue that every attack path passes through the check, and four provide evidence on what remains after a block. These evidence gaps have practical consequences: in NeMo Guardrails' default streaming configuration, the check blocked 94% of password leaks in our test, yet the client received the password in every leak, blocked or not. We identify which of the three properties each safeguard category's evaluations omit, and what to measure for each of four ways attacks bypass a check. Deployments that require these measurements before launch can choose and configure safeguards that limit what attackers obtain from the deployed system.
Source:https://arxiv.org/abs/2609.00519PreviousIoMT-SecAlarmBench: A Semi-Synthetic Counterfactual Benchmark for Integrity Attacks in IoMTNextMalware Clustering in the Wild: Taxonomy, Challenges, and Evaluation Metrics
Related posts

‘It’s a risk’: How Rockstar built the most detailed (and romantic) GTA yet
0The Verge
China dominates battery graphite, but this startup could break the monopoly
0TechCrunch
AI visibility: mentions, recommendations and citations need separate checks
0dev.to
Background AI Makes Inference a Scheduling Problem
0dev.to

DeepSeek vs Claude vs Kimi vs Codex vs Gemini: A 70-Call AI Ethics Test
0dev.to
Fluent English is not evidence of ASL understanding: lessons from building a video translator
0dev.to