By Pramin Pradeep, Co-founder & CEO, BotGauge AI
Every agent you ship is an attack surface
For most of the last few years, my job was making software prove it works. At BotGauge, we built autonomous QA that tested our customers' apps before every release. Then those same customers started shipping AI agents, and the old playbook stopped being enough.
A test suite asks one question: does this do what it should? An agent needs a second one: what will it do when someone wants it to misbehave? That second question is red teaming. It's why we launched our agent red-teaming platform on October 28, 2026 and the developer community noticed: it finished #3 Product of the Day on Product Hunt.
Why red teaming has to change for agents
Red teaming isn't new. It started as military war-gaming, became a staple of security teams paid to break in, and with LLMs turned into jailbreaking chatbots.
Agents raise the stakes because they act. They call tools, read inboxes, write to databases and hand tasks to other agents. A bad chatbot answer is embarrassing. A bad agent action is an incident.
And adoption is outrunning testing. Gartner predicts 40% of enterprise applications will feature task-specific AI agents by the end of 2026, up from under 5% in 2025. Yet most teams I talk to test agents like APIs: happy paths, a few edge cases, a vibe check. None of that simulates an adversary.
What attacks on agents actually look like
OWASP ranks prompt injection as the top risk for LLM applications, but for agents it's only the front door. Here's what we see:
• Indirect prompt injection: instructions hidden in a web page, PDF, email or support ticket the agent reads mid-task.
• Tool misuse: tricking the agent into calling a legitimate tool with harmful arguments, like a refund, a delete or an outbound email.
• Data exfiltration: coaxing the agent into leaking customer data, system prompts or credentials through its outputs or tool calls.
• Goal hijacking: slow, multi-turn manipulation that drifts the agent away from its task without tripping any single-message filter.
• Multi-agent cascades: one compromised agent passing poisoned context to others that trust it by default.
Why we rebuilt BotGauge around this
Our autonomous QA product had paying customers and a working business. But functional testing was maturing fast, and the problem our customers kept raising wasn't "does it work?" It was "can I trust it in production?"
So we rebuilt BotGauge as an Agent Control Plane, and made one calls early:
● Red teaming at the core, not a checkbox. Observability tells you what happened. Red teaming tells you what could happen, before a user or attacker finds it.
What the platform does
The goal is simple: attack your agent the way an adversary would, every time it changes, and show you exactly where it broke.
• 200+ custom attack strategies spanning injection, tool misuse, exfiltration, goal hijacking and jailbreaks, run against your real agent, not a sandboxed copy of the model.
• Full tracing: every failed attack links to its trace, so you see which input, which step and which tool call went wrong.
• Evals that persist: results are scored and compared across releases, so a fix doesn't quietly regress next sprint.
• Fits your stack: it sits on top of runtimes like LangGraph and AWS Bedrock AgentCore instead of replacing them.
Where to start, with or without us
- Inventory what your agent can touch: every tool, data source and permission. That list is your attack surface.
- Map threats to a framework. OWASP's LLM Top 10 and MITRE ATLAS give you shared language with your security team.
- Attack the inputs you don't control. Retrieved documents, emails and web pages are where injection hides.
- Trace, don't just score. A pass rate tells you that something failed. A trace tells you why.
- Red team every release. A new prompt, model or tool is a new agent. Test it like one.
- Default to least privilege. The safest tool call is the one the agent was never allowed to make.
The bottom line
Your agents will be attacked. Regulators are already moving: NIST's AI risk guidance and the EU AI Act both call for adversarial testing. The only real question is who finds the holes first, your team or someone else. I'd rather it be you.
About the author
Pramin Pradeep is co-founder and CEO of BotGauge AI, an Agent Control Plane for testing, observing and governing AI agents in production. BotGauge is backed by Surface Ventures, IA Seed Ventures and Saka Ventures. *
Sources
- Gartner, prediction on task-specific AI agents in enterprise apps (Aug 2025) [add link]
- OWASP Top 10 for LLM Applications
- MITRE ATLAS
- NIST AI Risk Management Framework
- EU AI Act, Article 55 (adversarial testing for general-purpose AI with systemic risk)
This story was distributed as a release by Jon Stojan under HackerNoon’s Business Blogging Program
