📺 Today’s recommended deep-dive video: https://www.youtube.com/watch?v=QDsbFLEt9ro

Guarding the Guardians: Building the Control Plane for the Agentic Era

As autonomous AI agents move from simple text generation to performing complex software tasks, the risk of catastrophic errors or illegitimate actions grows exponentially. Maxim Kogan, CEO of Onyx Security, explains why we need specialized, small-scale models to oversee these powerful agents before they manage critical infrastructure like our power grids.

Core Question: How can enterprises safely deploy autonomous agents when traditional security controls like identity permissions and API gateways are no longer sufficient to govern model intent?

Highlights

• The shift from simple LLM chat bots to autonomous agents that can execute code and tool calls independently.
• Why traditional security models fail when agents require broad, human-level permissions to be productive.
• The strategy of training “small but specialized” models to oversee frontier models with low latency and cost.
• The unique convergence of Israeli cyber-intelligence talent and deep learning research at Onyx.

⏱️ Reading time: approx. 6 minutes · Saves you about 35 minutes vs. watching.

Want to take notes while watching? Click the image below and let AI Notebook capture the key points for you 👇

The Rise of Autonomous Agents

From AutoGPT to Real-World Utility

The journey toward autonomous agents began in earnest with projects like AutoGPT, which captured the industry’s imagination by demonstrating how Large Language Models could move beyond simple text generation to iterative decision-making loops. While early versions were limited by model reasoning, they proved the concept of agents using APIs to act.

Today, we are seeing the maturation of this concept through tools like Cursor, Devin, and various autonomous coding assistants that now dominate enterprise AI adoption. These systems aren’t just summarizing documents; they are performing long-horizon tasks, rewriting software, and managing cloud infrastructure. This leap in capability brings a simultaneous leap in risk, as a single hallucination or misguided planning loop can lead to permanent data loss or the accidental exposure of sensitive security tokens.

The reality is that enterprises cannot stop the adoption of these tools because the productivity gains are too significant to ignore for long.

💡 Digging Deeper

Q: Why did AutoGPT fail to take off initially?
A: The models at the time, including early GPT-4, were not yet capable of stable long-horizon reasoning or reliable tool calling.

Q: What has changed in the current market?
A: The arrival of “reasoning” models and better orchestration frameworks has made agents useful enough that even conservative enterprises are now sanctioning their use.

Q: Is the goal to replace humans in the loop?
A: As actions scale 100x or 1,000x, human oversight becomes impossible, necessitating “agentic” security to watch the agents.

The Security Gap in Agentic Systems

Why Traditional Firewalls and Identity Management Fail

Traditional security relies on the principle of least privilege, where you limit a software system’s permissions to only what it strictly needs to function. However, with an autonomous assistant, we want it to act on our behalf, which often requires giving it broad, human-like permissions. If you restrict an agent too much, you destroy its utility; if you grant it full access, you risk a “rogue” action having catastrophic consequences across your entire technology stack.

Proxies and API gateways lack the necessary internal context to understand why an agent is suddenly choosing to delete a database or exfiltrate a specific block of code.

Because existing endpoint detection systems cannot peek into the “thought process” of a model, they are blind to illegitimate actions that appear technically valid. This creates a defensive debt where enterprises are deploying highly flexible, unpredictable systems without any foundational mechanism to govern their planning or intent. We are moving from a world of “static rules” to a world of “dynamic intent,” and our security tools must follow suit.

💡 Digging Deeper

Q: Can’t we just use a traditional proxy with a policy engine?
A: Proxies are technically non-viable for many cloud-native AIs and don’t provide the semantic understanding needed to judge if an action is “correct” based on the agent’s goal.

Q: How do agent actions differ from traditional automation?
A: Automations follow a script; agents make autonomous choices based on fuzzy instructions, making their failure modes unpredictable.

The Guardian Model Approach

Specialized Oversight for Frontier Models

To solve the oversight problem without ballooning costs, Onyx Security trains specialized, smaller models designed to act as “guardians” for larger frontier models. Instead of running a massive GPT-4 instance to watch another GPT-4 instance, which is prohibitively expensive and slow, these small models are trained specifically to identify high-risk moments.

These specialized models act like a chess player’s intuition, sensing danger under time pressure without needing to calculate every possible future branch.

When the small model detects a potentially dangerous action—such as an agent attempting to modify a core security setting or publish code to a public repository—it triggers a more intelligent oversight process. This multi-tiered approach allows for low-latency monitoring while ensuring that high-risk actions are underwritten by significant computational intelligence. It balances the need for enterprise-grade safety with the reality that security vendors cannot cost more than the AI being secured.

💡 Digging Deeper

Q: Why can’t the foundation model labs (OpenAI/Anthropic) do this themselves?
A: Enterprises often refuse to share long-term historical behavioral data with labs who might use it for training; an independent third party provides a “clean room” for governance.

Q: What is the role of mechanistic interpretability here?
A: Onyx believes that understanding the internal weights and activations of models is the long-term path to truly controlling super-intelligent systems.

Key Takeaways

Enterprises are rapidly adopting autonomous coding agents because the productivity benefits are undeniable, despite the growing risks of accidental data destruction or token leaks. This adoption is moving away from low-code, restricted platforms toward unleashed, autonomous assistants that require a new kind of oversight.

Modern security requires moving beyond simple Data Loss Prevention to a sophisticated “Control Plane” that understands the intent and historical behavior of autonomous systems. Traditional firewalls and identity locks are insufficient for agents that must act with human-level permissions to be useful.

Onyx Security bridges the gap between deep cyber-intelligence and AI research by utilizing specialized, “small but smart” models to govern frontier agents. By focusing on the intersection of mathematical reasoning and adversarial security, they aim to provide the foundational governance layer needed for the transition toward AGI.

Q&A

Q1: What is the fastest-growing category of AI in the enterprise today?
A: Autonomous coding agents and assistants are currently the fastest-growing category, representing over 50% of adoption in many large companies.

Q2: How does Onyx view the Israeli talent pool in relation to AI?
A: The Israeli ecosystem offers a unique blend of military-grade adversarial thinking and advanced mathematical research, which is essential for solving AI alignment and control.

Q3: What is “Mythos-level” risk in cybersecurity?
A: It refers to the plummeting cost of automated vulnerability finding by AI coding tools, which could allow attackers to find and exploit software flaws at an unprecedented scale.

Q4: Will AI labs eventually solve the security problem themselves?
A: It is structurally difficult for labs to be their own auditors; buyers prefer independent third-party verification and a tool that works across multiple different AI vendors.

Q5: How does identity management change in the age of agents?
A: It becomes less effective as a primary control because agents require the same broad permissions as the human users they represent to perform diverse tasks.

Q6: What is the “Jagged Intelligence” problem?
A: The phenomenon where models can perform incredibly complex tasks but still make very “silly” or basic mistakes that a human wouldn’t, necessitating a safety net.

Q7: How does historical data help in securing agents?
A: By looking at how an agent has behaved in the past, a security layer can detect anomalies in intent, even if the specific API call itself appears valid.