📺 Today’s recommended deep-dive video: https://www.youtube.com/watch?v=_ZZuzakjgxQ

Beyond the Chatbox: Mastering Agentic Patterns with Spring AI

Most developers treat Large Language Models (LLMs) as simple text-in, text-out pipes, but building production-ready enterprise applications requires much more control. By mastering the “Advisor” pattern and progressive disclosure, you can transform a stateless black box into a sophisticated, stateful agent capable of self-correction and complex reasoning.

Core Question: How can a few small architectural concepts in Spring AI be used to implement nearly any complex generative AI pattern?

Highlights

• The “Advisor” as the fundamental unit for intercepting and augmenting LLM inputs and outputs.
• Moving tool-calling logic to the advisor layer to improve observability and context management.
• Progressive tool disclosure: Using a “Tool Search Tool” to manage thousands of functions without overwhelming token limits.
• Orchestration patterns like “To-Do” lists and sub-agents that force LLMs to follow structured, multi-step logic.

⏱️ Reading time: approx. 8 minutes · Saves you about 44 minutes vs. watching.

Want to take notes while watching? Click the image below and let AI Notebook capture the key points for you 👇

The Core Abstraction: The Advisor

Intercepting the Black Box

Large Language Models are inherently stateless, non-deterministic, and limited by context windows. To build a functional application, we must treat the LLM as a black box and surround it with a “harness” that manages state and validation.

In Spring AI, the most powerful tool in your belt is the Advisor. Think of it as an interceptor or a filter, similar to those found in web frameworks, that sits on the path of communication between your code and the LLM. It allows you to augment what goes in—like injecting chat history or RAG context—and intercept what comes out to ensure quality.

Everything in the modern Spring AI ecosystem is being built on top of this simple concept.

By using advisors, you can implement memory, guardrails, and even complex tool-calling logic without cluttering your core business service. It separates the “intelligence” of the model from the “plumbing” of your infrastructure.

💡 Digging Deeper

Q: Why move tool calling into an Advisor instead of the Chat Model?
A: Historically, tool calling was buried deep in the model layer, making it invisible. Moving it to an advisor allows you to observe, log, and even intercept tool requests before they are executed.

Q: Can multiple advisors be used at once?
A: Yes, they can be chained. However, the order is critical; for example, a memory advisor should usually be placed inside a tool-calling advisor to ensure it captures the intermediate steps of a function call.

Q: Is an advisor the same as a Spring AOP aspect?
A: While similar in spirit, Spring AI Advisors are specifically designed for the request/response flow of LLM interactions, offering a more tailored API for manipulating chat messages and metadata.

Structured Output and Guardrails

Enforcing Predictability in a Creative Medium

LLMs are creative, which is another way of saying they are prone to hallucinating or ignoring formatting instructions. Even when you ask for JSON, there is a 1% chance the model will return malformed text that breaks your parser.

Spring AI solves this by using a Structured Output Advisor. This component doesn’t just ask for JSON; it validates the response against a schema and, if the validation fails, it automatically sends the error back to the LLM for a second attempt.

This self-correction happens behind the scenes.

The user never sees the failure. They only see the successful, structured result. This loop of “Act, Validate, Correct” is the hallmark of a resilient agentic system.

Advanced Guardrails with “Judge” Models

Sometimes deterministic validation isn’t enough. You might need to check if a response is helpful or if it contains sensitive information that shouldn’t leave the enterprise.

In these cases, you can plug in a “Judge” LLM advisor. This is often a smaller, faster, and cheaper model fine-tuned specifically to grade the output of the primary model. If the judge gives a “thumbs down,” the advisor intercepts the response and prompts the main model to try again with specific feedback.

Progressive Disclosure and Tool Search

Solving the “Too Many Tools” Problem

If you have a thousand tools—common in environments using Model Context Protocol (MCP) servers—you cannot send them all to the LLM at once. Doing so would skyrocket your token costs and confuse the model, leading to “intent drift.”

The solution is Progressive Tool Disclosure. Instead of registering 1,000 tools, you register one: the “Tool Search Tool.”

The LLM is told that if it needs help, it should call this search tool.

Based on the user’s intent, the search tool queries a local index of your 1,000 tools and “discloses” only the five most relevant ones to the LLM. The model then makes a second call using the newly discovered tools. This technique can reduce token consumption by over 50% while improving accuracy.

Orchestrating Complex Tasks

To-Do Lists and Sub-Agents

When an LLM faces a complex, multi-step problem, it can lose the plot halfway through. To prevent this, you can implement the “To-Do Right” pattern.

This pattern forces the LLM to create a structured to-do list before starting the work. Each step has a lifecycle (Pending, In Progress, Completed). The system message enforces a rule: the model cannot move to step two until step one is marked as completed in the conversation history.

This keeps the agent focused and ensures no steps are skipped.

Finally, for the most complex tasks, we use Sub-Agents. A sub-agent runs in an isolated context with its own set of tools. It performs a specific sub-task and returns only the final answer to the Lead Agent. This prevents the Lead Agent’s context from being polluted by the “messy” intermediate steps of every minor calculation or search.

Key Takeaways

We are moving rapidly away from simple prompt engineering and toward “Context Engineering” and “Agent Harnessing.” The goal is no longer just to write a better prompt, but to build a robust architectural framework around the model that manages its state, validates its logic, and optimizes its resources.

Spring AI’s Advisor pattern is the key to this evolution. By treating capabilities like memory, RAG, and tool calling as interceptors, you create a modular system where you can swap models, update guardrails, and scale toolsets without rewriting your core logic.

The future of AI development isn’t about the “black box” itself; it’s about everything else you build to keep that black box in check. Whether it’s through progressive disclosure of tools or the use of Judge LLMs to ensure quality, the patterns discussed here provide the blueprint for building reliable, enterprise-grade AI agents.

Q&A

Q1: What are the main characteristics of LLMs that make advisors necessary?
A1: LLMs are stateless, pre-trained on old data, produce non-structured text, are non-deterministic (prone to hallucinations), and have limited token windows for data exchange.

Q2: How does the “To-Do Right” pattern prevent the LLM from getting confused?
A2: It enforces a strict state machine (Pending -> In Progress -> Completed) on tasks, requiring the LLM to update the status of each step before proceeding, which anchors its reasoning in the conversation history.

Q3: What is “Progressive Disclosure” in the context of tools?
A3: It is the practice of only providing the LLM with the metadata for tools it actually needs for the current task, rather than overwhelming its context window with every available function in the system.

Q4: What is the benefit of the “Augmentor” pattern for tool calling?
A4: It allows you to hijack tool calls to add hidden parameters, like “inner thoughts” or “reasoning steps,” which the LLM fills out. This provides better observability into why an agent decided to call a specific tool.

Q5: Can I run shell scripts or Python code using Spring AI?
A5: Yes, via “Agent Skills.” This allows the LLM to use tools like Bash or Python to solve problems, though the speaker strongly recommends running these in a sandboxed or containerized environment for security.

Q6: What happens if the Structured Output Advisor fails to correct the LLM?
A6: You can configure the number of retry attempts. If it fails after the maximum number of attempts, the system will eventually return an error, but in practice, most models correct their formatting on the second try.

Q7: Is Spring AI compatible with different LLM providers like Anthropic and OpenAI?
A7: Yes, Spring AI is designed as a portable abstraction. Patterns like Advisors and Tool Calling work across various providers, including Anthropic, OpenAI, Google Gemini, and Bedrock.