Spring AI Tutorial: ChatClient, Advisors & Structured Output
This is post 7 of 10 in the AI for Java Developers track — Java-first, hype-free AI integration. New to the series? Start at the Java Learning Roadmap 2026 hub.
The customer problem
Your Spring Boot service already runs the company's ticketing API. Now support asks for an AI feature inside it: "Paste a ticket thread, get a one-paragraph summary and a category, so agents can route tickets faster." They don't want a new service, a new deploy pipeline, or a Python sidecar. They want an endpoint on the app they already have — one that feels like the rest of your Spring code: dependency injection, application.properties, and constructors you can unit-test.
That's exactly the gap Spring AI fills. You already know how to call an LLM over HTTP from plain Java; now you want the Spring way: auto-configuration, a fluent client that looks like RestClient, cross-cutting concerns expressed as advisors instead of scattered helper methods, and model output that lands in a Java record instead of a string you regex. By the end of this post you'll have one worked endpoint — POST /api/tickets/summarize — built this way.
OPENAI_API_KEY environment variable. Every sample model output below is labeled ILLUSTRATIVE — invented for this example, not a measured response. Spring AI moves fast: coordinates and property names below were checked against current community sources in October 2026, but verify anything marked "check current docs" before you depend on it.What Spring AI actually is
Spring AI is Spring's portable abstraction over AI model providers. The idea mirrors what JDBC did for databases or what Spring Data did for data stores: you program against a stable interface — ChatModel, EmbeddingModel, VectorStore — and a provider-specific starter (OpenAI, Anthropic, Ollama, Gemini, Azure, and others) supplies the implementation. The value is uniformity: DI-friendly beans, auto-configuration, retry handling, and a fluent ChatClient API that reads like WebClient or RestClient:
String answer = chatClient.prompt()
.user("Summarize this ticket thread: ...")
.call()
.content();Dependencies: the BOM and one starter
Spring AI ships a Bill of Materials so all its artifacts stay on one consistent version. Add the BOM to <dependencyManagement>, then pull exactly one model starter for your provider:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bom</artifactId>
<!-- Verify the latest version before you build; 2.0.1 was current when checked. -->
<version>2.0.1</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<!-- Pick ONE provider starter: -->
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>
<!-- or, for a local model: -->
<!-- <artifactId>spring-ai-starter-model-ollama</artifactId> -->
</dependencies>spring-ai-openai-spring-boot-starter no longer resolve — if a tutorial from 2024 tells you to use them, that's a stale coordinate. Check current docs.Configuration in application.properties
The starter auto-configures a ChatModel bean from properties. The key never goes in the file — it stays in the environment:
# The only secret. Never commit a real key here.
spring.ai.openai.api-key=${OPENAI_API_KEY}
# Optional: pin the model explicitly so a provider-side default change
# never silently changes your app's behavior.
# Property name for the model option shifted between 1.x and 2.0 — check current docs.
spring.ai.openai.chat.options.model=gpt-4o-miniIf OPENAI_API_KEY is unset, the application fails fast at startup rather than at the first user request — fail-fast beats fail-at-3am. For local development without a key, the Ollama starter points at a local server by default and needs no key at all.
The ChatClient fluent API
Auto-configuration also provides a ChatClient.Builder. Best practice: build one client per use case, with its system prompt baked in at construction — not concatenated into every call:
package com.example.ai;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class AiConfig {
@Bean
ChatClient ticketSummarizer(ChatClient.Builder builder) {
return builder
.defaultSystem("""
You are a support-ticket triage assistant.
Reply with a JSON object only: { "category": "...", "priority": "...", "summary": "..." }.
Keep the summary under 60 words. If you cannot categorize, use category "UNKNOWN".
""")
.build();
}
}The client is then injected like any other bean. The prompt spec builds fluently — .system(), .user(), .call() — and the response offers .content() (plain string), .entity(Class) (structured output, next section), or .stream() for token-by-token Flux streaming:
package com.example.ai;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.stereotype.Service;
@Service
public class TicketTriageService {
private final ChatClient summarizer;
public TicketTriageService(ChatClient ticketSummarizer) {
this.summarizer = ticketSummarizer;
}
public String summarize(String threadText) {
return summarizer.prompt()
.user("Ticket thread to summarize:\n\n" + threadText)
.call()
.content();
}
}Because TicketTriageService depends on ChatClient — an interface you can stub — the class is unit-testable without a provider key. That's the payoff of the Spring way: your business logic stays a plain constructor-injected bean.
Advisors: the cross-cutting-concern pattern
Advisors are Spring AI's answer to a familiar problem: logging, input guardrails, retries, and memory are concerns that cut across every AI call, and scattering them through each service method is how you get five inconsistent implementations. An advisor wraps the request chain like a servlet filter or a Spring AOP around-advice — it can inspect or rewrite the prompt on the way in and the response on the way out, then call chain.nextCall(request) to continue.
Here's a minimal custom advisor: a guardrail that refuses to send prompt-injection-shaped input to the model. (Interface names: CallAdvisor / adviseCall are the 2.0 shape; the 1.x line used CallAroundAdvisor with AdvisedRequest — check current docs for your version.) First, one small unchecked exception both the guardrail and the validator share:
package com.example.ai;
/** Unchecked so advisors (whose interface methods declare no checked exceptions)
* and validators can both raise it. */
public class LlmException extends RuntimeException {
private final String code;
public LlmException(String code, String message) {
super(message);
this.code = code;
}
public LlmException(String code, String message, Throwable cause) {
super(message, cause);
this.code = code;
}
public String code() {
return code;
}
}package com.example.ai;
import org.springframework.ai.chat.client.ChatClientRequest;
import org.springframework.ai.chat.client.ChatClientResponse;
import org.springframework.ai.chat.client.advisor.api.CallAdvisor;
import org.springframework.ai.chat.client.advisor.api.CallAdvisorChain;
/** Guardrail: reject input that looks like prompt injection before any model call. */
public class InjectionGuardAdvisor implements CallAdvisor {
@Override
public String getName() {
return "InjectionGuardAdvisor";
}
@Override
public int getOrder() {
// Lower order = earlier on the way in. Guardrails run before logging.
return 10;
}
@Override
public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain chain) {
String input = request.prompt().getUserMessages().stream()
.map(m -> m.getText())
.reduce("", (a, b) -> a + "\n" + b);
if (looksLikeInjection(input)) {
throw new LlmException("GUARDRAIL_BLOCKED",
"Input refused by injection guardrail; not sent to the model.");
}
return chain.nextCall(request);
}
private boolean looksLikeInjection(String input) {
String lower = input.toLowerCase(java.util.Locale.ROOT);
return lower.contains("ignore all previous instructions")
|| lower.contains("reveal your system prompt");
}
}Register it once, globally, on the client builder — every call through that client now passes the guardrail:
@Bean
ChatClient ticketSummarizer(ChatClient.Builder builder) {
return builder
.defaultAdvisors(new InjectionGuardAdvisor())
.defaultSystem("You are a support-ticket triage assistant. ...")
.build();
}You can also attach an advisor to a single call with .advisors(...) on the prompt spec — useful for per-endpoint concerns like a one-off audit log.
getOrder(), lowest first, and the chain is symmetric like a filter — the first advisor on the way in is the last on the way out. Put guardrails (low order) ahead of logging and retry logic, so a blocked call is logged as blocked, and a retried call's audit trail sees each attempt.Structured output: from text to a Java type
Real applications rarely want prose — they want fields. Spring AI's structured output converts a model reply into a Java type. The shortcut on the fluent API is .call().entity(MyRecord.class): Spring AI appends JSON-schema instructions to the prompt and parses the reply with Jackson:
public record TicketSummary(String category, String priority, String summary) { }
public TicketSummary summarizeStructured(String threadText) {
return summarizer.prompt()
.user("Summarize this ticket thread:\n\n" + threadText)
.call()
.entity(TicketSummary.class);
}For explicit control there is BeanOutputConverter, which generates the format instructions you inject into the prompt yourself and then parses via .call().entity(converter):
import org.springframework.ai.converter.BeanOutputConverter;
var converter = new BeanOutputConverter<>(TicketSummary.class);
TicketSummary summary = summarizer.prompt()
.user(u -> u.text("Summarize this ticket thread:\n\n{thread}\n\n{format}")
.param("thread", threadText)
.param("format", converter.getFormat()))
.call()
.entity(converter);package com.example.ai;
import java.util.Set;
/** Pure-Java validation of model-produced structured output. */
public final class TicketSummaryValidator {
private static final Set<String> CATEGORIES =
Set.of("BILLING", "TECHNICAL", "ACCOUNT", "SHIPPING", "UNKNOWN");
private static final Set<String> PRIORITIES = Set.of("LOW", "MEDIUM", "HIGH");
private TicketSummaryValidator() { }
public static TicketSummary validate(TicketSummary candidate) throws LlmException {
if (candidate == null) {
throw new LlmException("EMPTY_RESPONSE", "Model returned no structured output.");
}
String category = nullSafe(candidate.category()).toUpperCase(java.util.Locale.ROOT);
String priority = nullSafe(candidate.priority()).toUpperCase(java.util.Locale.ROOT);
String summary = nullSafe(candidate.summary());
if (!CATEGORIES.contains(category) || !PRIORITIES.contains(priority)
|| summary.isBlank() || summary.length() > 500) {
throw new LlmException("INVALID_OUTPUT",
"Model output failed validation; discarding it.");
}
return new TicketSummary(category, priority, summary);
}
private static String nullSafe(String value) {
return value == null ? "" : value.strip();
}
}The worked example: POST /api/tickets/summarize
Now the endpoint support actually asked for. Note the error handling: Spring AI distinguishes transient failures (rate limits, timeouts — retryable) from non-transient ones (bad key, invalid request — retrying is pointless), surfacing them as TransientAiException and NonTransientAiException. The controller maps each to the HTTP status the client can act on:
package com.example.ai;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.retry.NonTransientAiException;
import org.springframework.ai.retry.TransientAiException;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequestMapping("/api/tickets")
public class TicketController {
private final ChatClient summarizer;
public TicketController(ChatClient ticketSummarizer) {
this.summarizer = ticketSummarizer;
}
public record SummarizeRequest(String threadText) { }
public record SummarizeResponse(String category, String priority, String summary) { }
@PostMapping("/summarize")
public SummarizeResponse summarize(@RequestBody SummarizeRequest request) {
String thread = request.threadText();
if (thread == null || thread.isBlank()) {
throw new BadTicketRequestException("threadText must not be empty.");
}
if (thread.length() > 20_000) {
throw new BadTicketRequestException("threadText exceeds 20000 characters.");
}
try {
TicketSummary raw = summarizer.prompt()
.user("Summarize this ticket thread:\n\n" + thread)
.call()
.entity(TicketSummary.class);
TicketSummary checked = TicketSummaryValidator.validate(raw);
return new SummarizeResponse(checked.category(), checked.priority(), checked.summary());
} catch (LlmException e) {
// Guardrail blocks and validation failures: 422, nothing to retry.
throw new UnprocessableTicketException(e.getMessage(), e);
} catch (NonTransientAiException e) {
// Auth errors, bad requests: our problem, don't hammer the provider.
throw new ProviderMisconfiguredException("AI provider rejected the request.", e);
} catch (TransientAiException e) {
// Rate limits, timeouts: 503 so the caller can back off and retry.
throw new ProviderUnavailableException("AI provider temporarily unavailable.", e);
}
}
@ResponseStatus(HttpStatus.BAD_REQUEST)
public static class BadTicketRequestException extends RuntimeException {
public BadTicketRequestException(String message) { super(message); }
}
@ResponseStatus(HttpStatus.UNPROCESSABLE_ENTITY)
public static class UnprocessableTicketException extends RuntimeException {
public UnprocessableTicketException(String message, Throwable cause) { super(message, cause); }
}
@ResponseStatus(HttpStatus.INTERNAL_SERVER_ERROR)
public static class ProviderMisconfiguredException extends RuntimeException {
public ProviderMisconfiguredException(String message, Throwable cause) { super(message, cause); }
}
@ResponseStatus(HttpStatus.SERVICE_UNAVAILABLE)
public static class ProviderUnavailableException extends RuntimeException {
public ProviderUnavailableException(String message, Throwable cause) { super(message, cause); }
}
}{"category": "BILLING", "priority": "HIGH", "summary": "Customer was charged twice for invoice INV-2291 after a retry during checkout. They want one charge reversed and confirmation sent to billing@example.com."}When it breaks: 4 traps
| Trap | What happens | The fix |
|---|---|---|
| The abstraction leaks at the provider boundary. | You develop against one provider; in production the other rejects tool-call message ordering, caps tokens differently, or returns 400 on a system role you relied on. | Treat the provider as a real integration: keep provider options in config, and run your endpoint tests (at least smoke-level) against every provider you ship. "Portable is not identical" is the principle for a reason. |
| Advisor ordering surprises. | Two advisors with the same default order run in an unspecified sequence — or your logging advisor sits outside the retry advisor and each retry logs as a new request. | Give every advisor an explicit getOrder(). Guardrails first (lowest), then caching/audit, then retry closest to the model — and write a test asserting the order your chain resolves to. |
| Assuming structured output is validated. | .entity(TicketSummary.class) throws if the JSON is malformed — but silently returns a bean with a plausible-but-wrong category when the model invents one. |
Validate every model-produced value against your domain's allowed set (like TicketSummaryValidator above). Parsing is infrastructure; trust is your code. |
| Version-pairing drift. | Spring AI 2.x on Spring Boot 3 (or vice versa) compiles fine and then fails at startup with missing beans or NoSuchMethodError — the kind of error that looks like your code is wrong. |
Pin the pairing in your build: Boot 4 with AI 2.0, Boot 3 with AI 1.x, one provider starter, one BOM version. When you upgrade Boot, check the AI line before anything else. |
Wrapping up
You now have the Spring-idiomatic way to put an AI feature in a Spring Boot service: one starter, one BOM, a per-use-case ChatClient built by @Bean, cross-cutting concerns as ordered advisors, and structured output validated like any untrusted input. The endpoint is a plain controller with constructor injection — testable, configurable, and boring in the best sense.
Cost and timeout discipline still matter: set HTTP timeouts on the underlying client, watch token usage per endpoint, and remember that one slow provider response can hold a request thread for a long time. Continue the track at the Java Learning Roadmap 2026 hub.
Comments
Post a Comment