LangChain4j Tutorial: Chains, Agents & Tools in Java

By -

Your hand-rolled RAG pipeline works. It embeds your documents, searches for the best matches, stuffs them into a prompt template, and calls the model. It also looks like this: 400 lines of StringBuilder calls, a prompt template held together with + "\n\nContext:\n" +, and a parseAnswer() method that fails in a new and exciting way every third release. Then product asks for a second data source — the support ticket archive alongside the policy docs — and you realize adding it means rewriting the prompt assembly by hand. Again.

This is the moment frameworks earn their keep. Not by making the model smarter, but by taking the assembly work — prompt building, history stitching, retrieval injection, tool-call loops — out of your string-munging code and into something you can read, test, and change. LangChain4j is the Java-native way to do that.

The principle for this lesson: the framework assembles prompts; you still own what goes in them. LangChain4j will happily build a prompt from your interface, your retriever, and your memory — but the quality of each ingredient is still your job.

What LangChain4j is (and isn't)

LangChain4j is a Java library for orchestrating LLM-powered applications: chaining model calls together, giving the model tools to call, and wiring retrieval (RAG) and memory into the flow. It is inspired by the same ideas as Python's LangChain — chains, agents, tools — but it is a separate Java codebase, not a port of the Python library. It is Apache 2.0 licensed and integrates with the Java ecosystem you already use (plain Maven projects, Spring Boot, Quarkus).

One honest framing before we start, straight from the project's own docs: LangChain4j's Chains are legacy. The early chain classes were the first answer to "compose model calls," and the maintainers now recommend a different pattern — AI Services — as the Java-native way to orchestrate. "Agents," in this ecosystem, mostly means a model in a tool-calling loop: the model decides which of your Java methods to invoke, the framework executes them and feeds the results back. We'll show the shape of tools in this post; the full mechanics come later.

Also worth knowing: LangChain4j moves fast and renames things. In the 1.0 release, ChatLanguageModel became ChatModel, generate() became chat(), and builder methods like chatLanguageModel(...) became chatModel(...). If you learned LangChain4j from a 2024 tutorial, half the class names have changed. This post uses the current 1.x API.

Setup: Maven coordinates

LangChain4j ships as one module per integration, so you only pull in what you need. Two artifacts cover this post: the core (langchain4j, carrying the AI Services machinery) and the OpenAI integration (langchain4j-open-ai), both from group dev.langchain4j:

<properties>
    <!-- Check Maven Central / the docs for the latest 1.x; 1.20.0 is the stable line as of this writing. -->
    <langchain4j.version>1.20.0</langchain4j.version>
    <maven.compiler.release>21</maven.compiler.release>
</properties>

<dependencies>
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j</artifactId>
        <version>${langchain4j.version}</version>
    </dependency>
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-open-ai</artifactId>
        <version>${langchain4j.version}</version>
    </dependency>
</dependencies>

Keep every LangChain4j module on the same version — one property, referenced everywhere (there is also a langchain4j-bom if you prefer a bill of materials). Mixing versions across modules is the single most common source of "it compiled but throws at runtime" (more on that in the traps section). The library needs Java 17 or newer; we're on 21.

One more setup habit, before any model call: fail fast when the API key is missing. A null key produces a confusing 401 three layers deep; a clear error at startup saves everyone time:

public final class ApiKeys {
    private ApiKeys() {}

    public static String openAiKey() {
        String key = System.getenv("OPENAI_API_KEY");
        if (key == null || key.isBlank()) {
            throw new IllegalStateException(
                "OPENAI_API_KEY is not set. Export it before running: " +
                "export OPENAI_API_KEY=sk-...");
        }
        return key;
    }
}

Your first model call: ChatModel

The core abstraction is ChatModel — one interface, many providers. You build a provider-specific model (here, OpenAI), and the rest of your code depends only on the interface, so switching providers later is a builder swap:

import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.openai.OpenAiChatModel;

public class FirstCall {
    public static void main(String[] args) {
        ChatModel model = OpenAiChatModel.builder()
                .apiKey(ApiKeys.openAiKey())
                .modelName("gpt-4o-mini")
                .build();

        String answer = model.chat("In one sentence, what is a vector embedding?");
        System.out.println(answer);
    }
}

model.chat(String) returns the model's reply as a String — the simple overload for the simple case. There are richer overloads that take a list of messages and return a full ChatResponse (with token usage), which you'll want once you start tracking cost.

The standout feature: AiServices

Here is the pattern that makes LangChain4j worth learning. Instead of assembling prompts by hand, you declare a Java interface and let the library generate the implementation — the same idea as Spring Data JPA repositories or Retrofit clients:

import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;

interface PolicyAssistant {

    @SystemMessage("You are a support agent for Acme Outdoors. " +
                   "Answer using only the provided context. " +
                   "If the answer is not in the context, say so.")
    String answer(@UserMessage String question);
}

PolicyAssistant assistant = AiServices.create(PolicyAssistant.class, model);
String reply = assistant.answer("What is the return window for tents?");
// (illustrative) "Tents can be returned within 30 days of purchase."

Behind the scenes, AiServices creates a proxy. When you call answer(...), the proxy turns the argument into a UserMessage, prepends the SystemMessage from the annotation, sends both to the ChatModel, and converts the reply back into a String. The prompt assembly you used to hand-roll is now declarative — and here is the flow:

Your code calls answer("...") AiServices proxy assembles the prompt, sends it, parses reply, loops on tool calls ChatModel (OpenAiChatModel) String answer @SystemMessage template from annotation Retrieved segments from ContentRetriever Chat history from ChatMemory @Tool Java methods model requests a call, proxy executes, feeds result back

Prompt templates support variables. {{it}} refers to the single method argument; with several arguments, name them with @V:

import dev.langchain4j.service.V;

interface Translator {
    @UserMessage("Translate {{text}} to {{language}}. Reply with only the translation.")
    String translate(@V("text") String text, @V("language") String language);
}

The proxy also handles richer return types: boolean for classification, enums, dates, even POJOs (structured extraction — the proxy asks the model for JSON and deserializes it). Every one of these is a parse that can fail at runtime; the proxy throws, it doesn't return null. Remember that in the traps section.

Worked example: document QA with RAG

Let's rebuild the scenario from the opening — a document question-answerer — with LangChain4j doing the assembly. The knowledge base is small enough to inline: three policy snippets for a fictional outdoor store. In a real project these would come from files or a database; the mechanics don't change.

Step 1 — ingest the documents. We embed each text segment and store it. EmbeddingStoreIngestor handles the embed-and-store loop, and InMemoryEmbeddingStore keeps the demo self-contained (swap it for a persistent vector store later without touching the rest):

import dev.langchain4j.data.document.Document;
import dev.langchain4j.data.segment.TextSegment;
import dev.langchain4j.model.embedding.EmbeddingModel;
import dev.langchain4j.model.openai.OpenAiEmbeddingModel;
import dev.langchain4j.store.embedding.EmbeddingStore;
import dev.langchain4j.store.embedding.EmbeddingStoreIngestor;
import dev.langchain4j.store.embedding.inmemory.InMemoryEmbeddingStore;
import java.util.List;

public class PolicyDocs {

    public static List<String> policies() {
        return List.of(
            "Returns: Tents, sleeping bags, and backpacks can be returned within " +
            "30 days of purchase with a receipt. Items must be unused and in " +
            "original packaging. Clearance items are final sale.",
            "Shipping: Orders placed before 2pm ship the same business day. " +
            "Standard shipping takes 3-5 business days. Express shipping takes " +
            "1-2 business days and costs $12.95.",
            "Warranty: All Acme Outdoors tents carry a 2-year warranty against " +
            "manufacturing defects. Normal wear, UV damage, and improper setup " +
            "are not covered."
        );
    }

    public static EmbeddingStore<TextSegment> buildStore(EmbeddingModel embeddingModel) {
        EmbeddingStore<TextSegment> store = new InMemoryEmbeddingStore<>();
        for (String policy : policies()) {
            Document document = Document.from(policy);
            EmbeddingStoreIngestor.builder()
                    .embeddingModel(embeddingModel)
                    .embeddingStore(store)
                    .build()
                    .ingest(document);
        }
        return store;
    }
}

Step 2 — wire the retriever into an AI Service. EmbeddingStoreContentRetriever embeds the user's question, finds the most similar segments, and the AI Service injects them into the prompt automatically:

import dev.langchain4j.rag.content.retriever.ContentRetriever;
import dev.langchain4j.rag.content.retriever.EmbeddingStoreContentRetriever;
import dev.langchain4j.service.AiServices;

public class DocumentQa {

    interface DocQa {
        @SystemMessage("Answer using only the retrieved context. " +
                       "If the context does not contain the answer, say " +
                       "'I don't know based on the provided documents.'")
        String answer(String question);
    }

    public static void main(String[] args) {
        EmbeddingModel embeddingModel = OpenAiEmbeddingModel.builder()
                .apiKey(ApiKeys.openAiKey())
                .modelName("text-embedding-3-small")
                .build();

        ChatModel chatModel = OpenAiChatModel.builder()
                .apiKey(ApiKeys.openAiKey())
                .modelName("gpt-4o-mini")
                .build();

        EmbeddingStore<TextSegment> store = PolicyDocs.buildStore(embeddingModel);

        ContentRetriever retriever = EmbeddingStoreContentRetriever.builder()
                .embeddingStore(store)
                .embeddingModel(embeddingModel)
                .maxResults(2)
                .minScore(0.6)
                .build();

        DocQa qa = AiServices.builder(DocQa.class)
                .chatModel(chatModel)
                .contentRetriever(retriever)
                .build();

        System.out.println(qa.answer("How long do I have to return a tent?"));
        // (illustrative) "Tents can be returned within 30 days of purchase
        //  with a receipt, unused and in original packaging."

        System.out.println(qa.answer("Do you sell bicycles?"));
        // (illustrative) "I don't know based on the provided documents."
    }
}

Compare this to the hand-rolled version: no manual cosine similarity, no prompt string assembly, no separate retrieval step in your code. The ingestion is ~20 lines, the wiring is ~15. The second data source from the opening scenario — the ticket archive — is now a second EmbeddingStore and a retriever that searches both. That is the payoff: the framework owns the plumbing, and each new source is a component, not a rewrite. One tuning note: maxResults and minScore are knobs you tune against your own documents — there is no universally correct setting.

Tools: the shape of function calling

Sometimes the model needs more than documents — it needs to do something: check inventory, compute shipping, look up an order. LangChain4j exposes your Java methods to the model with the @Tool annotation. Here is the shape to recognize (the full mechanics of the tool-execution loop get their own treatment):

import dev.langchain4j.agent.tool.Tool;

public class StoreTools {

    @Tool("Estimate standard shipping days for a destination country")
    public int shippingDays(String country) {
        if (country == null || country.isBlank()) {
            throw new IllegalArgumentException("country must not be blank");
        }
        return switch (country.trim().toLowerCase()) {
            case "usa", "canada" -> 5;
            case "uk", "germany", "france" -> 8;
            default -> 12;
        };
    }
}

DocQa qaWithTools = AiServices.builder(DocQa.class)
        .chatModel(chatModel)
        .contentRetriever(retriever)
        .tools(new StoreTools())
        .build();

The loop the proxy runs: the model receives the tool's description, decides to call it, the proxy executes your Java method with the model's chosen arguments, and the result goes back to the model for the final answer. Notice the validation inside shippingDays — that is not decoration. The model chooses the argument values, so a @Tool method is a public entry point: validate inputs, keep side effects deliberate, and never expose a destructive operation casually. The honest version of "agents" is this loop, not magic.

Memory: conversations that remember

A plain AI Service call is stateless — every call is a fresh conversation. For back-and-forth dialogue, attach a ChatMemory. MessageWindowChatMemory keeps the last N messages and the proxy appends history to each prompt automatically. With @MemoryId, each user (or session) gets their own history:

import dev.langchain4j.memory.chat.MessageWindowChatMemory;
import dev.langchain4j.service.MemoryId;

interface SupportChat {
    String chat(@MemoryId String userId, @UserMessage String message);
}

SupportChat chat = AiServices.builder(SupportChat.class)
        .chatModel(chatModel)
        .chatMemoryProvider(memoryId ->
                MessageWindowChatMemory.builder()
                        .id(memoryId)
                        .maxMessages(20)
                        .build())
        .build();

chat.chat("user-42", "I bought a tent last month.");
chat.chat("user-42", "What is the return window?");
// The second call sees the first: the model knows "the tent" refers
// to the purchase (illustrative behavior).

Memory is convenient and quietly expensive: every turn resends the window, so token usage grows with conversation length. Cap the window, and for anything serious, persist memories in a real store.

When it breaks: 4 LangChain4j traps

Trap 1 — Version drift between modules

Symptom: everything compiles, and at runtime you get NoSuchMethodError or a proxy that throws on build(). The cause is almost always mixed module versions — langchain4j at 1.20.0 and langchain4j-open-ai at some older release pulled in transitively or pinned by a stale property. The modules evolve together and assume matching internals. The fix is the one property from the setup section (or the BOM): every dev.langchain4j artifact on the same version, upgraded together in one commit.

Trap 2 — Hiding prompts inside annotations where you can't version them

Symptom: the support team's answers drift, nobody knows which prompt text is live, and the "fix" is editing a string literal in an annotation and redeploying. @SystemMessage("...") with a long inline string is convenient for a demo and a liability in production: annotation strings can't be diffed against the last release, can't be reviewed by a non-developer, and can't be A/B tested.

LangChain4j gives you the escape hatch: load the prompt from a resource file — @SystemMessage(fromResource = "prompts/doc-qa-system.txt") (and the same fromResource on @UserMessage). Prompts become versioned files, reviewed in pull requests like any other text asset. Decision rule: the annotation is wiring; the prompt text lives in a file. If a prompt matters enough to tune, it matters enough to version.

Trap 3 — Assuming the declarative proxy validates anything

Symptom: your Sentiment analyze(String text) interface works in testing, then throws in production on the one review where the model returns "positive-ish" instead of a valid enum value. The proxy converts the method argument into a prompt and parses the reply into your return type — and parsing is a runtime operation with runtime failure modes. Structured extraction (POJOs, enums, dates) is a request to the model, not a guarantee.

Treat every AI Service method like a call to an unreliable external system: expect parsing exceptions, decide what "degraded" means (retry with a stricter instruction? fall back to a human queue? log and skip?), and never let a classification result drive an irreversible action without a second check. The proxy handles the plumbing; the error policy is yours.

Trap 4 — Forgetting the model chooses the tool arguments

Symptom: a @Tool method that emails customers gets called with a hallucinated address, or a lookup tool receives an empty string and your downstream code throws. It is easy to read .tools(new StoreTools()) as "I wired up a helper" and forget what actually happens: the model decides when to call the tool and what arguments to pass, based on the conversation. The tool description you write in the annotation is the model's only documentation.

So write tool methods like public API endpoints: validate every argument, keep the description precise about expected formats ("country as a full English name, e.g. 'Germany'"), prefer read-only tools while you're learning, and keep anything with real-world side effects behind an explicit confirmation step. The framework executes faithfully; it does not judge wisely.

What's next

You now have the full LangChain4j loop in one place: a ChatModel behind a provider-neutral interface, AiServices turning interfaces into working assistants, retrieval wired in as a component instead of string surgery, tools sketched as annotated methods, and memory giving conversations continuity. The honest summary is smaller than the feature list: the library takes over prompt assembly, history stitching, retrieval injection, and the tool-call loop — and hands you back clean seams where your judgment goes: the prompt text, the retrieval tuning, the tool contracts, the error policy.

Field check before you move on: take the DocumentQa example, swap the three inline policies for five paragraphs from a document you actually care about, and ask three questions — one it answers, one it half-answers, one it doesn't. Where it shines and where it hedges is the retrieval-tuning intuition no tutorial can give you.

Continue: Java Learning Roadmap 2026

Comments

Post a Comment

Popular posts from this blog

JSP Servlet Interview Questions For Freshers Series 1

By -
Image
We know that while preparing for an interview there are lots of things needs to keep in your mind, and for freshers not easy to cover all the questions of Core Java, Servlet, JSP, Database, Aptitude, Algorithm etc. So we have planned to design few very popular interview question askeds in an interview for freshers on different series based on there priority and of course priority start from series 1. In our previous post we have listed very popular 20 Core Java basic interview questions for freshers . Friends don't worry we will post maximum number of interview questions on JSP Servlet but, started from very popular and most respective questions. JSP Servlet Interview Questions For Freshers Series 1 1. What is the life cycle of Servlet? Yes, this is very popular interview question and interviewer expect from you to at least list down the process involved while serving request for a Servlet. Well, Servlet life cycle is managed by Servlet Container and javax.servlet.Servlet in...
Read more

What are the core classes and interfaces of hibernate 4

By -
Hibernate is an Open source Object Relational Mapping framework. It enables developer to develop classes to be persist in an object oriented way including inheritance and the Java collections framework. Current hibernate stable release is 4.3.5 that was released in April 2014. It has various classes and interfaces, but sometimes interviewer ask you to list down the core classes and interfaces of hibernate. Well hibernate has various classes and interfaces, but when we say core classes and interfaces those are listed below, they all are the exceptional role in hibernate library. Configuration (class) ServiceRegistry (interface) SessionFactory (interface) Session (interface) Transaction (interface) Query (interface) Configuration: Is a class resides inside the org.hibernate.cfg package. Configuration as it's name implies it is all about the configuration of the application, configuration of the application done only at the initialization time which is only once for t...
Read more

Java Banking Finance Services and Insurance (BFSI) domain interview questions

By -
Image
Banking, Financial Services and Insurance ( BFSI ) is a banking domain, nowadays in IT industry high pay, stability and most respected domain after R&D, which includes verity of products and services such as commercial banks, insurance companies, non-banking financial companies, cooperatives, pensions funds, mutual funds etc. In BFSI domain mostly questions are comes from the Class loader, Thread, Concurrency api, Array, Collection, JVM architecture, Java Memory Model, String pool, String, StringBuilder, StringBuffer, and Puzzle. Normally interviewer will ask you to rate yourself in the above given API's, it is again depends on interviewer state of mind. Interviewer: Ok lets start with Collection, What is the different between ArrayList and LinkedList ? The demand of this question is to just make you comfort while in a interview ground. Smart Candidate :) ArrayList and LinkedList both are implementation of List interface. ArrayList implementation is based on dyna...
Read more