How to Build AI Agents in Java with Koog Framework

Mar 25, 2026

Purpose

I wanted to build an AI agent for a Java backend service. Most AI agent frameworks are Python-based—LangChain, CrewAI, AutoGen—all assume you’re writing Python. But my codebase is Java. I didn’t want to spin up a separate Python microservice just for AI capabilities.

Then I found Koog, JetBrains’ Java-based AI agent framework. It lets you define tools using Java annotations and build agents with a fluent API. No Python required.

The Setup

First, I needed to understand how Koog works. The framework has three main pieces:

Tools — Java methods you annotate so the LLM knows what they do
LLM Provider — Configuration for OpenAI, Anthropic, or other model providers
Agent — Combines tools, prompts, and LLM into something you can run

Defining Tools

I started by defining tools. The key is using @Tool and @LLMDescription annotations. These tell the LLM what your method does and what parameters it expects.

public class BankingTools implements ToolSet {
    @Tool
    @LLMDescription("Sends money to a recipient")
    public Boolean sendMoney(
        @LLMDescription("Unique identifier of the recipient")
        String recipientId,
        Integer amount
    ) {
        return true; // Your implementation here
    }

    @Tool
    @LLMDescription("Account balance in $")
    public Integer getAccountBalance(String userId) {
        return 1000000; // Your implementation here
    }
}

The @LLMDescription is critical. The LLM reads these descriptions to understand when and how to call your tools. If the description is vague, the agent will make wrong decisions.

I initially wrote @LLMDescription("Send money") — too brief. The LLM didn’t know it needed a recipient ID and amount. I rewrote it to "Sends money to a recipient" and added parameter descriptions. Much better results.

Configuring the LLM Provider

Next, I needed to connect to an LLM. Koog supports multiple providers: OpenAI, Anthropic, Google, DeepSeek, and Ollama.

var promptExecutor = new MultiLLMPromptExecutor(
    new OpenAILLMClient("OPENAI_API_KEY"),
    new AnthropicLLMClient("ANTHROPIC_API_KEY")
);

You can configure multiple providers and let your agent choose. For my use case, I just needed OpenAI.

A mistake I made: I forgot to set the API key as an environment variable and hardcoded it initially. Don’t do that. Use System.getenv("OPENAI_API_KEY") or your preferred secrets management.

Building the Agent

Now I combined everything into an agent.

var bankingAgent = AIAgent.builder()
    .promptExecutor(promptExecutor)
    .llmModel(OpenAIModels.Chat.GPT5_2)
    .systemPrompt("You're a banking assistant")
    .toolRegistry(
        ToolRegistry.builder()
            .tools(new BankingTools())
            .build()
    )
    .build();

The builder pattern feels natural in Java. You specify:

Which LLM to use (promptExecutor + model)
What system prompt sets the agent’s behavior
Which tools the agent can call

Running the Agent

Finally, I ran the agent with a task:

bankingAgent.run("Send 100$ to my friend Mike (mike_1234) if I have enough money");

The agent parses the task, checks the account balance using getAccountBalance, decides if there’s enough money, and calls sendMoney if appropriate. All the reasoning happens inside the LLM based on your tool descriptions.

What I Learned

The @LLMDescription annotations are the most important part. Write clear, specific descriptions. The LLM has no other way to understand your tools.

Specify your model explicitly. I forgot .llmModel(...) at first and got an error. Koog doesn’t assume a default.

The Spring Boot integration is available if you’re in that ecosystem. For standalone Java apps, the builder API works fine.

Documentation

Official docs: https://docs.koog.ai/

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!