How to Capture LLM Reasoning During Tool Calls in Spring AI

Mar 26, 2026

The Problem

I was building an AI agent with Spring AI that could call different tools based on user requests. The agent worked, but I had no visibility into why it chose specific tools.

Here’s what my logs looked like:

2026-03-26 10:15:32 INFO  Tool called: retrievePatientHealthStatus
2026-03-26 10:15:32 INFO  Arguments: {patientId=PAT-001}

I could see which tool was called, but not why. When debugging unexpected behavior, this wasn’t enough. I needed to know the LLM’s reasoning.

The Solution

Spring AI’s Tool Argument Augmenter lets you capture LLM reasoning by dynamically extending tool JSON schemas with additional arguments. The LLM fills these arguments when selecting tools, giving you visibility into the decision-making process.

Step 1: Create a Reasoning DTO

First, define a record to capture the reasoning data:

import org.springframework.ai.tool.annotation.ToolParam;

public record AgentThinking(
    @ToolParam(description = """
      Your step-by-step reasoning for why you're calling this tool.
      Explain what information you need and why this tool provides it.
      """, required = true)
    String innerThought,

    @ToolParam(description = "Your confidence level (low, medium, high)", required = true)
    String confidence
) {}

The @ToolParam annotations tell the LLM what to include in each field.

Step 2: Set Up the Augmented Tool Callback Provider

Create the provider that intercepts tool calls and extracts reasoning:

import org.springframework.ai.tool.method.MethodToolCallbackProvider;
import org.springframework.ai.tool.augment.AugmentedToolCallbackProvider;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class ToolConfig {

    @Bean
    public AugmentedToolCallbackProvider&lt;AgentThinking&gt; augmentedToolProvider(
            PatientHealthInformationTools healthTools) {

        return AugmentedToolCallbackProvider
            .&lt;AgentThinking&gt;builder()
            .toolObject(healthTools)
            .argumentType(AgentThinking.class)
            .argumentConsumer(event -&gt; {
                AgentThinking thinking = event.arguments();

                // Log the reasoning
                log.info("Tool: {}", event.toolDefinition().name());
                log.info("Reasoning: {}", thinking.innerThought());
                log.info("Confidence: {}", thinking.confidence());
            })
            .build();
    }
}

Note: In the code above, I escape the generics syntax for MDX compatibility. In your actual Java code, use AugmentedToolCallbackProvider<AgentThinking>.

Step 3: Keep Your Tools Clean

The beauty of this approach is that your actual tool methods stay simple:

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.stereotype.Component;
import java.util.Map;
import java.util.stream.Collectors;

@Component
public class PatientHealthInformationTools {

    private static final Map&lt;String, PatientRecord&gt; HEALTH_DATA = Map.of(
        "PAT-001", new PatientRecord("John Doe", "Stable", "Room 302"),
        "PAT-002", new PatientRecord("Jane Smith", "Critical", "ICU 101")
    );

    @Tool(description = "Retrieve the current health status of a patient by their ID")
    public String retrievePatientHealthStatus(String patientId) {
        PatientRecord record = HEALTH_DATA.get(patientId);
        if (record == null) {
            return "Patient not found: " + patientId;
        }
        return String.format("Patient: %s, Status: %s, Location: %s",
            record.name(), record.status(), record.location());
    }

    @Tool(description = "List all patients with a specific health status")
    public String listPatientsByStatus(String status) {
        return HEALTH_DATA.values().stream()
            .filter(r -&gt; r.status().equalsIgnoreCase(status))
            .map(PatientRecord::name)
            .collect(Collectors.joining(", "));
    }
}

The tool has no knowledge of the reasoning capture. This keeps your business logic clean.

Step 4: Use the Augmented Provider in ChatClient

Wire everything together in your ChatClient:

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.stereotype.Service;

@Service
public class AgentService {

    private final ChatClient chatClient;

    public AgentService(ChatClient.Builder builder,
                       AugmentedToolCallbackProvider&lt;AgentThinking&gt; provider) {
        this.chatClient = builder
            .defaultToolCallbacks(provider)
            .build();
    }

    public String chat(String userInput) {
        return chatClient.prompt()
            .user(userInput)
            .call()
            .content();
    }
}

The Result

Now when I run the agent, I get much richer logs:

2026-03-26 10:20:15 INFO  Tool: retrievePatientHealthStatus
2026-03-26 10:20:15 INFO  Reasoning: The user asked about patient PAT-001. I need to check their current health status to provide accurate information. The retrievePatientHealthStatus tool is the most appropriate since it directly queries patient records.
2026-03-26 10:20:15 INFO  Confidence: high

Why This Matters

Production AI systems need accountability. When an agent makes a decision, you need to know:

Debugging: Why did the agent call this specific tool?
Auditing: What was the LLM’s reasoning for this action?
Compliance: Can you explain the decision chain to stakeholders?
Improvement: Are your tool descriptions clear enough?

Common Mistake: Modifying Tool Signatures

I initially tried adding reasoning parameters directly to my tool methods:

// DON'T DO THIS
@Tool(description = "Retrieve patient health status")
public String retrievePatientHealthStatus(
    String patientId,
    String reasoning,  // Wrong!
    String confidence  // Wrong!
) {
    // Business logic gets mixed with observability
}

This approach is wrong because:

Tools become coupled to observability concerns
You can’t reuse tools in different contexts
Testing becomes more complex
The tool’s purpose becomes unclear

The augmenter pattern keeps these concerns separate.

How It Works Under the Hood

Here’s the flow:

sequenceDiagram
    participant App as Your Application
    participant ACP as AugmentedToolCallbackProvider
    participant LLM as LLM
    participant Tool as Tool Method

    App-&gt;&gt;ACP: Request tool definitions
    ACP-&gt;&gt;ACP: Extend JSON schemas with reasoning args
    ACP-&gt;&gt;LLM: Send extended tool definitions
    LLM-&gt;&gt;LLM: Select tool and fill all arguments
    LLM-&gt;&gt;ACP: Return tool call with reasoning
    ACP-&gt;&gt;ACP: Extract and consume reasoning
    ACP-&gt;&gt;Tool: Invoke with original arguments only
    Tool-&gt;&gt;App: Return result

The LLM sees extended schemas, fills in reasoning, and the augmenter extracts it before invoking your tool.

Summary

Use Spring AI’s Tool Argument Augmenter to capture why your AI agent selects specific tools. Create a DTO with @ToolParam annotations, set up AugmentedToolCallbackProvider with an argument consumer, and keep your tool methods clean. This enables debugging, auditing, and compliance without polluting your business logic.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

👨‍💻 Spring AI Documentation
👨‍💻 Tool Argument Augmenter

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!