Skip to content
BSWEN
Blog

How to Switch Between OpenAI, Anthropic, and Ollama in a Spring AI Application

Cowrie
Cowrie
Dev @ Bswen

I was building an AI-powered application and hit a wall. Every time I wanted to switch from OpenAI to Anthropic or test locally with Ollama, I had to rewrite code. API keys were scattered. Model names were hardcoded. It was a mess.

Then I discovered Spring AI’s unified abstraction. Let me show you how to switch between LLM providers without touching your application code.

The Problem

Most Java developers integrating LLMs face these issues:

  • Each provider has different APIs and SDKs
  • Switching requires code changes
  • API keys scattered across config files
  • Local development with cloud providers gets expensive
  • Testing with different models needs separate configurations

I wanted one configuration to rule them all.

The Solution

Spring AI provides a unified ChatClient interface. You configure which provider to use, and Spring AI handles the rest.

Here’s the key property:

application.yaml
spring:
  ai:
    model:
      chat: openai  # Change to: anthropic, ollama

That’s it. Change one property, and your entire application switches providers.

Setting Up Dependencies

First, add the Spring AI starters you need:

build.gradle
dependencies {
    implementation 'org.springframework.ai:spring-ai-openai-spring-boot-starter'
    implementation 'org.springframework.ai:spring-ai-anthropic-spring-boot-starter'
    implementation 'org.springframework.ai:spring-ai-ollama-spring-boot-starter'
}

You don’t need all three. Add only the providers you plan to use.

Full Configuration Example

Here’s how I configured all three providers:

application.yaml
spring:
  ai:
    model:
      chat: openai  # Switch to: anthropic, ollama
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        options:
          model: gpt-4
    anthropic:
      api-key: ${ANTHROPIC_API_KEY}
      chat:
        options:
          model: claude-3-opus-20240229
    ollama:
      base-url: http://localhost:11434
      chat:
        options:
          model: llama3

The spring.ai.model.chat property tells Spring AI which provider to activate. The other configurations stay ready but inactive.

Writing Provider-Agnostic Code

Here’s my chat service that works with any provider:

AgentService.java
@Service
public class AgentService {


    private final ChatClient chatClient;


    public AgentService(ChatClient.Builder chatClientBuilder) {
        this.chatClient = chatClientBuilder.build();
    }


    public String chat(String userMessage) {
        return chatClient.call(userMessage);
    }


    public String chatWithSystem(String systemPrompt, String userMessage) {
        return chatClient.call(new Prompt(
            List.of(
                new SystemMessage(systemPrompt),
                new UserMessage(userMessage)
            )
        )).getResult().getOutput().getContent();
    }
}

Notice there’s no mention of OpenAI, Anthropic, or Ollama anywhere in this code. The abstraction handles it.

Provider Comparison

Here’s when I use each provider:

Provider selection guide
┌────────────┬─────────────────────────┬──────────────────────────────┐
│ Provider   │ Use Case                │ Configuration                │
├────────────┼─────────────────────────┼──────────────────────────────┤
│ OpenAI     │ Production, GPT-4/3.5   │ spring.ai.openai.api-key    │
│ Anthropic  │ Production, Claude      │ spring.ai.anthropic.api-key │
│ Ollama     │ Local, privacy-first    │ Local Ollama server         │
└────────────┴─────────────────────────┴──────────────────────────────┘

Why This Matters

I learned this the hard way. Here’s why provider switching matters:

Cost optimization: I use local Ollama for development. No API costs while building features. Switch to OpenAI or Anthropic for production.

Privacy: Some data should never leave my machine. Ollama runs entirely locally.

Flexibility: Different tasks need different models. Claude excels at analysis. GPT-4 handles creative tasks well. Llama is great for quick local tests.

Future-proofing: Spring AI keeps adding providers. When a new model appears, I just add a starter dependency and a config block.

Common Mistakes I Made

Hardcoding provider selection: I initially wrote if (provider.equals("openai")) everywhere. Wrong. Use configuration.

Ignoring model capabilities: Each model has different context windows and capabilities. I had to adjust prompts per provider.

Storing API keys in code: Don’t do this. Use environment variables.

Not testing locally: I was paying for API calls during development. Ollama saved me money.

Dynamic Provider Switching at Runtime

In my JavaClaw project, I let users choose their provider during onboarding. Here’s the conceptual flow:

Onboarding flow
Step 1: Welcome
    ↓
Step 2: Choose Provider (OpenAI/Anthropic/Ollama)
    ↓
Step 3: Enter Credentials
    ↓
Step 4: Configure Agent Prompt
    ↓
Step 5: Setup MCP Servers
    ↓
Step 6: Connect Telegram (optional)
    ↓
Step 7: Complete

Here’s how I handle the provider selection:

OnboardingController.java
@Controller
public class OnboardingController {


    @PostMapping("/onboarding/provider")
    public String selectProvider(
        @RequestParam String provider,
        @RequestParam(required = false) String apiKey,
        @RequestParam(required = false) String model
    ) {
        Map<String, String> updates = new HashMap<>();
        updates.put("spring.ai.model.chat", provider);


        switch (provider) {
            case "openai":
                updates.put("spring.ai.openai.api-key", apiKey);
                updates.put("spring.ai.openai.chat.options.model", model);
                break;
            case "anthropic":
                updates.put("spring.ai.anthropic.api-key", apiKey);
                updates.put("spring.ai.anthropic.chat.options.model", model);
                break;
            case "ollama":
                updates.put("spring.ai.ollama.chat.options.model", model);
                break;
        }


        configService.updateConfiguration(updates);
        return "redirect:/onboarding/agent-prompt";
    }
}

The configuration updates take effect immediately. Users can switch providers without restarting.

Summary

Spring AI makes LLM provider switching straightforward:

  1. Add starter dependencies for your chosen providers
  2. Configure all providers in application.yaml
  3. Set spring.ai.model.chat to select the active provider
  4. Write code against the ChatClient interface
  5. Switch providers by changing one configuration property

No code changes. No API rewrites. Just configuration.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!

Comments

Tags: