Skip to content
BSWEN
Blog

When Should AI Agents Use Free Public APIs Versus Paid API Aggregators?

Cowrie
Cowrie
Dev @ Bswen

I spent two hours registering for API keys last week. Twelve different services. Twelve confirmation emails. Twelve API keys stored in my environment variables. My AI agent needed access to weather data, stock prices, news feeds, geocoding, currency rates—you name it. By the seventh registration form, I started questioning my life choices.

Then I discovered my agent hit rate limits on three of those APIs within the first hour of testing.

This is the hidden complexity of building AI agents that interact with the real world. You either juggle dozens of API integrations or pay someone to do it for you. Let me walk through how I decided which approach makes sense.

The Problem Nobody Warns You About

When I started building my AI agent, connecting to external APIs seemed straightforward. I found the public-apis repository on GitHub—396,000 stars, massive list of free APIs. I thought I’d hit the jackpot.

Here’s what actually happened:

Fragmentation hit hard. Each API needed separate registration. Some wanted email verification. Others required OAuth flows. A few demanded credit cards for “identity verification” even on free tiers.

Rate limits varied wildly. One API allowed 1,000 requests per day. Another offered 10 per minute. A third had monthly caps that reset on arbitrary dates. My agent’s query patterns didn’t match any of them.

Reliability was unpredictable. That free weather API I integrated? It went down for three days. No SLA, no notification, just timeouts.

The CORS confusion. I wasted an hour debugging CORS errors before realizing server-side agents don’t face browser restrictions. The public-apis list includes CORS columns that are irrelevant for backend code.

What Actually Works

After three iterations, I landed on a decision framework that saved me from API integration hell.

When Free Public APIs Make Sense

I use free public APIs when:

I need 1-3 data sources maximum. Managing three API keys is fine. Managing fifteen is a maintenance nightmare.

Usage stays predictable and low. If my agent makes under 1,000 daily requests per source, free tiers usually hold.

The API has clear documentation and stability. Government APIs (like census.gov) and major platform APIs (GitHub, Stripe) treat free tiers seriously. Hobby projects on random domains don’t.

I’m prototyping. Before I commit to a paid aggregator, I need to prove the agent actually needs that data.

Here’s the decision logic I ended up coding:

api_strategy.py
def choose_api_strategy(sources: list, daily_requests: int, budget: float) -> str:
    """
    Decide between free APIs and aggregator based on constraints.


    Returns: 'free' | 'aggregator' | 'hybrid'
    """
    free_tier_friendly = daily_requests < 1000
    simple_integration = len(sources) <= 3
    budget_constrained = budget < 50  # monthly


    if simple_integration and free_tier_friendly:
        return "free"


    if len(sources) >= 5 or not budget_constrained:
        return "aggregator"


    # Mix of both for complex but budget-conscious setups
    return "hybrid"

When Paid Aggregators Win

I switched to a paid aggregator (Handler in my case) when:

My agent needed 5+ different data sources. The cognitive load of tracking five rate limits, five authentication methods, and five failure modes wasn’t worth the savings.

I needed gated content. Premium news APIs, financial data feeds, authenticated web scraping—these aren’t in the public-apis list.

I wanted unified billing. One invoice. One spending limit. One place to audit where my agent spent its budget.

Rate limit aggregation mattered. Aggregators often pool limits across services or negotiate higher caps than individual free tiers offer.

The handler approach someone mentioned on Reddit made sense:

“You sign up once, get an API key that you provide to your agent, set spending rules, and let the agent access the actual web / gated stuff.”

That’s exactly what I needed. My agent gets one API key, I set a $20 monthly cap, and it can access weather, news, web scraping, and geocoding through a single interface.

agent_with_aggregator.py
import os
from handler import HandlerClient


# One key for everything
client = HandlerClient(api_key=os.environ["HANDLER_KEY"])


# Built-in spending control
client.set_monthly_limit(20.00)


# Agent can now access multiple services
weather = client.weather.get(city="San Francisco")
news = client.news.search(query="AI agents", days=7)
location = client.geocode(address="1600 Pennsylvania Ave")


# Spending tracked automatically
print(f"Monthly spend: ${client.get_spend():.2f}")

The Hybrid Approach That Actually Scales

What I ended up with wasn’t purely free or purely paid. It’s a hybrid:

  • Free APIs for bulk, non-critical data (open government datasets, public GitHub repos)
  • Aggregator for gated content and services needing reliability (news, financial data, web scraping)
  • Fallback chains so one API failure doesn’t kill the agent
hybrid_fetcher.py
class HybridDataFetcher:
    """
    Free APIs for bulk data, aggregator for gated/reliable access.
    """


    def __init__(self):
        self.free_apis = {
            "github": GitHubClient(),  # Well-documented, reliable free tier
            "census": CensusClient(),  # Government API with SLA
        }
        self.aggregator = HandlerClient(api_key=os.environ["HANDLER_KEY"])
        self.aggregator.set_monthly_limit(20.00)


    def fetch(self, source: str, **params):
        # Try free tier first
        if source in self.free_apis:
            try:
                return self.free_apis[source].get(**params)
            except RateLimitExceeded:
                # Fall back to aggregator
                pass


        # Aggregator handles gated/reliable access
        return self.aggregator.fetch(source, **params)

Mistakes I Made (So You Don’t Have To)

Mistake 1: Skipping the “requires key” APIs.

I filtered the public-apis list to only show “no auth” endpoints. Bad idea. The comment that changed my thinking:

“APIs requiring free key are worth the two-minute registration because key-authenticated endpoints are almost always more capable and reliable.”

Key-authenticated endpoints get higher rate limits, better uptime, and more features. The two-minute registration is worth it.

Mistake 2: Not calculating actual query volume.

I assumed “free tier will be fine” without measuring. My agent needed 500 requests per hour during peak usage. That API with 1,000 daily requests? It failed within two hours of deployment.

Now I calculate before choosing:

volume_calculator.py
def estimate_monthly_requests(queries_per_hour: int, active_hours: int = 24) -> dict:
    """
    Calculate monthly API volume requirements.
    """
    daily = queries_per_hour * active_hours
    monthly = daily * 30


    return {
        "daily": daily,
        "monthly": monthly,
        "recommended_tier": "free" if monthly < 30000 else "paid",
        "buffer_needed": monthly * 1.5  # Always add 50% buffer
    }

Mistake 3: Ignoring failure isolation.

When the aggregator goes down, all my agent’s data sources died. Now each critical data source has at least one fallback:

resilient_fetcher.py
class ResilientFetcher:
    """
    Fallback chains for critical data sources.
    """


    def get_weather(self, city: str):
        # Primary: aggregator (reliable, unified billing)
        # Fallback: free Open-Meteo API


        try:
            return self.aggregator.weather.get(city)
        except (Timeout, ApiError) as e:
            print(f"Aggregator failed: {e}, falling back to free tier")
            return self.open_meteo.get(city)

The Decision Matrix I Actually Use

After all the trial and error, here’s the mental model I apply:

┌─────────────────────────────────────────────────────────────────────────────┐
│                     AI Agent API Strategy Decision Tree                     │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│   How many data sources?                                                    │
│   ├── 1-3 sources → Start with free APIs                                    │
│   └── 5+ sources → Consider aggregator                                       │
│                                                                             │
│   What's your monthly query volume?                                         │
│   ├── Under 30K per source → Free tiers might work                          │
│   └── Over 30K per source → Check paid tiers or aggregator                  │
│                                                                             │
│   Do you need gated content?                                                │
│   ├── Yes → Aggregator (or individual paid APIs)                            │
│   └── No → Free public APIs sufficient                                       │
│                                                                             │
│   Is this prototype or production?                                          │
│   ├── Prototype → Free APIs, accept lower reliability                       │
│   └── Production → Aggregator or paid APIs with SLAs                        │
│                                                                             │
│   Budget constraints?                                                       │
│   ├── Tight → Free APIs + careful rate limit management                     │
│   └── Flexible → Aggregator for simplicity                                  │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

What I’d Do Differently Starting Fresh

If I were building a new AI agent today, here’s my exact process:

  1. Check public-apis first. Browse the repository for my data needs. If it’s there, great. If not, I know I’ll need paid access.

  2. Register for free keys immediately. Don’t skip the two-minute registration. The improved reliability is worth it.

  3. Count my sources. If I need more than five different APIs, I’m going straight to an aggregator.

  4. Estimate volume. Calculate monthly requests and compare against free tier limits with a 1.5x buffer.

  5. Start hybrid. Use free APIs for well-documented sources (government, major platforms), aggregator for everything else.

  6. Build fallback chains from day one. Don’t wait for the first outage.

The 396,000 stars on the public-apis repository proves I’m not alone in needing API discovery. But discovery is just step one. The architecture decision—free vs aggregator vs hybrid—depends on your agent’s specific needs, not on which approach is “better” in the abstract.

My agent now runs smoothly with two API keys: one for the aggregator (gated content, spending control), one for a reliable free API (bulk data). That’s infinitely simpler than the twelve I started with.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!

Comments

Tags: