Skip to content
BSWEN
Blog

How to Choose the Right Python Concurrency Model When Building High-Performance Applications

Cowrie
Cowrie
Dev @ Bswen

I ran into a classic problem last week: my web scraper was taking 12 hours to process 1,000 URLs sequentially. “I’ll just add threads,” I thought. My server crashed with “can’t create new thread” errors. Then I tried multiprocessing—my CPU utilization hit 100% but the task still took 10 hours because each process was mostly waiting on network I/O.

After debugging for hours, I realized I had picked the wrong concurrency model entirely. Here’s what I learned about when to use asyncio, threading, and multiprocessing in Python.

The Real Problem: Mismatched Concurrency Models

The core issue isn’t which concurrency model is “best”—it’s which model matches your workload:

┌─────────────────────────────────────────────────────────────┐
│                    PYTHON CONCURRENCY GUIDE                   │
├─────────────────┬─────────────────┬───────────────────────────┤
│    I/O-Bound    │    I/O-Bound    │       CPU-Bound          │
│  High Concurrency │ Low Concurrency │   Heavy Computation     │
├─────────────────┼─────────────────┼───────────────────────────┤
│    asyncio      │    threading    │     multiprocessing       │
│  (best choice)  │  (okay choice)  │     (only choice)         │
├─────────────────┼─────────────────┼───────────────────────────┤
│ • 1000+ tasks   │ • <100 tasks    │ • Math computations       │
│ • Network I/O   │ • Simple I/O    │ • Data processing         │
│ • Web scrapers  │ • File I/O      │ • Image processing        │
│ • API clients   │ • Database ops  │ • Machine learning       │
└─────────────────┴─────────────────┴───────────────────────────┘

Let me show you the trial-and-error process I went through.

My First Mistake: Blocking Calls in Async Functions

I thought I could just sprinkle async keywords everywhere and get magic performance:

bad_async.py
import asyncio
import time


async def fetch_url(url):
    # This BLOCKS the entire event loop!
    time.sleep(1)  # ❌ Never use blocking calls in async
    return f"Result from {url}"


async def main():
    tasks = [fetch_url(f"http://example.com/{i}") for i in range(1000)]
    results = await asyncio.gather(*tasks)


asyncio.run(main())  # Still takes ~1000 seconds sequentially!

I ran this and got zero performance benefit. Why? Because time.sleep(1) is a blocking call—it freezes the entire event loop, preventing any other coroutines from running.

The correct approach is to use async-compatible operations:

good_async.py
import asyncio


async def fetch_url(url):
    # This YIELDS control to the event loop
    await asyncio.sleep(1)  # ✓ Non-blocking sleep
    return f"Result from {url}"


async def main():
    tasks = [fetch_url(f"http://example.com/{i}") for i in range(1000)]
    results = await asyncio.gather(*tasks)
    return results


asyncio.run(main())  # Now takes ~1 second for all 1000 URLs!

The difference is dramatic. The async version completes in roughly 1 second instead of 1000 seconds because all tasks run concurrently.

Performance Comparison: Synchronous vs Async

To understand the performance difference, I wrote a simple test:

perf_comparison.py
import asyncio
import time


# Synchronous version
def count_sync():
    print("One")
    time.sleep(1)
    print("Two")
    time.sleep(1)


def run_sync():
    start = time.time()
    count_sync()
    count_sync()
    count_sync()
    print(f"Synchronous: {time.time() - start:.2f} seconds")


# Async version
async def count_async():
    print("One")
    await asyncio.sleep(1)
    print("Two")
    await asyncio.sleep(1)


async def run_async():
    start = time.time()
    await asyncio.gather(count_async(), count_async(), count_async())
    print(f"Async: {time.time() - start:.2f} seconds")


if __name__ == "__main__":
    run_sync()
    asyncio.run(run_async())

Output:

Synchronous: 6.03 seconds
Async: 2.00 seconds

The synchronous version takes 6 seconds because it runs sequentially. The async version takes only 2 seconds because all three tasks run concurrently, each yielding control during the sleep periods.

Why Threading Failed My Web Scraper

Initially, I tried threading for my 1,000 URL scraper:

threading_attempt.py
import threading
import time
import requests


def fetch_url(url):
    response = requests.get(url)
    return response.status_code


threads = []
for i in range(1000):
    t = threading.Thread(target=fetch_url, args=(f"http://example.com/{i}",))
    threads.append(t)
    t.start()


for t in threads:
    t.join()

This crashed with:

OSError: [Errno 24] Too many open files

Or worse:

RuntimeError: can't start new thread

The problem? Threads are heavy system resources. Each thread requires:

  • A call stack (typically 8MB on Linux)
  • Operating system thread management overhead
  • Context switching costs

Creating 1,000 threads exhausted my system resources. Real Python’s article puts it bluntly: “Creating thousands of threads will fail on many machines or can slow down your code. In contrast, creating thousands of async I/O tasks is completely feasible.”

When to Use Threading (It’s Not Never)

Threading does have valid use cases, just not for high-concurrency I/O:

threading_useful.py
import threading
from queue import Queue
import time


# Worker pool pattern: limited threads for I/O-bound work
def worker(queue, results):
    while True:
        item = queue.get()
        if item is None:
            break
        # Process item (I/O-bound operation)
        time.sleep(0.1)  # Simulating I/O
        results.append(item * 2)
        queue.task_done()


queue = Queue()
results = []


# Create only 10 threads, not 1000
num_workers = 10
threads = []
for i in range(num_workers):
    t = threading.Thread(target=worker, args=(queue, results))
    t.start()
    threads.append(t)


# Add 100 tasks
for i in range(100):
    queue.put(i)


# Wait for completion
queue.join()


# Stop workers
for i in range(num_workers):
    queue.put(None)
for t in threads:
    t.join()


print(f"Processed {len(results)} items")

Threading works well when you need:

  • Less than 100 concurrent operations
  • Simple I/O-bound tasks (file operations, database queries)
  • Legacy code that can’t be easily converted to async

Multiprocessing: The CPU-Bound Workhorse

When I switched to multiprocessing for my scraper, I saw 100% CPU utilization but poor performance because network I/O is not CPU-bound. Multiprocessing shines for actual computation:

multiprocessing_example.py
import multiprocessing
import time


def cpu_intensive_task(n):
    """CPU-bound task: mathematical computation"""
    result = 0
    for i in range(n):
        result += i ** 2
    return result


def run_sequential():
    start = time.time()
    for _ in range(4):
        cpu_intensive_task(1000000)
    return time.time() - start


def run_parallel():
    start = time.time()
    with multiprocessing.Pool(processes=4) as pool:
        pool.map(cpu_intensive_task, [1000000] * 4)
    return time.time() - start


if __name__ == "__main__":
    seq_time = run_sequential()
    par_time = run_parallel()
    print(f"Sequential: {seq_time:.2f}s")
    print(f"Parallel: {par_time:.2f}s")
    print(f"Speedup: {seq_time/par_time:.2f}x")

Output on my 4-core machine:

Sequential: 1.24s
Parallel: 0.34s
Speedup: 3.65x

The speedup approaches 4x (matching my CPU cores) because the computation is truly CPU-bound.

The Chess Master Analogy

Real Python uses a brilliant analogy to explain async I/O:

Imagine chess master Judit Polgár playing 24 opponents simultaneously.

  • Sequential: Play one game to completion, then the next. Total time: 12 hours.
  • Async: Make one move, move to the next board while opponent thinks, repeat. Total time: 1 hour.

This is exactly how asyncio works. The chess master (event loop) doesn’t play faster—she just avoids waiting while opponents think (I/O operations). Each game takes the same time, but they all complete faster because she’s not blocked waiting.

Common Mistakes I Made (So You Don’t Have To)

Mistake 1: Mixing Blocking and Async Code

mixed_blocking.py
import asyncio
import time


async def bad_mixed():
    print("Starting async operation")
    # This blocks everything!
    time.sleep(5)  # ❌ Blocks event loop
    print("Done")


async def good_mixed():
    print("Starting async operation")
    await asyncio.sleep(5)  # ✓ Yields control
    print("Done")

Mistake 2: Using Multiprocessing for I/O-Bound Tasks

wrong_multiprocessing.py
import multiprocessing
import requests


def fetch_url(url):
    # I/O-bound: process creation overhead is wasted
    return requests.get(url).status_code


# Each process spends 99% of time waiting for network
# Process creation adds ~100ms overhead per process
# Much slower than async!

Mistake 3: Forgetting to Await Coroutines

forgotten_await.py
import asyncio


async def my_coroutine():
    await asyncio.sleep(1)
    return "result"


async def wrong():
    my_coroutine()  # ❌ Called but not awaited!
    print("This prints immediately")


async def correct():
    result = await my_coroutine()  # ✓ Properly awaited
    print(f"Got: {result}")

Decision Framework: Which Model to Choose?

Ask yourself these questions:

1. What type of work?
   ├─ CPU-bound (math, data processing, ML)
   │  └─ Use multiprocessing
   └─ I/O-bound (network, disk, database)
      └─ 2. How many concurrent operations?
         ├─ Thousands of connections
         │  └─ Use asyncio
         └─ Less than 100 connections
            └─ 3. Can you modify the code?
               ├─ Yes, can make async
               │  └─ Use asyncio
               └─ No, must use sync libraries
                  └─ Use threading with worker pool

Real-World Example: Web Scraper Done Right

Here’s how I finally fixed my scraper using asyncio with aiohttp:

async_scraper.py
import asyncio
import aiohttp
import time
from typing import List


async def fetch_url(session: aiohttp.ClientSession, url: str) -> tuple:
    try:
        async with session.get(url, timeout=aiohttp.ClientTimeout(total=10)) as response:
            return url, response.status, await response.text()
    except Exception as e:
        return url, None, str(e)


async def scrape_urls(urls: List[str], batch_size: int = 50) -> List[tuple]:
    """Scrape URLs in batches to avoid overwhelming servers"""
    results = []


    async with aiohttp.ClientSession() as session:
        for i in range(0, len(urls), batch_size):
            batch = urls[i:i + batch_size]
            tasks = [fetch_url(session, url) for url in batch]
            batch_results = await asyncio.gather(*tasks)
            results.extend(batch_results)
            print(f"Processed batch {i//batch_size + 1}: {len(batch)} URLs")


    return results


async def main():
    urls = [f"https://httpbin.org/delay/1" for _ in range(100)]


    start = time.time()
    results = await scrape_urls(urls, batch_size=20)
    elapsed = time.time() - start


    successful = sum(1 for _, status, _ in results if status == 200)
    print(f"Completed {len(urls)} URLs in {elapsed:.2f}s")
    print(f"Success rate: {successful}/{len(urls)}")


if __name__ == "__main__":
    asyncio.run(main())

This version:

  • Processes 100 URLs in ~15 seconds (vs 100+ seconds sequentially)
  • Uses bounded concurrency with batching to avoid overwhelming servers
  • Handles errors gracefully without crashing
  • Uses minimal memory compared to threading

Performance Characteristics Summary

┌─────────────────┬────────────────┬────────────────┬────────────────┐
│    Model        │ Best For        │ Scalability    │ Memory Usage   │
├─────────────────┼────────────────┼────────────────┼────────────────┤
│ asyncio         │ I/O-bound       │ Excellent      │ Low            │
│                 │ High concurrency│ (10K+ tasks)   │ (~1KB/task)    │
├─────────────────┼────────────────┼────────────────┼────────────────┤
│ threading       │ I/O-bound       │ Limited        │ High           │
│                 │ Low concurrency │ (<100 threads) │ (~8MB/thread)  │
├─────────────────┼────────────────┼────────────────┼────────────────┤
│ multiprocessing │ CPU-bound       │ Limited by     │ Very High      │
│                 │ Heavy compute   │ CPU cores      │ (full process  │
│                 │                 │ (4-64 cores)   │  per worker)   │
└─────────────────┴────────────────┴────────────────┴────────────────┘

Key Takeaways

After all this trial and error, here’s what I learned:

  1. asyncio is the default choice for I/O-bound work in modern Python—it scales to thousands of concurrent operations with minimal overhead

  2. multiprocessing is only for CPU-bound tasks—using it for I/O is wasteful because processes spend most time waiting, not computing

  3. threading is the legacy option—use it only when you can’t convert sync libraries to async, and keep thread counts low ({'<'}100)

  4. Never block the event loop—blocking calls inside async functions defeat the entire purpose and can make performance worse than sequential code

  5. Match the model to the workload—I/O-bound with high concurrency → asyncio, CPU-bound → multiprocessing, simple I/O → threading (as a last resort)

The chess master analogy captures it perfectly: asyncio isn’t about doing things faster, it’s about not waiting when you could be doing something else. My 12-hour scraper now completes in under an hour—all because I finally picked the right tool for the job.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!

Comments

Tags: