Understanding Python Async/Await: Why It's Confusing and How to Master It
I was building a simple web scraper that fetched 10 URLs sequentially. Each request took about 2 seconds. Total time: 20 seconds. I’d heard about Python’s async/await and thought it would magically make everything faster. So I wrote this:
import asyncio, time
async def fetch_data(): time.sleep(2) return "data"
async def main(): result = await fetch_data() print(result)
asyncio.run(main())Still took 2 seconds per call. When I tried running 10 of them concurrently, nothing changed. It still ran one at a time. I had no idea what async actually did — and I was not alone.
The Real Problem: You’re Thinking About Threads
The confusion starts here. Most developers heard “async = concurrent” and filled in the rest with “like threads, but lighter.” That’s wrong.
Threads run code in parallel (on multiple CPU cores). The operating system scheduler decides when to pause a thread and switch to another — at any instruction boundary. You don’t control it.
Async is different. It runs everything on one thread. There is no parallelism. What you get is cooperative multitasking: each task voluntarily yields control at explicit points (the await keywords) so other tasks can run.
The mental model that finally clicked for me: async is a scheduler running on a single thread. The event loop is that scheduler. It runs one coroutine until it hits await, then it switches to another waiting coroutine.
Sequential (no async): |---A---|---B---|---C---| (3 chunks of time)
Concurrent (async): |---A---|---B---|---A---|---C---|---B---|---C---| ^ task A hits await, ^ task B hits await, event loop switches event loop switches to B to CThe total wall time can be shorter for I/O-bound work because while task A is waiting for the network, the CPU is idle. The event loop uses that idle time to run B and C.
The Event Loop: The Engine You Never See
When you call asyncio.run(main()), Python creates an event loop that:
- Picks a coroutine from the ready queue
- Runs it until it hits
await - When it hits
await, the awaited operation (e.g., network I/O) is registered with the loop - The loop picks the next ready coroutine and runs it
- When the awaited I/O finishes, the original coroutine goes back to the ready queue
- Repeat until all coroutines finish
The await keyword is the surrender point. Without it, your function runs from start to finish without interruption, same as a normal function.
This is why my first scraper failed. time.sleep(2) is a blocking call. It doesn’t surrender to the event loop — it blocks the entire thread, including the event loop and every other coroutine waiting to run.
The Fix: Real async I/O
Replace blocking calls with their async equivalents:
import asyncio
async def fetch_data(): await asyncio.sleep(2) # surrenders control return "data"
async def main(): result = await fetch_data() print(result)
asyncio.run(main())asyncio.sleep() returns a coroutine that tells the loop “wake me in 2 seconds.” The loop records this, switches to another task, and comes back when time’s up. The thread is never blocked.
The pattern is: every blocking library call has an async equivalent.
| Blocking | Async |
|---|---|
time.sleep(n) | asyncio.sleep(n) |
requests.get(url) | httpx.AsyncClient().get(url) |
open(file) (sync) | aiofiles.open(file) |
socket.send() | asyncio.loop.sock_sendall() |
Running Multiple Tasks
The real win is concurrency. You use asyncio.create_task() to schedule a coroutine for execution, then asyncio.gather() to wait for all of them:
import asyncioimport httpx
async def fetch_endpoint(url): async with httpx.AsyncClient() as client: resp = await client.get(url) return resp.status_code
async def main(): urls = [ "https://api.example.com/1", "https://api.example.com/2", "https://api.example.com/3", ] tasks = [asyncio.create_task(fetch_endpoint(url)) for url in urls] results = await asyncio.gather(*tasks) print(results)
asyncio.run(main())With 3 URLs at 2 seconds each, this finishes in ~2 seconds instead of 6. The event loop interleaves execution: while fetch_endpoint("1") is waiting for the HTTP response, the loop switches to fetch_endpoint("2"), and so on.
Two Mistakes I Kept Making
Forgetting await
result = fetch_data() # returns a coroutine object, not data!print(result) # prints "<coroutine object fetch_data at 0x...>"A coroutine is a paused function. It doesn’t run until you await it (or pass it to asyncio.run()). If you forget await, you get a dangling coroutine object and a RuntimeWarning when the garbage collector finds it.
Blocking the Loop with CPU Work
async def compute(): for i in range(10_000_000): _ = i * i # no await, blocks everythingAsync is for I/O, not CPU. If you need to run CPU-bound work, use asyncio.to_thread() to offload it to a thread pool:
async def compute(): result = await asyncio.to_thread(heavy_cpu_work) return resultWhen To Actually Use Async
Async is worth it when your program spends most of its time waiting: web servers, API clients, database queries, file I/O. FastAPI, aiohttp, and asyncpg are async-native for a reason.
Async is not useful for CPU-bound scripts, simple scripts that run one thing at a time, or anything where the bottleneck is the CPU.
The Bottom Line
async def declares a function that can be paused. await is the pause button. The event loop is the invisible scheduler that decides what runs next. Everything runs on one thread. No parallelism. Just smarter waiting.
Final Words + More Resources
My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me
Here are also the most important links from this article along with some further resources that will help you in this scope:
- 👨💻 Official Python asyncio Documentation
- 👨💻 Real Python: Async IO in Python
- 👨💻 FastAPI Async Documentation
Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!
Comments