Does asyncio Have Race Conditions in Python? The Truth About Single-Threaded Concurrency
I was debugging a counter in my asyncio application. The code looked perfectly safe—no threads, no parallelism, just simple async coroutines. Yet after running 10,000 increments, my counter showed 1 instead of 10,000.
The Problem: asyncio Race Conditions Are Real
I had fallen for the single-thread myth. Like many developers, I assumed asyncio was immune to race conditions because it runs in a single thread. I was wrong.
import asyncio
value = 0
async def increment(): global value tmp = value # Step 1: Read await asyncio.sleep(0) # Step 2: YIELD - Event loop switches! tmp = tmp + 1 # Step 3: Compute await asyncio.sleep(0) # Step 4: YIELD - Event loop switches! value = tmp # Step 5: Write
async def main(): tasks = [increment() for _ in range(10000)] await asyncio.gather(*tasks) print(f"Result: {value}") # Expected 10000, got 1
asyncio.run(main())Output: Result: 1
Every coroutine read the initial value (0), incremented it to 1, and wrote back 1. The other 9,999 updates were lost.
Why This Happens: The await Yield Point
The key insight that changed my understanding: concurrency != parallelism.
Asyncio provides concurrency without parallelism. Multiple coroutines can be “in progress” simultaneously, even though only one actively executes. The event loop decides which coroutine runs next at each await point.
Timeline of what actually happens:
Task 1: read(0) ─── await ─── compute(1) ─── await ─── write(1) │ │ ▼ ▼Task 2: read(0) ─── await ─── compute(1) ─── await ─── write(1) │ │ ▼ ▼Task 3: read(0) ─── await ─── compute(1) ─── await ─── write(1)
All tasks read 0, all write 1. Final result: 1 (not 10000!)Every await statement creates a suspension point where the event loop can switch to another coroutine. This is cooperative multitasking—the coroutine voluntarily yields control.
The Solution: asyncio.Lock
Python’s asyncio provides synchronization primitives specifically for this problem. The most common is asyncio.Lock:
import asyncio
value = 0lock = asyncio.Lock()
async def increment(): global value async with lock: # Acquire lock - others must wait tmp = value await asyncio.sleep(0) # Safe: lock prevents interleaving tmp = tmp + 1 await asyncio.sleep(0) # Safe: still protected value = tmp
async def main(): tasks = [increment() for _ in range(10000)] await asyncio.gather(*tasks) print(f"Result: {value}") # Now correctly shows 10000
asyncio.run(main())Output: Result: 10000
The lock ensures only one coroutine can execute the critical section at a time, even with await points inside.
When You Need Locks (And When You Don’t)
Safe Operations (No Lock Needed)
Atomic operations that complete in one execution step without await:
# These are safe - no yield point between read and writecounter += 1 # Atomic in asyncioitems.append(item) # Atomic list appendcache[key] = value # Atomic dict assignment
# These operations have NO await, so they cannot be interruptedx = some_valuey = x + 1result = compute(x, y)Unsafe Operations (Lock Required)
Any read-modify-write pattern with await between steps:
# Pattern 1: Lost Updatebalance = await get_balance() # YIELD POINTbalance += 100 # Another coroutine can read old balance hereawait save_balance(balance) # YIELD POINT
# Pattern 2: Check-Then-Actif user not in cache: # Check await fetch_user(user) # YIELD - user might be fetched by another coroutine cache[user] = data # Act - potential duplicate work
# Pattern 3: Shared I/O Resourcesawait writer.write(data) # YIELD - interleaved writes can corrupt streamawait writer.flush() # YIELDCommon Mistakes I Made
Mistake 1: Using threading.Lock
import threadingimport asyncio
# WRONG: threading.Lock blocks the entire thread, including event looplock = threading.Lock()
async def bad_example(): with lock: # This blocks the event loop! await some_async_operation()
# CORRECT: Use asyncio.Locklock = asyncio.Lock()
async def good_example(): async with lock: # This cooperates with the event loop await some_async_operation()threading.Lock blocks the entire thread. Since asyncio runs in a single thread, this blocks the event loop itself—defeating the purpose of async.
Mistake 2: Locking Too Late
async def bad_example(): data = await fetch_data() # Race condition already happened here! async with lock: process(data) # Locking after the race point is useless
# The correct approach: protect the entire critical sectionasync def good_example(): async with lock: data = await fetch_data() # Protected from the start process(data)Mistake 3: Over-Locking
async def wasteful_example(): async with lock: local_result = compute() # Local variable - no need to lock await asyncio.sleep(1) # Not accessing shared state final = transform(local_result) # Still local await save_result(final) # Only this needs protection
# Better: minimize the locked sectionasync def efficient_example(): local_result = compute() # No lock needed final = transform(local_result) # No lock needed async with lock: await save_result(final) # Lock only the shared resource accessPractical Example: Shared Cache Pattern
A common real-world pattern—caching async fetch results:
import asyncio
cache = {}cache_lock = asyncio.Lock()
async def get_or_set(key, fetch_func): """Get from cache or fetch and cache the result.""" # Check-then-act pattern MUST use lock async with cache_lock: if key in cache: return cache[key] # Even with await inside lock, it's protected value = await fetch_func(key) cache[key] = value return value
# Usageasync def fetch_user(user_id): await asyncio.sleep(0.1) # Simulate API call return f"user_{user_id}"
async def main(): # Concurrent requests for same user - only one fetch happens results = await asyncio.gather( get_or_set(1, fetch_user), get_or_set(1, fetch_user), # Cache hit get_or_set(1, fetch_user), # Cache hit ) print(results) # All three get the same cached result
asyncio.run(main())Without the lock, multiple coroutines could simultaneously check if key in cache, all find it missing, and all trigger the expensive fetch operation.
The Full Toolkit: Other Synchronization Primitives
Python’s asyncio provides six synchronization primitives:
| Primitive | Purpose | Common Use Case |
|---|---|---|
asyncio.Lock | Mutual exclusion | Protecting shared state |
asyncio.Event | Signal multiple tasks | ”Data is ready” broadcast |
asyncio.Condition | Wait for event + exclusive access | Producer-consumer |
asyncio.Semaphore | Limit concurrent access | Rate limiting, connection pools |
asyncio.BoundedSemaphore | Semaphore with upper bound check | Resource pools with max limit |
asyncio.Barrier | Block until N tasks waiting | Parallel task synchronization |
import asyncio
# Limit to 3 concurrent API callsapi_semaphore = asyncio.Semaphore(3)
async def fetch_with_limit(url): async with api_semaphore: # Only 3 coroutines can proceed at once return await fetch(url)
# Now running 100 fetches won't overwhelm the APItasks = [fetch_with_limit(url) for url in urls]await asyncio.gather(*tasks)Key Takeaways
- asyncio has race conditions - Single-threaded does not mean single-execution-path
- Every
awaitis a yield point - The event loop can switch coroutines - Use
asyncio.Lockfor shared state - Notthreading.Lock - Lock the entire critical section - Not just the write operation
- Don’t over-lock - Local operations don’t need protection
The mental model shift: thinking of await as “pause here, someone else might run.” Any shared state accessed across await boundaries needs protection.
Final Words + More Resources
My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me
Here are also the most important links from this article along with some further resources that will help you in this scope:
Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!
Comments