Speeding Up I/O‑Bound Work in Python with async/await

The problem: waiting for network calls blocks your whole script

When you fetch data from multiple HTTP endpoints sequentially, each request pauses the program while it waits for a response. Even though the CPU is idle, the total runtime adds up linearly with the number of calls. For I/O‑bound workloads this waste is unnecessary; the program could be doing other work while the network is in transit.

Thesis: async/await lets a single thread drive many I/O operations concurrently

By declaring a function with async def and using await on awaitable objects (e.g., coroutines from aiohttp), you hand control back to the event loop whenever the function would block. The loop can then start another task, making progress on all pending requests without creating extra OS threads.

How asyncio structures the work

1. Define an async coroutine that performs the I/O operation.
2. Create many coroutine instances (tasks) for each URL.
3. Gather them with asyncio.gather so the loop runs them concurrently.
4. Run the event loop once with asyncio.run (Python 3.7+).

Worked example: concurrent scraping of ten URLs

Assume you have a list of ten public URLs you want to fetch. The sequential version looks like:

import requests, time
urls = [f'https://example.com/page{i}' for i in range(1, 11)]
start = time.perf_counter()
for url in urls:
    resp = requests.get(url, timeout=5)
    # process resp.text …
elapsed = time.perf_counter() - start
print(f'Sequential: {elapsed:.2f}s')

The async version replaces requests with aiohttp:

import aiohttp, asyncio, time

async def fetch(session, url):
    async with session.get(url, timeout=5) as resp:
        return await resp.text()

async def main(urls):
    async with aiohttp.ClientSession() as session:
        tasks = [fetch(session, u) for u in urls]
        return await asyncio.gather(*tasks)

if __name__ == '__main__':
    urls = [f'https://example.com/page{i}' for i in range(1, 11)]
    start = time.perf_counter()
    results = asyncio.run(main(urls))
    elapsed = time.perf_counter() - start
    print(f'Concurrent: {elapsed:.2f}s, fetched {len(results)} pages')

Where to run: any terminal with Python 3.11+ and aiohttp installed (pip install aiohttp). No special permissions are needed; the script only makes outbound HTTP requests.

Trade‑off and limitation

Async/await shines when the workload is I/O‑bound. If a coroutine performs heavy CPU work (e.g., image processing, cryptography) without yielding, it blocks the event loop and defeats concurrency. In such cases you must offload the CPU‑bound piece to a thread pool via loop.run_in_executor or rewrite it as a separate process. Additionally, debugging async code can be harder because stack traces are flattened; using asyncio.run with PYTHONASYNCIODEBUG=1 or a debugger that understands coroutines helps.

Actionable closing

Start by profiling your current I/O‑bound script with timeit or cProfile to confirm that most time is spent in blocking calls. Then replace the blocking library with its async counterpart (e.g., requests → aiohttp, psycopg2 → asyncpg) and wrap the calls in async def. Run the script with asyncio.run and compare the elapsed time. If you see a speed‑up proportional to the number of concurrent operations, you’ve successfully applied async/await. Remember to keep CPU‑heavy work out of the coroutine loop or delegate it to an executor.