Async Rust with tokio 1.29, A Production Mental Model

TL;DR — An async fn returns a state machine, not a running task; the runtime drives it / tokio 1.29 uses a work-stealing multi-threaded scheduler by default with cooperative yielding to prevent task starvation / Cancellation is implicit on every .await — design for it or it’ll bite you

Async in Rust took me longer to get comfortable with than the borrow checker. The borrow checker has clear rules; async has clear rules plus a runtime plus a futures model plus pinning plus Send bounds. Once it clicks, it’s productive. While it’s clicking, it’s frustrating.

This post is the mental model that worked for me. It assumes you’ve written enough Rust to be past the borrow-checker phase — see my memory safety post for the foundation. The goal here is to make tokio’s behavior predictable so you can write services that handle load without surprises.

What an async fn actually is

An async fn doesn’t run when you call it. It returns a Future — a struct generated by the compiler that holds the state machine for the function’s body. The function doesn’t make progress until something polls that future, which in practice means handing it to the runtime.

async fn fetch_user(id: u64) -> Result<User, Error> {
    let row = sqlx::query_as::<_, User>("SELECT * FROM users WHERE id = $1")
        .bind(id as i64)
        .fetch_one(&POOL)
        .await?;
    Ok(row)
}

fn main() {
    let _fut = fetch_user(42); // builds the future, runs nothing
    // Without a runtime, this future is just an idle state machine
}

To actually run it, you hand it to a tokio runtime:

#[tokio::main(flavor = "multi_thread", worker_threads = 8)]
async fn main() -> anyhow::Result<()> {
    let user = fetch_user(42).await?;
    println!("{:?}", user);
    Ok(())
}

The #[tokio::main] macro expands to building a Runtime and calling block_on with the body. tokio 1.29’s default is a multi-threaded work-stealing scheduler. Tasks (futures spawned via tokio::spawn) get distributed across worker threads, and idle workers steal work from busy ones.

The two runtime flavors

You’ll pick one of two flavors per binary:

Multi-threaded (default). N worker threads, work stealing. Use this for HTTP servers, anything CPU-parallel, anything where you want max throughput. Tasks spawned here must be Send because they might move between threads.

Current-thread. Single worker. Tasks don’t need to be Send. Use this for CLIs, tests, single-connection clients. Lighter weight, predictable scheduling, no synchronization overhead.

// Multi-threaded server
#[tokio::main]
async fn main() { /* ... */ }

// Current-thread CLI
#[tokio::main(flavor = "current_thread")]
async fn main() { /* ... */ }

// Explicit runtime if you need fine control
fn main() {
    let rt = tokio::runtime::Builder::new_multi_thread()
        .worker_threads(4)
        .max_blocking_threads(64)
        .enable_all()
        .build()
        .unwrap();
    rt.block_on(async {
        run_server().await
    });
}

The blocking thread pool (separate from worker threads) is for spawn_blocking and tokio::fs. Default size is 512, which is too many for most services — tune it down or you’ll see thread count explode under load.

The cardinal sin: blocking the runtime

A tokio worker thread polls futures cooperatively. If your future doesn’t yield (doesn’t hit an .await or doesn’t return Pending), the worker is stuck. One stuck worker on a four-worker runtime means 25% of capacity is gone. Two stuck workers and you’re at 50%. This is the most common production issue with async Rust.

// BAD: blocks the worker
async fn process(data: Vec<u8>) -> Hash {
    sha256::digest(&data) // CPU-heavy, no await, blocks the worker
}

// GOOD: offload CPU work
async fn process(data: Vec<u8>) -> Hash {
    tokio::task::spawn_blocking(move || sha256::digest(&data))
        .await
        .unwrap()
}

// BAD: blocking IO in async context
async fn load() -> String {
    std::fs::read_to_string("config.toml").unwrap() // blocks
}

// GOOD: async IO
async fn load() -> std::io::Result<String> {
    tokio::fs::read_to_string("config.toml").await
}

tokio 1.29 includes cooperative scheduling improvements that yield automatically inside many runtime primitives, which catches some accidental long-running tasks. It doesn’t catch pure CPU work; that’s still on you.

Rule of thumb: if a single operation takes more than ~100µs and isn’t doing IO, spawn_blocking it.

Cancellation is implicit and pervasive

Every .await is a potential cancellation point. If the task running the future is dropped (because someone called .abort() on the JoinHandle, or because a select! chose another branch, or because a tokio::time::timeout expired), the future is dropped at the next .await.

This is fine for most code. The local destructor chain runs, connections get returned to pools, file handles close. But it’s a problem for sequences that must be atomic.

async fn transfer(from: AccountId, to: AccountId, amount: u64) -> Result<()> {
    debit(from, amount).await?;   // what if cancellation happens here?
    credit(to, amount).await?;     // we'd debit without crediting
    Ok(())
}

Two defenses. First: do this kind of work inside a database transaction so the commit is the atomic point, and a cancellation between debit and credit rolls back. Second: structure operations so any partial state is recoverable — idempotency keys, reconciliation passes.

For “I really need to finish this work even if my caller goes away,” tokio::spawn decouples the lifetime. The spawned task runs to completion even if the spawner is dropped, unless you explicitly abort it.

let handle = tokio::spawn(async move {
    finish_critical_work().await
});
// the caller can return without cancelling the spawned task

select! and cancellation safety

tokio::select! polls multiple futures and runs the body of the first to complete. The other branches are dropped — cancelled — at whatever .await they were sitting on. This is fine if those branches are “cancellation safe,” meaning dropping them mid-await doesn’t lose data.

AsyncRead::read is not cancellation safe in general — bytes may have been consumed from the stream. tokio::sync::mpsc::Receiver::recv is cancellation safe. The tokio docs call out which methods are safe; check before using a future in select!.

loop {
    tokio::select! {
        msg = rx.recv() => {
            match msg {
                Some(m) => handle(m).await,
                None => break,
            }
        }
        _ = tokio::time::sleep(Duration::from_secs(30)) => {
            send_heartbeat().await;
        }
    }
}

This is a common pattern: receive messages with a periodic timeout for housekeeping. Both branches are cancellation-safe.

Structured concurrency with JoinSet

For fanning work out and joining the results, JoinSet is the right primitive:

use tokio::task::JoinSet;

async fn fetch_all(ids: Vec<u64>) -> Vec<Result<User, Error>> {
    let mut set = JoinSet::new();
    for id in ids {
        set.spawn(fetch_user(id));
    }
    let mut results = Vec::with_capacity(set.len());
    while let Some(res) = set.join_next().await {
        results.push(res.unwrap());
    }
    results
}

Dropping the JoinSet aborts all its tasks. This gives you “if I bail out of this function, none of the spawned work leaks.” Without JoinSet, naked tokio::spawn calls become orphaned tasks that keep running and you have to track JoinHandles by hand.

The Send bound problem

A future is Send if all the data it captures across .await points is Send. The most common offender is Rc<T> or RefCell<T> held across an await — both are !Send. The compiler will tell you, but the error messages can be hard to read.

async fn bad() {
    let cache = Rc::new(LocalCache::new());
    something_async().await; // ERROR: future not Send because Rc lives across await
    cache.lookup("x");
}

Two fixes: use Arc instead, or scope the non-Send value so it’s dropped before the .await:

async fn good() {
    let key = {
        let cache = Rc::new(LocalCache::new());
        cache.lookup_key("x") // returns an owned, Send value
    }; // Rc dropped here
    something_async().await;
    use_key(key);
}

For the multi-threaded runtime, every spawned task must be Send. For current-thread, no Send requirement.

Common Pitfalls

• Spawning without bounds. A handler that does tokio::spawn per request, without limits, becomes a DoS vector. Use a semaphore: Arc<Semaphore> with acquire_owned gives you backpressure.
• Mixing std::sync::Mutex with .await. Holding a std::sync::Mutex guard across an .await is technically a bug (the future becomes !Send in many cases) and a performance hazard. Use tokio::sync::Mutex when you need to hold a lock across an await; use std::sync::Mutex and drop it before the await otherwise.
• unwrap() on JoinHandle::await. A panicked task returns Err(JoinError). Log and continue, don’t propagate the panic up.
• Naive timeouts on non-cancellation-safe code. tokio::time::timeout(d, future) drops the future on expiry. If the inner future was halfway through a multi-step operation, you have partial state. Design the operation to be safe to drop, or wrap it in a transaction.
• Ignoring tracing instrumentation. Async debugging without #[tracing::instrument] on key functions is painful. Add it early.

// A solid default handler shape
#[tracing::instrument(skip(state))]
async fn handler(state: AppState, req: Request) -> Result<Response, Error> {
    let _permit = state.semaphore.clone().acquire_owned().await?;
    let result = tokio::time::timeout(
        Duration::from_secs(10),
        do_work(state, req),
    ).await??;
    Ok(result)
}

Wrapping Up

Async Rust on tokio 1.29 is genuinely productive once the model is in place. The friction comes from the fact that async forces you to think about cancellation, blocking, and Send bounds that you’d ignore in a synchronous codebase — but those concerns exist in the synchronous version too, you just don’t see them until production. The compiler making you confront them up front is a feature, even when it doesn’t feel like one.

Next post is on building secure CLIs with clap 4, which uses the current-thread tokio runtime as a base. After that, error handling patterns that scale beyond the toy ? operator examples.