io_uring learns to swap thread identities for the sake of speed

The io_uring subsystem is built on a promise not to block the calling thread unless explicitly asked to. The problem is that many code paths inside the kernel were never designed for asynchronous execution in the first place. Today, any operation that might block (for example, the equivalents of fdatasync(), statx(), or parts of openat()) is handed off by io_uring to a separate worker thread. This guarantees that io_uring_enter() won't hang, but it comes at a cost: waking a thread and switching context. What's annoying is that you always have to pay it, even when the operation doesn't end up blocking after all — and performance is usually the whole reason people reach for io_uring.

Swapping identities

Jens Axboe has published an RFC patch set with a solution he himself calls scary. The idea: start a potentially blocking operation right in the calling thread, and pay for a worker thread only when blocking actually happens.

Catching that moment required a change to the scheduler. A PF_IO_HANDOFF flag has been added to task_struct; if a thread marked with it is about to go to sleep anywhere in the kernel, the scheduler calls a new function, io_uring_task_sleeping(). This is how io_uring learns about the block without rewriting the code paths themselves.

What happens next is a "thread identity handoff." Work already done can't be unwound — blocking can happen almost anywhere, so the operation has to be carried through to the end. Instead, io_uring takes a thread from its pool and swaps the identities of the two threads: the worker thread receives the original's identifier, signal-handling settings, and other attributes, continues processing the queue, and eventually returns to user space. Meanwhile the original thread, having taken on the worker's identity, calmly blocks, finishes the operation once it wakes, and heads off into the pool.

Plenty of mines along the way

The main danger is references to task_struct. Before the swap, the kernel must make sure no one else is holding the structure, otherwise someone will reference the wrong thread, adding to the stream of kernel CVEs. Hence a long list of prohibitions on the handoff: tracing via ptrace() (in both directions), perf events, kernel-tracked futex ownership, the realtime scheduler, core scheduling cookies, vfork(), and others. It is precisely this check that looks like the most fragile part: task_struct is used everywhere, and it's hard to vouch that every reference has been accounted for — especially future ones, added by developers who aren't thinking about io_uring at all.

The payoff, meanwhile, is large. In benchmarks, fsync() on tmpfs gave a boost of nearly 700%. Other improvements are more modest, and tests where blocking always happens show a regression in places — especially at large queue depth, where sequential processing loses out to parallel dispatch in the workers. Axboe reckons that applications with such operations usually don't keep long queues anyway.

No one is planning to merge the patches any time soon — the author is only checking whether the approach is viable. Peter Zijlstra pointed out that banning the handoff when a shadow stack is active would rule the feature out for most systems; Axboe hopes to carry that over too. Gabriel Krisman Bertazzi summed up the general mood in a formula: "very cool and looks very dangerous."

Source: Lobsters