[PATCH 0/4] io_uring: improve alloc_cache branch prediction

[Caleb Sander Mateos <csander@purestorage.com>]

A call to io_uring_alloc_async_data() that provides a struct io_alloc_cache
currently has 2 unnecessary NULL checks in the fast path:
- Of the struct io_alloc_cache *, even though it points to a struct io_ring_ctx
  field
- Of the void * entry popped from the cache, even though all cache entries are
  non-NULL
The compiler also incorrectly assumes the success path in io_alloc_cache_get()
is the unlikely one because it's written as an early return.

Avoid the first NULL check by splitting io_uring_alloc_async_data()'s cache
logic to a separate function io_uring_alloc_cached_async_data() that uses the
cache unconditionally. Avoid the second by returning a separate success bool
from io_cache_alloc() instead of checking the NULL-ness of the returned entry.
Also mark io_alloc_cache_get() and io_alloc_cache_put()'s success paths as
likely so the compiler can avoid jumps in the fast path.

The final commit is a small code simplification in io_futex_wait() noticed while
touching the neighboring io_cache_alloc() call.

Caleb Sander Mateos (4):
  io_uring: split io_uring_alloc_async_data()
  io_uring/alloc_cache: mark cache success paths as likely
  io_uring/alloc_cache: assume io_alloc_cache_get() doesn't return NULL
  io_uring/futex: avoid ifd NULL check in io_futex_wait()

 io_uring/alloc_cache.c |  2 +-
 io_uring/alloc_cache.h | 25 ++++++++++++------------
 io_uring/filetable.c   |  3 +--
 io_uring/futex.c       | 13 +++++--------
 io_uring/io_uring.h    | 24 ++++++++++++++---------
 io_uring/net.c         |  8 ++++----
 io_uring/poll.c        | 13 ++++++++-----
 io_uring/rsrc.c        | 43 +++++++++++++++++++++---------------------
 io_uring/rsrc.h        |  3 ++-
 io_uring/rw.c          |  4 ++--
 io_uring/timeout.c     |  2 +-
 io_uring/uring_cmd.c   |  4 ++--
 io_uring/waitid.c      |  2 +-
 13 files changed, 76 insertions(+), 70 deletions(-)

-- 
2.56.0