[PATCH] eventpoll: compute timer slack lazily in ep_poll()

[PATCH] eventpoll: compute timer slack lazily in ep_poll()

[Usama Arif]

ep_poll() computes the timer slack via select_estimate_accuracy() up front,
before checking whether events are already available.
select_estimate_accuracy() reads the clock (ktime_get_ts64()), and the
resulting slack is only consumed by the schedule_hrtimeout_range() call on
the blocking path.

A busy poller such as an L7 proxy event loop calls epoll_wait() at a very
high rate and often finds events already pending, returning via
ep_try_send_events() without ever blocking.  In that case the up-front
slack estimation - including its clock read - is pure overhead.  read_tsc()
attributable to select_estimate_accuracy() sometimes shows up in perf profiles
of such a workload via the epoll_wait() path.

Move the slack estimation to the point where the thread is actually about
to sleep.  The timeout passed to ep_poll() is already an absolute deadline
(ep_timeout_to_timespec()), so deferring the estimate does not change the
wakeup time; taken closer to the sleep it is, if anything, marginally more
accurate.  On the common non-blocking path the clock read is skipped
entirely.

Measured on a host running a Meta production workload with the following
bpftrace script:

  #!/usr/bin/bpftrace
  fentry:__x64_sys_epoll_wait,
  fentry:__x64_sys_epoll_pwait   { @in[tid] = 1; }
  fexit:__x64_sys_epoll_wait,
  fexit:__x64_sys_epoll_pwait    { delete(@in, tid); }
  fentry:select_estimate_accuracy /@in[tid]/ { @sea++; }
  fentry:schedule_hrtimeout_range /@in[tid]/ { @shr++; }
  interval:s:30 {
      printf("sea=%lld shr=%lld wasted=%lld (%d%%)\n",
             @sea, @shr, @sea - @shr, (@sea - @shr) * 100 / @sea);
      exit();
  }

Over a 30s window:

sea=3,587,704 shr=3,003,920 wasted=583,784 (16%)

So ~16% of ep_poll invocations of select_estimate_accuracy have no
consumer.

Signed-off-by: Usama Arif <usama.arif@linux.dev>
---
 fs/eventpoll.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index 0e65c7431dfc..128d7fd3d0ea 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -2248,7 +2248,6 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events,
 	lockdep_assert_irqs_enabled();
 
 	if (timeout && (timeout->tv_sec | timeout->tv_nsec)) {
-		slack = select_estimate_accuracy(timeout);
 		to = &expires;
 		*to = timespec64_to_ktime(*timeout);
 	} else if (timeout) {
@@ -2327,10 +2326,13 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events,
 
 		spin_unlock_irq(&ep->lock);
 
-		if (!eavail)
+		if (!eavail) {
+			if (to)
+				slack = select_estimate_accuracy(timeout);
 			timed_out = !ep_schedule_timeout(to) ||
 				!schedule_hrtimeout_range(to, slack,
 							  HRTIMER_MODE_ABS);
+		}
 		__set_current_state(TASK_RUNNING);
 
 		/*
-- 
2.53.0-Meta

Re: [PATCH] eventpoll: compute timer slack lazily in ep_poll()

[Christian Brauner]

On Tue, 07 Jul 2026 12:02:38 -0700, Usama Arif wrote:
> eventpoll: compute timer slack lazily in ep_poll()

Applied to the vfs-7.3.misc branch of the vfs/vfs.git tree.
Patches in the vfs-7.3.misc branch should appear in linux-next soon.

Please report any outstanding bugs that were missed during review in a
new review to the original patch series allowing us to drop it.

It's encouraged to provide Acked-bys and Reviewed-bys even though the
patch has now been applied. If possible patch trailers will be updated.

Note that commit hashes shown below are subject to change due to rebase,
trailer updates or similar. If in doubt, please check the listed branch.

tree:   https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git
branch: vfs-7.3.misc

[1/1] eventpoll: compute timer slack lazily in ep_poll()
      https://git.kernel.org/vfs/vfs/c/d0c242dda99c

Re: [PATCH] eventpoll: compute timer slack lazily in ep_poll()

[Shakeel Butt]

On Tue, Jul 07, 2026 at 12:02:38PM -0700, Usama Arif wrote:
> ep_poll() computes the timer slack via select_estimate_accuracy() up front,
> before checking whether events are already available.
> select_estimate_accuracy() reads the clock (ktime_get_ts64()), and the
> resulting slack is only consumed by the schedule_hrtimeout_range() call on
> the blocking path.
> 
> A busy poller such as an L7 proxy event loop calls epoll_wait() at a very
> high rate and often finds events already pending, returning via
> ep_try_send_events() without ever blocking.  In that case the up-front
> slack estimation - including its clock read - is pure overhead.  read_tsc()
> attributable to select_estimate_accuracy() sometimes shows up in perf profiles
> of such a workload via the epoll_wait() path.
> 
> Move the slack estimation to the point where the thread is actually about
> to sleep.  The timeout passed to ep_poll() is already an absolute deadline
> (ep_timeout_to_timespec()), so deferring the estimate does not change the
> wakeup time; taken closer to the sleep it is, if anything, marginally more
> accurate.  On the common non-blocking path the clock read is skipped
> entirely.
> 
> Measured on a host running a Meta production workload with the following
> bpftrace script:
> 
>   #!/usr/bin/bpftrace
>   fentry:__x64_sys_epoll_wait,
>   fentry:__x64_sys_epoll_pwait   { @in[tid] = 1; }
>   fexit:__x64_sys_epoll_wait,
>   fexit:__x64_sys_epoll_pwait    { delete(@in, tid); }
>   fentry:select_estimate_accuracy /@in[tid]/ { @sea++; }
>   fentry:schedule_hrtimeout_range /@in[tid]/ { @shr++; }
>   interval:s:30 {
>       printf("sea=%lld shr=%lld wasted=%lld (%d%%)\n",
>              @sea, @shr, @sea - @shr, (@sea - @shr) * 100 / @sea);
>       exit();
>   }
> 
> Over a 30s window:
> 
> sea=3,587,704 shr=3,003,920 wasted=583,784 (16%)
> 
> So ~16% of ep_poll invocations of select_estimate_accuracy have no
> consumer.
> 
> Signed-off-by: Usama Arif <usama.arif@linux.dev>

Reviewed-by: Shakeel Butt <shakeel.butt@linux.dev>