From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andy Lutomirski Subject: Re: [PATCH 0/3] epoll: Add epoll_pwait1 syscall Date: Thu, 8 Jan 2015 17:28:45 -0800 Message-ID: References: <1420705550-24245-1-git-send-email-famz@redhat.com> <1420708372.18399.15.camel@suse.cz> <20150109011608.GA2924@fam-t430.nay.redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Return-path: In-Reply-To: <20150109011608.GA2924-+wGkCoP0yD+sDdueE5tM26fLeoKvNuZc@public.gmane.org> Sender: linux-api-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Fam Zheng Cc: Miklos Szeredi , "linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , X86 ML , Alexander Viro , Andrew Morton , Juri Lelli , Zach Brown , David Drysdale , Kees Cook , Alexei Starovoitov , David Herrmann , Dario Faggioli , Theodore Ts'o , Peter Zijlstra , Vivek Goyal , Mike Frysinger , Heiko Carstens , Rasmus Villemoes , Oleg Nesterov , Mathieu Desnoyers List-Id: linux-api@vger.kernel.org On Thu, Jan 8, 2015 at 5:25 PM, Fam Zheng wrote: > On Thu, 01/08 09:57, Andy Lutomirski wrote: >> On Thu, Jan 8, 2015 at 1:12 AM, Miklos Szeredi wrote: >> > On Thu, 2015-01-08 at 16:25 +0800, Fam Zheng wrote: >> >> Applications could use epoll interface when then need to poll a big number of >> >> files in their main loops, to achieve better performance than ppoll(2). Except >> >> for one concern: epoll only takes timeout parameters in microseconds, rather >> >> than nanoseconds. >> >> >> >> That is a drawback we should address. For a real case in QEMU, we run into a >> >> scalability issue with ppoll(2) when many devices are attached to guest, in >> >> which case many host fds, such as virtual disk images and sockets, need to be >> >> polled by the main loop. As a result we are looking at switching to epoll, but >> >> the coarse timeout precision is a trouble, as explained below. >> >> >> >> We're already using prctl(PR_SET_TIMERSLACK, 1) which is necessary to implement >> >> timers in the main loop; and we call ppoll(2) with the next firing timer as >> >> timeout, so when ppoll(2) returns, we know that we have more work to do (either >> >> handling IO events, or fire a timer callback). This is natual and efficient, >> >> except that ppoll(2) itself is slow. >> >> >> >> Now that we want to switch to epoll, to speed up the polling. However the timer >> >> slack setting will be effectively undone, because that way we will have to >> >> round up the timeout to microseconds honoring timer contract. But consequently, >> >> this hurts the general responsiveness. >> >> >> >> Note: there are two alternatives, without changing kernel: >> >> >> >> 1) Leading ppoll(2), with the epollfd only and a nanosecond timeout. It won't >> >> be slow as one fd is polled. No more scalability issue. And if there are >> >> events, we know from ppoll(2)'s return, then we do the epoll_wait(2) with >> >> timeout=0; otherwise, there can't be events for the epoll, skip the following >> >> epoll_wait and just continue with other work. >> >> >> >> 2) Setup and add a timerfd to epoll, then we do epoll_wait(..., timeout=-1). >> >> The timerfd will hopefully force epoll_wait to return when it timeouts, even if >> >> no other events have arrived. This will inheritly give us timerfd's precision. >> >> Note that for each poll, the desired timeout is different because the next >> >> timer is different, so that, before each epoll_wait(2), there will be a >> >> timerfd_settime syscall to set it to a proper value. >> >> >> >> Unfortunately, both approaches require one more syscall per iteration, compared >> >> to the original single ppoll(2), cost of which is unneglectable when we talk >> >> about nanosecond granularity. >> >> I'd like to see a more ambitious change, since the timer isn't the >> only problem like this. Specifically, I'd like a syscall that does a >> list of epoll-related things and then waits. The list of things could >> include, at least: >> >> - EPOLL_CTL_MOD actions: level-triggered epoll users are likely to >> want to turn on and off their requests for events on a somewhat >> regular basis. > > This sounds good to me. > >> >> - timerfd_settime actions: this allows a single syscall to wait and >> adjust *both* monotonic and real-time wakeups. > > I'm not sure, doesn't this break orthogonality between epoll and timerfd? Yes. It's not very elegant, and more elegant ideas are welcome. > >> >> Would this make sense? It could look like: >> >> int epoll_mod_and_pwait(int epfd, >> struct epoll_event *events, int maxevents, >> struct epoll_command *commands, int ncommands, >> const sigset_t *sigmask); > > What about flags? > No room. Maybe it should just be a struct for everything instead of separate args. > Fam -- Andy Lutomirski AMA Capital Management, LLC