Re: [Lsf-pc] [LSF/MM TOPIC] async buffered diskio read for userspace apps

[Jeremy Allison <jra@samba.org>]

On Fri, Jan 16, 2015 at 10:44:12AM -0500, Milosz Tanski wrote:
> On Thu, Jan 15, 2015 at 5:31 PM, Jan Kara <jack@suse.cz> wrote:
> > On Thu 15-01-15 12:43:23, Milosz Tanski wrote:
> >> I would like to talk about enhancing the user interfaces for doing
> >> async buffered disk IO for userspace applications. There's a whole
> >> class of distributed web applications (most new applications today)
> >> that would benefit from such an API. Most of them today rely on
> >> cobbling one together in user space using a threadpool.
> >>
> >> The current in kernel AIO interfaces that only support DIRECTIO, they
> >> were generally designed by and for big database vendors. The consensus
> >> is that the current AIO interfaces usually lead to decreased
> >> performance for those app.
> >>
> >> I've been developing a new read syscall that allows non-blocking
> >> diskio read (provided that data is in the page cache). It's analogous
> >> to what exists today in the network world with recvmsg with MSG_NOWAIT
> >> flag. The work has been previously described by LWN here:
> >> https://lwn.net/Articles/612483/
> >>
> >> Previous attempts (over the last 12+ years) at non-blocking buffered
> >> diskio has stalled due to their complexity. I would like to talk about
> >> the problem, my solution, and get feedback on the course of action.
> >>
> >> Over the years I've been building the low level guys of various "web
> >> applications". That usually involves async network based applications
> >> (epoll based servers) and the biggest pain point for the last 8+ years
> >> has been async disk IO.
> >   Maybe this topic will be sorted out before LSF/MM. I know Andrew had some
> > objections about doc and was suggesting a solution using fincore() (which
> > Christoph refuted as being racy). Also there was a pending question
> > regarding whether the async read in this form will be used by applications.
> > But if it doesn't get sorted out a short session on the pending issues
> > would be probably useful.
> >
> >                                                                 Honza
> > --
> > Jan Kara <jack@suse.cz>
> > SUSE Labs, CR
> 
> I've spent the better part of yesterday wrapping up the first cut of
> samba support to FIO so we can test a modified samba file server with
> these changes in a few scenarios. Right now it's only sync but I hope
> to have async in the future. I hope that by the time the summit rolls
> around I'll have data to share from samba and maybe some other common
> apps (node.js / twisted).

Don't forget to share the code changes :-). We @ Samba would
love to see them to keep track !