Re: Approaches to making io_submit not block

Andrew Morton
2011-08-30T23:12:27Z

On Tue, 30 Aug 2011 16:03:42 -0700
Jeremy Allison <jra@samba.org> wrote:

> On Tue, Aug 30, 2011 at 03:54:38PM -0700, Andrew Morton wrote:
> >
> > Also, glibc has userspace for POSIX AIO. A successful kernel-based
> > implementation would result in glibc migrating away from its current
> > implementation. So we should work with the glibc developers on ensuring
> > that the migration can happen.
>
> Unfortunately the glibc userspace POSIX AIO limits asynchronicity to
> one outstanding request per file descriptor. From aio_misc.c in glibc:
>
> if (runp != NULL
> && runp->aiocbp->aiocb.aio_fildes == aiocbp->aiocb.aio_fildes)
> {
> /* The current file descriptor is worked on. It makes no sense
> to start another thread since this new thread would fight
> with the running thread for the resources. But we also cannot
> say that the thread processing this desriptor shall immediately
> after finishing the current job process this request if there
> are other threads in the running queue which have a higher
> priority. */
>
> /* Simply enqueue it after the running one according to the
> priority. */
>
> I have often wondered if this is actually the case ? I created
> my own glibc with a patches AIO that removed this restriction
> (thus had multiple outstanding threads on a single fd). In testing
> I saw a dramatic increase in performance (2x speedup) but then
> testing with use in actual code (Samba smbd) it made the client
> throughput *worse*. I never got to the bottom of this and so
> didn't submit my fixes to glibc.
>
> Any ideas if this is still the case ? Or comments on why glibc
> insists on only one outstanding request per fd ? Is this really
> needed for kernel performance ?
>

I don't know. Uli cc'ed.