From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753931Ab1H3XM1 (ORCPT ); Tue, 30 Aug 2011 19:12:27 -0400 Received: from smtp1.linux-foundation.org ([140.211.169.13]:47851 "EHLO smtp1.linux-foundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753893Ab1H3XMX (ORCPT ); Tue, 30 Aug 2011 19:12:23 -0400 Date: Tue, 30 Aug 2011 16:11:30 -0700 From: Andrew Morton To: Jeremy Allison Cc: Daniel Ehrenberg , Jens Axboe , Jeff Moyer , linux-kernel@vger.kernel.org, linux-aio@kvack.org, Ulrich Drepper Subject: Re: Approaches to making io_submit not block Message-Id: <20110830161130.592df746.akpm@linux-foundation.org> In-Reply-To: <20110830230342.GB16326@samba2> References: <4E5D5817.6040704@kernel.dk> <4E5D64E8.7000102@kernel.dk> <20110830154157.d802d097.akpm@linux-foundation.org> <20110830155438.bc31ab99.akpm@linux-foundation.org> <20110830230342.GB16326@samba2> X-Mailer: Sylpheed 3.0.2 (GTK+ 2.20.1; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 30 Aug 2011 16:03:42 -0700 Jeremy Allison wrote: > On Tue, Aug 30, 2011 at 03:54:38PM -0700, Andrew Morton wrote: > > > > Also, glibc has userspace for POSIX AIO. A successful kernel-based > > implementation would result in glibc migrating away from its current > > implementation. So we should work with the glibc developers on ensuring > > that the migration can happen. > > Unfortunately the glibc userspace POSIX AIO limits asynchronicity to > one outstanding request per file descriptor. From aio_misc.c in glibc: > > if (runp != NULL > && runp->aiocbp->aiocb.aio_fildes == aiocbp->aiocb.aio_fildes) > { > /* The current file descriptor is worked on. It makes no sense > to start another thread since this new thread would fight > with the running thread for the resources. But we also cannot > say that the thread processing this desriptor shall immediately > after finishing the current job process this request if there > are other threads in the running queue which have a higher > priority. */ > > /* Simply enqueue it after the running one according to the > priority. */ > > I have often wondered if this is actually the case ? I created > my own glibc with a patches AIO that removed this restriction > (thus had multiple outstanding threads on a single fd). In testing > I saw a dramatic increase in performance (2x speedup) but then > testing with use in actual code (Samba smbd) it made the client > throughput *worse*. I never got to the bottom of this and so > didn't submit my fixes to glibc. > > Any ideas if this is still the case ? Or comments on why glibc > insists on only one outstanding request per fd ? Is this really > needed for kernel performance ? > I don't know. Uli cc'ed.