From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chris Mason Subject: Re: [PATCH RFC] O_DIRECT reads and writes without i_sem Date: Fri, 05 Nov 2004 13:06:20 -0500 Message-ID: <1099677980.15540.125.camel@watt.suse.com> References: <1099323127.23475.80.camel@watt.suse.com> <20041101160844.GA29251@infradead.org> Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit Cc: linux-fsdevel@vger.kernel.org, akpm@osdl.org Return-path: Received: from cantor.suse.de ([195.135.220.2]:52951 "EHLO Cantor.suse.de") by vger.kernel.org with ESMTP id S262734AbUKESHx (ORCPT ); Fri, 5 Nov 2004 13:07:53 -0500 To: Christoph Hellwig In-Reply-To: <20041101160844.GA29251@infradead.org> Sender: linux-fsdevel-owner@vger.kernel.org List-Id: linux-fsdevel.vger.kernel.org On Mon, 2004-11-01 at 16:08 +0000, Christoph Hellwig wrote: > On Mon, Nov 01, 2004 at 10:32:07AM -0500, Chris Mason wrote: > > Hello everyone, > > > > Right now, O_DIRECT reads and writes on regular files have to take i_sem > > while reading file metadata in order to make sure we don't race with > > hole filling. > > > > This patch tries to get around that by avoiding i_sem when we are doing > > an O_DIRECT read or write inside of i_size. Yet another rw semaphore is > > added to struct inode to protect against holes being filled during the > > O_DIRECT. direct-io.c gets another special case to be aware of the > > locking. > > > > This has only been lightly tested, I'm posting here for general comments > > before I go much further. I'm rounding up some hardware with enough > > disks to benchmark it properly. Looks like there's a long line to get to the big test machines here, so I did some more local performance tests. I've got two scsi drives striped into a lvm2 target (64k stripe size). FS is reiser. I did testing on writes because more of the io is done under i_sem for writes, so it should be easiest to show the benefit there. The test is a 16k random write test on a 15GB file. 50MB of io is done by each random writer, and everyone is writing to the same preallocated file. 1 random writer = .64MB/s 2 random writers (unpatched) = .71MB/s 2 random writers (patched) = 1.02MB/s In the patched kernel, the two writer procs finished at roughly the same time (< 1sec difference). In the unpatched kernel, writer #1 finished in about 1/2 the time as writer #2. -chris