From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ragnar =?iso-8859-15?Q?Kj=F8rstad?= Subject: Re: NFSv4/pNFS possible POSIX I/O API standards Date: Mon, 18 Dec 2006 03:57:59 +0100 Message-ID: <20061218025759.GO25199@vestdata.no> References: <20061206100614.GX5937@schatzie.adilger.int> <4576FBB0.2070704@redhat.com> <4576FD78.4040603@mcs.anl.gov> <4577011F.6040107@redhat.com> <20061206180124.GM17226@vestdata.no> <45770850.4030003@redhat.com> <20061217144130.GN25199@vestdata.no> <4585956F.80404@redhat.com> <20061217193821.GA21070@parisc-linux.org> <4585BBEA.6050705@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Matthew Wilcox , Rob Ross , Christoph Hellwig , Trond Myklebust , Sage Weil , Brad Boyer , Anton Altaparmakov , Gary Grider , linux-fsdevel@vger.kernel.org Return-path: Received: from stine.vestdata.no ([217.149.127.10]:36975 "EHLO stine.vestdata.no" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752109AbWLRC6u (ORCPT ); Sun, 17 Dec 2006 21:58:50 -0500 To: Ulrich Drepper Content-Disposition: inline In-Reply-To: <4585BBEA.6050705@redhat.com> Sender: linux-fsdevel-owner@vger.kernel.org List-Id: linux-fsdevel.vger.kernel.org On Sun, Dec 17, 2006 at 01:51:38PM -0800, Ulrich Drepper wrote: > Matthew Wilcox wrote: > >I know that the rsync load is a major factor on kernel.org right now= =2E >=20 > That should be quite easy to quantify then. Move the readdir and sta= t=20 > call next to each other in the sources, pass the struct stat around i= f=20 > necessary, and then count the stat calls which do not originate from = the=20 > stat following the readdir call. Of course we'll also need the actua= l=20 > improvement which can be achieved by combining the calls. Given the=20 > inodes are cached, is there more overhead then finding the right inod= e?=20 > Note that is rsync doesn't already use fstatat() it should do so and= =20 > this means then that there is no long file path to follow, all file=20 > names are local to the directory opened with opendir(). >=20 > My but feeling is that the improvements are minimal for normal (not=20 > cluster etc) filesystems and hence the improvements for kernel.org wo= uld=20 > be minimal. I don't think the overhead of finding the right inode or the system calls themselves makes a difference at all. E.g. the rsync numbers I listed spend more than 0.3ms per stat syscall. That kind of time is not spent in looking up kernel datastructures - it's spent doing IO. That part that I think is important (and please correct me if I've gotten it all wrong) is to do the IO in parallel. This applies both to local filesystems and clustered filesystems, allthough it would probabl= y be much more significant for clustered filesystems since they would typically have longer latency for each roundtrip. Today there is no go= od=20 way for an application to stat many files in parallel. You could do it through threading, but with significant overhead and complexity. I'm curious what results one would get by comparing performance of: * application doing readdir and then stat on every single file * application doing readdirplus * application doing readdir and then stat on every file using a lot of threads or an asyncronous stat interface As far as parallel IO goes, I would think that async stat would be nearly as fast as readdirplus? =46or the clustered filesystem case there may be locking issues that ma= kes readdirplus faster? --=20 Ragnar Kj=F8rstad Software Engineer Scali - http://www.scali.com Scaling the Linux Datacenter - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel= " in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html