From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andreas Dilger Subject: Re: fixing redundant network opens on Linux file creation Date: Mon, 6 Jan 2003 14:31:04 -0700 Sender: samba-technical-admin@lists.samba.org Message-ID: <20030106143104.L31555@schatzie.adilger.int> References: <20030106175958.GA632@vagabond> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Jan Hudec , linux-fsdevel@vger.kernel.org, Richard Sharpe , samba-technical@samba.org, Steven French , Lustre Development Mailing List Return-path: To: Bryan Henderson Content-Disposition: inline In-Reply-To: ; from hbryan@us.ibm.com on Mon, Jan 06, 2003 at 11:42:26AM -0800 Errors-To: samba-technical-admin@lists.samba.org List-Help: List-Post: List-Subscribe: , List-Unsubscribe: , List-Archive: List-Id: linux-fsdevel.vger.kernel.org On Jan 06, 2003 11:42 -0800, Bryan Henderson wrote: > >There is a lookup intent patch from lustre group. It can be found > >somewhere in the archives. Pushing that (or something along that lines) > >to mainline and using that would be IMHO most beneficial > > "Intent," as it's generally understood, is not a promise of future activity > -- it's either a hint to improve efficiency or it's a promise to restrict > future activity, but it should be possible simply to bail out before > exercising that intent. E.g. you can't open a file at the same time as it > is looked up just because the looker upper says he intends to open it > later. In our code, the lookup-with-intent actually performs both of the operations on the server, and it is up to the client methods to detect that the operation was done and deal with it appropriately. We have very well-tested code for 2.4 and 2.5 code is mostly functional (2.5 is a lot neater implementation but the changes mean that it isn't yet as functional as the 2.4 code). In the Lustre code, the premise is that the lookup-with-intent operation (called lookup2 for now) does one of: 1) the lookup + operation on the server in one RPC (i.e. lookup+create[+open], lookup+unlink, lookup+rename) and tells the client "I just did this for you, here are the attributes of the new entry and a lock on it if necessary, please fix up your local state to match", and the actual VFS operations are only doing the post-facto state cleanup. 2) OR it returns a lock to the client that grants the client exclusive control over the item(s) in question (normally the parent dir(s)) and lets the client do the operations locally and send the operations to the server separately. We currently implement (1) only right now, but the goal is to implement (2) in the future (which would be back to nearly what the VFS currently does, except that we are now granted the locks in advance) so that a client can do many operations locally without the need for getting lots of locks. For example, in the future, a Lustre client creating a new directory could be granted the lock on that directory, and it could then create files in that directory without further RPCs to the server very efficiently (e.g. untarring a file) until another client revokes the lock(s) and forces the client to flush all of its updates to the server. For an updated version of the intent patch, see: ftp://ftp.lustre.org/pub/kernels/patches/2.4.18-hp1_pnnl18_l5.patch ftp://ftp.lustre.org/pub/kernels/patches/37chaos-l5.patch The first patch is good for vanilla kernels, and the second for RH 2.4.18-17ish kernels. There is a bit of extra stuff therein which isn't really related to the intent changes. Cheers, Andreas PS - I've added lustre-devel to this thread so that the Lustre developers also see any discussion related to the intent changes. -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://www-mddsp.enel.ucalgary.ca/People/adilger/