Re: O_NONBLOCK for regular files

Re: O_NONBLOCK for regular files

[Greg Banks]

Jamie Lokier wrote:
> Christoph Hellwig wrote:
> > It's used to [...] return EAGAIN to the nfs server so it doesn't block all
> > nfsd threads on possible long-enduring HSM operations.
> 
> How does nfsd get notified when the operation is possible? 

It doesn't; the nfsd sends an RPC reply containing the NFSERR_JUKEBOX
error to the NFS client, which sleeps and retries the operation after a
few seconds.  If the operation is not yet possible the cycle repeats.

This is according to the RFC for NFSv3 and NFS clients (including Linux)
know how to handle it.  The point is to free the nfsd and corresponding
client-side resources for other NFS traffic while HSM operation continues.
Also, without this there's a pathological situation where the client
keeps retrying, getting a new nfsd each time, until all the nfsds are
blocked on the same file and all NFS traffic to the server stops.

Greg.
-- 
Greg Banks, R&D Software Engineer, SGI Australian Software Group.
I don't speak for SGI.

Re: O_NONBLOCK for regular files

[Greg Banks]

G'day,

On Sat, 2004-03-20 at 20:24, Greg Banks wrote:
> Jamie Lokier wrote:
> > Christoph Hellwig wrote:
> > > It's used to [...] return EAGAIN to the nfs server so it doesn't block all
> > > nfsd threads on possible long-enduring HSM operations.
> > 
> > How does nfsd get notified when the operation is possible? 
> 
> It doesn't; the nfsd sends an RPC reply containing the NFSERR_JUKEBOX
> error to the NFS client, which sleeps and retries the operation after a
> few seconds.  If the operation is not yet possible the cycle repeats.

http://marc.theaimsgroup.com/?l=linux-fsdevel&m=107959907311576&w=2

This thread was started by Christoph Hellwig 13 months ago to discuss
the possibility of using O_NONBLOCK in nfsd to make the combination
of NFS and Hierarchical Storage Managers play nicely together.  The
thread petered out with no firm conclusion but no major objections
either.

There's now a patch to actually do it, see

http://marc.theaimsgroup.com/?l=linux-nfs&m=111087383132762&w=2

Does anyone want to give a firm yea or nea to this approach?  I'd
really like to get this patch moving in the pipeline.

Greg.
-- 
Greg Banks, R&D Software Engineer, SGI Australian Software Group.
I don't speak for SGI.




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: O_NONBLOCK for regular files

[Trond Myklebust]

on den 13.04.2005 Klokka 10:45 (+1000) skreiv Greg Banks:

> There's now a patch to actually do it, see
> 
> http://marc.theaimsgroup.com/?l=linux-nfs&m=111087383132762&w=2
> 
> Does anyone want to give a firm yea or nea to this approach?  I'd
> really like to get this patch moving in the pipeline.

Nay...

if (rqstp->rq_vers == 3)

  is BAD! 

We're already busy with rq_vers == 4 (which has the same issues). This
sort of condition needs to be handled in a more version-independent
manner.

Cheers,
  Trond

-- 
Trond Myklebust <trond.myklebust@fys.uio.no>

Re: O_NONBLOCK for regular files

[Greg Banks]

On Wed, 2005-04-13 at 15:56, Trond Myklebust wrote:
> on den 13.04.2005 Klokka 10:45 (+1000) skreiv Greg Banks:
> 
> > There's now a patch to actually do it, see
> > 
> > http://marc.theaimsgroup.com/?l=linux-nfs&m=111087383132762&w=2
> > 
> > Does anyone want to give a firm yea or nea to this approach?  I'd
> > really like to get this patch moving in the pipeline.
> 
> Nay...
> 
> if (rqstp->rq_vers == 3)
> 
>   is BAD! 
> 
> We're already busy with rq_vers == 4 (which has the same issues). This
> sort of condition needs to be handled in a more version-independent
> manner.

Yes, Neil already made this point.  I have a new version of the patch
which does

if (rqstp->rq_vers >= 3)

which I'll resend if there are no other issues.  I assume that'll
be good enough.

Any other issues?

Greg.
-- 
Greg Banks, R&D Software Engineer, SGI Australian Software Group.
I don't speak for SGI.

Re: O_NONBLOCK for regular files

[Trond Myklebust]

on den 13.04.2005 Klokka 16:03 (+1000) skreiv Greg Banks:

> Any other issues?
> 
> Greg.

Well, I'm sa bit curious as to what "iap->ia_valid |= ATTR_NO_BLOCK"
will do. AFAICS, there is nothing that actually tests for it.

Trond
-- 
Trond Myklebust <trond.myklebust@fys.uio.no>

Re: O_NONBLOCK for regular files

[Greg Banks]

On Wed, 2005-04-13 at 16:31, Trond Myklebust wrote:
> on den 13.04.2005 Klokka 16:03 (+1000) skreiv Greg Banks:
> 
> > Any other issues?
> > 
> > Greg.
> 
> Well, I'm sa bit curious as to what "iap->ia_valid |= ATTR_NO_BLOCK"
> will do. AFAICS, there is nothing that actually tests for it.

fs/xfs/linux-2.6/xfs_iops.c:linvfs_setattr() does

    535 #ifdef ATTR_NO_BLOCK
    536         if ((ia_valid & ATTR_NO_BLOCK))
    537                 flags |= ATTR_NONBLOCK;
    538 #endif

and later ATTR_NONBLOCK is used to change the behaviour of
DMAPI callouts from XFS.

Greg.
-- 
Greg Banks, R&D Software Engineer, SGI Australian Software Group.
I don't speak for SGI.

O_NONBLOCK for regular files

[Christoph Hellwig]

Guy,

any opinion on whether we should allow O_NONBLOCK for regular files?
XFS currently has this in as a leftover from IRIX, it's used to to allow
the HSM support code in XFS (not in the kernel.org code due to the horrible
design, just in the oss.sgi.com tree) to return EAGAIN to the nfs server
so it doesn't block all nfsd threads on possible long-enduring HSM operations.

People want to bring that thing over to Linux now which brought that
O_NONBLOCK thing on my radar.   I don't see any direct problems with this
thing, but I don't want it as an undocumented XFS extension.

Re: O_NONBLOCK for regular files

[viro]

On Thu, Mar 18, 2004 at 08:36:28AM +0000, Christoph Hellwig wrote:
> Guy,
> 
> any opinion on whether we should allow O_NONBLOCK for regular files?
> XFS currently has this in as a leftover from IRIX, it's used to to allow
> the HSM support code in XFS (not in the kernel.org code due to the horrible
> design, just in the oss.sgi.com tree) to return EAGAIN to the nfs server
> so it doesn't block all nfsd threads on possible long-enduring HSM operations.
> 
> People want to bring that thing over to Linux now which brought that
> O_NONBLOCK thing on my radar.   I don't see any direct problems with this
> thing, but I don't want it as an undocumented XFS extension.

How do they handle fcntl() (re)setting O_NONBLOCK?

Re: O_NONBLOCK for regular files

[Christoph Hellwig]

On Thu, Mar 18, 2004 at 08:42:59AM +0000, viro@parcelfarce.linux.theplanet.co.uk wrote:
> How do they handle fcntl() (re)setting O_NONBLOCK?

The code checks file->f_flags & O_NONBLOCK, so it's exactly the same as for 
all other uses.

Re: O_NONBLOCK for regular files

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 931 bytes --]

On Thu, 18 Mar 2004 08:36:28 +0000 Christoph Hellwig <hch@infradead.org> wrote:
>
> any opinion on whether we should allow O_NONBLOCK for regular files?
> XFS currently has this in as a leftover from IRIX, it's used to to allow
> the HSM support code in XFS (not in the kernel.org code due to the horrible
> design, just in the oss.sgi.com tree) to return EAGAIN to the nfs server
> so it doesn't block all nfsd threads on possible long-enduring HSM operations.
> 
> People want to bring that thing over to Linux now which brought that
> O_NONBLOCK thing on my radar.   I don't see any direct problems with this
> thing, but I don't want it as an undocumented XFS extension.

We honour O_NONBLOCK when opening files with active leases i.e. the open
does not block (returns EWOULDBLOCK), but the lease holder is still
notified.

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: O_NONBLOCK for regular files

[Jamie Lokier]

Christoph Hellwig wrote:
> It's used to to allow the HSM support code in XFS (not in the
> kernel.org code due to the horrible design, just in the oss.sgi.com
> tree) to return EAGAIN to the nfs server so it doesn't block all
> nfsd threads on possible long-enduring HSM operations.

How does nfsd get notified when the operation is possible?  Does
poll() work?

If poll works that implies a system call returning EAGAIN triggers at
least some part of the HSM operation (whichever part is slow) in the
background -- in which case, isn't the existing AIO interface more
appropriate?

-- Jamie