Re: OFD ("file private") locks and NFS

Re: OFD ("file private") locks and NFS

[Michael Kerrisk (man-pages)]

[CC+= linux-nfs@]

On 04/29/2014 10:38 AM, Michael Kerrisk (man-pages) wrote:
> Hi Jeff,
> 
> I've been looking a bit at the fcntl() documentation of traditional 
> (F_SETLK) record locking, and a question just jumped out at me. Is 
> it worth considering some future-proofing in the design of OFD locks
> ("open file description locks", formerly known as "file-private locks")?
> 
> What I am thinking of here is that on some systems, the traditional
> 'struct flock' has a nonstandard field, l_sysid, that is used on F_GETLK 
> to identify the remote system on which a lock is held. Should the design
> of OFD locks allow for such a field (now, or in the future), which might 
> be useful in the context of locking on network file systems such as NFS.
> 
> Put more simply, should the new OFD locking system be using a new
> structure for describing locks, rather than the traditional 'struct
> flock'? Defining a new structure, might be useful to allow for
> future extensions to the API.

Just add one further detail here. What I'm thinking is, maybe instead there
should be something like:

struct flockx {
    int flags;
    /* Other fields like 'struct flock' */
    char reserved[32];	/* Or some suitable value */
}

That flags field might always be zero for now, but in the future it 
could be used on the setlk and getlk operations to indicate the presence
of additional fields in the structure.

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: OFD ("file private") locks and NFS

[Jeff Layton]

On Tue, 29 Apr 2014 10:47:07 +0200
"Michael Kerrisk (man-pages)" <mtk.manpages@gmail.com> wrote:

> [CC+= linux-nfs@]
> 
> On 04/29/2014 10:38 AM, Michael Kerrisk (man-pages) wrote:
> > Hi Jeff,
> > 
> > I've been looking a bit at the fcntl() documentation of traditional 
> > (F_SETLK) record locking, and a question just jumped out at me. Is 
> > it worth considering some future-proofing in the design of OFD locks
> > ("open file description locks", formerly known as "file-private locks")?
> > 
> > What I am thinking of here is that on some systems, the traditional
> > 'struct flock' has a nonstandard field, l_sysid, that is used on F_GETLK 
> > to identify the remote system on which a lock is held. Should the design
> > of OFD locks allow for such a field (now, or in the future), which might 
> > be useful in the context of locking on network file systems such as NFS.
> > 
> > Put more simply, should the new OFD locking system be using a new
> > structure for describing locks, rather than the traditional 'struct
> > flock'? Defining a new structure, might be useful to allow for
> > future extensions to the API.
> 
> Just add one further detail here. What I'm thinking is, maybe instead there
> should be something like:
> 
> struct flockx {
>     int flags;
>     /* Other fields like 'struct flock' */
>     char reserved[32];	/* Or some suitable value */
> }
> 
> That flags field might always be zero for now, but in the future it 
> could be used on the setlk and getlk operations to indicate the presence
> of additional fields in the structure.
> 
> Cheers,
> 
> Michael
> 

I considered that early on when I did this, but I don't think it's
really helpful. I'm just not a fan of padding out structs when it's not
clear what would eventually go in there.

The problem is that once actually go to try to convert those from
"reserved" to something usable, it becomes a nightmare for userland to
figure that out. How do I know whether my kernel supports the stuff I
put in those fields or will just ignore them?

If we really find later that we need to do something like this, I think
we'd be better off adding a new set of cmd values along with the
"extended" struct, or possibly a new syscall. Some of the samba folks
were interested in an async locking mechanism too, so something like
that could be added in conjunction with such an interface.

-- 
Jeff Layton <jlayton@poochiereds.net>

Re: OFD ("file private") locks and NFS

[Michael Kerrisk (man-pages)]

Hi Jeff,

On Tue, Apr 29, 2014 at 1:11 PM, Jeff Layton <jlayton@poochiereds.net> wrote:
> On Tue, 29 Apr 2014 10:47:07 +0200
> "Michael Kerrisk (man-pages)" <mtk.manpages@gmail.com> wrote:
>
>> [CC+= linux-nfs@]
>>
>> On 04/29/2014 10:38 AM, Michael Kerrisk (man-pages) wrote:
>> > Hi Jeff,
>> >
>> > I've been looking a bit at the fcntl() documentation of traditional
>> > (F_SETLK) record locking, and a question just jumped out at me. Is
>> > it worth considering some future-proofing in the design of OFD locks
>> > ("open file description locks", formerly known as "file-private locks")?
>> >
>> > What I am thinking of here is that on some systems, the traditional
>> > 'struct flock' has a nonstandard field, l_sysid, that is used on F_GETLK
>> > to identify the remote system on which a lock is held. Should the design
>> > of OFD locks allow for such a field (now, or in the future), which might
>> > be useful in the context of locking on network file systems such as NFS.
>> >
>> > Put more simply, should the new OFD locking system be using a new
>> > structure for describing locks, rather than the traditional 'struct
>> > flock'? Defining a new structure, might be useful to allow for
>> > future extensions to the API.
>>
>> Just add one further detail here. What I'm thinking is, maybe instead there
>> should be something like:
>>
>> struct flockx {
>>     int flags;
>>     /* Other fields like 'struct flock' */
>>     char reserved[32];        /* Or some suitable value */
>> }
>>
>> That flags field might always be zero for now, but in the future it
>> could be used on the setlk and getlk operations to indicate the presence
>> of additional fields in the structure.
>>
>> Cheers,
>>
>> Michael
>>
>
> I considered that early on when I did this, but I don't think it's
> really helpful. I'm just not a fan of padding out structs when it's not
> clear what would eventually go in there.

Okay.

> The problem is that once actually go to try to convert those from
> "reserved" to something usable, it becomes a nightmare for userland to
> figure that out. How do I know whether my kernel supports the stuff I
> put in those fields or will just ignore them?

(That was my purpose with the "flags" field.)

> If we really find later that we need to do something like this, I think
> we'd be better off adding a new set of cmd values along with the
> "extended" struct, or possibly a new syscall. Some of the samba folks
> were interested in an async locking mechanism too, so something like
> that could be added in conjunction with such an interface.

Sounds reasonable. I just thought it worth checking. Thanks for the
quick response.

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: OFD ("file private") locks and NFS

[Matt W. Benjamin]

Hi Jeff,

Something which came up on the last Ganesha conn call is that we have
a pretty strong need for some ability to wait on a set of locks, and perhaps
receive events.  Frank Filz believed that you had made a proposal which
would cover this.  Can you elaborate on that?

Thanks,

Matt

----- "Jeff Layton" <jlayton@poochiereds.net> wrote:

> 
> If we really find later that we need to do something like this, I
> think
> we'd be better off adding a new set of cmd values along with the
> "extended" struct, or possibly a new syscall. Some of the samba folks
> were interested in an async locking mechanism too, so something like
> that could be added in conjunction with such an interface.
> 


-- 
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI  48104

http://linuxbox.com

tel.  734-761-4689 
fax.  734-769-8938 
cel.  734-216-5309 

Re: OFD ("file private") locks and NFS

[Jeff Layton]

On Tue, 29 Apr 2014 07:40:08 -0400 (EDT)
"Matt W. Benjamin" <matt@linuxbox.com> wrote:

> Hi Jeff,
> 
> Something which came up on the last Ganesha conn call is that we have
> a pretty strong need for some ability to wait on a set of locks, and perhaps
> receive events.  Frank Filz believed that you had made a proposal which
> would cover this.  Can you elaborate on that?
> 
> Thanks,
> 
> Matt
> 

No, there's no mechanism to wait on a set of locks from within the
context of a single thread of execution or to receive "events". Again,
that would be a new API beyond what I've been proposing over the last
several months.
-- 
Jeff Layton <jlayton@poochiereds.net>

RE: OFD ("file private") locks and NFS

[Frank Filz]

> On Tue, 29 Apr 2014 07:40:08 -0400 (EDT) "Matt W. Benjamin"
> <matt@linuxbox.com> wrote:
> 
> > Hi Jeff,
> >
> > Something which came up on the last Ganesha conn call is that we have
> > a pretty strong need for some ability to wait on a set of locks, and
> > perhaps receive events.  Frank Filz believed that you had made a
> > proposal which would cover this.  Can you elaborate on that?
> >
> > Thanks,
> >
> > Matt
> >
> 
> No, there's no mechanism to wait on a set of locks from within the context
of
> a single thread of execution or to receive "events". Again, that would be
a
> new API beyond what I've been proposing over the last several months.

Some kind of facility to enable one user space thread to wait on multiple
blocked locks would definitely be helpful to user space servers.

Our current plan is to have a pool of threads, and dispatch blocking locks
to them. If that pool is exhausted, all further locks would be dispatched to
a single thread that would poll for locks.

Frank