[RFC][PATCH 0/2] glibc: header file and manual updates for file-private locks

[RFC][PATCH 0/2] glibc: header file and manual updates for file-private locks

[Jeff Layton]

I've been working on a set of patches to add a new type file record lock
to the Linux kernel that have less troublesome semantics on close and
that are usable by threaded applications.

I have a blog post here with more details:

    http://jtlayton.wordpress.com/2014/01/07/file-private-posix-locks-aka-un-posix-locks/

...and the latest kernel patchset is here:

    https://lkml.org/lkml/2014/1/14/554

This patchset represents a set of corresponding updates for glibc. It
just adds the new cmd values to fcntl.h, and a manual update that
discusses how to use the new locks.

I'm currently shooting for a merge in v3.15 kernel so I'm posting this
as an RFC for now.

Comments and suggestions appreciated...

Thanks,

Jeff Layton (2):
  fcntl.h: add new definitions for file-private lock cmd values
  manual: update manual to document file-private locks

 ChangeLog                                  |   5 +
 manual/llio.texi                           | 175 ++++++++++++++++++++++++++++-
 sysdeps/unix/sysv/linux/bits/fcntl-linux.h |  18 +++
 3 files changed, 196 insertions(+), 2 deletions(-)

-- 
1.8.5.3

[RFC][PATCH 1/2] fcntl.h: add new definitions for file-private lock cmd values

[Jeff Layton]

Signed-off-by: Jeff Layton <jlayton@redhat.com>
---
 ChangeLog                                  |  4 ++++
 sysdeps/unix/sysv/linux/bits/fcntl-linux.h | 18 ++++++++++++++++++
 2 files changed, 22 insertions(+)

diff --git a/ChangeLog b/ChangeLog
index 5e820f7e9900..db5b90385e2c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,7 @@
+2014-02-11  Jeff Layton  <jlayton@redhat.com>
+
+	* add new fcntl cmd macros for file-private locks
+
 2014-02-11  Stefan Liebler  <stli@linux.vnet.ibm.com>
 
 	[BZ #16447]
diff --git a/sysdeps/unix/sysv/linux/bits/fcntl-linux.h b/sysdeps/unix/sysv/linux/bits/fcntl-linux.h
index cd4722b836ea..71fd80dba2d8 100644
--- a/sysdeps/unix/sysv/linux/bits/fcntl-linux.h
+++ b/sysdeps/unix/sysv/linux/bits/fcntl-linux.h
@@ -117,6 +117,24 @@
 # define F_SETLKW64	14	/* Set record locking info (blocking).	*/
 #endif
 
+/*
+ * fd "private" POSIX locks.
+ *
+ * Usually POSIX locks held by a process are released on *any* close and are
+ * not inherited across a fork().
+ *
+ * These cmd values will set locks that conflict with normal POSIX locks, but
+ * are "owned" by the opened file, not the process. This means that they are
+ * inherited across fork() like BSD (flock) locks, and they are only released
+ * automatically when the last reference to the the open file against which
+ * they were acquired is put.
+ */
+#ifndef F_GETLKP
+# define F_GETLKP	36
+# define F_SETLKP	37
+# define F_SETLKPW	38
+#endif
+
 #ifdef __USE_LARGEFILE64
 # define O_LARGEFILE __O_LARGEFILE
 #endif
-- 
1.8.5.3

Re: [RFC][PATCH 1/2] fcntl.h: add new definitions for file-private lock cmd values

[Roland McGrath]

> +2014-02-11  Jeff Layton  <jlayton@redhat.com>
> +
> +	* add new fcntl cmd macros for file-private locks

Proper format is:

	* sysdeps/unix/sysv/linux/bits/fcntl-linux.h
	(F_GETLKP, F_SETLKP, F_SETLKPW): New macros.

> +/*
> + * fd "private" POSIX locks.
> + *
> + * Usually POSIX locks held by a process are released on *any* close and are
> + * not inherited across a fork().

Use GNU style for comments, as you see elsewhere in the file.  Use two
spaces between sentences.  Don't use "function()" when referring to a
function, just the name.

> +#ifndef F_GETLKP
> +# define F_GETLKP	36
> +# define F_SETLKP	37
> +# define F_SETLKPW	38
> +#endif

These need to be protected by #ifdef __USE_GNU.

Re: [RFC][PATCH 1/2] fcntl.h: add new definitions for file-private lock cmd values

[Jeff Layton]

On Tue, 11 Feb 2014 13:49:15 -0800 (PST)
Roland McGrath <roland@hack.frob.com> wrote:

> > +2014-02-11  Jeff Layton  <jlayton@redhat.com>
> > +
> > +	* add new fcntl cmd macros for file-private locks
> 
> Proper format is:
> 
> 	* sysdeps/unix/sysv/linux/bits/fcntl-linux.h
> 	(F_GETLKP, F_SETLKP, F_SETLKPW): New macros.
> 
> > +/*
> > + * fd "private" POSIX locks.
> > + *
> > + * Usually POSIX locks held by a process are released on *any* close and are
> > + * not inherited across a fork().
> 
> Use GNU style for comments, as you see elsewhere in the file.  Use two
> spaces between sentences.  Don't use "function()" when referring to a
> function, just the name.
> 
> > +#ifndef F_GETLKP
> > +# define F_GETLKP	36
> > +# define F_SETLKP	37
> > +# define F_SETLKPW	38
> > +#endif
> 
> These need to be protected by #ifdef __USE_GNU.

Thanks for the comments, Roland. I'll fix the patch accordingly.

My plan is to eventually submit this to become part of the POSIX
standard though they won't take it until there's at least one shipping
implementation. I assume that we just need to stick that inside of
__USE_GNU for now, and if POSIX eventually picks it up then we'll just
remove that protection?

Also, I'm a little curious in looking at this and have a (probably
stupid) question:

Why does glibc have its own definitions for the fcntl F_* cmd values
instead of using the ones in the uapi kernel headers? Would it not be
simpler to just have the fcntl.h include those instead of duplicating
them?

Thanks,
-- 
Jeff Layton <jlayton@redhat.com>

Re: [RFC][PATCH 1/2] fcntl.h: add new definitions for file-private lock cmd values

[Andreas Schwab]

Jeff Layton <jlayton@redhat.com> writes:

> Why does glibc have its own definitions for the fcntl F_* cmd values
> instead of using the ones in the uapi kernel headers? Would it not be
> simpler to just have the fcntl.h include those instead of duplicating
> them?

The uapi headers are useless for user space.  They refer to CONFIG_*
macros, and don't have suitable namespace control.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: [RFC][PATCH 1/2] fcntl.h: add new definitions for file-private lock cmd values

[Roland McGrath]

> My plan is to eventually submit this to become part of the POSIX
> standard though they won't take it until there's at least one shipping
> implementation. I assume that we just need to stick that inside of
> __USE_GNU for now, and if POSIX eventually picks it up then we'll just
> remove that protection?

Change it as appropriate, yes.

> Why does glibc have its own definitions for the fcntl F_* cmd values
> instead of using the ones in the uapi kernel headers? Would it not be
> simpler to just have the fcntl.h include those instead of duplicating
> them?

libc headers respect ISO C and POSIX name space constraints.
Kernel headers (even uapi ones) fail to.

[RFC][PATCH 2/2] manual: update manual to document file-private locks

[Jeff Layton]

Signed-off-by: Jeff Layton <jlayton@redhat.com>
---
 ChangeLog        |   1 +
 manual/llio.texi | 175 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 174 insertions(+), 2 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index db5b90385e2c..d2f9a52dd283 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,6 +1,7 @@
 2014-02-11  Jeff Layton  <jlayton@redhat.com>
 
 	* add new fcntl cmd macros for file-private locks
+	* add manual section about file-private locks
 
 2014-02-11  Stefan Liebler  <stli@linux.vnet.ibm.com>
 
diff --git a/manual/llio.texi b/manual/llio.texi
index 69b54c2838c9..533992a5b40b 100644
--- a/manual/llio.texi
+++ b/manual/llio.texi
@@ -57,6 +57,8 @@ directly.)
                                          flags associated with open files.
 * File Locks::                          Fcntl commands for implementing
                                          file locking.
+* File-private Locks::                  Fcntl commands for implementing
+                                         file-private locking.
 * Interrupt Input::                     Getting an asynchronous signal when
                                          input arrives.
 * IOCTLs::                              Generic I/O Control operations.
@@ -2890,7 +2892,7 @@ Get flags associated with the open file.  @xref{File Status Flags}.
 Set flags associated with the open file.  @xref{File Status Flags}.
 
 @item F_GETLK
-Get a file lock.  @xref{File Locks}.
+Get (test) a file lock.  @xref{File Locks}.
 
 @item F_SETLK
 Set or clear a file lock.  @xref{File Locks}.
@@ -2898,6 +2900,15 @@ Set or clear a file lock.  @xref{File Locks}.
 @item F_SETLKW
 Like @code{F_SETLK}, but wait for completion.  @xref{File Locks}.
 
+@item F_GETLKP
+Get (test) a file-private lock.  @xref{File Locks}.
+
+@item F_SETLKP
+Set or clear a file lock.  @xref{File Locks}.
+
+@item F_SETLKPW
+Like @code{F_SETLKP}, but wait for completion.  @xref{File Locks}.
+
 @item F_GETOWN
 Get process or process group ID to receive @code{SIGIO} signals.
 @xref{Interrupt Input}.
@@ -3576,6 +3587,10 @@ set_nonblock_flag (int desc, int value)
 
 @cindex file locks
 @cindex record locking
+This section describes "classic" record locks. There is also a newer type
+of record lock that is associated with the open file table entry instead
+of the process. @xref{File-private Locks}
+
 The remaining @code{fcntl} commands are used to support @dfn{record
 locking}, which permits multiple cooperating programs to prevent each
 other from simultaneously accessing parts of a file in error-prone
@@ -3641,7 +3656,9 @@ the file.
 @item pid_t l_pid
 This field is the process ID (@pxref{Process Creation Concepts}) of the
 process holding the lock.  It is filled in by calling @code{fcntl} with
-the @code{F_GETLK} command, but is ignored when making a lock.
+the @code{F_GETLK} command, but is ignored when making a lock. If the
+conflicting lock is a File-private lock (@pxref{File-private Locks}),
+then this field will be set to @math{-1}.
 @end table
 @end deftp
 
@@ -3817,6 +3834,160 @@ Remember that file locks are only a @emph{voluntary} protocol for
 controlling access to a file.  There is still potential for access to
 the file by programs that don't use the lock protocol.
 
+@node File-private Locks
+@section File-private Locks
+
+In contrast to classic record locks (@pxref{File Locks}), file-private
+locks are associated with an open file table entry rather than a
+process. File-private locks set on an open file descriptor will never
+conflict with existing file-private locks set on that file descriptor.
+
+File-private locks are also inherited by child processes across
+@code{fork} (@pxref{Creating a Process}), along with the file
+descriptor. For this reason, the @code{l_pid} field in @code{struct
+flock} is meaningless for file private locks. For the @code{F_GETLK} and
+@code{F_GETLKP} commands, the @code{l_pid} value is always set to
+@math{-1} if the blocking lock is a file-private one.
+
+Note that using @code{dup} (@pxref{Duplicating Descriptors}) to clone a
+file descriptor does not give you a new instance of the open file, but
+instead just clones a reference to an existing open file. Thus,
+file-private locks set on a file descriptor cloned by @code{dup} will
+never conflict with file-private locks set on the original descriptor.
+
+File-private locks always conflict with classic record locks, even if
+acquired by the same process or on the same open file descriptor.
+
+File-private locks use the same @code{struct flock} as classic POSIX
+locks as an argument (@pxref{File Locks}) and the macros for the
+cmd values are also declared in the header file @file{fcntl.h}.
+@pindex fcntl.h.
+
+@deftypevr Macro int F_GETLKP
+This macro is used as the @var{command} argument to @code{fcntl}, to
+specify that it should get information about a lock. This command
+requires a third argument of type @w{@code{struct flock *}} to be passed
+to @code{fcntl}, so that the form of the call is:
+
+If there is a lock already in place that would block the lock described
+by the @var{lockp} argument, information about that lock overwrites
+@code{*@var{lockp}}.  Existing locks are not reported if they are
+compatible with making a new lock as specified.  Thus, you should
+specify a lock type of @code{F_WRLCK} if you want to find out about both
+read and write locks, or @code{F_RDLCK} if you want to find out about
+write locks only.
+
+There might be more than one lock affecting the region specified by the
+@var{lockp} argument, but @code{fcntl} only returns information about
+one of them.  The @code{l_whence} member of the @var{lockp} structure is
+set to @code{SEEK_SET} and the @code{l_start} and @code{l_len} fields
+set to identify the locked region.
+
+If no lock applies, the only change to the @var{lockp} structure is to
+update the @code{l_type} to a value of @code{F_UNLCK}.
+
+The normal return value from @code{fcntl} with this command is an
+unspecified value other than @math{-1}, which is reserved to indicate an
+error.  The following @code{errno} error conditions are defined for
+this command:
+
+@table @code
+@item EBADF
+The @var{filedes} argument is invalid.
+
+@item EINVAL
+Either the @var{lockp} argument doesn't specify valid lock information,
+or the file associated with @var{filedes} doesn't support locks.
+@end table
+@end deftypevr
+
+@comment fcntl.h
+@comment POSIX.1
+@deftypevr Macro int F_SETLKP
+This macro is used as the @var{command} argument to @code{fcntl}, to
+specify that it should set or clear a lock.  This command requires a
+third argument of type @w{@code{struct flock *}} to be passed to
+@code{fcntl}, so that the form of the call is:
+
+@smallexample
+fcntl (@var{filedes}, F_SETLK, @var{lockp})
+@end smallexample
+
+If the opened file already has a lock on any part of the
+region, the old lock on that part is replaced with the new lock.  You
+can remove a lock by specifying a lock type of @code{F_UNLCK}.
+
+If the lock cannot be set, @code{fcntl} returns immediately with a value
+of @math{-1}.  This function does not block waiting for other tasks
+to release locks.  If @code{fcntl} succeeds, it returns a value other
+than @math{-1}.
+
+The following @code{errno} error conditions are defined for this
+function:
+
+@table @code
+@item EAGAIN
+@itemx EACCES
+The lock cannot be set because it is blocked by an existing lock on the
+file.  Some systems use @code{EAGAIN} in this case, and other systems
+use @code{EACCES}; your program should treat them alike, after
+@code{F_SETLKP}.  (@gnulinuxhurdsystems{} always use @code{EAGAIN}.)
+
+@item EBADF
+Either: the @var{filedes} argument is invalid; you requested a read lock
+but the @var{filedes} is not open for read access; or, you requested a
+write lock but the @var{filedes} is not open for write access.
+
+@item EINVAL
+Either the @var{lockp} argument doesn't specify valid lock information,
+or the file associated with @var{filedes} doesn't support locks.
+
+@item ENOLCK
+The system has run out of file lock resources; there are already too
+many file locks in place.
+
+Well-designed file systems never report this error, because they have no
+limitation on the number of locks.  However, you must still take account
+of the possibility of this error, as it could result from network access
+to a file system on another machine.
+@end table
+@end deftypevr
+
+@comment fcntl.h
+@comment POSIX.1
+@deftypevr Macro int F_SETLKPW
+This macro is used as the @var{command} argument to @code{fcntl}, to
+specify that it should set or clear a lock.  It is just like the
+@code{F_SETLKP} command, but causes the process to block (or wait)
+until the request can be specified.
+
+This command requires a third argument of type @code{struct flock *}, as
+for the @code{F_SETLKP} command.
+
+The @code{fcntl} return values and errors are the same as for the
+@code{F_SETLKP} command, but these additional @code{errno} error conditions
+are defined for this command:
+
+@table @code
+@item EINTR
+The function was interrupted by a signal while it was waiting.
+@xref{Interrupted Primitives}.
+
+@end table
+@end deftypevr
+
+File-private locks are useful in the same sorts of situations as classic
+record locks. They can also be used to synchronize file access between
+threads within the same process by giving each thread its own open file
+instance.
+
+Because they are only released automatically when the last reference to
+an open file is destroyed, file-private locks allow more assurance that
+the locks will not be released due to a library routine opening and
+closing a file without the application being aware.
+
+Like with classic record locks, file-private locks are also voluntary.
+
 @node Interrupt Input
 @section Interrupt-Driven Input
 
-- 
1.8.5.3

Re: [RFC][PATCH 0/2] glibc: header file and manual updates for file-private locks

[Mike Frysinger]

[-- Attachment #1: Type: text/plain, Size: 1465 bytes --]

On Tuesday, February 11, 2014 14:03:54 Jeff Layton wrote:
> I've been working on a set of patches to add a new type file record lock
> to the Linux kernel that have less troublesome semantics on close and
> that are usable by threaded applications.
> 
> I have a blog post here with more details:
> 
>    
> http://jtlayton.wordpress.com/2014/01/07/file-private-posix-locks-aka-un-po
> six-locks/
> 
> ...and the latest kernel patchset is here:
> 
>     https://lkml.org/lkml/2014/1/14/554
> 
> This patchset represents a set of corresponding updates for glibc. It
> just adds the new cmd values to fcntl.h, and a manual update that
> discusses how to use the new locks.
> 
> I'm currently shooting for a merge in v3.15 kernel so I'm posting this
> as an RFC for now.

disclaimer: haven't read the kernel patches.  just the patch summary and your 
blog post.

looks like the new API manipulates the existing POSIX lock only -- flock based 
locks are still parallel.  too crazy to have the new API grab both since 
you're bringing proper semantics that cover both ?

a good summary of Linux locking from the perspective of userspace was posted 
here sometime ago:
	http://0pointer.de/blog/projects/locking.html
random notes/limitations/bugs are also in the various man pages like fcntl(2)

would be nice if we could cover as many known limitations as possible with the 
new api so as to not need yet another in the future ;)
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [RFC][PATCH 0/2] glibc: header file and manual updates for file-private locks

[Jeff Layton]

[-- Attachment #1: Type: text/plain, Size: 2262 bytes --]

On Sun, 16 Feb 2014 03:27:05 -0500
Mike Frysinger <vapier@gentoo.org> wrote:

> On Tuesday, February 11, 2014 14:03:54 Jeff Layton wrote:
> > I've been working on a set of patches to add a new type file record lock
> > to the Linux kernel that have less troublesome semantics on close and
> > that are usable by threaded applications.
> > 
> > I have a blog post here with more details:
> > 
> >    
> > http://jtlayton.wordpress.com/2014/01/07/file-private-posix-locks-aka-un-po
> > six-locks/
> > 
> > ...and the latest kernel patchset is here:
> > 
> >     https://lkml.org/lkml/2014/1/14/554
> > 
> > This patchset represents a set of corresponding updates for glibc. It
> > just adds the new cmd values to fcntl.h, and a manual update that
> > discusses how to use the new locks.
> > 
> > I'm currently shooting for a merge in v3.15 kernel so I'm posting this
> > as an RFC for now.
> 
> disclaimer: haven't read the kernel patches.  just the patch summary and your 
> blog post.
> 
> looks like the new API manipulates the existing POSIX lock only -- flock based 
> locks are still parallel.  too crazy to have the new API grab both since 
> you're bringing proper semantics that cover both ?
> 

Correct, at least in the Linux implementation. The basic idea is that
these *are* POSIX locks, they just have different behavior wrt
inheritance and on close.

We should note that on BSD, flock() and fcntl() based locks do
conflict. If they end up implementing file-private locks on BSD, then
they may also conflict with flock() locks just like classic POSIX locks
do there.

> a good summary of Linux locking from the perspective of userspace was posted 
> here sometime ago:
> 	http://0pointer.de/blog/projects/locking.html
> random notes/limitations/bugs are also in the various man pages like fcntl(2)
> would be nice if we could cover as many known limitations as possible with the 
> new api so as to not need yet another in the future ;)
> -mike

Yep, I've seen Lennart's summary. I think that the only thing this
doesn't approach doesn't address is his comment about using locks
on world-readable files. I don't see that there's much we can do about
that.

-- 
Jeff Layton <jlayton@redhat.com>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 836 bytes --]