Linux filesystem development
 help / color / mirror / Atom feed
  • * Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024
           [not found]                 ` <20260120174659.GE6263@brightrain.aerifal.cx>
           [not found]                   ` <lhubjio5dsb.fsf@oldenburg.str.redhat.com>
    @ 2026-01-20 20:11                   ` Paul Eggert
      2026-01-20 20:35                   ` Alejandro Colomar
      2 siblings, 0 replies; 23+ messages in thread
    From: Paul Eggert @ 2026-01-20 20:11 UTC (permalink / raw)
      To: Rich Felker
      Cc: Alejandro Colomar, Vincent Lefevre, Jan Kara, Alexander Viro,
    	Christian Brauner, linux-fsdevel, linux-api, GNU libc development,
    	Zack Weinberg
    
    On 2026-01-20 09:46, Rich Felker wrote:
    > the job of the man pages absolutely is not "to tell people how to
    > program". It's to document behaviors.
    
    In practice man pages do both. When I type "man close" on GNU/Linux I 
    see text like the text quoted below, and as a C programmer I appreciate 
    getting advice like this when the situation is sufficiently tricky.
    
    ----
    
    Any record locks (see fcntl(2)) held on the file it was associated with, 
    and owned by the process, are removed regardless of the file descriptor 
    that was used to obtain the lock. This has some unfortunate consequences 
    and one should be extra careful when using advisory record locking. See 
    fcntl(2) for discussion of the risks and consequences as well as for the 
    (probably preferred) open file description locks.
    
    ^ permalink raw reply	[flat|nested] 23+ messages in thread
  • * Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024
           [not found]                 ` <20260120174659.GE6263@brightrain.aerifal.cx>
           [not found]                   ` <lhubjio5dsb.fsf@oldenburg.str.redhat.com>
      2026-01-20 20:11                   ` Paul Eggert
    @ 2026-01-20 20:35                   ` Alejandro Colomar
      2026-01-20 20:42                     ` Alejandro Colomar
      2 siblings, 1 reply; 23+ messages in thread
    From: Alejandro Colomar @ 2026-01-20 20:35 UTC (permalink / raw)
      To: Rich Felker
      Cc: Zack Weinberg, Vincent Lefevre, Jan Kara, Alexander Viro,
    	Christian Brauner, linux-fsdevel, linux-api, GNU libc development
    
    [-- Attachment #1: Type: text/plain, Size: 9114 bytes --]
    
    Hi Rich, Zack,
    
    On Tue, Jan 20, 2026 at 12:46:59PM -0500, Rich Felker wrote:
    > On Tue, Jan 20, 2026 at 12:05:52PM -0500, Zack Weinberg wrote:
    > > > On Fri, May 23, 2025 at 02:10:57PM -0400, Zack Weinberg wrote:
    
    [...]
    
    > > Now, the abstract correct behavior is secondary to the fact that we
    > > know there are both systems where close should not be retried after
    > > EINTR (Linux) and systems where the fd is still open after EINTR
    > > (HP-UX).  But it is my position that *portable code* should assume the
    > > Linux behavior, because that is the safest option.  If you assume the
    > > HP-UX behavior on a machine that implements the Linux behavior, you
    > > might close some unrelated file out from under yourself (probably but
    > > not necessarily a different thread).  If you assume the Linux behavior
    > > on a machine that implements the HP-UX behavior, you have leaked a
    > > file descriptor; the worst things that can do are much less severe.
    > 
    > Unfortunately, regardless of what happens, code portable to old
    > systems needs to avoid getting in the situation to begin with. By
    > either not installing interrupting signal handlers or blocking EINTR
    > around close.
    
    [...]
    
    > > > While I agree with all of this, I think the tone is way too
    > > > proscriptive. The man pages are to document the behaviors, not tell
    > > > people how to program.
    > > 
    > > I could be persuaded to tone it down a little but in this case I think
    > > the man page's job *is* to tell people how to program.  We know lots of
    > > existing code has gotten the fine details of close() wrong and we are
    > > trying to document how to do it right.
    > 
    > No, the job of the man pages absolutely is not "to tell people how to
    > program". It's to document behaviors. They are not a programming
    > tutorial. They are not polemic diatribes. They are unbiased statements
    > of facts. Facts of what the standards say and what implementations do,
    > that equip programmers with the knowledge they need to make their own
    > informed decisions, rather than blindly following what someone who
    > thinks they know better told them to do.
    
    This reminds me a little bit of the realloc(p,0) fiasco of C89 and
    glibc.
    
    In most cases, I agree with you that manual pages are and should be
    aseptic, there are cases where I think the manual page needs to be
    tutorial.  Especially when there's such a mess, we need to both explain
    all the possible behaviors (or at least mention them to some degree).
    
    But for example, there's the case of realloc(p,0), where we have
    a fiasco that was pushed by a compoundment of wrong decisions by the
    C Committee, and prior to that from System V.  We're a bit lucky that
    C17 accidentally broke it so badly that we now have it as UB, and that
    gives us the opportunity to fix it now (which BTW might also be the case
    for close(2)).
    
    In the case of realloc(3), I went and documented in the manual page that
    glibc is broken, and that ISO C is also broken.
    
    	STANDARDS
    	     malloc()
    	     free()
    	     calloc()
    	     realloc()
    		    C23, POSIX.1‐2024.
    
    	     reallocarray()
    		    POSIX.1‐2024.
    
    	   realloc(p, 0)
    	     The  behavior of realloc(p, 0) in glibc doesn’t conform to
    	     any of C99, C11, POSIX.1‐2001, POSIX.1‐2004, POSIX.1‐2008,
    	     POSIX.1‐2013,  POSIX.1‐2017,  or  POSIX.1‐2024.   The  C17
    	     specification  was changed to make it conforming, but that
    	     specification made it impossible to write code that  reli‐
    	     ably  determines if the input pointer is freed after real‐
    	     loc(p, 0), and C23 changed it again to make this undefined
    	     behavior, acknowledging that  the  C17  specification  was
    	     broad enough, so that undefined behavior wasn’t worse than
    	     that.
    
    	     reallocarray() suffers the same issues in glibc.
    
    	     musl  libc  and  the BSDs conform to all versions of ISO C
    	     and POSIX.1.
    
    	     gnulib provides the realloc‐posix module,  which  provides
    	     wrappers  realloc() and reallocarray() that conform to all
    	     versions of ISO C and POSIX.1.
    
    	     There’s a proposal to standardize the BSD behavior: https:
    	     //www.open-std.org/jtc1/sc22/wg14/www/docs/n3621.txt.
    
    	HISTORY
    	     malloc()
    	     free()
    	     calloc()
    	     realloc()
    		    POSIX.1‐2001, C89.
    
    	     reallocarray()
    		    glibc 2.26.  OpenBSD 5.6, FreeBSD 11.0.
    
    	     malloc() and related functions rejected sizes greater than
    	     PTRDIFF_MAX starting in glibc 2.30.
    
    	     free() preserved errno starting in glibc 2.33.
    
    	   realloc(p, 0)
    	     C89 was ambiguous in its specification of  realloc(p,  0).
    	     C99 partially fixed this.
    
    	     The  original implementation in glibc would have been con‐
    	     forming to C99.  However, and ironically, trying to comply
    	     with C99 before the standard was released,  glibc  changed
    	     its  behavior  in glibc 2.1.1 into something that ended up
    	     not conforming to the final C99 specification (but this is
    	     debated, as the wording of the standard seems self‐contra‐
    	     dicting).
    
    	...
    
    	BUGS
    	     Programmers  would  naturally  expect  by  induction  that
    	     realloc(p, size)  is  consistent  with  free(p)  and  mal‐
    	     loc(size),  as  that  is the behavior in the general case.
    	     This is not explicitly required by  POSIX.1‐2024  or  C11,
    	     but  all  conforming  implementations  are consistent with
    	     that.
    
    	     The glibc implementation of realloc()  is  not  consistent
    	     with  that,  and as a consequence, it is dangerous to call
    	     realloc(p, 0) in glibc.
    
    	     A  trivial  workaround  for  glibc  is   calling   it   as
    	     realloc(p, size?size:1).
    
    	     The  workaround for reallocarray() in glibc ——which shares
    	     the         same          bug——          would          be
    	     reallocarray(p, n?n:1, size?size:1).
    
    
    Apart from documenting that glibc and ISO C are broken, we document how
    to best deal with it (see the last paragraph in BUGS).  This is
    necessary because I fear that just by documenting the different
    behaviors, programmers would still not know what to do with that.
    Just take into account that even several members of the committee don't
    know how to deal with it.
    
    I'd be willing to have something similar for close(2).
    
    
    Have a lovely night!
    Alex
    
    P.S.:  I have great news about realloc(p,0)!  Microsoft is on-board with
    the change.  They told me they like the proposal, and are willing to
    fix their realloc(3) implementation.  They'll now conduct tests to make
    sure it doesn't break anything too badly, and will come back to me with
    any feedback they have from those tests.
    
    I'll put the standards proposal for realloc(3) on hold, waiting for
    Microsoft's feedback.
    
    > > > Aside: the reason EINTR *has to* be specified this way is that pthread
    > > > cancellation is aligned with EINTR. If EINTR were defined to have
    > > > closed the fd, then acting on cancellation during close would also
    > > > have closed the fd, but the cancellation handler would have no way to
    > > > distinguish this, leading to a situation where you're forced to either
    > > > leak fds or introduce a double-close vuln.
    > > 
    > > The correct way to address this would be to make close() not be a
    > > cancellation point.
    > 
    > This would also be a desirable change, one I would support if other
    > implementors are on-board with pushing for it.
    > 
    > > > An outline of what I'd like to see instead:
    > > >
    > > > - Clear explanation of why double-close is a serious bug that must
    > > >   always be avoided. (I think we all agree on this.)
    > > >
    > > > - Statement that the historical Linux/glibc behavior and current POSIX
    > > >   requirement differ, without language that tries to paint the POSIX
    > > >   behavior as a HP-UX bug/quirk. Possibly citing real sources/history
    > > >   of the issue (Austin Group tracker items 529, 614; maybe others).
    > > >
    > > > - Consequence of just assuming the Linux behavior (fd leaks on
    > > >   conforming systems).
    > > >
    > > > - Consequences of assuming the POSIX behavior (double-close vulns on
    > > >   GNU/Linux, maybe others).
    > > >
    > > > - Survey of methods for avoiding the problem (ways to preclude EINTR,
    > > >   possibly ways to infer behavior, etc).
    > > 
    > > This outline seems more or less reasonable to me but, if it's me
    > > writing the text, I _will_ characterize what POSIX currently says
    > > about EINTR returns from close() as a bug in POSIX.  As far as I'm
    > > concerned, that is a fact, not polemic.
    > > 
    > > I have found that arguing with you in particular, Rich, is generally
    > > not worth the effort.  Therefore, unless you reply and _accept_ that
    > > the final version of the close manpage will say that POSIX is buggy,
    > > I am not going to write another version of this text, nor will I be
    > > drawn into further debate.
    > 
    > I will not accept that because it's a gross violation of the
    > responsibility of document writing.
    > 
    > Rich
    
    -- 
    <https://www.alejandro-colomar.es>
    
    [-- Attachment #2: signature.asc --]
    [-- Type: application/pgp-signature, Size: 833 bytes --]
    
    ^ permalink raw reply	[flat|nested] 23+ messages in thread
  • * Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024
           [not found]       ` <20250517133251.GY1509@brightrain.aerifal.cx>
           [not found]         ` <5jm7pblkwkhh4frqjptrw4ll4nwncn22ep2v7sli6kz5wxg5ik@pbnj6wfv66af>
    @ 2026-02-06 15:13         ` Vincent Lefevre
      1 sibling, 0 replies; 23+ messages in thread
    From: Vincent Lefevre @ 2026-02-06 15:13 UTC (permalink / raw)
      To: Rich Felker
      Cc: Alejandro Colomar, Jan Kara, Alexander Viro, Christian Brauner,
    	linux-fsdevel, linux-api, libc-alpha
    
    On 2025-05-17 09:32:52 -0400, Rich Felker wrote:
    > On Fri, May 16, 2025 at 04:39:57PM +0200, Vincent Lefevre wrote:
    > > On 2025-05-16 09:05:47 -0400, Rich Felker wrote:
    > > > FWIW musl adopted the EINPROGRESS as soon as we were made aware of the
    > > > issue, and later changed it to returning 0 since applications
    > > > (particularly, any written prior to this interpretation) are prone to
    > > > interpret EINPROGRESS as an error condition rather than success and
    > > > possibly misinterpret it as meaning the fd is still open and valid to
    > > > pass to close again.
    > > 
    > > If I understand correctly, this is a poor choice. POSIX.1-2024 says:
    > > 
    > > ERRORS
    > >   The close() and posix_close() functions shall fail if:
    > > [...]
    > >   [EINPROGRESS]
    > >     The function was interrupted by a signal and fildes was closed
    > >     but the close operation is continuing asynchronously.
    > > 
    > > But this does not mean that the asynchronous close operation will
    > > succeed.
    > 
    > There are no asynchronous behaviors specified for there to be a
    > conformance distinction here. The only observable behaviors happen
    > instantly, mainly the release of the file descriptor and the process's
    > handle on the underlying resource. Abstractly, there is no async
    > operation that could succeed or fail.
    
    Sorry, this is old. But a consequence may be memory leak if something
    unexpected occurred during what was done asynchronously. There is no
    guarantee that *every* resource has been released.
    
    -- 
    Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
    100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
    Work: CR INRIA - computer arithmetic / Pascaline project (LIP, ENS-Lyon)
    
    ^ permalink raw reply	[flat|nested] 23+ messages in thread

  • end of thread, other threads:[~2026-02-06 15:20 UTC | newest]
    
    Thread overview: 23+ messages (download: mbox.gz follow: Atom feed
    -- links below jump to the message on this page --
         [not found] <a5tirrssh3t66q4vpwpgmxgxaumhqukw5nyxd4x6bevh7mtuvy@wtwdsb4oloh4>
         [not found] ` <efaffc5a404cf104f225c26dbc96e0001cede8f9.1747399542.git.alx@kernel.org>
         [not found]   ` <20250516130547.GV1509@brightrain.aerifal.cx>
         [not found]     ` <20250516143957.GB5388@qaa.vinc17.org>
         [not found]       ` <20250517133251.GY1509@brightrain.aerifal.cx>
         [not found]         ` <5jm7pblkwkhh4frqjptrw4ll4nwncn22ep2v7sli6kz5wxg5ik@pbnj6wfv66af>
         [not found]           ` <8c47e10a-be82-4d5b-a45e-2526f6e95123@app.fastmail.com>
         [not found]             ` <20250524022416.GB6263@brightrain.aerifal.cx>
         [not found]               ` <1571b14d-1077-4e81-ab97-36e39099761e@app.fastmail.com>
         [not found]                 ` <20260120174659.GE6263@brightrain.aerifal.cx>
         [not found]                   ` <lhubjio5dsb.fsf@oldenburg.str.redhat.com>
         [not found]                     ` <20260120190010.GF6263@brightrain.aerifal.cx>
    2026-01-20 20:05                       ` [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024 Florian Weimer
    2026-01-20 20:11                   ` Paul Eggert
    2026-01-20 20:35                   ` Alejandro Colomar
    2026-01-20 20:42                     ` Alejandro Colomar
    2026-01-23  0:33                       ` Zack Weinberg
    2026-01-23  1:02                         ` Alejandro Colomar
    2026-01-23  1:38                           ` Al Viro
    2026-01-23 14:44                             ` Alejandro Colomar
    2026-01-23 14:05                           ` Zack Weinberg
    2026-01-24 19:34                         ` The 8472
    2026-01-24 21:39                           ` Rich Felker
    2026-01-24 21:57                             ` The 8472
    2026-01-25 15:37                               ` Zack Weinberg
    2026-01-26  8:51                                 ` Florian Weimer
    2026-01-26 12:15                                 ` Jan Kara
    2026-01-26 13:53                                   ` The 8472
    2026-01-26 15:56                                     ` Jan Kara
    2026-01-26 16:43                                       ` Jeff Layton
    2026-01-26 23:01                                         ` Trevor Gross
    2026-01-27  0:49                                           ` Jeff Layton
    2026-01-28 16:58                                             ` Zack Weinberg
    2026-02-05  9:34                                               ` Jan Kara
    2026-02-06 15:13         ` Vincent Lefevre
    

    This is a public inbox, see mirroring instructions
    for how to clone and mirror all data and code used for this inbox