close(2) with EINTR has been changed by POSIX.1-2024

close(2) with EINTR has been changed by POSIX.1-2024

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2238 bytes --]

Hi,

I'm updating the manual pages for POSIX.1-2024, and have some doubts
about close(2).  The manual page for close(2) says (conforming to
POSIX.1-2008):

       The EINTR error is a somewhat special case.  Regarding the EINTR
       error, POSIX.1‐2008 says:

              If close() is interrupted by  a  signal  that  is  to  be
              caught,  it  shall  return -1 with errno set to EINTR and
              the state of fildes is unspecified.

       This permits the behavior that occurs on Linux  and  many  other
       implementations,  where,  as  with  other errors that may be re‐
       ported by close(), the  file  descriptor  is  guaranteed  to  be
       closed.   However, it also permits another possibility: that the
       implementation returns an EINTR error and  keeps  the  file  de‐
       scriptor open.  (According to its documentation, HP‐UX’s close()
       does this.)  The caller must then once more use close() to close
       the  file  descriptor, to avoid file descriptor leaks.  This di‐
       vergence in implementation behaviors provides a difficult hurdle
       for  portable  applications,  since  on  many   implementations,
       close() must not be called again after an EINTR error, and on at
       least one, close() must be called again.  There are plans to ad‐
       dress  this  conundrum for the next major release of the POSIX.1
       standard.

TL;DR: close(2) with EINTR is allowed to either leave the fd open or
closed, and Linux leaves it closed, while others (HP-UX only?) leaves it
open.

Now, POSIX.1-2024 says:

	If close() is interrupted by a signal that is to be caught, then
	it is unspecified whether it returns -1 with errno set to
	[EINTR] and fildes remaining open, or returns -1 with errno set
	to [EINPROGRESS] and fildes being closed, or returns 0 to
	indicate successful completion; [...]

<https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>

Which seems to bless HP-UX and screw all the others, requiring them to
report EINPROGRESS.

Was there any discussion about what to do in the Linux kernel?


Have a lovely night!
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Jan Kara]

Hi!

On Thu 15-05-25 23:33:22, Alejandro Colomar wrote:
> I'm updating the manual pages for POSIX.1-2024, and have some doubts
> about close(2).  The manual page for close(2) says (conforming to
> POSIX.1-2008):
> 
>        The EINTR error is a somewhat special case.  Regarding the EINTR
>        error, POSIX.1‐2008 says:
> 
>               If close() is interrupted by  a  signal  that  is  to  be
>               caught,  it  shall  return -1 with errno set to EINTR and
>               the state of fildes is unspecified.
> 
>        This permits the behavior that occurs on Linux  and  many  other
>        implementations,  where,  as  with  other errors that may be re‐
>        ported by close(), the  file  descriptor  is  guaranteed  to  be
>        closed.   However, it also permits another possibility: that the
>        implementation returns an EINTR error and  keeps  the  file  de‐
>        scriptor open.  (According to its documentation, HP‐UX’s close()
>        does this.)  The caller must then once more use close() to close
>        the  file  descriptor, to avoid file descriptor leaks.  This di‐
>        vergence in implementation behaviors provides a difficult hurdle
>        for  portable  applications,  since  on  many   implementations,
>        close() must not be called again after an EINTR error, and on at
>        least one, close() must be called again.  There are plans to ad‐
>        dress  this  conundrum for the next major release of the POSIX.1
>        standard.
> 
> TL;DR: close(2) with EINTR is allowed to either leave the fd open or
> closed, and Linux leaves it closed, while others (HP-UX only?) leaves it
> open.
> 
> Now, POSIX.1-2024 says:
> 
> 	If close() is interrupted by a signal that is to be caught, then
> 	it is unspecified whether it returns -1 with errno set to
> 	[EINTR] and fildes remaining open, or returns -1 with errno set
> 	to [EINPROGRESS] and fildes being closed, or returns 0 to
> 	indicate successful completion; [...]
> 
> <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
> 
> Which seems to bless HP-UX and screw all the others, requiring them to
> report EINPROGRESS.
> 
> Was there any discussion about what to do in the Linux kernel?

I'm not aware of any discussions but indeed we are returning EINTR while
closing the fd. Frankly, changing the error code we return in that case is
really asking for userspace regressions so I'm of the opinion we just
ignore the standard as in my opinion it goes against a long established
reality.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 844 bytes --]

Hi Jan!

On Fri, May 16, 2025 at 12:48:56PM +0200, Jan Kara wrote:
> > <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
> > 
> > Which seems to bless HP-UX and screw all the others, requiring them to
> > report EINPROGRESS.
> > 
> > Was there any discussion about what to do in the Linux kernel?
> 
> I'm not aware of any discussions but indeed we are returning EINTR while
> closing the fd. Frankly, changing the error code we return in that case is
> really asking for userspace regressions so I'm of the opinion we just
> ignore the standard as in my opinion it goes against a long established
> reality.

Yep, sounds like what I was expecting.  I'll document that we'll ignore
the new POSIX for close(2) on purpose.  Thanks!


Have a lovely day!
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Mateusz Guzik]

On Fri, May 16, 2025 at 12:48:56PM +0200, Jan Kara wrote:
> Hi!
> 
> On Thu 15-05-25 23:33:22, Alejandro Colomar wrote:
> > I'm updating the manual pages for POSIX.1-2024, and have some doubts
> > about close(2).  The manual page for close(2) says (conforming to
> > POSIX.1-2008):
> > 
> >        The EINTR error is a somewhat special case.  Regarding the EINTR
> >        error, POSIX.1‐2008 says:
> > 
> >               If close() is interrupted by  a  signal  that  is  to  be
> >               caught,  it  shall  return -1 with errno set to EINTR and
> >               the state of fildes is unspecified.
> > 
> >        This permits the behavior that occurs on Linux  and  many  other
> >        implementations,  where,  as  with  other errors that may be re‐
> >        ported by close(), the  file  descriptor  is  guaranteed  to  be
> >        closed.   However, it also permits another possibility: that the
> >        implementation returns an EINTR error and  keeps  the  file  de‐
> >        scriptor open.  (According to its documentation, HP‐UX’s close()
> >        does this.)  The caller must then once more use close() to close
> >        the  file  descriptor, to avoid file descriptor leaks.  This di‐
> >        vergence in implementation behaviors provides a difficult hurdle
> >        for  portable  applications,  since  on  many   implementations,
> >        close() must not be called again after an EINTR error, and on at
> >        least one, close() must be called again.  There are plans to ad‐
> >        dress  this  conundrum for the next major release of the POSIX.1
> >        standard.
> > 
> > TL;DR: close(2) with EINTR is allowed to either leave the fd open or
> > closed, and Linux leaves it closed, while others (HP-UX only?) leaves it
> > open.
> > 
> > Now, POSIX.1-2024 says:
> > 
> > 	If close() is interrupted by a signal that is to be caught, then
> > 	it is unspecified whether it returns -1 with errno set to
> > 	[EINTR] and fildes remaining open, or returns -1 with errno set
> > 	to [EINPROGRESS] and fildes being closed, or returns 0 to
> > 	indicate successful completion; [...]
> > 
> > <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
> > 
> > Which seems to bless HP-UX and screw all the others, requiring them to
> > report EINPROGRESS.
> > 
> > Was there any discussion about what to do in the Linux kernel?
> 
> I'm not aware of any discussions but indeed we are returning EINTR while
> closing the fd. Frankly, changing the error code we return in that case is
> really asking for userspace regressions so I'm of the opinion we just
> ignore the standard as in my opinion it goes against a long established
> reality.

I wonder what are they thinking there.

Any program which even bothers to check for EINTR assumes the fd is
already closed, so one has to assume augmenting behavior to support this
would result in fd leaks.

But that crappery aside, I do wonder if a close() variant which can fail
and leaves the fd intact would be warranted.

For example one of the error modes is ENOSPC (or at least the manpage
claims as much). As is the error is not actionable as the fd is gone.
If instead a magic flag was passed down to indicate what to do (e.g.,
leave the fd in place), the program could try to do some recovery (for
examples unlinking temp files it knows it stores there).

Similar deal with EINTR, albeit this error for close() would preferably get
eradicated instead.

Just some meh rambling.

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Theodore Ts'o]

On Fri, May 16, 2025 at 12:48:56PM +0200, Jan Kara wrote:
> > Now, POSIX.1-2024 says:
> > 
> > 	If close() is interrupted by a signal that is to be caught, then
> > 	it is unspecified whether it returns -1 with errno set to
> > 	[EINTR] and fildes remaining open, or returns -1 with errno set
> > 	to [EINPROGRESS] and fildes being closed, or returns 0 to
> > 	indicate successful completion; [...]
> > 
> > <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
> > 
> > Which seems to bless HP-UX and screw all the others, requiring them to
> > report EINPROGRESS.
> > 
> > Was there any discussion about what to do in the Linux kernel?
> 
> I'm not aware of any discussions but indeed we are returning EINTR while
> closing the fd. Frankly, changing the error code we return in that case is
> really asking for userspace regressions so I'm of the opinion we just
> ignore the standard as in my opinion it goes against a long established
> reality.

Yeah, it appears that the Austin Group has lost all connection with
reality, and we should treat POSIX 2024 accordingly.  Not breaking
userspace applications is way more important that POSIX 2024
compliance.  Which is sad, because I used to really care about POSIX.1
standard as being very useful.  But that seems to be no longer the
case...

						- Ted

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Al Viro]

On Fri, May 16, 2025 at 12:48:56PM +0200, Jan Kara wrote:

> I'm not aware of any discussions but indeed we are returning EINTR while
> closing the fd. Frankly, changing the error code we return in that case is
> really asking for userspace regressions so I'm of the opinion we just
> ignore the standard as in my opinion it goes against a long established
> reality.

AFAICS what happens is that relevance of Austin Group has dropped so low
that they stopped caring about any BS filters they used to have.  What
we are seeing now is assorted pet idiocies that used to sit in their
system, periodically getting shot down while there had been anyone who
cared to do that.

Sad, of course, but what can we do, other than politely ignoring the... output?

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Christian Brauner]

On Fri, May 16, 2025 at 12:48:56PM +0200, Jan Kara wrote:
> Hi!
> 
> On Thu 15-05-25 23:33:22, Alejandro Colomar wrote:
> > I'm updating the manual pages for POSIX.1-2024, and have some doubts
> > about close(2).  The manual page for close(2) says (conforming to
> > POSIX.1-2008):
> > 
> >        The EINTR error is a somewhat special case.  Regarding the EINTR
> >        error, POSIX.1‐2008 says:
> > 
> >               If close() is interrupted by  a  signal  that  is  to  be
> >               caught,  it  shall  return -1 with errno set to EINTR and
> >               the state of fildes is unspecified.
> > 
> >        This permits the behavior that occurs on Linux  and  many  other
> >        implementations,  where,  as  with  other errors that may be re‐
> >        ported by close(), the  file  descriptor  is  guaranteed  to  be
> >        closed.   However, it also permits another possibility: that the
> >        implementation returns an EINTR error and  keeps  the  file  de‐
> >        scriptor open.  (According to its documentation, HP‐UX’s close()
> >        does this.)  The caller must then once more use close() to close
> >        the  file  descriptor, to avoid file descriptor leaks.  This di‐
> >        vergence in implementation behaviors provides a difficult hurdle
> >        for  portable  applications,  since  on  many   implementations,
> >        close() must not be called again after an EINTR error, and on at
> >        least one, close() must be called again.  There are plans to ad‐
> >        dress  this  conundrum for the next major release of the POSIX.1
> >        standard.
> > 
> > TL;DR: close(2) with EINTR is allowed to either leave the fd open or
> > closed, and Linux leaves it closed, while others (HP-UX only?) leaves it
> > open.
> > 
> > Now, POSIX.1-2024 says:
> > 
> > 	If close() is interrupted by a signal that is to be caught, then
> > 	it is unspecified whether it returns -1 with errno set to
> > 	[EINTR] and fildes remaining open, or returns -1 with errno set
> > 	to [EINPROGRESS] and fildes being closed, or returns 0 to
> > 	indicate successful completion; [...]
> > 
> > <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
> > 
> > Which seems to bless HP-UX and screw all the others, requiring them to
> > report EINPROGRESS.
> > 
> > Was there any discussion about what to do in the Linux kernel?
> 
> I'm not aware of any discussions but indeed we are returning EINTR while
> closing the fd. Frankly, changing the error code we return in that case is
> really asking for userspace regressions so I'm of the opinion we just
> ignore the standard as in my opinion it goes against a long established
> reality.

Ignore. We've long since stopped designing apis with input from that
standard in mind. And I think that was a very wise decision.

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Steffen Nurpmeso]

Theodore Ts'o wrote in
 <20250516124147.GB7158@mit.edu>:
 |On Fri, May 16, 2025 at 12:48:56PM +0200, Jan Kara wrote:
 |>> Now, POSIX.1-2024 says:
 |>> 
 |>>  If close() is interrupted by a signal that is to be caught, then
 |>>  it is unspecified whether it returns -1 with errno set to
 |>>  [EINTR] and fildes remaining open, or returns -1 with errno set
 |>>  to [EINPROGRESS] and fildes being closed, or returns 0 to
 |>>  indicate successful completion; [...]
 |>> 
 |>> <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
 |>> 
 |>> Which seems to bless HP-UX and screw all the others, requiring them to
 |>> report EINPROGRESS.
 |>> 
 |>> Was there any discussion about what to do in the Linux kernel?
 |> 
 |> I'm not aware of any discussions but indeed we are returning EINTR while
 |> closing the fd. Frankly, changing the error code we return in that \
 |> case is
 |> really asking for userspace regressions so I'm of the opinion we just
 |> ignore the standard as in my opinion it goes against a long established
 |> reality.
 |
 |Yeah, it appears that the Austin Group has lost all connection with
 |reality, and we should treat POSIX 2024 accordingly.  Not breaking
 |userspace applications is way more important that POSIX 2024
 |compliance.  Which is sad, because I used to really care about POSIX.1
 |standard as being very useful.  But that seems to be no longer the
 |case...

They could not do otherwise than talking the status quo, i think.
They have explicitly added posix_close() which overcomes the
problem (for those operating systems which actually act like
that).  There is a long RATIONALE on this, it starts on page 747 :)

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Theodore Ts'o]

On Tue, May 20, 2025 at 01:19:19AM +0200, Steffen Nurpmeso wrote:
> 
> They could not do otherwise than talking the status quo, i think.
> They have explicitly added posix_close() which overcomes the
> problem (for those operating systems which actually act like
> that).  There is a long RATIONALE on this, it starts on page 747 :)

They could have just added posix_close() which provided well-defined
semantics without demanding that existing implementations make
non-backwards compatible changes to close(2).  Personally, while they
were adding posix_close(2) they could have also fixed the disaster
which is the semantics around close(2) and how advisory locks get
released that were held by other file descriptors and add a profound
apologies over the insane semantics demanded by POSIX[1].

[1] "POSIX advisory locks are broken by design."
    https://www.sqlite.org/src/artifact/c230a7a24?ln=994-1081

					- Ted

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Steffen Nurpmeso]

Theodore Ts'o wrote in
 <20250520133705.GE38098@mit.edu>:
 |On Tue, May 20, 2025 at 01:19:19AM +0200, Steffen Nurpmeso wrote:
 |> They could not do otherwise than talking the status quo, i think.
 |> They have explicitly added posix_close() which overcomes the
 |> problem (for those operating systems which actually act like
 |> that).  There is a long RATIONALE on this, it starts on page 747 :)
 |
 |They could have just added posix_close() which provided well-defined
 |semantics without demanding that existing implementations make
 |non-backwards compatible changes to close(2).  Personally, while they
 |were adding posix_close(2) they could have also fixed the disaster
 |which is the semantics around close(2) [.]

Well it was a lot of trouble, not only in bug 529[1], with
follow-ups like a thread started by Michael Kerrisk, with an
interesting response by Rich Felker of Musl[2].
In [1] Erik Blake of RedHat/libvirt said for example

  The Linux kernel currently always frees the file descriptor (no
  chance for a retry; the filedes can immediately be reused by
  another open()), for both EINTR and EIO. Maybe it is safer to
  state that the fd is _always_ closed, even if failure is
  reported?

etc, but Geoff Clare then (this also was in 2012, where one
possibly could have hoped that more operating systems survive /
continue with money/manpower backing by serious companies; just
in case that mattered) came via

  HP got it right with HP-UX; AIX and Linux do the wrong thing.

and he has quite some reasoning for descriptors like ttys etc,
where close can linger, which resulted in Erik Blake quoting

  Let me make it very, very clear - no matter how many times these
  guys assert HP-UX insane behaviour correct, no "fixes" to Linux
  one are going to be accepted.  Consider it vetoed.  By me, in
  role of Linux VFS maintainer.  And I'm _very_ certain that
  getting Linus to agree will be a matter of minutes.

  [1] https://www.austingroupbugs.net/view.php?id=529
  [2] https://www.mail-archive.com/austin-group-l@opengroup.org/msg00579.html

 |[.] and how advisory locks get
 |released that were held by other file descriptors and add a profound
 |apologies over the insane semantics demanded by POSIX[1].

The new standard added the Linux-style F_OFD_* fcntl(2) locks!
They are yet Linux-only, but NetBSD at least has an issue by
a major contributor (bug 59241):

  NetBSD seems to lack the following:

  3.237 OFD-Owned File Lock
  ...
  https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap03.html#tag_03_237
  >How-To-Repeat:
  standards inspection
  >Fix:
  Yes, please!  (That or write down a reason why we eschew it.)

 |[1] "POSIX advisory locks are broken by design."
 |    https://www.sqlite.org/src/artifact/c230a7a24?ln=994-1081
 |
 |     - Ted
 --End of <20250520133705.GE38098@mit.edu>

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)