close(2) with EINTR has been changed by POSIX.1-2024

close(2) with EINTR has been changed by POSIX.1-2024

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2238 bytes --]

Hi,

I'm updating the manual pages for POSIX.1-2024, and have some doubts
about close(2).  The manual page for close(2) says (conforming to
POSIX.1-2008):

       The EINTR error is a somewhat special case.  Regarding the EINTR
       error, POSIX.1‐2008 says:

              If close() is interrupted by  a  signal  that  is  to  be
              caught,  it  shall  return -1 with errno set to EINTR and
              the state of fildes is unspecified.

       This permits the behavior that occurs on Linux  and  many  other
       implementations,  where,  as  with  other errors that may be re‐
       ported by close(), the  file  descriptor  is  guaranteed  to  be
       closed.   However, it also permits another possibility: that the
       implementation returns an EINTR error and  keeps  the  file  de‐
       scriptor open.  (According to its documentation, HP‐UX’s close()
       does this.)  The caller must then once more use close() to close
       the  file  descriptor, to avoid file descriptor leaks.  This di‐
       vergence in implementation behaviors provides a difficult hurdle
       for  portable  applications,  since  on  many   implementations,
       close() must not be called again after an EINTR error, and on at
       least one, close() must be called again.  There are plans to ad‐
       dress  this  conundrum for the next major release of the POSIX.1
       standard.

TL;DR: close(2) with EINTR is allowed to either leave the fd open or
closed, and Linux leaves it closed, while others (HP-UX only?) leaves it
open.

Now, POSIX.1-2024 says:

	If close() is interrupted by a signal that is to be caught, then
	it is unspecified whether it returns -1 with errno set to
	[EINTR] and fildes remaining open, or returns -1 with errno set
	to [EINPROGRESS] and fildes being closed, or returns 0 to
	indicate successful completion; [...]

<https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>

Which seems to bless HP-UX and screw all the others, requiring them to
report EINPROGRESS.

Was there any discussion about what to do in the Linux kernel?


Have a lovely night!
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Jan Kara]

Hi!

On Thu 15-05-25 23:33:22, Alejandro Colomar wrote:
> I'm updating the manual pages for POSIX.1-2024, and have some doubts
> about close(2).  The manual page for close(2) says (conforming to
> POSIX.1-2008):
> 
>        The EINTR error is a somewhat special case.  Regarding the EINTR
>        error, POSIX.1‐2008 says:
> 
>               If close() is interrupted by  a  signal  that  is  to  be
>               caught,  it  shall  return -1 with errno set to EINTR and
>               the state of fildes is unspecified.
> 
>        This permits the behavior that occurs on Linux  and  many  other
>        implementations,  where,  as  with  other errors that may be re‐
>        ported by close(), the  file  descriptor  is  guaranteed  to  be
>        closed.   However, it also permits another possibility: that the
>        implementation returns an EINTR error and  keeps  the  file  de‐
>        scriptor open.  (According to its documentation, HP‐UX’s close()
>        does this.)  The caller must then once more use close() to close
>        the  file  descriptor, to avoid file descriptor leaks.  This di‐
>        vergence in implementation behaviors provides a difficult hurdle
>        for  portable  applications,  since  on  many   implementations,
>        close() must not be called again after an EINTR error, and on at
>        least one, close() must be called again.  There are plans to ad‐
>        dress  this  conundrum for the next major release of the POSIX.1
>        standard.
> 
> TL;DR: close(2) with EINTR is allowed to either leave the fd open or
> closed, and Linux leaves it closed, while others (HP-UX only?) leaves it
> open.
> 
> Now, POSIX.1-2024 says:
> 
> 	If close() is interrupted by a signal that is to be caught, then
> 	it is unspecified whether it returns -1 with errno set to
> 	[EINTR] and fildes remaining open, or returns -1 with errno set
> 	to [EINPROGRESS] and fildes being closed, or returns 0 to
> 	indicate successful completion; [...]
> 
> <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
> 
> Which seems to bless HP-UX and screw all the others, requiring them to
> report EINPROGRESS.
> 
> Was there any discussion about what to do in the Linux kernel?

I'm not aware of any discussions but indeed we are returning EINTR while
closing the fd. Frankly, changing the error code we return in that case is
really asking for userspace regressions so I'm of the opinion we just
ignore the standard as in my opinion it goes against a long established
reality.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 844 bytes --]

Hi Jan!

On Fri, May 16, 2025 at 12:48:56PM +0200, Jan Kara wrote:
> > <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
> > 
> > Which seems to bless HP-UX and screw all the others, requiring them to
> > report EINPROGRESS.
> > 
> > Was there any discussion about what to do in the Linux kernel?
> 
> I'm not aware of any discussions but indeed we are returning EINTR while
> closing the fd. Frankly, changing the error code we return in that case is
> really asking for userspace regressions so I'm of the opinion we just
> ignore the standard as in my opinion it goes against a long established
> reality.

Yep, sounds like what I was expecting.  I'll document that we'll ignore
the new POSIX for close(2) on purpose.  Thanks!


Have a lovely day!
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Alejandro Colomar]

POSIX.1-2024 now mandates a behavior different from what Linux (and many
other implementations) does.  It requires that we report EINPROGRESS for
what now is EINTR.

There are no plans to conform to POSIX.1-2024 within the Linux kernel,
so document this divergence.  Keep POSIX.1-2008 as the standard to
which we conform in STANDARDS.

Link: <https://sourceware.org/bugzilla/show_bug.cgi?id=14627>
Link: <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
Cc: Jan Kara <jack@suse.cz>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Christian Brauner <brauner@kernel.org>
Cc: Rich Felker <dalias@libc.org>
Cc: <linux-fsdevel@vger.kernel.org>
Cc: <linux-api@vger.kernel.org>
Cc: <libc-alpha@sourceware.org>
Signed-off-by: Alejandro Colomar <alx@kernel.org>
---

Hi,

I've prepared this draft for discussion.  While doing so, I've noticed
the glibc bug ticket, which sounds possibly reasonable: returning 0
instead of reporting an error on EINTR.  That would be an option that
would make us conforming to POSIX.1-2024.  And given that a user can
(and must) do nothing after seeing EINTR, returning 0 wouldn't change
things.

So, I'll leave this patch open for discussion.


Have a lovely day!
Alex

 man/man2/close.2 | 21 ++++++---------------
 1 file changed, 6 insertions(+), 15 deletions(-)

diff --git a/man/man2/close.2 b/man/man2/close.2
index b25ea4de9..9d5e26eed 100644
--- a/man/man2/close.2
+++ b/man/man2/close.2
@@ -191,10 +191,7 @@ .SS Dealing with error returns from close()
 meaning that the file descriptor was invalid)
 even if they subsequently report an error on return from
 .BR close ().
-POSIX.1 is currently silent on this point,
-but there are plans to mandate this behavior in the next major release
-.\" Issue 8
-of the standard.
+POSIX.1-2008 was silent on this point.
 .P
 A careful programmer who wants to know about I/O errors may precede
 .BR close ()
@@ -206,7 +203,7 @@ .SS Dealing with error returns from close()
 error is a somewhat special case.
 Regarding the
 .B EINTR
-error, POSIX.1-2008 says:
+error, POSIX.1-2008 said:
 .P
 .RS
 If
@@ -243,16 +240,10 @@ .SS Dealing with error returns from close()
 error, and on at least one,
 .BR close ()
 must be called again.
-There are plans to address this conundrum for
-the next major release of the POSIX.1 standard.
-.\" FIXME . for later review when Issue 8 is one day released...
-.\" POSIX proposes further changes for EINTR
-.\" http://austingroupbugs.net/tag_view_page.php?tag_id=8
-.\" http://austingroupbugs.net/view.php?id=529
-.\"
-.\" FIXME .
-.\" Review the following glibc bug later
-.\" https://sourceware.org/bugzilla/show_bug.cgi?id=14627
+.P
+POSIX.1-2024 standardized the behavior of HP-UX,
+making Linux and many other implementations non-conforming.
+There are no plans to change the behavior on Linux.
 .SH SEE ALSO
 .BR close_range (2),
 .BR fcntl (2),

Range-diff against v0:
-:  --------- > 1:  efaffc5a4 man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

base-commit: 978b017d93e4e32b752b33877e44a8365644630c
-- 
2.49.0

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Rich Felker]

On Fri, May 16, 2025 at 02:52:05PM +0200, Alejandro Colomar wrote:
> POSIX.1-2024 now mandates a behavior different from what Linux (and many
> other implementations) does.  It requires that we report EINPROGRESS for
> what now is EINTR.
> 
> There are no plans to conform to POSIX.1-2024 within the Linux kernel,
> so document this divergence.  Keep POSIX.1-2008 as the standard to
> which we conform in STANDARDS.
> 
> Link: <https://sourceware.org/bugzilla/show_bug.cgi?id=14627>
> Link: <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
> Cc: Jan Kara <jack@suse.cz>
> Cc: Alexander Viro <viro@zeniv.linux.org.uk>
> Cc: Christian Brauner <brauner@kernel.org>
> Cc: Rich Felker <dalias@libc.org>
> Cc: <linux-fsdevel@vger.kernel.org>
> Cc: <linux-api@vger.kernel.org>
> Cc: <libc-alpha@sourceware.org>
> Signed-off-by: Alejandro Colomar <alx@kernel.org>
> ---
> 
> Hi,
> 
> I've prepared this draft for discussion.  While doing so, I've noticed
> the glibc bug ticket, which sounds possibly reasonable: returning 0
> instead of reporting an error on EINTR.  That would be an option that
> would make us conforming to POSIX.1-2024.  And given that a user can
> (and must) do nothing after seeing EINTR, returning 0 wouldn't change
> things.
> 
> So, I'll leave this patch open for discussion.

FWIW musl adopted the EINPROGRESS as soon as we were made aware of the
issue, and later changed it to returning 0 since applications
(particularly, any written prior to this interpretation) are prone to
interpret EINPROGRESS as an error condition rather than success and
possibly misinterpret it as meaning the fd is still open and valid to
pass to close again.

In general, raw kernel interfaces do not conform to any version of
POSIX; they're just a low-impedance-mismatch set of inferfaces that
facilitate implementing POSIX at the userspace libc layer. So I don't
think this should be documented as "Linux doesn't conform" but
(hopefully, once glibc fixes this) "old versions of glibc did not
conform".

Rich

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Theodore Ts'o]

On Fri, May 16, 2025 at 09:05:47AM -0400, Rich Felker wrote:
 
> In general, raw kernel interfaces do not conform to any version of
> POSIX; they're just a low-impedance-mismatch set of inferfaces that
> facilitate implementing POSIX at the userspace libc layer. So I don't
> think this should be documented as "Linux doesn't conform" but
> (hopefully, once glibc fixes this) "old versions of glibc did not
> conform".

If glibc maintainers want to deal with breaking userspace, then as a
kernel developer, I'm happy to let them deal with the
angry/disappointed users and application programmers.  :-)

						- Ted

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 1912 bytes --]

Hi Ted, Rich,

On Fri, May 16, 2025 at 09:05:47AM -0400, Rich Felker wrote:
> FWIW musl adopted the EINPROGRESS as soon as we were made aware of the
> issue, and later changed it to returning 0 since applications
> (particularly, any written prior to this interpretation) are prone to
> interpret EINPROGRESS as an error condition rather than success and
> possibly misinterpret it as meaning the fd is still open and valid to
> pass to close again.

Hmmm, this page will need a kernel/libc differences section where I
should explain this.

On Fri, May 16, 2025 at 10:20:24AM -0400, Theodore Ts'o wrote:
> On Fri, May 16, 2025 at 09:05:47AM -0400, Rich Felker wrote:
>  
> > In general, raw kernel interfaces do not conform to any version of
> > POSIX; they're just a low-impedance-mismatch set of inferfaces that
> > facilitate implementing POSIX at the userspace libc layer. So I don't
> > think this should be documented as "Linux doesn't conform" but
> > (hopefully, once glibc fixes this) "old versions of glibc did not
> > conform".
> 
> If glibc maintainers want to deal with breaking userspace, then as a
> kernel developer, I'm happy to let them deal with the
> angry/disappointed users and application programmers.  :-)

Which breakage do you expect from the behavior that musl has chosen?

I agree that the POSIX invention of EINPROGRESS is something that would
break users.  However, in removing the error completely and making it a
success, I don't see the same problem.  That is, if a program calls
close(2) and sees a return of 0, or sees a return of -1 with EINTR on
Linux, both mean "the file descriptor has been closed, and the contents
of the file will *eventually* reach the file".

In which cases do you expect any existing Linux program to behave
differently on 0 and on EINTR?


Have a lovely day!
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 2731 bytes --]

Hi,

On Sat, May 17, 2025 at 07:46:48AM +0200, Alejandro Colomar wrote:
> Hi Ted, Rich,
> 
> On Fri, May 16, 2025 at 09:05:47AM -0400, Rich Felker wrote:
> > FWIW musl adopted the EINPROGRESS as soon as we were made aware of the
> > issue, and later changed it to returning 0 since applications
> > (particularly, any written prior to this interpretation) are prone to
> > interpret EINPROGRESS as an error condition rather than success and
> > possibly misinterpret it as meaning the fd is still open and valid to
> > pass to close again.

BTW, I don't think that's a correct interpretation.  The manual page
clearly says after close(2), even on error, the fd is closed and not
usable.  The issue I see is a program thinking it failed and trying to
copy the file again or reporting an error.

On the other hand, as Vincent said, maybe this is not so bad.  For
certain files, fsync(2) is only described for storage devices, so in
some cases there's no clear way to make sure close(2) won't fail after
EINTR (maybe calling sync(2)?).  So, maybe considering it an error
wouldn't be a terrible idea.

I don't know.


Cheers,
Alex

> 
> Hmmm, this page will need a kernel/libc differences section where I
> should explain this.
> 
> On Fri, May 16, 2025 at 10:20:24AM -0400, Theodore Ts'o wrote:
> > On Fri, May 16, 2025 at 09:05:47AM -0400, Rich Felker wrote:
> >  
> > > In general, raw kernel interfaces do not conform to any version of
> > > POSIX; they're just a low-impedance-mismatch set of inferfaces that
> > > facilitate implementing POSIX at the userspace libc layer. So I don't
> > > think this should be documented as "Linux doesn't conform" but
> > > (hopefully, once glibc fixes this) "old versions of glibc did not
> > > conform".
> > 
> > If glibc maintainers want to deal with breaking userspace, then as a
> > kernel developer, I'm happy to let them deal with the
> > angry/disappointed users and application programmers.  :-)
> 
> Which breakage do you expect from the behavior that musl has chosen?
> 
> I agree that the POSIX invention of EINPROGRESS is something that would
> break users.  However, in removing the error completely and making it a
> success, I don't see the same problem.  That is, if a program calls
> close(2) and sees a return of 0, or sees a return of -1 with EINTR on
> Linux, both mean "the file descriptor has been closed, and the contents
> of the file will *eventually* reach the file".
> 
> In which cases do you expect any existing Linux program to behave
> differently on 0 and on EINTR?
> 
> 
> Have a lovely day!
> Alex
> 
> -- 
> <https://www.alejandro-colomar.es/>



-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Rich Felker]

On Sat, May 17, 2025 at 03:03:52PM +0200, Alejandro Colomar wrote:
> Hi,
> 
> On Sat, May 17, 2025 at 07:46:48AM +0200, Alejandro Colomar wrote:
> > Hi Ted, Rich,
> > 
> > On Fri, May 16, 2025 at 09:05:47AM -0400, Rich Felker wrote:
> > > FWIW musl adopted the EINPROGRESS as soon as we were made aware of the
> > > issue, and later changed it to returning 0 since applications
> > > (particularly, any written prior to this interpretation) are prone to
> > > interpret EINPROGRESS as an error condition rather than success and
> > > possibly misinterpret it as meaning the fd is still open and valid to
> > > pass to close again.
> 
> BTW, I don't think that's a correct interpretation.  The manual page
> clearly says after close(2), even on error, the fd is closed and not
> usable.  The issue I see is a program thinking it failed and trying to
> copy the file again or reporting an error.

The authoritative source here is POSIX not the man page, assuming
you're writing a portable application and not a "Linux application".

Until the lastest issue (POSIX 2024/Issue 8), the state of the fd
after EINTR was explicitly unspecified, and after other errors was
unspecified by omission. So there is no way for a program written to
prior versions of the standard to have known how to safely handle
getting EINPROGRESS -- or any error from close for that matter.
Really, the only safe error for close to return, *ever*, is EBADF. On
valid input, it *must succeed*. This is a general principle for
"deallocation/destruction functions". Not an explicit requirement of
this or any standard; just a logical requirement for forward progress
to be possible.

> On the other hand, as Vincent said, maybe this is not so bad.  For
> certain files, fsync(2) is only described for storage devices, so in
> some cases there's no clear way to make sure close(2) won't fail after
> EINTR (maybe calling sync(2)?).  So, maybe considering it an error
> wouldn't be a terrible idea.

Whether data is committed to physical storage in a way that's robust
against machine faults is a completely separate issue from whether
it's committed to the abstract storage. The latter happens at the
moment of write, not close.

If an application is trying to ensure that kind of robustness, the
return value of close is not the tool. It needs the Synchronized IO
interfaces (fsync, etc.) or something specific to whatever it's
writing to.

Rich

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Vincent Lefevre]

On 2025-05-16 09:05:47 -0400, Rich Felker wrote:
> FWIW musl adopted the EINPROGRESS as soon as we were made aware of the
> issue, and later changed it to returning 0 since applications
> (particularly, any written prior to this interpretation) are prone to
> interpret EINPROGRESS as an error condition rather than success and
> possibly misinterpret it as meaning the fd is still open and valid to
> pass to close again.

If I understand correctly, this is a poor choice. POSIX.1-2024 says:

ERRORS
  The close() and posix_close() functions shall fail if:
[...]
  [EINPROGRESS]
    The function was interrupted by a signal and fildes was closed
    but the close operation is continuing asynchronously.

But this does not mean that the asynchronous close operation will
succeed.

So the application could incorrectly deduce that the close operation
was done without any error.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Pascaline project (LIP, ENS-Lyon)

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Florian Weimer]

* Vincent Lefevre:

> On 2025-05-16 09:05:47 -0400, Rich Felker wrote:
>> FWIW musl adopted the EINPROGRESS as soon as we were made aware of the
>> issue, and later changed it to returning 0 since applications
>> (particularly, any written prior to this interpretation) are prone to
>> interpret EINPROGRESS as an error condition rather than success and
>> possibly misinterpret it as meaning the fd is still open and valid to
>> pass to close again.
>
> If I understand correctly, this is a poor choice. POSIX.1-2024 says:
>
> ERRORS
>   The close() and posix_close() functions shall fail if:
> [...]
>   [EINPROGRESS]
>     The function was interrupted by a signal and fildes was closed
>     but the close operation is continuing asynchronously.
>
> But this does not mean that the asynchronous close operation will
> succeed.
>
> So the application could incorrectly deduce that the close operation
> was done without any error.

But on Linux, close traditionally has poor error reporting anyway.  You
have to fsync (or equivalent) before calling close if you want error
checking.  On other systems, the fsync is more or less implied by the
close, leading to rather poor performance.

Thanks,
Florian

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Vincent Lefevre]

On 2025-05-16 16:52:48 +0200, Florian Weimer wrote:
> * Vincent Lefevre:
> 
> > On 2025-05-16 09:05:47 -0400, Rich Felker wrote:
> >> FWIW musl adopted the EINPROGRESS as soon as we were made aware of the
> >> issue, and later changed it to returning 0 since applications
> >> (particularly, any written prior to this interpretation) are prone to
> >> interpret EINPROGRESS as an error condition rather than success and
> >> possibly misinterpret it as meaning the fd is still open and valid to
> >> pass to close again.
> >
> > If I understand correctly, this is a poor choice. POSIX.1-2024 says:
> >
> > ERRORS
> >   The close() and posix_close() functions shall fail if:
> > [...]
> >   [EINPROGRESS]
> >     The function was interrupted by a signal and fildes was closed
> >     but the close operation is continuing asynchronously.
> >
> > But this does not mean that the asynchronous close operation will
> > succeed.
> >
> > So the application could incorrectly deduce that the close operation
> > was done without any error.
> 
> But on Linux, close traditionally has poor error reporting anyway.  You
> have to fsync (or equivalent) before calling close if you want error
> checking.  On other systems, the fsync is more or less implied by the
> close, leading to rather poor performance.

According to its documentation, fsync is only for storage devices,
while not all file descriptors are associated with storage devices.
So I'm wondering the consequences in the other cases.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / Pascaline project (LIP, ENS-Lyon)

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Rich Felker]

On Fri, May 16, 2025 at 04:39:57PM +0200, Vincent Lefevre wrote:
> On 2025-05-16 09:05:47 -0400, Rich Felker wrote:
> > FWIW musl adopted the EINPROGRESS as soon as we were made aware of the
> > issue, and later changed it to returning 0 since applications
> > (particularly, any written prior to this interpretation) are prone to
> > interpret EINPROGRESS as an error condition rather than success and
> > possibly misinterpret it as meaning the fd is still open and valid to
> > pass to close again.
> 
> If I understand correctly, this is a poor choice. POSIX.1-2024 says:
> 
> ERRORS
>   The close() and posix_close() functions shall fail if:
> [...]
>   [EINPROGRESS]
>     The function was interrupted by a signal and fildes was closed
>     but the close operation is continuing asynchronously.
> 
> But this does not mean that the asynchronous close operation will
> succeed.

It always succeeds in the way that's important: the file descriptor is
freed and the process no longer has this reference to the open file
description.

What might or might not succeed is:

(1) other ancient legacy behaviors coupled to close(), like rewinding
a tape drive. If the application cares how that behaves, it needs to
be performing an explicit rewind *before* calling close, when it still
has a handle on the open file so that it can respond to exceptional
conditions, not relying on a legacy behavior like "close also rewinds"
that's device-specific and outside the scope of any modern
cross-platform standard.

(2) deferred operations in unsafe async NFS setups. This is a huge
mess with no real reliable solution except "don't configure your NFS
to have unsafe and nonconforming behaviors in the pursuit of
performance".

Rich

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Rich Felker]

On Fri, May 16, 2025 at 04:39:57PM +0200, Vincent Lefevre wrote:
> On 2025-05-16 09:05:47 -0400, Rich Felker wrote:
> > FWIW musl adopted the EINPROGRESS as soon as we were made aware of the
> > issue, and later changed it to returning 0 since applications
> > (particularly, any written prior to this interpretation) are prone to
> > interpret EINPROGRESS as an error condition rather than success and
> > possibly misinterpret it as meaning the fd is still open and valid to
> > pass to close again.
> 
> If I understand correctly, this is a poor choice. POSIX.1-2024 says:
> 
> ERRORS
>   The close() and posix_close() functions shall fail if:
> [...]
>   [EINPROGRESS]
>     The function was interrupted by a signal and fildes was closed
>     but the close operation is continuing asynchronously.
> 
> But this does not mean that the asynchronous close operation will
> succeed.

There are no asynchronous behaviors specified for there to be a
conformance distinction here. The only observable behaviors happen
instantly, mainly the release of the file descriptor and the process's
handle on the underlying resource. Abstractly, there is no async
operation that could succeed or fail.

> So the application could incorrectly deduce that the close operation
> was done without any error.

This deduction is correct, not incorrect. Rather, failing with
EINPROGRESS would make the application incorrectly deduce that there
might be some error it missed (even if it's aware of the new error
code), and absolutely does make all existing applications written
prior to the new text in POSIX 2024 unable to determine if the fd was
even released and needs to be passed to close again or not.

Rich

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Alejandro Colomar]

[-- Attachment #1: Type: text/plain, Size: 3465 bytes --]

On Sat, May 17, 2025 at 09:32:52AM -0400, Rich Felker wrote:
> On Fri, May 16, 2025 at 04:39:57PM +0200, Vincent Lefevre wrote:
> > On 2025-05-16 09:05:47 -0400, Rich Felker wrote:
> > > FWIW musl adopted the EINPROGRESS as soon as we were made aware of the
> > > issue, and later changed it to returning 0 since applications
> > > (particularly, any written prior to this interpretation) are prone to
> > > interpret EINPROGRESS as an error condition rather than success and
> > > possibly misinterpret it as meaning the fd is still open and valid to
> > > pass to close again.
> > 
> > If I understand correctly, this is a poor choice. POSIX.1-2024 says:
> > 
> > ERRORS
> >   The close() and posix_close() functions shall fail if:
> > [...]
> >   [EINPROGRESS]
> >     The function was interrupted by a signal and fildes was closed
> >     but the close operation is continuing asynchronously.
> > 
> > But this does not mean that the asynchronous close operation will
> > succeed.
> 
> There are no asynchronous behaviors specified for there to be a
> conformance distinction here. The only observable behaviors happen
> instantly, mainly the release of the file descriptor and the process's
> handle on the underlying resource. Abstractly, there is no async
> operation that could succeed or fail.
> 
> > So the application could incorrectly deduce that the close operation
> > was done without any error.
> 
> This deduction is correct, not incorrect. Rather, failing with
> EINPROGRESS would make the application incorrectly deduce that there
> might be some error it missed (even if it's aware of the new error
> code), and absolutely does make all existing applications written
> prior to the new text in POSIX 2024 unable to determine if the fd was
> even released and needs to be passed to close again or not.

Hi Rich,

I think this is not correct; at least on Linux.  The manual page is very
clear that close(2) should not be retried on error:

   Dealing with error returns from close()
       A  careful  programmer  will  check the return value of close(),
       since it is quite possible that errors on  a  previous  write(2)
       operation  are  reported only on the final close() that releases
       the open file description.  Failing to check  the  return  value
       when  closing  a file may lead to silent loss of data.  This can
       especially be observed with NFS and with disk quota.

       Note, however, that a failure return should be used only for di‐
       agnostic purposes (i.e., a warning to the application that there
       may still be I/O pending or there may have been failed  I/O)  or
       remedial  purposes (e.g., writing the file once more or creating
       a backup).

       Retrying the close() after a failure return is the  wrong  thing
       to  do,  since  this may cause a reused file descriptor from an‐
       other thread to be closed.  This can  occur  because  the  Linux
       kernel  always  releases  the file descriptor early in the close
       operation, freeing it for reuse; the steps that  may  return  an
       error,  such as flushing data to the filesystem or device, occur
       only later in the close operation.

	...

       A careful programmer who wants to know about I/O errors may pre‐
       cede close() with a call to fsync(2).


Cheers,
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Zack Weinberg]

Taking everything said in this thread into account, I have attempted to
wordsmith new language for the close(2) manpage.  Please let me know
what you think, and please help me with the bits marked in square
brackets. I can make this into a proper patch for the manpages
when everyone is happy with it.

zw

---

DESCRIPTION
    ... existing text ...

    close() always succeeds.  That is, after it returns, _fd_ has
    always been disconnected from the open file it formerly referred
    to, and its number can be recycled to refer to some other file.
    Furthermore, if _fd_ was the last reference to the underlying
    open file description, the resources associated with the open file
    description will always have been scheduled to be released.

    However, close may report _delayed errors_ from a previous I/O
    operation.  Therefore, its return value should not be ignored.

RETURN VALUE
    close() returns zero if there are no delayed errors to report,
    or -1 if there _might_ be delayed errors.

    When close() returns -1, check _errno_ to see what the situation
    actually is.  Most, but not all, _errno_ codes indicate a delayed
    I/O error that should be reported to the user.  See ERRORS and
    NOTES for more detail.

    [QUERY: Is it ever possible to get delayed errors on close() from
    a file that was opened with O_RDONLY?  What about a file that was
    opened with O_RDWR but never actually written to?  If people only
    have to worry about delayed errors if the file was actually
    written to, we should say so at this point.

    It would also be good to mention whether it is possible to get a
    delayed error on close() even if a previous call to fsync() or
    fdatasync() succeeded and there haven’t been any more writes to
    that file *description* (not necessarily via the fd being closed)
    since.]

ERRORS
    EBADF  _fd_ wasn’t open in the first place, or is outside the
           valid numeric range for file descriptors.

    EINPROGRESS
    EINTR
           There are no delayed errors to report, but the kernel is
           still doing some clean-up work in the background.  This
           situation should be treated the same as if close() had
           returned zero.  Do not retry the close(), and do not report
           an error to the user.

    EDQUOT
    EFBIG
    EIO
    ENOSPC
           These are the most common errno codes associated with
           delayed I/O errors.  They should be treated as a hard
           failure to write to the file that was formerly associated
           with _fd_, the same as if an earlier write(2) had failed
           with one of these codes.  The file has still been closed!
           Do not retry the close().  But do report an error to the user.

    Depending on the underlying file, close() may return other errno
    codes; these should generally also be treated as delayed I/O errors.

NOTES
  Dealing with error returns from close()

    As discussed above, close() always closes the file.  Except when
    errno is set to EBADF, EINPROGRESS, or EINTR, an error return from
    close() reports a _delayed I/O error_ from a previous write()
    operation.

    It is vital to report delayed I/O errors to the user; failing to
    check the return value of close() can cause _silent_ loss of data.
    The most common situations where this actually happens involve
    networked filesystems, where, in the name of throughput, write()
    often returns success before the server has actually confirmed a
    successful write.

    However, it is also vital to understand that _no matter what_
    close() returns, and _no matter what_ it sets errno to, when it
    returns, _the file descriptor passed to close() has been closed_,
    and its number is _immediately_ available for reuse by open(2),
    dup(2), etc.  Therefore, one should never retry a close(), not
    even if it set errno to a value that normally indicates the
    operation needs to be retried (e.g. EINTR).  Retrying a close()
    is a serious bug, particularly in a multithreaded program; if
    the file descriptor number has already been reused, _that file_
    will get closed out from under whatever other thread opened it.

    [Possibly something about fsync/fdatasync here?]

BUGS
    Prior to POSIX.1-2024, there was no official guarantee that
    close() would always close the file descriptor, even on error.
    Linux has always closed the file descriptor, even on error,
    but other implementations might not have.

    The only such implementation we have heard of is HP-UX; at least
    some versions of HP-UX’s man page for close() said it should be
    retried if it returned -1 with errno set to EINTR.  (If you know
    exactly which versions of HP-UX are affected, or of any other
    Unix where close() doesn’t always close the file descriptor,
    please contact us about it.)

    Portable code should nonetheless never retry a failed close(); the
    consequences of a file descriptor leak are far less dangerous than
    the consequences of closing a file out from under another thread.

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Rich Felker]

On Fri, May 23, 2025 at 02:10:57PM -0400, Zack Weinberg wrote:
> Taking everything said in this thread into account, I have attempted to
> wordsmith new language for the close(2) manpage.  Please let me know
> what you think, and please help me with the bits marked in square
> brackets. I can make this into a proper patch for the manpages
> when everyone is happy with it.
> 
> zw
> 
> ---
> 
> DESCRIPTION
>     ... existing text ...
> 
>     close() always succeeds.  That is, after it returns, _fd_ has
>     always been disconnected from the open file it formerly referred
>     to, and its number can be recycled to refer to some other file.
>     Furthermore, if _fd_ was the last reference to the underlying
>     open file description, the resources associated with the open file
>     description will always have been scheduled to be released.
> 
>     However, close may report _delayed errors_ from a previous I/O
>     operation.  Therefore, its return value should not be ignored.
> 
> RETURN VALUE
>     close() returns zero if there are no delayed errors to report,
>     or -1 if there _might_ be delayed errors.
> 
>     When close() returns -1, check _errno_ to see what the situation
>     actually is.  Most, but not all, _errno_ codes indicate a delayed
>     I/O error that should be reported to the user.  See ERRORS and
>     NOTES for more detail.
> 
>     [QUERY: Is it ever possible to get delayed errors on close() from
>     a file that was opened with O_RDONLY?  What about a file that was
>     opened with O_RDWR but never actually written to?  If people only
>     have to worry about delayed errors if the file was actually
>     written to, we should say so at this point.
> 
>     It would also be good to mention whether it is possible to get a
>     delayed error on close() even if a previous call to fsync() or
>     fdatasync() succeeded and there haven’t been any more writes to
>     that file *description* (not necessarily via the fd being closed)
>     since.]
> 
> ERRORS
>     EBADF  _fd_ wasn’t open in the first place, or is outside the
>            valid numeric range for file descriptors.
> 
>     EINPROGRESS
>     EINTR
>            There are no delayed errors to report, but the kernel is
>            still doing some clean-up work in the background.  This
>            situation should be treated the same as if close() had
>            returned zero.  Do not retry the close(), and do not report
>            an error to the user.

Since this behavior for EINTR is non-conforming (and even prior to the
POSIX 2024 update, it was contrary to the general semantics for EINTR,
that no non-ignoreable side-effects have taken place), it should be
noted that it's Linux/glibc-specific.

>     EDQUOT
>     EFBIG
>     EIO
>     ENOSPC
>            These are the most common errno codes associated with
>            delayed I/O errors.  They should be treated as a hard
>            failure to write to the file that was formerly associated
>            with _fd_, the same as if an earlier write(2) had failed
>            with one of these codes.  The file has still been closed!
>            Do not retry the close().  But do report an error to the user.
> 
>     Depending on the underlying file, close() may return other errno
>     codes; these should generally also be treated as delayed I/O errors.
> 
> NOTES
>   Dealing with error returns from close()
> 
>     As discussed above, close() always closes the file.  Except when
>     errno is set to EBADF, EINPROGRESS, or EINTR, an error return from
>     close() reports a _delayed I/O error_ from a previous write()
>     operation.
> 
>     It is vital to report delayed I/O errors to the user; failing to
>     check the return value of close() can cause _silent_ loss of data.
>     The most common situations where this actually happens involve
>     networked filesystems, where, in the name of throughput, write()
>     often returns success before the server has actually confirmed a
>     successful write.
> 
>     However, it is also vital to understand that _no matter what_
>     close() returns, and _no matter what_ it sets errno to, when it
>     returns, _the file descriptor passed to close() has been closed_,
>     and its number is _immediately_ available for reuse by open(2),
>     dup(2), etc.  Therefore, one should never retry a close(), not
>     even if it set errno to a value that normally indicates the
>     operation needs to be retried (e.g. EINTR).  Retrying a close()
>     is a serious bug, particularly in a multithreaded program; if
>     the file descriptor number has already been reused, _that file_
>     will get closed out from under whatever other thread opened it.
> 
>     [Possibly something about fsync/fdatasync here?]

While I agree with all of this, I think the tone is way too
proscriptive. The man pages are to document the behaviors, not tell
people how to program. And again, it should be noted that the standard
behavior is that you *do* have to retry on EINTR, or arrange to ensure
it never happens (e.g. by not installing interrupting signal handlers,
or blocking signals across calls to close), and that treating EINTR as
"fd has been closed" is something you should only do on
known-nonconforming systems.

Aside: the reason EINTR *has to* be specified this way is that pthread
cancellation is aligned with EINTR. If EINTR were defined to have
closed the fd, then acting on cancellation during close would also
have closed the fd, but the cancellation handler would have no way to
distinguish this, leading to a situation where you're forced to either
leak fds or introduce a double-close vuln.

> BUGS
>     Prior to POSIX.1-2024, there was no official guarantee that
>     close() would always close the file descriptor, even on error.
>     Linux has always closed the file descriptor, even on error,
>     but other implementations might not have.
> 
>     The only such implementation we have heard of is HP-UX; at least
>     some versions of HP-UX’s man page for close() said it should be
>     retried if it returned -1 with errno set to EINTR.  (If you know
>     exactly which versions of HP-UX are affected, or of any other
>     Unix where close() doesn’t always close the file descriptor,
>     please contact us about it.)
> 
>     Portable code should nonetheless never retry a failed close(); the
>     consequences of a file descriptor leak are far less dangerous than
>     the consequences of closing a file out from under another thread.

This is explicitly the opposite of what's specified for portable code.

It sounds like you are intentionally omitting that POSIX says the
opposite of what you want it to, and treating the standard behavior as
a historical HP-UX quirk/bug. This is polemic, not the sort of
documentation that belongs in a man page.

An outline of what I'd like to see instead:

- Clear explanation of why double-close is a serious bug that must
  always be avoided. (I think we all agree on this.)

- Statement that the historical Linux/glibc behavior and current POSIX
  requirement differ, without language that tries to paint the POSIX
  behavior as a HP-UX bug/quirk. Possibly citing real sources/history
  of the issue (Austin Group tracker items 529, 614; maybe others).

- Consequence of just assuming the Linux behavior (fd leaks on
  conforming systems).

- Consequences of assuming the POSIX behavior (double-close vulns on
  GNU/Linux, maybe others).

- Survey of methods for avoiding the problem (ways to preclude EINTR,
  possibly ways to infer behavior, etc).


Rich

Re: [RFC v1] man/man2/close.2: CAVEATS: Document divergence from POSIX.1-2024

[Florian Weimer]

* Zack Weinberg:

> BUGS
>     Prior to POSIX.1-2024, there was no official guarantee that
>     close() would always close the file descriptor, even on error.
>     Linux has always closed the file descriptor, even on error,
>     but other implementations might not have.
>
>     The only such implementation we have heard of is HP-UX; at least
>     some versions of HP-UX’s man page for close() said it should be
>     retried if it returned -1 with errno set to EINTR.  (If you know
>     exactly which versions of HP-UX are affected, or of any other
>     Unix where close() doesn’t always close the file descriptor,
>     please contact us about it.)

The AIX documentation also says this:

| The success of the close subroutine is undetermined if the following
| is true:
|	
| EINTR The state of the FileDescriptor is undetermined. Retry the
|       close routine to ensure that the FileDescriptor is closed.

<https://www.ibm.com/docs/en/aix/7.2.0?topic=c-close-subroutine>

So it's not just HP-UX.

For z/OS, it looks like some other errors leave the descriptor open:

| EAGAIN
|
|   The call did not complete because the specified socket descriptor
|   is currently being used by another thread in the same process.
|
|    For example, in a multithreaded environment, close() fails and
|    returns EAGAIN when the following sequence of events occurs (1)
|    thread is blocked in a read() or select() call on a given file or
|    socket descriptor and (2) another thread issues a simultaneous
|    close() call for the same descriptor.
| […]
| EBADF
|    fildes is not a valid open file descriptor, or the socket
|    parameter is not a valid socket descriptor.

<https://www.ibm.com/docs/en/zos/2.1.0?topic=functions-close-close-file>

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Mateusz Guzik]

On Fri, May 16, 2025 at 12:48:56PM +0200, Jan Kara wrote:
> Hi!
> 
> On Thu 15-05-25 23:33:22, Alejandro Colomar wrote:
> > I'm updating the manual pages for POSIX.1-2024, and have some doubts
> > about close(2).  The manual page for close(2) says (conforming to
> > POSIX.1-2008):
> > 
> >        The EINTR error is a somewhat special case.  Regarding the EINTR
> >        error, POSIX.1‐2008 says:
> > 
> >               If close() is interrupted by  a  signal  that  is  to  be
> >               caught,  it  shall  return -1 with errno set to EINTR and
> >               the state of fildes is unspecified.
> > 
> >        This permits the behavior that occurs on Linux  and  many  other
> >        implementations,  where,  as  with  other errors that may be re‐
> >        ported by close(), the  file  descriptor  is  guaranteed  to  be
> >        closed.   However, it also permits another possibility: that the
> >        implementation returns an EINTR error and  keeps  the  file  de‐
> >        scriptor open.  (According to its documentation, HP‐UX’s close()
> >        does this.)  The caller must then once more use close() to close
> >        the  file  descriptor, to avoid file descriptor leaks.  This di‐
> >        vergence in implementation behaviors provides a difficult hurdle
> >        for  portable  applications,  since  on  many   implementations,
> >        close() must not be called again after an EINTR error, and on at
> >        least one, close() must be called again.  There are plans to ad‐
> >        dress  this  conundrum for the next major release of the POSIX.1
> >        standard.
> > 
> > TL;DR: close(2) with EINTR is allowed to either leave the fd open or
> > closed, and Linux leaves it closed, while others (HP-UX only?) leaves it
> > open.
> > 
> > Now, POSIX.1-2024 says:
> > 
> > 	If close() is interrupted by a signal that is to be caught, then
> > 	it is unspecified whether it returns -1 with errno set to
> > 	[EINTR] and fildes remaining open, or returns -1 with errno set
> > 	to [EINPROGRESS] and fildes being closed, or returns 0 to
> > 	indicate successful completion; [...]
> > 
> > <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
> > 
> > Which seems to bless HP-UX and screw all the others, requiring them to
> > report EINPROGRESS.
> > 
> > Was there any discussion about what to do in the Linux kernel?
> 
> I'm not aware of any discussions but indeed we are returning EINTR while
> closing the fd. Frankly, changing the error code we return in that case is
> really asking for userspace regressions so I'm of the opinion we just
> ignore the standard as in my opinion it goes against a long established
> reality.

I wonder what are they thinking there.

Any program which even bothers to check for EINTR assumes the fd is
already closed, so one has to assume augmenting behavior to support this
would result in fd leaks.

But that crappery aside, I do wonder if a close() variant which can fail
and leaves the fd intact would be warranted.

For example one of the error modes is ENOSPC (or at least the manpage
claims as much). As is the error is not actionable as the fd is gone.
If instead a magic flag was passed down to indicate what to do (e.g.,
leave the fd in place), the program could try to do some recovery (for
examples unlinking temp files it knows it stores there).

Similar deal with EINTR, albeit this error for close() would preferably get
eradicated instead.

Just some meh rambling.

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Theodore Ts'o]

On Fri, May 16, 2025 at 12:48:56PM +0200, Jan Kara wrote:
> > Now, POSIX.1-2024 says:
> > 
> > 	If close() is interrupted by a signal that is to be caught, then
> > 	it is unspecified whether it returns -1 with errno set to
> > 	[EINTR] and fildes remaining open, or returns -1 with errno set
> > 	to [EINPROGRESS] and fildes being closed, or returns 0 to
> > 	indicate successful completion; [...]
> > 
> > <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
> > 
> > Which seems to bless HP-UX and screw all the others, requiring them to
> > report EINPROGRESS.
> > 
> > Was there any discussion about what to do in the Linux kernel?
> 
> I'm not aware of any discussions but indeed we are returning EINTR while
> closing the fd. Frankly, changing the error code we return in that case is
> really asking for userspace regressions so I'm of the opinion we just
> ignore the standard as in my opinion it goes against a long established
> reality.

Yeah, it appears that the Austin Group has lost all connection with
reality, and we should treat POSIX 2024 accordingly.  Not breaking
userspace applications is way more important that POSIX 2024
compliance.  Which is sad, because I used to really care about POSIX.1
standard as being very useful.  But that seems to be no longer the
case...

						- Ted

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Steffen Nurpmeso]

Theodore Ts'o wrote in
 <20250516124147.GB7158@mit.edu>:
 |On Fri, May 16, 2025 at 12:48:56PM +0200, Jan Kara wrote:
 |>> Now, POSIX.1-2024 says:
 |>> 
 |>>  If close() is interrupted by a signal that is to be caught, then
 |>>  it is unspecified whether it returns -1 with errno set to
 |>>  [EINTR] and fildes remaining open, or returns -1 with errno set
 |>>  to [EINPROGRESS] and fildes being closed, or returns 0 to
 |>>  indicate successful completion; [...]
 |>> 
 |>> <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
 |>> 
 |>> Which seems to bless HP-UX and screw all the others, requiring them to
 |>> report EINPROGRESS.
 |>> 
 |>> Was there any discussion about what to do in the Linux kernel?
 |> 
 |> I'm not aware of any discussions but indeed we are returning EINTR while
 |> closing the fd. Frankly, changing the error code we return in that \
 |> case is
 |> really asking for userspace regressions so I'm of the opinion we just
 |> ignore the standard as in my opinion it goes against a long established
 |> reality.
 |
 |Yeah, it appears that the Austin Group has lost all connection with
 |reality, and we should treat POSIX 2024 accordingly.  Not breaking
 |userspace applications is way more important that POSIX 2024
 |compliance.  Which is sad, because I used to really care about POSIX.1
 |standard as being very useful.  But that seems to be no longer the
 |case...

They could not do otherwise than talking the status quo, i think.
They have explicitly added posix_close() which overcomes the
problem (for those operating systems which actually act like
that).  There is a long RATIONALE on this, it starts on page 747 :)

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Theodore Ts'o]

On Tue, May 20, 2025 at 01:19:19AM +0200, Steffen Nurpmeso wrote:
> 
> They could not do otherwise than talking the status quo, i think.
> They have explicitly added posix_close() which overcomes the
> problem (for those operating systems which actually act like
> that).  There is a long RATIONALE on this, it starts on page 747 :)

They could have just added posix_close() which provided well-defined
semantics without demanding that existing implementations make
non-backwards compatible changes to close(2).  Personally, while they
were adding posix_close(2) they could have also fixed the disaster
which is the semantics around close(2) and how advisory locks get
released that were held by other file descriptors and add a profound
apologies over the insane semantics demanded by POSIX[1].

[1] "POSIX advisory locks are broken by design."
    https://www.sqlite.org/src/artifact/c230a7a24?ln=994-1081

					- Ted

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Steffen Nurpmeso]

Theodore Ts'o wrote in
 <20250520133705.GE38098@mit.edu>:
 |On Tue, May 20, 2025 at 01:19:19AM +0200, Steffen Nurpmeso wrote:
 |> They could not do otherwise than talking the status quo, i think.
 |> They have explicitly added posix_close() which overcomes the
 |> problem (for those operating systems which actually act like
 |> that).  There is a long RATIONALE on this, it starts on page 747 :)
 |
 |They could have just added posix_close() which provided well-defined
 |semantics without demanding that existing implementations make
 |non-backwards compatible changes to close(2).  Personally, while they
 |were adding posix_close(2) they could have also fixed the disaster
 |which is the semantics around close(2) [.]

Well it was a lot of trouble, not only in bug 529[1], with
follow-ups like a thread started by Michael Kerrisk, with an
interesting response by Rich Felker of Musl[2].
In [1] Erik Blake of RedHat/libvirt said for example

  The Linux kernel currently always frees the file descriptor (no
  chance for a retry; the filedes can immediately be reused by
  another open()), for both EINTR and EIO. Maybe it is safer to
  state that the fd is _always_ closed, even if failure is
  reported?

etc, but Geoff Clare then (this also was in 2012, where one
possibly could have hoped that more operating systems survive /
continue with money/manpower backing by serious companies; just
in case that mattered) came via

  HP got it right with HP-UX; AIX and Linux do the wrong thing.

and he has quite some reasoning for descriptors like ttys etc,
where close can linger, which resulted in Erik Blake quoting

  Let me make it very, very clear - no matter how many times these
  guys assert HP-UX insane behaviour correct, no "fixes" to Linux
  one are going to be accepted.  Consider it vetoed.  By me, in
  role of Linux VFS maintainer.  And I'm _very_ certain that
  getting Linus to agree will be a matter of minutes.

  [1] https://www.austingroupbugs.net/view.php?id=529
  [2] https://www.mail-archive.com/austin-group-l@opengroup.org/msg00579.html

 |[.] and how advisory locks get
 |released that were held by other file descriptors and add a profound
 |apologies over the insane semantics demanded by POSIX[1].

The new standard added the Linux-style F_OFD_* fcntl(2) locks!
They are yet Linux-only, but NetBSD at least has an issue by
a major contributor (bug 59241):

  NetBSD seems to lack the following:

  3.237 OFD-Owned File Lock
  ...
  https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap03.html#tag_03_237
  >How-To-Repeat:
  standards inspection
  >Fix:
  Yes, please!  (That or write down a reason why we eschew it.)

 |[1] "POSIX advisory locks are broken by design."
 |    https://www.sqlite.org/src/artifact/c230a7a24?ln=994-1081
 |
 |     - Ted
 --End of <20250520133705.GE38098@mit.edu>

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Al Viro]

On Fri, May 16, 2025 at 12:48:56PM +0200, Jan Kara wrote:

> I'm not aware of any discussions but indeed we are returning EINTR while
> closing the fd. Frankly, changing the error code we return in that case is
> really asking for userspace regressions so I'm of the opinion we just
> ignore the standard as in my opinion it goes against a long established
> reality.

AFAICS what happens is that relevance of Austin Group has dropped so low
that they stopped caring about any BS filters they used to have.  What
we are seeing now is assorted pet idiocies that used to sit in their
system, periodically getting shot down while there had been anyone who
cared to do that.

Sad, of course, but what can we do, other than politely ignoring the... output?

Re: close(2) with EINTR has been changed by POSIX.1-2024

[Christian Brauner]

On Fri, May 16, 2025 at 12:48:56PM +0200, Jan Kara wrote:
> Hi!
> 
> On Thu 15-05-25 23:33:22, Alejandro Colomar wrote:
> > I'm updating the manual pages for POSIX.1-2024, and have some doubts
> > about close(2).  The manual page for close(2) says (conforming to
> > POSIX.1-2008):
> > 
> >        The EINTR error is a somewhat special case.  Regarding the EINTR
> >        error, POSIX.1‐2008 says:
> > 
> >               If close() is interrupted by  a  signal  that  is  to  be
> >               caught,  it  shall  return -1 with errno set to EINTR and
> >               the state of fildes is unspecified.
> > 
> >        This permits the behavior that occurs on Linux  and  many  other
> >        implementations,  where,  as  with  other errors that may be re‐
> >        ported by close(), the  file  descriptor  is  guaranteed  to  be
> >        closed.   However, it also permits another possibility: that the
> >        implementation returns an EINTR error and  keeps  the  file  de‐
> >        scriptor open.  (According to its documentation, HP‐UX’s close()
> >        does this.)  The caller must then once more use close() to close
> >        the  file  descriptor, to avoid file descriptor leaks.  This di‐
> >        vergence in implementation behaviors provides a difficult hurdle
> >        for  portable  applications,  since  on  many   implementations,
> >        close() must not be called again after an EINTR error, and on at
> >        least one, close() must be called again.  There are plans to ad‐
> >        dress  this  conundrum for the next major release of the POSIX.1
> >        standard.
> > 
> > TL;DR: close(2) with EINTR is allowed to either leave the fd open or
> > closed, and Linux leaves it closed, while others (HP-UX only?) leaves it
> > open.
> > 
> > Now, POSIX.1-2024 says:
> > 
> > 	If close() is interrupted by a signal that is to be caught, then
> > 	it is unspecified whether it returns -1 with errno set to
> > 	[EINTR] and fildes remaining open, or returns -1 with errno set
> > 	to [EINPROGRESS] and fildes being closed, or returns 0 to
> > 	indicate successful completion; [...]
> > 
> > <https://pubs.opengroup.org/onlinepubs/9799919799/functions/close.html>
> > 
> > Which seems to bless HP-UX and screw all the others, requiring them to
> > report EINPROGRESS.
> > 
> > Was there any discussion about what to do in the Linux kernel?
> 
> I'm not aware of any discussions but indeed we are returning EINTR while
> closing the fd. Frankly, changing the error code we return in that case is
> really asking for userspace regressions so I'm of the opinion we just
> ignore the standard as in my opinion it goes against a long established
> reality.

Ignore. We've long since stopped designing apis with input from that
standard in mind. And I think that was a very wise decision.