[PATCH] fsync.2 updates

[PATCH] fsync.2 updates

[Christoph Hellwig]

 - explain the situation with disk caches better
 - remove the duplicate fdatasync explanation in the NOTE section
 - remove an incorrect note about fsync generally requiring two writes
 - remove an obsolete ext2 example note
 - fsync works on any fd and does not require a writeable one,
   correct the EBADF error code explanation.

Signed-off-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

diff --git a/man2/fsync.2 b/man2/fsync.2
index 58d325a..9b74774 100644
--- a/man2/fsync.2
+++ b/man2/fsync.2
@@ -63,12 +63,15 @@ transfers ("flushes") all modified in-core data of
 (i.e., modified buffer cache pages for) the
 file referred to by the file descriptor
 .I fd
-to the disk device (or other permanent storage device)
-where that file resides.
+to the disk device (or other permanent storage device) so that all
+changed information can be retrieved even after the system crashed or
+was rebooted.  This includes writing through or flushing a disk cache
+if present.
 The call blocks until the device reports that the transfer has completed.
 It also flushes metadata information associated with the file (see
 .BR stat (2)).
 
+
 Calling
 .BR fsync ()
 does not necessarily ensure
@@ -111,7 +114,7 @@ is set appropriately.
 .TP
 .B EBADF
 .I fd
-is not a valid file descriptor open for writing.
+is not a valid open file descriptor.
 .TP
 .B EIO
 An error occurred during synchronization.
@@ -135,49 +138,21 @@ to a value greater than 0.
 .\" -1: unavailable, 0: ask using sysconf().
 .\" glibc defines them to 1.
 .SH NOTES
-Applications that access databases or log files often write a tiny
-data fragment (e.g., one line in a log file) and then call
-.BR fsync ()
-immediately in order to ensure that the written data is physically
-stored on the harddisk.
-Unfortunately,
-.BR fsync ()
-will always initiate two write operations: one for the newly written
-data and another one in order to update the modification time stored
-in the inode.
-If the modification time is not a part of the transaction
-concept
-.BR fdatasync ()
-can be used to avoid unnecessary inode disk write operations.
-
-If the underlying hard disk has write caching enabled, then
-the data may not really be on permanent storage when
-.BR fsync ()
-/
-.BR fdatasync ()
-return.
-.\" See
-.\" .BR hdparm (8)
-.\" for how to disable that cache for IDE disks.
-.LP
-When an ext2 file system is mounted with the
-.I sync
-option, directory entries are also implicitly synced by
-.BR fsync ().
-.LP
-On kernels before 2.4,
-.BR fsync ()
-on big files can be inefficient.
-An alternative might be to use the
-.B O_SYNC
-flag to
-.BR open (2).

Re: [PATCH] fsync.2 updates

[Guillem Jover]

Hi!

On 2011-11-04 6:12:04, Christoph Hellwig wrote:
> - fsync works on any fd and does not require a writeable one,
>   correct the EBADF error code explanation.

The problem is, while this is true for Linux, that's not a safe portable
assumption to make on POSIX in general as that behaviour is not
specified and as such is implementation specific, some Unix systems
do actually fail on read-only file descriptors, for example:

  <http://h30097.www3.hp.com/docs/base_doc/DOCUMENTATION/V50_HTML/MAN/MAN2/0033____.HTM>
  <http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=/com.ibm.aix.basetechref/doc/basetrf1/fsync.htm>

So if this part of the patch gets applied it would be nice to add a
note stating this is Linux specific behaviour, and that other systems
might still fail on such condition.

regards,
guillem
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] fsync.2 updates

[Christoph Hellwig]

ping?

On Fri, Nov 04, 2011 at 02:12:03AM -0400, Christoph Hellwig wrote:
>  - explain the situation with disk caches better
>  - remove the duplicate fdatasync explanation in the NOTE section
>  - remove an incorrect note about fsync generally requiring two writes
>  - remove an obsolete ext2 example note
>  - fsync works on any fd and does not require a writeable one,
>    correct the EBADF error code explanation.
> 
> Signed-off-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
> 
> diff --git a/man2/fsync.2 b/man2/fsync.2
> index 58d325a..9b74774 100644
> --- a/man2/fsync.2
> +++ b/man2/fsync.2
> @@ -63,12 +63,15 @@ transfers ("flushes") all modified in-core data of
>  (i.e., modified buffer cache pages for) the
>  file referred to by the file descriptor
>  .I fd
> -to the disk device (or other permanent storage device)
> -where that file resides.
> +to the disk device (or other permanent storage device) so that all
> +changed information can be retrieved even after the system crashed or
> +was rebooted.  This includes writing through or flushing a disk cache
> +if present.
>  The call blocks until the device reports that the transfer has completed.
>  It also flushes metadata information associated with the file (see
>  .BR stat (2)).
>  
> +
>  Calling
>  .BR fsync ()
>  does not necessarily ensure
> @@ -111,7 +114,7 @@ is set appropriately.
>  .TP
>  .B EBADF
>  .I fd
> -is not a valid file descriptor open for writing.
> +is not a valid open file descriptor.
>  .TP
>  .B EIO
>  An error occurred during synchronization.
> @@ -135,49 +138,21 @@ to a value greater than 0.
>  .\" -1: unavailable, 0: ask using sysconf().
>  .\" glibc defines them to 1.
>  .SH NOTES
> -Applications that access databases or log files often write a tiny
> -data fragment (e.g., one line in a log file) and then call
> -.BR fsync ()
> -immediately in order to ensure that the written data is physically
> -stored on the harddisk.
> -Unfortunately,
> -.BR fsync ()
> -will always initiate two write operations: one for the newly written
> -data and another one in order to update the modification time stored
> -in the inode.
> -If the modification time is not a part of the transaction
> -concept
> -.BR fdatasync ()
> -can be used to avoid unnecessary inode disk write operations.
> -
> -If the underlying hard disk has write caching enabled, then
> -the data may not really be on permanent storage when
> -.BR fsync ()
> -/
> -.BR fdatasync ()
> -return.
> -.\" See
> -.\" .BR hdparm (8)
> -.\" for how to disable that cache for IDE disks.
> -.LP
> -When an ext2 file system is mounted with the
> -.I sync
> -option, directory entries are also implicitly synced by
> -.BR fsync ().
> -.LP
> -On kernels before 2.4,
> -.BR fsync ()
> -on big files can be inefficient.
> -An alternative might be to use the
> -.B O_SYNC
> -flag to
> -.BR open (2).
> -
>  In Linux 2.2 and earlier,
>  .BR fdatasync ()
>  is equivalent to
>  .BR fsync (),
>  and so has no performance advantage.
> +
> +The
> +.BR fsync ()
> +implementations in older kernels and lesser used filesystems
> +does not know how to flush disk caches.  In these cases disk caches need to
> +be disabled using
> +.BR hdparm (8)
> +or
> +.BR sdparm (8)
> +to guarantee safe operation.
>  .SH "SEE ALSO"
>  .BR bdflush (2),
>  .BR open (2),
---end quoted text---
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] fsync.2 updates

[Michael Kerrisk]

Hi Christoph,

I've applied this patch for man-pages-3.36.

Cheers,

Michael


On Fri, Nov 4, 2011 at 7:12 PM, Christoph Hellwig <hch-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org> wrote:
>  - explain the situation with disk caches better
>  - remove the duplicate fdatasync explanation in the NOTE section
>  - remove an incorrect note about fsync generally requiring two writes
>  - remove an obsolete ext2 example note
>  - fsync works on any fd and does not require a writeable one,
>   correct the EBADF error code explanation.
>
> Signed-off-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
>
> diff --git a/man2/fsync.2 b/man2/fsync.2
> index 58d325a..9b74774 100644
> --- a/man2/fsync.2
> +++ b/man2/fsync.2
> @@ -63,12 +63,15 @@ transfers ("flushes") all modified in-core data of
>  (i.e., modified buffer cache pages for) the
>  file referred to by the file descriptor
>  .I fd
> -to the disk device (or other permanent storage device)
> -where that file resides.
> +to the disk device (or other permanent storage device) so that all
> +changed information can be retrieved even after the system crashed or
> +was rebooted.  This includes writing through or flushing a disk cache
> +if present.
>  The call blocks until the device reports that the transfer has completed.
>  It also flushes metadata information associated with the file (see
>  .BR stat (2)).
>
> +
>  Calling
>  .BR fsync ()
>  does not necessarily ensure
> @@ -111,7 +114,7 @@ is set appropriately.
>  .TP
>  .B EBADF
>  .I fd
> -is not a valid file descriptor open for writing.
> +is not a valid open file descriptor.
>  .TP
>  .B EIO
>  An error occurred during synchronization.
> @@ -135,49 +138,21 @@ to a value greater than 0.
>  .\" -1: unavailable, 0: ask using sysconf().
>  .\" glibc defines them to 1.
>  .SH NOTES
> -Applications that access databases or log files often write a tiny
> -data fragment (e.g., one line in a log file) and then call
> -.BR fsync ()
> -immediately in order to ensure that the written data is physically
> -stored on the harddisk.
> -Unfortunately,
> -.BR fsync ()
> -will always initiate two write operations: one for the newly written
> -data and another one in order to update the modification time stored
> -in the inode.
> -If the modification time is not a part of the transaction
> -concept
> -.BR fdatasync ()
> -can be used to avoid unnecessary inode disk write operations.
> -
> -If the underlying hard disk has write caching enabled, then
> -the data may not really be on permanent storage when
> -.BR fsync ()
> -/
> -.BR fdatasync ()
> -return.
> -.\" See
> -.\" .BR hdparm (8)
> -.\" for how to disable that cache for IDE disks.
> -.LP
> -When an ext2 file system is mounted with the
> -.I sync
> -option, directory entries are also implicitly synced by
> -.BR fsync ().
> -.LP
> -On kernels before 2.4,
> -.BR fsync ()
> -on big files can be inefficient.
> -An alternative might be to use the
> -.B O_SYNC
> -flag to
> -.BR open (2).
> -
>  In Linux 2.2 and earlier,
>  .BR fdatasync ()
>  is equivalent to
>  .BR fsync (),
>  and so has no performance advantage.
> +
> +The
> +.BR fsync ()
> +implementations in older kernels and lesser used filesystems
> +does not know how to flush disk caches.  In these cases disk caches need to
> +be disabled using
> +.BR hdparm (8)
> +or
> +.BR sdparm (8)
> +to guarantee safe operation.
>  .SH "SEE ALSO"
>  .BR bdflush (2),
>  .BR open (2),
> --
> To unsubscribe from this list: send the line "unsubscribe linux-man" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface"; http://man7.org/tlpi/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] fsync.2 updates

[Michael Kerrisk]

Guillem,

On Sun, Jan 1, 2012 at 3:49 PM, Guillem Jover <guillem-+FW4gsLVM0RAfugRpC6u6w@public.gmane.org> wrote:
> Hi!
>
> On 2011-11-04 6:12:04, Christoph Hellwig wrote:
>> - fsync works on any fd and does not require a writeable one,
>>   correct the EBADF error code explanation.
>
> The problem is, while this is true for Linux, that's not a safe portable
> assumption to make on POSIX in general as that behaviour is not
> specified and as such is implementation specific, some Unix systems
> do actually fail on read-only file descriptors, for example:
>
>  <http://h30097.www3.hp.com/docs/base_doc/DOCUMENTATION/V50_HTML/MAN/MAN2/0033____.HTM>
>  <http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=/com.ibm.aix.basetechref/doc/basetrf1/fsync.htm>
>
> So if this part of the patch gets applied it would be nice to add a
> note stating this is Linux specific behaviour, and that other systems
> might still fail on such condition.

Good catch! I applied the patch below.

Cheers,

Michael


--- a/man2/fsync.2
+++ b/man2/fsync.2
@@ -139,6 +139,12 @@ to a value greater than 0.
 .\" -1: unavailable, 0: ask using sysconf().
 .\" glibc defines them to 1.
 .SH NOTES
+On some UNIX systems (but not Linux),
+.I fd
+must be a
+.I writable
+file descriptor.
+
 In Linux 2.2 and earlier,
 .BR fdatasync ()
 is equivalent to

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface"; http://man7.org/tlpi/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html