sync_file_range.2: add some big WARNINGS

sync_file_range.2: add some big WARNINGS

[Christoph Hellwig]

This system call is by design completely unsuitable for any data
integrity operations.  Make that very clear in the manpage.


Signed-off-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>

Index: man-pages/man2/sync_file_range.2
===================================================================
--- man-pages.orig/man2/sync_file_range.2	2009-08-27 14:51:51.373360594 -0300
+++ man-pages/man2/sync_file_range.2	2009-08-27 14:57:35.213854927 -0300
@@ -80,11 +80,22 @@ after performing any write.
 Specifying
 .I flags
 as 0 is permitted, as a no-op.
-.SS Some details
-None of these operations write out the file's metadata.
+.SS WARNING
+This system call is extremly dangerous and should not be used in portable
+programs.  None of these operations write out the file's metadata.
 Therefore, unless the application is strictly performing overwrites of
-already-instantiated disk blocks,
-there are no guarantees that the data will be available after a crash.
+already-instantiated disk blocks, there are no guarantees that the data will
+be available after a crash.  There is no user interface to know if a
+write is purely an overwrite. On filesystem using copy on write semantics
+like
+.IR btrfs
+an over write of existing allocated blocks is impossible.  Writing into
+pre-allocated space many filesystems also require calls into the block
+allocator which this system call does not sync out to disk.
+This system call does not flush disk write caches and thus does not provide
+any data integrity on systems with volatile disk write caches.
+
+.SS Some details
 
 .B SYNC_FILE_RANGE_WAIT_BEFORE
 and
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: sync_file_range.2: add some big WARNINGS

[Michael Kerrisk]

Hi Christoph,

Thanks for this patch. Since Andrew was the original author of this
syscall, I'd like him to take a look at these changes, to get some
review.

Andrew?

Thanks

Michael

On Thu, Aug 27, 2009 at 8:01 PM, Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org> wrote:
> This system call is by design completely unsuitable for any data
> integrity operations.  Make that very clear in the manpage.
>
>
> Signed-off-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
>
> Index: man-pages/man2/sync_file_range.2
> ===================================================================
> --- man-pages.orig/man2/sync_file_range.2       2009-08-27 14:51:51.373360594 -0300
> +++ man-pages/man2/sync_file_range.2    2009-08-27 14:57:35.213854927 -0300
> @@ -80,11 +80,22 @@ after performing any write.
>  Specifying
>  .I flags
>  as 0 is permitted, as a no-op.
> -.SS Some details
> -None of these operations write out the file's metadata.
> +.SS WARNING
> +This system call is extremly dangerous and should not be used in portable
> +programs.  None of these operations write out the file's metadata.
>  Therefore, unless the application is strictly performing overwrites of
> -already-instantiated disk blocks,
> -there are no guarantees that the data will be available after a crash.
> +already-instantiated disk blocks, there are no guarantees that the data will
> +be available after a crash.  There is no user interface to know if a
> +write is purely an overwrite. On filesystem using copy on write semantics
> +like
> +.IR btrfs
> +an over write of existing allocated blocks is impossible.  Writing into
> +pre-allocated space many filesystems also require calls into the block
> +allocator which this system call does not sync out to disk.
> +This system call does not flush disk write caches and thus does not provide
> +any data integrity on systems with volatile disk write caches.
> +
> +.SS Some details
>
>  .B SYNC_FILE_RANGE_WAIT_BEFORE
>  and
>



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Watch my Linux system programming book progress to publication!
http://blog.man7.org/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: sync_file_range.2: add some big WARNINGS

[Andrew Morton]

On Sun, 20 Sep 2009 07:34:40 +0200
Michael Kerrisk <mtk.manpages-gM/Ye1E23mwN+BqQ9rBEUg@public.gmane.org> wrote:

> Hi Christoph,
> 
> Thanks for this patch. Since Andrew was the original author of this
> syscall, I'd like him to take a look at these changes, to get some
> review.
> 
> Andrew?
> 
> Thanks
> 
> Michael
> 
> On Thu, Aug 27, 2009 at 8:01 PM, Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org> wrote:
> > This system call is by design completely unsuitable for any data
> > integrity operations. __Make that very clear in the manpage.
> >
> >
> > Signed-off-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
> >
> > Index: man-pages/man2/sync_file_range.2
> > ===================================================================
> > --- man-pages.orig/man2/sync_file_range.2 __ __ __ 2009-08-27 14:51:51.373360594 -0300
> > +++ man-pages/man2/sync_file_range.2 __ __2009-08-27 14:57:35.213854927 -0300
> > @@ -80,11 +80,22 @@ after performing any write.
> > __Specifying
> > __.I flags
> > __as 0 is permitted, as a no-op.
> > -.SS Some details
> > -None of these operations write out the file's metadata.
> > +.SS WARNING
> > +This system call is extremly dangerous and should not be used in portable
> > +programs. __None of these operations write out the file's metadata.
> > __Therefore, unless the application is strictly performing overwrites of
> > -already-instantiated disk blocks,
> > -there are no guarantees that the data will be available after a crash.
> > +already-instantiated disk blocks, there are no guarantees that the data will
> > +be available after a crash. __There is no user interface to know if a
> > +write is purely an overwrite. On filesystem using copy on write semantics
> > +like
> > +.IR btrfs
> > +an over write of existing allocated blocks is impossible. __Writing into
> > +pre-allocated space many filesystems also require calls into the block
> > +allocator which this system call does not sync out to disk.
> > +This system call does not flush disk write caches and thus does not provide
> > +any data integrity on systems with volatile disk write caches.
> > +
> > +.SS Some details
> >

Shrug.  If you want.

The syscall is all about dirty memory management and userspace control
over IO timing/volume.  Trying to use it for data integrity purposes is a
bit perverse.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: sync_file_range.2: add some big WARNINGS

[Christoph Hellwig]

On Mon, Sep 21, 2009 at 01:15:19PM -0700, Andrew Morton wrote:
> The syscall is all about dirty memory management and userspace control
> over IO timing/volume.  Trying to use it for data integrity purposes is a
> bit perverse.

Yes, its is.  But that needs to be clearly documented.  If you have
better wording to make it crystal clear you're welcome.

Additionally I have a half-backed patch to allow it calling into ->fsync
to actually make it usable as a fdatasync_range/fsync_range, but I
wonder if we should just add those as syscalls instead.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: sync_file_range.2: add some big WARNINGS

[Andrew Morton]

On Tue, 22 Sep 2009 13:40:52 +0200 Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org> wrote:

> On Mon, Sep 21, 2009 at 01:15:19PM -0700, Andrew Morton wrote:
> > The syscall is all about dirty memory management and userspace control
> > over IO timing/volume.  Trying to use it for data integrity purposes is a
> > bit perverse.
> 
> Yes, its is.  But that needs to be clearly documented.  If you have
> better wording to make it crystal clear you're welcome.
> 
> Additionally I have a half-backed patch to allow it calling into ->fsync
> to actually make it usable as a fdatasync_range/fsync_range, but I
> wonder if we should just add those as syscalls instead.

hm, hard call.

I guess we should avoid permitting people to sync the metadata before
the data, because that would provide an easy way to expose huge amounts
of unwritten blocks if the machine crashes at the right time.  Or
perhaps it's not worth bothering about that.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: sync_file_range.2: add some big WARNINGS

[Christoph Hellwig]

So what are we going to do with it?  The current manpage still suggests
it might actuall be safe for overwrites which it really isn't.

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: sync_file_range.2: add some big WARNINGS

[Christoph Hellwig]

ping?

On Thu, Aug 27, 2009 at 08:01:16PM +0200, Christoph Hellwig wrote:
> This system call is by design completely unsuitable for any data
> integrity operations.  Make that very clear in the manpage.
> 
> 
> Signed-off-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
> 
> Index: man-pages/man2/sync_file_range.2
> ===================================================================
> --- man-pages.orig/man2/sync_file_range.2	2009-08-27 14:51:51.373360594 -0300
> +++ man-pages/man2/sync_file_range.2	2009-08-27 14:57:35.213854927 -0300
> @@ -80,11 +80,22 @@ after performing any write.
>  Specifying
>  .I flags
>  as 0 is permitted, as a no-op.
> -.SS Some details
> -None of these operations write out the file's metadata.
> +.SS WARNING
> +This system call is extremly dangerous and should not be used in portable
> +programs.  None of these operations write out the file's metadata.
>  Therefore, unless the application is strictly performing overwrites of
> -already-instantiated disk blocks,
> -there are no guarantees that the data will be available after a crash.
> +already-instantiated disk blocks, there are no guarantees that the data will
> +be available after a crash.  There is no user interface to know if a
> +write is purely an overwrite. On filesystem using copy on write semantics
> +like
> +.IR btrfs
> +an over write of existing allocated blocks is impossible.  Writing into
> +pre-allocated space many filesystems also require calls into the block
> +allocator which this system call does not sync out to disk.
> +This system call does not flush disk write caches and thus does not provide
> +any data integrity on systems with volatile disk write caches.
> +
> +.SS Some details
>  
>  .B SYNC_FILE_RANGE_WAIT_BEFORE
>  and
---end quoted text---
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: sync_file_range.2: add some big WARNINGS

[Christoph Hellwig]

Are you going to put this or some equivalent note in?   Currently the
manpages is extremly dangerous as it doesn't warn people about this
system call not actually guaranteeing any data integrity.

On Thu, Aug 27, 2009 at 08:01:16PM +0200, Christoph Hellwig wrote:
> This system call is by design completely unsuitable for any data
> integrity operations.  Make that very clear in the manpage.
> 
> 
> Signed-off-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
> 
> Index: man-pages/man2/sync_file_range.2
> ===================================================================
> --- man-pages.orig/man2/sync_file_range.2	2009-08-27 14:51:51.373360594 -0300
> +++ man-pages/man2/sync_file_range.2	2009-08-27 14:57:35.213854927 -0300
> @@ -80,11 +80,22 @@ after performing any write.
>  Specifying
>  .I flags
>  as 0 is permitted, as a no-op.
> -.SS Some details
> -None of these operations write out the file's metadata.
> +.SS WARNING
> +This system call is extremly dangerous and should not be used in portable
> +programs.  None of these operations write out the file's metadata.
>  Therefore, unless the application is strictly performing overwrites of
> -already-instantiated disk blocks,
> -there are no guarantees that the data will be available after a crash.
> +already-instantiated disk blocks, there are no guarantees that the data will
> +be available after a crash.  There is no user interface to know if a
> +write is purely an overwrite. On filesystem using copy on write semantics
> +like
> +.IR btrfs
> +an over write of existing allocated blocks is impossible.  Writing into
> +pre-allocated space many filesystems also require calls into the block
> +allocator which this system call does not sync out to disk.
> +This system call does not flush disk write caches and thus does not provide
> +any data integrity on systems with volatile disk write caches.
> +
> +.SS Some details
>  
>  .B SYNC_FILE_RANGE_WAIT_BEFORE
>  and
---end quoted text---
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: sync_file_range.2: add some big WARNINGS

[Michael Kerrisk]

Hi Christoph,

On Sun, Jan 10, 2010 at 3:49 AM, Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org> wrote:
> Are you going to put this or some equivalent note in?   Currently the
> manpages is extremly dangerous as it doesn't warn people about this
> system call not actually guaranteeing any data integrity.

I lightly edited your patch, and applied it. Sorry for the long delay.

Cheers,

Michael

> On Thu, Aug 27, 2009 at 08:01:16PM +0200, Christoph Hellwig wrote:
>> This system call is by design completely unsuitable for any data
>> integrity operations.  Make that very clear in the manpage.
>>
>>
>> Signed-off-by: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
>>
>> Index: man-pages/man2/sync_file_range.2
>> ===================================================================
>> --- man-pages.orig/man2/sync_file_range.2     2009-08-27 14:51:51.373360594 -0300
>> +++ man-pages/man2/sync_file_range.2  2009-08-27 14:57:35.213854927 -0300
>> @@ -80,11 +80,22 @@ after performing any write.
>>  Specifying
>>  .I flags
>>  as 0 is permitted, as a no-op.
>> -.SS Some details
>> -None of these operations write out the file's metadata.
>> +.SS WARNING
>> +This system call is extremly dangerous and should not be used in portable
>> +programs.  None of these operations write out the file's metadata.
>>  Therefore, unless the application is strictly performing overwrites of
>> -already-instantiated disk blocks,
>> -there are no guarantees that the data will be available after a crash.
>> +already-instantiated disk blocks, there are no guarantees that the data will
>> +be available after a crash.  There is no user interface to know if a
>> +write is purely an overwrite. On filesystem using copy on write semantics
>> +like
>> +.IR btrfs
>> +an over write of existing allocated blocks is impossible.  Writing into
>> +pre-allocated space many filesystems also require calls into the block
>> +allocator which this system call does not sync out to disk.
>> +This system call does not flush disk write caches and thus does not provide
>> +any data integrity on systems with volatile disk write caches.
>> +
>> +.SS Some details
>>
>>  .B SYNC_FILE_RANGE_WAIT_BEFORE
>>  and
> ---end quoted text---
>



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface" http://blog.man7.org/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: sync_file_range.2: add some big WARNINGS

[Christoph Hellwig]

On Sun, Jan 17, 2010 at 06:48:03PM +1300, Michael Kerrisk wrote:
> Hi Christoph,
> 
> On Sun, Jan 10, 2010 at 3:49 AM, Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org> wrote:
> > Are you going to put this or some equivalent note in? ? Currently the
> > manpages is extremly dangerous as it doesn't warn people about this
> > system call not actually guaranteeing any data integrity.
> 
> I lightly edited your patch, and applied it. Sorry for the long delay.

Thanks a lot!

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html