* Re: fsync, rename, O_ATOMIC/O_PONIES
@ 2012-03-02 12:39 Olaf van der Spek
2012-03-02 13:12 ` Christoph Hellwig
0 siblings, 1 reply; 6+ messages in thread
From: Olaf van der Spek @ 2012-03-02 12:39 UTC (permalink / raw)
To: xfs
Hi XFS devs,
I'm a bit late to the party, but I'd like to respond to some comments at
http://lwn.net/Articles/476263/#Comments
http://lwn.net/Articles/484709/
> Could any of the fsync advocates post real code that does the atomic
variant of open, write, close?
> Hint: it's not possible without tons of regressions.
> Linux devs should really provide a proper solution (like O_ATOMIC)
instead of blaming app devs for not doing the impossible.
I'd like to ask:
- Is there a tool to log all unsafe operations?
- What is the *right* way to update a file?
I've asked the last question multiple times, but nobody has been able to
give me a proper answer.
Greetings,
Olaf
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: fsync, rename, O_ATOMIC/O_PONIES
2012-03-02 12:39 fsync, rename, O_ATOMIC/O_PONIES Olaf van der Spek
@ 2012-03-02 13:12 ` Christoph Hellwig
2012-03-04 17:14 ` Olaf van der Spek
0 siblings, 1 reply; 6+ messages in thread
From: Christoph Hellwig @ 2012-03-02 13:12 UTC (permalink / raw)
To: Olaf van der Spek; +Cc: xfs
> I'd like to ask:
> - Is there a tool to log all unsafe operations?
What is an unsafe operation? Anything that's dumb and might lose data?
The possibilities are sheer endless.
> - What is the *right* way to update a file?
fd = open(tmpfile, ...);
write(fd, ...); // or any other update
fdatasync(fd);
rename(tmpfile, realfile);
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: fsync, rename, O_ATOMIC/O_PONIES
2012-03-02 13:12 ` Christoph Hellwig
@ 2012-03-04 17:14 ` Olaf van der Spek
2012-03-05 1:02 ` Dave Chinner
0 siblings, 1 reply; 6+ messages in thread
From: Olaf van der Spek @ 2012-03-04 17:14 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: xfs
On 2-3-2012 14:12, Christoph Hellwig wrote:
>> I'd like to ask:
>> - Is there a tool to log all unsafe operations?
>
> What is an unsafe operation? Anything that's dumb and might lose data?
For example
> The possibilities are sheer endless.
And?
>> - What is the *right* way to update a file?
>
> fd = open(tmpfile, ...);
> write(fd, ...); // or any other update
> fdatasync(fd);
> rename(tmpfile, realfile);
Argh, come on.
That's not real and it's not complete. tmpfile is undefined, errors
aren't handled and you have lots of unlisted assumptions or regressions.
Olaf
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: fsync, rename, O_ATOMIC/O_PONIES
2012-03-04 17:14 ` Olaf van der Spek
@ 2012-03-05 1:02 ` Dave Chinner
2012-03-05 15:44 ` Olaf van der Spek
0 siblings, 1 reply; 6+ messages in thread
From: Dave Chinner @ 2012-03-05 1:02 UTC (permalink / raw)
To: Olaf van der Spek; +Cc: Christoph Hellwig, xfs
On Sun, Mar 04, 2012 at 06:14:39PM +0100, Olaf van der Spek wrote:
> On 2-3-2012 14:12, Christoph Hellwig wrote:
> >>I'd like to ask:
> >>- Is there a tool to log all unsafe operations?
> >
> >What is an unsafe operation? Anything that's dumb and might lose data?
>
> For example
>
> >The possibilities are sheer endless.
>
> And?
>
> >>- What is the *right* way to update a file?
> >
> > fd = open(tmpfile, ...);
> > write(fd, ...); // or any other update
> > fdatasync(fd);
> > rename(tmpfile, realfile);
>
> Argh, come on.
> That's not real and it's not complete. tmpfile is undefined, errors
> aren't handled and you have lots of unlisted assumptions or
> regressions.
The above is perfectly reasonable psuedo code for quickly describing
how to safely overwriting a file. If you want to know about error
handling and assumptions, read the man pages for operation.
To make it easy for you, I'll just point out this has already been
dealt with early in in the LWN thread for that article, here's the
link to the comment and the relevant LWN article providing you with
all the information you want:
http://lwn.net/Articles/476606/
https://lwn.net/Articles/457667/
And that my stance on this atomic rename subject is simply this: If
we want to change reanme behaviour, then the filesystem is not the
right place to do it - atomic rename semantics need to be defined
and enforced at the VFS. See here:
http://lwn.net/Articles/479152/
This has all been hashed out before - we don't need to revisit this
discussion here. If you want to solve the problem once and for all,
submit patches to LKML to implement the new rename semantics in a
new syscall....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: fsync, rename, O_ATOMIC/O_PONIES
2012-03-05 1:02 ` Dave Chinner
@ 2012-03-05 15:44 ` Olaf van der Spek
2012-03-05 23:17 ` Dave Chinner
0 siblings, 1 reply; 6+ messages in thread
From: Olaf van der Spek @ 2012-03-05 15:44 UTC (permalink / raw)
To: Dave Chinner; +Cc: Christoph Hellwig, xfs
On 5-3-2012 2:02, Dave Chinner wrote:
>> Argh, come on.
>> That's not real and it's not complete. tmpfile is undefined, errors
>> aren't handled and you have lots of unlisted assumptions or
>> regressions.
>
> The above is perfectly reasonable psuedo code for quickly describing
> how to safely overwriting a file. If you want to know about error
> handling and assumptions, read the man pages for operation.
But I don't have a psuedo code compiler. Using psuedo code hides
complexity and bugs. Even the code from Jeff Moyer in the article you're
refering too contained bugs.
Don't you think it's quite strange there's no real code available to
handle this widespread problem?
> To make it easy for you, I'll just point out this has already been
> dealt with early in in the LWN thread for that article, here's the
> link to the comment and the relevant LWN article providing you with
> all the information you want:
>
> http://lwn.net/Articles/476606/
> https://lwn.net/Articles/457667/
It also doesn't handle the assumptions / regressions:
http://lwn.net/Articles/485182/
Assuming the target is not a symlink to a different volume
Assuming you are allowed to create the tmp file
Assuming you are allowed to overwrite an existing file having the same
name as your tmp file
Assuming it's ok to reset meta-data, like file owner, permissions, acls,
creation timestamp, etc.
Assuming the performance regression due to fsync is ok (request was for
atomic, not durable)
> And that my stance on this atomic rename subject is simply this: If
> we want to change reanme behaviour, then the filesystem is not the
> right place to do it - atomic rename semantics need to be defined
> and enforced at the VFS. See here:
>
> http://lwn.net/Articles/479152/
I don't think rename is part of the solution.
Olaf
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: fsync, rename, O_ATOMIC/O_PONIES
2012-03-05 15:44 ` Olaf van der Spek
@ 2012-03-05 23:17 ` Dave Chinner
0 siblings, 0 replies; 6+ messages in thread
From: Dave Chinner @ 2012-03-05 23:17 UTC (permalink / raw)
To: Olaf van der Spek; +Cc: Christoph Hellwig, xfs
On Mon, Mar 05, 2012 at 04:44:26PM +0100, Olaf van der Spek wrote:
> On 5-3-2012 2:02, Dave Chinner wrote:
> >>Argh, come on.
> >>That's not real and it's not complete. tmpfile is undefined, errors
> >>aren't handled and you have lots of unlisted assumptions or
> >>regressions.
> >
> >The above is perfectly reasonable psuedo code for quickly describing
> >how to safely overwriting a file. If you want to know about error
> >handling and assumptions, read the man pages for operation.
>
> But I don't have a psuedo code compiler. Using psuedo code hides
> complexity and bugs. Even the code from Jeff Moyer in the article
> you're refering too contained bugs.
>
> Don't you think it's quite strange there's no real code available to
> handle this widespread problem?
What you are saying is this:
"I'm too lazy to understand data integrity principles and apply them
to my application. Can someone please write some code perfect for my
needs so I can copy and paste them into my application."
Stop wasting our time by being obtuse and repeating silly arguments
as to why this isn't your problem and instead go read the man pages
and modify the example code you've already been pointed at to do
exactly what you need.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-03-05 23:17 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-03-02 12:39 fsync, rename, O_ATOMIC/O_PONIES Olaf van der Spek
2012-03-02 13:12 ` Christoph Hellwig
2012-03-04 17:14 ` Olaf van der Spek
2012-03-05 1:02 ` Dave Chinner
2012-03-05 15:44 ` Olaf van der Spek
2012-03-05 23:17 ` Dave Chinner
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox