Re: [patch 0/9] writeback data integrity and other fixes (take 3)

Re: [patch 0/9] writeback data integrity and other fixes (take 3)

[Nick Piggin]

On Wed, Oct 29, 2008 at 05:44:17AM -0400, Christoph Hellwig wrote:
> On Wed, Oct 29, 2008 at 10:21:43AM +0100, Nick Piggin wrote:
> > Please do.
> 
> Well, there's one stumling block I haven't made progress on yet:
> 
> I've changed the prototype of ->fsync to lose the dentry as we should
> always have a valid file struct.  Except that nfsd doesn't on
> directories.  So I either need to fake up a file there, or bail out
> and add a ->dir_sync export operation that needs just a dentry.

OK. I don't know much about hthat code, but I would think nfsd
should look as close to the syscall layer as possible. I guess
there must be something prohibitive (some protocol semantics?).

Is there anything that particularly makes it a file operation
as opposed to an inode operation?

Re: [patch 0/9] writeback data integrity and other fixes (take 3)

[Jamie Lokier]

Nick Piggin wrote:
> On Wed, Oct 29, 2008 at 05:44:17AM -0400, Christoph Hellwig wrote:
> > On Wed, Oct 29, 2008 at 10:21:43AM +0100, Nick Piggin wrote:
> > > Please do.
> > 
> > Well, there's one stumling block I haven't made progress on yet:
> > 
> > I've changed the prototype of ->fsync to lose the dentry as we should
> > always have a valid file struct.  Except that nfsd doesn't on
> > directories.  So I either need to fake up a file there, or bail out
> > and add a ->dir_sync export operation that needs just a dentry.
> 
> OK. I don't know much about hthat code, but I would think nfsd
> should look as close to the syscall layer as possible. I guess
> there must be something prohibitive (some protocol semantics?).
> 
> Is there anything that particularly makes it a file operation
> as opposed to an inode operation?

In principle, is fsync() required to flush all dirty data written
through any file descriptor ever, or just dirty data written through
the file descriptor used for fsync()?

-- Jamie

Re: [patch 0/9] writeback data integrity and other fixes (take 3)

[Ric Wheeler]

Jamie Lokier wrote:
> Nick Piggin wrote:
>   
>> On Wed, Oct 29, 2008 at 05:44:17AM -0400, Christoph Hellwig wrote:
>>     
>>> On Wed, Oct 29, 2008 at 10:21:43AM +0100, Nick Piggin wrote:
>>>       
>>>> Please do.
>>>>         
>>> Well, there's one stumling block I haven't made progress on yet:
>>>
>>> I've changed the prototype of ->fsync to lose the dentry as we should
>>> always have a valid file struct.  Except that nfsd doesn't on
>>> directories.  So I either need to fake up a file there, or bail out
>>> and add a ->dir_sync export operation that needs just a dentry.
>>>       
>> OK. I don't know much about hthat code, but I would think nfsd
>> should look as close to the syscall layer as possible. I guess
>> there must be something prohibitive (some protocol semantics?).
>>
>> Is there anything that particularly makes it a file operation
>> as opposed to an inode operation?
>>     
>
> In principle, is fsync() required to flush all dirty data written
> through any file descriptor ever, or just dirty data written through
> the file descriptor used for fsync()?
>
> -- Jamie
> --
>   
http://www.opengroup.org/onlinepubs/009695399/functions/fsync.html

Is a pointer to what seems to be the official posix spec for this - it 
is definitely per file descriptor, not per file system, etc...

What can happen by side effect (depending on the implementation) is that 
you can actually force out all data for any file. I found that you can 
approach non-fsync speeds for an fsync per file workload by simply 
writing all of the files out, then going back and fsync'ing them one at 
a time (last file first makes a bit of a difference). With that 
technique, you do get the hard promise of full data integrity and high 
speed. This is useful when you want to do bulk writes (tar, etc)

ric

Re: [patch 0/9] writeback data integrity and other fixes (take 3)

[Chris Mason]

On Wed, 2008-10-29 at 09:32 -0400, Ric Wheeler wrote:
> Jamie Lokier wrote:

> >> Is there anything that particularly makes it a file operation
> >> as opposed to an inode operation?
> >>     
> >
> > In principle, is fsync() required to flush all dirty data written
> > through any file descriptor ever, or just dirty data written through
> > the file descriptor used for fsync()?
> >
> > -- Jamie
> > --
> >   
> http://www.opengroup.org/onlinepubs/009695399/functions/fsync.html
> 
> Is a pointer to what seems to be the official posix spec for this - it 
> is definitely per file descriptor, not per file system, etc...
> 

Maybe I'm reading Jamie's question wrong, but I think he's saying:

/* open exactly the same file twice */
fd = open("file");
fd2 = open("file");

write(fd, "stuff")
write(fd2, "more stuff")
fsync(fd);

Does the fsync promise "more stuff" will be on disk?  I think the answer
should be yes.

-chris

Re: [patch 0/9] writeback data integrity and other fixes (take 3)

[Nick Piggin]

On Wed, Oct 29, 2008 at 10:56:36AM -0400, Chris Mason wrote:
> On Wed, 2008-10-29 at 09:32 -0400, Ric Wheeler wrote:
> > Jamie Lokier wrote:
> 
> > >> Is there anything that particularly makes it a file operation
> > >> as opposed to an inode operation?
> > >>     
> > >
> > > In principle, is fsync() required to flush all dirty data written
> > > through any file descriptor ever, or just dirty data written through
> > > the file descriptor used for fsync()?
> > >
> > > -- Jamie
> > > --
> > >   
> > http://www.opengroup.org/onlinepubs/009695399/functions/fsync.html
> > 
> > Is a pointer to what seems to be the official posix spec for this - it 
> > is definitely per file descriptor, not per file system, etc...
> > 
> 
> Maybe I'm reading Jamie's question wrong, but I think he's saying:
> 
> /* open exactly the same file twice */
> fd = open("file");
> fd2 = open("file");
> 
> write(fd, "stuff")
> write(fd2, "more stuff")
> fsync(fd);
> 
> Does the fsync promise "more stuff" will be on disk?  I think the answer
> should be yes.

I think so. And this is in the context of making ->fsync an inode
operation and avoid the NFS NULL-file problem... I don't think there
is any fd specific metadata that fsync has to deal with? Any other
reasons it has to be a file operation?

Re: [patch 0/9] writeback data integrity and other fixes (take 3)

[jim owens]

Nick Piggin wrote:
> On Wed, Oct 29, 2008 at 10:56:36AM -0400, Chris Mason wrote:
>> On Wed, 2008-10-29 at 09:32 -0400, Ric Wheeler wrote:
>>> Jamie Lokier wrote:
>>>>> Is there anything that particularly makes it a file operation
>>>>> as opposed to an inode operation?
>>>>>     
>>>> In principle, is fsync() required to flush all dirty data written
>>>> through any file descriptor ever, or just dirty data written through
>>>> the file descriptor used for fsync()?
>>>>
>>>> -- Jamie
>>>> --
>>>>   
>>> http://www.opengroup.org/onlinepubs/009695399/functions/fsync.html
>>>
>>> Is a pointer to what seems to be the official posix spec for this - it 
>>> is definitely per file descriptor, not per file system, etc...
>>>
>> Maybe I'm reading Jamie's question wrong, but I think he's saying:
>>
>> /* open exactly the same file twice */
>> fd = open("file");
>> fd2 = open("file");
>>
>> write(fd, "stuff")
>> write(fd2, "more stuff")
>> fsync(fd);
>>
>> Does the fsync promise "more stuff" will be on disk?  I think the answer
>> should be yes.
> 
> I think so. And this is in the context of making ->fsync an inode
> operation and avoid the NFS NULL-file problem... I don't think there
> is any fd specific metadata that fsync has to deal with? Any other
> reasons it has to be a file operation?

NO, or at least *not the posix definition*.  It is normal
in unix-like operating systems to always flush everything
dirty on the inode no matter what stream it arrived on.

Flushing everything is permitted but not the requirement so
applications must not expect this is *promised* or they
will not be portable.  It is only guaranteed that "stuff"
in this example will be on disk.

AFAIK the fsync semantic comes from the days of dinosaurs,
mainframes, and minicomputers... when a lot of operating
systems had user-space libraries that buffered the I/O.
On fsync(fd), the "fd2" data would still be in user-space.

jim

Re: [patch 0/9] writeback data integrity and other fixes (take 3)

[Jim Rees]

jim owens wrote:

  AFAIK the fsync semantic comes from the days of dinosaurs,
  mainframes, and minicomputers... when a lot of operating
  systems had user-space libraries that buffered the I/O.
  On fsync(fd), the "fd2" data would still be in user-space.

User space buffering happens in stdio, which is above the system call
level.  It's been that way since fsync() was first introduced, and is still
that way today.

Re: [patch 0/9] writeback data integrity and other fixes (take 3)

[Dave Chinner]

On Wed, Oct 29, 2008 at 12:22:35PM +0000, Jamie Lokier wrote:
> Nick Piggin wrote:
> > On Wed, Oct 29, 2008 at 05:44:17AM -0400, Christoph Hellwig wrote:
> > > On Wed, Oct 29, 2008 at 10:21:43AM +0100, Nick Piggin wrote:
> > > > Please do.
> > > 
> > > Well, there's one stumling block I haven't made progress on yet:
> > > 
> > > I've changed the prototype of ->fsync to lose the dentry as we should
> > > always have a valid file struct.  Except that nfsd doesn't on
> > > directories.  So I either need to fake up a file there, or bail out
> > > and add a ->dir_sync export operation that needs just a dentry.
> > 
> > OK. I don't know much about hthat code, but I would think nfsd
> > should look as close to the syscall layer as possible. I guess
> > there must be something prohibitive (some protocol semantics?).
> > 
> > Is there anything that particularly makes it a file operation
> > as opposed to an inode operation?
> 
> In principle, is fsync() required to flush all dirty data written
> through any file descriptor ever, or just dirty data written through
> the file descriptor used for fsync()?

fsync() is required to flush the data that is dirty at the time of the
call, as well as any associated metadata needed to reference that data. It
doesn't matter who wrote the data in the first place....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com