Re: [PATCH 1/3] block: Create sysfs knobs to override FLUSH/FUA support flags

Re: [PATCH 1/3] block: Create sysfs knobs to override FLUSH/FUA support flags

[Tejun Heo]

On Tue, Jan 25, 2011 at 11:16:26PM -0800, Darrick J. Wong wrote:
> This patch is the first in a series to refactor the barrier= mount options out
> of the filesystem code.  This patch adds sysfs knobs to disable flush and FUA;
> of course, the automatic default is the safe choice, i.e. to leave them
> enabled.  Obviously, only a seasoned administrator should ever be overriding
> the defaults.

Hmmm... wouldn't it be better to just export flush and fua instead of
ignore_*?  So that the admin can turn things on and off as [s]he seems
fit?  Also, it might be better to export them in a single attribute,
say cache_control or something.  Only subset of the combinations make
sense anyway - none, flush, flush_fua.

Thanks.

-- 
tejun

Re: [PATCH 1/3] block: Create sysfs knobs to override FLUSH/FUA support flags

[Darrick J. Wong]

On Wed, Jan 26, 2011 at 10:30:51AM +0100, Tejun Heo wrote:
> On Tue, Jan 25, 2011 at 11:16:26PM -0800, Darrick J. Wong wrote:
> > This patch is the first in a series to refactor the barrier= mount options out
> > of the filesystem code.  This patch adds sysfs knobs to disable flush and FUA;
> > of course, the automatic default is the safe choice, i.e. to leave them
> > enabled.  Obviously, only a seasoned administrator should ever be overriding
> > the defaults.
> 
> Hmmm... wouldn't it be better to just export flush and fua instead of
> ignore_*?  So that the admin can turn things on and off as [s]he seems

I considered having a general knob to override the automatic FLUSH/FUA
detection, but I thought that it wasn't a good idea to provide a mechanism to
enable features that devices don't advertise.  Mostly I was imagining horror
scenarios like USB storage devices that claim no write cache and but then catch
on fire if someone sends flush anyway.  Not using advertised features seemed
less risky.

> fit?  Also, it might be better to export them in a single attribute,
> say cache_control or something.  Only subset of the combinations make
> sense anyway - none, flush, flush_fua.

I agree.  It could be simplified even further to a simple boolean that means
"use neither" or "use whatever's supported".

--D

Re: [PATCH 1/3] block: Create sysfs knobs to override FLUSH/FUA support flags

[Greg KH]

On Tue, Jan 25, 2011 at 11:16:26PM -0800, Darrick J. Wong wrote:
> This patch is the first in a series to refactor the barrier= mount options out
> of the filesystem code.  This patch adds sysfs knobs to disable flush and FUA;

If you add sysfs files, you must also add documentation for these files
in Documentation/ABI/.

Please do that in this patch series.

thanks,

greg k-h

Re: [PATCH 3/3] ext4: Deprecate barrier= and nobarrier mount options

[Tejun Heo]

Hello,

On Tue, Jan 25, 2011 at 11:23:29PM -0800, Darrick J. Wong wrote:
> As part of migrating the FLUSH/FUA knob to the block layer, remove the support
> code for the barrier-related mount options and remove the conditionals around
> flushes in favor of always issuing the flush.  The block layer will handle
> gracefully the situation where a FLUSH or FUA request is issued to a device
> that doesn't support it.  Modify the option parsing code to print a warning if
> someone tries to use the old mount option.
> 
> Note: The nobarrier bit in the default mount flags is now useless.

The option is something which users are already quite familiar with.
I think we'll just have to carry this around.  What we can do, tho, is
moving the actual control mechanism to block layer -
ie. blkdev_skip_flush() or something like that which ignores flush
requests for the current exclusive opener.

Thanks.

-- 
tejun

Re: [PATCH 3/3] ext4: Deprecate barrier= and nobarrier mount options

[Jan Kara]

  Hello,

On Wed 26-01-11 10:36:32, Tejun Heo wrote:
> On Tue, Jan 25, 2011 at 11:23:29PM -0800, Darrick J. Wong wrote:
> > As part of migrating the FLUSH/FUA knob to the block layer, remove the support
> > code for the barrier-related mount options and remove the conditionals around
> > flushes in favor of always issuing the flush.  The block layer will handle
> > gracefully the situation where a FLUSH or FUA request is issued to a device
> > that doesn't support it.  Modify the option parsing code to print a warning if
> > someone tries to use the old mount option.
> > 
> > Note: The nobarrier bit in the default mount flags is now useless.
> 
> The option is something which users are already quite familiar with.
> I think we'll just have to carry this around.  What we can do, tho, is
> moving the actual control mechanism to block layer -
> ie. blkdev_skip_flush() or something like that which ignores flush
> requests for the current exclusive opener.
  Ted should have a final word about this but I believe it's possible to
deprecate the mount options. Maybe with some transition period where
deprecation message is shown but the option actually still works. That
being said I'm not sure what we should do when someone has a disk with two
partitions and one partition is mounted with barriers and another one
without them - sure, one has to think hard to find a sane use case for this
(possibly if user does not care about data after a crash on one of the
partitions, in which case he should probably use nojournal mode) but it
should probably work.

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

Re: [PATCH 3/3] ext4: Deprecate barrier= and nobarrier mount options

[Tejun Heo]

Hello,

On Wed, Jan 26, 2011 at 11:47:34AM +0100, Jan Kara wrote:
>   Ted should have a final word about this but I believe it's possible to
> deprecate the mount options. Maybe with some transition period where
> deprecation message is shown but the option actually still works. That
> being said I'm not sure what we should do when someone has a disk with two
> partitions and one partition is mounted with barriers and another one
> without them - sure, one has to think hard to find a sane use case for this
> (possibly if user does not care about data after a crash on one of the
> partitions, in which case he should probably use nojournal mode) but it
> should probably work.

The policy can be made per-bdev (which maps to per-partition), so I
don't think that's a big problem.

Thanks.

-- 
tejun

Re: [PATCH 3/3] ext4: Deprecate barrier= and nobarrier mount options

[Ric Wheeler]

On 01/26/2011 05:47 AM, Jan Kara wrote:
>    Hello,
>
> On Wed 26-01-11 10:36:32, Tejun Heo wrote:
>> On Tue, Jan 25, 2011 at 11:23:29PM -0800, Darrick J. Wong wrote:
>>> As part of migrating the FLUSH/FUA knob to the block layer, remove the support
>>> code for the barrier-related mount options and remove the conditionals around
>>> flushes in favor of always issuing the flush.  The block layer will handle
>>> gracefully the situation where a FLUSH or FUA request is issued to a device
>>> that doesn't support it.  Modify the option parsing code to print a warning if
>>> someone tries to use the old mount option.
>>>
>>> Note: The nobarrier bit in the default mount flags is now useless.
>> The option is something which users are already quite familiar with.
>> I think we'll just have to carry this around.  What we can do, tho, is
>> moving the actual control mechanism to block layer -
>> ie. blkdev_skip_flush() or something like that which ignores flush
>> requests for the current exclusive opener.
>    Ted should have a final word about this but I believe it's possible to
> deprecate the mount options. Maybe with some transition period where
> deprecation message is shown but the option actually still works. That
> being said I'm not sure what we should do when someone has a disk with two
> partitions and one partition is mounted with barriers and another one
> without them - sure, one has to think hard to find a sane use case for this
> (possibly if user does not care about data after a crash on one of the
> partitions, in which case he should probably use nojournal mode) but it
> should probably work.
>
> 								Honza

I would also like to suggest that mount is the right time to change a per file 
system behaviour. We have well known paths (add the mount options to /etc/fstab) 
and sys admins understand it.

If we have to poke a file, what is the suggested mechanism for doing this? 
Another, new script or user space utility that needs to be run at mount time for 
each file system?

If we did want to deprecate the "barrier" name, I would leave it in place and 
replace it with something meaningful (at mount time!) to users like 
"data_safety" and "no_data_safety" modes :)

Ric

Re: [PATCH 3/3] ext4: Deprecate barrier= and nobarrier mount options

[Tejun Heo]

Hello,

On Wed, Jan 26, 2011 at 07:16:18AM -0500, Ric Wheeler wrote:
> I would also like to suggest that mount is the right time to change
> a per file system behaviour. We have well known paths (add the mount
> options to /etc/fstab) and sys admins understand it.
> 
> If we have to poke a file, what is the suggested mechanism for doing
> this? Another, new script or user space utility that needs to be run
> at mount time for each file system?
> 
> If we did want to deprecate the "barrier" name, I would leave it in
> place and replace it with something meaningful (at mount time!) to
> users like "data_safety" and "no_data_safety" modes :)

With barrier on by default these days, I don't think it's necessary to
fiddle with it too much.  It's not as important as before.  Let's
leave it as it is.  Barrier, flush, it doesn't really matter how it's
called.

In the long run tho, I think filesystems should just always enable
flush/barrier and the fiddling belongs to the block layer.

Thanks.

-- 
tejun

Re: [PATCH 3/3] ext4: Deprecate barrier= and nobarrier mount options

[torn5]

On 01/26/2011 11:47 AM, Jan Kara wrote:
>   Hello,
>
> On Wed 26-01-11 10:36:32, Tejun Heo wrote:
>   Ted should have a final word about this but I believe it's possible to
> deprecate the mount options. Maybe with some transition period where
> deprecation message is shown but the option actually still works.

+1 for this from a user.

I'd like to see this moved to blockdev sysfs in the long term.

During the transition period there could be both the old barrier mount
option (with a deprecated warning upon use) and the sysfs knobs.

Flush and FUA would be delivered to disk only if both the barriers are
enabled in the mount options (which is also the default) and the sysfs
are at "noignore" value. Alternatively, the barrier mount option could
affect the sysfs knobs values upon mount. Eventually the barrier mount
option would be removed becoming always-on, leaving only the sysfs knobs.

Re: [PATCHSET] Refactor barrier=/nobarrier flags from fs to block layer

[Ric Wheeler]

On 01/26/2011 02:12 AM, Darrick J. Wong wrote:
> Hello,
>
>  From what I can tell, most of the filesystems that know how to issue commands
> to flush the write cache also have some mechanism for the user to override
> whether or not the filesystem actually issues those flushes.  Unfortunately,
> the term "barrier" is obsolete having been changed into flushes in 2.6.36, and
> many of the filesystems implement the mount options with slightly different
> syntaxes (barrier=[0|1|none|flush], nobarrier, etc).

Why remove the mount option? We have been using that term and educating users 
about how to use it for many, many years.

I see no reason to remove mount options at all.

Ric

> This patchset adds to the block layer a sysfs knob that an administrator can
> use to disable flushes, and removes the mount options from the filesystem code.
> As a starting point, I'm removing the mount options and flush toggle from
> jbd2/ext4.
>
> Anyway, I'm looking for some feedback about refactoring the barrier/flush
> control knob into the block layer.  It sounds like we want a knob that picks
> the safest option (issue flushes when supported) unless the administrator
> decides that it is appropriate to do otherwise.  I suspect that there are good
> arguments for not having a knob at all, and good arguments for a safe knob.
> However, since I don't see the barrier options being removed en masse, I'm
> assuming that we still want a knob somewhere.  Do we need the ignore_fua knob
> too?  Is this the proper way to deprecate mount options out of filesystems?
>
> --D

Re: [PATCHSET] Refactor barrier=/nobarrier flags from fs to block layer

[Ric Wheeler]

On 01/26/2011 02:12 AM, Darrick J. Wong wrote:
> Hello,
>
>  From what I can tell, most of the filesystems that know how to issue commands
> to flush the write cache also have some mechanism for the user to override
> whether or not the filesystem actually issues those flushes.  Unfortunately,
> the term "barrier" is obsolete having been changed into flushes in 2.6.36, and
> many of the filesystems implement the mount options with slightly different
> syntaxes (barrier=[0|1|none|flush], nobarrier, etc).
>
> This patchset adds to the block layer a sysfs knob that an administrator can
> use to disable flushes, and removes the mount options from the filesystem code.
> As a starting point, I'm removing the mount options and flush toggle from
> jbd2/ext4.
>
> Anyway, I'm looking for some feedback about refactoring the barrier/flush
> control knob into the block layer.  It sounds like we want a knob that picks
> the safest option (issue flushes when supported) unless the administrator
> decides that it is appropriate to do otherwise.  I suspect that there are good
> arguments for not having a knob at all, and good arguments for a safe knob.
> However, since I don't see the barrier options being removed en masse, I'm
> assuming that we still want a knob somewhere.  Do we need the ignore_fua knob
> too?  Is this the proper way to deprecate mount options out of filesystems?
>
> --D

Just to be clear, I strongly object to remove the mount options.

"Barrier" and "poke a control knob in the block" layer are both equally 
mysterious and meaningless to real users, so I do not see this as a gain,

Ric

Re: [PATCHSET] Refactor barrier=/nobarrier flags from fs to block layer

[Eric Sandeen]

On 1/26/11 5:49 AM, Ric Wheeler wrote:
> On 01/26/2011 02:12 AM, Darrick J. Wong wrote:
>> Hello,
>>
>>  From what I can tell, most of the filesystems that know how to issue commands
>> to flush the write cache also have some mechanism for the user to override
>> whether or not the filesystem actually issues those flushes.  Unfortunately,
>> the term "barrier" is obsolete having been changed into flushes in 2.6.36, and
>> many of the filesystems implement the mount options with slightly different
>> syntaxes (barrier=[0|1|none|flush], nobarrier, etc).
>>
>> This patchset adds to the block layer a sysfs knob that an administrator can
>> use to disable flushes, and removes the mount options from the filesystem code.
>> As a starting point, I'm removing the mount options and flush toggle from
>> jbd2/ext4.
>>
>> Anyway, I'm looking for some feedback about refactoring the barrier/flush
>> control knob into the block layer.  It sounds like we want a knob that picks
>> the safest option (issue flushes when supported) unless the administrator
>> decides that it is appropriate to do otherwise.  I suspect that there are good
>> arguments for not having a knob at all, and good arguments for a safe knob.
>> However, since I don't see the barrier options being removed en masse, I'm
>> assuming that we still want a knob somewhere.  Do we need the ignore_fua knob
>> too?  Is this the proper way to deprecate mount options out of filesystems?
>>
>> --D
> 
> Just to be clear, I strongly object to remove the mount options.

Agreed, we are just finally, barely starting to win the education battle here.
Removing or changing the option now will just set us back.  It should at
LEAST remain as a deprecated option, with the deprecation message pointing
to crystal-clear documentation.

-Eric

> "Barrier" and "poke a control knob in the block" layer are both equally mysterious and meaningless to real users, so I do not see this as a gain,
> 
> Ric
> 
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHSET] Refactor barrier=/nobarrier flags from fs to block layer

[Darrick J. Wong]

On Wed, Jan 26, 2011 at 10:41:35AM -0600, Eric Sandeen wrote:
> On 1/26/11 5:49 AM, Ric Wheeler wrote:
> > On 01/26/2011 02:12 AM, Darrick J. Wong wrote:
> >> Hello,
> >>
> >>  From what I can tell, most of the filesystems that know how to issue commands
> >> to flush the write cache also have some mechanism for the user to override
> >> whether or not the filesystem actually issues those flushes.  Unfortunately,
> >> the term "barrier" is obsolete having been changed into flushes in 2.6.36, and
> >> many of the filesystems implement the mount options with slightly different
> >> syntaxes (barrier=[0|1|none|flush], nobarrier, etc).
> >>
> >> This patchset adds to the block layer a sysfs knob that an administrator can
> >> use to disable flushes, and removes the mount options from the filesystem code.
> >> As a starting point, I'm removing the mount options and flush toggle from
> >> jbd2/ext4.
> >>
> >> Anyway, I'm looking for some feedback about refactoring the barrier/flush
> >> control knob into the block layer.  It sounds like we want a knob that picks
> >> the safest option (issue flushes when supported) unless the administrator
> >> decides that it is appropriate to do otherwise.  I suspect that there are good
> >> arguments for not having a knob at all, and good arguments for a safe knob.
> >> However, since I don't see the barrier options being removed en masse, I'm
> >> assuming that we still want a knob somewhere.  Do we need the ignore_fua knob
> >> too?  Is this the proper way to deprecate mount options out of filesystems?
> >>
> >> --D
> > 
> > Just to be clear, I strongly object to remove the mount options.
> 
> Agreed, we are just finally, barely starting to win the education battle here.
> Removing or changing the option now will just set us back.  It should at
> LEAST remain as a deprecated option, with the deprecation message pointing
> to crystal-clear documentation.

Ok, how about a second proposal:

1. Put the sysfs knob and the toggle code in the block layer, similar to patch
#1, only make it a per-bdev toggle so each mount can have its own override
parameters.

2. Add some sort of "nocacheflush" option to the VFS layer to adjust the knob.
With this we gain a consistent mount option syntax across all the filesystems,
though what it means for a networked fs is questionable.  I guess you could
reject the mount option if there's no block device under the fs.  Also, any fs
that someday grows an issue-flush feature won't have to add its own option
parsing code.

At umount time, do we undo whatever overrides we set up at mount time?  Seems
sane to me, just wanted to run it by everyone.

3. Change the per-fs option handling code to call the same code as the VFS'
nocacheflush option.  Any fs that wants to deprecate its per-fs option handler
can do so.  Or they can stay forever.

4. Remove all the flush conditionals from the fs code in favor of letting the
block layer handle it.

Hopefully "nocacheflush" is a little more obvious.

--D

Re: [PATCHSET] Refactor barrier=/nobarrier flags from fs to block layer

[Dave Chinner]

On Wed, Jan 26, 2011 at 09:24:13AM -0800, Darrick J. Wong wrote:
> On Wed, Jan 26, 2011 at 10:41:35AM -0600, Eric Sandeen wrote:
> > On 1/26/11 5:49 AM, Ric Wheeler wrote:
> > > On 01/26/2011 02:12 AM, Darrick J. Wong wrote:
> > >> Hello,
> > >>
> > >>  From what I can tell, most of the filesystems that know how to issue commands
> > >> to flush the write cache also have some mechanism for the user to override
> > >> whether or not the filesystem actually issues those flushes.  Unfortunately,
> > >> the term "barrier" is obsolete having been changed into flushes in 2.6.36, and
> > >> many of the filesystems implement the mount options with slightly different
> > >> syntaxes (barrier=[0|1|none|flush], nobarrier, etc).
> > >>
> > >> This patchset adds to the block layer a sysfs knob that an administrator can
> > >> use to disable flushes, and removes the mount options from the filesystem code.
> > >> As a starting point, I'm removing the mount options and flush toggle from
> > >> jbd2/ext4.
> > >>
> > >> Anyway, I'm looking for some feedback about refactoring the barrier/flush
> > >> control knob into the block layer.  It sounds like we want a knob that picks
> > >> the safest option (issue flushes when supported) unless the administrator
> > >> decides that it is appropriate to do otherwise.  I suspect that there are good
> > >> arguments for not having a knob at all, and good arguments for a safe knob.
> > >> However, since I don't see the barrier options being removed en masse, I'm
> > >> assuming that we still want a knob somewhere.  Do we need the ignore_fua knob
> > >> too?  Is this the proper way to deprecate mount options out of filesystems?
> > >>
> > >> --D
> > > 
> > > Just to be clear, I strongly object to remove the mount options.
> > 
> > Agreed, we are just finally, barely starting to win the education battle here.
> > Removing or changing the option now will just set us back.  It should at
> > LEAST remain as a deprecated option, with the deprecation message pointing
> > to crystal-clear documentation.
> 
> Ok, how about a second proposal:
> 
> 1. Put the sysfs knob and the toggle code in the block layer, similar to patch
> #1, only make it a per-bdev toggle so each mount can have its own override
> parameters.

A sysfs knob just seems wrong for this. What do you do with
filesystems or block devices that span multiple block devices,
either via md, dm, mount options (XFS - separate data, log and
realtime devices) or other means (btrfs w/ multiple devices)? 

IMO, the only sane way to control this sort of behaviour is from the
top down (i.e. from the filesystem) and not from the bottom up (i.e.
from the lowest level of block devices) because the cache flushes
are only useful to the filesystem if they are consistently
implemented from the top of the storage stack to the bottom...

Also, if you allow block devices at the bottom of the stack to be
configured to ignore flushes dynamically, we need some method to
inform the upper layers that this has happened. At minimum the
filesystem needs to log the fact that their crash/power fail
consistency guarantees have changed - there's no way I'm going to
assume that users won't do something stupid if there's a knob to
tweak....

> 2. Add some sort of "nocacheflush" option to the VFS layer to adjust the knob.
> With this we gain a consistent mount option syntax across all the filesystems,
> though what it means for a networked fs is questionable.  I guess you could
> reject the mount option if there's no block device under the fs.  Also, any fs
> that someday grows an issue-flush feature won't have to add its own option
> parsing code.

We already have a relatively widely implemented mount option pair -
barrier/nobarrier is supported by ext3, ext4, btrfs, gfs2, xfs,
hfsplus and nilfs2 - so I'd suggest that this is the best paaaaaaah
to take for implementing a generic mount option...

> At umount time, do we undo whatever overrides we set up at mount time?  Seems
> sane to me, just wanted to run it by everyone.

Does it really matter? The next mount will set it to whatever is
necessary...

> 3. Change the per-fs option handling code to call the same code as the VFS'
> nocacheflush option.  Any fs that wants to deprecate its per-fs option handler
> can do so.  Or they can stay forever.
> 
> 4. Remove all the flush conditionals from the fs code in favor of letting the
> block layer handle it.
>
> Hopefully "nocacheflush" is a little more obvious.

What cache does "nocacheflush" refer to? The page, inode, dentry, or
buffer caches? Or some other per filesystem cache? Perhaps the MD
stripe cache? Maybe something else? There are many different caches
in a storage system even before we consider hardware, so I think
"nocacheflush" is much more ambiguous than barrier/nobarrier...

Just my 2c worth....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com