Re: fsync_mode mount option for ext4

["Theodore Ts'o" <tytso@mit.edu>]

On Wed, May 29, 2019 at 09:37:58AM +0530, Sahitya Tummala wrote:
> 
> Here is what I think on these mount options. Please correct me if my
> understanding is wrong.
> 
> The nobarrier mount option poses risk even if there is a battery
> protection against sudden power down, as it doesn't guarantee the ordering
> of important data such as journal writes on the disk. On the storage
> devices with internal cache, if the cache flush policy is out-of-order,
> then the places where FS is trying to enforce barriers will be at risk,
> causing FS to be inconsistent.

If you have protection against sudden shutdown, then nobarrier is
perfectly safe --- which is to say, if it is guaranteed that any
writes sent to device will be persisted after a crash, then nobarrier
is perfectly safe.  So for example, if you are using ext4 connected to
a million dollar EMC Storage Array, which has battery backup, using
nobarrier is perfectly safe.

That's because we still send writes to the device in an appropriate
order in nobarrier mode --- in particular, we send the journal updates
to the device in order.  The cache flush policy on the HDD is
out-of-order, but so long as they all make it out to persistant store
in the end, it'll be fine.

> But whereas with fsync_mode=nobarrier, FS is not trying to enforce
> any ordering of data on the disk except to ensure the data is flushed
> from the internal cache to non-volatile memory. Thus, I see this
> fsync_mode=nobarrier is much better than a general nobarrier. And it
> provides better performance too as with nobarrier but without
> compromising much on FS consistency.

"without compomising much on FS consistency" doesn't have any meaning.
If you care about FS consistency, and you don't have power fail
protection, then at least for ext4, you *must* send a CACHE FLUSH
after any time that you modify any file system metadata block --- and
that's true for 99% of all fsync(2)'s.

I suppose you could do something where if there are times when no
metadata updates are necessary, but just data block writes, the CACHE
FLUSH could be suppressed.  But (a) this won't actually provide much
performance improvements for the vast majority of workloads,
especially on an Android system, and (b) you're making a value
judgement that FS consistency is more important than application data
consistency.


You didn't answer my question directly --- exactly what is your goal
that you are trying to achieve, and what assumptions you are willing
to make?  If you have power fail protection (this might require making
some adjustments to the EC), then you can use nobarrier and just not
worry about it.

If you don't have power fail protection, and you care about FS
consistency, then you pretty much have to leave the CACHE FLUSH
commands in.

If the problem is that some applications are fsync-happy, then I'd
suggest fixing the applications.  Or if you really don't care about
the applications working correctly or users suffering application data
loss after a crash, you could hack in a mode, so that for non-root
users, or maybe certain specific users, fsync is turned into a no-op,
or a background, asynchronous (non-integrity) writeback.

Are you trying to hit some benchmark target?  I'm really confused why
you would want to be so cavalier with application data safety.

    	       	     		      - Ted