Re: Enable asynchronous commits by default patch revoked?

[Ric Wheeler <rwheeler@redhat.com>]

On 08/25/2009 01:52 PM, Andreas Dilger wrote:
> On Aug 24, 2009  20:15 -0400, Theodore Ts'o wrote:
>> On Mon, Aug 24, 2009 at 05:43:36PM -0600, Andreas Dilger wrote:
>>> Without transaction checksums waiting on all of the blocks together
>>> is NOT safe.  If the commit record is on disk, but the rest of the
>>> transaction's blocks are not then during replay it may cause garbage
>>> to be written from the journal into the filesystem metadata.
>>
>> That's the one optimization we using journal checksums buys us.
>> Unfortunately it does not allow us to omit the barrier
>> operation.... and have real-world testing experience that without the
>> barrier, a power drop can cause significant filesystem corruption and
>> potential data loss.
>>
>> Try using Chris Mason's torture-test workload with async-checksums
>> without this patch; you will get data corruption if you try dropping
>> power while his torture-test is running.  I know you really don't like
>> the barrier, but I'm afraid it's not safe to run without it, even with
>> journal checksums.
>
> In our performance testing of barriers (not with Chris' program), it
> was FAR better to disable the disk cache and wait for IO completion
> (i.e. barriers disabled) on just the journal blocks than to enable the
> cache and cause a cache flush for each "barrier".  The problem is that at
> high IO rates there is much more data in the cache vs. the actual journal
> blocks, and forcing the whole cache to be flushed each transaction commit
> hurt our performance noticably.
>
> Cheers, Andreas


Just for completeness, I ran a quick test on ext3 which was marginally better 
with barriers and xfs which was much better with barriers...

EXT3:

[root@ricdesktop ~]# mkfs.ext3 /dev/sdb
<snip>
[root@ricdesktop ~]# hdparm -W0 /dev/sdb

/dev/sdb:
  setting drive write-caching to 0 (off)
  write-caching =  0 (off)
[root@ricdesktop ~]# mount -o barrier=0 /dev/sdb /mnt/
[root@ricdesktop ~]# rm -f /mnt/bigfile
[root@ricdesktop ~]# dd if=/dev/zero of=/mnt/bigfile bs=10M count=100
100+0 records in
100+0 records out
1048576000 bytes (1.0 GB) copied, 9.26707 s, 113 MB/s
[root@ricdesktop ~]# umount /mnt
[root@ricdesktop ~]# hdparm -W1 /dev/sdb

/dev/sdb:
  setting drive write-caching to 1 (on)
  write-caching =  1 (on)
[root@ricdesktop ~]# mount -o barrier=1 /dev/sdb /mnt/
[root@ricdesktop ~]# rm -f /mnt/bigfile
[root@ricdesktop ~]# dd if=/dev/zero of=/mnt/bigfile bs=10M count=100
100+0 records in
100+0 records out
1048576000 bytes (1.0 GB) copied, 8.90897 s, 118 MB/s
[root@ricdesktop ~]# umount /mnt

XFS:

[root@ricdesktop ~]# umount /mnt
[root@ricdesktop ~]# mkfs.xfs -f /dev/sdb
<snip>
[root@ricdesktop ~]# hdparm -W0 /dev/sdb

/dev/sdb:
  setting drive write-caching to 0 (off)
  write-caching =  0 (off)
[root@ricdesktop ~]# mount -o nobarrier /dev/sdb /mnt
[root@ricdesktop ~]# rm -f /mnt/bigfile
[root@ricdesktop ~]# dd if=/dev/zero of=/mnt/bigfile bs=10M count=100
100+0 records in
100+0 records out
1048576000 bytes (1.0 GB) copied, 4.04406 s, 259 MB/s
[root@ricdesktop ~]# umount /mnt
[root@ricdesktop ~]# hdparm -W1 /dev/sdb

/dev/sdb:
  setting drive write-caching to 1 (on)
  write-caching =  1 (on)
[root@ricdesktop ~]# mount -o barrier /dev/sdb /mnt
[root@ricdesktop ~]# rm -f /mnt/bigfile
[root@ricdesktop ~]# dd if=/dev/zero of=/mnt/bigfile bs=10M count=100
100+0 records in
100+0 records out
1048576000 bytes (1.0 GB) copied, 3.03633 s, 345 MB/s