Re: btrfs write-bandwidth performance regression of 6.5-rc4/rc3

[Wang Yugui <wangyugui@e16-tech.com>]

Hi,


> On Wed, Aug 02, 2023 at 08:04:57AM +0800, Wang Yugui wrote:
> > > And with only a revert of
> > > 
> > > "btrfs: submit IO synchronously for fast checksum implementations"?
> > 
> > GOOD performance when only (Revert "btrfs: submit IO synchronously for fast
> > checksum implementations") 
> 
> Ok, so you have a case where the offload for the checksumming generation
> actually helps (by a lot).  Adding Chris to the Cc list as he was
> involved with this.
> 
> > > > -       if (test_bit(BTRFS_FS_CSUM_IMPL_FAST, &bbio->fs_info->flags))
> > > > +       if ((bbio->bio.bi_opf & REQ_META) && test_bit(BTRFS_FS_CSUM_IMPL_FAST, &bbio->fs_info->flags))
> > > >                 return false;
> > > 
> > > This disables synchronous checksum calculation entirely for data I/O.
> > 
> > without this fix, data I/O checksum is always synchronous?
> > this is a feature change of "btrfs: submit IO synchronously for fast checksum implementations"?
> 
> It is never with the above patch.
> 
> > 
> > > Also I'm curious if you see any differents for a non-RAID0 (i.e.
> > > single profile) workload.
> > 
> > '-m single -d single' is about 10% slow that '-m raid1 -d raid0' in this test
> > case.
> 
> How does it compare with and without the revert?  Can you add the numbers?

the patch from wangyugui
diff --git a/fs/btrfs/bio.c b/fs/btrfs/bio.c
index 12b12443efaa..8ef968f0957d 100644
--- a/fs/btrfs/bio.c
+++ b/fs/btrfs/bio.c
@@ -598,7 +598,7 @@ static void run_one_async_free(struct btrfs_work *work)
 static bool should_async_write(struct btrfs_bio *bbio)
 {
 	/* Submit synchronously if the checksum implementation is fast. */
-	if (test_bit(BTRFS_FS_CSUM_IMPL_FAST, &bbio->fs_info->flags))
+	if ((bbio->bio.bi_opf & REQ_META) && test_bit(BTRFS_FS_CSUM_IMPL_FAST, &bbio->fs_info->flags))
 		return false;
 
 	/*

btrfs device: NVMe SSD PCIe3  x4

fio(-numjobs=4 ) on linux 6.5-rc5 with the patch, btrfs '-m raid1 -d raid0' 
  WRITE: bw=3858MiB/s (4045MB/s)
  WRITE: bw=3781MiB/s (3965MB/s)

fio(-numjobs=4 ) on linux 6.5-rc5 with the patch, btrfs '-m single -d single' 
  WRITE: bw=3004MiB/s (3149MB/s),
  WRITE: bw=2851MiB/s (2990MB/s)


fio(-numjobs=4 ) on linux 6.5-rc5 without the patch, btrfs '-m raid1 -d raid0' 
  WRITE: bw=1311MiB/s (1375MB/s)
  WRITE: bw=1435MiB/s (1504MB/s)

fio(-numjobs=4 ) on linux 6.5-rc5 without the patch, btrfs '-m single -d single' 
  WRITE: bw=1337MiB/s (1402MB/s)
  WRITE: bw=1413MiB/s (1481MB/s),


Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2023/08/11