Re: reduce memory allocation in the btrfs direct I/O path v2

[Anand Jain <anand.jain@oracle.com>]

On 5/6/22 01:41, Christoph Hellwig wrote:
> Hi all,
> 
> this series adds two minor improvements to iomap that allow btrfs
> to avoid a memory allocation per read/write system call and another
> one per submitted bio.  I also have at last two other pending uses
> for the iomap functionality later on, so they are not really btrfs
> specific either.
> 
> Changes since v1:
>   - pass the private data direct to iomap_dio_rw instead of through the
>     iocb
>   - better document the bio_set in iomap_dio_ops
>   - split a patch into three
>   - use kcalloc to allocate the checksums
> 
> Diffstat:
>   fs/btrfs/btrfs_inode.h |   25 --------
>   fs/btrfs/ctree.h       |    6 -
>   fs/btrfs/file.c        |    6 -
>   fs/btrfs/inode.c       |  152 +++++++++++++++++++++++--------------------------
>   fs/erofs/data.c        |    2
>   fs/ext4/file.c         |    4 -
>   fs/f2fs/file.c         |    4 -
>   fs/gfs2/file.c         |    4 -
>   fs/iomap/direct-io.c   |   26 ++++++--
>   fs/xfs/xfs_file.c      |    6 -
>   fs/zonefs/super.c      |    4 -
>   include/linux/iomap.h  |   16 ++++-
>   12 files changed, 123 insertions(+), 132 deletions(-)

This patch got me curious a couple of days back while I was tracing
a dio read performance issue on nvme. I am sharing the results as below.
[1]. There is no performance difference. Thx.


[1]
Before:
  4971MB/s 4474GB iocounts: nvme3n1 545220968 nvme0n1 547007640 
single_d2/5.18.0-rc5+_misc-next_1

After:
  4968MB/s 4471GB iocounts: nvme3n1 544207371 nvme1n1 547458037 
single_d2/5.18.0-rc5+_dio_cleanup_hch_1

  readstat /btrfs fio --eta=auto --output=$CMDLOG \
--name fiotest --directory=/btrfs --rw=randread \
--bs=4k --size=4G --ioengine=libaio --iodepth=16 --direct=1 \
--time_based=1 --runtime=900 --randrepeat=1 --gtod_reduce=1 \
--group_reporting=1 --numjobs=64