exfat: slow write performance

exfat: slow write performance

[Vincent]

I've found that write performance of exFAT is a lot lower than other
filesystems on our setup with a fast NVMe SSD:

- Kernel: 6.10.0-364.vanilla.fc40.x86_64
- Test cmd: fio -name=fioWr -ioengine=libaio -directory=<mount point>
-blocksize=1M -readwrite=write -filesize=10G -end_fsync=1 -numjobs=8
-direct=1 -group_reporting
- Benchmarks:
    - exFAT: Direct I/O: ~1370 MiB/s (Buffered: ~1250 MiB/s)
(mkfs.exfat -c 1M -b 1M <device>)
    - ext4: Direct I/O: ~2230 MiB/s (Buffered: ~2150 MiB/s)
    - xfs: Direct I/O: ~2220 MiB/s (Buffered: ~2200 MiB/s)

I found that direct I/O is disabled for most of these writes in exFAT,
code responsible is "exfat_direct_IO()" in "fs/exfat/inode.c". As the
written file is being expanded it is falling back to normal buffered
write. The relevant code also has the following FIXME comment
(inherited from the fat driver):

/*
* FIXME: blockdev_direct_IO() doesn't use ->write_begin(),
* so we need to update the ->i_size_aligned to block boundary.
*
* But we must fill the remaining area or hole by nul for
* updating ->i_size_aligned
*
* Return 0, and fallback to normal buffered write.
*/

I have tried working around this problem by preallocating the file. I
see good write speeds in the direct I/O case (matching ext4 ~2200
MiB/s, Buffered: ~1700 MiB/s), the problem is that preallocation is
slow since native fallocate is not supported.

What would the maintainers recommend as the best course of action for
us here? I have noticed [1] that FALLOC_FL_KEEP_SIZE was implemented
in fat, so perhaps that would be simple to work on first?

Does anyone with more experience in the exFAT driver know the full
reasons behind the direct I/O disabling, and what it would take to
support direct I/O during file extension? Perhaps recent changes may
have made fixing this simpler?

Thanks.

1: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/fs/fat?id=b13bb33eacb7266d66a3adf03adaa0886d091789

Re: exfat: slow write performance

[Namjae Jeon]

2024년 7월 29일 (월) 오후 11:47, Vincent <laivcf@gmail.com>님이 작성:
>
> I've found that write performance of exFAT is a lot lower than other
> filesystems on our setup with a fast NVMe SSD:
>
> - Kernel: 6.10.0-364.vanilla.fc40.x86_64
> - Test cmd: fio -name=fioWr -ioengine=libaio -directory=<mount point>
> -blocksize=1M -readwrite=write -filesize=10G -end_fsync=1 -numjobs=8
> -direct=1 -group_reporting
> - Benchmarks:
>     - exFAT: Direct I/O: ~1370 MiB/s (Buffered: ~1250 MiB/s)
> (mkfs.exfat -c 1M -b 1M <device>)
>     - ext4: Direct I/O: ~2230 MiB/s (Buffered: ~2150 MiB/s)
>     - xfs: Direct I/O: ~2220 MiB/s (Buffered: ~2200 MiB/s)
>
> I found that direct I/O is disabled for most of these writes in exFAT,
> code responsible is "exfat_direct_IO()" in "fs/exfat/inode.c". As the
> written file is being expanded it is falling back to normal buffered
> write. The relevant code also has the following FIXME comment
> (inherited from the fat driver):
>
> /*
> * FIXME: blockdev_direct_IO() doesn't use ->write_begin(),
> * so we need to update the ->i_size_aligned to block boundary.
> *
> * But we must fill the remaining area or hole by nul for
> * updating ->i_size_aligned
> *
> * Return 0, and fallback to normal buffered write.
> */
>
> I have tried working around this problem by preallocating the file. I
> see good write speeds in the direct I/O case (matching ext4 ~2200
> MiB/s, Buffered: ~1700 MiB/s), the problem is that preallocation is
> slow since native fallocate is not supported.
>
> What would the maintainers recommend as the best course of action for
> us here? I have noticed [1] that FALLOC_FL_KEEP_SIZE was implemented
> in fat, so perhaps that would be simple to work on first?
It is difficult to implement due to exfat on-disk layout constraints.
>
> Does anyone with more experience in the exFAT driver know the full
> reasons behind the direct I/O disabling, and what it would take to
> support direct I/O during file extension? Perhaps recent changes may
> have made fixing this simpler?
I suggest you try direct IO write after extending the file size you
want with ftruncate.
>
> Thanks.
>
> 1: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/fs/fat?id=b13bb33eacb7266d66a3adf03adaa0886d091789
>