Re: Really bad I/O lock contention between non-overlapping reader and writer

[Dave Chinner <dgc@kernel.org>]

On Fri, Oct 16, 2026 at 12:06:54PM +1100, Dave Chinner wrote:
> On Wed, Oct 14, 2026 at 04:10:30PM -0700, Vitali Lovich wrote:
> > Ok. I have a patch series I'm happy to share. It has no meaningful
> > performance degradation / scaling problems and largely fixes the
> > head-of-line blocking problem without the challenges of range
> > tracking. Before I post it, some stylistic questions for you:
> > 
> > * Do you prefer it as a new RFC e-mail chain or continue here
> 
> New patch series.
> 
> > * Are there specific benchmarks / workloads you'd like me to gather numbers?
> 
> FIO exercising random rw 4kB DIO at differing levels of concurrency
> is the minimum.
> 
> > * Is a VM with RAM XFS filesystem sufficient or is raw metal w/ an
> > NVMe drive required?
> 
> I'd typically test this on a machine with 32+ CPU cores and an IO
> subsystem that can sustain >2 million random 4kB writes
> indefinitely. I'd then use fio using pre-written files to hit it
> with increasing IO concurrency to the same file with either io_uring
> or AIO with direct IO to exercise shared access scalability.

Oh, I should have mentioned: RAM disks are -slow- compared to a
single high performance NVMe SSD, let alone a machine with a couple
of dozen of them.

THe problem is that RAM disks always do synchronous IO and they
consume CPU time copying the data to/from the RAM disk to/from user
buffers. Hence you can only have one IO per CPU in flight at a time,
instead of a few thousand IOs in flight per CPU when using
asynchronous IO on spread across many NVMe SSDs.

IOWs, never use ram disks for performance testing XFS filesystems -
the results never reflect real world performance on actual storage
hardware.

-Dave.

-- 
Dave Chinner
dgc@kernel.org