Re: [RFC] md/raid5: optimize RAID5 performance.

[Paul E Luse <paul.e.luse@linux.intel.com>]

On Sun, 26 Nov 2023 22:44:45 -0800
Christoph Hellwig <hch@infradead.org> wrote:

> Hi Shushu,
> 
> the work certainly l-ooks interesting!
> 
> However:
> 
> > Optimized by using fine-grained locks, customized data structures,
> > and scattered address space. Achieves significant improvements in
> > both throughput and latency.
> 
> this is a lot of work for a single Linux patch, we usually do that
> work pice by pice instead of complete rewrite, and for such
> signigicant changes the commit logs also tend to be a bit extensive.
> 
> I'm also not quite sure what scattered address spaces are - I bet
> reading the paper (I plan to get to that) would explain it, but it
> also helps to explain the idea in the commit message.
> 
> That's my high level nitpicking for now, I'll try to read the paper
> and the patch in detail and come back later.
> 
> 

Hi Everyone,

I went ahead and ran a series of performance tests on this patch to
help the community understand the value.Here's a summary of what have
completed and am happy to run some more to keep the patch moving.

I have not yet reviewed the code as I wanted to make sure it provided
good benefit first and it does for sure. I will be reviewing shortly.
Here is a summary of my tests:

* Kioxia CM7 drives
  https://americas.kioxia.com/content/dam/kioxia/shared/business/ssd/enterprise-ssd/asset/productbrief/eSSD-CM7-V-product-brief.pdf
* Dual Socket Xeon 8368 2.4GHz 256G RAM
* Results are the average of just 2 60 second runs per data point, if
  interest continues I can re-run to eliminate any potential anomalies
* I used 8 fio jobs per disk and 2 group_thread_cnt per disk so when
  reading the graph, for example, 8DR5_patch_64j15gtc means an 8 Disk
  RAID5 run against the patch with 64 fio jobs and group-thread_cnt set
  to 16.  'base' in the name is md-next branch as of yesterday.
* Sample fio command: fio --filename=/dev/md0 --direct=1
  --output=/root/remote/8DR5_patch_64j16gtc_1/randrw_131072_1.json
  --rw=randrw --bs=131072 --ioengine=libaio --ramp_time=3 --runtime=60
  --iodepth=1 --numjobs=64 --time_based --group_reporting
  --name=131072_1_randrw --output-format=json --numa_cpu_nodes=0

Results: https://photos.app.goo.gl/Cip1rU3spbD8nvG28 

-Paul