Re: [PATCH v4 1/3] btrfs: add read_policy latency

[Anand Jain <anand.jain@oracle.com>]

Hi Michal,

  Did you get any chance to run the evaluation with this patchset?

Thanks, Anand

On 1/30/2021 9:08 AM, Anand Jain wrote:
> 
>>> 500m is really small data size for such measurement
> 
> I reran the read policy tests with some changes in the fio command
> options. Mainly to measure IOPS throughput and latency on the filesystem
> with latency-policy and pid-policy.
> 
> Each of these tests was run for 3 iterations and the best and worst of
> those 3 iterations are shown below.
> 
> These workloads are performing read-write which is the most commonly
> used workload, on a single type of device (which is nvme here) and two
> devices are configured for RAID1.
> 
> In all these read-write workloads, pid-policy performed ~25% better than
> the latency-policy for both throughput and IOPS, and 3% better on the
> latency parameter.
> 
> I haven't analyzed these read-write workloads on RAID1c3/RAID1c4 yet,
> but RAID1 is more common than other types, IMO.
> 
> So I think pid-policy should remain as our default read policy.
> 
> However as shown before, pid-policy perform worst in the case of special
> configs such as volumes with mixed types of devices. For those special
> mixed types of devices, latency-policy performs better than pid-policy.
> As tested before typically latency-policy provided ~175% better
> throughput performance in the case of mixed types of devices (SSD and
> nvme).
> 
> Feedbacks welcome.
> 
> Fio logs below.
> 
> 
> IOPS focused readwrite workload:
> fio --filename=/btrfs/foo --size=500GB --direct=1 --rw=randrw --bs=4k 
> --ioengine=libaio --iodepth=256 --runtime=120 --numjobs=4 --time_based 
> --group_reporting --name=iops-randomreadwrite --eta-newline=1
> 
> pid [latency] device roundrobin ( 00)
>    read: IOPS=40.6k, BW=159MiB/s (166MB/s)(18.6GiB/120002msec)
> 
> [pid] latency device roundrobin ( 00)
>    read: IOPS=50.7k, BW=198MiB/s (208MB/s)(23.2GiB/120001msec)
> 
> IOPS is 25% better with pid policy.
> 
> 
> Throughput focused readwrite workload:
> fio --filename=/btrfs/foo --size=500GB --direct=1 --rw=randrw --bs=64k 
> --ioengine=libaio --iodepth=64 --runtime=120 --numjobs=4 --time_based 
> --group_reporting --name=throughput-randomreadwrite --eta-newline=1
> 
> pid [latency] device roundrobin ( 00)
>    read: IOPS=8525, BW=533MiB/s (559MB/s)(62.4GiB/120003msec)
> 
> [pid] latency device roundrobin ( 00)
>    read: IOPS=10.7k, BW=670MiB/s (702MB/s)(78.5GiB/120005msec)
> 
> Throughput is 25% better with pid policy
> 
> Latency focused readwrite workload:
> fio --filename=/btrfs/foo --size=500GB --direct=1 --rw=randrw --bs=4k 
> --ioengine=libaio --iodepth=1 --runtime=120 --numjobs=4 --time_based 
> --group_reporting --name=latency-randomreadwrite --eta-newline=1
> pid [latency] device roundrobin ( 00)
>    read: IOPS=59.8k, BW=234MiB/s (245MB/s)(27.4GiB/120003msec)
>       lat (usec): min=68, max=826930, avg=1917.20, stdev=4210.90
> 
> [pid] latency device roundrobin ( 00)
>    read: IOPS=61.9k, BW=242MiB/s (253MB/s)(28.3GiB/120001msec)
>       lat (usec): min=64, max=751557, avg=1846.07, stdev=4082.97
> 
> Latency is 3% better with pid policy.