From: Chaitanya Kulkarni <kch@nvidia.com>
To: <linux-block@vger.kernel.org>
Cc: <axboe@kernel.dk>, Chaitanya Kulkarni <kch@nvidia.com>
Subject: [PATCH 0/2] brd: small improvements
Date: Mon, 31 Jul 2023 19:29:27 -0700 [thread overview]
Message-ID: <20230801022929.8972-1-kch@nvidia.com> (raw)
Hi,
Small improvements for brd to remove deprecated APIs and add pr_fmt
macro to remove the repetative brd prefix in the pr_xxx() string.
I've ran few fio read/write/verify tests on this, they seems to pass.
-ck
Chaitanya Kulkarni (2):
brd: use memcpy_to|from_page() in copy_to|from_brd()
brd: remove redundent prefix with pr_fmt macro
drivers/block/brd.c | 37 +++++++++++++++----------------------
1 file changed, 15 insertions(+), 22 deletions(-)
linux-block # sh test-brd-memcpy-perf.sh
+ ./compile_brd.sh
+ umount /mnt/brd
umount: /mnt/brd: not mounted.
+ dmesg -c
+ modprobe -r brd
+ lsmod
+ grep brd
++ nproc
+ make -j 48 M=drivers/block modules
+ HOST=drivers/block/brd.ko
++ uname -r
+ HOST_DEST=/lib/modules/6.5.0-rc3lblk+/kernel/drivers/block/
+ cp drivers/block/brd.ko /lib/modules/6.5.0-rc3lblk+/kernel/drivers/block//
+ ls -lrth /lib/modules/6.5.0-rc3lblk+/kernel/drivers/block//brd.ko
-rw-r--r--. 1 root root 378K Jul 31 19:13 /lib/modules/6.5.0-rc3lblk+/kernel/drivers/block//brd.ko
+ dmesg -c
+ lsmod
+ grep brd
+ modprobe brd rd_nr=1 rd_size=20971520
+ sleep 1
+ dmesg -c
[12164.622572] brd: module loaded
+ test_perf
+ git log --oneline -2
3dbc8dd05802 (HEAD -> for-next) brd: remove redundent prefix with pr_fmt macro
5644af5fe8d1 brd: use memcpy_to|from_page() in copy_to|from_brd()
+ ./compile_brd.sh
+ umount /mnt/brd
umount: /mnt/brd: not mounted.
+ dmesg -c
+ modprobe -r brd
+ lsmod
+ grep brd
++ nproc
+ make -j 48 M=drivers/block modules
+ HOST=drivers/block/brd.ko
++ uname -r
+ HOST_DEST=/lib/modules/6.5.0-rc3lblk+/kernel/drivers/block/
+ cp drivers/block/brd.ko /lib/modules/6.5.0-rc3lblk+/kernel/drivers/block//
+ ls -lrth /lib/modules/6.5.0-rc3lblk+/kernel/drivers/block//brd.ko
-rw-r--r--. 1 root root 378K Jul 31 19:13 /lib/modules/6.5.0-rc3lblk+/kernel/drivers/block//brd.ko
+ dmesg -c
[12165.683470] brd: module unloaded
+ lsmod
+ grep brd
+ modprobe brd rd_nr=1 rd_size=20971520
+ sleep 1
+ fio fio/randwrite.fio --filename=/dev/ram0 --ioengine=libaio --size=20G
RANDWRITE: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=2
...
fio-3.34
Starting 48 processes
Jobs: 48 (f=48): [w(48)][100.0%][w=26.5GiB/s][w=6948k IOPS][eta 00m:00s]
RANDWRITE: (groupid=0, jobs=48): err= 0: pid=6654: Mon Jul 31 19:14:50 2023
write: IOPS=6580k, BW=25.1GiB/s (27.0GB/s)(1506GiB/60002msec); 0 zone resets
slat (nsec): min=992, max=48184k, avg=6291.87, stdev=44125.08
clat (nsec): min=741, max=48194k, avg=7820.21, stdev=48267.24
lat (usec): min=2, max=48215, avg=14.11, stdev=65.65
clat percentiles (usec):
| 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 6], 20.00th=[ 6],
| 30.00th=[ 6], 40.00th=[ 7], 50.00th=[ 7], 60.00th=[ 7],
| 70.00th=[ 8], 80.00th=[ 8], 90.00th=[ 9], 95.00th=[ 10],
| 99.00th=[ 26], 99.50th=[ 64], 99.90th=[ 137], 99.95th=[ 404],
| 99.99th=[ 594]
bw ( MiB/s): min= 4967, max=30851, per=100.00%, avg=25716.15, stdev=107.71, samples=5712
iops : min=1271674, max=7898038, avg=6583332.27, stdev=27573.66, samples=5712
lat (nsec) : 750=0.01%, 1000=0.01%
lat (usec) : 2=0.01%, 4=0.93%, 10=94.93%, 20=3.07%, 50=0.34%
lat (usec) : 100=0.58%, 250=0.07%, 500=0.07%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
cpu : usr=12.86%, sys=84.29%, ctx=26503, majf=0, minf=627
IO depths : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,394811720,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=2
Run status group 0 (all jobs):
WRITE: bw=25.1GiB/s (27.0GB/s), 25.1GiB/s-25.1GiB/s (27.0GB/s-27.0GB/s), io=1506GiB (1617GB), run=60002-60002msec
Disk stats (read/write):
ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
+ for i in 1 2 3
+ fio fio/randread.fio --filename=/dev/ram0 --ioengine=libaio --size=20G
RANDREAD: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=2
...
fio-3.34
Starting 48 processes
Jobs: 48 (f=48): [r(48)][100.0%][r=36.4GiB/s][r=9542k IOPS][eta 00m:00s]
RANDREAD: (groupid=0, jobs=48): err= 0: pid=6704: Mon Jul 31 19:15:50 2023
read: IOPS=9635k, BW=36.8GiB/s (39.5GB/s)(2205GiB/60002msec)
slat (nsec): min=1342, max=64089k, avg=4179.67, stdev=20589.24
clat (nsec): min=771, max=64095k, avg=5428.32, stdev=23628.88
lat (usec): min=2, max=64107, avg= 9.61, stdev=31.37
clat percentiles (nsec):
| 1.00th=[ 3440], 5.00th=[ 3728], 10.00th=[ 3920], 20.00th=[ 4192],
| 30.00th=[ 4384], 40.00th=[ 4640], 50.00th=[ 4896], 60.00th=[ 5216],
| 70.00th=[ 5600], 80.00th=[ 6048], 90.00th=[ 6752], 95.00th=[ 7456],
| 99.00th=[ 9536], 99.50th=[34048], 99.90th=[41216], 99.95th=[43264],
| 99.99th=[66048]
bw ( MiB/s): min=28759, max=39419, per=100.00%, avg=37689.01, stdev=32.63, samples=5712
iops : min=7362346, max=10091438, avg=9648387.52, stdev=8354.07, samples=5712
lat (nsec) : 1000=0.01%
lat (usec) : 2=0.01%, 4=12.70%, 10=86.45%, 20=0.17%, 50=0.66%
lat (usec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%
cpu : usr=14.07%, sys=84.64%, ctx=27355, majf=0, minf=732
IO depths : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=578147226,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=2
Run status group 0 (all jobs):
READ: bw=36.8GiB/s (39.5GB/s), 36.8GiB/s-36.8GiB/s (39.5GB/s-39.5GB/s), io=2205GiB (2368GB), run=60002-60002msec
Disk stats (read/write):
ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
+ for i in 1 2 3
+ fio fio/randread.fio --filename=/dev/ram0 --ioengine=libaio --size=20G
RANDREAD: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=2
...
fio-3.34
Starting 48 processes
Jobs: 48 (f=48): [r(48)][100.0%][r=36.5GiB/s][r=9568k IOPS][eta 00m:00s]
RANDREAD: (groupid=0, jobs=48): err= 0: pid=6772: Mon Jul 31 19:16:50 2023
read: IOPS=9544k, BW=36.4GiB/s (39.1GB/s)(2185GiB/60005msec)
slat (nsec): min=1292, max=52199k, avg=4216.56, stdev=30223.82
clat (nsec): min=761, max=52205k, avg=5481.37, stdev=34604.93
lat (usec): min=2, max=52218, avg= 9.70, stdev=46.06
clat percentiles (nsec):
| 1.00th=[ 3440], 5.00th=[ 3728], 10.00th=[ 3920], 20.00th=[ 4192],
| 30.00th=[ 4384], 40.00th=[ 4640], 50.00th=[ 4896], 60.00th=[ 5216],
| 70.00th=[ 5600], 80.00th=[ 6048], 90.00th=[ 6752], 95.00th=[ 7392],
| 99.00th=[ 9408], 99.50th=[35584], 99.90th=[41216], 99.95th=[42752],
| 99.99th=[80384]
bw ( MiB/s): min=25868, max=40243, per=100.00%, avg=37331.79, stdev=46.77, samples=5712
iops : min=6622274, max=10302360, avg=9556939.51, stdev=11972.54, samples=5712
lat (nsec) : 1000=0.01%
lat (usec) : 2=0.01%, 4=12.55%, 10=86.62%, 20=0.15%, 50=0.66%
lat (usec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%
cpu : usr=12.19%, sys=85.45%, ctx=25680, majf=0, minf=686
IO depths : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=572715634,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=2
Run status group 0 (all jobs):
READ: bw=36.4GiB/s (39.1GB/s), 36.4GiB/s-36.4GiB/s (39.1GB/s-39.1GB/s), io=2185GiB (2346GB), run=60005-60005msec
Disk stats (read/write):
ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
+ for i in 1 2 3
+ fio fio/randread.fio --filename=/dev/ram0 --ioengine=libaio --size=20G
RANDREAD: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=2
...
fio-3.34
Starting 48 processes
Jobs: 48 (f=48): [r(48)][100.0%][r=36.4GiB/s][r=9549k IOPS][eta 00m:00s]
RANDREAD: (groupid=0, jobs=48): err= 0: pid=6823: Mon Jul 31 19:17:51 2023
read: IOPS=9544k, BW=36.4GiB/s (39.1GB/s)(2185GiB/60003msec)
slat (nsec): min=1332, max=40081k, avg=4213.58, stdev=30581.29
clat (nsec): min=761, max=40087k, avg=5479.64, stdev=34518.58
lat (usec): min=2, max=40100, avg= 9.69, stdev=46.18
clat percentiles (nsec):
| 1.00th=[ 3472], 5.00th=[ 3760], 10.00th=[ 3952], 20.00th=[ 4192],
| 30.00th=[ 4384], 40.00th=[ 4640], 50.00th=[ 4896], 60.00th=[ 5216],
| 70.00th=[ 5600], 80.00th=[ 6048], 90.00th=[ 6752], 95.00th=[ 7392],
| 99.00th=[ 9408], 99.50th=[35584], 99.90th=[40704], 99.95th=[42240],
| 99.99th=[82432]
bw ( MiB/s): min=25167, max=40648, per=100.00%, avg=37327.34, stdev=47.56, samples=5712
iops : min=6442770, max=10406041, avg=9555798.60, stdev=12175.34, samples=5712
lat (nsec) : 1000=0.01%
lat (usec) : 2=0.01%, 4=12.35%, 10=86.82%, 20=0.16%, 50=0.65%
lat (usec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%
cpu : usr=12.36%, sys=85.26%, ctx=25612, majf=0, minf=774
IO depths : 1=0.1%, 2=100.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=572691650,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=2
Run status group 0 (all jobs):
READ: bw=36.4GiB/s (39.1GB/s), 36.4GiB/s-36.4GiB/s (39.1GB/s-39.1GB/s), io=2185GiB (2346GB), run=60003-60003msec
Disk stats (read/write):
ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
+ fio fio/verify.fio --filename=/dev/ram0 --ioengine=io_uring --size=10G
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=io_uring, iodepth=16
fio-3.34
Starting 1 process
Jobs: 1 (f=1): [V(1)][77.8%][r=1829MiB/s][r=468k IOPS][eta 00m:04s]
write-and-verify: (groupid=0, jobs=1): err= 0: pid=6876: Mon Jul 31 19:18:06 2023
read: IOPS=468k, BW=1827MiB/s (1916MB/s)(6472MiB/3542msec)
slat (nsec): min=1022, max=96523, avg=1313.44, stdev=369.23
clat (nsec): min=711, max=128955, avg=32147.10, stdev=2033.64
lat (nsec): min=1974, max=130207, avg=33460.54, stdev=2081.94
clat percentiles (nsec):
| 1.00th=[30592], 5.00th=[31104], 10.00th=[31104], 20.00th=[31360],
| 30.00th=[31360], 40.00th=[31616], 50.00th=[31616], 60.00th=[31872],
| 70.00th=[32128], 80.00th=[32128], 90.00th=[32640], 95.00th=[34560],
| 99.00th=[40192], 99.50th=[43264], 99.90th=[51456], 99.95th=[55552],
| 99.99th=[84480]
write: IOPS=237k, BW=926MiB/s (971MB/s)(10.0GiB/11061msec); 0 zone resets
slat (nsec): min=1954, max=71735, avg=3897.72, stdev=938.16
clat (nsec): min=430, max=136489, avg=63372.98, stdev=5430.72
lat (usec): min=4, max=142, avg=67.27, stdev= 5.70
clat percentiles (usec):
| 1.00th=[ 48], 5.00th=[ 55], 10.00th=[ 58], 20.00th=[ 61],
| 30.00th=[ 62], 40.00th=[ 63], 50.00th=[ 64], 60.00th=[ 65],
| 70.00th=[ 66], 80.00th=[ 67], 90.00th=[ 70], 95.00th=[ 73],
| 99.00th=[ 80], 99.50th=[ 83], 99.90th=[ 91], 99.95th=[ 99],
| 99.99th=[ 118]
bw ( KiB/s): min=93464, max=1144224, per=96.18%, avg=911805.22, stdev=187568.47, samples=23
iops : min=23366, max=286056, avg=227951.30, stdev=46892.12, samples=23
lat (nsec) : 500=0.01%, 750=0.01%
lat (usec) : 4=0.01%, 10=0.01%, 20=0.01%, 50=39.72%, 100=60.25%
lat (usec) : 250=0.03%
cpu : usr=55.59%, sys=44.35%, ctx=20, majf=0, minf=45314
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1656716,2621440,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=16
Run status group 0 (all jobs):
READ: bw=1827MiB/s (1916MB/s), 1827MiB/s-1827MiB/s (1916MB/s-1916MB/s), io=6472MiB (6786MB), run=3542-3542msec
WRITE: bw=926MiB/s (971MB/s), 926MiB/s-926MiB/s (971MB/s-971MB/s), io=10.0GiB (10.7GB), run=11061-11061msec
Disk stats (read/write):
ram0: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
+ modprobe -r brd
+ modprobe -r brd
+ set +x
--
2.40.0
next reply other threads:[~2023-08-01 2:30 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-01 2:29 Chaitanya Kulkarni [this message]
2023-08-01 2:29 ` [PATCH 1/2] brd: use memcpy_to|from_page() in copy_to|from_brd() Chaitanya Kulkarni
2023-08-01 2:29 ` [PATCH 2/2] brd: remove redundent prefix with pr_fmt macro Chaitanya Kulkarni
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230801022929.8972-1-kch@nvidia.com \
--to=kch@nvidia.com \
--cc=axboe@kernel.dk \
--cc=linux-block@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.