From: pg@lxra2.for.sabi.co.UK (Peter Grandi)
To: Linux RAID <linux-raid@vger.kernel.org>
Subject: Re: strange mirror write perf
Date: Mon, 20 Mar 2017 00:53:50 +0000 [thread overview]
Message-ID: <22735.10270.726966.698144@tree.ty.sabi.co.uk> (raw)
In-Reply-To: <c0981a7c8b13facace1228ed0f99efdc@grim-is-a-geek.fr>
[ ... ]
> abyss ~ # dd bs=1M count=2024 if=/dev/zero of=/dev/sdd conv=fdatasync
> 2024+0 records in
> 2024+0 records out
> 2122317824 bytes (2,1 GB, 2,0 GiB) copied, 14,1238 s, 150 MB/s
> abyss ~ # dd bs=1M count=2024 if=/dev/zero of=/dev/sdc conv=fdatasync
> 2024+0 records in
> 2024+0 records out
> 2122317824 bytes (2,1 GB, 2,0 GiB) copied, 14,7046 s, 144 MB/s
[ ... ]
> mdadm: automatically enabling write-intent bitmap on large array
[ ... ]
> abyss ~ # dd bs=1M count=2024 if=/dev/zero of=/dev/md/md6to conv=fdatasync
> 2024+0 records in
> 2024+0 records out
> 2122317824 bytes (2,1 GB, 2,0 GiB) copied, 21,1528 s, 100 MB/s
[ ... ]
This is not about "performance" but raw speed in a very
particular situation. The difference is not large and could be
meaningless for a variety of reasons:
* Had the initial 6TB sync finished (would take 15-17 hours)?
* Internal bitmaps can steal a fair bit of speed.
* For whatever weirdness is related to the appalling
implementation of the Linux page cache, IO reading (but less
so writing) an MD block device is often slower than to a file
on a filetree on that block device, plus writing via the page
cache costs a lot of CPU time.
Replication of the experiment on similar smaller disks:
soft# mdadm --create --verbose /dev/md/md6to --level=1 --raid-devices=2 /dev/sde3 /dev/sdf3
mdadm: size set to 243269440K
mdadm: array /dev/md/md6to started.
soft# grep -A2 'sd[ef]3' /proc/mdstat
md127 : active raid1 sdf3[1] sde3[0]
243269440 blocks super 1.0 [2/2] [UU]
[====>................] resync = 20.5% (49954880/243269440) finish=19.5min speed=164445K/sec
soft# grep -A2 'sd[ef]3' /proc/mdstat; dd bs=1M count=2024 if=/dev/zero of=/dev
/md/md6to conv=fdatasync
md127 : active raid1 sdf3[1] sde3[0]
243269440 blocks super 1.0 [2/2] [UU]
2024+0 records in
2024+0 records out
2122317824 bytes (2.1 GB) copied, 13.8561 s, 153 MB/s
prev parent reply other threads:[~2017-03-20 0:53 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <eda41097b3c3a385f83041f0df372912@grim-is-a-geek.fr>
[not found] ` <6ec5f2dbefa2b7489d9f39d4833d5113@grim-is-a-geek.fr>
2017-03-19 15:14 ` strange mirror write perf Youenn Gestin
2017-03-19 23:42 ` Adam Goryachev
2017-03-20 0:44 ` Youenn Gestin
2017-03-20 5:07 ` Roman Mamedov
2017-03-20 0:53 ` Peter Grandi [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=22735.10270.726966.698144@tree.ty.sabi.co.uk \
--to=pg@lxra2.for.sabi.co.uk \
--cc=linux-raid@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox