From: Qiliang Yuan <qiliang.yuan@linux.dev>
To: Song Liu <song@kernel.org>, Yu Kuai <yukuai@fygo.io>,
Li Nan <magiclinan@didiglobal.com>, Xiao Ni <xiao@kernel.org>
Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org,
Qiliang Yuan <qiliang.yuan@linux.dev>
Subject: [PATCH] md: wait for a single grace period when stopping an array
Date: Sat, 21 Nov 2026 18:59:13 -0500 [thread overview]
Message-ID: <20261121-feat-md-export-array-gp-v1-1-b1925100be19@linux.dev> (raw)
When an array is stopped, export_array() takes the members off
mddev->disks one by one with md_kick_rdev_from_array(), which unlinks a
member with list_del_rcu() and calls synchronize_rcu() before handing it
to mddev_unlock() for kobject_del().
Each member thus waits for a full grace period of its own, so stopping
an array takes one grace period per member, about 6 ms each in a guest,
with reconfig_mutex held.
Release the serialization resources of all members first, while
mddev_destroy_serial_pool() still sees the members that need the pool,
then unlink all members at once with list_splice_init_rcu() and a single
synchronize_rcu(). Clear rdev->mddev and the sysfs references only after
the grace period, so a reader walking mddev->disks never finds a member
that has already been detached.
Stopping an array made of null_blk devices in a guest with 32 vCPUs,
base and patched kernels alternating over 2 rounds of 3 runs, median of
the 6 runs and range in parentheses:
level, disks before (ms) after (ms) change
raid0, 16 132.4 (126.3-147.5) 37.7 (34.5-41.4) -72%
raid1, 8 88.5 (86.3-103.1) 43.7 (40.7-45.4) -51%
raid5, 8 98.8 (97.6-107.3) 57.7 (55.0-59.1) -42%
raid10, 8 87.4 (85.0-89.8) 41.7 (37.8-43.9) -52%
Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
drivers/md/md.c | 39 +++++++++++++++++++++++++++++----------
1 file changed, 29 insertions(+), 10 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 03e35386a8028..11fa541bce32e 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -2676,14 +2676,9 @@ static void export_rdev(struct md_rdev *rdev)
kobject_put(&rdev->kobj);
}
-static void md_kick_rdev_from_array(struct md_rdev *rdev)
+static void md_unbind_rdev(struct md_rdev *rdev)
{
- struct mddev *mddev = rdev->mddev;
-
- bd_unlink_disk_holder(rdev->bdev, rdev->mddev->gendisk);
- list_del_rcu(&rdev->same_set);
pr_debug("md: unbind<%pg>\n", rdev->bdev);
- mddev_destroy_serial_pool(rdev->mddev, rdev);
WRITE_ONCE(rdev->mddev, NULL);
sysfs_remove_link(&rdev->kobj, "block");
sysfs_put(rdev->sysfs_state);
@@ -2693,6 +2688,16 @@ static void md_kick_rdev_from_array(struct md_rdev *rdev)
rdev->sysfs_unack_badblocks = NULL;
rdev->sysfs_badblocks = NULL;
rdev->badblocks.count = 0;
+}
+
+static void md_kick_rdev_from_array(struct md_rdev *rdev)
+{
+ struct mddev *mddev = rdev->mddev;
+
+ bd_unlink_disk_holder(rdev->bdev, mddev->gendisk);
+ list_del_rcu(&rdev->same_set);
+ mddev_destroy_serial_pool(mddev, rdev);
+ md_unbind_rdev(rdev);
synchronize_rcu();
@@ -2707,12 +2712,26 @@ static void md_kick_rdev_from_array(struct md_rdev *rdev)
static void export_array(struct mddev *mddev)
{
struct md_rdev *rdev;
+ LIST_HEAD(kicked);
- while (!list_empty(&mddev->disks)) {
- rdev = list_first_entry(&mddev->disks, struct md_rdev,
- same_set);
- md_kick_rdev_from_array(rdev);
+ /*
+ * Release the serialization resources while every member is still on
+ * the list, as mddev_destroy_serial_pool() counts the members that
+ * still need the pool.
+ */
+ rdev_for_each(rdev, mddev) {
+ bd_unlink_disk_holder(rdev->bdev, mddev->gendisk);
+ mddev_destroy_serial_pool(mddev, rdev);
}
+
+ /* Unlink all members at once to wait for a single grace period */
+ list_splice_init_rcu(&mddev->disks, &kicked, synchronize_rcu);
+
+ list_for_each_entry(rdev, &kicked, same_set)
+ md_unbind_rdev(rdev);
+
+ /* Leave kobject_del() to mddev_unlock() */
+ list_splice(&kicked, &mddev->deleting);
mddev->raid_disks = 0;
mddev->major_version = 0;
}
---
base-commit: 5f22fb20497f4983094131f00a734362300d22d2
change-id: 20261120-feat-md-export-array-gp-1381456062ec
Best regards,
--
Qiliang Yuan <qiliang.yuan@linux.dev>
reply other threads:[~2026-11-21 23:59 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261121-feat-md-export-array-gp-v1-1-b1925100be19@linux.dev \
--to=qiliang.yuan@linux.dev \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-raid@vger.kernel.org \
--cc=magiclinan@didiglobal.com \
--cc=song@kernel.org \
--cc=xiao@kernel.org \
--cc=yukuai@fygo.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox