Linux RAID subsystem development
 help / color / mirror / Atom feed
* [PATCH] md: wait for a single grace period when stopping an array
@ 2026-11-21 23:59 Qiliang Yuan
  0 siblings, 0 replies; only message in thread
From: Qiliang Yuan @ 2026-11-21 23:59 UTC (permalink / raw)
  To: Song Liu, Yu Kuai, Li Nan, Xiao Ni; +Cc: linux-raid, linux-kernel, Qiliang Yuan

When an array is stopped, export_array() takes the members off
mddev->disks one by one with md_kick_rdev_from_array(), which unlinks a
member with list_del_rcu() and calls synchronize_rcu() before handing it
to mddev_unlock() for kobject_del().

Each member thus waits for a full grace period of its own, so stopping
an array takes one grace period per member, about 6 ms each in a guest,
with reconfig_mutex held.

Release the serialization resources of all members first, while
mddev_destroy_serial_pool() still sees the members that need the pool,
then unlink all members at once with list_splice_init_rcu() and a single
synchronize_rcu(). Clear rdev->mddev and the sysfs references only after
the grace period, so a reader walking mddev->disks never finds a member
that has already been detached.

Stopping an array made of null_blk devices in a guest with 32 vCPUs,
base and patched kernels alternating over 2 rounds of 3 runs, median of
the 6 runs and range in parentheses:

  level, disks           before (ms)         after (ms)   change
  raid0, 16      132.4 (126.3-147.5)   37.7 (34.5-41.4)     -72%
  raid1, 8         88.5 (86.3-103.1)   43.7 (40.7-45.4)     -51%
  raid5, 8         98.8 (97.6-107.3)   57.7 (55.0-59.1)     -42%
  raid10, 8         87.4 (85.0-89.8)   41.7 (37.8-43.9)     -52%

Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
 drivers/md/md.c | 39 +++++++++++++++++++++++++++++----------
 1 file changed, 29 insertions(+), 10 deletions(-)

diff --git a/drivers/md/md.c b/drivers/md/md.c
index 03e35386a8028..11fa541bce32e 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -2676,14 +2676,9 @@ static void export_rdev(struct md_rdev *rdev)
 	kobject_put(&rdev->kobj);
 }
 
-static void md_kick_rdev_from_array(struct md_rdev *rdev)
+static void md_unbind_rdev(struct md_rdev *rdev)
 {
-	struct mddev *mddev = rdev->mddev;
-
-	bd_unlink_disk_holder(rdev->bdev, rdev->mddev->gendisk);
-	list_del_rcu(&rdev->same_set);
 	pr_debug("md: unbind<%pg>\n", rdev->bdev);
-	mddev_destroy_serial_pool(rdev->mddev, rdev);
 	WRITE_ONCE(rdev->mddev, NULL);
 	sysfs_remove_link(&rdev->kobj, "block");
 	sysfs_put(rdev->sysfs_state);
@@ -2693,6 +2688,16 @@ static void md_kick_rdev_from_array(struct md_rdev *rdev)
 	rdev->sysfs_unack_badblocks = NULL;
 	rdev->sysfs_badblocks = NULL;
 	rdev->badblocks.count = 0;
+}
+
+static void md_kick_rdev_from_array(struct md_rdev *rdev)
+{
+	struct mddev *mddev = rdev->mddev;
+
+	bd_unlink_disk_holder(rdev->bdev, mddev->gendisk);
+	list_del_rcu(&rdev->same_set);
+	mddev_destroy_serial_pool(mddev, rdev);
+	md_unbind_rdev(rdev);
 
 	synchronize_rcu();
 
@@ -2707,12 +2712,26 @@ static void md_kick_rdev_from_array(struct md_rdev *rdev)
 static void export_array(struct mddev *mddev)
 {
 	struct md_rdev *rdev;
+	LIST_HEAD(kicked);
 
-	while (!list_empty(&mddev->disks)) {
-		rdev = list_first_entry(&mddev->disks, struct md_rdev,
-					same_set);
-		md_kick_rdev_from_array(rdev);
+	/*
+	 * Release the serialization resources while every member is still on
+	 * the list, as mddev_destroy_serial_pool() counts the members that
+	 * still need the pool.
+	 */
+	rdev_for_each(rdev, mddev) {
+		bd_unlink_disk_holder(rdev->bdev, mddev->gendisk);
+		mddev_destroy_serial_pool(mddev, rdev);
 	}
+
+	/* Unlink all members at once to wait for a single grace period */
+	list_splice_init_rcu(&mddev->disks, &kicked, synchronize_rcu);
+
+	list_for_each_entry(rdev, &kicked, same_set)
+		md_unbind_rdev(rdev);
+
+	/* Leave kobject_del() to mddev_unlock() */
+	list_splice(&kicked, &mddev->deleting);
 	mddev->raid_disks = 0;
 	mddev->major_version = 0;
 }

---
base-commit: 5f22fb20497f4983094131f00a734362300d22d2
change-id: 20261120-feat-md-export-array-gp-1381456062ec

Best regards,
-- 
Qiliang Yuan <qiliang.yuan@linux.dev>


^ permalink raw reply related	[flat|nested] only message in thread

only message in thread, other threads:[~2026-11-21 23:59 UTC | newest]

Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-21 23:59 [PATCH] md: wait for a single grace period when stopping an array Qiliang Yuan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox