From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F86E606A331 for ; Thu, 19 Nov 2026 11:07:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=nXINq/Ee; dkim=pass (1024-bit key, unprotected) header.d=linux.dev header.i=@linux.dev header.a=rsa-sha256 header.s=key1 header.b=KWMD6u6e DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=yMR9n4XMip3+AkiagmhEADJx6wMLDNfeDj5uYbdfVxY=; b=nXINq/Eee5ugyQ fbKlatyx3zJJVeYyCvy95ViBMFnIBi2i620/+x0Zno3zU0o08FtJmKoSa/c1kHLJZXbRK0Mx96yRm zlNY9zw0SwzjYoybDP1cSEvd51p3xF7C8pONRyqvMgacg8bODQ4X0LwZ9uxiJLpowhsvhN5DOex0C k4WaPAYuZSuTt14ltGy0hk1VpSIVmBl1tjOgJvP8SQtp7V0qmlSVrVCJOu0bGVp0Nv95LJXlZacq4 uLyCJIfR4gQLAgACnXx7NufLRdFCUavCdebbmAUakxT0kGBpGcTePXP+11oJsxXD0oVqORkC3erDd 6UVzyiVldegJ+yh+2DzQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.100.1 #2 (Red Hat Linux)) id 1xTzz7-0000000BfD7-17VR; Thu, 19 Nov 2026 11:07:21 +0000 Received: from out-227.mta1.migadu.com ([95.215.58.227] helo=mta1.migadu.com) by bombadil.infradead.org with esmtps (Exim 4.100.1 #2 (Red Hat Linux)) id 1xTzz4-0000000BfBR-2WTU for linux-nvme@lists.infradead.org; Thu, 19 Nov 2026 11:07:20 +0000 X-Envelope-To: linux-nvme@lists.infradead.org DKIM-Signature: a=rsa-sha256; bh=GU/oTQIMoj85YJlo1iReKPWz2s1WS2YXPY5CYYSNvRs=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1795086436; v=1; x=1795691236; b=KWMD6u6eOeDrdo06zOMpFewXbAvviT+JvOgrYJnVyQQZzcxEOSSUa1MKLIc4YzAMFzTKrVnM 0x9fVMe2TnekPgGMwqlvAjbbkEHPoZ08qBw6bwOeya+eRsmpZVEfqfHkj+0KuBjWKqN2JjOjNYj yhvrmQ68oQf2s4OKBbLlagew= X-Envelope-To: linux-nvme@lists.infradead.org Received: by smtp.migadu.com with ESMTPS id 859c1671daba923c; Thu, 19 Nov 2026 11:07:16 +0000 X-Mizu-Trace-ID: 859c1671daba923c X-Migadu-Flow: FLOW_OUT From: Qiliang Yuan Date: Thu, 19 Nov 2026 06:07:01 -0500 Subject: [PATCH v5 2/2] nvme-multipath: don't wait for a grace period in nvme_mpath_set_live() X-Archived-By: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20261119-bug-nvme-mpath-set-live-srcu-v5-2-e1483e2e447f@linux.dev> References: <20261119-bug-nvme-mpath-set-live-srcu-v5-0-e1483e2e447f@linux.dev> In-Reply-To: <20261119-bug-nvme-mpath-set-live-srcu-v5-0-e1483e2e447f@linux.dev> To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Nilay Shroff , Hannes Reinecke Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Qiliang Yuan , Hannes Reinecke X-Mailer: b4 0.13.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20261119_030718_797174_9CA684FB X-CRM114-Status: GOOD ( 14.38 ) X-Spam-Score: -2.8 (--) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-nvme" X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on aws-us-west-2-korg-listserv-alma10-1 When a path goes live, nvme_mpath_set_live() waits for an SRCU grace period of the namespace head before kicking the requeue work, so that bios which found no usable path are on the requeue list by the time it runs. The ANA work calls nvme_mpath_set_live() for every namespace that becomes optimized or non-optimized, one after another, so an ANA change waits for one grace period per namespace. Start the grace period with start_poll_synchronize_srcu() instead and have nvme_requeue_work() wait for it only if it has not completed yet. Take the cookie under head->lock, so that concurrent callers store their cookies in order and the requeue work never sees an older one. The grace periods of all namespaces then run in parallel, and requeue work kicked for other reasons finds the cookie completed and does not wait. Flipping an ANA group from inaccessible to optimized ten times on each of two boots over nvme-tcp loopback, nvme_read_ana_log() takes (median, min-max in parentheses): namespaces before (ms) after (ms) change 256 52.0 (35.4-74.7) 8.7 (2.5-19.6) -83% 1024 154.3 (123.9-193.8) 43.1 (18.2-58.3) -72% Signed-off-by: Qiliang Yuan Reviewed-by: Hannes Reinecke Reviewed-by: Nilay Shroff --- drivers/nvme/host/multipath.c | 15 ++++++++++++++- drivers/nvme/host/nvme.h | 1 + 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index 51c59cc3ea246..ed930dd2af24d 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -712,6 +712,11 @@ static void nvme_requeue_work(struct work_struct *work) container_of(work, struct nvme_ns_head, requeue_work); struct bio *bio, *next; + /* Let bios that found no usable path reach the requeue list first. */ + if (!poll_state_synchronize_srcu(&head->srcu, + READ_ONCE(head->requeue_srcu_cookie))) + synchronize_srcu(&head->srcu); + spin_lock_irq(&head->requeue_lock); next = bio_list_get(&head->requeue_list); spin_unlock_irq(&head->requeue_lock); @@ -767,6 +772,7 @@ int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl, struct nvme_ns_head *head) mutex_init(&head->lock); spin_lock_init(&head->requeue_lock); INIT_WORK(&head->requeue_work, nvme_requeue_work); + head->requeue_srcu_cookie = get_completed_synchronize_srcu(); INIT_WORK(&head->partition_scan_work, nvme_partition_scan_work); INIT_DELAYED_WORK(&head->remove_work, nvme_remove_head_work); @@ -849,9 +855,16 @@ static void nvme_mpath_set_live(struct nvme_ns *ns) __nvme_find_path(head, node); srcu_read_unlock(&head->srcu, srcu_idx); } + /* + * Have nvme_requeue_work() wait for the grace period instead of + * blocking here, so that namespaces going live together, e.g. on + * an ANA change, do not wait for one grace period each. Start it + * under head->lock so that the newest cookie is stored last. + */ + WRITE_ONCE(head->requeue_srcu_cookie, + start_poll_synchronize_srcu(&head->srcu)); mutex_unlock(&head->lock); - synchronize_srcu(&head->srcu); nvme_mpath_revalidate_zones(head); kblockd_schedule_work(&head->requeue_work); } diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 8833a463f1c84..3fff9b394b1cd 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -587,6 +587,7 @@ struct nvme_ns_head { __guarded_by(&requeue_lock); spinlock_t requeue_lock; struct work_struct requeue_work; + unsigned long requeue_srcu_cookie; struct work_struct partition_scan_work; struct mutex lock; unsigned long flags; -- 2.43.0