Re: [PATCH] nvme: core: freeze multipath queue early in nvme_update_ns_info()

[Niklas Cassel <cassel@kernel.org>]

On Fri, Aug 23, 2024 at 05:26:56PM +0200, Martin Wilck wrote:
> On Fri, 2024-08-23 at 15:38 +0200, Niklas Cassel wrote:
> > On Thu, Aug 22, 2024 at 10:14:13PM +0200, Martin Wilck wrote:
> > > For multipath devices, nvme_update_ns_info() needs to freeze both
> > > the queue of the path and the queue of the multipath device. For
> > > both operations, it waits for one RCU grace period to pass, ~25ms
> > > on my test system. By calling blk_freeze_queue_start() for the
> > > multipath queue early, we avoid waiting twice; tests using ftrace
> > > have shown that the second blk_mq_freeze_queue_wait() call finishes
> > > in just a few microseconds. The path queue is unfrozen before
> > > calling blk_mq_freeze_queue_wait() on the multipath queue, so that
> > > possibly outstanding IO in the multipath queue can be flushed.
> > > 
> > > I tested this using the "controller rescan under I/O load" test
> > > I submitted recently [1].
> > > 
> > > [1]
> > > https://lore.kernel.org/linux-nvme/20240822193814.106111-3-mwilck@suse.com/T/#u
> > > 
> > > Signed-off-by: Martin Wilck <mwilck@suse.com>
> > > ---
> > >  drivers/nvme/host/core.c | 8 ++++++--
> > >  1 file changed, 6 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> > > index 33fa01c599ad..e2454398c660 100644
> > > --- a/drivers/nvme/host/core.c
> > > +++ b/drivers/nvme/host/core.c
> > > @@ -2217,6 +2217,9 @@ static int nvme_update_ns_info(struct nvme_ns
> > > *ns, struct nvme_ns_info *info)
> > >  	bool unsupported = false;
> > >  	int ret;
> > >  
> > > +	if (nvme_ns_head_multipath(ns->head))
> > > +		blk_freeze_queue_start(ns->head->disk->queue);
> > > +
> > 
> > From someone reading this code, it looks quite similar to
> > nvme_mpath_start_freeze().
> 
> That function takes a struct nvme_subsystem as argument, and walks over
> all namespaces in that subsystem, whereas here we're just acting on a
> single namespace.
> 
> > Perhaps create a new helper, with proper kdoc, and possibly also add
> > kdoc to
> > nvme_mpath_start_freeze(), so that a user can easily tell (from the
> > kdoc)
> > when to use which function.
> 
> What's the benefit of introducing such a trivial helper, used only in a
> single place of the code?

To make nvme_update_ns_info() easier to read, and to keep the existing code
style of calling nvme_mpath_*() functions unconditionally.

Sure, with my suggestion you would need a mpath_freeze() and mpath_unfreeze()
helper.

But if you add a helper that simply does:
if (nvme_ns_head_multipath(ns->head))
	blk_freeze_queue_start(ns->head->disk->queue);

nvme_update_ns_info() could call the helper unconditionally,
just like how e.g. nvme_passthru_start() calls nvme_mpath_start_freeze()
unconditionally for both multipath and non-multipath NS heads:
https://github.com/torvalds/linux/blob/v6.11-rc4/drivers/nvme/host/core.c#L1205

(And how core.c calls many other nvme_mpath_*() functions unconditionally.)

> Thanks for the suggestion, but I'd like to see other maintainers'
> opinions about this.

Of course, a suggestion is just a suggestion :)


Kind regards,
Niklas