Re: [PATCHv2] nvme-mpath: delete disk after last connection

From: Hannes Reinecke <hare@suse.de>
To: Keith Busch <kbusch@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>,
	Keith Busch <keith.busch@wdc.com>,
	linux-nvme@lists.infradead.org,
	Daniel Wagner <daniel.wagner@suse.de>
Subject: Re: [PATCHv2] nvme-mpath: delete disk after last connection
Date: Tue, 20 Apr 2021 18:21:57 +0200	[thread overview]
Message-ID: <5cc5da66-e2aa-3c97-ddbc-70eeb4b5a8ea@suse.de> (raw)
In-Reply-To: <20210420141436.GA14300@redsun51.ssa.fujisawa.hgst.com>

On 4/20/21 4:14 PM, Keith Busch wrote:
> On Tue, Apr 20, 2021 at 03:19:10PM +0200, Hannes Reinecke wrote:
>> On 4/20/21 10:05 AM, Christoph Hellwig wrote:
>>> On Fri, Apr 16, 2021 at 08:24:11AM +0200, Hannes Reinecke wrote:
>>>> With the proposed patch, the following messages appear:
>>>>
>>>>    [  227.516807] md/raid1:md0: Disk failure on nvme3n1, disabling device.
>>>>    [  227.516807] md/raid1:md0: Operation continuing on 1 devices.
>>>
>>> So how is this going to work for e.g. a case where the device
>>> disappear due to resets or fabrics connection problems?  This now
>>> directly teards down the device.
>>>
>> Yes, that is correct; the nshead will be removed once the last path is
>> _removed_.
> 
> The end result is also how non-multipath nvme behaves, so I think that's
> what users have come to expect.
> 
Nope. It doesn't.

This patch precisely _aligns_ the behaviour between multipathed (or, to 
be precise, namespaces with CMIC != 0) and non-multipathed ones.

When CMIC == 0 this patch doesn't change the behaviour at all.

>> But key point here is that once the system finds itself in that
>> situation it's impossible to recover, as the refcounts are messed.
>> Even a manual connect call with the same parameter will _not_ restore
>> operation, but rather result in a new namespace.
> 
> I haven't looked at this yet, but is it really not possible to restore
> the original namespace upon the reestablished connection?
> 

No. As mentioned, refcount is messed up.

With the current behaviour, once the last path drops, the namespace will 
_vanish_ from the list of namespaces.
Once you reconnect the original namespace 'magically' reappears in 'nvme 
list', but with size '0', _and_ the new namespace beside it:

Needless to say, the current opener (mount, MD, you name it) still refer 
to the original namespace, and you can't correct the situation as I/O is 
still pending you have no way to abort it.

(Funny, though; it was you who suggested the patch in the first place; 
feels kinda strange to explain the patch to its author ...)

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare@suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme