Re: nvme/pcie hot plug results in /dev name change

[Ming Lei <ming.lei@redhat.com>]

On Tue, Feb 14, 2023 at 11:18:20AM +0200, Sagi Grimberg wrote:
> 
> > > > > Also not sure if this is going to work out, but it looks like a good start to
> > > > > me.
> > > > > 
> > > > > Instead of user pinning the virtual device so that it never goes away though, I
> > > > > considered a user tunable "device missing delay" parameter to debounce link
> > > > > events. New IOs would be deferred to the requeue_list while the timer is
> > > > > active, and then EIO'ed if the timer expires without a successful LIVE
> > > > > controller attachment. The use cases I'm considering are short bounces from
> > > > > transient link resets, so I'd expect timers to be from a few seconds to maybe a
> > > > > minute.
> > > > 
> > > > Isn't this equivalent to dm-mpath queue_if_no_path or no_path_timeout ?
> > > 
> > > Similiar, but generic to non-multipath devices.
> > > > We can keep the mpath device around, but if not, what is the desired
> > > > behavior from the upper layers?
> > > 
> > > I don't think we're looking for any behavioral changes in the upper layers.
> > That also means no matter if the nvme mpath layer is added or not, upper
> > layer still has to handle this kind of failure, so what is the
> > difference made from the added nvme-mpath?
> 
> What do you expect from the upper layer? The device went away...

From developer viewpoint, FS has to be re-configured(re-mount after
new device comes back).

> 
> The suggestion is that when the device comes back the upper layer will
> detect it and reconfigure it (restore it to a raid/dm or remount the
> FS)?

I understand userspace is capable of recognizing if same device comes
back, and userspace can get notified with device add/del. Why bother kernel
to do that?

Also suppose the mpath device is invented for this purpose, we can't
guarantee that the mapth dev won't be gone and back, so userspace has
to deal with the situation, right?

Also the issue isn't nvme unique, and it shouldn't be implemented in
nvme-mpath suppose agreement is reached to solve it in kernel.

Thanks,
Ming