Re: [RFC PATCH V3 0/8] hw/block/nvme: support multi-path for ctrl/ns

[Minwoo Im <minwoo.im.dev@gmail.com>]

On 21-01-20 08:52:14, Klaus Jensen wrote:
> On Jan 20 09:44, Minwoo Im wrote:
> > On 21-01-19 19:18:16, Klaus Jensen wrote:
> > > On Jan 20 02:01, Minwoo Im wrote:
> > > > Hello,
> > > > 
> > > > This patch series is third one to support multi-controller and namespace
> > > > sharing in multi-path.  This series introduced subsystem scheme to
> > > > manage controller(s) and namespace(s) in the subsystem.
> > > > 
> > > > This series has new patches from the V2:  'detached' parameter has been
> > > > added to the nvme-ns device.  This will decide whether to attach the
> > > > namespace to controller(s) in the current subsystem or not.  If it's
> > > > given with true, then it will be just allocated in the subsystem, but
> > > > not attaching to any controllers in the subsystem.  Otherwise, it will
> > > > automatically attach to all the controllers in the subsystem.  The other
> > > > t hing is that the last patch implemented Identify Active Namespace ID
> > > > List command handler apart from the Allocated Namespace ID List.
> > > > 
> > > > Run with:
> > > >   -device nvme,serial=qux,id=nvme3
> > > >   -device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3
> > > > 
> > > >   -device nvme-subsys,id=subsys0
> > > >   -device nvme,serial=foo,id=nvme0,subsys=subsys0
> > > >   -device nvme,serial=bar,id=nvme1,subsys=subsys0
> > > >   -device nvme,serial=baz,id=nvme2,subsys=subsys0
> > > >   -device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true
> > > >   -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2
> > > > 
> > > > nvme-cli:
> > > >   root@vm:~/work# nvme list -v                                                                                                      
> > > >   NVM Express Subsystems                                                                                                 
> > > >                                                                                                                                      
> > > >   Subsystem        Subsystem-NQN                                                                                    Controllers
> > > >   ---------------- ------------------------------------------------------------------------------------------------ ----------------
> > > >   nvme-subsys0     nqn.2019-08.org.qemu:qux                                                                         nvme0
> > > >   nvme-subsys1     nqn.2019-08.org.qemu:subsys0                                                                     nvme1, nvme2, nvme3
> > > >                                                                                                                                    
> > > >   NVM Express Controllers                                                                                           
> > > >                                                                                                                   
> > > >   Device   SN                   MN                                       FR       TxPort Address        Subsystem    Namespaces
> > > >   -------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------
> > > >   nvme0    qux                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:06.0   nvme-subsys0
> > > 
> > > Shouldn't nvme0n1 be listed under Namespaces for nvme0?
> > 
> > Oh, I missed that one from the output.  As Keith mentioned, I ran the
> > list command again based on the latest nvme-cli.git:
> > 
> > Please refer the following result.  I think it's okay not to send the
> > cover letter again :)
> > 
> > # nvme --version
> > nvme version 1.13.48.g33c6
> > 
> > # nvme list -v
> > NVM Express Subsystems
> > 
> > Subsystem        Subsystem-NQN                                                                                    Controllers
> > ---------------- ------------------------------------------------------------------------------------------------ ----------------
> > nvme-subsys0     nqn.2019-08.org.qemu:qux                                                                         nvme0
> > nvme-subsys1     nqn.2019-08.org.qemu:subsys0                                                                     nvme1, nvme2, nvme3
> > 
> > NVM Express Controllers
> > 
> > Device   SN                   MN                                       FR       TxPort Address        Subsystem    Namespaces      
> > -------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------
> > nvme0    qux                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:06.0   nvme-subsys0 nvme0n1
> > nvme1    foo                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:07.0   nvme-subsys1 
> > nvme2    bar                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:08.0   nvme-subsys1 
> > nvme3    baz                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:09.0   nvme-subsys1 nvme1c3n1
> > 
> > NVM Express Namespaces
> > 
> > Device       NSID     Usage                      Format           Controllers     
> > ------------ -------- -------------------------- ---------------- ----------------
> > nvme0n1      3        268.44  MB / 268.44  MB    512   B +  0 B   nvme0
> > nvme1n1      2        268.44  MB / 268.44  MB    512   B +  0 B   nvme3
> 
> That looks better, but hmm. Shouldnt the namespace be named `nvme1c3n1`
> here has well? Is that also an issue with nvme-cli?

No it isn't in this series.  The 'nvme3' controller has not been
provided with 'subsys' parameter which means it does not support CMIC
multi-controller as nvme_init_ctrl() does not set the flag for
NVME_CMIC_MULTI_CTRL:

  -device nvme,serial=qux,id=nvme3

If this has been given with 'subsys' which means it supports for
multi-controller, then it will be given like:

  Device   SN                   MN                                       FR       TxPort Address        Subsystem    Namespaces
  -------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------
  nvme0    qux                  QEMU NVMe Ctrl                           1.0      pcie   0000:00:06.0   nvme-subsys0 nvme0c0n1

In short, if 'nvme' device controller is not given with 'subsys', then
it will not support multi-controller and it will never have
GENHD_FL_HIDDEN (nvmeXcYnZ convention).