[PATCH v2 00/13] mpt3sas driver NVMe support:

[James.Bottomley@HansenPartnership.com (James Bottomley)]

On Mon, 2017-08-07@20:01 +0530, Kashyap Desai wrote:
> > 
> > -----Original Message-----
> > From: James Bottomley [mailto:James.Bottomley at HansenPartnership.com
> > ]
> > Sent: Monday, August 07, 2017 7:48 PM
> > To: Kashyap Desai; Christoph Hellwig; Hannes Reinecke
> > Cc: Suganath Prabu Subramani; martin.petersen at oracle.com; linux-
> > scsi at vger.kernel.org; Sathya Prakash Veerichetty; linux-
> > kernel at vger.kernel.org; Chaitra Basappa; Sreekanth Reddy; linux-
> > nvme at lists.infradead.org
> > Subject: Re: [PATCH v2 00/13] mpt3sas driver NVMe support:
> > 
> > On Mon, 2017-08-07@19:26 +0530, Kashyap Desai wrote:
> > > 
> > > > 
> > > > 
> > > > -----Original Message-----
> > > > From: James Bottomley [mailto:James.Bottomley at HansenPartnership
> > > > .com
> > > > ]
> > > > Sent: Saturday, August 05, 2017 8:12 PM
> > > > To: Christoph Hellwig; Hannes Reinecke
> > > > Cc: Suganath Prabu S; martin.petersen at oracle.com; linux-
> > > > scsi at vger.kernel.org; Sathya.Prakash at broadcom.com;
> > > > kashyap.desai at broadcom.com; linux-kernel at vger.kernel.org;
> > > > chaitra.basappa at broadcom.com; sreekanth.reddy at broadcom.com;
> > > > linux-
> > > > nvme at lists.infradead.org
> > > > Subject: Re: [PATCH v2 00/13] mpt3sas driver NVMe support:
> > > > 
> > > > On Sat, 2017-08-05@06:53 -0700, Christoph Hellwig wrote:
> > > > > 
> > > > > 
> > > > > On Wed, Aug 02, 2017 at 10:14:40AM +0200, Hannes Reinecke
> > > > > wrote:
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > I'm not happy with this approach.
> > > > > > NVMe devices should _not_ appear as SCSI devices; this will
> > > > > > just
> > > > > > confuse matters _and_ will be incompatible with 'normal'
> > > > > > NVMe
> > > > > > devices.
> > > > > > 
> > > > > > Rather I would like to see the driver to hook into the
> > > > > > existing
> > > > > > NVMe framework (which essentially means to treat the
> > > > > > mpt3sas as
> > > > > > a weird NVMe-over-Fabrics HBA), and expose the NVMe devices
> > > > > > like
> > > > > > any other NVMe HBA.
> > > > > 
> > > > > That doesn't make any sense.??The devices behind the mpt
> > > > > adapter
> > > > > don't look like NVMe devices at all for the hosts - there are
> > > > > no
> > > > > NVMe commands or queues involved at all, they hide behind the
> > > > > same
> > > > > somewhat leaky scsi abstraction as other devices behind the
> > > > > mpt
> > > > > controller.
> > > > 
> > > > You might think about what we did for SAS: split the generic
> > > > handler
> > > > into two pieces, libsas for driving the devices, which mpt
> > > > didn't
> > > > need because of the fat firmware and the SAS transport class so
> > > > mpt
> > > > could at least show the same sysfs files as everything else for
> > > > SAS
> > > > devices.
> > > 
> > > ?Ventura generation of controllers are adding connectivity of
> > > NVME
> > > ?drives seamlessly and protocol handling is in Firmware.
> > > ?Same as SCSI to ATA translation done in firmware, Ventura
> > > controller
> > > ?is doing SCSI to NVME translation and for end user protocol
> > > handling
> > > ?is abstracted.
> > > 
> > > ?This product handles new Transport protocol (NVME) same as ATA
> > > and
> > > ?transport is abstracted for end user.
> > > 
> > > NVME pass-through related driver code, it is just a big tunnel
> > > for
> > > user space application. It is just a basic framework like SATA
> > > PASS-
> > > Through in existing mpt3sas driver.
> > 
> > I know how it works ... and I'm on record as not liking your SATL
> > approach
> > because we keep tripping across bugs in the SATL that we have to
> > fix in
> > the
> > driver.
> 
> We discussed about NVME device support behind <mpt3sas> to Hannes and
> he suggested to describe to product behavior to wider audience to be
> aware. Just wanted to share the notes.

Great, thanks! it's certainly better than releasing the product and
having us have to support it.

> > However, at least for bot SAS and SATA they appear to the system as
> > SCSI
> > devices regardless of HBA, so we've largely smoothed over any
> > problems if
> > you
> > transfer from mp3sas to another SAS/SATA controller.
> > 
> > I believe your current proposal is to have NVMe devices appear as
> > SCSI,
> > which
> > isn't how the native NVMe driver handles them at all.??This is
> > going to
> > have to
> > be special cased in any tool designed to handle nvme devices and
> > it's
> > going to
> > cause big problems if someone changes controller (or moves the
> > device).??What's the proposal for making this as painless as
> > possible?
> 
> We have to attempt this use case and see how it behaves. I have not
> tried this, so not sure if things are really bad or just some tuning
> may be helpful. I will revert back to you on this.
> 
> I understood request as -??We need some udev rules to be working well
> for *same* NVME drives if it is behind <mpt3sas> or native <nvme>.
> Example - If user has OS installed on NVME drive which is behind
> <mpt3sas> driver as SCSI disk should be able to boot if he/she hooked
> same NVME drive which is detected by native <nvme> driver (and vice
> versa.)

It's not just the udev rules, it's the tools as well; possibly things
like that nvme-cli toolkit Intel is doing.

James