Re: Best practice for new linux block driver device naming?

[scameron@beardog.cce.hp.com]

On Fri, Mar 08, 2013 at 04:56:32PM -0500, Lennart Sorensen wrote:
> On Fri, Mar 08, 2013 at 02:07:18PM -0600, scameron@beardog.cce.hp.com wrote:
> > I'm just wondering if there are best pratices for new linux block
> > drivers that are adding new devices nodes of which grub is currently
> > not cognizant.
> > 
> > E.g. when we added the HP Smart Array cciss driver to the kernel
> > many years ago, it had device nodes like /dev/cciss/c*d*, and there's
> > code in grub to handle this in util/getroot.c, in
> > convert_system_partition_to_system_disk():
> > 
> >       /* If this is a CCISS disk.  */
> >       if (strncmp ("cciss/c", p, sizeof ("cciss/c") - 1) == 0)
> >         {
> >           /* /dev/cciss/c[0-9]+d[0-9]+(p[0-9]+)? */
> >           p = strchr (p, 'p');
> >           if (p)
> >             {
> >               *is_part = 1;
> >               *p = '\0';
> >             }
> > 
> >           return path;
> >         }
> > 
> > And there is similar code for other weird device names is in there
> > as well.
> > 
> > Ideally, I'm hoping there's a way to introduce new devices nodes
> > with a new block driver which does not any require grub modifications.
> > Looking over the code, it's not clear to me whether or not this is
> > possible, and if it is, how to do it, what the constraints may be, etc.
> > 
> > Currently I have a new driver which adds devices like /dev/sop0  with
> > partitions like /dev/sop0p1, /dev/sop0p2, etc.
> > 
> > If there is some better way to do this to enable grub to work with
> > these devices, it is not yet too late for me to change it.
> > 
> > Or on the other hand, if it turns out that it is not possible to add
> > new block devices to linux and have grub support for those devices without
> > also modifying grub, then I wonder if it might be worth looking into to
> > adding some kind of shared device namespace for block devices to linux, so
> > new block drivers could use that and have a common naming system for block devices,
> > and grub could be modified to support this new common naming system,
> > much as scsi hba device drivers share the /dev/sd* namespace for their attached
> > disks, so it is easy to add new scsi hba drivers to linux and automatically have
> > grub support for them.  It would be nice if it were similarly easy to add new
> > block device drivers to linux without also requiring modifications to grub.
> > (It also occurs to me that this is such an obvious desire that if it is not 
> > already supported, perhaps there's a good reason why not, but if that's the
> > case, I'm don't know what the reason might be.)
> > 
> > Thoughts?
> 
> Well currently, SCSI, SATA, IDE, most well behaved raid controllers,
> USB storage, and many others all show up simple as /dev/sd*.  You better
> have a really good reason to not do so if you make a new controller.
> Certainly the IBM serveraid cards I have worked with just present sd*
> devices (as well as some sg* devices for the controller and hotswap
> backplane and such).  

I get ~4x the IOPSs with a block driver vs. scsi driver due to contention
for locks in the scsi mid layer (in scsi_request_fn).  It's the
difference between the device being worth manufacturing vs. not.

See this thread: http://marc.info/?l=linux-scsi&m=135518042125008&w=2

Driver is similar to nvme (also a new block driver), but this one is
for SCSI over PCIe, basically highly parallelized access to very low
latency devices and trying to use the SCSI midlayer kills the IOPS.
 
> I think the CCISS is a sample of a horrible design
> as far as the device names in linux are concerned.

There were reasons back then for doing that one as a block driver
which are no longer extant (hence the existence of the hpsa driver
which supplanted cciss for new smart array devices.)

> 
> By creating a new type of block device you force everyone else to do work
> to support it (or choose to ignore your device because no one cares).
> By emulating a plain old scsi device interface, everything else just
> works, and all the work is done by you in your device driver to pretend
> to my just a scsi disk.  If you want your device taken seriously,
> I don't think you have a choice.  Sysadmin's hate things that are
> different for no good reason.

All other things being equal, I would also prefer a scsi driver.
Heck, it's called SCSI over PCIe -- I tried like hell to get it 
to perform adequately as a SCSI driver but all other things are
not equal, not even close, the block driver was ~4x as fast.

So we reluctantly go with a block driver, just like nvme did.

-- steve