Re: [RFC 07/22] dm: blk: add UCLASS_PARTITION

[AKASHI Takahiro <takahiro.akashi@linaro.org>]

On Thu, Nov 04, 2021 at 08:02:05PM -0600, Simon Glass wrote:
> Hi,
> 
> On Tue, 2 Nov 2021 at 01:43, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >
> >
> >
> > On 11/1/21 03:14, Simon Glass wrote:
> > > Hi Takahiro,
> > >
> > > On Sun, 31 Oct 2021 at 19:52, AKASHI Takahiro
> > > <takahiro.akashi@linaro.org> wrote:
> > >>
> > >> On Sun, Oct 31, 2021 at 07:15:17PM -0600, Simon Glass wrote:
> > >>> Hi Takahiro,
> > >>>
> > >>> On Sun, 31 Oct 2021 at 18:36, AKASHI Takahiro
> > >>> <takahiro.akashi@linaro.org> wrote:
> > >>>>
> > >>>> On Sat, Oct 30, 2021 at 07:45:14AM +0200, Heinrich Schuchardt wrote:
> > >>>>>
> > >>>>>
> > >>>>> Am 29. Oktober 2021 23:17:56 MESZ schrieb Simon Glass <sjg@chromium.org>:
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> On Fri, 29 Oct 2021 at 13:26, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Am 29. Oktober 2021 08:15:56 MESZ schrieb AKASHI Takahiro <takahiro.akashi@linaro.org>:
> > >>>>>>>> On Fri, Oct 29, 2021 at 06:57:24AM +0200, Heinrich Schuchardt wrote:
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>> I agree with Heinrich that we are better to leave BLK as it is, both
> > >>>>>>>>>> in name and meaning. I think maybe I am missing the gist of your
> > >>>>>>>>>> argument.
> > >>>>>>>>>>
> > >>>>>>>>>> If we use UCLASS_PART, for example, can we have that refer to both s/w
> > >>>>>>>>>> and h/w partitions, as Herinch seems to allude to below? What would
> > >>>>>>>>>> the picture look like the, and would it get us closer to agreement?
> > >>>>>>>>>
> > >>>>>>>>> In the driver model:
> > >>>>>>>>>
> > >>>>>>>>> A UCLASS is a class of drivers that share the same interface.
> > >>>>>>>>> A UDEVICE is a logical device that belongs to exactly one UCLASS and is
> > >>>>>>>>> accessed through this UCLASS's interface.
> > >>>>>>>>
> > >>>>>>>> Please be careful about "accessed through" which is a quite confusing
> > >>>>>>>> expression. I don't always agree with this view.
> > >>>>>>>>
> > >>>>>>>>> A hardware partition is an object that exposes only a single interface
> > >>>>>>>>> for block IO.
> > >>>>>>>>>
> > >>>>>>>>> A software partition is an object that may expose two interfaces: one
> > >>>>>>>>> for block IO, the other for file IO.
> > >>>>>>>>
> > >>>>>>>> Are you talking about UEFI world or U-Boot?
> > >>>>>>>> Definitely, a hw partitions can provide a file system
> > >>>>>>>> if you want.
> > >>>>>>>> It's a matter of usage.
> > >>>>>>>>
> > >>>>>>>> I remember that we had some discussion about whether block devices
> > >>>>>>>> on UEFI system should always have a (sw) partition table or not.
> > >>>>>>>> But it is a different topic.
> > >>>>>>>>
> > >>>>>>>>> The UEFI model does not have a problem with this because on a handle you
> > >>>>>>>>> can install as many different protocols as you wish. But U-Boot's driver
> > >>>>>>>>> model only allows a single interface per device. Up to now U-Boot has
> > >>>>>>>>> overcome this limitation by creating child devices for the extra interfaces.
> > >>>>>>>>
> > >>>>>>>>> We have the following logical levels:
> > >>>>>>>>>
> > >>>>>>>>> Controller      | Block device | Software Partition| File system
> > >>>>>>>>> ----------------+--------------+-------------------+------------
> > >>>>>>>>> NVMe Drive      | Namespace    | Partition 1..n    | FAT, EXT4
> > >>>>>>>>> ATA Controller  | ATA-Drive    |                   |
> > >>>>>>>>> SCSI Controller | LUN          |                   |
> > >>>>>>>>> MMC Controller  | HW-Partition |                   |
> > >>>>>>>>> MMC Controller  | SD-Card      |                   |
> > >>>>>>>>> USB-Node        | USB-Drive    |                   |
> > >>>>>>>>>
> > >>>>>>>>> In the device tree this could be modeled as:
> > >>>>>>>>>
> > >>>>>>>>> |-- Controller (UCLASS_CTRL)
> > >>>>>>>>> | |-- Block device / HW Partition (UCLASS_BLK)    (A)
> > >>>>>>>>> | | |-- Partition table (UCLASS_PARTITION_TABLE)  (B)
> > >>>>>>>>> | |   |-- Software Partition (UCLASS_BLK)
> > >>>>>>>>> | |     |-- File system (UCLASS_FS)
> > >>>>>>>>> | |
> > >>>>>>>>> | |-- Block device (UCLASS_BLK)
> > >>>>>>>>> |   |-- File system (UCLASS_FS)
> > >>>>>>>>
> > >>>>>>>> I don't know why we expect PARTITION_TABLE and FS to appear in DM tree.
> > >>>>>>>> What is the benefit?
> > >>>>>>>> (A) and (B) always have 1:1 relationship.
> > >>>>>>>
> > >>>>>>> No. You can have a bare device without a partition table.
> > >>>>>>
> > >>>>>> I can have a DOS partition that covers the whole device, without a
> > >>>>>> partition table. This is supported in U-Boot and Linux.
> > >>>>>>
> > >>>>>>>
> > >>>>>>> We have several partition table drivers: DOS, GPT, OSX, ... . In future we should also have one for the NOR Flash partitions. All of these drivers have a common interface. As the partition table type is mostly independent of the block device type we should use separate uclasses and udevices.
> > >>>>>>>
> > >>>>>>>> I also remember that you claimed that not all efi objects(handles and
> > >>>>>>>> protocols like SIMPE_FILE_SYSTEM_PROTOCOL) need to have corresponding
> > >>>>>>>> U-Boot counterparts in our 2019 discussion.
> > >>>>>>>>
> > >>>>>>>> If we *need* PARTITION_TALBLE, why don't we have HW_PARTITION_TABLE,
> > >>>>>>>> which should support other type of hw partitions as well?
> > >>>>>>>
> > >>>>>>> How hardware partitions, LUNs, ATA drives are enumerated is specific to the type of controller while the type of software partition table  is independent of the block device.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> |-- eMMC controller (UCLASS_MMC)
> > >>>>>>>> | |-- eMMC device1 / Physical media (UCLASS_HW_PARTITION_TABLE?)
> > >>>>>>>> |   |-- Block device / HW Partition:user data (UCLASS_BLK)
> > >>>>>>>> |   | |-- Partition table (UCLASS_PARTITION_TABLE)
> > >>>>>>>> |   |   |-- Software Partition (UCLASS_BLK)
> > >>>>>>>> |   |     |-- File system (UCLASS_FS)
> > >>>>>>>> |   |
> > >>>>>>>> |   |-- Block device / HW Partition:boot0 (UCLASS_BLK)
> > >>>>>>>> |   |-- Block device / HW Partition:boot1 (UCLASS_BLK)
> > >>>>>>>>           ...
> > >>>>>>>> | |-- eMMC device2 / Physical media (UCLASS_HW_PARTITION_TABLE?)
> > >>>>>>>>
> > >>>>>>>> |-- scsi controller (UCLASS_SCSI)
> > >>>>>>>> | |-- scsi disk / Physical media (UCLASS_HW_PARTITION_TABLE?)
> > >>>>>>>> |   |-- scsi LUN1 (UCLASS_HW_PARTITION_TABLE?)
> > >>>>>>>> |   | |-- Partition table (UCLASS_PARTITION_TABLE)
> > >>>>>>>> |   |   |-- Software Partition (UCLASS_BLK)
> > >>>>>>>> |   |-- scsi LUN2 (UCLASS_HW_PARTITION_TABLE?)
> > >>>>>>>>           ...
> > >>>>>>>>
> > >>>>>>>> (Here I ignored scsi buses/channels which make things more complicated.)
> > >>>>>>>>
> > >>>>>>>> This kind of complex hierarchy doesn't benefit anybody.
> > >>>>>>>
> > >>>>>>> All these levels exist already. We simply do not model them yet in the DM way.
> > >>>>>>>
> > >>>>>>> The device tree depth is the outcome of the udevice exposing always only a single interface defined by the uclass.
> > >>>>>>>
> > >>>>>>> The UEFI design allows installing multiple protocol interfaces on a single handle. This may result in simpler device trees in some cases.
> > >>>>>>
> > >>>>>> Yes, the complexity has to go somewhere. With driver model I chose to
> > >>>>>> have a single interface per uclass, since it is simpler to understand,
> > >>>>>> no need to request a protocol for a device, etc.
> > >>>>>>
> > >>>>>> Our current setup is similar to this
> > >>>>>>
> > >>>>>> |-- Controller (UCLASS_MMC)
> > >>>>>> | |-- Block device (UCLASS_BLK)     - 'usual' HW partition
> > >>>>>> | |-- Block device (UCLASS_BLK)     - e.g. for a different HW partition*
> > >>>>>>
> > >>>>>> * although I don't think the MMC code actually supports it - SCSI does though
> > >>>>>>
> > >>>>>> We want to add devices for the partition table and the filesystem, so could do:
> > >>>>>>
> > >>>>>> |-- Controller (UCLASS_MMC)
> > >>>>>> | |-- Block device (UCLASS_BLK)     - 'usual' HW partition (the whole device)
> > >>>>>> | | |-- Partition table (UCLASS_PART)  - DOS partition (or EFI)
> > >>>>>> | | | |-- Block device (UCLASS_BLK)  - partition 1
> > >>>>>> | | | | |-- Filesystem (UCLASS_FS) - DOS filesystem
> > >>>>>> | | | |-- Block device (UCLASS_BLK)  - partition 2
> > >>>>>> | | | | |-- Filesystem (UCLASS_FS) - ext5 filesystem
> > >>>>>> | |-- Block device (UCLASS_BLK)     - e.g. for a different HW
> > >>>>>> partition (the whole device)
> > >>>>>>
> > >>>>>> This is similar to Heinrich's, but without the top-level
> > >>>>>> UCLASS_HW_PARTITION_TABLE which I am not sure is necessary.
> > >>>>>
> > >>>>> Are further MMC hw partitions, multiple SCSI LUNs and multiple NVME  namespaces already treated as separate BLK devices?
> > >>>>
> > >>>> Yes.
> > >>>> What I meant to say is that, if we don't need a partition table 'udevice'
> > >>>> for hw partitions, we don't need such a device for sw partitions neither.
> > >>>>
> > >>>> Meanwhile, what about UCLASS_FS? Why do we need this?
> > >>>
> > >>> We don't need it for our current discussion, but if we want to 'open'
> > >>> the filesystem and keep the metadata around, rather than reading it
> > >>> again every time we access a file, we might find it useful. Open files
> > >>> could be children of the FS uclass, perhaps, if we go a step further
> > >>> and create devices for them.
> > >>
> > >> Do you want to invent linux-like mount-point concepts or procfs?
> > >> I remember that you didn't want to have child nodes under BLK devices.
> > >> I'm getting confused about our goal.
> > >
> > > I think we are all a bit unsure.
> > >
> > > I think BLK devices can have children, sorry if I said the wrong thing
> > > somewhere along the way. For example, a partition would be under a BLK
> > > device, or a FS.
> > >
> > >> What should DM represent in U-Boot world?
> > >
> > > That is what we are trying to figure out.
> > >
> > > I think the minimum is to have a a way to represent partitions (s/w
> > > and hw/). As I understand it, that's what we've been discussing.
> >
> > The discovery of hardware partitions is specific to the block device
> > controller SCSI/MMC/ATA/NVMe. We currently do not provide any
> > manipulation commands to create hardware partitions (e.g. NVMe
> > namespaces, SCSI LUNs). This is why extracting a uclass for hardware
> > partitions does not seem necessary.
> 
> I can see the reasoning here. It might not stand the test of time but
> how about we go with it for now? For MMC hardware partition we would
> just end up with multiple BLK devices, like we do with SCSI LUNs at
> present, which seems like it should work (with some code tweaks).
> 
> >
> > Software partitioning (MBR, GPT, ...) is independent of the harboring
> > block device.
> >
> > We already have a set of drivers for software partition tables in disk/.
> > Currently the available methods of the drivers are defined in
> > U_BOOT_PART_TYPE referring to struct part_driver.
> >
> > Currently struct part_driver knows only the following methods:
> >
> > - get_info()
> > - print()
> > - test()
> >
> > These drivers should be ome a uclass.
> >
> > gpt.c and mbr.c allow to create and delete partitions. I think we should add
> >
> > - create_partition()
> > - delete_partition()
> >
> > to the uclass methods.
> 
> That sounds good to me, although since it is a partition uclass, we
> can just use create() and delete().

I don't know why we need a "partition table" device in the middle
of DM hierarchy.
I believe that it simply makes the view of DM tree complicated
without any explicit benefit.

-Takahiro Akashi


> >
> > The partitions handled by cmd/mtdparts.c, cmd/nand.c are also software
> > partitions. The difference to MBR, GPT is that the partition table is
> > held in memory and not on disk. These partitions could be modeled in the
> > same uclass.
> 
> For future work!
> 
> Regards,
> SImon
> 
> 
> > >>>
> > >>>>>>
> > >>>>>> It is compatible with what we have now and we could enable/disable the
> > >>>>>> extra devices with a Kconfig.
> > >>>>>>
> > >>>>>> Regards,
> > >>>>>> Simon
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>>
> > >>>>>>>>> UCLASS_PARTITION_TABLE would be for the drivers in disk/.
> > >>>>>>>>> UCLASS_FS would be for the drivers in fs/.
> > >>>>>>>>> UCLASS_BLK will be for any objects exposing raw block IO. A software
> > >>>>>>>>> partition does the same. It is created by the partition table driver as
> > >>>>>>>>> child of the partition table udevice.
> > >>>>>>>>>
> > >>>>>>>>> In this model an eMMC device will not be a UCLASS_BLK device because it
> > >>>>>>>>> does not expose block IO. It is the hardware partition that exposes this
> > >>>>>>>>> interface.
> > >>>>>>>>>
> > >>>>>>>>> The suggested model will allow a clean description of nested partition
> > >>>>>>>>> tables.
> > >>>>>>>>>
> > >>>>>>>>> In the UEFI world the software partition and its file system must be
> > >>>>>>>>> mapped to a single handle with device path node type HD(). For the
> > >>>>>>>>> parent block device we may create a child handle with partition number 0
> > >>>>>>>>> (HD(0)). For the partition table we will not create a handle.
> > >>>>>>>>>
> > >>>>>>>>> Best regards
> > >>>>>>>>>
> > >>>>>>>>> Heinrich