Proposal: U-Boot MTD (rewriting old story)

Proposal: U-Boot MTD (rewriting old story)

[Jagan Teki]

Yes, this is the old discussion and triggered again now since we had a
discussion on the last U-Boot contributor call.

I will brief the main points here, as most of the details are
mentioned in previous threads.

Here are the couple of old and new implementations on the SPI-NOR side.

http://u-boot.10912.n7.nabble.com/PATCH-v10-00-27-dm-Generic-MTD-Subsystem-with-SPI-NOR-interface-tt315694.html#none
http://u-boot.10912.n7.nabble.com/PATCH-0-5-mtd-Implement-MTD-UCLASS-use-SPINOR-td419538.html#a419541
https://patchwork.ozlabs.org/project/uboot/cover/20200709111709.68904-1-jagan@amarulasolutions.com/

Idea is to
1. Create MTD_UCLASS, already present
2. Create DM MTD Operations, above patchset does.
3. Create Flash specific MTD UClass's like
    UCLASS_NAND
    UCLASS_SPI_NAND
    UCLASS_SPI_NOR
    UCLASS_NOR
    UCLASS_UBI
and register to UCLASS_MTD as interface slaves like how MMC, SCSI are
registered to UCLASS_BLK
4. Sync only respective flash specific code, and drop __UBOOT__
5. Interface mtd, sf, and nand commands to UCLASS_MTD.

Hard points:
1. Hard to arrange Linux specific code and drop unneeded _UBOOT_ code,
but possible.
2. Requires a lot of testing.

Any inputs?
Jagan.

Re: Proposal: U-Boot MTD (rewriting old story)

[Simon Glass]

Hi Jagan,

On Tue, 3 Aug 2021 at 09:01, Jagan Teki <jagan@amarulasolutions.com> wrote:
>
> Yes, this is the old discussion and triggered again now since we had a
> discussion on the last U-Boot contributor call.
>
> I will brief the main points here, as most of the details are
> mentioned in previous threads.
>
> Here are the couple of old and new implementations on the SPI-NOR side.
>
> http://u-boot.10912.n7.nabble.com/PATCH-v10-00-27-dm-Generic-MTD-Subsystem-with-SPI-NOR-interface-tt315694.html#none
> http://u-boot.10912.n7.nabble.com/PATCH-0-5-mtd-Implement-MTD-UCLASS-use-SPINOR-td419538.html#a419541
> https://patchwork.ozlabs.org/project/uboot/cover/20200709111709.68904-1-jagan@amarulasolutions.com/
>
> Idea is to
> 1. Create MTD_UCLASS, already present
> 2. Create DM MTD Operations, above patchset does.
> 3. Create Flash specific MTD UClass's like
>     UCLASS_NAND
>     UCLASS_SPI_NAND
>     UCLASS_SPI_NOR
>     UCLASS_NOR
>     UCLASS_UBI
> and register to UCLASS_MTD as interface slaves like how MMC, SCSI are
> registered to UCLASS_BLK
> 4. Sync only respective flash specific code, and drop __UBOOT__
> 5. Interface mtd, sf, and nand commands to UCLASS_MTD.

This all seems good to me.

>
> Hard points:
> 1. Hard to arrange Linux specific code and drop unneeded _UBOOT_ code,
> but possible.
> 2. Requires a lot of testing.

Let's add simple emulators for NAND, SPI_NAND and UBI. We already have
one for SPI_NOR I believe. That will be a good long-term investment.

Regards,
Simon

Re: Proposal: U-Boot MTD (rewriting old story)

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 1731 bytes --]

On Wed, Aug 04, 2021 at 10:09:55AM -0600, Simon Glass wrote:
> Hi Jagan,
> 
> On Tue, 3 Aug 2021 at 09:01, Jagan Teki <jagan@amarulasolutions.com> wrote:
> >
> > Yes, this is the old discussion and triggered again now since we had a
> > discussion on the last U-Boot contributor call.
> >
> > I will brief the main points here, as most of the details are
> > mentioned in previous threads.
> >
> > Here are the couple of old and new implementations on the SPI-NOR side.
> >
> > http://u-boot.10912.n7.nabble.com/PATCH-v10-00-27-dm-Generic-MTD-Subsystem-with-SPI-NOR-interface-tt315694.html#none
> > http://u-boot.10912.n7.nabble.com/PATCH-0-5-mtd-Implement-MTD-UCLASS-use-SPINOR-td419538.html#a419541
> > https://patchwork.ozlabs.org/project/uboot/cover/20200709111709.68904-1-jagan@amarulasolutions.com/
> >
> > Idea is to
> > 1. Create MTD_UCLASS, already present
> > 2. Create DM MTD Operations, above patchset does.
> > 3. Create Flash specific MTD UClass's like
> >     UCLASS_NAND
> >     UCLASS_SPI_NAND
> >     UCLASS_SPI_NOR
> >     UCLASS_NOR
> >     UCLASS_UBI
> > and register to UCLASS_MTD as interface slaves like how MMC, SCSI are
> > registered to UCLASS_BLK
> > 4. Sync only respective flash specific code, and drop __UBOOT__
> > 5. Interface mtd, sf, and nand commands to UCLASS_MTD.
> 
> This all seems good to me.
> 
> >
> > Hard points:
> > 1. Hard to arrange Linux specific code and drop unneeded _UBOOT_ code,
> > but possible.
> > 2. Requires a lot of testing.
> 
> Let's add simple emulators for NAND, SPI_NAND and UBI. We already have
> one for SPI_NOR I believe. That will be a good long-term investment.

What are our viable options here via QEMU?

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: Proposal: U-Boot MTD (rewriting old story)

[Simon Glass]

Hi Tom,

On Wed, 4 Aug 2021 at 12:49, Tom Rini <trini@konsulko.com> wrote:
>
> On Wed, Aug 04, 2021 at 10:09:55AM -0600, Simon Glass wrote:
> > Hi Jagan,
> >
> > On Tue, 3 Aug 2021 at 09:01, Jagan Teki <jagan@amarulasolutions.com> wrote:
> > >
> > > Yes, this is the old discussion and triggered again now since we had a
> > > discussion on the last U-Boot contributor call.
> > >
> > > I will brief the main points here, as most of the details are
> > > mentioned in previous threads.
> > >
> > > Here are the couple of old and new implementations on the SPI-NOR side.
> > >
> > > http://u-boot.10912.n7.nabble.com/PATCH-v10-00-27-dm-Generic-MTD-Subsystem-with-SPI-NOR-interface-tt315694.html#none
> > > http://u-boot.10912.n7.nabble.com/PATCH-0-5-mtd-Implement-MTD-UCLASS-use-SPINOR-td419538.html#a419541
> > > https://patchwork.ozlabs.org/project/uboot/cover/20200709111709.68904-1-jagan@amarulasolutions.com/
> > >
> > > Idea is to
> > > 1. Create MTD_UCLASS, already present
> > > 2. Create DM MTD Operations, above patchset does.
> > > 3. Create Flash specific MTD UClass's like
> > >     UCLASS_NAND
> > >     UCLASS_SPI_NAND
> > >     UCLASS_SPI_NOR
> > >     UCLASS_NOR
> > >     UCLASS_UBI
> > > and register to UCLASS_MTD as interface slaves like how MMC, SCSI are
> > > registered to UCLASS_BLK
> > > 4. Sync only respective flash specific code, and drop __UBOOT__
> > > 5. Interface mtd, sf, and nand commands to UCLASS_MTD.
> >
> > This all seems good to me.
> >
> > >
> > > Hard points:
> > > 1. Hard to arrange Linux specific code and drop unneeded _UBOOT_ code,
> > > but possible.
> > > 2. Requires a lot of testing.
> >
> > Let's add simple emulators for NAND, SPI_NAND and UBI. We already have
> > one for SPI_NOR I believe. That will be a good long-term investment.
>
> What are our viable options here via QEMU?

I don't mean QEMU, just sandbox emulators. They are pretty simple to
write and faster/easier to run and debug than qemu.

Regards,
Simon

Re: Proposal: U-Boot MTD (rewriting old story)

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 2551 bytes --]

On Thu, Aug 05, 2021 at 01:20:51PM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On Wed, 4 Aug 2021 at 12:49, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Wed, Aug 04, 2021 at 10:09:55AM -0600, Simon Glass wrote:
> > > Hi Jagan,
> > >
> > > On Tue, 3 Aug 2021 at 09:01, Jagan Teki <jagan@amarulasolutions.com> wrote:
> > > >
> > > > Yes, this is the old discussion and triggered again now since we had a
> > > > discussion on the last U-Boot contributor call.
> > > >
> > > > I will brief the main points here, as most of the details are
> > > > mentioned in previous threads.
> > > >
> > > > Here are the couple of old and new implementations on the SPI-NOR side.
> > > >
> > > > http://u-boot.10912.n7.nabble.com/PATCH-v10-00-27-dm-Generic-MTD-Subsystem-with-SPI-NOR-interface-tt315694.html#none
> > > > http://u-boot.10912.n7.nabble.com/PATCH-0-5-mtd-Implement-MTD-UCLASS-use-SPINOR-td419538.html#a419541
> > > > https://patchwork.ozlabs.org/project/uboot/cover/20200709111709.68904-1-jagan@amarulasolutions.com/
> > > >
> > > > Idea is to
> > > > 1. Create MTD_UCLASS, already present
> > > > 2. Create DM MTD Operations, above patchset does.
> > > > 3. Create Flash specific MTD UClass's like
> > > >     UCLASS_NAND
> > > >     UCLASS_SPI_NAND
> > > >     UCLASS_SPI_NOR
> > > >     UCLASS_NOR
> > > >     UCLASS_UBI
> > > > and register to UCLASS_MTD as interface slaves like how MMC, SCSI are
> > > > registered to UCLASS_BLK
> > > > 4. Sync only respective flash specific code, and drop __UBOOT__
> > > > 5. Interface mtd, sf, and nand commands to UCLASS_MTD.
> > >
> > > This all seems good to me.
> > >
> > > >
> > > > Hard points:
> > > > 1. Hard to arrange Linux specific code and drop unneeded _UBOOT_ code,
> > > > but possible.
> > > > 2. Requires a lot of testing.
> > >
> > > Let's add simple emulators for NAND, SPI_NAND and UBI. We already have
> > > one for SPI_NOR I believe. That will be a good long-term investment.
> >
> > What are our viable options here via QEMU?
> 
> I don't mean QEMU, just sandbox emulators. They are pretty simple to
> write and faster/easier to run and debug than qemu.

If something exists in QEMU today, there's nothing to write.  There are
many things sandbox is good for, and that I wish we made more use of (it
would be amazing if allyesconfig was viable for sandbox so everything
outside of arch/ could be static analyzed for example).  But when it
comes to driver testing, I really want to know what we can do with QEMU
first.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: Proposal: U-Boot MTD (rewriting old story)

[Bin Meng]

On Fri, Aug 6, 2021 at 3:40 AM Tom Rini <trini@konsulko.com> wrote:
>
> On Thu, Aug 05, 2021 at 01:20:51PM -0600, Simon Glass wrote:
> > Hi Tom,
> >
> > On Wed, 4 Aug 2021 at 12:49, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Wed, Aug 04, 2021 at 10:09:55AM -0600, Simon Glass wrote:
> > > > Hi Jagan,
> > > >
> > > > On Tue, 3 Aug 2021 at 09:01, Jagan Teki <jagan@amarulasolutions.com> wrote:
> > > > >
> > > > > Yes, this is the old discussion and triggered again now since we had a
> > > > > discussion on the last U-Boot contributor call.
> > > > >
> > > > > I will brief the main points here, as most of the details are
> > > > > mentioned in previous threads.
> > > > >
> > > > > Here are the couple of old and new implementations on the SPI-NOR side.
> > > > >
> > > > > http://u-boot.10912.n7.nabble.com/PATCH-v10-00-27-dm-Generic-MTD-Subsystem-with-SPI-NOR-interface-tt315694.html#none
> > > > > http://u-boot.10912.n7.nabble.com/PATCH-0-5-mtd-Implement-MTD-UCLASS-use-SPINOR-td419538.html#a419541
> > > > > https://patchwork.ozlabs.org/project/uboot/cover/20200709111709.68904-1-jagan@amarulasolutions.com/
> > > > >
> > > > > Idea is to
> > > > > 1. Create MTD_UCLASS, already present
> > > > > 2. Create DM MTD Operations, above patchset does.
> > > > > 3. Create Flash specific MTD UClass's like
> > > > >     UCLASS_NAND
> > > > >     UCLASS_SPI_NAND
> > > > >     UCLASS_SPI_NOR
> > > > >     UCLASS_NOR
> > > > >     UCLASS_UBI
> > > > > and register to UCLASS_MTD as interface slaves like how MMC, SCSI are
> > > > > registered to UCLASS_BLK
> > > > > 4. Sync only respective flash specific code, and drop __UBOOT__
> > > > > 5. Interface mtd, sf, and nand commands to UCLASS_MTD.
> > > >
> > > > This all seems good to me.
> > > >
> > > > >
> > > > > Hard points:
> > > > > 1. Hard to arrange Linux specific code and drop unneeded _UBOOT_ code,
> > > > > but possible.
> > > > > 2. Requires a lot of testing.
> > > >
> > > > Let's add simple emulators for NAND, SPI_NAND and UBI. We already have
> > > > one for SPI_NOR I believe. That will be a good long-term investment.
> > >
> > > What are our viable options here via QEMU?
> >
> > I don't mean QEMU, just sandbox emulators. They are pretty simple to
> > write and faster/easier to run and debug than qemu.
>
> If something exists in QEMU today, there's nothing to write.  There are
> many things sandbox is good for, and that I wish we made more use of (it
> would be amazing if allyesconfig was viable for sandbox so everything
> outside of arch/ could be static analyzed for example).  But when it
> comes to driver testing, I really want to know what we can do with QEMU
> first.

Agree, I believe we can do something on QEMU ARM/RISC-V virt (NOR) and
RISC-V sifive_u (spi-nor).

Regards,
Bin

Re: Proposal: U-Boot MTD (rewriting old story)

[Simon Glass]

Hi,

On Thu, 5 Aug 2021 at 19:07, Bin Meng <bmeng.cn@gmail.com> wrote:
>
> On Fri, Aug 6, 2021 at 3:40 AM Tom Rini <trini@konsulko.com> wrote:
> >
> > On Thu, Aug 05, 2021 at 01:20:51PM -0600, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Wed, 4 Aug 2021 at 12:49, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Wed, Aug 04, 2021 at 10:09:55AM -0600, Simon Glass wrote:
> > > > > Hi Jagan,
> > > > >
> > > > > On Tue, 3 Aug 2021 at 09:01, Jagan Teki <jagan@amarulasolutions.com> wrote:
> > > > > >
> > > > > > Yes, this is the old discussion and triggered again now since we had a
> > > > > > discussion on the last U-Boot contributor call.
> > > > > >
> > > > > > I will brief the main points here, as most of the details are
> > > > > > mentioned in previous threads.
> > > > > >
> > > > > > Here are the couple of old and new implementations on the SPI-NOR side.
> > > > > >
> > > > > > http://u-boot.10912.n7.nabble.com/PATCH-v10-00-27-dm-Generic-MTD-Subsystem-with-SPI-NOR-interface-tt315694.html#none
> > > > > > http://u-boot.10912.n7.nabble.com/PATCH-0-5-mtd-Implement-MTD-UCLASS-use-SPINOR-td419538.html#a419541
> > > > > > https://patchwork.ozlabs.org/project/uboot/cover/20200709111709.68904-1-jagan@amarulasolutions.com/
> > > > > >
> > > > > > Idea is to
> > > > > > 1. Create MTD_UCLASS, already present
> > > > > > 2. Create DM MTD Operations, above patchset does.
> > > > > > 3. Create Flash specific MTD UClass's like
> > > > > >     UCLASS_NAND
> > > > > >     UCLASS_SPI_NAND
> > > > > >     UCLASS_SPI_NOR
> > > > > >     UCLASS_NOR
> > > > > >     UCLASS_UBI
> > > > > > and register to UCLASS_MTD as interface slaves like how MMC, SCSI are
> > > > > > registered to UCLASS_BLK
> > > > > > 4. Sync only respective flash specific code, and drop __UBOOT__
> > > > > > 5. Interface mtd, sf, and nand commands to UCLASS_MTD.
> > > > >
> > > > > This all seems good to me.
> > > > >
> > > > > >
> > > > > > Hard points:
> > > > > > 1. Hard to arrange Linux specific code and drop unneeded _UBOOT_ code,
> > > > > > but possible.
> > > > > > 2. Requires a lot of testing.
> > > > >
> > > > > Let's add simple emulators for NAND, SPI_NAND and UBI. We already have
> > > > > one for SPI_NOR I believe. That will be a good long-term investment.
> > > >
> > > > What are our viable options here via QEMU?
> > >
> > > I don't mean QEMU, just sandbox emulators. They are pretty simple to
> > > write and faster/easier to run and debug than qemu.
> >
> > If something exists in QEMU today, there's nothing to write.  There are
> > many things sandbox is good for, and that I wish we made more use of (it
> > would be amazing if allyesconfig was viable for sandbox so everything
> > outside of arch/ could be static analyzed for example).  But when it
> > comes to driver testing, I really want to know what we can do with QEMU
> > first.
>
> Agree, I believe we can do something on QEMU ARM/RISC-V virt (NOR) and
> RISC-V sifive_u (spi-nor).

Actually I think people are missing the point with sandbox emulators.
They are not the same thing as qemu.

Emulators have these advantages that I can think of:

- very little code - see for example
https://github.com/u-boot/u-boot/blob/master/drivers/mmc/sandbox_mmc.c
    - it supports detecting MMC and reading/writing. I have a patch to
use a backing file on disk with another 20 lines

- easy to debug - can use gdb and everything is in U-Boot, no need to
chase through qemu code to understand what is wrong

- just enough features - we only implement what we need, keeping the
code simple and avoiding complexity

- can define emulators in the DT (as with sandbox_mmc and others) so
it is easy to add and remove emulated devices

- can run in CI very quickly (fraction of a second) rather than
needing qemu to spin up, etc.

This isn't about writing lots of useless code that is duplicating
functionality elsewhere. The sandbox emulators are much more flexible
and easy to use.

Regards,
Simon