Porting Linux's MTD/NAND changes into U-Boot

Porting Linux's MTD/NAND changes into U-Boot

[Boris Brezillon]

Hi Scott,

I've recently contributed a lot of MTD/NAND related patches (and intend
to continue doing so). Some of them are transversal changes touching the
MTD and NAND framework internals, which implies patching all NAND
drivers along with the core changes.

All those changes are required to properly handle modern NANDs (MLC/TLC
NANDs), and I need them to add proper NAND support to the sunxi
platform (and more particularly to the C.H.I.P from NextThing Co.).

So my question is, how should I port those changes to U-Boot? I see
that your doing "synchronization commits", but in my case this mean
including a bunch of driver specific changes into this "sync commit".

I think it's also worth mentioning that I plan to heavily rework the
Linux NAND framework to improve NAND performances on modern NAND
controllers and clarify the NAND chip / NAND controller concepts, and
other people are also working on merging the BBT code of the NAND and
OneNAND framework. Which unfortunately means that we're not done porting
invasive changes to U-Boot :-/.

Any advice is welcome.

Thanks,

Boris

-- 
Boris Brezillon, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [U-Boot] Porting Linux's MTD/NAND changes into U-Boot

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 1534 bytes --]

On Mon, Apr 25, 2016 at 04:43:14PM +0200, Boris Brezillon wrote:

> Hi Scott,
> 
> I've recently contributed a lot of MTD/NAND related patches (and intend
> to continue doing so). Some of them are transversal changes touching the
> MTD and NAND framework internals, which implies patching all NAND
> drivers along with the core changes.
> 
> All those changes are required to properly handle modern NANDs (MLC/TLC
> NANDs), and I need them to add proper NAND support to the sunxi
> platform (and more particularly to the C.H.I.P from NextThing Co.).
> 
> So my question is, how should I port those changes to U-Boot? I see
> that your doing "synchronization commits", but in my case this mean
> including a bunch of driver specific changes into this "sync commit".
> 
> I think it's also worth mentioning that I plan to heavily rework the
> Linux NAND framework to improve NAND performances on modern NAND
> controllers and clarify the NAND chip / NAND controller concepts, and
> other people are also working on merging the BBT code of the NAND and
> OneNAND framework. Which unfortunately means that we're not done porting
> invasive changes to U-Boot :-/.
> 
> Any advice is welcome.

I suppose my first suggestion would be to sync the kernel back into
U-Boot more frequently.  With our bi-monthly release cycle it shouldn't
be too hard to pick a window to grab the current kernel release and
bring it over.  I think the more stuff we let build up prior to syncing
the harder it will be.

-- 
Tom

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [U-Boot] Porting Linux's MTD/NAND changes into U-Boot

[Boris Brezillon]

On Mon, 25 Apr 2016 13:36:46 -0400
Tom Rini <trini@konsulko.com> wrote:

> On Mon, Apr 25, 2016 at 04:43:14PM +0200, Boris Brezillon wrote:
> 
> > Hi Scott,
> > 
> > I've recently contributed a lot of MTD/NAND related patches (and intend
> > to continue doing so). Some of them are transversal changes touching the
> > MTD and NAND framework internals, which implies patching all NAND
> > drivers along with the core changes.
> > 
> > All those changes are required to properly handle modern NANDs (MLC/TLC
> > NANDs), and I need them to add proper NAND support to the sunxi
> > platform (and more particularly to the C.H.I.P from NextThing Co.).
> > 
> > So my question is, how should I port those changes to U-Boot? I see
> > that your doing "synchronization commits", but in my case this mean
> > including a bunch of driver specific changes into this "sync commit".
> > 
> > I think it's also worth mentioning that I plan to heavily rework the
> > Linux NAND framework to improve NAND performances on modern NAND
> > controllers and clarify the NAND chip / NAND controller concepts, and
> > other people are also working on merging the BBT code of the NAND and
> > OneNAND framework. Which unfortunately means that we're not done porting
> > invasive changes to U-Boot :-/.
> > 
> > Any advice is welcome.  
> 
> I suppose my first suggestion would be to sync the kernel back into
> U-Boot more frequently.  With our bi-monthly release cycle it shouldn't
> be too hard to pick a window to grab the current kernel release and
> bring it over.  I think the more stuff we let build up prior to syncing
> the harder it will be.
> 

Okay, so this means I should be the one back-porting Linux changes into
U-Boot? I was planning on porting my current work on the sunxi NAND
driver (including all the core dependencies) and submitting it to
U-Boot, but honestly, I don't see how I can keep things in sync in the
long run. From my experience, porting things to U-Boot is never as
easy at it seems, and I don't think I'll have the time to maintain that
by myself. Scott, what's your policy regarding Linux -> U-Boot syncs?
Can I expect some help from your side?


-- 
Boris Brezillon, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [U-Boot] Porting Linux's MTD/NAND changes into U-Boot

[Scott Wood]

On Mon, 2016-04-25 at 22:53 +0200, Boris Brezillon wrote:
> On Mon, 25 Apr 2016 13:36:46 -0400
> Tom Rini <trini@konsulko.com> wrote:
> 
> > On Mon, Apr 25, 2016 at 04:43:14PM +0200, Boris Brezillon wrote:
> > 
> > > Hi Scott,
> > > 
> > > I've recently contributed a lot of MTD/NAND related patches (and intend
> > > to continue doing so). Some of them are transversal changes touching the
> > > MTD and NAND framework internals, which implies patching all NAND
> > > drivers along with the core changes.
> > > 
> > > All those changes are required to properly handle modern NANDs (MLC/TLC
> > > NANDs), and I need them to add proper NAND support to the sunxi
> > > platform (and more particularly to the C.H.I.P from NextThing Co.).
> > > 
> > > So my question is, how should I port those changes to U-Boot? I see
> > > that your doing "synchronization commits", but in my case this mean
> > > including a bunch of driver specific changes into this "sync commit".
> > > 
> > > I think it's also worth mentioning that I plan to heavily rework the
> > > Linux NAND framework to improve NAND performances on modern NAND
> > > controllers and clarify the NAND chip / NAND controller concepts, and
> > > other people are also working on merging the BBT code of the NAND and
> > > OneNAND framework. Which unfortunately means that we're not done porting
> > > invasive changes to U-Boot :-/.
> > > 
> > > Any advice is welcome.  
> > 
> > I suppose my first suggestion would be to sync the kernel back into
> > U-Boot more frequently.  With our bi-monthly release cycle it shouldn't
> > be too hard to pick a window to grab the current kernel release and
> > bring it over.  I think the more stuff we let build up prior to syncing
> > the harder it will be.
> > 
> 
> Okay, so this means I should be the one back-porting Linux changes into
> U-Boot? I was planning on porting my current work on the sunxi NAND
> driver (including all the core dependencies) and submitting it to
> U-Boot, but honestly, I don't see how I can keep things in sync in the
> long run. From my experience, porting things to U-Boot is never as
> easy at it seems, and I don't think I'll have the time to maintain that
> by myself. Scott, what's your policy regarding Linux -> U-Boot syncs?
> Can I expect some help from your side?

I usually do syncs when someone complains about needing a sync.  Every release
seems a bit much.  I'll try to do one soon.

-Scott

Re: [U-Boot] Porting Linux's MTD/NAND changes into U-Boot

[Boris Brezillon]

Hi Scott,

On Tue, 03 May 2016 00:06:45 -0500
Scott Wood <oss@buserror.net> wrote:

> On Mon, 2016-04-25 at 22:53 +0200, Boris Brezillon wrote:
> > On Mon, 25 Apr 2016 13:36:46 -0400
> > Tom Rini <trini@konsulko.com> wrote:
> >   
> > > On Mon, Apr 25, 2016 at 04:43:14PM +0200, Boris Brezillon wrote:
> > >   
> > > > Hi Scott,
> > > > 
> > > > I've recently contributed a lot of MTD/NAND related patches (and intend
> > > > to continue doing so). Some of them are transversal changes touching the
> > > > MTD and NAND framework internals, which implies patching all NAND
> > > > drivers along with the core changes.
> > > > 
> > > > All those changes are required to properly handle modern NANDs (MLC/TLC
> > > > NANDs), and I need them to add proper NAND support to the sunxi
> > > > platform (and more particularly to the C.H.I.P from NextThing Co.).
> > > > 
> > > > So my question is, how should I port those changes to U-Boot? I see
> > > > that your doing "synchronization commits", but in my case this mean
> > > > including a bunch of driver specific changes into this "sync commit".
> > > > 
> > > > I think it's also worth mentioning that I plan to heavily rework the
> > > > Linux NAND framework to improve NAND performances on modern NAND
> > > > controllers and clarify the NAND chip / NAND controller concepts, and
> > > > other people are also working on merging the BBT code of the NAND and
> > > > OneNAND framework. Which unfortunately means that we're not done porting
> > > > invasive changes to U-Boot :-/.
> > > > 
> > > > Any advice is welcome.    
> > > 
> > > I suppose my first suggestion would be to sync the kernel back into
> > > U-Boot more frequently.  With our bi-monthly release cycle it shouldn't
> > > be too hard to pick a window to grab the current kernel release and
> > > bring it over.  I think the more stuff we let build up prior to syncing
> > > the harder it will be.
> > >   
> > 
> > Okay, so this means I should be the one back-porting Linux changes into
> > U-Boot? I was planning on porting my current work on the sunxi NAND
> > driver (including all the core dependencies) and submitting it to
> > U-Boot, but honestly, I don't see how I can keep things in sync in the
> > long run. From my experience, porting things to U-Boot is never as
> > easy at it seems, and I don't think I'll have the time to maintain that
> > by myself. Scott, what's your policy regarding Linux -> U-Boot syncs?
> > Can I expect some help from your side?  
> 
> I usually do syncs when someone complains about needing a sync.  Every release
> seems a bit much.  I'll try to do one soon.

Great! Let me know if you need some help.

Thanks,

Boris

-- 
Boris Brezillon, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [U-Boot] Porting Linux's MTD/NAND changes into U-Boot

[Boris Brezillon]

Hi Scott,

On Tue, 03 May 2016 00:06:45 -0500
Scott Wood <oss@buserror.net> wrote:

> On Mon, 2016-04-25 at 22:53 +0200, Boris Brezillon wrote:
> > On Mon, 25 Apr 2016 13:36:46 -0400
> > Tom Rini <trini@konsulko.com> wrote:
> >   
> > > On Mon, Apr 25, 2016 at 04:43:14PM +0200, Boris Brezillon wrote:
> > >   
> > > > Hi Scott,
> > > > 
> > > > I've recently contributed a lot of MTD/NAND related patches (and intend
> > > > to continue doing so). Some of them are transversal changes touching the
> > > > MTD and NAND framework internals, which implies patching all NAND
> > > > drivers along with the core changes.
> > > > 
> > > > All those changes are required to properly handle modern NANDs (MLC/TLC
> > > > NANDs), and I need them to add proper NAND support to the sunxi
> > > > platform (and more particularly to the C.H.I.P from NextThing Co.).
> > > > 
> > > > So my question is, how should I port those changes to U-Boot? I see
> > > > that your doing "synchronization commits", but in my case this mean
> > > > including a bunch of driver specific changes into this "sync commit".
> > > > 
> > > > I think it's also worth mentioning that I plan to heavily rework the
> > > > Linux NAND framework to improve NAND performances on modern NAND
> > > > controllers and clarify the NAND chip / NAND controller concepts, and
> > > > other people are also working on merging the BBT code of the NAND and
> > > > OneNAND framework. Which unfortunately means that we're not done porting
> > > > invasive changes to U-Boot :-/.
> > > > 
> > > > Any advice is welcome.    
> > > 
> > > I suppose my first suggestion would be to sync the kernel back into
> > > U-Boot more frequently.  With our bi-monthly release cycle it shouldn't
> > > be too hard to pick a window to grab the current kernel release and
> > > bring it over.  I think the more stuff we let build up prior to syncing
> > > the harder it will be.
> > >   
> > 
> > Okay, so this means I should be the one back-porting Linux changes into
> > U-Boot? I was planning on porting my current work on the sunxi NAND
> > driver (including all the core dependencies) and submitting it to
> > U-Boot, but honestly, I don't see how I can keep things in sync in the
> > long run. From my experience, porting things to U-Boot is never as
> > easy at it seems, and I don't think I'll have the time to maintain that
> > by myself. Scott, what's your policy regarding Linux -> U-Boot syncs?
> > Can I expect some help from your side?  
> 
> I usually do syncs when someone complains about needing a sync.  Every release
> seems a bit much.  I'll try to do one soon.

Sorry to bother you with that, but can I ask when you plan to do this
sync (I guess you'll sync with 4.6)? As I explained, I depend on
features introduced in 4.6 to make the sunxi NAND driver usable on
sunxi based boards (the concept of data randomizer/scrambler is
required on some MLC NANDs, and most Allwinner boards embed this
kind of NAND chips).

Don't hesitate to let me know if you need some help.

Thanks,

Boris

-- 
Boris Brezillon, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

Re: [U-Boot] Porting Linux's MTD/NAND changes into U-Boot

[Scott Wood]

On Mon, 2016-05-16 at 14:02 +0200, Boris Brezillon wrote:
> Hi Scott,
> 
> On Tue, 03 May 2016 00:06:45 -0500
> Scott Wood <oss@buserror.net> wrote:
> 
> > I usually do syncs when someone complains about needing a sync.  Every
> > release
> > seems a bit much.  I'll try to do one soon.
> 
> Sorry to bother you with that, but can I ask when you plan to do this
> sync (I guess you'll sync with 4.6)? As I explained, I depend on
> features introduced in 4.6 to make the sunxi NAND driver usable on
> sunxi based boards (the concept of data randomizer/scrambler is
> required on some MLC NANDs, and most Allwinner boards embed this
> kind of NAND chips).
> 
> Don't hesitate to let me know if you need some help.

I'm working on it now.  Yes, with 4.6.

-Scott

Re: [U-Boot] Porting Linux's MTD/NAND changes into U-Boot

[Heiko Schocher]

Hello Tom, Boris,

Am 25.04.2016 um 19:36 schrieb Tom Rini:
> On Mon, Apr 25, 2016 at 04:43:14PM +0200, Boris Brezillon wrote:
>
>> Hi Scott,
>>
>> I've recently contributed a lot of MTD/NAND related patches (and intend
>> to continue doing so). Some of them are transversal changes touching the
>> MTD and NAND framework internals, which implies patching all NAND
>> drivers along with the core changes.
>>
>> All those changes are required to properly handle modern NANDs (MLC/TLC
>> NANDs), and I need them to add proper NAND support to the sunxi
>> platform (and more particularly to the C.H.I.P from NextThing Co.).
>>
>> So my question is, how should I port those changes to U-Boot? I see
>> that your doing "synchronization commits", but in my case this mean
>> including a bunch of driver specific changes into this "sync commit".
>>
>> I think it's also worth mentioning that I plan to heavily rework the
>> Linux NAND framework to improve NAND performances on modern NAND
>> controllers and clarify the NAND chip / NAND controller concepts, and
>> other people are also working on merging the BBT code of the NAND and
>> OneNAND framework. Which unfortunately means that we're not done porting
>> invasive changes to U-Boot :-/.
>>
>> Any advice is welcome.
>
> I suppose my first suggestion would be to sync the kernel back into
> U-Boot more frequently.  With our bi-monthly release cycle it shouldn't
> be too hard to pick a window to grab the current kernel release and
> bring it over.  I think the more stuff we let build up prior to syncing
> the harder it will be.

Yes, the same applies for UBI/UBIFS. I have no real strategy yet for
this. I think also, we need when doing such a sync a better testframe-
work, which tests on ideally a lot of boards the new versions ...

Without this, it is very difficult not to break existing boards with
such a sync ... especially MTD is tricky, I think, because a lot of
boards use it ...

I know I am nerving with my tbot [1], but it is may a solution for this.

Also, as the UBI/UIFS case may show, it is not always a trivial job
to do such a sync, as concepts in Linux changes ...

bye,
Heiko
[1] https://github.com/hsdenx/tbot
-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

Re: [U-Boot] Porting Linux's MTD/NAND changes into U-Boot

[Richard Weinberger]

Am 26.04.2016 um 07:17 schrieb Heiko Schocher:
> Also, as the UBI/UIFS case may show, it is not always a trivial job
> to do such a sync, as concepts in Linux changes ...

While we are at it, how do you QA?
Did you ever port port UBI or MTD tests to u-boot and let them run?
Maybe there are more issues hidden which didn't came up in quick "builds and boots a kernel" tests. ;-)
As you offer the full UBI and UBIFS feature sets advanced users may get surprised.

Thanks,
//richard

Re: [U-Boot] Porting Linux's MTD/NAND changes into U-Boot

[Heiko Schocher]

Hello Richard,

Am 26.04.2016 um 09:20 schrieb Richard Weinberger:
> Am 26.04.2016 um 07:17 schrieb Heiko Schocher:
>> Also, as the UBI/UIFS case may show, it is not always a trivial job
>> to do such a sync, as concepts in Linux changes ...
>
> While we are at it, how do you QA?

I do some very basic ubi/ubifs tests with tbot [1] on the aristainetos2 [2]
and dxr2 board [3].

> Did you ever port port UBI or MTD tests to u-boot and let them run?

No, patches are welcome.

> Maybe there are more issues hidden which didn't came up in quick "builds and boots a kernel" tests. ;-)
> As you offer the full UBI and UBIFS feature sets advanced users may get surprised.

Yes, possible.

bye,
Heiko

[1] https://github.com/hsdenx/tbot
[2] U-Boot cyclic tests on the aristainetos2 board
     (used UBI on sf nor and nand, UBIFS on the nand, U-Boot test only)
     http://xeidos.ddns.net/buildbot/builders/ari_ubi/builds/50
[3] U-Boot / Linux UBI/UBIFS tests, last log:
     http://xeidos.ddns.net/buildbot/builders/dxr2/builds/12/steps/shell/logs/tbotlog

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany