Re: [PATCH 1/6] ARM: dts: am335x-phycore-som: Update NAND partition table

Re: [PATCH 1/6] ARM: dts: am335x-phycore-som: Update NAND partition table

[Brian Norris]

On Thu, Jan 05, 2017 at 09:18:45AM -0800, Tony Lindgren wrote:
> * Tony Lindgren <tony@atomide.com> [170105 07:37]:
> > * Teresa Remmet <t.remmet@phytec.de> [170105 06:57]:
> > > To improve NAND safety we updated the partition layout.
> > > Added barebox backup partition and removed kernel and oftree
> > > partition. They are kept in ubi now.
> > 
> > What about the users with earlier partition tables?
> > 
> > Please read about "flag day" changes, typically they are not
> > acceptable.
> 
> Adding Brian and Adam to Cc. Can you guys come up with some
> solution on this?

I don't have much context for this thread, and no I don't plan to solve
your problems for you. But I can provide tips!

> I'm suggesting we leave the kernel nodes empty and let u-boot
> populate them, so maybe you guys can discuss this on the related
> lists.

That's an option. I've worked with platforms that did something like
this, and that's really one of the only ways you can handle putting
partition information in the device tree. You're really hamstringing
yourself when you put all the partition information in the device tree.
And it's just dumb once that gets codified in the kernel source tree.

The best solution would be to try to migrate away from static device
tree representations of partition info entirely. UBI volumes are best
where possible. If not, then some other kind of on-flash data structures
(along the lines of a GPT) with a corresponding MTD partition parser is
an OK alternative. Unfortunately, there isn't any good standard format
for this on MTD, so it's typically all custom -- and so people use the
easiest approach: device tree. And it's even more difficult with NAND,
which has reliability problems, especially with static data (e.g., read
disturb).

Anyway, the parser solution is helpful only if one can properly fix the
"flag day" first. And it requires a little bit more work to be generally
useful; I posted some work for this over a year ago, but bikeshedding
brought it down.

> The rest of the series looks fine to me so applying it into
> omap-for-v4.11/dt.

Brian

Re: [PATCH 1/6] ARM: dts: am335x-phycore-som: Update NAND partition table

[Teresa Remmet]

Hello Brian,

Am Donnerstag, den 05.01.2017, 09:56 -0800 schrieb Brian Norris:
> On Thu, Jan 05, 2017 at 09:18:45AM -0800, Tony Lindgren wrote:
> > 
> > * Tony Lindgren <tony@atomide.com> [170105 07:37]:
> > > 
> > > * Teresa Remmet <t.remmet@phytec.de> [170105 06:57]:
> > > > 
> > > > To improve NAND safety we updated the partition layout.
> > > > Added barebox backup partition and removed kernel and oftree
> > > > partition. They are kept in ubi now.
> > > What about the users with earlier partition tables?
> > > 
> > > Please read about "flag day" changes, typically they are not
> > > acceptable.
> > Adding Brian and Adam to Cc. Can you guys come up with some
> > solution on this?
> I don't have much context for this thread, and no I don't plan to
> solve
> your problems for you. But I can provide tips!
> 
> > 
> > I'm suggesting we leave the kernel nodes empty and let u-boot
> > populate them, so maybe you guys can discuss this on the related
> > lists.
> That's an option. I've worked with platforms that did something like
> this, and that's really one of the only ways you can handle putting
> partition information in the device tree. You're really hamstringing
> yourself when you put all the partition information in the device
> tree.
> And it's just dumb once that gets codified in the kernel source tree.
> 

In our case the bootloader does pass the partition table to the kernel.
So it gets overwritten anyway. This was just more for backup, 
if someone uses a different bootloader. But I'm fine with removing the
nand partition table completely from the kernel device tree.
Same with the SPI nor partition table.

I will send patches for this.

Regards,
Teresa

> The best solution would be to try to migrate away from static device
> tree representations of partition info entirely. UBI volumes are best
> where possible. If not, then some other kind of on-flash data
> structures
> (along the lines of a GPT) with a corresponding MTD partition parser
> is
> an OK alternative. Unfortunately, there isn't any good standard
> format
> for this on MTD, so it's typically all custom -- and so people use
> the
> easiest approach: device tree. And it's even more difficult with
> NAND,
> which has reliability problems, especially with static data (e.g.,
> read
> disturb).
> 
> Anyway, the parser solution is helpful only if one can properly fix
> the
> "flag day" first. And it requires a little bit more work to be
> generally
> useful; I posted some work for this over a year ago, but bikeshedding
> brought it down.
> 
> > 
> > The rest of the series looks fine to me so applying it into
> > omap-for-v4.11/dt.
> Brian
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH 1/6] ARM: dts: am335x-phycore-som: Update NAND partition table

[Tony Lindgren]

* Teresa Remmet <t.remmet@phytec.de> [170106 01:28]:
> Hello Brian,
> 
> Am Donnerstag, den 05.01.2017, 09:56 -0800 schrieb Brian Norris:
> > On Thu, Jan 05, 2017 at 09:18:45AM -0800, Tony Lindgren wrote:
> > > 
> > > * Tony Lindgren <tony@atomide.com> [170105 07:37]:
> > > > 
> > > > * Teresa Remmet <t.remmet@phytec.de> [170105 06:57]:
> > > > > 
> > > > > To improve NAND safety we updated the partition layout.
> > > > > Added barebox backup partition and removed kernel and oftree
> > > > > partition. They are kept in ubi now.
> > > > What about the users with earlier partition tables?
> > > > 
> > > > Please read about "flag day" changes, typically they are not
> > > > acceptable.
> > > Adding Brian and Adam to Cc. Can you guys come up with some
> > > solution on this?
> > I don't have much context for this thread, and no I don't plan to
> > solve
> > your problems for you. But I can provide tips!
> > 
> > > 
> > > I'm suggesting we leave the kernel nodes empty and let u-boot
> > > populate them, so maybe you guys can discuss this on the related
> > > lists.
> > That's an option. I've worked with platforms that did something like
> > this, and that's really one of the only ways you can handle putting
> > partition information in the device tree. You're really hamstringing
> > yourself when you put all the partition information in the device
> > tree.
> > And it's just dumb once that gets codified in the kernel source tree.
> > 
> 
> In our case the bootloader does pass the partition table to the kernel.
> So it gets overwritten anyway. This was just more for backup, 
> if someone uses a different bootloader. But I'm fine with removing the
> nand partition table completely from the kernel device tree.
> Same with the SPI nor partition table.
> 
> I will send patches for this.

OK thanks! Also thank you Brian for your comments.

Regards,

Tony

Re: [PATCH 1/6] ARM: dts: am335x-phycore-som: Update NAND partition table

[Adam Ford]

On Fri, Jan 6, 2017 at 10:02 AM, Tony Lindgren <tony@atomide.com> wrote:
> * Teresa Remmet <t.remmet@phytec.de> [170106 01:28]:
>> Hello Brian,
>>
>> Am Donnerstag, den 05.01.2017, 09:56 -0800 schrieb Brian Norris:
>> > On Thu, Jan 05, 2017 at 09:18:45AM -0800, Tony Lindgren wrote:
>> > >
>> > > * Tony Lindgren <tony@atomide.com> [170105 07:37]:
>> > > >
>> > > > * Teresa Remmet <t.remmet@phytec.de> [170105 06:57]:
>> > > > >
>> > > > > To improve NAND safety we updated the partition layout.
>> > > > > Added barebox backup partition and removed kernel and oftree
>> > > > > partition. They are kept in ubi now.
>> > > > What about the users with earlier partition tables?
>> > > >
>> > > > Please read about "flag day" changes, typically they are not
>> > > > acceptable.
>> > > Adding Brian and Adam to Cc. Can you guys come up with some
>> > > solution on this?
>> > I don't have much context for this thread, and no I don't plan to
>> > solve
>> > your problems for you. But I can provide tips!
>> >
>> > >
>> > > I'm suggesting we leave the kernel nodes empty and let u-boot
>> > > populate them, so maybe you guys can discuss this on the related
>> > > lists.
>> > That's an option. I've worked with platforms that did something like
>> > this, and that's really one of the only ways you can handle putting
>> > partition information in the device tree. You're really hamstringing
>> > yourself when you put all the partition information in the device
>> > tree.
>> > And it's just dumb once that gets codified in the kernel source tree.
>> >
>>
>> In our case the bootloader does pass the partition table to the kernel.
>> So it gets overwritten anyway. This was just more for backup,
>> if someone uses a different bootloader. But I'm fine with removing the
>> nand partition table completely from the kernel device tree.
>> Same with the SPI nor partition table.
>>
>> I will send patches for this.
>
> OK thanks! Also thank you Brian for your comments.
>

Tony - I tested leaving the partition info as-is with the updates from
the bootloader and it works.  Would you prefer that I match Brian and
remove the partition table completely, or should I just leave my board
alone?

I am good either way.

> Regards,
>
> Tony

adam

Re: [PATCH 1/6] ARM: dts: am335x-phycore-som: Update NAND partition table

[Tony Lindgren]

* Adam Ford <aford173@gmail.com> [170106 08:06]:
> On Fri, Jan 6, 2017 at 10:02 AM, Tony Lindgren <tony@atomide.com> wrote:
> > * Teresa Remmet <t.remmet@phytec.de> [170106 01:28]:
> >> Hello Brian,
> >>
> >> Am Donnerstag, den 05.01.2017, 09:56 -0800 schrieb Brian Norris:
> >> > On Thu, Jan 05, 2017 at 09:18:45AM -0800, Tony Lindgren wrote:
> >> > >
> >> > > * Tony Lindgren <tony@atomide.com> [170105 07:37]:
> >> > > >
> >> > > > * Teresa Remmet <t.remmet@phytec.de> [170105 06:57]:
> >> > > > >
> >> > > > > To improve NAND safety we updated the partition layout.
> >> > > > > Added barebox backup partition and removed kernel and oftree
> >> > > > > partition. They are kept in ubi now.
> >> > > > What about the users with earlier partition tables?
> >> > > >
> >> > > > Please read about "flag day" changes, typically they are not
> >> > > > acceptable.
> >> > > Adding Brian and Adam to Cc. Can you guys come up with some
> >> > > solution on this?
> >> > I don't have much context for this thread, and no I don't plan to
> >> > solve
> >> > your problems for you. But I can provide tips!
> >> >
> >> > >
> >> > > I'm suggesting we leave the kernel nodes empty and let u-boot
> >> > > populate them, so maybe you guys can discuss this on the related
> >> > > lists.
> >> > That's an option. I've worked with platforms that did something like
> >> > this, and that's really one of the only ways you can handle putting
> >> > partition information in the device tree. You're really hamstringing
> >> > yourself when you put all the partition information in the device
> >> > tree.
> >> > And it's just dumb once that gets codified in the kernel source tree.
> >> >
> >>
> >> In our case the bootloader does pass the partition table to the kernel.
> >> So it gets overwritten anyway. This was just more for backup,
> >> if someone uses a different bootloader. But I'm fine with removing the
> >> nand partition table completely from the kernel device tree.
> >> Same with the SPI nor partition table.
> >>
> >> I will send patches for this.
> >
> > OK thanks! Also thank you Brian for your comments.
> >
> 
> Tony - I tested leaving the partition info as-is with the updates from
> the bootloader and it works.  Would you prefer that I match Brian and
> remove the partition table completely, or should I just leave my board
> alone?
> 
> I am good either way.

OK. How about let's remove the partitions from kernel for v4.11 as
clean-up then for cases where the bootloader might change them?

Regards,

Tony

Re: [PATCH 1/6] ARM: dts: am335x-phycore-som: Update NAND partition table

[Brian Norris]

On Fri, Jan 06, 2017 at 08:14:22AM -0800, Tony Lindgren wrote:
> * Adam Ford <aford173@gmail.com> [170106 08:06]:
> > On Fri, Jan 6, 2017 at 10:02 AM, Tony Lindgren <tony@atomide.com> wrote:
> > > * Teresa Remmet <t.remmet@phytec.de> [170106 01:28]:
> > >> Am Donnerstag, den 05.01.2017, 09:56 -0800 schrieb Brian Norris:
> > >> > On Thu, Jan 05, 2017 at 09:18:45AM -0800, Tony Lindgren wrote:
> > >> > > I'm suggesting we leave the kernel nodes empty and let u-boot
> > >> > > populate them, so maybe you guys can discuss this on the related
> > >> > > lists.
> > >> > That's an option. I've worked with platforms that did something like
> > >> > this, and that's really one of the only ways you can handle putting
> > >> > partition information in the device tree. You're really hamstringing
> > >> > yourself when you put all the partition information in the device
> > >> > tree.
> > >> > And it's just dumb once that gets codified in the kernel source tree.
> > >> >
> > >>
> > >> In our case the bootloader does pass the partition table to the kernel.
> > >> So it gets overwritten anyway. This was just more for backup,
> > >> if someone uses a different bootloader. But I'm fine with removing the
> > >> nand partition table completely from the kernel device tree.
> > >> Same with the SPI nor partition table.

Ah, well if these are essentially just a backup, then why do they need
changed at all? To be more precise about my "dumb" statement: it seems
unwise to assume that all systems using a particular board will have the
same partition layout. So *relying* on the in-kernel DTS(I) files to
have a universal partition layout would likely cause problems.

I don't have much problem with keeping a sample layout there as a
backup, if you find it useful. But it seems like it will be hard to
argue that you can ever change it in the future.

Anyway, the ofpart mechanism is there, and it's fine to use it, but it's
up to you to understand the implications and manage the backwards
compatibility problems :)

> > >> I will send patches for this.
> > >
> > > OK thanks! Also thank you Brian for your comments.
> > >
> > 
> > Tony - I tested leaving the partition info as-is with the updates from
> > the bootloader and it works.  Would you prefer that I match Brian and
> > remove the partition table completely, or should I just leave my board
> > alone?
> > 
> > I am good either way.
> 
> OK. How about let's remove the partitions from kernel for v4.11 as
> clean-up then for cases where the bootloader might change them?

I don't have much stake in this but...if there were any systems using
the backup layout (i.e., the in-kernel partition layout), wouldn't that
break them? Is there a problem with just leaving them alone?

Brian

Re: [PATCH 1/6] ARM: dts: am335x-phycore-som: Update NAND partition table

[Tony Lindgren]

* Brian Norris <briannorris@chromium.org> [170106 10:21]:
> On Fri, Jan 06, 2017 at 08:14:22AM -0800, Tony Lindgren wrote:
> > * Adam Ford <aford173@gmail.com> [170106 08:06]:
> > > On Fri, Jan 6, 2017 at 10:02 AM, Tony Lindgren <tony@atomide.com> wrote:
> > > > * Teresa Remmet <t.remmet@phytec.de> [170106 01:28]:
> > > >> Am Donnerstag, den 05.01.2017, 09:56 -0800 schrieb Brian Norris:
> > > >> > On Thu, Jan 05, 2017 at 09:18:45AM -0800, Tony Lindgren wrote:
> > > >> > > I'm suggesting we leave the kernel nodes empty and let u-boot
> > > >> > > populate them, so maybe you guys can discuss this on the related
> > > >> > > lists.
> > > >> > That's an option. I've worked with platforms that did something like
> > > >> > this, and that's really one of the only ways you can handle putting
> > > >> > partition information in the device tree. You're really hamstringing
> > > >> > yourself when you put all the partition information in the device
> > > >> > tree.
> > > >> > And it's just dumb once that gets codified in the kernel source tree.
> > > >> >
> > > >>
> > > >> In our case the bootloader does pass the partition table to the kernel.
> > > >> So it gets overwritten anyway. This was just more for backup,
> > > >> if someone uses a different bootloader. But I'm fine with removing the
> > > >> nand partition table completely from the kernel device tree.
> > > >> Same with the SPI nor partition table.
> 
> Ah, well if these are essentially just a backup, then why do they need
> changed at all? To be more precise about my "dumb" statement: it seems
> unwise to assume that all systems using a particular board will have the
> same partition layout. So *relying* on the in-kernel DTS(I) files to
> have a universal partition layout would likely cause problems.
> 
> I don't have much problem with keeping a sample layout there as a
> backup, if you find it useful. But it seems like it will be hard to
> argue that you can ever change it in the future.
> 
> Anyway, the ofpart mechanism is there, and it's fine to use it, but it's
> up to you to understand the implications and manage the backwards
> compatibility problems :)
> 
> > > >> I will send patches for this.
> > > >
> > > > OK thanks! Also thank you Brian for your comments.
> > > >
> > > 
> > > Tony - I tested leaving the partition info as-is with the updates from
> > > the bootloader and it works.  Would you prefer that I match Brian and
> > > remove the partition table completely, or should I just leave my board
> > > alone?
> > > 
> > > I am good either way.
> > 
> > OK. How about let's remove the partitions from kernel for v4.11 as
> > clean-up then for cases where the bootloader might change them?
> 
> I don't have much stake in this but...if there were any systems using
> the backup layout (i.e., the in-kernel partition layout), wouldn't that
> break them? Is there a problem with just leaving them alone?

Yeah I guess it depends on the device and it's bootloader versions.

Regards,

Tony

Re: [PATCH 1/6] ARM: dts: am335x-phycore-som: Update NAND partition table

[Ladislav Michl]

On Thu, Jan 05, 2017 at 09:56:20AM -0800, Brian Norris wrote:
[snip]
> The best solution would be to try to migrate away from static device
> tree representations of partition info entirely. UBI volumes are best
> where possible. If not, then some other kind of on-flash data structures
> (along the lines of a GPT) with a corresponding MTD partition parser is
> an OK alternative. Unfortunately, there isn't any good standard format
> for this on MTD, so it's typically all custom -- and so people use the
> easiest approach: device tree. And it's even more difficult with NAND,
> which has reliability problems, especially with static data (e.g., read
> disturb).

Just as a side note, there is some work in this area:
https://www.mail-archive.com/u-boot@lists.denx.de/msg232759.html

> Anyway, the parser solution is helpful only if one can properly fix the
> "flag day" first. And it requires a little bit more work to be generally
> useful; I posted some work for this over a year ago, but bikeshedding
> brought it down.

Best regards,
	ladis