Re: Linux MTD: Per Partition ECC

Re: Linux MTD: Per Partition ECC

[Andreas Bießmann]

[-- Attachment #1: Type: text/plain, Size: 3192 bytes --]

Dear star@gmx.li,

cc linux-mtd, one may have more information on that topic there.

On 02/11/2014 12:58 PM, star@gmx.li wrote:
> Dear Peter, dear Andreas,
> 
> I just found this during my research:
> 
> http://lists.infradead.org/pipermail/linux-mtd/2012-September/044120.html 
>
> I  have simular Problem, to update a device which runs on 1 bit SW-ECC out of linux to 'on-die-ECC'
> 
> I duplicated the board partition table (Linux 2.6.33) in a first step. Now each partition can be access  with 2 names
> /dev/mtd4 or /dev/mtd8 as example.

D'oh why the hell should one do that? I dunno if this is sane with
synchronisation between different access layers (e.g. two different
instances of filesystem).

> But how can I addict a ECC scheme to a certain partition? Do you think this is a possible way?
> So access /dev/mtd4 with software ECC and /dev/mtd8 with on-die-ecc?

AFAIK there is no way to do per partition ecc currently. So there is no
way to support this without changing the whole mtd framework.
But maybe there was some movement since my last check. Maybe one of the
mtd developers could comment on that.

Nevertheless the ECC scheme of Linux 1Bit hamming SW ECC is likely to be
different than the Micron (or whatever manufacturer you use)
'on-die-ECC', therefore there is no chance to support 'partition A' and
'partition B' working on the very same hardware with these different ECC
schemes. You can only support scheme A _or_ scheme B at a time. I mean
for the whole process of storing the data and reading it out of the
device. Is the data physically written with scheme A it would fail to
read with scheme B and vice versa.

> Andreas can you provide the complete patch for you version? 

You just found the complete patch [2] if you red the link mentioned above.

> Problem for me is, I am always running on a mtd partition.

Well, my patch provides a solution for a very special case. Doing system
update of embedded devices in the field and switching the ECC scheme of
one partition (the first one, which boot ROM is reading) to OMAP3 1 bit
hamming HW ECC scheme while the rest of the device is 8 bit BCH SW ECC
scheme (it is 'HW-assisted', cause of missing ELM in OMAP3; I know there
were some patches cleaning this up, dunno the current state).
I can do that cause we kexec the update firmware which brings the
omap-nand mtd driver as a module. That module then can be configured to
switch the ECC scheme when probing. The trick is to load it twice,
sequential with different setup.
As discussed in [1] this solution is only useful when doing such very
special update strategies. So sorry, I think it won't help you.

> As a workaround I do have 2 root partitions which can be boot alternatively.

Sorry, I fail to understand how that would help here. The point is you
need to patch your SPL, bootlaoder and kernel to support the newer ECC
scheme and update all of them at once. If the procedure fail then your
device is likely to be bricked ...

Best regards

Andreas Bießmann

[1] http://thread.gmane.org/gmane.linux.drivers.mtd/50364
[2] http://article.gmane.org/gmane.linux.drivers.mtd/43804


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]

Aw: Re: Linux MTD: Per Partition ECC

[star]

Dear Andreas,
thanks for quick answer.

It is clear to me, that I can't mix ECC at same time.
I would like to switch at a certain point of time.

In uboot there is a command (nandecc) which does this for me.
But howto do this in linux?

The idea is to boot in a normal way (1 bit SW ECC).
I need to copy all needed tools from flash to RAM (flash_eraseall, nandwrite, busybox)
After that all partitions (uboot, kernel, root)  will be erased, written with new ECC scheme and rebooted.

Thinking about this - once I read that it is possible to boot a new kernel out of linux.
Not remember the preconditions, but this is probably the more complex way.
 
mhh.

Did I get it right: You managed it to boot your system (using flash content ?) and than unload the mtd module,
reload it with a differen parameter and you are able to flash xloader?
Of course you cannot access all other partitions of flash during that.

In general that seems simular to my challenge - with the difference that I have to flash complete chain ...
(at least uboot, environment, kernel, root).

You mentioned it is not possible to stick a /dev/mtdx to a certain ECC scheme. 
My kernel is also rather old, (2.6.33) - so no hope that there is included this already.
In kernel source it seems that each partition uses the copy of a so called master. Is more than one master possible?
I didn't understand the precedure until now-

If your are right, it is useless to double my partition scheme. Idea was to boot uboot from /dev/mtd1 but rewrite it with other ECC scheme at /dev/mtd6 as every partition is available with 2 names.


Best regards
Steffen




> Gesendet: Dienstag, 11. Februar 2014 um 13:44 Uhr
> Von: "Andreas Bießmann" <andreas.biessmann@corscience.de>
> An: star@gmx.li, biessmann@corscience.de, peter.barada@gmail.com, linux-mtd@lists.infradead.org
> Betreff: Re: Linux MTD: Per Partition ECC
>
> Dear star@gmx.li,
> 
> cc linux-mtd, one may have more information on that topic there.
> 
> On 02/11/2014 12:58 PM, star@gmx.li wrote:
> > Dear Peter, dear Andreas,
> > 
> > I just found this during my research:
> > 
> > http://lists.infradead.org/pipermail/linux-mtd/2012-September/044120.html 
> >
> > I  have simular Problem, to update a device which runs on 1 bit SW-ECC out of linux to 'on-die-ECC'
> > 
> > I duplicated the board partition table (Linux 2.6.33) in a first step. Now each partition can be access  with 2 names
> > /dev/mtd4 or /dev/mtd8 as example.
> 
> D'oh why the hell should one do that? I dunno if this is sane with
> synchronisation between different access layers (e.g. two different
> instances of filesystem).
> 
> > But how can I addict a ECC scheme to a certain partition? Do you think this is a possible way?
> > So access /dev/mtd4 with software ECC and /dev/mtd8 with on-die-ecc?
> 
> AFAIK there is no way to do per partition ecc currently. So there is no
> way to support this without changing the whole mtd framework.
> But maybe there was some movement since my last check. Maybe one of the
> mtd developers could comment on that.
> 
> Nevertheless the ECC scheme of Linux 1Bit hamming SW ECC is likely to be
> different than the Micron (or whatever manufacturer you use)
> 'on-die-ECC', therefore there is no chance to support 'partition A' and
> 'partition B' working on the very same hardware with these different ECC
> schemes. You can only support scheme A _or_ scheme B at a time. I mean
> for the whole process of storing the data and reading it out of the
> device. Is the data physically written with scheme A it would fail to
> read with scheme B and vice versa.
> 
> > Andreas can you provide the complete patch for you version? 
> 
> You just found the complete patch [2] if you red the link mentioned above.
> 
> > Problem for me is, I am always running on a mtd partition.
> 
> Well, my patch provides a solution for a very special case. Doing system
> update of embedded devices in the field and switching the ECC scheme of
> one partition (the first one, which boot ROM is reading) to OMAP3 1 bit
> hamming HW ECC scheme while the rest of the device is 8 bit BCH SW ECC
> scheme (it is 'HW-assisted', cause of missing ELM in OMAP3; I know there
> were some patches cleaning this up, dunno the current state).
> I can do that cause we kexec the update firmware which brings the
> omap-nand mtd driver as a module. That module then can be configured to
> switch the ECC scheme when probing. The trick is to load it twice,
> sequential with different setup.
> As discussed in [1] this solution is only useful when doing such very
> special update strategies. So sorry, I think it won't help you.
> 
> > As a workaround I do have 2 root partitions which can be boot alternatively.
> 
> Sorry, I fail to understand how that would help here. The point is you
> need to patch your SPL, bootlaoder and kernel to support the newer ECC
> scheme and update all of them at once. If the procedure fail then your
> device is likely to be bricked ...
> 
> Best regards
> 
> Andreas Bießmann
> 
> [1] http://thread.gmane.org/gmane.linux.drivers.mtd/50364
> [2] http://article.gmane.org/gmane.linux.drivers.mtd/43804
> 
>

Re: Aw: Re: Linux MTD: Per Partition ECC

[Peter Barada]

On 02/11/2014 10:43 AM, star@gmx.li wrote:
> Dear Andreas,
> thanks for quick answer.
>
> It is clear to me, that I can't mix ECC at same time.
> I would like to switch at a certain point of time.
>
> In uboot there is a command (nandecc) which does this for me.
> But howto do this in linux?

Currently there is no method to switch ECCs at runtime within Linux w/o
unloading/reloading the NAND driver module and specify a different ECC. 
This works great if your rootfs is not NAND-based (where unloading the
NAND driver can cause it to hang/crash), but in my case I need it to
work with a NAND-based rootfs.

It may be possible to burn the bootloader into NAND using MTD_RAW_MODE
which _should_ disable all ECC for the operation, but in my case the ECC
generator is internal to the NAND chip itself and this won't work.

I'm working on a possible solution in my linux-3.0 kernel, to modify
nand_get_device to take an extra pramater "no_ecc" which keys off
whether the NAND ops mode (if known) is MTD_OOB_RAW, and if so shut off
all ECC (including the in-chip ECC my NAND requires).    Then every NAND
operation will enable/disable ECC while getting the controller lock.

In the process of using flashtool to burn NAND, it uses write() to burn
the data portion, and ioctl(MEMWRITEOOB) to burn the OOB area, but this
causes two writes to the same page, which violates Np for my NAND (i.e.
only one write per page).  I'm looking at creating a new ioctl entry
(MEMWRITEDATAOOB?) that takes both the data and oob buffers from
userspace and then burns them in one operation.

>
> The idea is to boot in a normal way (1 bit SW ECC).
> I need to copy all needed tools from flash to RAM (flash_eraseall, nandwrite, busybox)
> After that all partitions (uboot, kernel, root)  will be erased, written with new ECC scheme and rebooted.
>
> Thinking about this - once I read that it is possible to boot a new kernel out of linux.
> Not remember the preconditions, but this is probably the more complex way.
>  
> mhh.
>
> Did I get it right: You managed it to boot your system (using flash content ?) and than unload the mtd module,
> reload it with a differen parameter and you are able to flash xloader?
> Of course you cannot access all other partitions of flash during that.
>
> In general that seems simular to my challenge - with the difference that I have to flash complete chain ...
> (at least uboot, environment, kernel, root).
>
> You mentioned it is not possible to stick a /dev/mtdx to a certain ECC scheme. 
> My kernel is also rather old, (2.6.33) - so no hope that there is included this already.
> In kernel source it seems that each partition uses the copy of a so called master. Is more than one master possible?
> I didn't understand the precedure until now-
>
> If your are right, it is useless to double my partition scheme. Idea was to boot uboot from /dev/mtd1 but rewrite it with other ECC scheme at /dev/mtd6 as every partition is available with 2 names.
>
>
> Best regards
> Steffen
>
>
>
>
>> Gesendet: Dienstag, 11. Februar 2014 um 13:44 Uhr
>> Von: "Andreas Bießmann" <andreas.biessmann@corscience.de>
>> An: star@gmx.li, biessmann@corscience.de, peter.barada@gmail.com, linux-mtd@lists.infradead.org
>> Betreff: Re: Linux MTD: Per Partition ECC
>>
>> Dear star@gmx.li,
>>
>> cc linux-mtd, one may have more information on that topic there.
>>
>> On 02/11/2014 12:58 PM, star@gmx.li wrote:
>>> Dear Peter, dear Andreas,
>>>
>>> I just found this during my research:
>>>
>>> http://lists.infradead.org/pipermail/linux-mtd/2012-September/044120.html 
>>>
>>> I  have simular Problem, to update a device which runs on 1 bit SW-ECC out of linux to 'on-die-ECC'
>>>
>>> I duplicated the board partition table (Linux 2.6.33) in a first step. Now each partition can be access  with 2 names
>>> /dev/mtd4 or /dev/mtd8 as example.
>> D'oh why the hell should one do that? I dunno if this is sane with
>> synchronisation between different access layers (e.g. two different
>> instances of filesystem).
>>
>>> But how can I addict a ECC scheme to a certain partition? Do you think this is a possible way?
>>> So access /dev/mtd4 with software ECC and /dev/mtd8 with on-die-ecc?
>> AFAIK there is no way to do per partition ecc currently. So there is no
>> way to support this without changing the whole mtd framework.
>> But maybe there was some movement since my last check. Maybe one of the
>> mtd developers could comment on that.
>>
>> Nevertheless the ECC scheme of Linux 1Bit hamming SW ECC is likely to be
>> different than the Micron (or whatever manufacturer you use)
>> 'on-die-ECC', therefore there is no chance to support 'partition A' and
>> 'partition B' working on the very same hardware with these different ECC
>> schemes. You can only support scheme A _or_ scheme B at a time. I mean
>> for the whole process of storing the data and reading it out of the
>> device. Is the data physically written with scheme A it would fail to
>> read with scheme B and vice versa.
>>
>>> Andreas can you provide the complete patch for you version? 
>> You just found the complete patch [2] if you red the link mentioned above.
>>
>>> Problem for me is, I am always running on a mtd partition.
>> Well, my patch provides a solution for a very special case. Doing system
>> update of embedded devices in the field and switching the ECC scheme of
>> one partition (the first one, which boot ROM is reading) to OMAP3 1 bit
>> hamming HW ECC scheme while the rest of the device is 8 bit BCH SW ECC
>> scheme (it is 'HW-assisted', cause of missing ELM in OMAP3; I know there
>> were some patches cleaning this up, dunno the current state).
>> I can do that cause we kexec the update firmware which brings the
>> omap-nand mtd driver as a module. That module then can be configured to
>> switch the ECC scheme when probing. The trick is to load it twice,
>> sequential with different setup.
>> As discussed in [1] this solution is only useful when doing such very
>> special update strategies. So sorry, I think it won't help you.
>>
>>> As a workaround I do have 2 root partitions which can be boot alternatively.
>> Sorry, I fail to understand how that would help here. The point is you
>> need to patch your SPL, bootlaoder and kernel to support the newer ECC
>> scheme and update all of them at once. If the procedure fail then your
>> device is likely to be bricked ...
>>
>> Best regards
>>
>> Andreas Bießmann
>>
>> [1] http://thread.gmane.org/gmane.linux.drivers.mtd/50364
>> [2] http://article.gmane.org/gmane.linux.drivers.mtd/43804
>>
>>


-- 
Peter Barada
peter.barada@gmail.com

Aw: Re: Re: Linux MTD: Per Partition ECC

[star]

Dear Peter,

that sounds like that you haven't nailed it for your needs right now?

If unloading works fine, maybe best method is to follow this approach of Andreas (if I get it right:
- modify environmont to boot somehow from a ramdisk (of course content of this is in flash initially - where else).
- reboot into this ramdisk which just contains tools required for reflash and the content of partitions
- unload module / load module with different ecc scheme
- erase and reflash the complete boot chain (uboot, env, kernel, root)

Sorry, if I am asking again - I haven't the experience with that and I am far away from understanding mtd at the moment.
If I am looking into mtpart.c 

add_one_partition(struct mtd_info *master,
		const struct mtd_partition *part, int partno,
		uint64_t cur_offset)

It looks like that all parameters for this new added partition are copied from a master.
Couldn't modify (even quick and dirty hardcoded) this a solution together with doubling the partition table.
Hence I could access all partitions below x with Soft-ECC and >=x with on-die-ecc?

I guess you guys spend already some time on this, maybe you can explain why this or other approaches for a switch will fail?

Best regards
Steffen


> Gesendet: Dienstag, 11. Februar 2014 um 17:23 Uhr
> Von: "Peter Barada" <peter.barada@gmail.com>
> An: star@gmx.li, "Andreas Bießmann" <andreas.biessmann@corscience.de>
> Cc: biessmann@corscience.de, linux-mtd@lists.infradead.org, "Peter Barada" <peter.barada@logicpd.com>
> Betreff: Re: Aw: Re: Linux MTD: Per Partition ECC
>
> On 02/11/2014 10:43 AM, star@gmx.li wrote:
> > Dear Andreas,
> > thanks for quick answer.
> >
> > It is clear to me, that I can't mix ECC at same time.
> > I would like to switch at a certain point of time.
> >
> > In uboot there is a command (nandecc) which does this for me.
> > But howto do this in linux?
> 
> Currently there is no method to switch ECCs at runtime within Linux w/o
> unloading/reloading the NAND driver module and specify a different ECC. 
> This works great if your rootfs is not NAND-based (where unloading the
> NAND driver can cause it to hang/crash), but in my case I need it to
> work with a NAND-based rootfs.
> 
> It may be possible to burn the bootloader into NAND using MTD_RAW_MODE
> which _should_ disable all ECC for the operation, but in my case the ECC
> generator is internal to the NAND chip itself and this won't work.
> 
> I'm working on a possible solution in my linux-3.0 kernel, to modify
> nand_get_device to take an extra pramater "no_ecc" which keys off
> whether the NAND ops mode (if known) is MTD_OOB_RAW, and if so shut off
> all ECC (including the in-chip ECC my NAND requires).    Then every NAND
> operation will enable/disable ECC while getting the controller lock.
> 
> In the process of using flashtool to burn NAND, it uses write() to burn
> the data portion, and ioctl(MEMWRITEOOB) to burn the OOB area, but this
> causes two writes to the same page, which violates Np for my NAND (i.e.
> only one write per page).  I'm looking at creating a new ioctl entry
> (MEMWRITEDATAOOB?) that takes both the data and oob buffers from
> userspace and then burns them in one operation.
> 
> >
> > The idea is to boot in a normal way (1 bit SW ECC).
> > I need to copy all needed tools from flash to RAM (flash_eraseall, nandwrite, busybox)
> > After that all partitions (uboot, kernel, root)  will be erased, written with new ECC scheme and rebooted.
> >
> > Thinking about this - once I read that it is possible to boot a new kernel out of linux.
> > Not remember the preconditions, but this is probably the more complex way.
> >  
> > mhh.
> >
> > Did I get it right: You managed it to boot your system (using flash content ?) and than unload the mtd module,
> > reload it with a differen parameter and you are able to flash xloader?
> > Of course you cannot access all other partitions of flash during that.
> >
> > In general that seems simular to my challenge - with the difference that I have to flash complete chain ...
> > (at least uboot, environment, kernel, root).
> >
> > You mentioned it is not possible to stick a /dev/mtdx to a certain ECC scheme. 
> > My kernel is also rather old, (2.6.33) - so no hope that there is included this already.
> > In kernel source it seems that each partition uses the copy of a so called master. Is more than one master possible?
> > I didn't understand the precedure until now-
> >
> > If your are right, it is useless to double my partition scheme. Idea was to boot uboot from /dev/mtd1 but rewrite it with other ECC scheme at /dev/mtd6 as every partition is available with 2 names.
> >
> >
> > Best regards
> > Steffen
> >
> >
> >
> >
> >> Gesendet: Dienstag, 11. Februar 2014 um 13:44 Uhr
> >> Von: "Andreas Bießmann" <andreas.biessmann@corscience.de>
> >> An: star@gmx.li, biessmann@corscience.de, peter.barada@gmail.com, linux-mtd@lists.infradead.org
> >> Betreff: Re: Linux MTD: Per Partition ECC
> >>
> >> Dear star@gmx.li,
> >>
> >> cc linux-mtd, one may have more information on that topic there.
> >>
> >> On 02/11/2014 12:58 PM, star@gmx.li wrote:
> >>> Dear Peter, dear Andreas,
> >>>
> >>> I just found this during my research:
> >>>
> >>> http://lists.infradead.org/pipermail/linux-mtd/2012-September/044120.html 
> >>>
> >>> I  have simular Problem, to update a device which runs on 1 bit SW-ECC out of linux to 'on-die-ECC'
> >>>
> >>> I duplicated the board partition table (Linux 2.6.33) in a first step. Now each partition can be access  with 2 names
> >>> /dev/mtd4 or /dev/mtd8 as example.
> >> D'oh why the hell should one do that? I dunno if this is sane with
> >> synchronisation between different access layers (e.g. two different
> >> instances of filesystem).
> >>
> >>> But how can I addict a ECC scheme to a certain partition? Do you think this is a possible way?
> >>> So access /dev/mtd4 with software ECC and /dev/mtd8 with on-die-ecc?
> >> AFAIK there is no way to do per partition ecc currently. So there is no
> >> way to support this without changing the whole mtd framework.
> >> But maybe there was some movement since my last check. Maybe one of the
> >> mtd developers could comment on that.
> >>
> >> Nevertheless the ECC scheme of Linux 1Bit hamming SW ECC is likely to be
> >> different than the Micron (or whatever manufacturer you use)
> >> 'on-die-ECC', therefore there is no chance to support 'partition A' and
> >> 'partition B' working on the very same hardware with these different ECC
> >> schemes. You can only support scheme A _or_ scheme B at a time. I mean
> >> for the whole process of storing the data and reading it out of the
> >> device. Is the data physically written with scheme A it would fail to
> >> read with scheme B and vice versa.
> >>
> >>> Andreas can you provide the complete patch for you version? 
> >> You just found the complete patch [2] if you red the link mentioned above.
> >>
> >>> Problem for me is, I am always running on a mtd partition.
> >> Well, my patch provides a solution for a very special case. Doing system
> >> update of embedded devices in the field and switching the ECC scheme of
> >> one partition (the first one, which boot ROM is reading) to OMAP3 1 bit
> >> hamming HW ECC scheme while the rest of the device is 8 bit BCH SW ECC
> >> scheme (it is 'HW-assisted', cause of missing ELM in OMAP3; I know there
> >> were some patches cleaning this up, dunno the current state).
> >> I can do that cause we kexec the update firmware which brings the
> >> omap-nand mtd driver as a module. That module then can be configured to
> >> switch the ECC scheme when probing. The trick is to load it twice,
> >> sequential with different setup.
> >> As discussed in [1] this solution is only useful when doing such very
> >> special update strategies. So sorry, I think it won't help you.
> >>
> >>> As a workaround I do have 2 root partitions which can be boot alternatively.
> >> Sorry, I fail to understand how that would help here. The point is you
> >> need to patch your SPL, bootlaoder and kernel to support the newer ECC
> >> scheme and update all of them at once. If the procedure fail then your
> >> device is likely to be bricked ...
> >>
> >> Best regards
> >>
> >> Andreas Bießmann
> >>
> >> [1] http://thread.gmane.org/gmane.linux.drivers.mtd/50364
> >> [2] http://article.gmane.org/gmane.linux.drivers.mtd/43804
> >>
> >>
> 
> 
> -- 
> Peter Barada
> peter.barada@gmail.com
> 
>

Re: Aw: Re: Linux MTD: Per Partition ECC

[Brian Norris]

On Tue, Feb 11, 2014 at 11:23:00AM -0500, Peter Barada wrote:
> On 02/11/2014 10:43 AM, star@gmx.li wrote:
> > It is clear to me, that I can't mix ECC at same time.
> > I would like to switch at a certain point of time.
> >
> > In uboot there is a command (nandecc) which does this for me.
> > But howto do this in linux?
> 
> Currently there is no method to switch ECCs at runtime within Linux w/o
> unloading/reloading the NAND driver module and specify a different ECC. 
> This works great if your rootfs is not NAND-based (where unloading the
> NAND driver can cause it to hang/crash), but in my case I need it to
> work with a NAND-based rootfs.
> 
> It may be possible to burn the bootloader into NAND using MTD_RAW_MODE
> which _should_ disable all ECC for the operation, but in my case the ECC
> generator is internal to the NAND chip itself and this won't work.
> 
> I'm working on a possible solution in my linux-3.0 kernel, to modify
> nand_get_device to take an extra pramater "no_ecc" which keys off
> whether the NAND ops mode (if known) is MTD_OOB_RAW, and if so shut off
> all ECC (including the in-chip ECC my NAND requires).    Then every NAND
> operation will enable/disable ECC while getting the controller lock.

It is renamed to MTD_OPS_RAW, and yes, drivers should support turning
off ECC when using RAW. I don't think anybody has ever considered
supporting raw mode with on-die ECC, though. You can probably do it with
some modest modifications to the NAND layer (esp. to the
chip->{read,write}_*_raw callbacks). I would recommend trying this
before trying to develop per-partition ECC configuration. Others are
trying, but I'm not confident it will go anywhere:

http://lists.infradead.org/pipermail/linux-mtd/2014-February/051896.html

> In the process of using flashtool to burn NAND, it uses write() to burn
> the data portion, and ioctl(MEMWRITEOOB) to burn the OOB area, but this
> causes two writes to the same page, which violates Np for my NAND (i.e.
> only one write per page).  I'm looking at creating a new ioctl entry
> (MEMWRITEDATAOOB?) that takes both the data and oob buffers from
> userspace and then burns them in one operation.

Please do not create a new one. ioctl(MEMWRITE) already exists for this
purpose. It can do page-at-a-time programming of data+OOB, and it even
has a 'mode' field for choosing ECC vs. raw (no-ECC) programming. This
is useful for any flash that doesn't support partial page programming.
The nandwrite utility from mtd-utils supports this in Linux.

Brian

Re: Aw: Re: Linux MTD: Per Partition ECC

[Ricard Wanderlof]

On Wed, 12 Feb 2014, Brian Norris wrote:

>> In the process of using flashtool to burn NAND, it uses write() to burn
>> the data portion, and ioctl(MEMWRITEOOB) to burn the OOB area, but this
>> causes two writes to the same page, which violates Np for my NAND (i.e.
>> only one write per page).  I'm looking at creating a new ioctl entry
>> (MEMWRITEDATAOOB?) that takes both the data and oob buffers from
>> userspace and then burns them in one operation.
>
> Please do not create a new one. ioctl(MEMWRITE) already exists for this
> purpose. It can do page-at-a-time programming of data+OOB, and it even
> has a 'mode' field for choosing ECC vs. raw (no-ECC) programming. This
> is useful for any flash that doesn't support partial page programming.
> The nandwrite utility from mtd-utils supports this in Linux.

I don't know if it has changed lately, but I was just looking at how 
nandwrite handles writes in RAW mode, and it appears that it does two 
separate writes in this case, one for the OOB and one for the main area.

It seems to be done that way just because it was the easiest thing to do: 
at some point the nandwrite checks for raw mode write, and writes the oob 
then and there, followed by the ordinary write to the main area for all 
modes. Having a combined write to both areas at once would increase 
complexity, and I suppose it was just not needed at the time.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Aw: Re: Linux MTD: Per Partition ECC

[Brian Norris]

On Wed, Feb 12, 2014 at 08:48:18AM +0100, Ricard Wanderlof wrote:
> On Wed, 12 Feb 2014, Brian Norris wrote:
> >>In the process of using flashtool to burn NAND, it uses write() to burn
> >>the data portion, and ioctl(MEMWRITEOOB) to burn the OOB area, but this
> >>causes two writes to the same page, which violates Np for my NAND (i.e.
> >>only one write per page).  I'm looking at creating a new ioctl entry
> >>(MEMWRITEDATAOOB?) that takes both the data and oob buffers from
> >>userspace and then burns them in one operation.
> >
> >Please do not create a new one. ioctl(MEMWRITE) already exists for this
> >purpose. It can do page-at-a-time programming of data+OOB, and it even
> >has a 'mode' field for choosing ECC vs. raw (no-ECC) programming. This
> >is useful for any flash that doesn't support partial page programming.
> >The nandwrite utility from mtd-utils supports this in Linux.
> 
> I don't know if it has changed lately, but I was just looking at how
> nandwrite handles writes in RAW mode, and it appears that it does
> two separate writes in this case, one for the OOB and one for the
> main area.

It has changed.

http://git.infradead.org/mtd-utils.git/commitdiff/6a8889fbc0ea7a198628c9e3901fd88038e12d09

Brian

Re: Aw: Re: Linux MTD: Per Partition ECC

[Ricard Wanderlof]

On Wed, 12 Feb 2014, Brian Norris wrote:

>> I don't know if it has changed lately, but I was just looking at how
>> nandwrite handles writes in RAW mode, and it appears that it does
>> two separate writes in this case, one for the OOB and one for the
>> main area.
>
> It has changed.
>
> http://git.infradead.org/mtd-utils.git/commitdiff/6a8889fbc0ea7a198628c9e3901fd88038e12d09

Ah, great. Thanks for pointing this out.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Aw: Re: Linux MTD: Per Partition ECC

[Ricard Wanderlof]

On Tue, 11 Feb 2014, Peter Barada wrote:

> It may be possible to burn the bootloader into NAND using MTD_RAW_MODE
> which _should_ disable all ECC for the operation, but in my case the ECC
> generator is internal to the NAND chip itself and this won't work.
>
> I'm working on a possible solution in my linux-3.0 kernel, to modify
> nand_get_device to take an extra pramater "no_ecc" which keys off
> whether the NAND ops mode (if known) is MTD_OOB_RAW, and if so shut off
> all ECC (including the in-chip ECC my NAND requires).    Then every NAND
> operation will enable/disable ECC while getting the controller lock.

I'm sure I don't understand this properly, but in the first paragraph you 
state that the ECC generator is internal to the NAND chip so that you 
can't disable ECC on a per-write basis using MTD_RAW_MODE, then in the 
second paragraph above you say that you want to add a parameter which 
disables ECC. But that is what MTD_OOB_RAW is supposed to do isn't it?

Or is the real problem here that in the default routines in nand_base.c 
there is no way of notifying the _nand_chip_ which mode we're in? It 
should be possible though if you replace the appropriate default callbacks 
with chip-specific ones.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Aw: Re: Linux MTD: Per Partition ECC

[Peter Barada]

On 02/12/2014 02:44 AM, Ricard Wanderlof wrote:
>
> On Tue, 11 Feb 2014, Peter Barada wrote:
>
>> It may be possible to burn the bootloader into NAND using MTD_RAW_MODE
>> which _should_ disable all ECC for the operation, but in my case the ECC
>> generator is internal to the NAND chip itself and this won't work.
>>
>> I'm working on a possible solution in my linux-3.0 kernel, to modify
>> nand_get_device to take an extra pramater "no_ecc" which keys off
>> whether the NAND ops mode (if known) is MTD_OOB_RAW, and if so shut off
>> all ECC (including the in-chip ECC my NAND requires).    Then every NAND
>> operation will enable/disable ECC while getting the controller lock.
>
> I'm sure I don't understand this properly, but in the first paragraph
> you state that the ECC generator is internal to the NAND chip so that
> you can't disable ECC on a per-write basis using MTD_RAW_MODE, then in
> the second paragraph above you say that you want to add a parameter
> which disables ECC. But that is what MTD_OOB_RAW is supposed to do
> isn't it?
>
> Or is the real problem here that in the default routines in
> nand_base.c there is no way of notifying the _nand_chip_ which mode
> we're in? It should be possible though if you replace the appropriate
> default callbacks with chip-specific ones.
>
> /Ricard
My apologies, I should have mentioned I'm working within linux-3.0 and
am trying to implement a solution for units that are already in the
field - updating to a newer kernel isn't feasible at this time. Some of
the issues I'm up against may already been addressed in newer kernels.
I'll pull a new kernel and look at backporting the new ioctl(MEMWRITE)
into my kernel to fix the write()/ioctl(MEMWRITEOOB) multiple page-write
problem.

Rather than adding multiple chip-specific callbacks (and subsequent
duplicated code), I think it might be cleaner to add a parameter to
nand_get_device() for the mode, and a chip-specific call that
enables/disables chip ECC if the mode changes from what the chip is in.

-- 
Peter Barada
peter.barada@logicpd.com

Re: Aw: Re: Linux MTD: Per Partition ECC

[Andreas Bießmann]

[-- Attachment #1: Type: text/plain, Size: 3726 bytes --]

Hi Steffen,

On 02/11/2014 04:43 PM, star@gmx.li wrote:
> It is clear to me, that I can't mix ECC at same time.
> I would like to switch at a certain point of time.
> 
> In uboot there is a command (nandecc) which does this for me.

Ok, understand. This command however is only available in omap3 devices
and exactly for that purpose. It was refused to implement a generic way
to switch ECC scheme for other SoC. The way to go seems to use the very
same ECC scheme as the ROM code uses (that is the case for at least Ti
devices like AM3xx family). Unfortunately this won't scale with changes
in NAND technology. So, yes, there might be the need for more
flexibility in NAND partitioning and ECC schemes.

> But howto do this in linux?

There is no builtin mechanism, you need to build one for your needs.

> The idea is to boot in a normal way (1 bit SW ECC).
> I need to copy all needed tools from flash to RAM (flash_eraseall, nandwrite, busybox)
> After that all partitions (uboot, kernel, root)  will be erased, written with new ECC scheme and rebooted.

That is the right way out.

> Thinking about this - once I read that it is possible to boot a new kernel out of linux.
> Not remember the preconditions, but this is probably the more complex way.

In my opinion this is the clean way. The 'copying required files to
ramfs' approach described above is widespread (at least in my feelings,
we used this approach in former devices too). The problem with this
approach is that you can't adopt the tools you need. The kexec approach
is more flexible here. Just my 2¢ and of course it is off topic here.

> Did I get it right: You managed it to boot your system (using flash content ?) and than unload the mtd module,
> reload it with a differen parameter and you are able to flash xloader?

Right, with a freshly booted linux out of the running linux. We switched
our update procedure to the kexec approach exactly for that reason. We
have a clean, controllable environment and can do just everything from
that running update system, including change of ecc scheme or even
change of mtd partitioning.

> Of course you cannot access all other partitions of flash during that.
> 
> In general that seems simular to my challenge - with the difference that I have to flash complete chain ...
> (at least uboot, environment, kernel, root).

We do exactly that, we do it sequentially and switch ecc scheme where
needed.

> You mentioned it is not possible to stick a /dev/mtdx to a certain ECC scheme. 

Well, it seems there is some work done regarding 'Per Partition ECC',
Boris Brezillon sent an RFC [1].

> My kernel is also rather old, (2.6.33) - so no hope that there is included this already.

That would be your job to get it working then. I can't say anything
about architectural changes in mtd framework since then.

> In kernel source it seems that each partition uses the copy of a so called master. Is more than one master possible?
> I didn't understand the precedure until now-
>
> If your are right, it is useless to double my partition scheme. Idea was to boot uboot from /dev/mtd1 but rewrite it with other ECC scheme at /dev/mtd6 as every partition is available with 2 names.

Regardless the meaningfulness of your approach, you need to flash a
kernel which is capable to do so and boot it again. So this would be a
two step approach to update your device. Why not just boot another linux
out of the running one which is capable to access your NAND with
different ECC schemes (however it is implemented) and do your update
procedure.

Best regards

Andreas Bießmann

[1] http://article.gmane.org/gmane.linux.drivers.mtd/51605



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 901 bytes --]