Preventing JFFS2 partial page writes?

Preventing JFFS2 partial page writes?

[Peter Barada]

I'm using a Micron 512MB NAND which has an internal 4-bit ECC engine, 
and am finding lots of ECC errors while using JFFS2.  The same NAND 
works find using YAFFS so I know the underlying MTD driver works properly.

The big question I have is whether JFFS2 does partial page writes and if 
so how to disable them.

Thanks in advance!

-- 
Peter Barada
peter.barada@gmail.com

Re: Preventing JFFS2 partial page writes?

[Artem Bityutskiy]

On Tue, 2011-06-14 at 12:19 -0400, Peter Barada wrote:
> I'm using a Micron 512MB NAND which has an internal 4-bit ECC engine, 
> and am finding lots of ECC errors while using JFFS2.  The same NAND 
> works find using YAFFS so I know the underlying MTD driver works properly.
> 
> The big question I have is whether JFFS2 does partial page writes and if 
> so how to disable them.

JFFS2 does not use sub-pages so it should never do partial writes. I
think the ECC errors you see are because of JFFS2 using OOB area to
store clean markers.

-- 
Best Regards,
Artem Bityutskiy

Re: Preventing JFFS2 partial page writes?

[Peter Barada]

On 06/22/2011 02:04 AM, Artem Bityutskiy wrote:
> On Tue, 2011-06-14 at 12:19 -0400, Peter Barada wrote:
>> I'm using a Micron 512MB NAND which has an internal 4-bit ECC engine,
>> and am finding lots of ECC errors while using JFFS2.  The same NAND
>> works find using YAFFS so I know the underlying MTD driver works properly.
>>
>> The big question I have is whether JFFS2 does partial page writes and if
>> so how to disable them.
> JFFS2 does not use sub-pages so it should never do partial writes. I
> think the ECC errors you see are because of JFFS2 using OOB area to
> store clean markers.
Sorry for using the wrong terminology - I assumed a "partial write" is 
where data/oob information is written into a page multiple times.

Yes, the issue I'm seeing is that JFFS2 writes into the OOB area into
bytes that perturb the ECC, then writes data (w/o OOB) and the next read 
fails.  Googling around I see GETECCLAYOUT is now obsolete, so it 
doesn't look like extending the structure returned from it is not the 
way to go.

I'm thinking a new ioctl (and data in the kernel accessible to the NAND 
FS layers) is needed (GETOOBLAYOUT?) that returns the oobfree array, as 
well as other arrays in a structure:
1) which OOB bytes perturb the data ECCs,
2) which OOB bytes are ECC'd (within the OOB area and separate from the 
data ECCs).

I think given the new large-sized NAND that these lists will have to be 
dynamic (and another GETOOBLAYOUTSIZE ioctl call to return its size) to 
prevent moving massive structure to userspace).

Then userspace can figure out:

1) what bytes can be written in the OOB.
2) what bytes in the OOB can only be written as part of a data write (or 
sub-page write)
3) if no OOB bytes are internally ECC'd(outside of those covered by a 
data area ECC) then the code needs to either use an ECC as part of the 
data written to the OOB, or use a bitflip-friendly way to compare the 
returned data - unless the OOB data it wants to write is part of a data 
area write.

In my initial attempt to solve this I ran into the issue (in 
linux-2.6.32) that nand_transfer_oob/nand_fill_oob make the assumption 
that chip->ecc.layout->oobfree is *always* zero terminated (I needed 
eight entries to describe my OOB area).  I'll put together a patch 
against the latest kernel and send it along.

Thoughts?

-- 
Peter Barada
peter.barada@gmail.com

Re: Preventing JFFS2 partial page writes?

[Ivan Djelic]

On Wed, Jun 22, 2011 at 04:28:54PM +0100, Peter Barada wrote:
(...)
> 
> I'm thinking a new ioctl (and data in the kernel accessible to the NAND 
> FS layers) is needed (GETOOBLAYOUT?) that returns the oobfree array, as 
> well as other arrays in a structure:
> 1) which OOB bytes perturb the data ECCs,
> 2) which OOB bytes are ECC'd (within the OOB area and separate from the 
> data ECCs).
> 
> I think given the new large-sized NAND that these lists will have to be 
> dynamic (and another GETOOBLAYOUTSIZE ioctl call to return its size) to 
> prevent moving massive structure to userspace).
> 
> Then userspace can figure out:
> 
> 1) what bytes can be written in the OOB.
> 2) what bytes in the OOB can only be written as part of a data write (or 
> sub-page write)
> 3) if no OOB bytes are internally ECC'd(outside of those covered by a 
> data area ECC) then the code needs to either use an ECC as part of the 
> data written to the OOB, or use a bitflip-friendly way to compare the 
> returned data - unless the OOB data it wants to write is part of a data 
> area write.
> 
> In my initial attempt to solve this I ran into the issue (in 
> linux-2.6.32) that nand_transfer_oob/nand_fill_oob make the assumption 
> that chip->ecc.layout->oobfree is *always* zero terminated (I needed 
> eight entries to describe my OOB area).  I'll put together a patch 
> against the latest kernel and send it along.
> 
> Thoughts?

On one hand it sure would be nice (and a bit complicated) to accurately
describe OOB write constraints, for easier JFFS2/YAFFS2 integration.
On the other hand, I am not sure such a complication is really worth the
trouble, given that on next nand generation:
- OOB areas will not be usable anymore for metadata storage (8-bit ecc leaves
  only 6 spare bytes out of 64)
- partial writes will probably be limited to 1 (like in MLC), meaning that
  JFFS2 clean marking step will be forbidden anyway
Furthermore, userspace will probably need to handle case 3) anyway (no
protected oob bytes) to stay portable...

--
Ivan

Re: Preventing JFFS2 partial page writes?

[Peter Barada]

On 06/22/2011 01:07 PM, Ivan Djelic wrote:
>
> On one hand it sure would be nice (and a bit complicated) to accurately
> describe OOB write constraints, for easier JFFS2/YAFFS2 integration.
> On the other hand, I am not sure such a complication is really worth the
> trouble, given that on next nand generation:
> - OOB areas will not be usable anymore for metadata storage (8-bit ecc leaves
>    only 6 spare bytes out of 64)
> - partial writes will probably be limited to 1 (like in MLC), meaning that
>    JFFS2 clean marking step will be forbidden anyway
> Furthermore, userspace will probably need to handle case 3) anyway (no
> protected oob bytes) to stay portable...
>
SLC parts (including this one) look to be around for a while, as well as 
the products they are already in.  So I think the issue of JFFS2's 
cleanmarker in the context of this Micron NAND needs to be solved, 
preferably in a general way so the next oddball NAND chip that comes 
along doesn't cause MTD, mtd-utils, JFFS2 (and other NAND FS) fits.

Does UBIFS do any of the "partial writes" (i.e. write OOB/data area 
multiple times) as JFFS2 does?



-- 
Peter Barada
peter.barada@gmail.com

Re: Preventing JFFS2 partial page writes?

[Ivan Djelic]

On Wed, Jun 22, 2011 at 08:17:33PM +0100, Peter Barada wrote:
> On 06/22/2011 01:07 PM, Ivan Djelic wrote:
> >
> > On one hand it sure would be nice (and a bit complicated) to accurately
> > describe OOB write constraints, for easier JFFS2/YAFFS2 integration.
> > On the other hand, I am not sure such a complication is really worth the
> > trouble, given that on next nand generation:
> > - OOB areas will not be usable anymore for metadata storage (8-bit ecc leaves
> >    only 6 spare bytes out of 64)
> > - partial writes will probably be limited to 1 (like in MLC), meaning that
> >    JFFS2 clean marking step will be forbidden anyway
> > Furthermore, userspace will probably need to handle case 3) anyway (no
> > protected oob bytes) to stay portable...
> >
> SLC parts (including this one) look to be around for a while, as well as 
> the products they are already in.  So I think the issue of JFFS2's 
> cleanmarker in the context of this Micron NAND needs to be solved, 
> preferably in a general way so the next oddball NAND chip that comes 
> along doesn't cause MTD, mtd-utils, JFFS2 (and other NAND FS) fits.
> 
> Does UBIFS do any of the "partial writes" (i.e. write OOB/data area 
> multiple times) as JFFS2 does?
 
No it doesn't. UBIFS relies on UBI, which itself does not rely on oob for
storing metadata. BTW, have you considered using UBIFS instead of JFFS2/YAFFS2?

--
Ivan

Re: Preventing JFFS2 partial page writes?

[Peter Barada]

On 06/22/2011 04:06 PM, Ivan Djelic wrote:
> On Wed, Jun 22, 2011 at 08:17:33PM +0100, Peter Barada wrote:
>> On 06/22/2011 01:07 PM, Ivan Djelic wrote:
>>> On one hand it sure would be nice (and a bit complicated) to accurately
>>> describe OOB write constraints, for easier JFFS2/YAFFS2 integration.
>>> On the other hand, I am not sure such a complication is really worth the
>>> trouble, given that on next nand generation:
>>> - OOB areas will not be usable anymore for metadata storage (8-bit ecc leaves
>>>     only 6 spare bytes out of 64)
>>> - partial writes will probably be limited to 1 (like in MLC), meaning that
>>>     JFFS2 clean marking step will be forbidden anyway
>>> Furthermore, userspace will probably need to handle case 3) anyway (no
>>> protected oob bytes) to stay portable...
>>>
>> SLC parts (including this one) look to be around for a while, as well as
>> the products they are already in.  So I think the issue of JFFS2's
>> cleanmarker in the context of this Micron NAND needs to be solved,
>> preferably in a general way so the next oddball NAND chip that comes
>> along doesn't cause MTD, mtd-utils, JFFS2 (and other NAND FS) fits.
>>
>> Does UBIFS do any of the "partial writes" (i.e. write OOB/data area
>> multiple times) as JFFS2 does?
>
> No it doesn't. UBIFS relies on UBI, which itself does not rely on oob for
> storing metadata. BTW, have you considered using UBIFS instead of JFFS2/YAFFS2?
Yes.  However customers are already using JFFS2/YAFFS2 and are reluctant 
to change.
> --
> Ivan
>
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/


-- 
Peter Barada
peter.barada@gmail.com

Re: Preventing JFFS2 partial page writes?

[Artem Bityutskiy]

On Wed, 2011-06-22 at 11:28 -0400, Peter Barada wrote:
> Thoughts?

Sorry, could you please define the problem you are trying to solve?
Sorry if you did define it in your long post, but I could not easily
find it.

-- 
Best Regards,
Artem Bityutskiy (Битюцкий Артём)

Re: Preventing JFFS2 partial page writes?

[Peter Barada]

On 06/24/2011 03:26 PM, Artem Bityutskiy wrote:
> On Wed, 2011-06-22 at 11:28 -0400, Peter Barada wrote:
>> Thoughts?
> Sorry, could you please define the problem you are trying to solve?
> Sorry if you did define it in your long post, but I could not easily
> find it.
The problem I'm trying to solve is that the Micron NAND  I'm using has 
an internal 4-bit ECC engine and uses four 8-byte ECCs that provide 
4-bit protection per 512 data bytes + four OOB bytes.  The ecclayout I'm 
using is:

ecclayout = {
         eccbytes = 32,
         eccpos = { 8, 9, 10, 11, 12, 13, 14, 15, /* ECC data bytes 
0-511 + OOB bytes 4-7 */
                 24, 25, 26, 27, 28, 19, 30, 31, /* ECC data bytes 
512-1023 + OOB bytes 20-23 */
                 40, 41, 42, 43, 44, 45, 46, 47,  /* ECC data bytes 
1024-1535 + OOB bytes 36-39 */
                 56, 57, 58, 59, 60, 61, 62, 63},  /* ECC data bytes 
1536-2047 + OOB bytes 52-55 */
         .oobfree = {
                 { .offset = 4,
                   .length = 4},
                 { .offset = 20,
                   .length = 4},
                 { .offset = 36,
                   .length = 4},
                 { .offset = 52,
                   .length = 4},
         },
};

After the JFFS2 cleanmarker is written into bytes 4-7 and 16-23 of the 
OOB, nanddump shows:

   OOB Data: ff ff ff ff 85 19 03 20 5a e3 da 69 01 40 f1 36
   OOB Data: ff ff ff ff 08 00 00 00 91 99 3c 05 01 d0 5d b3
   OOB Data: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
   OOB Data: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

Note that the ECC bytes 8-15 and 24-31 are no longer FF due to bytes 4-7 
and bytes 20-23 being written with non-zero data.

When data is later written to this same page (w/o an intervening erase 
of its block) reading the page causes an uncorrectable ECC error.

There are eight additional bytes of OOB space available for writing, but 
they are not ECC'd.

The issue I'm trying to solve is how to communicate from MTD to JFFS2 
that some bytes of the oobfree array perturb the data ECC and can not be 
used to write the cleanmarker.

-- 
Peter Barada
peter.barada@gmail.com

Re: Preventing JFFS2 partial page writes?

[Artem Bityutskiy]

On Mon, 2011-06-27 at 10:31 -0400, Peter Barada wrote:
> On 06/24/2011 03:26 PM, Artem Bityutskiy wrote:
> > On Wed, 2011-06-22 at 11:28 -0400, Peter Barada wrote:
> >> Thoughts?
> > Sorry, could you please define the problem you are trying to solve?
> > Sorry if you did define it in your long post, but I could not easily
> > find it.
> The problem I'm trying to solve is that the Micron NAND  I'm using has 
> an internal 4-bit ECC engine and uses four 8-byte ECCs that provide 
> 4-bit protection per 512 data bytes + four OOB bytes.  The ecclayout I'm 
> using is:
> 
> ecclayout = {
>          eccbytes = 32,
>          eccpos = { 8, 9, 10, 11, 12, 13, 14, 15, /* ECC data bytes 
> 0-511 + OOB bytes 4-7 */
>                  24, 25, 26, 27, 28, 19, 30, 31, /* ECC data bytes 
> 512-1023 + OOB bytes 20-23 */
>                  40, 41, 42, 43, 44, 45, 46, 47,  /* ECC data bytes 
> 1024-1535 + OOB bytes 36-39 */
>                  56, 57, 58, 59, 60, 61, 62, 63},  /* ECC data bytes 
> 1536-2047 + OOB bytes 52-55 */
>          .oobfree = {
>                  { .offset = 4,
>                    .length = 4},
>                  { .offset = 20,
>                    .length = 4},
>                  { .offset = 36,
>                    .length = 4},
>                  { .offset = 52,
>                    .length = 4},
>          },
> };
> 
> After the JFFS2 cleanmarker is written into bytes 4-7 and 16-23 of the 
> OOB, nanddump shows:
> 
>    OOB Data: ff ff ff ff 85 19 03 20 5a e3 da 69 01 40 f1 36
>    OOB Data: ff ff ff ff 08 00 00 00 91 99 3c 05 01 d0 5d b3
>    OOB Data: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>    OOB Data: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
> 
> Note that the ECC bytes 8-15 and 24-31 are no longer FF due to bytes 4-7 
> and bytes 20-23 being written with non-zero data.
> 
> When data is later written to this same page (w/o an intervening erase 
> of its block) reading the page causes an uncorrectable ECC error.
> 
> There are eight additional bytes of OOB space available for writing, but 
> they are not ECC'd.
> 
> The issue I'm trying to solve is how to communicate from MTD to JFFS2 
> that some bytes of the oobfree array perturb the data ECC and can not be 
> used to write the cleanmarker.

OK, thanks for explanation. I am not very good in this area as I do not
have much experience dealing with OOB, but here is what I thing.

1. Linux MTD code was _not_ designed for "ECC'ed OOB".
2. I do not really know what MTD_OOB_RAW is, and the comment in mtd.h
   is not very verbose.
3. But in my opinion MTD_OOB_AUTO makes most sense and should be used
   everywhere except for some tricky cases when you want to test things
   by writing incorrect ECC, or you have an image with ECC and you want
   to flash it as is.
4. In general, OOB should be considered as belonging to the driver, and
   modern software should not rely on OOB at all.
5. So MTD_OOB_AUTO make free bytes in OOB look like a contiguous buffer
   which the _user_ can freely and _independently_ use.
6. In your case only this assumption does not work and your ecclayout is
   incorrect because the OOB areas you expose are not independent.
7. So in your case your ecclayout should be changed and you should
   expose only independent ECC bytes.

-- 
Best Regards,
Artem Bityutskiy

Re: Preventing JFFS2 partial page writes?

[Artem Bityutskiy]

On Tue, 2011-06-28 at 12:34 +0300, Artem Bityutskiy wrote:
> OK, thanks for explanation. I am not very good in this area as I do not
> have much experience dealing with OOB, but here is what I thing.
> 
> 1. Linux MTD code was _not_ designed for "ECC'ed OOB".
> 2. I do not really know what MTD_OOB_RAW is, and the comment in mtd.h
>    is not very verbose.
> 3. But in my opinion MTD_OOB_AUTO makes most sense and should be used
>    everywhere except for some tricky cases when you want to test things
>    by writing incorrect ECC, or you have an image with ECC and you want
>    to flash it as is.
> 4. In general, OOB should be considered as belonging to the driver, and
>    modern software should not rely on OOB at all.
> 5. So MTD_OOB_AUTO make free bytes in OOB look like a contiguous buffer
>    which the _user_ can freely and _independently_ use.
> 6. In your case only this assumption does not work and your ecclayout is
>    incorrect because the OOB areas you expose are not independent.
> 7. So in your case your ecclayout should be changed and you should
>    expose only independent ECC bytes.

To put it differently, I current model does not distinguish (I think,
correct me if I am wrong) between ECC'd OOB bytes and ECC'less OOB
bytes. BTW, does your flash has the latter?

So MTD would need some work to make it distinguish between those 2 types
of OOB bytes - probably additional info could be added to the ooblayout
structure, and the interfaces could be improved. How exactly - dunno,
I'd first need to figure out what MTD_OOB_RAW is - may be Brian or Ivan
could comment.

-- 
Best Regards,
Artem Bityutskiy

Re: Preventing JFFS2 partial page writes?

[Peter Barada]

On 06/28/2011 05:34 AM, Artem Bityutskiy wrote:
> On Mon, 2011-06-27 at 10:31 -0400, Peter Barada wrote:
>> On 06/24/2011 03:26 PM, Artem Bityutskiy wrote:
>>> On Wed, 2011-06-22 at 11:28 -0400, Peter Barada wrote:
>>>> Thoughts?
>>> Sorry, could you please define the problem you are trying to solve?
>>> Sorry if you did define it in your long post, but I could not easily
>>> find it.
>> The problem I'm trying to solve is that the Micron NAND  I'm using has
>> an internal 4-bit ECC engine and uses four 8-byte ECCs that provide
>> 4-bit protection per 512 data bytes + four OOB bytes.  The ecclayout I'm
>> using is:
>>
>> ecclayout = {
>>           eccbytes = 32,
>>           eccpos = { 8, 9, 10, 11, 12, 13, 14, 15, /* ECC data bytes
>> 0-511 + OOB bytes 4-7 */
>>                   24, 25, 26, 27, 28, 19, 30, 31, /* ECC data bytes
>> 512-1023 + OOB bytes 20-23 */
>>                   40, 41, 42, 43, 44, 45, 46, 47,  /* ECC data bytes
>> 1024-1535 + OOB bytes 36-39 */
>>                   56, 57, 58, 59, 60, 61, 62, 63},  /* ECC data bytes
>> 1536-2047 + OOB bytes 52-55 */
>>           .oobfree = {
>>                   { .offset = 4,
>>                     .length = 4},
>>                   { .offset = 20,
>>                     .length = 4},
>>                   { .offset = 36,
>>                     .length = 4},
>>                   { .offset = 52,
>>                     .length = 4},
>>           },
>> };
>>
>> After the JFFS2 cleanmarker is written into bytes 4-7 and 16-23 of the
>> OOB, nanddump shows:
>>
>>     OOB Data: ff ff ff ff 85 19 03 20 5a e3 da 69 01 40 f1 36
>>     OOB Data: ff ff ff ff 08 00 00 00 91 99 3c 05 01 d0 5d b3
>>     OOB Data: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>     OOB Data: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>
>> Note that the ECC bytes 8-15 and 24-31 are no longer FF due to bytes 4-7
>> and bytes 20-23 being written with non-zero data.
>>
>> When data is later written to this same page (w/o an intervening erase
>> of its block) reading the page causes an uncorrectable ECC error.
>>
>> There are eight additional bytes of OOB space available for writing, but
>> they are not ECC'd.
>>
>> The issue I'm trying to solve is how to communicate from MTD to JFFS2
>> that some bytes of the oobfree array perturb the data ECC and can not be
>> used to write the cleanmarker.
> OK, thanks for explanation. I am not very good in this area as I do not
> have much experience dealing with OOB, but here is what I thing.
>
> 1. Linux MTD code was _not_ designed for "ECC'ed OOB".
> 2. I do not really know what MTD_OOB_RAW is, and the comment in mtd.h
>     is not very verbose.
> 3. But in my opinion MTD_OOB_AUTO makes most sense and should be used
>     everywhere except for some tricky cases when you want to test things
>     by writing incorrect ECC, or you have an image with ECC and you want
>     to flash it as is.
> 4. In general, OOB should be considered as belonging to the driver, and
>     modern software should not rely on OOB at all.
> 5. So MTD_OOB_AUTO make free bytes in OOB look like a contiguous buffer
>     which the _user_ can freely and _independently_ use.
> 6. In your case only this assumption does not work and your ecclayout is
>     incorrect because the OOB areas you expose are not independent.
> 7. So in your case your ecclayout should be changed and you should
>     expose only independent ECC bytes.
The independent ECC bytes available only total to eight, which makes it 
impossible to use YAFFS (which needs at least 16 bytes to stor e its 
metadata in).  I can add the independent bytes at the end of the layout 
(and tweak YAFFS to ignore them since it needs only 16 bytes if the 
metadata is ECC'ed), but I still need to create a fix in JFFS2 (and 
u-boot and utilities) to skip the first 16 bytes of oobfree area.  
That's why I think another ioctl call (GETOOBLAYOUT?) would be useful to 
describe the oobfree list, as well as lists(or bitfields) that indicates 
which bytes in oobfree are either ECC'ed as part of the data ECC, or 
ECC'ed independently.

Another issue this exposes is that JFFS2 reads/compares the cleanmarker 
w/o any ECC in the marker data to verify its validity - if a bitflip in 
an unECC'd cleanmarker is read back, then I think JFFS2 will fail to use 
that block.

Also, from what I can find, MTD does not provide a method of programming 
OOB NAND data using MTD_OOB_AUTO as mtd_do_writeoob (called in mtdchar.c 
from the MTDWRITEOOB ioctl) uses only MTD_OOB_PLACE.

-- 
Peter Barada
peter.barada@gmail.com

Re: Preventing JFFS2 partial page writes?

[Artem Bityutskiy]

On Tue, 2011-06-28 at 14:56 -0400, Peter Barada wrote:
> On 06/28/2011 05:34 AM, Artem Bityutskiy wrote:
> > On Mon, 2011-06-27 at 10:31 -0400, Peter Barada wrote:
> >> On 06/24/2011 03:26 PM, Artem Bityutskiy wrote:
> >>> On Wed, 2011-06-22 at 11:28 -0400, Peter Barada wrote:
> >>>> Thoughts?
> >>> Sorry, could you please define the problem you are trying to solve?
> >>> Sorry if you did define it in your long post, but I could not easily
> >>> find it.
> >> The problem I'm trying to solve is that the Micron NAND  I'm using has
> >> an internal 4-bit ECC engine and uses four 8-byte ECCs that provide
> >> 4-bit protection per 512 data bytes + four OOB bytes.  The ecclayout I'm
> >> using is:
> >>
> >> ecclayout = {
> >>           eccbytes = 32,
> >>           eccpos = { 8, 9, 10, 11, 12, 13, 14, 15, /* ECC data bytes
> >> 0-511 + OOB bytes 4-7 */
> >>                   24, 25, 26, 27, 28, 19, 30, 31, /* ECC data bytes
> >> 512-1023 + OOB bytes 20-23 */
> >>                   40, 41, 42, 43, 44, 45, 46, 47,  /* ECC data bytes
> >> 1024-1535 + OOB bytes 36-39 */
> >>                   56, 57, 58, 59, 60, 61, 62, 63},  /* ECC data bytes
> >> 1536-2047 + OOB bytes 52-55 */
> >>           .oobfree = {
> >>                   { .offset = 4,
> >>                     .length = 4},
> >>                   { .offset = 20,
> >>                     .length = 4},
> >>                   { .offset = 36,
> >>                     .length = 4},
> >>                   { .offset = 52,
> >>                     .length = 4},
> >>           },
> >> };
> >>
> >> After the JFFS2 cleanmarker is written into bytes 4-7 and 16-23 of the
> >> OOB, nanddump shows:
> >>
> >>     OOB Data: ff ff ff ff 85 19 03 20 5a e3 da 69 01 40 f1 36
> >>     OOB Data: ff ff ff ff 08 00 00 00 91 99 3c 05 01 d0 5d b3
> >>     OOB Data: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
> >>     OOB Data: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
> >>
> >> Note that the ECC bytes 8-15 and 24-31 are no longer FF due to bytes 4-7
> >> and bytes 20-23 being written with non-zero data.
> >>
> >> When data is later written to this same page (w/o an intervening erase
> >> of its block) reading the page causes an uncorrectable ECC error.
> >>
> >> There are eight additional bytes of OOB space available for writing, but
> >> they are not ECC'd.
> >>
> >> The issue I'm trying to solve is how to communicate from MTD to JFFS2
> >> that some bytes of the oobfree array perturb the data ECC and can not be
> >> used to write the cleanmarker.
> > OK, thanks for explanation. I am not very good in this area as I do not
> > have much experience dealing with OOB, but here is what I thing.
> >
> > 1. Linux MTD code was _not_ designed for "ECC'ed OOB".
> > 2. I do not really know what MTD_OOB_RAW is, and the comment in mtd.h
> >     is not very verbose.
> > 3. But in my opinion MTD_OOB_AUTO makes most sense and should be used
> >     everywhere except for some tricky cases when you want to test things
> >     by writing incorrect ECC, or you have an image with ECC and you want
> >     to flash it as is.
> > 4. In general, OOB should be considered as belonging to the driver, and
> >     modern software should not rely on OOB at all.
> > 5. So MTD_OOB_AUTO make free bytes in OOB look like a contiguous buffer
> >     which the _user_ can freely and _independently_ use.
> > 6. In your case only this assumption does not work and your ecclayout is
> >     incorrect because the OOB areas you expose are not independent.
> > 7. So in your case your ecclayout should be changed and you should
> >     expose only independent ECC bytes.
> The independent ECC bytes available only total to eight, which makes it 
> impossible to use YAFFS (which needs at least 16 bytes to stor e its 
> metadata in). 

Are you sure YAFFS can use the dependent bytes? I am not sure, but I
thought YAFFS wants OOB to be independently writable, no?

>  I can add the independent bytes at the end of the layout 
> (and tweak YAFFS to ignore them since it needs only 16 bytes if the 
> metadata is ECC'ed), but I still need to create a fix in JFFS2 (and 
> u-boot and utilities) to skip the first 16 bytes of oobfree area. 

As I said, I think you need to take a good look at MTD and think how you
can teach it to distinguish with dependent and independent OOB bytes.

> That's why I think another ioctl call (GETOOBLAYOUT?) would be useful to 
> describe the oobfree list, as well as lists(or bitfields) that indicates 
> which bytes in oobfree are either ECC'ed as part of the data ECC, or 
> ECC'ed independently.

Why new ioctl? Why not add another OOB access type (like MTD_OOB_AUTO)?
Or may be one of the existing can be re-use? This should be carefully
analyzed.

> Another issue this exposes is that JFFS2 reads/compares the cleanmarker 
> w/o any ECC in the marker data to verify its validity - if a bitflip in 
> an unECC'd cleanmarker is read back, then I think JFFS2 will fail to use 
> that block.

No, I think what JFFS2 should do is just assume the eraseblock needs
erasure and just erase it. The whole purpose of clean markers is to
erase less. If it is corrupted - we just do an extra erase - not big
deal.

> Also, from what I can find, MTD does not provide a method of programming 
> OOB NAND data using MTD_OOB_AUTO as mtd_do_writeoob (called in mtdchar.c 
> from the MTDWRITEOOB ioctl) uses only MTD_OOB_PLACE.

Probably MTD_OOB_AUTO is just a layer on top of MTD_OOB_PLACE? Anyway,
this whole area of OOB is complex and needs some analysis, and even
better documentation, and then you can choose the best way to solve your
issue.

-- 
Best Regards,
Artem Bityutskiy

Re: Preventing JFFS2 partial page writes?

[Peter Barada]

On 06/29/2011 02:33 AM, Artem Bityutskiy wrote:
> On Tue, 2011-06-28 at 14:56 -0400, Peter Barada wrote:
>> On 06/28/2011 05:34 AM, Artem Bityutskiy wrote:
>>> On Mon, 2011-06-27 at 10:31 -0400, Peter Barada wrote:
>>>> On 06/24/2011 03:26 PM, Artem Bityutskiy wrote:
>>>>> On Wed, 2011-06-22 at 11:28 -0400, Peter Barada wrote:
>>>>>> Thoughts?
>>>>> Sorry, could you please define the problem you are trying to solve?
>>>>> Sorry if you did define it in your long post, but I could not easily
>>>>> find it.
>>>> The problem I'm trying to solve is that the Micron NAND  I'm using has
>>>> an internal 4-bit ECC engine and uses four 8-byte ECCs that provide
>>>> 4-bit protection per 512 data bytes + four OOB bytes.  The ecclayout I'm
>>>> using is:
>>>>
>>>> ecclayout = {
>>>>            eccbytes = 32,
>>>>            eccpos = { 8, 9, 10, 11, 12, 13, 14, 15, /* ECC data bytes
>>>> 0-511 + OOB bytes 4-7 */
>>>>                    24, 25, 26, 27, 28, 19, 30, 31, /* ECC data bytes
>>>> 512-1023 + OOB bytes 20-23 */
>>>>                    40, 41, 42, 43, 44, 45, 46, 47,  /* ECC data bytes
>>>> 1024-1535 + OOB bytes 36-39 */
>>>>                    56, 57, 58, 59, 60, 61, 62, 63},  /* ECC data bytes
>>>> 1536-2047 + OOB bytes 52-55 */
>>>>            .oobfree = {
>>>>                    { .offset = 4,
>>>>                      .length = 4},
>>>>                    { .offset = 20,
>>>>                      .length = 4},
>>>>                    { .offset = 36,
>>>>                      .length = 4},
>>>>                    { .offset = 52,
>>>>                      .length = 4},
>>>>            },
>>>> };
>>>>
>>>> After the JFFS2 cleanmarker is written into bytes 4-7 and 16-23 of the
>>>> OOB, nanddump shows:
>>>>
>>>>      OOB Data: ff ff ff ff 85 19 03 20 5a e3 da 69 01 40 f1 36
>>>>      OOB Data: ff ff ff ff 08 00 00 00 91 99 3c 05 01 d0 5d b3
>>>>      OOB Data: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>>>      OOB Data: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>>>>
>>>> Note that the ECC bytes 8-15 and 24-31 are no longer FF due to bytes 4-7
>>>> and bytes 20-23 being written with non-zero data.
>>>>
>>>> When data is later written to this same page (w/o an intervening erase
>>>> of its block) reading the page causes an uncorrectable ECC error.
>>>>
>>>> There are eight additional bytes of OOB space available for writing, but
>>>> they are not ECC'd.
>>>>
>>>> The issue I'm trying to solve is how to communicate from MTD to JFFS2
>>>> that some bytes of the oobfree array perturb the data ECC and can not be
>>>> used to write the cleanmarker.
>>> OK, thanks for explanation. I am not very good in this area as I do not
>>> have much experience dealing with OOB, but here is what I thing.
>>>
>>> 1. Linux MTD code was _not_ designed for "ECC'ed OOB".
>>> 2. I do not really know what MTD_OOB_RAW is, and the comment in mtd.h
>>>      is not very verbose.
>>> 3. But in my opinion MTD_OOB_AUTO makes most sense and should be used
>>>      everywhere except for some tricky cases when you want to test things
>>>      by writing incorrect ECC, or you have an image with ECC and you want
>>>      to flash it as is.
>>> 4. In general, OOB should be considered as belonging to the driver, and
>>>      modern software should not rely on OOB at all.
>>> 5. So MTD_OOB_AUTO make free bytes in OOB look like a contiguous buffer
>>>      which the _user_ can freely and _independently_ use.
>>> 6. In your case only this assumption does not work and your ecclayout is
>>>      incorrect because the OOB areas you expose are not independent.
>>> 7. So in your case your ecclayout should be changed and you should
>>>      expose only independent ECC bytes.
>> The independent ECC bytes available only total to eight, which makes it
>> impossible to use YAFFS (which needs at least 16 bytes to stor e its
>> metadata in).
> Are you sure YAFFS can use the dependent bytes? I am not sure, but I
> thought YAFFS wants OOB to be independently writable, no?
YAFFS2 only writes a page once so it can use dependent bytes.  I have 
YAFFS2 working with the current ECC
layout where it *only* uses the dependent bytes, and units are already 
in the field.  YAFFS1 would delete a page by writing into the OOB area a 
2nd time.
>>   I can add the independent bytes at the end of the layout
>> (and tweak YAFFS to ignore them since it needs only 16 bytes if the
>> metadata is ECC'ed), but I still need to create a fix in JFFS2 (and
>> u-boot and utilities) to skip the first 16 bytes of oobfree area.
> As I said, I think you need to take a good look at MTD and think how you
> can teach it to distinguish with dependent and independent OOB bytes.
I assume you imply that the mtd-utils need to be changed in addition to 
MTD (to at least communicate to MTD not to write OOB data into dependent 
bytes).
>> That's why I think another ioctl call (GETOOBLAYOUT?) would be useful to
>> describe the oobfree list, as well as lists(or bitfields) that indicates
>> which bytes in oobfree are either ECC'ed as part of the data ECC, or
>> ECC'ed independently.
> Why new ioctl? Why not add another OOB access type (like MTD_OOB_AUTO)?
> Or may be one of the existing can be re-use? This should be carefully
> analyzed.
Grepping through a current 2.6 kernel I don't see any way to set the OOB 
method from usersapce.  MEMSETOOBSEL only exists in 
include/mtd/mtd-abi.h and nowhere in driver/mtd.  It looks like it was 
removed way back by Vitaly Wool back in 11/2005.  The current 
MEMWRITEOOB only uses MTD_OOB_PLACE in the kernel (see mtd_do_writeoob() 
in drivers/mtd/mtdchar.c).  Would adding a field (oobtype) in struct 
mtd_oob_buf be the way to go?
>> Another issue this exposes is that JFFS2 reads/compares the cleanmarker
>> w/o any ECC in the marker data to verify its validity - if a bitflip in
>> an unECC'd cleanmarker is read back, then I think JFFS2 will fail to use
>> that block.
> No, I think what JFFS2 should do is just assume the eraseblock needs
> erasure and just erase it. The whole purpose of clean markers is to
> erase less. If it is corrupted - we just do an extra erase - not big
> deal.
How's the best way to do this?  That would make this whole problem just 
go away, and make MLC devices much more happy with JFFS2.
>> Also, from what I can find, MTD does not provide a method of programming
>> OOB NAND data using MTD_OOB_AUTO as mtd_do_writeoob (called in mtdchar.c
>> from the MTDWRITEOOB ioctl) uses only MTD_OOB_PLACE.
> Probably MTD_OOB_AUTO is just a layer on top of MTD_OOB_PLACE? Anyway,
> this whole area of OOB is complex and needs some analysis, and even
> better documentation, and then you can choose the best way to solve your
> issue.
MTD_OOB_PLACE and MTD_OOB_AUTO follow the same path and use 
nand_fill_oob() and nand_transfer_oob() to write/extract the OOB buffer 
used in the actual write.


-- 
Peter Barada
peter.barada@gmail.com

Re: Preventing JFFS2 partial page writes?

[Ivan Djelic]

On Tue, Jun 28, 2011 at 10:39:47AM +0100, Artem Bityutskiy wrote:
> On Tue, 2011-06-28 at 12:34 +0300, Artem Bityutskiy wrote:
> > OK, thanks for explanation. I am not very good in this area as I do not
> > have much experience dealing with OOB, but here is what I thing.
> > 
> > 1. Linux MTD code was _not_ designed for "ECC'ed OOB".
> > 2. I do not really know what MTD_OOB_RAW is, and the comment in mtd.h
> >    is not very verbose.
> > 3. But in my opinion MTD_OOB_AUTO makes most sense and should be used
> >    everywhere except for some tricky cases when you want to test things
> >    by writing incorrect ECC, or you have an image with ECC and you want
> >    to flash it as is.
> > 4. In general, OOB should be considered as belonging to the driver, and
> >    modern software should not rely on OOB at all.
> > 5. So MTD_OOB_AUTO make free bytes in OOB look like a contiguous buffer
> >    which the _user_ can freely and _independently_ use.
> > 6. In your case only this assumption does not work and your ecclayout is
> >    incorrect because the OOB areas you expose are not independent.
> > 7. So in your case your ecclayout should be changed and you should
> >    expose only independent ECC bytes.
> 
> To put it differently, I current model does not distinguish (I think,
> correct me if I am wrong) between ECC'd OOB bytes and ECC'less OOB
> bytes. BTW, does your flash has the latter?
> 
> So MTD would need some work to make it distinguish between those 2 types
> of OOB bytes - probably additional info could be added to the ooblayout
> structure, and the interfaces could be improved. How exactly - dunno,
> I'd first need to figure out what MTD_OOB_RAW is - may be Brian or Ivan
> could comment.

I agree with the idea that OOB should be considered as belonging to the driver.
I think the problem should be solved as follows:

1. Expose only unprotected (or "independent") bytes in your ecclayout. Those
bytes will be used by JFFS2 for its cleanmarker.

2. Use YAFFS2 "inband-tags" option to prevent YAFFS2 from using oob for storing
metadata.

If for some reason you really cannot use inband-tags, then patch YAFFS2 and add
an option so that it can use MTD_OOB_PLACE instead of MTD_OOB_AUTO, and
store its metadata into a specified list of protected OOB bytes.

Rationale: you would have to configure YAFFS2 for this specific device anyway,
by using YAFFS_DISABLE_TAGS_ECC or tags_ecc_off in order to let nand on-die ecc
protect metadata.

I would rather not add new complexity in mtd ecclayout to solve your problem,
because it is a bit too specific (your client insists on not using UBIFS which
would be better suited for this generation of nand devices) and this new
interface would probably be short-lived (as discussed in
http://lists.infradead.org/pipermail/linux-mtd/2011-June/036549.html).

What do you think ?

--
Best Regards,

Ivan

Re: Preventing JFFS2 partial page writes?

[Ivan Djelic]

On Thu, Jun 30, 2011 at 07:05:25PM +0100, Peter Barada wrote:
(...)
> >> Another issue this exposes is that JFFS2 reads/compares the cleanmarker
> >> w/o any ECC in the marker data to verify its validity - if a bitflip in
> >> an unECC'd cleanmarker is read back, then I think JFFS2 will fail to use
> >> that block.
> > No, I think what JFFS2 should do is just assume the eraseblock needs
> > erasure and just erase it. The whole purpose of clean markers is to
> > erase less. If it is corrupted - we just do an extra erase - not big
> > deal.
> How's the best way to do this?  That would make this whole problem just 
> go away, and make MLC devices much more happy with JFFS2.

I think Artem was just explaining that a single corrupted cleanmarker is no
big deal, it just forces JFFS2 to perform an extra erase. That does not mean
you can get rid of cleanmarkers, because systematically erasing a block before
programming it would seriously degrade write performance.

--
Best Regards,

Ivan

Re: Preventing JFFS2 partial page writes?

[Artem Bityutskiy]

On Fri, 2011-07-01 at 22:48 +0200, Ivan Djelic wrote:
> On Tue, Jun 28, 2011 at 10:39:47AM +0100, Artem Bityutskiy wrote:
> > On Tue, 2011-06-28 at 12:34 +0300, Artem Bityutskiy wrote:
> > > OK, thanks for explanation. I am not very good in this area as I do not
> > > have much experience dealing with OOB, but here is what I thing.
> > > 
> > > 1. Linux MTD code was _not_ designed for "ECC'ed OOB".
> > > 2. I do not really know what MTD_OOB_RAW is, and the comment in mtd.h
> > >    is not very verbose.
> > > 3. But in my opinion MTD_OOB_AUTO makes most sense and should be used
> > >    everywhere except for some tricky cases when you want to test things
> > >    by writing incorrect ECC, or you have an image with ECC and you want
> > >    to flash it as is.
> > > 4. In general, OOB should be considered as belonging to the driver, and
> > >    modern software should not rely on OOB at all.
> > > 5. So MTD_OOB_AUTO make free bytes in OOB look like a contiguous buffer
> > >    which the _user_ can freely and _independently_ use.
> > > 6. In your case only this assumption does not work and your ecclayout is
> > >    incorrect because the OOB areas you expose are not independent.
> > > 7. So in your case your ecclayout should be changed and you should
> > >    expose only independent ECC bytes.
> > 
> > To put it differently, I current model does not distinguish (I think,
> > correct me if I am wrong) between ECC'd OOB bytes and ECC'less OOB
> > bytes. BTW, does your flash has the latter?
> > 
> > So MTD would need some work to make it distinguish between those 2 types
> > of OOB bytes - probably additional info could be added to the ooblayout
> > structure, and the interfaces could be improved. How exactly - dunno,
> > I'd first need to figure out what MTD_OOB_RAW is - may be Brian or Ivan
> > could comment.
> 
> I agree with the idea that OOB should be considered as belonging to the driver.
> I think the problem should be solved as follows:
> 
> 1. Expose only unprotected (or "independent") bytes in your ecclayout. Those
> bytes will be used by JFFS2 for its cleanmarker.
> 
> 2. Use YAFFS2 "inband-tags" option to prevent YAFFS2 from using oob for storing
> metadata.
> 
> If for some reason you really cannot use inband-tags, then patch YAFFS2 and add
> an option so that it can use MTD_OOB_PLACE instead of MTD_OOB_AUTO, and
> store its metadata into a specified list of protected OOB bytes.
> 
> Rationale: you would have to configure YAFFS2 for this specific device anyway,
> by using YAFFS_DISABLE_TAGS_ECC or tags_ecc_off in order to let nand on-die ecc
> protect metadata.
> 
> I would rather not add new complexity in mtd ecclayout to solve your problem,
> because it is a bit too specific (your client insists on not using UBIFS which
> would be better suited for this generation of nand devices) and this new
> interface would probably be short-lived (as discussed in
> http://lists.infradead.org/pipermail/linux-mtd/2011-June/036549.html).
> 
> What do you think ?

Yes, I agree, this sounds much saner than trying to teach MTD to
distinguish between protected and unprotected areas. However, if Peter
is able to come up with a really really nice patch-set which adds the
functionality he needs in a nice way (good docs, good code separation,
clean-up of the current stuff, good testing) - why not? But I think he'd
need to do _a lot of_ work to achieve this.

-- 
Best Regards,
Artem Bityutskiy

Re: Preventing JFFS2 partial page writes?

[Peter Barada]

On 07/01/2011 04:52 PM, Ivan Djelic wrote:
> On Thu, Jun 30, 2011 at 07:05:25PM +0100, Peter Barada wrote:
> (...)
>>>> Another issue this exposes is that JFFS2 reads/compares the cleanmarker
>>>> w/o any ECC in the marker data to verify its validity - if a bitflip in
>>>> an unECC'd cleanmarker is read back, then I think JFFS2 will fail to use
>>>> that block.
>>> No, I think what JFFS2 should do is just assume the eraseblock needs
>>> erasure and just erase it. The whole purpose of clean markers is to
>>> erase less. If it is corrupted - we just do an extra erase - not big
>>> deal.
>> How's the best way to do this?  That would make this whole problem just
>> go away, and make MLC devices much more happy with JFFS2.
> I think Artem was just explaining that a single corrupted cleanmarker is no
> big deal, it just forces JFFS2 to perform an extra erase. That does not mean
> you can get rid of cleanmarkers, because systematically erasing a block before
> programming it would seriously degrade write performance.
What I meant was that for the first write to a block (which I hope will 
be to the page that contains the cleanmarker), erase the block before 
the write, merge the page data and clean marker into one write).  This 
shouldn't degrade performance that much (and could be enabled/disabled 
by a mount option).  If that is possible (and someone can provide a 
pointer to me on how to do such) its a much better solution than 
butchering ecclayout.

> --
> Best Regards,
>
> Ivan


-- 
Peter Barada
peter.barada@gmail.com