more flexible HW ECC support for nand_base

more flexible HW ECC support for nand_base

[Vitaly Wool]

Hi  Thomas,

as you might recall, during the PNX4008 LSP development we came across 
the MTD problem which turned to be a shipstopper for hardware ECC 
support on SanDisk MLC controller.  The patch (quite inmature) suggested 
and a kind of struggle around it are in September linux-mtd e-mail 
archive ;-)
The problem basically is that the MTD NAND core assumes that layout is 
like "(data bytes) (possible ecc data) (oob data)", and more often just 
"(data bytes) (oob data)". To perform ECC data reading, the NAND chip 
driver needs to set the special option like NAND_HWECC_SYNDROME.
On the other hand, the page layout may vary from chip to chip. For 
SanDisk MLC it looks like 
"(data)(ecc)(oob)(data)(ecc)(oob)(data)(ecc)(oob)(data)(ecc)(oob)" what 
can't be handled properly by nand_base since it doesn't even try to read 
OOB data in loop (i. e. eccsteps times).

After some local discussions, we suggest the following modification to 
the MTD NAND core to allow driver to specify its own layout of nand 
page. Namely, the solution proposed is to provide the `page_layout' 
field in nand_chip structure, that will point to array of structures like

struct page_layout_item {
    int length;
    enum {
        SPARE, DATA, OOB, ECC,
    } type;
}

Any standard types like NAND_HW512_6 will be replaced then by standard 
layouts like
    nand_layout_512_6 = {
        { 512, DATA },
        { 6, ECC },
    };
The rest of page, if any, might be considered as OOB data or explicitly 
defined, like
        { 10, OOB }

Another part of the problem is nand_read_oob()/nand_write_oob() 
functions that also make assumptions on NAND flash page layout. It's 
suggested that they go to nand_chip structure, i. e. it will become a 
specific chip driver's responsibility to provide those; and if driver 
does not provide one, the current implementation can be used. If 
nand_chip provides its own layout, it must read/write oob data according 
to the layout.

These steps should provide backward compatibility and the ability for 
particular driver owners to update their drivers accordingly with as 
less pain as possible.

Any comments are welcome,

Best regards,
  Vitaly

Re: more flexible HW ECC support for nand_base

[Charles Manning]

On Friday 30 September 2005 23:57, Vitaly Wool wrote:

> Another part of the problem is nand_read_oob()/nand_write_oob()
> functions that also make assumptions on NAND flash page layout. It's
> suggested that they go to nand_chip structure, i. e. it will become a
> specific chip driver's responsibility to provide those; and if driver
> does not provide one, the current implementation can be used. If
> nand_chip provides its own layout, it must read/write oob data according
> to the layout.
>
> These steps should provide backward compatibility and the ability for
> particular driver owners to update their drivers accordingly with as
> less pain as possible.
>
> Any comments are welcome,
>

Hi Vitaly

I'm sorry I could not access the list archive to see if there has been further 
progress on this topic, so I apologise upfront if this wrecks the flow of the 
thread.

I think this might also be just what we're looking for to fix some problems we 
see in YAFFSland.

YAFFS2 uses AUTOPLACE for read_ecc and write_ecc, with the idea that YAFFS2 
can just pass in a buffer and mtd can do all the byte placement. YAFFS2 does 
not need (and should not have) knowledge of the actual physical placement. 

That part works fine, but YAFFS2 also needs to read only the tags (in the oob) 
during mount scanning (and at a few other times like gc). We obviously don't 
want to read the entire data area and do ECC etc because that is just wasted 
work. Instead we want to just do read_oob.

At the moment we have the situation that read_oob does a raw read, so you 
don't get the same bytes as you wrote with AUTOPLACE.  Some people hope that 
the only difference [with 2k pages] is the 2 bytes of bad block marker and so 
read at an offset of 2. That works for some, but won't work for everyone.

If I understand it correctly, your proposal would use an implied AUTOPLACE for 
read_oob. In other words read_oob will provide the same oob bytes as read_ecc 
with AUTOPLACE. 

That would get you some fans in YAFFSland.

-- CHarles

Re: more flexible HW ECC support for nand_base

[Vitaly Wool]

Hi Charles,

as for now, I was mainly targeting just to enable hardware ECC support 
for some complicated page layouts within the linix-mtd scope. I was also 
trying to minimize the impact of my changes, i. e. make them as 
transparent as possible for all the nand_base users, and it looks like I 
had some success with it.

I do agree with you that in future read_oob better return OOB data in 
the same format as read_ecc/write_ecc but as of now, this is not 
impemented in my patch. The layout-based approach however is powerful 
enough to allow doing that... in the closest future, if the current 
piece of work gets accepted.

Best regards,
   Vitaly


Charles Manning wrote:

>On Friday 30 September 2005 23:57, Vitaly Wool wrote:
>
>  
>
>>Another part of the problem is nand_read_oob()/nand_write_oob()
>>functions that also make assumptions on NAND flash page layout. It's
>>suggested that they go to nand_chip structure, i. e. it will become a
>>specific chip driver's responsibility to provide those; and if driver
>>does not provide one, the current implementation can be used. If
>>nand_chip provides its own layout, it must read/write oob data according
>>to the layout.
>>
>>These steps should provide backward compatibility and the ability for
>>particular driver owners to update their drivers accordingly with as
>>less pain as possible.
>>
>>Any comments are welcome,
>>
>>    
>>
>
>Hi Vitaly
>
>I'm sorry I could not access the list archive to see if there has been further 
>progress on this topic, so I apologise upfront if this wrecks the flow of the 
>thread.
>
>I think this might also be just what we're looking for to fix some problems we 
>see in YAFFSland.
>
>YAFFS2 uses AUTOPLACE for read_ecc and write_ecc, with the idea that YAFFS2 
>can just pass in a buffer and mtd can do all the byte placement. YAFFS2 does 
>not need (and should not have) knowledge of the actual physical placement. 
>
>That part works fine, but YAFFS2 also needs to read only the tags (in the oob) 
>during mount scanning (and at a few other times like gc). We obviously don't 
>want to read the entire data area and do ECC etc because that is just wasted 
>work. Instead we want to just do read_oob.
>
>At the moment we have the situation that read_oob does a raw read, so you 
>don't get the same bytes as you wrote with AUTOPLACE.  Some people hope that 
>the only difference [with 2k pages] is the 2 bytes of bad block marker and so 
>read at an offset of 2. That works for some, but won't work for everyone.
>
>If I understand it correctly, your proposal would use an implied AUTOPLACE for 
>read_oob. In other words read_oob will provide the same oob bytes as read_ecc 
>with AUTOPLACE. 
>
>That would get you some fans in YAFFSland.
>
>-- CHarles
>
>
>  
>