mtd, mtdblock and nand ecc.

mtd, mtdblock and nand ecc.

[David Daney]

Is there some reason that the mtd and mtdblock drivers do not use ECC 
when the backing device is nand with ECC enabled?

I guess that kind of sums up my question.

If the answer is no, I am thinking of patching them so that they do use 
ECC when available.

I suppose you could run into problems if the backing device were a 
concatenation of ECC and non-ECC devices, but that could probably be 
detected either on open or at each page read/write.

Thanks in advance,

David Daney.

Re: mtd, mtdblock and nand ecc.

[Thomas Gleixner]

On Wednesday 14 April 2004 06:37, David Daney wrote:
> Is there some reason that the mtd and mtdblock drivers do not use ECC
> when the backing device is nand with ECC enabled?

Yes. 
If the oobsel structure == NULL, then the nand driver uses the structure, 
which is associated to this mtd device. This can be done from userspace using 
MEMSETOOBSEL ioctl.
NAND aware filesystem drivers provide their own oobsel structure and use the 
xxx_ecc functions.

> I suppose you could run into problems if the backing device were a
> concatenation of ECC and non-ECC devices, but that could probably be
> detected either on open or at each page read/write.

Concatenation of NAND and NOR will not work anyway, as the fs must be aware of 
the FLASH type.

-- 
Thomas
________________________________________________________________________
"Free software" is a matter of liberty, not price. To understand the concept,
you should think of "free" as in "free speech,'' not as in "free beer".
________________________________________________________________________
linutronix - competence in embedded & realtime linux
http://www.linutronix.de
mail: tglx@linutronix.de

Re: mtd, mtdblock and nand ecc.

[David Daney]

Thomas Gleixner wrote:

>On Wednesday 14 April 2004 06:37, David Daney wrote:
>  
>
>>Is there some reason that the mtd and mtdblock drivers do not use ECC
>>when the backing device is nand with ECC enabled?
>>    
>>
>
>Yes. 
>If the oobsel structure == NULL, then the nand driver uses the structure, 
>which is associated to this mtd device. This can be done from userspace using 
>MEMSETOOBSEL ioctl.
>NAND aware filesystem drivers provide their own oobsel structure and use the 
>xxx_ecc functions.
>  
>
I am using the cramfs on a NAND partition as my root file system.  
cramfs is not NAND aware, and I cannot be running userspace programs 
before mounting as it is the root file system.

I have not completely educated myself on the mtdblock driver.  Since the 
mtdblock driver can be used by non-mtd-aware filesystems, I am proposing 
making mtdblock NAND aware so that it uses the xxx_ecc functions iff ECC 
is available.  Perhaps there would be a kernel/module command line 
switch to help manage the behavior.

Thoughts?

David Daney

Re: mtd, mtdblock and nand ecc.

[Thomas Gleixner]

On Wednesday 14 April 2004 16:11, David Daney wrote:

> >NAND aware filesystem drivers provide their own oobsel structure and use
> > the xxx_ecc functions.
>
> I am using the cramfs on a NAND partition as my root file system.
> cramfs is not NAND aware, and I cannot be running userspace programs
> before mounting as it is the root file system.

I know, but why must you use cramfs ? Why dont you use jffs2 or yaffs as your 
root fs. Mount it r/o, so you have no hassle at all.

> I have not completely educated myself on the mtdblock driver.  Since the
> mtdblock driver can be used by non-mtd-aware filesystems, I am proposing
> making mtdblock NAND aware so that it uses the xxx_ecc functions iff ECC
> is available.  Perhaps there would be a kernel/module command line
> switch to help manage the behavior.
>
> Thoughts?

mtdblock is a block device driver and only provides an interface. It must not 
be aware of anything. 
Using NAND unaware filesystems on NAND is nothing we want to support.
ECC is only one part of NAND support. What about bad blocks? NAND chips can 
have bad blocks, even when they are new. Only block 0 is guaranteed to be not 
bad at delivery time. How want you deal with a board, where a bad block is in 
the partition which is reserved for your cramfs ?

We have two reliable working NAND aware filesystems around. I don't see any 
reason to provide support for predictable trouble. 

-- 
Thomas
________________________________________________________________________
"Free software" is a matter of liberty, not price. To understand the concept,
you should think of "free" as in "free speech,'' not as in "free beer".
________________________________________________________________________
linutronix - competence in embedded & realtime linux
http://www.linutronix.de
mail: tglx@linutronix.de

Re: mtd, mtdblock and nand ecc.

[David Daney]

Let me start by saying that I am not trying to cause problems, but just 
to understand the best options...

Thomas Gleixner wrote:

>On Wednesday 14 April 2004 16:11, David Daney wrote:
>
>  
>
>>>NAND aware filesystem drivers provide their own oobsel structure and use
>>>the xxx_ecc functions.
>>>      
>>>
>>I am using the cramfs on a NAND partition as my root file system.
>>cramfs is not NAND aware, and I cannot be running userspace programs
>>before mounting as it is the root file system.
>>    
>>
>
>I know, but why must you use cramfs ? Why dont you use jffs2 or yaffs as your 
>root fs. Mount it r/o, so you have no hassle at all.
>  
>
With my NAND drivers, booting the linux kernel and mounting a minimal 
root file system on a 16MB flash takes 1:08 for yaffs and 1:25 for 
jffs2.  Using cramfs it boots in under 0:10.

That is why I am thinking about using a non NAND aware file system for 
things that can be read-only.

>  
>
>>I have not completely educated myself on the mtdblock driver.  Since the
>>mtdblock driver can be used by non-mtd-aware filesystems, I am proposing
>>making mtdblock NAND aware so that it uses the xxx_ecc functions iff ECC
>>is available.  Perhaps there would be a kernel/module command line
>>switch to help manage the behavior.
>>
>>Thoughts?
>>    
>>
>
>mtdblock is a block device driver and only provides an interface. It must not 
>be aware of anything. 
>
That is not quite correct.  mtdblock is well aware of the mtd backend.  
It does this:
    ret = MTD_WRITE (mtd, pos, len, &retlen, buf);

All I am suggesting is to have it do MTD_WRITE_ECC when possible.

>Using NAND unaware filesystems on NAND is nothing we want to support.
>ECC is only one part of NAND support. What about bad blocks? NAND chips can 
>have bad blocks, even when they are new. Only block 0 is guaranteed to be not 
>bad at delivery time. How want you deal with a board, where a bad block is in 
>the partition which is reserved for your cramfs ?
>

>We have two reliable working NAND aware filesystems around. I don't see any 
>reason to provide support for predictable trouble. 
>
>  
>
You already support it.  /dev/mtd and /dev/mtdblock work off-the-shelf 
with NAND devices and allow arbitrary programs/filesystems to overwrite 
bad blocks if they choose.

All I was thinking about was the ECC issue.

David Daney.

Re: mtd, mtdblock and nand ecc.

[Thomas Gleixner]

On Wednesday 14 April 2004 17:13, David Daney wrote:
> Let me start by saying that I am not trying to cause problems, but just
> to understand the best options...

I was not saying, that you are causing problems :)

> Thomas Gleixner wrote:
> >On Wednesday 14 April 2004 16:11, David Daney wrote:
> >>>NAND aware filesystem drivers provide their own oobsel structure and use
> >>>the xxx_ecc functions.
> >>
> >>I am using the cramfs on a NAND partition as my root file system.
> >>cramfs is not NAND aware, and I cannot be running userspace programs
> >>before mounting as it is the root file system.
> >
> >I know, but why must you use cramfs ? Why dont you use jffs2 or yaffs as
> > your root fs. Mount it r/o, so you have no hassle at all.
>
> With my NAND drivers, booting the linux kernel and mounting a minimal
> root file system on a 16MB flash takes 1:08 for yaffs and 1:25 for
> jffs2.  Using cramfs it boots in under 0:10.

He ? 1 minute ? Where is the time spent ?
Thats totaly out of the usual time. My boot time with JFFS2 is well below 30s 
on an ARM7. I have no YAFFS root fs handy, but it is much faster.

The solution is checking where the time is lost and fixing it.

> That is why I am thinking about using a non NAND aware file system for
> things that can be read-only.

But this does not answer my question how you want to deal with bad blocks ?

> >
> >mtdblock is a block device driver and only provides an interface. It must
> > not be aware of anything.
>
> That is not quite correct.  mtdblock is well aware of the mtd backend.
> It does this:
>     ret = MTD_WRITE (mtd, pos, len, &retlen, buf);

Yes, thats a wrapper nothing else. The wrapper is not aware of anything else 
than the MTD API itself. It does not care of FLASH types. And it will not 
care of FLASH types. There are a bunch of new FLASH types in the pipeline. 
Each of them has seperate problems. They have to be handled in the driver, 
nowhere else.

> All I am suggesting is to have it do MTD_WRITE_ECC when possible.

The nand driver provides a mechanism for using ecc without calling the _ecc 
functions already.
Provide a valid oobsel structure in your hardware driver. Place it into 
mtd->oobinfo. Then the nand driver uses this when reading from the chip
even when it is called by the non ecc functions. 
Thats the way we provide ecc to userspace too. As the user must be able to 
select an ecc / oob scheme for the partiton he wants to access.
see utils/nandwrite.c

I'm reworking the driver at the moment. It will have a builtin default ecc 
scheme then.

> >Using NAND unaware filesystems on NAND is nothing we want to support.
> >ECC is only one part of NAND support. What about bad blocks? NAND chips
> > can have bad blocks, even when they are new. Only block 0 is guaranteed
> > to be not bad at delivery time. How want you deal with a board, where a
> > bad block is in the partition which is reserved for your cramfs ?
> >
> >We have two reliable working NAND aware filesystems around. I don't see
> > any reason to provide support for predictable trouble.
>
> You already support it.  /dev/mtd and /dev/mtdblock work off-the-shelf
> with NAND devices and allow arbitrary programs/filesystems to overwrite
> bad blocks if they choose.

No, bad blocks are protected in the nand driver itself. Try to erase a bad 
block.

We _cannot_ hold anybody off to use it with any driver, userspace app which is 
able to access /dev/mtdxxxx. Then we would lock out the NAND aware fs too :)

You can also write nonsense to your harddisk. There is no real limitation. 

-- 
Thomas
________________________________________________________________________
"Free software" is a matter of liberty, not price. To understand the concept,
you should think of "free" as in "free speech,'' not as in "free beer".
________________________________________________________________________
linutronix - competence in embedded & realtime linux
http://www.linutronix.de
mail: tglx@linutronix.de

Re: mtd, mtdblock and nand ecc.

[David Daney]

Thomas Gleixner wrote:

>On Wednesday 14 April 2004 17:13, David Daney wrote:
>  
>
>>Let me start by saying that I am not trying to cause problems, but just
>>to understand the best options...
>>    
>>
>
>I was not saying, that you are causing problems :)
>
>  
>
>>Thomas Gleixner wrote:
>>    
>>
>>>On Wednesday 14 April 2004 16:11, David Daney wrote:
>>>      
>>>
>>>>>NAND aware filesystem drivers provide their own oobsel structure and use
>>>>>the xxx_ecc functions.
>>>>>          
>>>>>
>>>>I am using the cramfs on a NAND partition as my root file system.
>>>>cramfs is not NAND aware, and I cannot be running userspace programs
>>>>before mounting as it is the root file system.
>>>>        
>>>>
>>>I know, but why must you use cramfs ? Why dont you use jffs2 or yaffs as
>>>your root fs. Mount it r/o, so you have no hassle at all.
>>>      
>>>
>>With my NAND drivers, booting the linux kernel and mounting a minimal
>>root file system on a 16MB flash takes 1:08 for yaffs and 1:25 for
>>jffs2.  Using cramfs it boots in under 0:10.
>>    
>>
>
>He ? 1 minute ? Where is the time spent ?
>
How should I know?


>Thats totaly out of the usual time. My boot time with JFFS2 is well below 30s 
>on an ARM7. I have no YAFFS root fs handy, but it is much faster.
>
>The solution is checking where the time is lost and fixing it.
>  
>
I have a custom nand driver (based on nand.c, but adapted to a non 
standard nand physical interface) that uses software ECC.  On a 300 MHz 
mips32 cpu.

When ever jffs2 or yaffs are mounted, they both seem to read many 
pages.  Perhaps the ECC overhead of reading all that data.

I suppose I could turn off ECC and see how fast it is...

>  
>
>>That is why I am thinking about using a non NAND aware file system for
>>things that can be read-only.
>>    
>>
>
>But this does not answer my question how you want to deal with bad blocks ?
>  
>
I want a file system very much like cramfs, but that can have holes in 
it so that it works on NAND.

When mounted it would start scanning blocks starting from the beginning 
of the NAND partition until it found a valid superblock (or what ever 
you would call it).  Then it would be done because all of the indexes 
would be built to work around the bad blocks.  Since this filesystem 
would be read-only with a static structure, you would never have to read 
more than necessary.

David Daney

Re: mtd, mtdblock and nand ecc.

[Thomas Gleixner]

On Wednesday 14 April 2004 18:49, David Daney wrote:
> >
> >He ? 1 minute ? Where is the time spent ?
> How should I know?

Profiling :)

> >Thats totaly out of the usual time. My boot time with JFFS2 is well below
> > 30s on an ARM7. I have no YAFFS root fs handy, but it is much faster.
> >
> >The solution is checking where the time is lost and fixing it.
>
> I have a custom nand driver (based on nand.c, but adapted to a non
> standard nand physical interface) that uses software ECC.  On a 300 MHz
> mips32 cpu.

Why did you modify nand.c ? 
Almost everything in nand.c can be overridden by the board driver. Therefor we 
call all the functions through chip->xxx().

> When ever jffs2 or yaffs are mounted, they both seem to read many
> pages.  Perhaps the ECC overhead of reading all that data.
> I suppose I could turn off ECC and see how fast it is...

Sure, that would give an estimation.

> >>That is why I am thinking about using a non NAND aware file system for
> >>things that can be read-only.
> >
> >But this does not answer my question how you want to deal with bad blocks
> > ?
>
> I want a file system very much like cramfs, but that can have holes in
> it so that it works on NAND.
> When mounted it would start scanning blocks starting from the beginning
> of the NAND partition until it found a valid superblock (or what ever
> you would call it).  Then it would be done because all of the indexes
> would be built to work around the bad blocks.  Since this filesystem
> would be read-only with a static structure, you would never have to read
> more than necessary.

OK, so you are going to write a fs driver, which is NAND aware and behaves 
similar to cramfs. 
Whatfor then the discussion about making mtdblock nand aware ?
If you write a nand aware fs then you just call the appropriate functions. 
This fs will be specific for nand or selectable for nand, so what ?
No need to touch anything in mtdblock.c

-- 
Thomas
________________________________________________________________________
"Free software" is a matter of liberty, not price. To understand the concept,
you should think of "free" as in "free speech,'' not as in "free beer".
________________________________________________________________________
linutronix - competence in embedded & realtime linux
http://www.linutronix.de
mail: tglx@linutronix.de

Re: mtd, mtdblock and nand ecc.

[David Daney]

Thomas Gleixner wrote:

>Why did you modify nand.c ? 
>Almost everything in nand.c can be overridden by the board driver. Therefor we 
>call all the functions through chip->xxx().
>  
>
We need almost everything in nand.c, but the physical access to the NAND 
is through a hardware state machine that hides the raw chip registers 
that nand.c uses.  So we made a copy of nand.c and hacked it to work.

>>When ever jffs2 or yaffs are mounted, they both seem to read many
>>pages.  Perhaps the ECC overhead of reading all that data.
>>I suppose I could turn off ECC and see how fast it is...
>>    
>>
>
>Sure, that would give an estimation.
>
>  
>
>>>>That is why I am thinking about using a non NAND aware file system for
>>>>things that can be read-only.
>>>>        
>>>>
>>>But this does not answer my question how you want to deal with bad blocks
>>>?
>>>      
>>>
>>I want a file system very much like cramfs, but that can have holes in
>>it so that it works on NAND.
>>When mounted it would start scanning blocks starting from the beginning
>>of the NAND partition until it found a valid superblock (or what ever
>>you would call it).  Then it would be done because all of the indexes
>>would be built to work around the bad blocks.  Since this filesystem
>>would be read-only with a static structure, you would never have to read
>>more than necessary.
>>    
>>
>
>OK, so you are going to write a fs driver, which is NAND aware and behaves 
>similar to cramfs. 
>
I MAY do it, but would rather that someone else did all the hard work :)

>Whatfor then the discussion about making mtdblock nand aware ?
>If you write a nand aware fs then you just call the appropriate functions. 
>This fs will be specific for nand or selectable for nand, so what ?
>No need to touch anything in mtdblock.c
>  
>
I guess I am abandoning that for now...

David Daney.