Is it an atomic operation for writing a page in NAND flash

Is it an atomic operation for writing a page in NAND flash

[Liu Hui]

Hi guys,

This is a question confused me for a long time. As I know, writing a
sector for a hard disk is atomic. That is to say, when we are writing
a sector to hard disk and power failure happen, the sector will be
written completely or not at all.

For NAND flash, I didn't see the atomic guarantee in any material.
Could you please tell me if writing a page for NAND flash is atomic?
This is very important for a transaction based file system.


--
Thanks & Best Regards
Liu Hui
--

Re: Is it an atomic operation for writing a page in NAND flash

[Ricard Wanderlof]

On Wed, 20 Jan 2010, Liu Hui wrote:

> Hi guys,
>
> This is a question confused me for a long time. As I know, writing a
> sector for a hard disk is atomic. That is to say, when we are writing
> a sector to hard disk and power failure happen, the sector will be
> written completely or not at all.
>
> For NAND flash, I didn't see the atomic guarantee in any material.
> Could you please tell me if writing a page for NAND flash is atomic?
> This is very important for a transaction based file system.

It is my understanding that if you get a power failure while the nand 
flash chip is writing the page the page could get partly written.

The only way around something like this would be to monitor the power line 
prior to the supply regulator, and not start a write if it can be detected 
that a power failure has occurred and there is not enough power to 
complete the write.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Is it an atomic operation for writing a page in NAND flash

[Liu Hui]

Richard,

Thank you for your confirmation and good idea.

I also think about your idea before, that is, when power failure
happens, generate an interrupt and blocks any other write requests in
interrupt handler. But this is a little complex.

Now, I think I can use ECC to check the partial write, if a write was
not finished, the ECC should be wrong, so we can detect this partial
write and discard this write. Do you think this is a good idea?

Thanks,
Hui

2010/1/20 Ricard Wanderlof <ricard.wanderlof@axis.com>:
>
> On Wed, 20 Jan 2010, Liu Hui wrote:
>
>> Hi guys,
>>
>> This is a question confused me for a long time. As I know, writing a
>> sector for a hard disk is atomic. That is to say, when we are writing
>> a sector to hard disk and power failure happen, the sector will be
>> written completely or not at all.
>>
>> For NAND flash, I didn't see the atomic guarantee in any material.
>> Could you please tell me if writing a page for NAND flash is atomic?
>> This is very important for a transaction based file system.
>
> It is my understanding that if you get a power failure while the nand flash
> chip is writing the page the page could get partly written.
>
> The only way around something like this would be to monitor the power line
> prior to the supply regulator, and not start a write if it can be detected
> that a power failure has occurred and there is not enough power to complete
> the write.
>
> /Ricard
> --
> Ricard Wolf Wanderlöf                           ricardw(at)axis.com
> Axis Communications AB, Lund, Sweden            www.axis.com
> Phone +46 46 272 2016                           Fax +46 46 13 61 30
>



-- 
Thanks & Best Regards
Liu Hui
--

Re: Is it an atomic operation for writing a page in NAND flash

[Jamie Lokier]

Liu Hui wrote:
> Richard,
> 
> Thank you for your confirmation and good idea.
> 
> I also think about your idea before, that is, when power failure
> happens, generate an interrupt and blocks any other write requests in
> interrupt handler. But this is a little complex.

Ideally, you would design the hardware so that power failure can be
detected early near the power input, but with enough on-board power
retention (i.e. capacitor) that there is guaranteed enough continuous
power for the CPU to react and the NAND chip to have enough stable
power to complete the write reliably.

There is no need for an interrupt, if you have a fast GPIO that you
can read before each write command that tells if the input power has
not dropped.

> Now, I think I can use ECC to check the partial write, if a write was
> not finished, the ECC should be wrong, so we can detect this partial
> write and discard this write. Do you think this is a good idea?

It's good, but not perfect: In principle a power-failed write could
successfully store the correct bits including ECC so they read back
correctly, but with the cell charges not completely stable.  But I
guess that's rare enough that it is just included in the normal NAND
bad block possibilities.

-- Jamie

Re: Is it an atomic operation for writing a page in NAND flash

[Jamie Lokier]

Liu Hui wrote:
> This is a question confused me for a long time. As I know, writing a
> sector for a hard disk is atomic. That is to say, when we are writing
> a sector to hard disk and power failure happen, the sector will be
> written completely or not at all.

Are you sure about that?
I have never seen a reliable confirmation of it.

-- Jamie

Re: Is it an atomic operation for writing a page in NAND flash

[Artem Bityutskiy]

On Wed, 2010-01-20 at 13:41 +0000, Jamie Lokier wrote:
> Liu Hui wrote:
> > This is a question confused me for a long time. As I know, writing a
> > sector for a hard disk is atomic. That is to say, when we are writing
> > a sector to hard disk and power failure happen, the sector will be
> > written completely or not at all.
> 
> Are you sure about that?
> I have never seen a reliable confirmation of it.

Theo sent a longish e-mail some time ago to fs-devel or lkml, explaining
why this almost never true in practice.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Is it an atomic operation for writing a page in NAND flash

[Liu Hui]

http://kerneltrap.org/node/6741
In this article, it said:"Disks assure atomicity at the sector level.
This means that a write to a sector either goes through completely or
not at all."

I also know, some transaction based file system depend on the atomic
sector write feature of hard disk.

But I am not very confirmed about this...

2010/1/20 Jamie Lokier <jamie@shareable.org>:
> Liu Hui wrote:
>> This is a question confused me for a long time. As I know, writing a
>> sector for a hard disk is atomic. That is to say, when we are writing
>> a sector to hard disk and power failure happen, the sector will be
>> written completely or not at all.
>
> Are you sure about that?
> I have never seen a reliable confirmation of it.
>
> -- Jamie
>



-- 
Thanks & Best Regards
Liu Hui
--

Re: Is it an atomic operation for writing a page in NAND flash

[Liu Hui]

I didn't see this e-mail of Theo and I also can't find it in my
fs-devel mail list, could you please share it with us?

much appreciated!

2010/1/20 Artem Bityutskiy <dedekind1@gmail.com>:
> On Wed, 2010-01-20 at 13:41 +0000, Jamie Lokier wrote:
>> Liu Hui wrote:
>> > This is a question confused me for a long time. As I know, writing a
>> > sector for a hard disk is atomic. That is to say, when we are writing
>> > a sector to hard disk and power failure happen, the sector will be
>> > written completely or not at all.
>>
>> Are you sure about that?
>> I have never seen a reliable confirmation of it.
>
> Theo sent a longish e-mail some time ago to fs-devel or lkml, explaining
> why this almost never true in practice.
>
> --
> Best Regards,
> Artem Bityutskiy (Артём Битюцкий)
>
>



-- 
Thanks & Best Regards
Liu Hui
--

Re: Is it an atomic operation for writing a page in NAND flash

[Liu Hui]

Ok, In fact, I am a soft developer, I want to find a way to design a
power-cut safe FTL. I can't control the design of hardware, so I have
to find a common way to ensure atomic operation.

> There is no need for an interrupt, if you have a fast GPIO that you
> can read before each write command that tells if the input power has
> not dropped.
This is no good for performance.

> It's good, but not perfect: In principle a power-failed write could
> successfully store the correct bits including ECC so they read back
> correctly, but with the cell charges not completely stable.  But I
> guess that's rare enough that it is just included in the normal NAND
> bad block possibilities.
Ok, ECC can detect partial write but can't detect unstable cell
charges, I think this is enough since NAND flash is unstable media.

Thanks for your information!
Hui



2010/1/20 Jamie Lokier <jamie@shareable.org>:
> Liu Hui wrote:
>> Richard,
>>
>> Thank you for your confirmation and good idea.
>>
>> I also think about your idea before, that is, when power failure
>> happens, generate an interrupt and blocks any other write requests in
>> interrupt handler. But this is a little complex.
>
> Ideally, you would design the hardware so that power failure can be
> detected early near the power input, but with enough on-board power
> retention (i.e. capacitor) that there is guaranteed enough continuous
> power for the CPU to react and the NAND chip to have enough stable
> power to complete the write reliably.
>
> There is no need for an interrupt, if you have a fast GPIO that you
> can read before each write command that tells if the input power has
> not dropped.
>
>> Now, I think I can use ECC to check the partial write, if a write was
>> not finished, the ECC should be wrong, so we can detect this partial
>> write and discard this write. Do you think this is a good idea?
>
> It's good, but not perfect: In principle a power-failed write could
> successfully store the correct bits including ECC so they read back
> correctly, but with the cell charges not completely stable.  But I
> guess that's rare enough that it is just included in the normal NAND
> bad block possibilities.
>
> -- Jamie
>



-- 
Thanks & Best Regards
Liu Hui
--

Re: Is it an atomic operation for writing a page in NAND flash

[Artem Bityutskiy]

On Wed, 2010-01-20 at 22:06 +0800, Liu Hui wrote:
> I didn't see this e-mail of Theo and I also can't find it in my
> fs-devel mail list, could you please share it with us?
> 
> much appreciated!

Found it on lkml. Enjoy. IMO, very good writing.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Is it an atomic operation for writing a page in NAND flash

[Artem Bityutskiy]

On Wed, 2010-01-20 at 16:38 +0200, Artem Bityutskiy wrote:
> On Wed, 2010-01-20 at 22:06 +0800, Liu Hui wrote:
> > I didn't see this e-mail of Theo and I also can't find it in my
> > fs-devel mail list, could you please share it with us?
> > 
> > much appreciated!
> 
> Found it on lkml. Enjoy. IMO, very good writing.

Sorry:
http://lkml.org/lkml/2009/8/24/156

(forgot to paste the link)

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Re: Is it an atomic operation for writing a page in NAND flash

[Liu Hui]

Great, so kind of you.

Thanks,
Hui

2010/1/20 Artem Bityutskiy <dedekind1@gmail.com>:
> On Wed, 2010-01-20 at 16:38 +0200, Artem Bityutskiy wrote:
>> On Wed, 2010-01-20 at 22:06 +0800, Liu Hui wrote:
>> > I didn't see this e-mail of Theo and I also can't find it in my
>> > fs-devel mail list, could you please share it with us?
>> >
>> > much appreciated!
>>
>> Found it on lkml. Enjoy. IMO, very good writing.
>
> Sorry:
> http://lkml.org/lkml/2009/8/24/156
>
> (forgot to paste the link)
>
> --
> Best Regards,
> Artem Bityutskiy (Артём Битюцкий)
>
>



-- 
Thanks & Best Regards
Liu Hui
--

Re: Is it an atomic operation for writing a page in NAND flash

[Ricard Wanderlof]

On Wed, 20 Jan 2010, Liu Hui wrote:

>> It's good, but not perfect: In principle a power-failed write could
>> successfully store the correct bits including ECC so they read back
>> correctly, but with the cell charges not completely stable.  But I
>> guess that's rare enough that it is just included in the normal NAND
>> bad block possibilities.
> Ok, ECC can detect partial write but can't detect unstable cell
> charges, I think this is enough since NAND flash is unstable media.

ECC is designed to correct a small number of bits (1 bit for the software 
ECC algorithm used by mtd) and detect failure if a couple of more bits are 
bad (2 bits for the mtd algorithm). Beyond that, the results cannot be 
trusted. That means, that if there are, say, 16 incorrect bits in the 
data, the ECC algorithm will not necessarily indicate that there is a 
failure. It might very well indicate that there is a single bit that needs 
correction, or that all bits are correct. It is not a CRC.

The end result is that you can't say "if the ECC says it's ok, the data 
hasn't been corrupted" (which you could with a CRC). The only thing you 
can say (in the case of the mtd ECC algorithm) is "if there is a one-bit 
error in the data, the ECC will correct it" and "if there is a two-bit 
error in the data, the ECC will detect it".

If you really need that kind of check for data integrity, I suppose you 
could add a CRC algorithm to the ECC calculations already being performed 
by mtd, so that any change in the data would be flagged due to a 
mismatching CRC.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Is it an atomic operation for writing a page in NAND flash

[Liu Hui]

Thanks you very much, CRC is the real solution.

But I don't understand, if a partial write happens, we use ECC to
correct the data, we will find the data can't be corrected, then
-EBADMSG will be returned(see nand_correct_data()), then we can know
this page are corrupted. IMHO, this works.

Anyway, I will think about CRC seriously.

Thanks,
Hui

2010/1/20 Ricard Wanderlof <ricard.wanderlof@axis.com>:
>
> On Wed, 20 Jan 2010, Liu Hui wrote:
>
>>> It's good, but not perfect: In principle a power-failed write could
>>> successfully store the correct bits including ECC so they read back
>>> correctly, but with the cell charges not completely stable.  But I
>>> guess that's rare enough that it is just included in the normal NAND
>>> bad block possibilities.
>>
>> Ok, ECC can detect partial write but can't detect unstable cell
>> charges, I think this is enough since NAND flash is unstable media.
>
> ECC is designed to correct a small number of bits (1 bit for the software
> ECC algorithm used by mtd) and detect failure if a couple of more bits are
> bad (2 bits for the mtd algorithm). Beyond that, the results cannot be
> trusted. That means, that if there are, say, 16 incorrect bits in the data,
> the ECC algorithm will not necessarily indicate that there is a failure. It
> might very well indicate that there is a single bit that needs correction,
> or that all bits are correct. It is not a CRC.
>
> The end result is that you can't say "if the ECC says it's ok, the data
> hasn't been corrupted" (which you could with a CRC). The only thing you can
> say (in the case of the mtd ECC algorithm) is "if there is a one-bit error
> in the data, the ECC will correct it" and "if there is a two-bit error in
> the data, the ECC will detect it".
>
> If you really need that kind of check for data integrity, I suppose you
> could add a CRC algorithm to the ECC calculations already being performed by
> mtd, so that any change in the data would be flagged due to a mismatching
> CRC.
>
> /Ricard
> --
> Ricard Wolf Wanderlöf                           ricardw(at)axis.com
> Axis Communications AB, Lund, Sweden            www.axis.com
> Phone +46 46 272 2016                           Fax +46 46 13 61 30
>



-- 
Thanks & Best Regards
Liu Hui
--

Re: Is it an atomic operation for writing a page in NAND flash

[Ricard Wanderlof]

On Wed, 20 Jan 2010, Liu Hui wrote:

> Thanks you very much, CRC is the real solution.
>
> But I don't understand, if a partial write happens, we use ECC to
> correct the data, we will find the data can't be corrected, then
> -EBADMSG will be returned(see nand_correct_data()), then we can know
> this page are corrupted. IMHO, this works.

Assuming the ECC algorithm used by mtd, it only produces correct results 
in the case of 0, 1 or 2 bit errors in the data. For more bit errors than 
that, the result is undefined.

Let's assume a partial write occurs, which leads to 57 bit errors compared 
to what was originally supposed to be there. Since there are more than 2 
bit errors, the algorthm output is undefined; it may say that the data 
can't be corrected, or it may say that the data is ok, or it may say that 
the data can be corrected; it's impossible to tell. As far as I 
understand, it is not uncommon for ECC to say the data is correct when it 
fact it isn't.

A slightly trivial case:

Again assuming the ECC algorithm used by mtd, the ECC bytes for a chunk of 
data where all the bytes have the same value is 0xFFFFFF, regardless of 
the actual value. So, say you have a page full of 0xA3; the ECC is then 
0xFFFFFFF. Now, assuming a partial write causes bit 2 of all bit cells to 
not change from 1 to 0 when programming. The result is a page full of 
0xA7, in effect, 256 bit errors (assuming a page size of 256 bits, or at 
least, assuming an ECC calculation encompassing that many bytes). But the 
ECC will still be 0xFFFFFF, and the corresponding ECC calculation will say 
that the data is correct. That is, as I mentioned before, because the 
result of an ECC calculation on data with >2 bit errors is undefined.

Note that there are other ECC algorithms which can correct more error 
bits. For MLC flash it is recommended to use an algorithm which corrects 4 
bit errors rather than a single bit error in a block of data. Such 
algorithms require more ECC bits though.

One has a tendency to think of ECC as a checking algorithm. It is not. It 
corrects and detects bit errors under certain circumstances. Outside those 
circumstances it is worthless. For the case of the software algorithm used 
in mtd, it is worthless if there are more than 2 bit errors. A failed 
write could cause any number of bit errors, so it is worthless to check 
the result using the ECC algorithm. The normal failure mode of a nand 
flash chip is random single random bit errors with low probability. ECC 
handles this elegantly.


Elaborating on this slightly, a devil's advocate would considere ECC 
worthless as a correction algorithm for a flash chip. Assume one bit error 
occurs, which is corrected by ECC. Then another bit error occurs in the 
same page. ECC then detects a failure. Then another bit error occurrs. The 
ECC algorithm is now worthless, it may detect the error, it may say that 
the data is correct, or it may even try to correct it (erroneously).

The reason that all this works in practice is that the probability of a 
bit error occurring is so low that the probability of two bit errors 
occurring in the same page is very low, in some respect lower than other 
failure modes in the system, so that we don't have to worry about it. (For 
example: What about bit errors occurring in RAM chips from cosmic 
radiation? It is a real risk, but so small that most systems don't have to 
worry about it.)

It can be a real concern though, and that is why things like UBI provide 
so-called bit scrubbing: whenever it detects that ECC has done a bit 
correction in a block, it erases that block and rewrites the data [lots of 
details omitted here] so that the chance of two bit errors ever occurring 
in the same page will be very small indeed.

Especially in these days of larger and larger flash chips resulting from 
shrinking chip geometries this is problem that is getting worse and worse. 
It also tends to vary hugely among manufacturers.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Is it an atomic operation for writing a page in NAND flash

[David Parkinson]

At 14:54 20/01/2010, Ricard Wanderlof wrote:
 >...
 >The end result is that you can't say "if the ECC says it's ok, the data
 >hasn't been corrupted" (which you could with a CRC).
 >...

Apologies for nit-picking (and small digression), but a CRC is no 
guarantee either.  Whilst error correcting codes have additional 
information so that small errors can be corrected both CRCs and ECCs 
work in the same way in detecting likely errors in the communications 
channel.  (It's all maths and statistics....).

A side question here is have the check algorithms been matched to the 
characteristics of the MTDs?  For example a weakish radio signal is 
likely to have errors randomly distributed across the message.  With 
a magnetic disk drive the errors are likely be caused by a blemish on 
the surface and therefore will come in bursts.  Some algorithms will 
be better than others in the respective cases.

Regards

David

Re: Is it an atomic operation for writing a page in NAND flash

[Ricard Wanderlof]

On Wed, 20 Jan 2010, David Parkinson wrote:

> At 14:54 20/01/2010, Ricard Wanderlof wrote:
> >...
> >The end result is that you can't say "if the ECC says it's ok, the data
> >hasn't been corrupted" (which you could with a CRC).
> >...
>
> Apologies for nit-picking (and small digression), but a CRC is no
> guarantee either.  Whilst error correcting codes have additional
> information so that small errors can be corrected both CRCs and ECCs
> work in the same way in detecting likely errors in the communications
> channel.  (It's all maths and statistics....).

You are right of course. Indeed, any mapping of N bits to n bits (where N 
> n) must result in a number of bit patterns for N which map to identical 
bit patterns for n. Still, CRC's used for data checking are designed so 
that the different bit patterns for N that map to the same n n are 
reasonably different from each other, so that a CRC is unlikely to show a 
correct result if there has been a 'typical' failure on the channel. At 
least the ECC algorithm used for mtd has is not intended for that level of 
error detection; it is optimized for correcting single-bit errors.

> A side question here is have the check algorithms been matched to the
> characteristics of the MTDs?  For example a weakish radio signal is
> likely to have errors randomly distributed across the message.  With
> a magnetic disk drive the errors are likely be caused by a blemish on
> the surface and therefore will come in bursts.  Some algorithms will
> be better than others in the respective cases.

The algorithm used in mtd comes from Toshiba I think and was 
originally designed for an old 256 page flash of theirs. But I would think 
all 1-bit-error-correction ECC's are basically the same.

I don't know, but I think the basic premise is that bit errors are rare, 
and when they do occur, they will be single bit errors occurring in random 
places. Indeed, the algorithm used seems to be ideally suited to this 
case.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Is it an atomic operation for writing a page in NAND flash

[Charles Manning]

On Thursday 21 January 2010 05:35:09 Ricard Wanderlof wrote:
> On Wed, 20 Jan 2010, David Parkinson wrote:
> > At 14:54 20/01/2010, Ricard Wanderlof wrote:
> > >...
> > >The end result is that you can't say "if the ECC says it's ok, the data
> > >hasn't been corrupted" (which you could with a CRC).
> > >...
> >
> > Apologies for nit-picking (and small digression), but a CRC is no
> > guarantee either.  Whilst error correcting codes have additional
> > information so that small errors can be corrected both CRCs and ECCs
> > work in the same way in detecting likely errors in the communications
> > channel.  (It's all maths and statistics....).

While you might be technically and theoretically correct, in practical terms 
CRC is a copper-bottomed guarantee when compared with ECC.

A 32-bit CRC is very difficult to randomly spoof. An ECC is extremely easy to 
spoof with random errors. The difference is in the order of millions.

The best protection for getting good NAND writes/erases is to make sure you 
don't launch a programming (write/erase op) unless you know power is good. If 
your system has a "power OK" flag then check it before doing the write.

Don't wire the WP pin to hardware power fail flags either since a falling WP 
will abort the current write.


>
> You are right of course. Indeed, any mapping of N bits to n bits (where N
>
> > n) must result in a number of bit patterns for N which map to identical
>
> bit patterns for n. Still, CRC's used for data checking are designed so
> that the different bit patterns for N that map to the same n n are
> reasonably different from each other, so that a CRC is unlikely to show a
> correct result if there has been a 'typical' failure on the channel. At
> least the ECC algorithm used for mtd has is not intended for that level of
> error detection; it is optimized for correcting single-bit errors.
>
> > A side question here is have the check algorithms been matched to the
> > characteristics of the MTDs?  For example a weakish radio signal is
> > likely to have errors randomly distributed across the message.  With
> > a magnetic disk drive the errors are likely be caused by a blemish on
> > the surface and therefore will come in bursts.  Some algorithms will
> > be better than others in the respective cases.

Actually radio errors are often bursty due to interference. The electric fence 
clicks I get here knocks out a few adjacent bits.

>
> The algorithm used in mtd comes from Toshiba I think and was
> originally designed for an old 256 page flash of theirs. But I would think
> all 1-bit-error-correction ECC's are basically the same.

This still treats the pages as blocks of 256 bytes so if you have a 512-byte 
page it will be treated as two 256-byte ECC regions.
>
> I don't know, but I think the basic premise is that bit errors are rare,
> and when they do occur, they will be single bit errors occurring in random
> places. Indeed, the algorithm used seems to be ideally suited to this
> case.

It depends...

For MLC flash you can expect quite a few errors which is why there has been a 
shift to multi-bit ECC for these.  If you have, say, 4-bit ECC then you might 
choose to treat 2 or less errors as "no error".

There are basically two types of multi-bit ECC: RS and BCH. RS is more suited 
to "burst" errors like you'll see on CD or radio. BCH is more suited to 
random errors.

>
> /Ricard

Re: Is it an atomic operation for writing a page in NAND flash

[Liu Hui]

Very informative, it expand my view. Thanks!

2010/1/21 Ricard Wanderlof <ricard.wanderlof@axis.com>:
>
> On Wed, 20 Jan 2010, Liu Hui wrote:
>
>> Thanks you very much, CRC is the real solution.
>>
>> But I don't understand, if a partial write happens, we use ECC to
>> correct the data, we will find the data can't be corrected, then
>> -EBADMSG will be returned(see nand_correct_data()), then we can know
>> this page are corrupted. IMHO, this works.
>
> Assuming the ECC algorithm used by mtd, it only produces correct results in
> the case of 0, 1 or 2 bit errors in the data. For more bit errors than that,
> the result is undefined.
>
> Let's assume a partial write occurs, which leads to 57 bit errors compared
> to what was originally supposed to be there. Since there are more than 2 bit
> errors, the algorthm output is undefined; it may say that the data can't be
> corrected, or it may say that the data is ok, or it may say that the data
> can be corrected; it's impossible to tell. As far as I understand, it is not
> uncommon for ECC to say the data is correct when it fact it isn't.
>
> A slightly trivial case:
>
> Again assuming the ECC algorithm used by mtd, the ECC bytes for a chunk of
> data where all the bytes have the same value is 0xFFFFFF, regardless of the
> actual value. So, say you have a page full of 0xA3; the ECC is then
> 0xFFFFFFF. Now, assuming a partial write causes bit 2 of all bit cells to
> not change from 1 to 0 when programming. The result is a page full of 0xA7,
> in effect, 256 bit errors (assuming a page size of 256 bits, or at least,
> assuming an ECC calculation encompassing that many bytes). But the ECC will
> still be 0xFFFFFF, and the corresponding ECC calculation will say that the
> data is correct. That is, as I mentioned before, because the result of an
> ECC calculation on data with >2 bit errors is undefined.
>
> Note that there are other ECC algorithms which can correct more error bits.
> For MLC flash it is recommended to use an algorithm which corrects 4 bit
> errors rather than a single bit error in a block of data. Such algorithms
> require more ECC bits though.
>
> One has a tendency to think of ECC as a checking algorithm. It is not. It
> corrects and detects bit errors under certain circumstances. Outside those
> circumstances it is worthless. For the case of the software algorithm used
> in mtd, it is worthless if there are more than 2 bit errors. A failed write
> could cause any number of bit errors, so it is worthless to check the result
> using the ECC algorithm. The normal failure mode of a nand flash chip is
> random single random bit errors with low probability. ECC handles this
> elegantly.
>
>
> Elaborating on this slightly, a devil's advocate would considere ECC
> worthless as a correction algorithm for a flash chip. Assume one bit error
> occurs, which is corrected by ECC. Then another bit error occurs in the same
> page. ECC then detects a failure. Then another bit error occurrs. The ECC
> algorithm is now worthless, it may detect the error, it may say that the
> data is correct, or it may even try to correct it (erroneously).
>
> The reason that all this works in practice is that the probability of a bit
> error occurring is so low that the probability of two bit errors occurring
> in the same page is very low, in some respect lower than other failure modes
> in the system, so that we don't have to worry about it. (For example: What
> about bit errors occurring in RAM chips from cosmic radiation? It is a real
> risk, but so small that most systems don't have to worry about it.)
>
> It can be a real concern though, and that is why things like UBI provide
> so-called bit scrubbing: whenever it detects that ECC has done a bit
> correction in a block, it erases that block and rewrites the data [lots of
> details omitted here] so that the chance of two bit errors ever occurring in
> the same page will be very small indeed.
>
> Especially in these days of larger and larger flash chips resulting from
> shrinking chip geometries this is problem that is getting worse and worse.
> It also tends to vary hugely among manufacturers.
>
> /Ricard
> --
> Ricard Wolf Wanderlöf                           ricardw(at)axis.com
> Axis Communications AB, Lund, Sweden            www.axis.com
> Phone +46 46 272 2016                           Fax +46 46 13 61 30
>



-- 
Thanks & Best Regards
Liu Hui
--