Compact Flash Question

Compact Flash Question

[Yigal Sadgat]

We wrote the Kernel to support CF drives (in IDE mode) several years ago but
now there are strange things coming back from the field that compel us to go
back and re-evaluate many design decisions.

For instance,
(1) Can you really ignore bit(2) (CORR) in the Status register offset 7 that 
tells
you that the CF has detected and corrected a soft error?, etc.
(2) An engineer at SanDisk Engineering told me NOT to do wear leveling.
The file allocation table is written very frequently back into the flash. So
is it really safe to assume that I don't need wear leveling???
(3) Re. the BUSY bit in the status register (offset 7, bit D7), anybody 
experienced
time outs?
(4) Re Error register (offset 1) bit D7 (BBK), again, I was told that it 
cannot (???)
happen since the CF performs read-after-write and it automatically switches 
good blocks
for bad ones... Is this correct?

We get many "soft" errors and I cannot wait to put my hand on a system to 
test but
thought I would draw on your collective experience before we dive into these 
issues again.

Any comments would be greatly appreciated.

Yigal Sadgat
General Computer Technology (GCT)
YSadgat1@gcte.com

Re: Compact Flash Question

[Alan Cox]

> (1) Can you really ignore bit(2) (CORR) in the Status register offset 7 that 
> tells
> you that the CF has detected and corrected a soft error?, etc.

I guess it might be interesting to log the error rate but we don't
currently do that. A corrected error is just that however - corrected by
the card itself.

> (2) An engineer at SanDisk Engineering told me NOT to do wear leveling.
> The file allocation table is written very frequently back into the flash. So
> is it really safe to assume that I don't need wear leveling???

Depends on your hardware vendor. Wear management is done within the CF
card and only the hardware vendor can tell you what they do. 

> (3) Re. the BUSY bit in the status register (offset 7, bit D7), anybody 
> experienced
> time outs?

Yes - both from failing CF cards and also other random events (bad
connections, people removing live cards etc)

> (4) Re Error register (offset 1) bit D7 (BBK), again, I was told that it 
> cannot (???)
> happen since the CF performs read-after-write and it automatically switches 
> good blocks
> for bad ones... Is this correct?

Depends on your hardware vendor. It certainly *can* occur with some CF
cards perhaps when they run out of spare blocks.

Alan

Re: Compact Flash Question

[Yigal Sadgat]

First, thank you to all that responded.

My original thought was that perhaps we're experiencing "soft errors". We 
shipped several thousands of systems and a very small number of them (3 to 4 
units) are showing this problem. I'm still waiting for these systems to come 
back to us...

Meanwhile I found out that although we (Engineering) specified "SanDisk 
ONLY" (only because they do wear leveling), our purchasing bought from other 
manufacturers as well (Toshiba, Crucial, Kingstone). Crucial is absolutely 
the LEAST reliable CF (system down!) and furthermore, their "lifetime 
warranty" is a joke.

Regards,
Yigal Sadgat

Re: Compact Flash Question

[linux-os (Dick Johnson)]

I was told by SanDisk when CF first came out to ignore
what the "lynix guys" (sic) were saying when I was
excoriated on this list for saying that everything
one needs to do is already incorporated into the
Compact Flash and it needs only to be treated as a
disk, nothing else!

Compact Flash is not just some flash RAM with an
IDE interface plug. It contains both static RAM
and flash RAM. It goes by pages and a stale page
is flushed to flash when a new page is being
accessed, or when other (propriatary) events are
true. The result is that most of the accesses
go to static RAM.

We have devices in the field with Linux embedded
in them for about 4 years that continuously write
to Compact Flash because the CF is mounted as
the root file-system and there is both log activity
and NTPD activity. The stuff still works and, in fact,
we have lost processors, power supplies, and Ethernet
controllers, but never CompactFlash.

There was some version of Linux that blacklisted the
CompactFlash that we were using. It thought it was
some hard disk that it didn't like. I removed the
black-listing and it worked fine. That was in a
device that had a RAM disk mounted on /tmp and
/var to minimize "wear." Eventually, we got rid of that.

On Tue, 6 May 2008, Yigal Sadgat wrote:

> We wrote the Kernel to support CF drives (in IDE mode) several years ago but
> now there are strange things coming back from the field that compel us to go
> back and re-evaluate many design decisions.
>
> For instance,
> (1) Can you really ignore bit(2) (CORR) in the Status register offset 7 that
> tells
> you that the CF has detected and corrected a soft error?, etc.
> (2) An engineer at SanDisk Engineering told me NOT to do wear leveling.
> The file allocation table is written very frequently back into the flash. So
> is it really safe to assume that I don't need wear leveling???
> (3) Re. the BUSY bit in the status register (offset 7, bit D7), anybody
> experienced
> time outs?
> (4) Re Error register (offset 1) bit D7 (BBK), again, I was told that it
> cannot (???)
> happen since the CF performs read-after-write and it automatically switches
> good blocks
> for bad ones... Is this correct?
>
> We get many "soft" errors and I cannot wait to put my hand on a system to
> test but
> thought I would draw on your collective experience before we dive into these
> issues again.
>
> Any comments would be greatly appreciated.
>
> Yigal Sadgat
> General Computer Technology (GCT)
> YSadgat1@gcte.com
>

Cheers,
Dick Johnson
Penguin : Linux version 2.6.22.1 on an i686 machine (5588.29 BogoMips).
My book : http://www.AbominableFirebug.com/
_


****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Compact Flash Question

[Bart Van Assche]

On Tue, May 6, 2008 at 11:59 PM, Yigal Sadgat <YSadgat1@gcte.com> wrote:
> (2) An engineer at SanDisk Engineering told me NOT to do wear leveling.
> The file allocation table is written very frequently back into the flash. So
> is it really safe to assume that I don't need wear leveling???

For most Linux filesystems you really need wear leveling. E.g. ext3's
superblock is at a fixed location and gets overwritten frequently.
Without wear leveling you risk that the flash sector where the
superblock resides wears out early.

When using ext3 on a CompactFlash, you can limit the number of writes
to the CompactFlash significantly by mounting the medium with
parameters like noatime,nodiratime,commit=300.

Bart.

Re: Compact Flash Question

[Alan Cox]

> For most Linux filesystems you really need wear leveling. E.g. ext3's
> superblock is at a fixed location and gets overwritten frequently.
> Without wear leveling you risk that the flash sector where the
> superblock resides wears out early.

CF sector mappings are not simple 1:1 mappings with flash blocks so this
is not the case. It is true with raw flash but not with CF. What CF
requires is vendor dependent but most vendors are pretty sensible.

The noatime advice is however good.

Re: Compact Flash Question

[Tomasz Chmielewski]

Bart Van Assche wrote:

> For most Linux filesystems you really need wear leveling. E.g. ext3's
> superblock is at a fixed location and gets overwritten frequently.
> Without wear leveling you risk that the flash sector where the
> superblock resides wears out early.

Compact Flash (and other similar media) does wear levelling, so essentially,
even if we write to the same fixed location, in reality, it will mostly go to a
different area on flash each time.

As Compact Flash and its wear levelling does not know about free space on the
filesystem, the wear levelling's effectiveness can be only limited - writes
won't spread on the whole free area of the flash.

Does anyone know how wear levelling is done in these devices? Perhaps it will
differ from a manufacturer to manufacturer, but I guess they have a free area we
normally use to store data, and some reserved area used just for wear levelling
and bad block handling, but that's just my guess.


> When using ext3 on a CompactFlash, you can limit the number of writes
> to the CompactFlash significantly by mounting the medium with
> parameters like noatime,nodiratime,commit=300.

noatime implies nodiratime, so there is no need to add the latter.


-- 
Tomasz Chmielewski
http://wpkg.org

Re: Compact Flash Question

[Tomasz Chmielewski]

Bart Van Assche schrieb:
> On Wed, May 7, 2008 at 9:12 AM, Tomasz Chmielewski <mangoo@wpkg.org> wrote:
>> As Compact Flash and its wear levelling does not know about free space on
>> the filesystem, the wear levelling's effectiveness can be only limited -
>> writes won't spread on the whole free area of the flash.
>>
>> Does anyone know how wear levelling is done in these devices? Perhaps it
>> will differ from a manufacturer to manufacturer, but I guess they have a
>> free area we normally use to store data, and some reserved area used just
>> for wear levelling and bad block handling, but that's just my guess.
> 
> The wear leveling algorithm depends on the CompactFlash manufacturer.
> Most manufacturers define a certain number of areas on a CompactFlash.
> Each area consists of two or more sectors which are counted in the
> total size of the CompactFlash, and one or more spare sectors. Wear
> leveling is carried out within an area by spreading writes over the
> sectors in an area. The controller in the CompactFlash stores the data
> about which sectors are used for storing which data persistently. I
> know of only one manufacturer who makes the CompactFlash controllers
> carry out wear leveling over the whole CompactFlash. 

How does it work, then?
How can it do wear levelling over the whole CF if some (or most) area of CF is 
already used by our precious data/metadata?
It would have to know the areas where no data is stored, but it contradicts the 
CF <-> filesystem separation.


-- 
Tomasz Chmielewski
http://wpkg.org

Re: Compact Flash Question

[Bart Van Assche]

On Wed, May 7, 2008 at 9:54 AM, Tomasz Chmielewski <mangoo@wpkg.org> wrote:
> How does it work, then?
> How can it do wear levelling over the whole CF if some (or most) area of CF
> is already used by our precious data/metadata?
> It would have to know the areas where no data is stored, but it contradicts
> the CF <-> filesystem separation.

The wear leveling algorithm even works if the CompactFlash controller
has no knowledge of the sectors on which filesystem data is stored.
All that is needed is some spare space on the CompactFlash that is not
made visible through the IDE interface, a controller that is able to
remap sectors and that stores this mapping persistently.

Bart.

Re: Compact Flash Question

[Helge Hafting]

Tomasz Chmielewski wrote:
>
> How does it work, then?
> How can it do wear levelling over the whole CF if some (or most) area 
> of CF is already used by our precious data/metadata?
> It would have to know the areas where no data is stored, but it 
> contradicts the CF <-> filesystem separation.
It don't necessarily need to know. It can swap two used blocks, one 
often-used and one
rarely-used.  That way the rarely-used block is rewriten over the 
previously busy
block, and the busy block is moved to the rarely used area that isn't worn.
This implies an extra write whenever a busy block is moved.  Don't know if
anybody do this, but the technique is simple enough.

CF-filesystem separation is necessary, for they can't know in advance what
filesystem or partitioning scheme will be used.  (I have ext3 on CF, for 
example...)

Helge Hafting

Re: Compact Flash Question

[Stéphane ANCELOT]

experience report :
In June 2004 we migrated to CF after bad hard disk crashes experience...

Most of the CF do NOT do wear leveling, you have to ask the manufacturer
if it does it or not, generally it is called industrial... and normally,
cost much more .
So, for low cost cf I assume you have to do wear leveling with the
kernel ...otherwise it is already "wear leveling inside"
Since the technology evolves very fast it may be possible today that the
production's cost of wear leveled CF will be the same as standard one
and wear leveling will be a standard ?


I think it should be possible to know if the Cf supports wear leveling
using the the identify device command (0xEC) and look at words 82 to 87 
(cf. CF specifications http://www.compactflash.org/specdl1.htm)
Depending on the application, some system hacks like noatime... need to
be setted up.

we use  silicon systems CF  with wear leveling inside  , we write a few
kb data back each 1/4 h  on it.(log /tmp files  are redirected to ram)

Since June 2004 ALL  hard disk  systems based replaced with CF (env. 100
units)   have not failed.

If somebody could report experience with other brand CF including wear
leveling , it will be fine
some other brand I know only the name :
www.apro-tw.com
http://www.afaya.com.tw/ (spec tells it supports wear leveling)


Best Regards
S.Ancelot

Helge Hafting a écrit :
> Tomasz Chmielewski wrote:
>>
>> How does it work, then?
>> How can it do wear levelling over the whole CF if some (or most) area
>> of CF is already used by our precious data/metadata?
>> It would have to know the areas where no data is stored, but it
>> contradicts the CF <-> filesystem separation.
> It don't necessarily need to know. It can swap two used blocks, one
> often-used and one
> rarely-used.  That way the rarely-used block is rewriten over the
> previously busy
> block, and the busy block is moved to the rarely used area that isn't
> worn.
> This implies an extra write whenever a busy block is moved.  Don't
> know if
> anybody do this, but the technique is simple enough.
>
> CF-filesystem separation is necessary, for they can't know in advance
> what
> filesystem or partitioning scheme will be used.  (I have ext3 on CF,
> for example...)
>
> Helge Hafting
>
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe
> linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
>

Re: Compact Flash Question

[Bart Van Assche]

On Wed, May 7, 2008 at 5:01 PM, Stéphane ANCELOT <sancelot@free.fr> wrote:
>
> we use  silicon systems CF  with wear leveling inside  , we write a few
> kb data back each 1/4 h  on it.(log /tmp files  are redirected to ram)

Silicon Systems CompactFlashes are the among the most reliable
CompactFlashes I have used in embedded devices. See also
http://www.siliconsystems.com/silicondrive/whitepapers/SSWP03-Endurance-R.pdf
for a whitepaper that explains their wear leveling and error
correction algorithms. Furthermore, Silicon Systems has a technology
called SiSMART that allows to monitor by how far the CompactFlash is
worn out, such that it can be monitored whether or not it is time to
replace the CompactFlash. (Note: I am not affiliated in any way to
Silicon Systems.)

Bart.

Re: Compact Flash Question

[Mark Lord]

Bart Van Assche wrote:
> On Wed, May 7, 2008 at 5:01 PM, Stéphane ANCELOT <sancelot@free.fr> wrote:
>> we use  silicon systems CF  with wear leveling inside  , we write a few
>> kb data back each 1/4 h  on it.(log /tmp files  are redirected to ram)
> 
> Silicon Systems CompactFlashes are the among the most reliable
> CompactFlashes I have used in embedded devices. See also
> http://www.siliconsystems.com/silicondrive/whitepapers/SSWP03-Endurance-R.pdf
> for a whitepaper that explains their wear leveling and error
> correction algorithms. Furthermore, Silicon Systems has a technology
> called SiSMART that allows to monitor by how far the CompactFlash is
> worn out, such that it can be monitored whether or not it is time to
> replace the CompactFlash. (Note: I am not affiliated in any way to
> Silicon Systems.)
..

Mmm.. according to that document, the "dynamic wear leveling" devices
might benefit from having Linux issue CF-Erase commands to unallocated sectors,
so that the onboard CF controller can then add those sectors to the wear-leveling
rotation scheme.

There was discussion at LSF'08 about possible filesystem hooks that could
be used for this (and other) purposes when deleting files.

Cheers

Re: Compact Flash Question

[Lennart Sorensen]

On Wed, May 07, 2008 at 06:46:25PM +0200, Bart Van Assche wrote:
> Silicon Systems CompactFlashes are the among the most reliable
> CompactFlashes I have used in embedded devices. See also
> http://www.siliconsystems.com/silicondrive/whitepapers/SSWP03-Endurance-R.pdf
> for a whitepaper that explains their wear leveling and error
> correction algorithms. Furthermore, Silicon Systems has a technology
> called SiSMART that allows to monitor by how far the CompactFlash is
> worn out, such that it can be monitored whether or not it is time to
> replace the CompactFlash. (Note: I am not affiliated in any way to
> Silicon Systems.)

We too have switched to Silicon Systems and are very happy with them.
And even industrial temperature versions are hardly expensive.  They may
cost more than you pay for a generic slow piece of junk at a retail
store, but you are getting a better card.

-- 
Len Sorensen

Re: Compact Flash Question

[Willy Tarreau]

On Thu, May 08, 2008 at 10:27:14AM -0400, Lennart Sorensen wrote:
> On Wed, May 07, 2008 at 06:46:25PM +0200, Bart Van Assche wrote:
> > Silicon Systems CompactFlashes are the among the most reliable
> > CompactFlashes I have used in embedded devices. See also
> > http://www.siliconsystems.com/silicondrive/whitepapers/SSWP03-Endurance-R.pdf
> > for a whitepaper that explains their wear leveling and error
> > correction algorithms. Furthermore, Silicon Systems has a technology
> > called SiSMART that allows to monitor by how far the CompactFlash is
> > worn out, such that it can be monitored whether or not it is time to
> > replace the CompactFlash. (Note: I am not affiliated in any way to
> > Silicon Systems.)
> 
> We too have switched to Silicon Systems and are very happy with them.
> And even industrial temperature versions are hardly expensive.  They may
> cost more than you pay for a generic slow piece of junk at a retail
> store, but you are getting a better card.

Interestingly, we once got a bunch of those and had a lot of problems
with them. Typically unbootable systems. We finally noticed that there
was a real corruption problem, because 3 MD5s of the kernel in a row
returned 3 different values. Our provider assured us that he never heard
about that, and insisted that we try again on a new batch. We have had
no trouble since. I guess we really got a bad batch. At least it was
noticeable early in the deployment process (the worst ones being a few
days after installation). However, we never had any problem with PQI
industrial.

> Len Sorensen

Willy