RE: How to protect DoC 2000 from power fail?

RE: How to protect DoC 2000 from power fail?

[Cfowler]

I do not use a journalling file system.  I load a ext2 fs into memory from a
raw file on DOC2000.  If you loose power then you just reboot.  No loss of
data.  Of course only config changes not the root fs.  I store config in a
shared memory sgement.  I only write it to DOC when the user issues the
'save' command.


--------- Original message --------
From: "Wil Taphoorn" <wil@wtms.nl>
To: "linux-mtd@lists.infradead.org" <linux-mtd@lists.infradead.org>
Subject: How to protect DoC 2000 from power fail?
Date: 03-18-02 12:23

Gentlemen,

I am looking for some rigid means of power fail protection
for a DoC 2000 based embedded system. From what I have read
so far I understand that almost any brand of journalling file
system should do well but, then again, those readings also
mention raw FLASH and not DoC, in other words, I think I am
lost. Would someone be so kind to enlighten me?

The system in question runs on a customized 486 SoC, boots
from EEPROM and loads LILO from DoC. The kernel uses initrd
to load the M-sys supplied DoC driver.

Thanks,
Wil

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: How to protect DoC 2000 from power fail?

[Wil Taphoorn]

Cfowler wrote:
 
> Wil Taphoorn wrote:
> > I am looking for some rigid means of power fail protection
> > for a DoC 2000 based embedded system. [..snip..]
> 
> I do not use a journalling file system.  I load a ext2 fs into memory from a
> raw file on DOC2000.  If you loose power then you just reboot.  No loss of
> data.  Of course only config changes not the root fs.  I store config in a
> shared memory sgement.  I only write it to DOC when the user issues the
> 'save' command.

Thanks for you input, Chris, but I can not use that scenario. The system has to
save not only run-time information but also system update files received from
the network like data, drivers and even the kernel itself.

I think I first have to read (and re-read) David's contribution on the subject.

Thanks,
Wil

How to protect DoC 2000 from power fail?

[Wil Taphoorn]

Gentlemen,

I am looking for some rigid means of power fail protection
for a DoC 2000 based embedded system. From what I have read
so far I understand that almost any brand of journalling file
system should do well but, then again, those readings also
mention raw FLASH and not DoC, in other words, I think I am
lost. Would someone be so kind to enlighten me?

The system in question runs on a customized 486 SoC, boots
from EEPROM and loads LILO from DoC. The kernel uses initrd
to load the M-sys supplied DoC driver.

Thanks,
Wil

Re: How to protect DoC 2000 from power fail?

[David Woodhouse]

wil@wtms.nl said:
>  I am looking for some rigid means of power fail protection for a DoC
> 2000 based embedded system. From what I have read so far I understand
> that almost any brand of journalling file system should do well but,
> then again, those readings also mention raw FLASH and not DoC, in
> other words, I think I am lost. Would someone be so kind to enlighten
> me? 

The M-Systems driver, and also the FTL and NFTL code in the Linux kernel, 
use a kind of pseudo-filesystem on the flash to emulate a normal block 
device. That pseudo-filesystem (or 'translation layer') is expected to 
be resilient to power failure, and generally expected to do wear levelling. 
It's essentially a journalling file system all of its own.

CompactFlash also uses such a pseudo-filesystem, but does it completely
internally. Electrically, it behaves just like an IDE drive. Reportedly CF 
devices are very bad w.r.t to power failure and wear levelling though, but 
the concept is similar. 

So now you have what appears to be a normal block device. You can use VFAT 
or ext2 on that, but obviously those are not resilient to power failure - 
the underlying translation layer will be fine, but if the file system on 
top will get corrupted, you're still screwed. Using VFAT or ext2 on CF or 
FTL/NFTL in most applications is fairly insane, if you're writing to it 
during normal operation.

So you probably want to use a real journalling file system on your pretend 
block device instead. But that has problems too...

Often, such journalling file systems work by keeping a part of the block
device for a 'log' or 'journal', then doing changes as follows:

	1. Write the change you're about to make to the log.
	2. Make the actual changes in the file system.
	3. Write a 'commit block' to the log.

If power is lost during step 1, the change we were about to make is lost - 
it's as if it never happened, and note that we didn't touch the traditional 
part of the file system.

If power is lost during step 2, when we reboot we'll 'play back' the log 
and go ahead and finish what we were doing.

If power is lost during step 3, we'll make the same changes to the file 
system again, but that won't matter.

So whenever we lose power, all the data are safe.

However, it should be immediately obvious that this form of journalling 
involves a lot more writes to the block device than a normal file system. 
On a real hard drive, that's fine. But on flash, that's precisely the kind 
of thing you want to avoid. The underlying translation layer will (normally)
try to distribute that wear around the real physical flash, but there will 
still be more writes than are necessary, and hence it will destroy the 
hardware faster and generally operate slower than necessary.

To be honest, I cannot recommend _any_ block-based journalling file system
for use on the DiskOnChip or similar devices. I would suggest that for most
situations where you actually want to write to the file system, using _any_
blkdev-based file system on top of such an emulated block device is insane.

The only justification for pretending to be a block device is that DOS
drivers were far easier to do that way; you could just hook into the INT 13h
handler and pretend to be a BIOS-supported hard drive. Porting to Windows 
was easier that way too. 

If you want to be able to write to it then the correct solution, IMO, is to
write a _real_ file system which operates on flash directly instead of
operating on a block device. Build in the wear levelling, journalling, etc. 

For many years I made this statement in the hope that someone would 
eventually get round to writing such a flash file system. Then eventually, 
Axis did it - see http://developer.axis.com/software/jffs/. I worked on 
this for a while before succumbing to Fred Brooks' wisdom and starting 
again from scratch.

The result is JFFS2 - http://sources.redhat.com/jffs2/. Originally it only 
worked on NOR flash and not the cheaper NAND flash which is in the 
DiskOnChip. But recently we've added basic support for NAND devices and it 
ought to be approaching beta quality on NAND. More testing would be useful. 

--
dwmw2

Re: How to protect DoC 2000 from power fail?

[Charles Manning]

On Tue, 19 Mar 2002 01:10, David Woodhouse wrote:
> wil@wtms.nl said:
> >  I am looking for some rigid means of power fail protection for a DoC
> > 2000 based embedded system. From what I have read so far I understand
> > that almost any brand of journalling file system should do well but,
> > then again, those readings also mention raw FLASH and not DoC, in
> > other words, I think I am lost. Would someone be so kind to enlighten
> > me?
>[snip]
> CompactFlash also uses such a pseudo-filesystem, but does it completely
> internally. Electrically, it behaves just like an IDE drive. Reportedly CF
> devices are very bad w.r.t to power failure and wear levelling though, but
> the concept is similar.


CF should have at least 15ms of good power after the last write. This is 
required to complete the write internally.

Dunno what goes on in a DOC, but basically it needs some power to complete 
the last write. Dunno how clever it is at fixing itself. Maybe read the 
msystems www. They have some good application notes - maybe one covers these 
issues.

Hard disks get around the power issue by using the inertia of the spinning 
disk to generate sufficient power (well that's what I believe anyway). No 
such luxury with flash. You either need a resiliant file system - preferably 
one designed specifically for flash - or some way to guarantee power to 
complete a shutdown.

I've done both in various situations.

One way to guarantee good power into the FS is to do something like follows:
* Have the hardware report an early warning of imminent power failure. eg. if 
you have a two-stage switchmode, detecting the regulation loss flag on the 
first stage might do the job.
* Protect all critical FS entry points with power checks:
   write_func()
   {
      while(power is bad) { /* spin */}
      rest of function
   }
*  If you've done this right, you never go into a critical section with bad 
power. If power turns bad during the run of the function then hopefully the 
residual energy in the rail etc will be enough to power the system through 
the rest of the function call.

This can work well if your hardware supports this kind of detection and has 
sufficient residual energy. Dunno how long you need for a DOC, but I would 
hunch that 100ms would be enough.


> If you want to be able to write to it then the correct solution, IMO, is to
> write a _real_ file system which operates on flash directly instead of
> operating on a block device. Build in the wear levelling, journalling, etc.

Too right!

-- Charles

Re: How to protect DoC 2000 from power fail?

[David Woodhouse]

On Tue, 19 Mar 2002, Charles Manning wrote:

> CF should have at least 15ms of good power after the last write. This is 
> required to complete the write internally.
> 
> Dunno what goes on in a DOC, but basically it needs some power to complete 
> the last write. Dunno how clever it is at fixing itself. Maybe read the 
> msystems www. They have some good application notes - maybe one covers these 
> issues.

DiskOnChip is just a bunch of NAND flash chips with an ASIC to do the ECC 
for you. If you lose power during a block write, the ECC on that write 
will fail or you won't yet have written the metadata, and you get to use 
the previous version of that sector when you reboot. You don't _need_ to 
guarantee that you'll have enough power, just that you won't screw up if 
you don't.

> *  If you've done this right, you never go into a critical section with bad 
> power. If power turns bad during the run of the function then hopefully the 
> residual energy in the rail etc will be enough to power the system through 
> the rest of the function call.
> 
> This can work well if your hardware supports this kind of detection and has 
> sufficient residual energy. Dunno how long you need for a DOC, but I would 
> hunch that 100ms would be enough.

But doing it at the hardware level isn't sufficient. The problem is with
the (fat/ext2/etc.) file system, which would need to have the same checks
for power and manage to do their own transactions in the same timescales,
including all the garbage-collection that the underlying translation layer
needs to do, etc.

> > If you want to be able to write to it then the correct solution, IMO, is to
> > write a _real_ file system which operates on flash directly instead of
> > operating on a block device. Build in the wear levelling, journalling, etc.
> 
> Too right!

:)

-- 
dwmw2