open question on flash speed/app blocking

open question on flash speed/app blocking

[Cam Mayor]

Hi all,

We're making an app that periodically writes a persistance file to flash.  I 
haven't actually gotten a flash file system area working yet on my board, so 
i can't test this yet for speed.   I know that the results will vary with the 
hardware, the filesystem used, and a handful of other factors.  I'm assuming 
that a write to the flash will be blocking - that is, nothing else will be 
allowed to happen on the bus while that function is being performed.

For a file the size of 32bytes, 1kByte, and 32kBtyes, what kind of blocking 
delay might one expect from linux writing to flash for each of those file 
sizes?  What would be an optimum flash filesystem to use for something like 
this?  (if there is one)

cheers,
cam

ps. i'm using linux 2.4.6-rmk1-rayl1 and 2.4.16-rmk2.  I could use the latest 
kernel, too, i just haven't gotten around to it.  For development purposes, 
i'm using a cirrus CDB89712 development board, which has the cs89712 
processor and some Intel 28f320B3 flash on it.

Re: open question on flash speed/app blocking

[Christopher Fowler]

I've noticed that when I write to a file on my DOC2000 that they bus 
does not block until the bufffers are flushd.  My flashing program 
issues a sync() before unmounting the flash.  I even write a 21kb 
config to a raw partition.  That can take a while too.    I'm not sure 
if there is a way around this.  On the DOC2000 you need to use at least 
a block size of 4096.  Since the M-SYS driver reads and writes in those 
block sizes.  Maybe some other people here can talk about this subject.

Chris

> Hi all,
> 
> We're making an app that periodically writes a persistance file to 
flash.  I 
> haven't actually gotten a flash file system area working yet on my 
board, so 
> i can't test this yet for speed.   I know that the results will vary 
with the 
> hardware, the filesystem used, and a handful of other factors.  I'm 
assuming 
> that a write to the flash will be blocking - that is, nothing else 
will be 
> allowed to happen on the bus while that function is being performed.
> 
> For a file the size of 32bytes, 1kByte, and 32kBtyes, what kind of 
blocking 
> delay might one expect from linux writing to flash for each of those 
file 
> sizes?  What would be an optimum flash filesystem to use for 
something like 
> this?  (if there is one)
> 
> cheers,
> cam
> 
> ps. i'm using linux 2.4.6-rmk1-rayl1 and 2.4.16-rmk2.  I could use 
the latest 
> kernel, too, i just haven't gotten around to it.  For development 
purposes, 
> i'm using a cirrus CDB89712 development board, which has the cs89712 
> processor and some Intel 28f320B3 flash on it.
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 
> 

Re: open question on flash speed/app blocking

[Herman Oosthuysen]

Hmm, the problem is not writing to flash per se, but rather garbage
collection, which requires the erase of dirty Flash sectors.  Erasing a
sector on an Intel chip can take from a couple hundred milliseconds to about
15 seconds, depending on the age of the device - by the time a chip gets to
15 seconds, I usually toss them away.  Flash chips tend to get slower with
age.  So, you have to repeat your tests a few hundred thousand times before
you can draw any real conclusions.  This also means that the worst case
performance will be about the same, irrespective of the file system used,
since the hardware constraints will eventually dominate.

If your system has to write to flash every few seconds, then consider using
a file system that has a large write cache, to enable you to survive the
periods when the system is busy erasing a sector.
--
Herman Oosthuysen
Herman@WirelessNetworksInc.com
Suite 300, #3016, 5th Ave NE,
Calgary, Alberta, T2A 6K4, Canada
Phone: (403) 569-5688, Fax: (403) 235-3965
----- Original Message -----
From: Cam Mayor <cmayor@iders.ca>
To: linux MTD mailing list <linux-mtd@lists.infradead.org>
Sent: Tuesday, January 15, 2002 11:18 AM
Subject: open question on flash speed/app blocking


Hi all,

We're making an app that periodically writes a persistance file to flash.  I
haven't actually gotten a flash file system area working yet on my board, so
i can't test this yet for speed.   I know that the results will vary with
the
hardware, the filesystem used, and a handful of other factors.  I'm assuming
that a write to the flash will be blocking - that is, nothing else will be
allowed to happen on the bus while that function is being performed.

For a file the size of 32bytes, 1kByte, and 32kBtyes, what kind of blocking
delay might one expect from linux writing to flash for each of those file
sizes?  What would be an optimum flash filesystem to use for something like
this?  (if there is one)

cheers,
cam

ps. i'm using linux 2.4.6-rmk1-rayl1 and 2.4.16-rmk2.  I could use the
latest
kernel, too, i just haven't gotten around to it.  For development purposes,
i'm using a cirrus CDB89712 development board, which has the cs89712
processor and some Intel 28f320B3 flash on it.

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: open question on flash speed/app blocking

[Vipin Malik]

http://www.embeddedlinuxworks.com/articles/jffs_guide.html

then

http://www.embeddedlinuxworks.com/articles/db_project.html

I've also done quite a bit of blocking time tests on JFFS2 (about 6-8 months
ago). See the (JFFS) list archives for details and graphs etc.

Vipin


Herman Oosthuysen wrote:

> Hmm, the problem is not writing to flash per se, but rather garbage
> collection, which requires the erase of dirty Flash sectors.  Erasing a
> sector on an Intel chip can take from a couple hundred milliseconds to about
> 15 seconds, depending on the age of the device - by the time a chip gets to
> 15 seconds, I usually toss them away.  Flash chips tend to get slower with
> age.  So, you have to repeat your tests a few hundred thousand times before
> you can draw any real conclusions.  This also means that the worst case
> performance will be about the same, irrespective of the file system used,
> since the hardware constraints will eventually dominate.
>
> If your system has to write to flash every few seconds, then consider using
> a file system that has a large write cache, to enable you to survive the
> periods when the system is busy erasing a sector.
> --
> Herman Oosthuysen
> Herman@WirelessNetworksInc.com
> Suite 300, #3016, 5th Ave NE,
> Calgary, Alberta, T2A 6K4, Canada
> Phone: (403) 569-5688, Fax: (403) 235-3965
> ----- Original Message -----
> From: Cam Mayor <cmayor@iders.ca>
> To: linux MTD mailing list <linux-mtd@lists.infradead.org>
> Sent: Tuesday, January 15, 2002 11:18 AM
> Subject: open question on flash speed/app blocking
>
> Hi all,
>
> We're making an app that periodically writes a persistance file to flash.  I
> haven't actually gotten a flash file system area working yet on my board, so
> i can't test this yet for speed.   I know that the results will vary with
> the
> hardware, the filesystem used, and a handful of other factors.  I'm assuming
> that a write to the flash will be blocking - that is, nothing else will be
> allowed to happen on the bus while that function is being performed.
>
> For a file the size of 32bytes, 1kByte, and 32kBtyes, what kind of blocking
> delay might one expect from linux writing to flash for each of those file
> sizes?  What would be an optimum flash filesystem to use for something like
> this?  (if there is one)
>
> cheers,
> cam
>
> ps. i'm using linux 2.4.6-rmk1-rayl1 and 2.4.16-rmk2.  I could use the
> latest
> kernel, too, i just haven't gotten around to it.  For development purposes,
> i'm using a cirrus CDB89712 development board, which has the cs89712
> processor and some Intel 28f320B3 flash on it.
>
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
>
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: open question on flash speed/app blocking

[Mark Sienkiewicz]

At 12:47 PM 1/16/2002 -0700, Herman Oosthuysen wrote:
>Hmm, the problem is not writing to flash per se, but rather garbage
>collection, which requires the erase of dirty Flash sectors.
...
 >If your system has to write to flash every few seconds, then consider using
>a file system that has a large write cache, to enable you to survive the
>periods when the system is busy erasing a sector.

Or you can use a system that starts erasing sectors well in advance of
needing them, so you never need to wait for the erase operation.

For example, suppose the only thing you need to store in the flash is a
stream of records where you only need the most recent XXX bytes.  You could
make a circular buffer in the flash, and start an erase operation any time
there are not at least N already-erased sectors ahead of where you are
writing.  If your data rate has a known upper bound, you can compute N from
the maximum erase time on the spec sheet for your flash chip.

That doesn't exactly make a filesystem, but it is very easy to do from user
mode.  I have a similar application:  I save a big chunk of data at shutdown
time.  At boot time, I find the newest record and restore it, but I also
find the _oldest_ record and start erasing it.  At shutdown time, I don't
need to erase anything -- it is already erased.

I'm not that familiar with jffs2, but if I remember correctly, it has a
filesystem parameter for how many empty flash sectors it must maintain.
Maybe if you set it higher than the minimum needed to make garbage
collection work, it will erase things early enough to avoid blocking anybody.
If you want to use jffs2, it would be worth looking into.

Mark S.