NILFS usage on NAND and NOR flash memory

NILFS usage on NAND and NOR flash memory

[Jesse Campbell]

Good day everyone, I do hope this has not already been asked as I had
trouble searching the archive, and nothing meaningful came up in a
search.

My question is this:
I am using android on an ARM-based pda phone.
Android works by having a system partition and a data partition. In
our environment, everything is mounted loop, as we have everything
sitting on an SD card formatted FAT for windows mobile (our launcher
runs from windows mobile)

How well would NILFS function as the loop data partition? We're
looking at 128MB, and it is used extensively for cache and sqlite3
database reads/writes. Currently we're using ext2, but it has issues
with fault tolerance, and on a device that doesn't unmount on
shutdown, faults happen every time you turn it off (lots of data
corruption).
ext3 was tried, but there were so many writes that the location of the
journal on the SD card burned and the card became unusable.

Does NILFS handle this kind of thing better? Is there a log that is
constantly written and overwritten, or does it sort of log as it goes?
My understanding of log-centric filesystems is better than a few days
ago (but still have lots to learn), it treats the whole partition as a
sequential loop for writes, with random access reads (right?). I would
think this would work better than ext2 for load leveling too.

Any ideas or suggestions would be greatly appreciated, my co-developer
things it would be "doing unnecessary writes" and that ext2 "should
work fine"... but I don't want to burn my card, and I'd like some
better fault tolerance.

Thanks all,
Jesse Campbell

Re: NILFS usage on NAND and NOR flash memory

[James M Long]

[-- Attachment #1.1: Type: text/plain, Size: 3106 bytes --]

Hello Jesse, I have been doing testing with sqlite on small nilfs ( not on
ARM ) partitions of about 1Gig in size so maybe I can be of some help.

> Good day everyone, I do hope this has not already been asked as I had
> trouble searching the archive, and nothing meaningful came up in a
> search.
>
> My question is this:
> I am using android on an ARM-based pda phone.
> Android works by having a system partition and a data partition. In
> our environment, everything is mounted loop, as we have everything
> sitting on an SD card formatted FAT for windows mobile (our launcher
> runs from windows mobile)
>
> How well would NILFS function as the loop data partition? We're
> looking at 128MB, and it is used extensively for cache and sqlite3
> database reads/writes. Currently we're using ext2, but it has issues
> with fault tolerance, and on a device that doesn't unmount on
> shutdown, faults happen every time you turn it off (lots of data
> corruption).
> ext3 was tried, but there were so many writes that the location of the
> journal on the SD card burned and the card became unusable.
>
> Does NILFS handle this kind of thing better? Is there a log that is
> constantly written and overwritten, or does it sort of log as it goes?
> My understanding of log-centric filesystems is better than a few days
> ago (but still have lots to learn), it treats the whole partition as a
> sequential loop for writes, with random access reads (right?). I would
> think this would work better than ext2 for load leveling too.
>
> Any ideas or suggestions would be greatly appreciated, my co-developer
> things it would be "doing unnecessary writes" and that ext2 "should
> work fine"... but I don't want to burn my card, and I'd like some
> better fault tolerance.
>
> Thanks all,
> Jesse Campbell
>
>
Well Jesse, I have been doing a lot of testing with various applications
including lucene indexes, postgresql databases and also sqlite3 databases.
From watching how all of these systems write to disk in the normal usage
case. ( No or limited use of in memory storage and most if not all changes
written to disk in real time ) I can tell you that you will be doing a lot
of writes to disk as you make changes to the sqlite db. Now if you are only
using sqlite as a storage mechanism, ie reading from the db upon application
startup and dumping the information back to disk upon app shutdown, you
shouldn't have to worry at all. On the other hand, if you are using it as a
normal db... your limited amount of space will require a lot of garbage
collection to prevent the partition from being filled. An example... one of
the applications that I have been testing will create a sqlite3 db of about
160M in size. The massive number of row inserts and updates to the db was
causing the 1Gig ( 976M formatted ) partition to be filled long before the
default setting for cleanerd got around to doing gc. Matter of fact the
partition would fill in less that 10 minutes.
Hopefully the above examples and information can be useful to you and anyone
else thinking about using sqlite on a nilfs partition.

James...

[-- Attachment #1.2: Type: text/html, Size: 3384 bytes --]

[-- Attachment #2: Type: text/plain, Size: 158 bytes --]

_______________________________________________
users mailing list
users-JrjvKiOkagjYtjvyW6yDsg@public.gmane.org
https://www.nilfs.org/mailman/listinfo/users

Re: NILFS usage on NAND and NOR flash memory

[Chris Samuel]

[-- Attachment #1.1: Type: Text/Plain, Size: 586 bytes --]

On Wed, 24 Jun 2009 11:42:32 am Jesse Campbell wrote:

> ext3 was tried, but there were so many writes that the location of the
> journal on the SD card burned and the card became unusable.

I know that btrfs (another COW filesystem) has an SSD mode (and an SSD mode 
for non-enterprise SSD's for 2.6.31), I wonder if NILFS could pick its brains 
for these devices ?

cheers,
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC

This email may come with a PGP signature as a file. Do not panic.
For more info see: http://en.wikipedia.org/wiki/OpenPGP

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 481 bytes --]

[-- Attachment #2: Type: text/plain, Size: 158 bytes --]

_______________________________________________
users mailing list
users-JrjvKiOkagjYtjvyW6yDsg@public.gmane.org
https://www.nilfs.org/mailman/listinfo/users