Run UBIFS on top of IDE mode NAND disk?

Run UBIFS on top of IDE mode NAND disk?

[Subodh Nijsure]

Hi,
 
I have board with 4GB NAND memory chip, it is configured to operate in IDE
mode. And currently I am creating ext3 file-system on it. 
This chip also have on-board controller that does wear-levelling, ECC so the
quesitons are:
 
Can I (Should I) run UBIFS on of it and gain more of wear-levelling or its
not worth it? 
 
/Subodh Nijsure
(Standard Disclaimers Apply)

Re: Run UBIFS on top of IDE mode NAND disk?

[Artem Bityutskiy]

Hi,

On Tue, 2009-04-14 at 23:10 -0700, Subodh Nijsure wrote: 
> I have board with 4GB NAND memory chip, it is configured to operate in IDE
> mode. And currently I am creating ext3 file-system on it. 
> This chip also have on-board controller that does wear-levelling, ECC so the
> quesitons are:
>  
> Can I (Should I) run UBIFS on of it and gain more of wear-levelling or its
> not worth it? 

Vs "Can I": if this is raw NAND, you just need an MTD driver for it.
Then you can use UBI/UBIFS on top of that.

Vs "Should I": you really should not ask questions like this. It is a
question of your design. Look at your NAND and how many erase cycles
each eraseblock survives. Then roughly calculate how many years or
months it should survive with ext3, which has fixed-position journal
and inode table and bitmap. Then decide whether you need WL or not.
Everything depends on your requirements.

Also, if this is raw NAND, then HW does not hide bad blocks for you,
right? How will you manage bad eraseblocks then, ext3 cannot do this.
It just panics in case of any I/O error.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: Run UBIFS on top of IDE mode NAND disk?

[Jamie Lokier]

Artem Bityutskiy wrote:
> Hi,
> 
> On Tue, 2009-04-14 at 23:10 -0700, Subodh Nijsure wrote: 
> > I have board with 4GB NAND memory chip, it is configured to
> > operate in IDE mode. And currently I am creating ext3 file-system
> > on it.  This chip also have on-board controller that does
> > wear-levelling, ECC so the quesitons are:
> >  
> > Can I (Should I) run UBIFS on of it and gain more of wear-levelling or its
> > not worth it? 
> 
> Vs "Can I": if this is raw NAND, you just need an MTD driver for it.
> Then you can use UBI/UBIFS on top of that.
> 
> Vs "Should I": you really should not ask questions like this. It is a
> question of your design. Look at your NAND and how many erase cycles
> each eraseblock survives. Then roughly calculate how many years or
> months it should survive with ext3, which has fixed-position journal
> and inode table and bitmap. Then decide whether you need WL or not.
> Everything depends on your requirements.
> 
> Also, if this is raw NAND, then HW does not hide bad blocks for you,
> right? How will you manage bad eraseblocks then, ext3 cannot do this.
> It just panics in case of any I/O error.

But the on board controller does wear-levelling, he said.

I think the question is whether the on board controller's
wear-levelling is good or not.  If it's low quality, UBI will do it
better.  If the controller is good quality, there's no need for UBI on
top of it.

I guess that two layers of good quality wear levelling just adds more
wear, because they both copy data around.

If it can be used as raw NAND, and the chip really is a NAND with
standard NAND properties, _and_ if it's possible to disable the
controller's wear levelling and get direct NAND access, then at least
you know that UBI provides a good quality wear levelling
implementation, and you have access to some statistics too.

-- Jamie

Re: Run UBIFS on top of IDE mode NAND disk?

[Artem Bityutskiy]

On Wed, 2009-04-15 at 17:29 +0100, Jamie Lokier wrote:
> Artem Bityutskiy wrote:
> > Hi,
> > 
> > On Tue, 2009-04-14 at 23:10 -0700, Subodh Nijsure wrote: 
> > > I have board with 4GB NAND memory chip, it is configured to
> > > operate in IDE mode. And currently I am creating ext3 file-system
> > > on it.  This chip also have on-board controller that does
> > > wear-levelling, ECC so the quesitons are:
> > >  
> > > Can I (Should I) run UBIFS on of it and gain more of wear-levelling or its
> > > not worth it? 
> > 
> > Vs "Can I": if this is raw NAND, you just need an MTD driver for it.
> > Then you can use UBI/UBIFS on top of that.
> > 
> > Vs "Should I": you really should not ask questions like this. It is a
> > question of your design. Look at your NAND and how many erase cycles
> > each eraseblock survives. Then roughly calculate how many years or
> > months it should survive with ext3, which has fixed-position journal
> > and inode table and bitmap. Then decide whether you need WL or not.
> > Everything depends on your requirements.
> > 
> > Also, if this is raw NAND, then HW does not hide bad blocks for you,
> > right? How will you manage bad eraseblocks then, ext3 cannot do this.
> > It just panics in case of any I/O error.
> 
> But the on board controller does wear-levelling, he said.
> 
> I think the question is whether the on board controller's
> wear-levelling is good or not.  If it's low quality, UBI will do it
> better.  If the controller is good quality, there's no need for UBI on
> top of it.
> 
> I guess that two layers of good quality wear levelling just adds more
> wear, because they both copy data around.
> 
> If it can be used as raw NAND, and the chip really is a NAND with
> standard NAND properties, _and_ if it's possible to disable the
> controller's wear levelling and get direct NAND access, then at least
> you know that UBI provides a good quality wear levelling
> implementation, and you have access to some statistics too.

Right, he responded me in private (do not know why) and I
apologized for misunderstanding, and shared my thoughts.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: Run UBIFS on top of IDE mode NAND disk?

[Miles Nordin]

[-- Attachment #1: Type: text/plain, Size: 2082 bytes --]

>>>>> "sn" == Subodh Nijsure <sunijsur@cisco.com> writes:

    sn> Hi, I have board with 4GB NAND memory chip

    sn> Can I (Should I) run UBIFS on of it and gain more of
    sn> wear-levelling or its not worth it?

I tried to do this with 16GB USB sticks.  There is currently some
limit in the mtd-utils around 2GB or 3GB.  I think you will have to
wait for the 64-bit ioctl patches to go in.

I would not expect bad-block remapping of ubifs layer to work at all
over an IDE device because IDE error handling is too goofy, and once a
bad block is showing through the proprietary wear-leveling layer, the
proprietary wear leveler may not be working sanely any more.  Probably
once the stick/board/whatever-it-is develops one bad block which shows
through the wear leveling, you will have to throw it out.

The reason I wanted to use it on my USB stick was not only for wear
leveling but because ubifs does checksums so I'll know if the cheapo
stick is corrupting my data silently.

And I think ubi has a scrub feature (which it claims is constantly
running in the background, thus much better than 'zpool scrub'.  on
ZFS they leave it as user's bother to initiate scrubs and thus user's
fault rather than the developers' if scrubs cut performance in half).

Also I hoped a log-structured filesystem would perform faster for
writing and have fewer of the weird problems described here:

 http://sqlite.org/atomiccommit.html

that plague overwrite filesystems like ext3/XFS/JFS/HFS+.  I wanted to
try it out.  

In general it looks to me like a really good idea to use the flash
filesystems over a layer of proprietary wear-leveling, but
supplimenting the proprietary wear leveling might not be the best
argument for doing it (because if the proprietary wear leveling is bad
in any but a few specific ways(like the old 16MB-chunk way), then you
are screwed anyway).  There are other good reasons for doing it
though.  Unfortunately there are scalability issues w.r.t. flash size,
and almost no one is doing this yet so it's likely to be clumsy.  I've
given up on it for a few years.

[-- Attachment #2: Type: application/pgp-signature, Size: 304 bytes --]