UBI: question about ubi_leb_change

UBI: question about ubi_leb_change

[Nancy]

Hi,
    As I know, ubi_leb_change only accept a whole LEB data as one of
its parameter.
    If file system modify a little data which in a old mapped LEB, it
needs to read the whole old mapped LEB data in its buffer first, then
do the little modification, finally deliver to ubi_leb_change?  Is
that right?
    when remove files, there's only inodes .....some file system meta
data changed,
but the real file data still exist which means the LEB still mapped.
Then when you write new data to this very LEB( but contains only
trash), maybe this time, your new data is a whole LEB size.  Still
goes the process:  read the mapped LEB old data to buffer,  replace
the old data with the new one in buffer, deliver to ubi_leb_change?
Really like that?  If the answer is yes, in this situation, waste a
whole block reading time. Maybe UBIFS does not do this.
     But FTL on UBI will face this problem.  And there's no suitable
UBI API for that needs.
     It need :  old_pnum = ubi_leb_unmap( ); but do not erase the mapped PEB
                  new_pnum = ubi_leb_map( );
                  ubi_read_peb( old_pnum,.....)  just like
ubi_read_leb  function
                  ubi_srub_peb( old_pnum ,....)

the write_cache in FTL layer only store changed data, when flush it to
Nand flash
it call " flush_write_cache" function
void flush_write_cache(.......){
         fill_write_cache(......);
        ubi_leb_change(......);
        .....
}
fill_write_cache() will call ubi_read_peb() to fill write_cache with
needed pages which come from the PEB(old_pnum)

      Or there still have another way to solve this, please give me a hint!

      Many thanks!


--
Best wishes,
Nancy

Re: UBI: question about ubi_leb_change

[Artem Bityutskiy]

Hi,

On Thu, 2008-03-06 at 20:26 +0800, Nancy wrote:
> Hi,
>     As I know, ubi_leb_change only accept a whole LEB data as one of
> its parameter.
Correct.

>     If file system modify a little data which in a old mapped LEB, it
> needs to read the whole old mapped LEB data in its buffer first, then
> do the little modification, finally deliver to ubi_leb_change?  Is
> that right?
This is what can be done. But file systems should not normally do this,
because it is inefficient. It is more efficient to do out-of-place
updates, which means writing data to somewhere else (a free LEB), and
treat the older data as obsolete. This is what JFFS2 and UBIFS do.

UBIFS does use the in-place update method you have described, but very
rarely. It uses it only when the FS is nearly full, and only as
last-resort method of updating the FS index, when out-of-place updates
are impossible.

>     when remove files, there's only inodes .....some file system meta
> data changed,
> but the real file data still exist which means the LEB still mapped.
> Then when you write new data to this very LEB( but contains only
> trash), maybe this time, your new data is a whole LEB size.  Still
> goes the process:  read the mapped LEB old data to buffer,  replace
> the old data with the new one in buffer, deliver to ubi_leb_change?
> Really like that?

If you have an LEB, and the contents of LEB is not needed, you should
just unmap it and write your new data. No need to read it.

>   If the answer is yes, in this situation, waste a
> whole block reading time. Maybe UBIFS does not do this.

Of course UBIFS does not do it - it would be slow if it did :-)

>      But FTL on UBI will face this problem.  And there's no suitable
> UBI API for that needs.
>      It need :  old_pnum = ubi_leb_unmap( ); but do not erase the mapped PEB
>                   new_pnum = ubi_leb_map( );
>                   ubi_read_peb( old_pnum,.....)  just like
> ubi_read_leb  function
>                   ubi_srub_peb( old_pnum ,....)
> 
> the write_cache in FTL layer only store changed data, when flush it to
> Nand flash
> it call " flush_write_cache" function
> void flush_write_cache(.......){
>          fill_write_cache(......);
>         ubi_leb_change(......);
>         .....
> }
> fill_write_cache() will call ubi_read_peb() to fill write_cache with
> needed pages which come from the PEB(old_pnum)

I do not really get it what you meant. There are many ways to create an
FTL, and I do not know which one you choose. I suggested a simple one
here:
http://lists.infradead.org/pipermail/linux-mtd/2008-January/020381.html

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: UBI: question about ubi_leb_change

[Nancy]

> Hi,
>
> On Thu, 2008-03-06 at 20:26 +0800, Nancy wrote:
> > Hi,
> >     As I know, ubi_leb_change only accept a whole LEB data as one of
> > its parameter.
> Correct.
>
> >     If file system modify a little data which in a old mapped LEB, it
> > needs to read the whole old mapped LEB data in its buffer first, then
> > do the little modification, finally deliver to ubi_leb_change?  Is
> > that right?
> This is what can be done. But file systems should not normally do this,
> because it is inefficient. It is more efficient to do out-of-place
> updates, which means writing data to somewhere else (a free LEB), and
> treat the older data as obsolete. This is what JFFS2 and UBIFS do.
>
> UBIFS does use the in-place update method you have described, but very
> rarely. It uses it only when the FS is nearly full, and only as
> last-resort method of updating the FS index, when out-of-place updates
> are impossible.

Thank you for your information!  It do help a lot !

> >     when remove files, there's only inodes .....some file system meta
> > data changed,
> > but the real file data still exist which means the LEB still mapped.
> > Then when you write new data to this very LEB( but contains only
> > trash), maybe this time, your new data is a whole LEB size.  Still
> > goes the process:  read the mapped LEB old data to buffer,  replace
> > the old data with the new one in buffer, deliver to ubi_leb_change?
> > Really like that?
>
> If you have an LEB, and the contents of LEB is not needed, you should
> just unmap it and write your new data. No need to read it.
>

   Yes, but in FTL layer, it don't know whether the old mapped LEB is
useful or not until
the new mapped LEB which data stored in a write_cache have to be write
to PEB. The
write_cache maybe full filled with new data or just a little new data,
others have to be read from the old mapped PEB

http://lists.infradead.org/pipermail/linux-mtd/2008-January/020381.html
 suggest to use
ubi_leb_change.   Which force the write_cache to read a whole LEB(old
mapped) back first.
That waste so much time, in fact, can't bare, too slow!

    FTL is using in-place update,although write to different PEB,  the
unchange data have to be read back no matter those data is useful or
not ( cause in FTL layer it doesn't know it is useful or not)

    Or FTL can have a same idea like UBIFS, implement log to make
out-place change become reality .....  Is that possible?

    Thanks in advance!


--
Best wishes,
Nancy

Re: UBI: question about ubi_leb_change

[Artem Bityutskiy]

On Thu, 2008-03-06 at 21:39 +0800, Nancy wrote:
>    Yes, but in FTL layer, it don't know whether the old mapped LEB is
> useful or not until
> the new mapped LEB which data stored in a write_cache have to be write
> to PEB. The
> write_cache maybe full filled with new data or just a little new data,
> others have to be read from the old mapped PEB

Ok, probably I understand what you are talking about. If you have a huge
file, and you remove it, the file-system does few changes in meta-data,
and that's it. Right, FTL does not know that the FS blocks the contents
of the file are stored at, are now in fact obsolete.

But UBI in now way can help you to solve this problem. How could it?
Invent an interface which would help.

The only way to go is to teach the FS which sits on top of you to inform
you about all blocks which become free when something is deleted. But
the existing FSes do not do this. An yes, this means you'll move garbage
all over the place, which is slow.

This is why we started UBIFS instead of doing FTL - because we assumed a
native FS should be significantly more efficient then FTL.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: UBI: question about ubi_leb_change

[Nancy]

> But UBI in now way can help you to solve this problem. How could it?
> Invent an interface which would help.
>
> The only way to go is to teach the FS which sits on top of you to inform
> you about all blocks which become free when something is deleted. But
> the existing FSes do not do this. An yes, this means you'll move garbage
> all over the place, which is slow.
>
> This is why we started UBIFS instead of doing FTL - because we assumed a
> native FS should be significantly more efficient then FTL.

But Windows do not recognize UBIFS.  I have to support VFAT to work on
UBI, how can I do? Guess I have nothing to do but suffer the slow
time. Now I just want to save a little time, do not
read the whole old mapped LEB back, but only necessary pages.  The
VFAT partitions usuallly use for storing  media data (mp3, mp4, avi
....). Only the filesystem meta-data stored area has much chance to be
little change.

So I needs some functions which I discribe before.  I met some problem
when I created them.
Maybe one more day will be fix,  or still can't, if it last too long
time still can't fix, hope I can get some help from you if you are
free.

Thank you :-)

---
Best wishes,
Nancy