Using UBIFS as an FTL

Using UBIFS as an FTL

[Daniel Ehrenberg]

Hi Artem, others,

For my project, I'm looking into using ext4 on top of NAND flash, in
order to use some fancy ext4 features. For this, I need an FTL in the
middle. I'm wondering which would be good to use. Options that I've
looked at are:
- ubiblock--the read-modify-write sounds unacceptable to me, even if
wear leveling and atomicity are handled.
- Some coworkers have suggested a new effort to build a new block
device, but that that's a huge project and takes a long time to get
right.
- loopback-mounting a file on ubifs--From skimming the code, it looks
to me like ubifs uses some nice datastructures to handle writes within
a file without doing read-modify-writes all the time as ubiblock
forces. ubifs authors/maintainers, do you see any downside to using
ubifs this way?

Thanks,
Dan

Re: Using UBIFS as an FTL

[hujianyang]

On 2014/7/26 2:21, Daniel Ehrenberg wrote:
> - loopback-mounting a file on ubifs--From skimming the code, it looks
> to me like ubifs uses some nice datastructures to handle writes within
> a file without doing read-modify-writes all the time as ubiblock
> forces. ubifs authors/maintainers, do you see any downside to using
> ubifs this way?

This way seems most easily to realize. Synchronize the loop file and
maybe you will never worry about power cut.

How about using UBI-layer and writing a new block driver to handle
bio? This driver should care about which leb on the flash has data
and other thing else(I don't have a clear idea now). The 'leb_change'
ops in UBI-layer maybe helpful for in-place update and atomic write.
But This way is harder than using a loop file.

Re: Using UBIFS as an FTL

[Richard Weinberger]

On Sat, Jul 26, 2014 at 1:41 AM, hujianyang <hujianyang@huawei.com> wrote:
> On 2014/7/26 2:21, Daniel Ehrenberg wrote:
>> - loopback-mounting a file on ubifs--From skimming the code, it looks
>> to me like ubifs uses some nice datastructures to handle writes within
>> a file without doing read-modify-writes all the time as ubiblock
>> forces. ubifs authors/maintainers, do you see any downside to using
>> ubifs this way?
>
> This way seems most easily to realize. Synchronize the loop file and
> maybe you will never worry about power cut.
>
> How about using UBI-layer and writing a new block driver to handle
> bio? This driver should care about which leb on the flash has data
> and other thing else(I don't have a clear idea now). The 'leb_change'
> ops in UBI-layer maybe helpful for in-place update and atomic write.
> But This way is harder than using a loop file.

Instead of adding another UBI block layer, please integrate such functionality
in Ezequiel's UBI block.

-- 
Thanks,
//richard

Re: Using UBIFS as an FTL

[hujianyang]

On 2014/7/27 15:20, Richard Weinberger wrote:
> On Sat, Jul 26, 2014 at 1:41 AM, hujianyang <hujianyang@huawei.com> wrote:
>> On 2014/7/26 2:21, Daniel Ehrenberg wrote:
>>> - loopback-mounting a file on ubifs--From skimming the code, it looks
>>> to me like ubifs uses some nice datastructures to handle writes within
>>> a file without doing read-modify-writes all the time as ubiblock
>>> forces. ubifs authors/maintainers, do you see any downside to using
>>> ubifs this way?
>>
>> This way seems most easily to realize. Synchronize the loop file and
>> maybe you will never worry about power cut.
>>
>> How about using UBI-layer and writing a new block driver to handle
>> bio? This driver should care about which leb on the flash has data
>> and other thing else(I don't have a clear idea now). The 'leb_change'
>> ops in UBI-layer maybe helpful for in-place update and atomic write.
>> But This way is harder than using a loop file.
> 
> Instead of adding another UBI block layer, please integrate such functionality
> in Ezequiel's UBI block.
> 

Oh, sorry~! I didn't know his work before. I should have done some
more researches.

Did you mean MTD_UBI_BLOCK? It seems a read-only interface. What's
your plan on it?

Add cc: Ezequiel Garcia

Richard, Thank you very much~!

Re: Using UBIFS as an FTL

[Ezequiel Garcia]

On 27 Jul 09:20 AM, Richard Weinberger wrote:
> On Sat, Jul 26, 2014 at 1:41 AM, hujianyang <hujianyang@huawei.com> wrote:
> > On 2014/7/26 2:21, Daniel Ehrenberg wrote:
> >> - loopback-mounting a file on ubifs--From skimming the code, it looks
> >> to me like ubifs uses some nice datastructures to handle writes within
> >> a file without doing read-modify-writes all the time as ubiblock
> >> forces. ubifs authors/maintainers, do you see any downside to using
> >> ubifs this way?
> >
> > This way seems most easily to realize. Synchronize the loop file and
> > maybe you will never worry about power cut.
> >
> > How about using UBI-layer and writing a new block driver to handle
> > bio? This driver should care about which leb on the flash has data
> > and other thing else(I don't have a clear idea now). The 'leb_change'
> > ops in UBI-layer maybe helpful for in-place update and atomic write.
> > But This way is harder than using a loop file.
> 
> Instead of adding another UBI block layer, please integrate such functionality
> in Ezequiel's UBI block.
> 

Daniel already mentioned it didn't git his needs. Quoting him:

"""
- ubiblock--the read-modify-write sounds unacceptable to me, even if
wear leveling and atomicity are handled.
- Some coworkers have suggested a new effort to build a new block
device, but that that's a huge project and takes a long time to get
right.
"""

Daniel: Maybe you can mention why you believe it's unacceptable?
-- 
Ezequiel García, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com

Re: Using UBIFS as an FTL

[Richard Weinberger]

Am 28.07.2014 05:06, schrieb Ezequiel Garcia:
> On 27 Jul 09:20 AM, Richard Weinberger wrote:
>> On Sat, Jul 26, 2014 at 1:41 AM, hujianyang <hujianyang@huawei.com> wrote:
>>> On 2014/7/26 2:21, Daniel Ehrenberg wrote:
>>>> - loopback-mounting a file on ubifs--From skimming the code, it looks
>>>> to me like ubifs uses some nice datastructures to handle writes within
>>>> a file without doing read-modify-writes all the time as ubiblock
>>>> forces. ubifs authors/maintainers, do you see any downside to using
>>>> ubifs this way?
>>>
>>> This way seems most easily to realize. Synchronize the loop file and
>>> maybe you will never worry about power cut.
>>>
>>> How about using UBI-layer and writing a new block driver to handle
>>> bio? This driver should care about which leb on the flash has data
>>> and other thing else(I don't have a clear idea now). The 'leb_change'
>>> ops in UBI-layer maybe helpful for in-place update and atomic write.
>>> But This way is harder than using a loop file.
>>
>> Instead of adding another UBI block layer, please integrate such functionality
>> in Ezequiel's UBI block.
>>
> 
> Daniel already mentioned it didn't git his needs. Quoting him:

I did not promote it as the ultimate solution.
But if we add such functionality it should reuse existing stuff instead
of adding another UBI block layer....

Thanks,
//richard

Re: Using UBIFS as an FTL

[Artem Bityutskiy]

On Fri, 2014-07-25 at 11:21 -0700, Daniel Ehrenberg wrote:
> Hi Artem, others,
> 
> For my project, I'm looking into using ext4 on top of NAND flash, in
> order to use some fancy ext4 features. For this, I need an FTL in the
> middle. I'm wondering which would be good to use. Options that I've
> looked at are:
> - ubiblock--the read-modify-write sounds unacceptable to me, even if
> wear leveling and atomicity are handled.

This may not give you the optimal performance.

> - Some coworkers have suggested a new effort to build a new block
> device, but that that's a huge project and takes a long time to get
> right.

Yes.

> - loopback-mounting a file on ubifs--From skimming the code, it looks
> to me like ubifs uses some nice datastructures to handle writes within
> a file without doing read-modify-writes all the time as ubiblock
> forces. ubifs authors/maintainers, do you see any downside to using
> ubifs this way?

I never tried loopback over UBIFS. Should work in theory.
Synchronization (as in 'fsync()') is something which comes to mind.

What I mean is that when ext4 handles an fsync(), or a commit, it
eventually needs to make sure the data goes to the underlying media. It
sends REQ_FLUSH bios, and it seems that the loopback driver will map
that to UBIFS's 'vfs_fsync()'. That should be fine, but may be too
coarse? You could just write the right pages instead, to gain better
performance?

Then ext4 commit has various ordering requirements, and I do not know
how this works with loopback.

This all matters if you need to have some kind of power-cut tolerance.

Thanks!

-- 
Best Regards,
Artem Bityutskiy

Re: Using UBIFS as an FTL

[Daniel Ehrenberg]

On Mon, Jul 28, 2014 at 9:54 AM, Artem Bityutskiy
<artem.bityutskiy@linux.intel.com> wrote:
>
> On Fri, 2014-07-25 at 11:21 -0700, Daniel Ehrenberg wrote:
> > Hi Artem, others,
> >
> > For my project, I'm looking into using ext4 on top of NAND flash, in
> > order to use some fancy ext4 features. For this, I need an FTL in the
> > middle. I'm wondering which would be good to use. Options that I've
> > looked at are:
> > - ubiblock--the read-modify-write sounds unacceptable to me, even if
> > wear leveling and atomicity are handled.
>
> This may not give you the optimal performance.

My understanding of ubiblock is that, when you do 4k writes scattered
around the device, they will magnify into block-sized
read-modify-writes. This causes ~64x as much wear (depending on the
device type) and goes at 1/64th the speed. I don't think I could
change ubiblock to not have this property because that wouldn't be
backwards-compatible--I'd need an extra layer of indirection to
separate logical pages from logical erase blocks. Isn't ubiblock
usable for, say, copying off or on a ubifs filesystem image? I
wouldn't want to remove that functionality (and maybe I'll end up
using it). It just seems like ubiblock and a real FTL need totally
different pieces of software.
>
> > - Some coworkers have suggested a new effort to build a new block
> > device, but that that's a huge project and takes a long time to get
> > right.
>
> Yes.
>
> > - loopback-mounting a file on ubifs--From skimming the code, it looks
> > to me like ubifs uses some nice datastructures to handle writes within
> > a file without doing read-modify-writes all the time as ubiblock
> > forces. ubifs authors/maintainers, do you see any downside to using
> > ubifs this way?
>
> I never tried loopback over UBIFS. Should work in theory.
> Synchronization (as in 'fsync()') is something which comes to mind.
>
> What I mean is that when ext4 handles an fsync(), or a commit, it
> eventually needs to make sure the data goes to the underlying media. It
> sends REQ_FLUSH bios, and it seems that the loopback driver will map
> that to UBIFS's 'vfs_fsync()'. That should be fine, but may be too
> coarse? You could just write the right pages instead, to gain better
> performance?
>
> Then ext4 commit has various ordering requirements, and I do not know
> how this works with loopback.
>
> This all matters if you need to have some kind of power-cut tolerance.
>
> Thanks!
>
> --
> Best Regards,
> Artem Bityutskiy
>
These are good points that I hadn't thought about. I'm also worrying
about the overhead of loopback, but that's really outside of the scope
of UBIFS.

Thanks a lot everyone for this help! I think I'll push back on the
application requirements and just use UBIFS for now, and then in a
year or two if the requirements push back, either try to optimize
loopback and handle the issues Artem raises, or implement an FTL, or
port the fancy ext4 features to UBIFS. It's good to know that I'm not
missing some big piece of code here that would solve my problems.

Thanks,
Dan

Re: Using UBIFS as an FTL

[Artem Bityutskiy]

On Mon, 2014-07-28 at 10:47 -0700, Daniel Ehrenberg wrote:
> My understanding of ubiblock is that, when you do 4k writes scattered
> around the device, they will magnify into block-sized
> read-modify-writes. This causes ~64x as much wear (depending on the
> device type) and goes at 1/64th the speed.

Yes, I think so.

>  I don't think I could
> change ubiblock to not have this property because that wouldn't be
> backwards-compatible--I'd need an extra layer of indirection to
> separate logical pages from logical erase blocks.

Say, ext4 can mount ancient ext3 images, and it supports many media
formats, etc. Some thing like this could in theory be done with
ubiblock. But this is not necessarily the _best_ way to go.

>  Isn't ubiblock
> usable for, say, copying off or on a ubifs filesystem image?

To copy the UBIFS image you can just read from the underlying UBI device
(/dev/ubiX_Y).

>  I
> wouldn't want to remove that functionality (and maybe I'll end up
> using it). It just seems like ubiblock and a real FTL need totally
> different pieces of software.

There are many aspects in FTL - how it scales WRT mount time, memory
consumption, etc. If someone decides to tackle only one small aspect at
a time, may be it makes sense to keep adding relatively small
improvements to ubiblock, maintain backward compatibility, several
on-the-media formats, etc. Keep adding features and keep doing
improvements.

But if a couple of smart folks decide to spend a year and create
something several orders of magnitude more superior, it is probably more
feasible to just implement a separate driver, say ubiftl or whatever.

If you ask me, I'd say it is largely up to the people doing the actual
work. Although talking to the community is a good idea in any case :-)

Thanks!

-- 
Best Regards,
Artem Bityutskiy

Re: Using UBIFS as an FTL

[Daniel Ehrenberg]

On Tue, Jul 29, 2014 at 12:16 AM, Artem Bityutskiy
<artem.bityutskiy@linux.intel.com> wrote:
> On Mon, 2014-07-28 at 10:47 -0700, Daniel Ehrenberg wrote:
>> My understanding of ubiblock is that, when you do 4k writes scattered
>> around the device, they will magnify into block-sized
>> read-modify-writes. This causes ~64x as much wear (depending on the
>> device type) and goes at 1/64th the speed.
>
> Yes, I think so.
>
>>  I don't think I could
>> change ubiblock to not have this property because that wouldn't be
>> backwards-compatible--I'd need an extra layer of indirection to
>> separate logical pages from logical erase blocks.
>
> Say, ext4 can mount ancient ext3 images, and it supports many media
> formats, etc. Some thing like this could in theory be done with
> ubiblock. But this is not necessarily the _best_ way to go.
>
>>  Isn't ubiblock
>> usable for, say, copying off or on a ubifs filesystem image?
>
> To copy the UBIFS image you can just read from the underlying UBI device
> (/dev/ubiX_Y).

Does this block device also work well for writing an image (with that
logic to delay writing out until you write to a different block) or
should I be using some other block device for that purpose?
>
>>  I
>> wouldn't want to remove that functionality (and maybe I'll end up
>> using it). It just seems like ubiblock and a real FTL need totally
>> different pieces of software.
>
> There are many aspects in FTL - how it scales WRT mount time, memory
> consumption, etc. If someone decides to tackle only one small aspect at
> a time, may be it makes sense to keep adding relatively small
> improvements to ubiblock, maintain backward compatibility, several
> on-the-media formats, etc. Keep adding features and keep doing
> improvements.
>
> But if a couple of smart folks decide to spend a year and create
> something several orders of magnitude more superior, it is probably more
> feasible to just implement a separate driver, say ubiftl or whatever.
>
> If you ask me, I'd say it is largely up to the people doing the actual
> work. Although talking to the community is a good idea in any case :-)
>
> Thanks!
>
> --
> Best Regards,
> Artem Bityutskiy
>