Question: background GC on UBIFS

Question: background GC on UBIFS

[Wang Fangpeng]

Hi, folks,

We did some performance test of writing data to a ubifs volume by dd. The block
size of dd is 1MB.

When the volume has more than 20% free space, the speed of writing is about
1.4MB/s ~ 1.2MB/s.

However, when the free space is less and less, the speed decreases to about
83KB/s. We think the overhead is caused by GC, since GC tries to find a free
space and move valid data to another place to get a free block.

Although we already have a kthread to collect freeable blocks, it doesn't
touch blocks which have both valid and invalid nodes. My question is why don't
we do such consolidation/defragmentation periodicly, so that the overhead
could be decreased when there is not enough free space?

thanks.

Re: Question: background GC on UBIFS

[Richard Weinberger]

Am 31.03.2016 um 08:25 schrieb Wang Fangpeng:
> Hi, folks,
> 
> We did some performance test of writing data to a ubifs volume by dd. The block
> size of dd is 1MB.
> 
> When the volume has more than 20% free space, the speed of writing is about
> 1.4MB/s ~ 1.2MB/s.
> 
> However, when the free space is less and less, the speed decreases to about
> 83KB/s. We think the overhead is caused by GC, since GC tries to find a free
> space and move valid data to another place to get a free block.
> 
> Although we already have a kthread to collect freeable blocks, it doesn't
> touch blocks which have both valid and invalid nodes. My question is why don't
> we do such consolidation/defragmentation periodicly, so that the overhead
> could be decreased when there is not enough free space?

AFAIK UBIFS does this on demand to be friendly to the flash.
Otherwise it would move blocks around for nothing (if there is no writer).

Doing it periodically is doable but how often and when? Only when the system
is completely idle?
I'm not sure if there will be a huge performance gain. Basically you shift the
IO load to another location.

But you can give it a try. Add a new debugfs file to trigger an aggressive GC run.
Currently GC tries to produce as much space as we budget. You can change GC to
produce as much space as possible.

Thanks,
//richard

Re: Question: background GC on UBIFS

[Artem Bityutskiy]

On Thu, 2016-03-31 at 10:53 +0200, Richard Weinberger wrote:
> Doing it periodically is doable but how often and when? Only when the
> system
> is completely idle?

I would say - we should give a knob (mechanism) and let the userland
decide when and how much to run it (policy).

> But you can give it a try. Add a new debugfs file to trigger an
> aggressive GC run.

I'd say the trigger should be - GC one eraseblock. Or better, clean
some dirt, or make some free space. Internally this will be GC one
eraseblock, but later on this may change. UBIFS will pick the best
eraseblock to GC automatically. Userland just triggers it at the right
poing. I guess this should be a blocking API - userland blocks until GC
is done. Then it can force more GC.

You can play with this knob, see if it helps.

However, fundamentally, UBIFS will always get slower when the space is
short, by its design. You can make it a bit faster when you have little
space, but you can't make it as fast as when there is more space,
unfortunately. Not easily, at least.

Artem.