About GC

About GC

[Steve Tsai]

The recent CVS code has a great improvement at mounting time. It's
great. I test it with the 32Mbytes NAND flash and the mounting time
reduce to 10 seconds(the original time is 50 seconds). After mounting, I
found that the GC thread will take the most CPU time(99.9)  in my system
for a while. How can I make jffs2_garbage_collection_pass to reduce CPU
time?

Steve Tsai

Re: About GC

[David Woodhouse]

(redirected to jffs list)

startec@ms11.hinet.net said:
> The recent CVS code has a great improvement at mounting time. It's
> great. I test it with the 32Mbytes NAND flash and the mounting time
> reduce to 10 seconds(the original time is 50 seconds). After mounting,
> I found that the GC thread will take the most CPU time(99.9)  in my
> system for a while. How can I make jffs2_garbage_collection_pass to
> reduce CPU time?

I just realised that the long mail I just composed on GC performance isn't 
actually related to your question :) But I'm going to send it anyway. First 
though, I'll answer the question.

Aside from GC, the GC thread is now _also_ going through the flash doing all
the CRC checks that the scan code used to do. The main difference is that
you don't have to wait for it. You can even write to the file system before 
it's finished -- you just can't do any GC. So as long as the GC thread was 
staying ahead of you making enough space before you rebooted, you'll be 
able to do some writing before you catch up to it again afterwards.

While it's doing that, it _should_ be being fairly nice and yielding
to other processes which want the CPU. Perhaps the code in 
jffs2_get_inode_nodes() doesn't do that, come to think of it -- sticking a 
cond_resched() in there can't go amiss. 

If you want to get an app running really quickly after boot, mount the 
rootfs readonly, and the GC thread won't start. Then you can get your app 
started and remount read/write, and the GC thread will start up. Or mount 
read/write and send the GC thread a SIGSTOP till you want it to go on. 

I suspect just identifying the places where it needs a cond_resched() would 
probably be sufficient though.

--
dwmw2

Re: About GC

[David Woodhouse]

(redirected to jffs list)

startec@ms11.hinet.net said:
> The recent CVS code has a great improvement at mounting time. It's
> great. I test it with the 32Mbytes NAND flash and the mounting time
> reduce to 10 seconds(the original time is 50 seconds). 

We can probably do better than that. I think we're still not page-aligning 
our reads during scan.

> After mounting, I found that the GC thread will take the most CPU 
> time(99.9)  in my system for a while. How can I make 
> jffs2_garbage_collection_pass to reduce CPU time?

(not really answering the question but I've written it now...)

Well, telling me it takes 99% CPU time isn't wonderfully useful. What's 
more useful is telling me _what_ it's doing. But as it happens, I was 
looking at that yesterday. http://www.infradead.org/~dwmw2/holey-profile
is a profile run from about a couple of minutes of GC-intensive writes on a 
fs which is about 80% full. 

We already have code to mark nodes as 'pristine' when they can be copied 
intact without having to iget the inode to which they belong and then read 
and rewrite the data. That will help a lot with memory usage (far less 
thrashing of icache) and allow us to remove the zlib traces from the 
profile. (You don't see the read_inode time in the trace because the icache 
was already fully populated with _every_ inode in the fs before I started).

However, the amount of time spent in zlib decompressing and then 
recompressing each node we GC isn't actually as much as I thought it was. 
We could possibly get 10% improvement when we finish that code and make the 
GC use it, but not a lot more, AFAICT.

The vast majority of the time is spent in __delay, which will have been 
used from the erase routine. The logic there is "if(need_resched()) do_so() 
else udelay()" so on an unloaded system it will hog your CPU and check more 
frequently for completion than once per jiffie, but if there's other stuff 
to run it'll be kinder.

I don't think there's anything I can do there locally -- we're waiting for
hardware. What we need to do is ensure that we erase less. At the moment, 
we have a single block to which we are currently writing. GC'd nodes get 
written there mixed up with new nodes with writes from the user. The former 
has a high probability of being static long-lived data, while the latter is 
more likely to be volatile. The result is that we tend to end up with a lot
of erase blocks which are about half-full of long-lived data and half 
dirty. for each pair of those, what we _want_ is a completely full clean
one and a completely dirty one. 

We can probably get much closer to that ideal by splitting up the writes. 
If we have two blocks 'on the go' at a time, one of which is taking new 
writes from the user, the other of which is taking GC'd nodes from elsewhere
with older data, we will tend to group clean and dirty stuff more usefully, 
and hence have to do less erasing and copying to make progress when we come 
to GC.

We already have separate allocation routines for GC writes anyway, for other
reasons, so implementing this shouldn't be too painful. It's just a case of
convincing myself it's actually going to be worth it and getting round to it
-- as ever, in the absence of customers causing my boss to schedule my time
for it, it has to wait till I'm sufficiently disgusted by what I'm
_supposed_ to be working on that I steal enough cycles to play with it.

--
dwmw2