Re: nilfs_cleanerd using a lot of disk-write bandwidth

[dexen deVries <dexen.devries-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

Hi Gordan,


On Tuesday 09 of August 2011 12:18:12 you wrote:
>  I'm seeing nilfs_cleanerd using a lot of disk write bandwidth according
>  to iotop. It seems to be performing approximately equal amounts of reads
>  and writes when it is running. Reads I can understand, but why is it
>  writing so much in order to garbage collect? Should it not be just
>  trying to mark blocks as free? The disk I/O r/w symmetry implies that it
>  is trying to do something like defragment the file system. Is there a
>  way to configure this behaviour in some way? The main use-case I have
>  for nilfs is cheap flash media that suffers from terrible random-write
>  performance, but on such media this many writes are going to cause media
>  failure very quickly. What can be done about this?


I'm not a NILFS2 developer, so don't rely too much on the following remarks!

NILFS2 consider filesystem as a (wrapped around) list of segments, by default 
each 8MB. Those segments contain both file data and metadata.

cleanerd operates on whole segments; normally either 2 or 4 in one pass 
(depending on remaining free space). It seems to me a segment is reclaimed 
when there is any amount of garbage in it, no matter how small. Thus you see, 
in some cases, about as much of read as of write.

One way could be be to make cleanerd configurable so it doesn't reclaim 
segments that have only very little garbage in them. That would probably be a 
trade-off between wasted diskspace and lessened bandwidth use.

As for wearing flash media down, I believe NILFS2 is still very good for them, 
because it tends to write in large chunks -- much larger than the original 
512B sector -- and not over-write once written areas (untill reclaimed by 
cleanerd, often much, much later). Once the flash' large erase unit is erased, 
NILFS2 append-writes to it, but not over-writes already written data. Which 
means the flash is erased almost as little as possible.


Regards,
-- 
dexen deVries

[[[↓][→]]]

For example, if the first thing in the file is:
   <?kzy irefvba="1.0" rapbqvat="ebg13"?>
an XML parser will recognize that the document is stored in the traditional 
ROT13 encoding.

(( Joe English, http://www.flightlab.com/~joe/sgml/faq-not.txt ))
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html