Re: More on nilfs_cleanerd and excessive writes (1 month flash card life expetancy)

["Jérôme Poulin" <jeromepoulin-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

I do not know what version of the NILFS-Tools you're using but the
latest is configurable in this way, that it will only clean when space
is critical.

Envoyé de mon appareil mobile.

Jérôme Poulin
Solutions G.A.

On 2011-08-12, at 06:38, Gordan Bobic <gordan-UpbECiGlrmGsTnJN9+BGXg@public.gmane.org> wrote:

> I just did some basic measuring and it looks like the total writes by nilfs_cleanerd on my SD card total about 1GB/minute (16MB/second, all my card can handle). Since the system is used all the time while it is on, that involves there always being things that need to be garbage collected, so it runs all the time. Even assuming it performance isn't an issue (running at nice 19 and ionice -c3, and performance IS an issue), that still means that the SD card will get 1,440GB of writes/day (1,4TB!). It's a 32GB MLC flash card, so assuming a 5,000 erase cycle life of 32nm MLC (ignoring any inevitable write amplification), that gives life expectancy of 160TB, or at the given rate of nilfs_cleanerd churn, about 12 days of usage. Call it a month with the assumption the machine isn't used all day every day.
>
> This is quite thoroughly unacceptable for usage on any flash media. Ignoring any other optimizations that might be applicable (e.g. smaller block size to minimize the number of blocks that have to be re-written), my immediate redneck solution is running this every minute as a cron job:
>
>
> ==========
> #!/bin/bash
>
> # Substitute /dev/mmcblk1p4 for your nilfs partition
> used=`df | grep /dev/mmcblk1p4 | awk '{ print $5; }' | sed -e 's/%//'`
>
> # If disk usage is more than 90%...
> if [ $used -gt 90 ]; then
>    # If nilfs_cleanerd is not running...
>    if (! pgrep nilfs_cleanerd > /dev/null ); then
>        nohup nice -n 19 ionice -c 3 /sbin/nilfs_cleanerd > /dev/null 2>&1 &
>    fi
> # If disk usage is less than 90%...
> elif [ $used -lt 80 ]; then
>    pkill nilfs_cleanerd > /dev/null 2>&1
> fi
> ==========
>
> This could of course be improved and "enterpriseified" further, e.g. check for all nilfs partitions and do the checks on all of them, make the free space amount thresholds based on 1/3 and 2/3 of free space (fs size - du), but this problem shouldn't really be looking for a solution in a cron job.
>
> It's not ideal and nilfs_cleanerd should be configurable to moderate itself in a similar way, but until that happens, I don't see any alternative to the above cron job. The write performance is fantastic for tasks that do a lot of writing, but the life expectancy issue is a very real one.
>
> Gordan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html