Endless garbage collection

Endless garbage collection

[Simon Munton]

Hi,

It's possible to get the garbage collection going in an endless loop by
creating lots of small files. eg, if I create 25 files, each 0 bytes long,
on an empty filesystem, garbage collection starts and goes on and on.

Perhaps the test for too much RAM being used in thread_should_wake() ought
to take the number of files into account? ie should it be:

	/* If there is too much RAM used by the various structures, GC */
	if (jffs_get_node_inuse() >
(c->fmc->used_size/c->fmc->max_chunk_size * 5 + 25 + jffs_get_file_count()))
{

where  jffs_get_file_count() returns the number of files on the filesystem.

Simon



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: Endless garbage collection

[David Woodhouse]

Simon.Munton@m4data.co.uk said:
> Perhaps the test for too much RAM being used in thread_should_wake()
> ought to take the number of files into account? ie should it be:

Probably something along the lines of what you suggested. I think we should
probably put it _after_ the check for dirty_size < sector_size too, 
until/unless we can be 100% sure that it'll never cause endless GC.

--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: Endless garbage collection

[Martin Gadbois]

Simon Munton wrote:

> It's possible to get the garbage collection going in an endless loop by
> creating lots of small files. eg, if I create 25 files, each 0 bytes long,
> on an empty filesystem, garbage collection starts and goes on and on.
>
> Perhaps the test for too much RAM being used in thread_should_wake() ought
> to take the number of files into account? ie should it be:
>
>         /* If there is too much RAM used by the various structures, GC */
>         if (jffs_get_node_inuse() >
> (c->fmc->used_size/c->fmc->max_chunk_size * 5 + 25 + jffs_get_file_count()))
> {
>
> where  jffs_get_file_count() returns the number of files on the filesystem.

You're absolutly right. The only issues are performance of the
jffs_get_file_count(), because it is executed every times an insert_node() is
called. We should keep a running count of how many files exists in the
partition, but I did not do that yet.
Here's my version:

if (jffs_get_node_inuse() > (c->fmc->used_size/c->fmc->max_chunk_size * 5
                        + jffs_foreach_file(c, jffs_file_count) * 2 + 50))
  return 1;

I suggest to raise 25 to 50, to avoid silly problems like this.



--
Martin Gadbois
S/W designer
Colubris Networks (http://www.colubris.com)





To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org