Re: Limit hash table size

Nick Piggin
2004-02-06T03:24:57Z

Andrew Morton wrote:

>Steve Lord <lord@xfs.org> wrote:
>
>> I have seen some dire cases with the dcache, SGI had some boxes with
>> millions of files out there, and every night a cron job would come
>> along and suck them all into memory. Resources got tight at some point,
>> and as more inodes and dentries were being read in, the try to free
>> pages path was continually getting called. There was always something
>> in filesystem cache which could get freed, and the inodes and dentries
>> kept getting more and more of the memory.
>>
>
>There are a number of variables here. Certainly, the old
>inodes-pinned-by-highmem pagecache will cause this to happen - badly. 2.6
>is pretty aggressive at killing off those inodes.
>
>What kernel was it?
>
>Was it a highmem box? If so, was the filesystem in question placing
>directory pagecache in highmem? If so, that was really bad on older 2.4:
>the directory pagecache in highmem pins down all directory inodes.
>
>

2.6.2-mm1 should fix this I think.
In particular, this hunk in vm-shrink-zone.patch

@@ -918,6 +917,15 @@ int try_to_free_pages(struct zone **zone
get_page_state(&ps);
nr_reclaimed += shrink_caches(zones, priority, &total_scanned,
gfp_mask, nr_pages, &ps);
+
+ if (zones[0] - zones[0]->zone_pgdat->node_zones < ZONE_HIGHMEM) {
+ shrink_slab(total_scanned, gfp_mask);
+ if (reclaim_state) {
+ nr_reclaimed += reclaim_state->reclaimed_slab;
+ reclaim_state->reclaimed_slab = 0;
+ }
+ }
+
if (nr_reclaimed >= nr_pages) {
ret = 1;
goto out;
@@ -933,13 +941,6 @@ int try_to_free_pages(struct zone **zone

/* Take a nap, wait for some writeback to complete */
blk_congestion_wait(WRITE, HZ/10);
- if (zones[0] - zones[0]->zone_pgdat->node_zones < ZONE_HIGHMEM) {
- shrink_slab(total_scanned, gfp_mask);
- if (reclaim_state) {
- nr_reclaimed += reclaim_state->reclaimed_slab;
- reclaim_state->reclaimed_slab = 0;
- }
- }
}
if ((gfp_mask & __GFP_FS) && !(gfp_mask & __GFP_NORETRY))
out_of_memory();