2.4.10 swap behaviour

2.4.10 swap behaviour

[Osma Ahvenlampi]

Cheers,

Yesterday, after upgrading to 2.4.10 and deciding it was significantly
better about memory usage under light load than 2.4.9, I ran a
pathological test case with rather interesting results..

My machine is a 900MHz P3 laptop with 512MB of memory (just upgraded
from 256M - most of my experience with earlier 2.4 kernels up to 2.4.9
is with 256M, which adds some variability into the subjective
experience..). I ran these tests with a vanilla 2.4.9 kernel and 2.4.10
patched with Robert Love's 2.4.10-preempt patches.

Under both kernels, I ran a simulated medium-load session (Ximian Gnome,
Evolution, a Java server, XEmacs and a 128MB VMware session open but not
particularly active, no net connection), and then started a cat /dev/dvd
>/dev/null process, watching the memory usage pattern with vmstat and
free. Before the cat, I had about 190 megs of memory used (disregarding
buffers/cache) - the VMware session had not allocated all of it's 128MB
limit, remaining around 80 megs at the point.

2.4.9 would gradually increase its buffer and cache memory until were
around 150-200 megs. At this point, a lot of the applications had been
swapped out, with 60 megs of swap in use, and the system was very
sluggish. I aborted the cat with a ctrl-c and the system regained
responsiveness fairly quickly (paging heavily until most code was back
in).

2.4.10 remained more responsive in the beginning, and I noticed that the
buffer allocation would actually drop from the initial 15 megs to under
5. However, the cache usage kept on going up, until about 440 megs of
RAM was used by the cache (and less than a meg in buffers!). At this
point, about 90 megs of swap had been allocated, and the machine quite
quickly lost all response. The mouse pointer would not move, the active
terminal window showed no response to keyboard, I could not switch to
another VC. Except for the constant disk access, the machine might as
well been completely hung, and for a moment I thought it was. Eventually
I was forced to eject the entire DVD drive from the machine (good thing
I was testing on a laptop! :)), at which point the cat process obviously
would be aborted. After that, it took about 20-30 seconds until the
terminal started to respond (slowly), and a couple of minutes before the
all the applications were again usable (after heavy paging again,
obviously).

Conclusion: 2.4.10 buffer behaviour is dramatically different, and the
system remains responsive somewhat longer under medium load - some of
that might be attributed to the preempt patches, however. Under heavy
sequential IO load, the cache still has a pathological behaviour of
forcing everything else out of memory, which can make the system act as
if completely dead, unless the IO process can be aborted. The improved
behaviour under moderate load makes the transition from "paging but
decent" to "completely unusable" somewhat delayed but more sudden when
it does happen.

let me know if you want me to test something different..

-- 
Osma Ahvenlampi <oa@iki.fi>

Re: 2.4.10 swap behaviour (with vm-tweaks-2)

[Andrea Arcangeli]

On Wed, Sep 26, 2001 at 10:41:48AM +0300, Osma Ahvenlampi wrote:
> quickly lost all response. The mouse pointer would not move, the active

You really want to apply vm-tweaks-1, (or better vm-tweaks-2 inlined
here with the bugfix for Cary's div by zero, woops) and try again. It
applies cleanly to vanilla 2.4.10.

Linus I'd suggest for inclusion, I know you hate it but think at least
at the huge benefit in light vm-pressure load when just heavy non-mapped
cache pollution is going on, even if you disagree about the
pageout-trigger condition (but swapout behaviour is much better too this
way, and I disagree in constantly wasting cpu to know when to swap
anyways).

(btw, after this patch we can also drop the "survive" cruft into the
page fault handler, not included here since not very important cleanup)

diff -urN 2.4.10/include/linux/fs.h vm/include/linux/fs.h
--- 2.4.10/include/linux/fs.h	Sun Sep 23 21:11:42 2001
+++ vm/include/linux/fs.h	Wed Sep 26 15:34:35 2001
@@ -282,7 +282,7 @@
 
 extern void set_bh_page(struct buffer_head *bh, struct page *page, unsigned long offset);
 
-#define touch_buffer(bh)	SetPageReferenced(bh->b_page)
+#define touch_buffer(bh)	mark_page_accessed(bh->b_page)
 
 
 #include <linux/pipe_fs_i.h>
diff -urN 2.4.10/mm/filemap.c vm/mm/filemap.c
--- 2.4.10/mm/filemap.c	Sun Sep 23 21:11:43 2001
+++ vm/mm/filemap.c	Wed Sep 26 15:34:35 2001
@@ -1704,7 +1704,7 @@
 	 * and possibly copy it over to another page..
 	 */
 	old_page = page;
-	mark_page_accessed(page);
+	activate_page(page);
 	if (no_share) {
 		struct page *new_page = alloc_page(GFP_HIGHUSER);
 
diff -urN 2.4.10/mm/swap.c vm/mm/swap.c
--- 2.4.10/mm/swap.c	Sun Sep 23 21:11:43 2001
+++ vm/mm/swap.c	Wed Sep 26 15:34:35 2001
@@ -54,6 +54,7 @@
 		del_page_from_active_list(page);
 		add_page_to_inactive_list(page);
 	}
+	SetPageReferenced(page);
 }	
 
 void deactivate_page(struct page * page)
@@ -72,6 +73,7 @@
 		del_page_from_inactive_list(page);
 		add_page_to_active_list(page);
 	}
+	ClearPageReferenced(page);
 }
 
 void activate_page(struct page * page)
diff -urN 2.4.10/mm/vmscan.c vm/mm/vmscan.c
--- 2.4.10/mm/vmscan.c	Sun Sep 23 21:11:43 2001
+++ vm/mm/vmscan.c	Wed Sep 26 15:35:03 2001
@@ -55,6 +55,9 @@
 		return 0;
 	}
 
+	if (PageActive(page) || (PageInactive(page) && PageReferenced(page)))
+		return 0;
+
 	if (TryLockPage(page))
 		return 0;
 
@@ -329,7 +332,6 @@
 {
 	struct list_head * entry;
 
-	spin_lock(&pagemap_lru_lock);
 	while (max_scan && (entry = inactive_list.prev) != &inactive_list) {
 		struct page * page;
 		swp_entry_t swap;
@@ -358,8 +360,10 @@
 			continue;
 
 		/* Racy check to avoid trylocking when not worthwhile */
-		if (!page->buffers && page_count(page) != 1)
+		if (!page->buffers && page_count(page) != 1) {
+			activate_page_nolock(page);
 			continue;
+		}
 
 		/*
 		 * The page is locked. IO in progress?
@@ -514,7 +518,6 @@
 {
 	struct list_head * entry;
 
-	spin_lock(&pagemap_lru_lock);
 	entry = active_list.prev;
 	while (nr_pages-- && entry != &active_list) {
 		struct page * page;
@@ -529,23 +532,28 @@
 
 		del_page_from_active_list(page);
 		add_page_to_inactive_list(page);
+		SetPageReferenced(page);
 	}
-	spin_unlock(&pagemap_lru_lock);
 }
 
 static int FASTCALL(shrink_caches(int priority, zone_t * classzone, unsigned int gfp_mask, int nr_pages));
 static int shrink_caches(int priority, zone_t * classzone, unsigned int gfp_mask, int nr_pages)
 {
-	int max_scan = nr_inactive_pages / priority;
+	int max_scan;
+	int chunk_size = nr_pages;
+	unsigned long ratio;
 
 	nr_pages -= kmem_cache_reap(gfp_mask);
 	if (nr_pages <= 0)
 		return 0;
 
-	/* Do we want to age the active list? */
-	if (nr_inactive_pages < nr_active_pages*2)
-		refill_inactive(nr_pages);
+	spin_lock(&pagemap_lru_lock);
+	nr_pages = chunk_size;
+	/* try to keep the active list 2/3 of the size of the cache */
+	ratio = (unsigned long) nr_pages * nr_active_pages / ((nr_inactive_pages + 1) * 2);
+	refill_inactive(ratio);
 
+	max_scan = nr_inactive_pages / priority;
 	nr_pages = shrink_cache(nr_pages, max_scan, classzone, gfp_mask);
 	if (nr_pages <= 0)
 		return 0;
@@ -558,17 +566,28 @@
 
 int try_to_free_pages(zone_t * classzone, unsigned int gfp_mask, unsigned int order)
 {
-	int priority = DEF_PRIORITY;
-	int ret = 0;
+	int priority, nr_pages, ret = 0;
 
-	do {
-		int nr_pages = SWAP_CLUSTER_MAX;
-		nr_pages = shrink_caches(priority, classzone, gfp_mask, nr_pages);
-		if (nr_pages <= 0)
-			return 1;
+	for (;;) {
+		priority = DEF_PRIORITY;
+		nr_pages = SWAP_CLUSTER_MAX;
+
+		do {
+			nr_pages = shrink_caches(priority, classzone, gfp_mask, nr_pages);
+			if (nr_pages <= 0)
+				return 1;
+
+			ret |= swap_out(priority, classzone, gfp_mask, SWAP_CLUSTER_MAX << 2);
+		} while (--priority);
 
-		ret |= swap_out(priority, classzone, gfp_mask, SWAP_CLUSTER_MAX << 2);
-	} while (--priority);
+		if (likely(ret))
+			break;
+		if (likely(current->pid != 1))
+			break;
+		current->policy |= SCHED_YIELD;
+		__set_current_state(TASK_RUNNING);
+		schedule();
+	}
 
 	return ret;
 }

Andrea

Re: 2.4.10 swap behaviour (with vm-tweaks-2)

[Osma Ahvenlampi]

Thanks Andrea,

after applying the patch, I repeated the experiment. Since the cat
/dev/dvd >/dev/null no longer really made any difference, I started at
the same time a dd if=/dev/hda2 of=/dev/null (a 6 GB partition) and
eight find / processes at slightly difference points in time (to
excercise the same disk blocks repeatedly). I let the whole thing run
for about 5 times longer than the original test, and while the system
still pushed a lot of programs onto swap (from an initial used memory of
190 megs, up to 80 was pushed onto swap while I kept "using" most of the
programs by clicking on buttons etc), the system remained fairly
responsive (a couple of individual apps became sluggish enough to
categorize as unresponsive, but a couple of other programs kept
responding almost as well as in a no-load condition). To top it off, I
tried to play some .ogg files with XMMS - there were breakups in the
output every 10-15 seconds, but other than that, XMMS worked fine as
well. I don't run XMMS with realtime priority, so that might have fixed
even the sound output problems.

I'd have to say I now (finally!) have a kernel which doesn't eat up all
my memory.. :) For the record, the setup now is 2.4.10 +
rml-preempt-kernel-1 + aa-vm-tweaks-2.

Cheers,

 Osma

On Wed, 2001-09-26 at 16:42, Andrea Arcangeli wrote:
> You really want to apply vm-tweaks-1, (or better vm-tweaks-2 inlined
> here with the bugfix for Cary's div by zero, woops) and try again. It
> applies cleanly to vanilla 2.4.10.

-- 
Osma Ahvenlampi <oa@iki.fi>