Re: Swappiness vs. mmap() and interactive response

[Rik van Riel <riel@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 1434 bytes --]

KOSAKI Motohiro wrote:

>> Next, I set the following:
>>
>> echo 0 > /proc/sys/vm/swappiness
>>
>> ... hoping it would prevent paging out of the UI in favor of file data that's
>> only used once.  It did appear to help to a small degree, but not much.  The
>> system is still effectively unusable while a file copy is going on.
>>
>> From this, I diagnosed that most likely, the kernel was paging out all my
>> application file mmap() data (such as my executables and shared libraries) in
>> favor of total garbage VM load from the file copy.

I believe your analysis is correct.

When merging the split LRU code upstream, some code was changed
(for scalability reasons) that results in active file pages being
moved to the inactive list any time we evict inactive file pages.

Even if the active file pages are referenced, they are not
protected from the streaming IO.

However, the use-once policy in the VM depends on the active
pages being protected from streaming IO.

A little before the decision to no longer honor the referenced
bit on active file pages was made, we dropped an ugly patch (by
me) after deciding it was just too much of a hack.  However, now
that we have _no_ protection for active file pages against large
amounts of streaming IO, we may want to reinstate something like
it.  Hopefully in a prettier way...

The old patch is attached for inspiration, discussion and maybe
testing :)

-- 
All rights reversed.

[-- Attachment #2: evict-cache-first.patch --]
[-- Type: text/plain, Size: 2986 bytes --]

When there is a lot of streaming IO going on, we do not want
to scan or evict pages from the working set.  The old VM used
to skip any mapped page, but still evict indirect blocks and
other data that is useful to cache.

This patch adds logic to skip scanning the anon lists and
the active file list if most of the file pages are on the
inactive file list (where streaming IO pages live), while
at the lowest scanning priority.

If the system is not doing a lot of streaming IO, eg. the
system is running a database workload, then more often used
file pages will be on the active file list and this logic
is automatically disabled.

Signed-off-by: Rik van Riel <riel@redhat.com>
---
 include/linux/mmzone.h |    1 +
 mm/vmscan.c            |   18 ++++++++++++++++--
 2 files changed, 17 insertions(+), 2 deletions(-)

Index: linux-2.6.26-rc8-mm1/include/linux/mmzone.h
===================================================================
--- linux-2.6.26-rc8-mm1.orig/include/linux/mmzone.h	2008-07-07 15:41:32.000000000 -0400
+++ linux-2.6.26-rc8-mm1/include/linux/mmzone.h	2008-07-15 14:58:50.000000000 -0400
@@ -453,6 +453,7 @@ static inline int zone_is_oom_locked(con
  * queues ("queue_length >> 12") during an aging round.
  */
 #define DEF_PRIORITY 12
+#define PRIO_CACHE_ONLY DEF_PRIORITY+1
 
 /* Maximum number of zones on a zonelist */
 #define MAX_ZONES_PER_ZONELIST (MAX_NUMNODES * MAX_NR_ZONES)
Index: linux-2.6.26-rc8-mm1/mm/vmscan.c
===================================================================
--- linux-2.6.26-rc8-mm1.orig/mm/vmscan.c	2008-07-07 15:41:33.000000000 -0400
+++ linux-2.6.26-rc8-mm1/mm/vmscan.c	2008-07-15 15:10:05.000000000 -0400
@@ -1481,6 +1481,20 @@ static unsigned long shrink_zone(int pri
 		}
 	}
 
+	/*
+	 * If there is a lot of sequential IO going on, most of the
+	 * file pages will be on the inactive file list.  We start
+	 * out by reclaiming those pages, without putting pressure on
+	 * the working set.  We only do this if the bulk of the file pages
+	 * are not in the working set (on the active file list).
+	 */
+	if (priority == PRIO_CACHE_ONLY &&
+			(nr[LRU_INACTIVE_FILE] > nr[LRU_ACTIVE_FILE]))
+		for_each_evictable_lru(l)
+			/* Scan only the inactive_file list. */
+			if (l != LRU_INACTIVE_FILE)
+				nr[l] = 0;
+
 	while (nr[LRU_INACTIVE_ANON] || nr[LRU_ACTIVE_FILE] ||
 					nr[LRU_INACTIVE_FILE]) {
 		for_each_evictable_lru(l) {
@@ -1609,7 +1623,7 @@ static unsigned long do_try_to_free_page
 		}
 	}
 
-	for (priority = DEF_PRIORITY; priority >= 0; priority--) {
+	for (priority = PRIO_CACHE_ONLY; priority >= 0; priority--) {
 		sc->nr_scanned = 0;
 		if (!priority)
 			disable_swap_token();
@@ -1771,7 +1785,7 @@ loop_again:
 	for (i = 0; i < pgdat->nr_zones; i++)
 		temp_priority[i] = DEF_PRIORITY;
 
-	for (priority = DEF_PRIORITY; priority >= 0; priority--) {
+	for (priority = PRIO_CACHE_ONLY; priority >= 0; priority--) {
 		int end_zone = 0;	/* Inclusive.  0 = ZONE_DMA */
 		unsigned long lru_pages = 0;