* [PATCH] Autoregulate vm swappiness 2.6.0-test8
@ 2003-10-23 13:37 Con Kolivas
2003-10-23 14:42 ` Martin J. Bligh
0 siblings, 1 reply; 9+ messages in thread
From: Con Kolivas @ 2003-10-23 13:37 UTC (permalink / raw)
To: linux kernel mailing list; +Cc: Andrew Morton
[-- Attachment #1: Type: text/plain, Size: 1853 bytes --]
The vm_swappiness dial in 2.6 was never quite the right setting without me
constantly changing it depending on the workload. If I was copying large
files or encoding video it was best at 0. If I was using lots of applications
it was best much higher. Furthermore it depended on the amount of ram in the
machine I was using. This patch was done just for fun a while back but it
turned out to be quite effectual so I thought I'd make it available for the
wider community to play with. Do whatever you like with it.
This patch autoregulates the vm_swappiness dial in 2.6 by making it equal to
the percentage of physical ram consumed by application pages.
This has the effect of preventing applications from being swapped out if the
ram is filling up with cached data.
Conversely, if many applications are in ram the swappiness increases which
means the application currently in use gets to stay in physical ram while
other less used applications are swapped out.
For desktop enthusiasts this means if you are copying large files around like
ISO images or leave your machine unattended for a while it will not swap out
your applications. Conversely if the machine has a lot of applications
currently loaded it will give the currently running applications preference
and swap out the less used ones.
The performance effect on larger boxes seems to be either unchanged or slight
improvement (1%) in database benchmarks.
The value in vm_swappiness is updated only when the vm is under pressure to
swap and you can check the last vm_swappiness value under pressure by
cat /proc/sys/vm/swappiness
Manually setting the swappiness with this patch in situ has no effect. This
patch has been heavily tested without noticable harm. Note I am not sure of
the best way to do this so it may look rather crude.
Patch against 2.6.0-test8
Con
[-- Attachment #2: patch-test8-am --]
[-- Type: text/x-diff, Size: 1420 bytes --]
--- linux-2.6.0-test8-base/mm/vmscan.c 2003-10-19 20:24:36.000000000 +1000
+++ linux-2.6.0-test8-am/mm/vmscan.c 2003-10-22 17:56:18.501329888 +1000
@@ -47,7 +47,7 @@
/*
* From 0 .. 100. Higher means more swappy.
*/
-int vm_swappiness = 60;
+int vm_swappiness = 0;
static long total_memory;
#ifdef ARCH_HAS_PREFETCH
@@ -595,11 +595,13 @@ refill_inactive_zone(struct zone *zone,
int pgmoved;
int pgdeactivate = 0;
int nr_pages = nr_pages_in;
+ int pg_size;
LIST_HEAD(l_hold); /* The pages which were snipped off */
LIST_HEAD(l_inactive); /* Pages to go onto the inactive_list */
LIST_HEAD(l_active); /* Pages to go onto the active_list */
struct page *page;
struct pagevec pvec;
+ struct sysinfo i;
int reclaim_mapped = 0;
long mapped_ratio;
long distress;
@@ -642,6 +644,16 @@ refill_inactive_zone(struct zone *zone,
mapped_ratio = (ps->nr_mapped * 100) / total_memory;
/*
+ * Autoregulate vm_swappiness to be application pages % -ck.
+ */
+ si_meminfo(&i);
+ si_swapinfo(&i);
+ pg_size = get_page_cache_size() - i.bufferram ;
+ vm_swappiness = 100 - (((i.freeram + i.bufferram +
+ (pg_size - swapper_space.nrpages)) * 100) /
+ (i.totalram ? i.totalram : 1));
+
+ /*
* Now decide how much we really want to unmap some pages. The mapped
* ratio is downgraded - just because there's a lot of mapped memory
* doesn't necessarily mean that page reclaim isn't succeeding.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness 2.6.0-test8
2003-10-23 13:37 [PATCH] Autoregulate vm swappiness 2.6.0-test8 Con Kolivas
@ 2003-10-23 14:42 ` Martin J. Bligh
2003-10-23 15:03 ` Con Kolivas
0 siblings, 1 reply; 9+ messages in thread
From: Martin J. Bligh @ 2003-10-23 14:42 UTC (permalink / raw)
To: Con Kolivas, linux kernel mailing list; +Cc: Andrew Morton
> + * Autoregulate vm_swappiness to be application pages % -ck.
> + */
> + si_meminfo(&i);
> + si_swapinfo(&i);
> + pg_size = get_page_cache_size() - i.bufferram ;
> + vm_swappiness = 100 - (((i.freeram + i.bufferram +
> + (pg_size - swapper_space.nrpages)) * 100) /
> + (i.totalram ? i.totalram : 1));
> +
> + /*
It seems that you don't need si_swapinfo here, do you? i.freeram,
i.bufferram, and i.totalram all come from meminfo, as far as I can
see? Maybe I'm missing a bit ...
M.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness 2.6.0-test8
2003-10-23 14:42 ` Martin J. Bligh
@ 2003-10-23 15:03 ` Con Kolivas
2003-10-25 6:58 ` [PATCH] Autoregulate vm swappiness cleanup Con Kolivas
0 siblings, 1 reply; 9+ messages in thread
From: Con Kolivas @ 2003-10-23 15:03 UTC (permalink / raw)
To: Martin J. Bligh, linux kernel mailing list; +Cc: Andrew Morton
[-- Attachment #1: Type: text/plain, Size: 727 bytes --]
On Friday 24 October 2003 00:42, Martin J. Bligh wrote:
> > + * Autoregulate vm_swappiness to be application pages % -ck.
> > + */
> > + si_meminfo(&i);
> > + si_swapinfo(&i);
> > + pg_size = get_page_cache_size() - i.bufferram ;
> > + vm_swappiness = 100 - (((i.freeram + i.bufferram +
> > + (pg_size - swapper_space.nrpages)) * 100) /
> > + (i.totalram ? i.totalram : 1));
> > +
> > + /*
>
> It seems that you don't need si_swapinfo here, do you? i.freeram,
> i.bufferram, and i.totalram all come from meminfo, as far as I can
> see? Maybe I'm missing a bit ...
Well I did do it a while ago and it seems I got carried away adding and
subtracting info indeed. :-) Here's a simpler patch that does the same thing.
Con
[-- Attachment #2: patch-test8-am-2 --]
[-- Type: text/x-diff, Size: 1132 bytes --]
--- linux-2.6.0-test8-base/mm/vmscan.c 2003-10-19 20:24:36.000000000 +1000
+++ linux-2.6.0-test8-am/mm/vmscan.c 2003-10-24 00:46:52.000000000 +1000
@@ -47,7 +47,7 @@
/*
* From 0 .. 100. Higher means more swappy.
*/
-int vm_swappiness = 60;
+int vm_swappiness = 0;
static long total_memory;
#ifdef ARCH_HAS_PREFETCH
@@ -600,6 +600,7 @@ refill_inactive_zone(struct zone *zone,
LIST_HEAD(l_active); /* Pages to go onto the active_list */
struct page *page;
struct pagevec pvec;
+ struct sysinfo i;
int reclaim_mapped = 0;
long mapped_ratio;
long distress;
@@ -642,6 +643,13 @@ refill_inactive_zone(struct zone *zone,
mapped_ratio = (ps->nr_mapped * 100) / total_memory;
/*
+ * Autoregulate vm_swappiness to be application pages% -ck
+ */
+ si_meminfo(&i);
+ vm_swappiness = 100 - (((i.freeram + get_page_cache_size() -
+ swapper_space.nrpages) * 100) / (i.totalram ? i.totalram : 1));
+
+ /*
* Now decide how much we really want to unmap some pages. The mapped
* ratio is downgraded - just because there's a lot of mapped memory
* doesn't necessarily mean that page reclaim isn't succeeding.
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH] Autoregulate vm swappiness cleanup
2003-10-23 15:03 ` Con Kolivas
@ 2003-10-25 6:58 ` Con Kolivas
2003-10-26 11:22 ` Nick Piggin
2003-10-28 11:04 ` Pavel Machek
0 siblings, 2 replies; 9+ messages in thread
From: Con Kolivas @ 2003-10-25 6:58 UTC (permalink / raw)
To: Martin J. Bligh, linux kernel mailing list; +Cc: Andrew Morton
[-- Attachment #1: Type: text/plain, Size: 894 bytes --]
On Fri, 24 Oct 2003 01:03, Con Kolivas wrote:
> On Friday 24 October 2003 00:42, Martin J. Bligh wrote:
> > It seems that you don't need si_swapinfo here, do you? i.freeram,
> > i.bufferram, and i.totalram all come from meminfo, as far as I can
> > see? Maybe I'm missing a bit ...
>
> Well I did do it a while ago and it seems I got carried away adding and
> subtracting info indeed. :-) Here's a simpler patch that does the same
> thing.
The off-list enthusiasm has been rather strong so here is a patch done the
right way (tm). There is no need for the check of totalram being zero (the
original version of this patch modified the swappiness every tick which was
wasteful and had a divide by zero on init). Adjusting vm_swappiness only when
there is pressure to swap means totalram shouldn't be ever be zero. The
sysctl is made read only since writing to it would be ignored now.
Con
[-- Attachment #2: patch-test8-am-3 --]
[-- Type: text/x-diff, Size: 1725 bytes --]
--- linux-2.6.0-test8-base/kernel/sysctl.c 2003-10-19 20:24:49.000000000 +1000
+++ linux-2.6.0-test8-am/kernel/sysctl.c 2003-10-25 16:37:44.384824976 +1000
@@ -664,11 +664,8 @@ static ctl_table vm_table[] = {
.procname = "swappiness",
.data = &vm_swappiness,
.maxlen = sizeof(vm_swappiness),
- .mode = 0644,
- .proc_handler = &proc_dointvec_minmax,
- .strategy = &sysctl_intvec,
- .extra1 = &zero,
- .extra2 = &one_hundred,
+ .mode = 0444 /* read-only*/,
+ .proc_handler = &proc_dointvec,
},
#ifdef CONFIG_HUGETLB_PAGE
{
--- linux-2.6.0-test8-base/mm/vmscan.c 2003-10-19 20:24:36.000000000 +1000
+++ linux-2.6.0-test8-am/mm/vmscan.c 2003-10-25 16:40:33.099176496 +1000
@@ -47,7 +47,7 @@
/*
* From 0 .. 100. Higher means more swappy.
*/
-int vm_swappiness = 60;
+int vm_swappiness = 0;
static long total_memory;
#ifdef ARCH_HAS_PREFETCH
@@ -600,6 +600,7 @@ refill_inactive_zone(struct zone *zone,
LIST_HEAD(l_active); /* Pages to go onto the active_list */
struct page *page;
struct pagevec pvec;
+ struct sysinfo i;
int reclaim_mapped = 0;
long mapped_ratio;
long distress;
@@ -642,6 +643,14 @@ refill_inactive_zone(struct zone *zone,
mapped_ratio = (ps->nr_mapped * 100) / total_memory;
/*
+ * Autoregulate vm_swappiness to be equal to the percentage of
+ * pages in physical ram that are application pages. -ck
+ */
+ si_meminfo(&i);
+ vm_swappiness = 100 - (((i.freeram + get_page_cache_size() -
+ swapper_space.nrpages) * 100) / i.totalram);
+
+ /*
* Now decide how much we really want to unmap some pages. The mapped
* ratio is downgraded - just because there's a lot of mapped memory
* doesn't necessarily mean that page reclaim isn't succeeding.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness cleanup
2003-10-26 11:22 ` Nick Piggin
@ 2003-10-26 10:36 ` Con Kolivas
2003-10-26 11:42 ` Nick Piggin
0 siblings, 1 reply; 9+ messages in thread
From: Con Kolivas @ 2003-10-26 10:36 UTC (permalink / raw)
To: Nick Piggin
Cc: Con Kolivas, Martin J. Bligh, linux kernel mailing list,
Andrew Morton
Nick Piggin wrote:
>
> Hi Con,
> If this indeed makes VM behaviour better, why not just merge the
> calculation
> with the swap_tendancy calculation and leave vm_swappiness there as a
> tunable?
Because the whole point of it is to remove the tunable and make it auto
tuning. We could do away with the vm_swappiness variable altogether too
(which I would actually prefer to do) but this leaves it intact to see
what the vm is doing.
Con
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness cleanup
2003-10-25 6:58 ` [PATCH] Autoregulate vm swappiness cleanup Con Kolivas
@ 2003-10-26 11:22 ` Nick Piggin
2003-10-26 10:36 ` Con Kolivas
2003-10-28 11:04 ` Pavel Machek
1 sibling, 1 reply; 9+ messages in thread
From: Nick Piggin @ 2003-10-26 11:22 UTC (permalink / raw)
To: Con Kolivas; +Cc: Martin J. Bligh, linux kernel mailing list, Andrew Morton
Con Kolivas wrote:
>On Fri, 24 Oct 2003 01:03, Con Kolivas wrote:
>
>>On Friday 24 October 2003 00:42, Martin J. Bligh wrote:
>>
>>>It seems that you don't need si_swapinfo here, do you? i.freeram,
>>>i.bufferram, and i.totalram all come from meminfo, as far as I can
>>>see? Maybe I'm missing a bit ...
>>>
>>Well I did do it a while ago and it seems I got carried away adding and
>>subtracting info indeed. :-) Here's a simpler patch that does the same
>>thing.
>>
>
>The off-list enthusiasm has been rather strong so here is a patch done the
>right way (tm). There is no need for the check of totalram being zero (the
>original version of this patch modified the swappiness every tick which was
>wasteful and had a divide by zero on init). Adjusting vm_swappiness only when
>there is pressure to swap means totalram shouldn't be ever be zero. The
>sysctl is made read only since writing to it would be ignored now.
>
>Con
>
>
>
>------------------------------------------------------------------------
>
>--- linux-2.6.0-test8-base/kernel/sysctl.c 2003-10-19 20:24:49.000000000 +1000
>+++ linux-2.6.0-test8-am/kernel/sysctl.c 2003-10-25 16:37:44.384824976 +1000
>@@ -664,11 +664,8 @@ static ctl_table vm_table[] = {
> .procname = "swappiness",
> .data = &vm_swappiness,
> .maxlen = sizeof(vm_swappiness),
>- .mode = 0644,
>- .proc_handler = &proc_dointvec_minmax,
>- .strategy = &sysctl_intvec,
>- .extra1 = &zero,
>- .extra2 = &one_hundred,
>+ .mode = 0444 /* read-only*/,
>+ .proc_handler = &proc_dointvec,
> },
> #ifdef CONFIG_HUGETLB_PAGE
> {
>--- linux-2.6.0-test8-base/mm/vmscan.c 2003-10-19 20:24:36.000000000 +1000
>+++ linux-2.6.0-test8-am/mm/vmscan.c 2003-10-25 16:40:33.099176496 +1000
>@@ -47,7 +47,7 @@
> /*
> * From 0 .. 100. Higher means more swappy.
> */
>-int vm_swappiness = 60;
>+int vm_swappiness = 0;
> static long total_memory;
>
> #ifdef ARCH_HAS_PREFETCH
>@@ -600,6 +600,7 @@ refill_inactive_zone(struct zone *zone,
> LIST_HEAD(l_active); /* Pages to go onto the active_list */
> struct page *page;
> struct pagevec pvec;
>+ struct sysinfo i;
> int reclaim_mapped = 0;
> long mapped_ratio;
> long distress;
>@@ -642,6 +643,14 @@ refill_inactive_zone(struct zone *zone,
> mapped_ratio = (ps->nr_mapped * 100) / total_memory;
>
> /*
>+ * Autoregulate vm_swappiness to be equal to the percentage of
>+ * pages in physical ram that are application pages. -ck
>+ */
>+ si_meminfo(&i);
>+ vm_swappiness = 100 - (((i.freeram + get_page_cache_size() -
>+ swapper_space.nrpages) * 100) / i.totalram);
>+
>+ /*
> * Now decide how much we really want to unmap some pages. The mapped
> * ratio is downgraded - just because there's a lot of mapped memory
> * doesn't necessarily mean that page reclaim isn't succeeding.
>
Hi Con,
If this indeed makes VM behaviour better, why not just merge the calculation
with the swap_tendancy calculation and leave vm_swappiness there as a
tunable?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness cleanup
2003-10-26 10:36 ` Con Kolivas
@ 2003-10-26 11:42 ` Nick Piggin
0 siblings, 0 replies; 9+ messages in thread
From: Nick Piggin @ 2003-10-26 11:42 UTC (permalink / raw)
To: Con Kolivas
Cc: Con Kolivas, Martin J. Bligh, linux kernel mailing list,
Andrew Morton
Con Kolivas wrote:
> Nick Piggin wrote:
>
>>
>> Hi Con,
>> If this indeed makes VM behaviour better, why not just merge the
>> calculation
>> with the swap_tendancy calculation and leave vm_swappiness there as a
>> tunable?
>
>
> Because the whole point of it is to remove the tunable and make it
> auto tuning. We could do away with the vm_swappiness variable
> altogether too (which I would actually prefer to do) but this leaves
> it intact to see what the vm is doing.
Right. This just had me a bit confused. No worries.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness cleanup
2003-10-25 6:58 ` [PATCH] Autoregulate vm swappiness cleanup Con Kolivas
2003-10-26 11:22 ` Nick Piggin
@ 2003-10-28 11:04 ` Pavel Machek
2003-10-28 12:40 ` Con Kolivas
1 sibling, 1 reply; 9+ messages in thread
From: Pavel Machek @ 2003-10-28 11:04 UTC (permalink / raw)
To: Con Kolivas; +Cc: Martin J. Bligh, linux kernel mailing list, Andrew Morton
Hi!
> > > It seems that you don't need si_swapinfo here, do you? i.freeram,
> > > i.bufferram, and i.totalram all come from meminfo, as far as I can
> > > see? Maybe I'm missing a bit ...
> >
> > Well I did do it a while ago and it seems I got carried away adding and
> > subtracting info indeed. :-) Here's a simpler patch that does the same
> > thing.
>
> The off-list enthusiasm has been rather strong so here is a patch done the
> right way (tm). There is no need for the check of totalram being zero (the
> original version of this patch modified the swappiness every tick which was
> wasteful and had a divide by zero on init). Adjusting vm_swappiness only when
> there is pressure to swap means totalram shouldn't be ever be zero. The
> sysctl is made read only since writing to it would be ignored now.
I believe swappiness == 100 was "I want max throughput, I don't care
about latency going through roof", while swappiness == 0 was "I don't
want you to swap too much, behave reasonably".
As you don't know if user cares about latency or not, I don't see how
you can autotune this.
--
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness cleanup
2003-10-28 11:04 ` Pavel Machek
@ 2003-10-28 12:40 ` Con Kolivas
0 siblings, 0 replies; 9+ messages in thread
From: Con Kolivas @ 2003-10-28 12:40 UTC (permalink / raw)
To: Pavel Machek; +Cc: Martin J. Bligh, linux kernel mailing list, Andrew Morton
On Tue, 28 Oct 2003 22:04, Pavel Machek wrote:
> Hi!
Hello!
> I believe swappiness == 100 was "I want max throughput, I don't care
> about latency going through roof", while swappiness == 0 was "I don't
> want you to swap too much, behave reasonably".
>
> As you don't know if user cares about latency or not, I don't see how
> you can autotune this.
Well I guess you either see merit in what my patch does based on what I said,
or you don't... so I guess you don't. That's fine; I just offered why I felt
this helped in my varied workloads more than a static value did.
Con
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2003-10-28 12:40 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2003-10-23 13:37 [PATCH] Autoregulate vm swappiness 2.6.0-test8 Con Kolivas
2003-10-23 14:42 ` Martin J. Bligh
2003-10-23 15:03 ` Con Kolivas
2003-10-25 6:58 ` [PATCH] Autoregulate vm swappiness cleanup Con Kolivas
2003-10-26 11:22 ` Nick Piggin
2003-10-26 10:36 ` Con Kolivas
2003-10-26 11:42 ` Nick Piggin
2003-10-28 11:04 ` Pavel Machek
2003-10-28 12:40 ` Con Kolivas
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).