* [PATCH] Autoregulate vm swappiness 2.6.0-test8 @ 2003-10-23 13:37 Con Kolivas 2003-10-23 14:42 ` Martin J. Bligh 0 siblings, 1 reply; 9+ messages in thread From: Con Kolivas @ 2003-10-23 13:37 UTC (permalink / raw) To: linux kernel mailing list; +Cc: Andrew Morton [-- Attachment #1: Type: text/plain, Size: 1853 bytes --] The vm_swappiness dial in 2.6 was never quite the right setting without me constantly changing it depending on the workload. If I was copying large files or encoding video it was best at 0. If I was using lots of applications it was best much higher. Furthermore it depended on the amount of ram in the machine I was using. This patch was done just for fun a while back but it turned out to be quite effectual so I thought I'd make it available for the wider community to play with. Do whatever you like with it. This patch autoregulates the vm_swappiness dial in 2.6 by making it equal to the percentage of physical ram consumed by application pages. This has the effect of preventing applications from being swapped out if the ram is filling up with cached data. Conversely, if many applications are in ram the swappiness increases which means the application currently in use gets to stay in physical ram while other less used applications are swapped out. For desktop enthusiasts this means if you are copying large files around like ISO images or leave your machine unattended for a while it will not swap out your applications. Conversely if the machine has a lot of applications currently loaded it will give the currently running applications preference and swap out the less used ones. The performance effect on larger boxes seems to be either unchanged or slight improvement (1%) in database benchmarks. The value in vm_swappiness is updated only when the vm is under pressure to swap and you can check the last vm_swappiness value under pressure by cat /proc/sys/vm/swappiness Manually setting the swappiness with this patch in situ has no effect. This patch has been heavily tested without noticable harm. Note I am not sure of the best way to do this so it may look rather crude. Patch against 2.6.0-test8 Con [-- Attachment #2: patch-test8-am --] [-- Type: text/x-diff, Size: 1420 bytes --] --- linux-2.6.0-test8-base/mm/vmscan.c 2003-10-19 20:24:36.000000000 +1000 +++ linux-2.6.0-test8-am/mm/vmscan.c 2003-10-22 17:56:18.501329888 +1000 @@ -47,7 +47,7 @@ /* * From 0 .. 100. Higher means more swappy. */ -int vm_swappiness = 60; +int vm_swappiness = 0; static long total_memory; #ifdef ARCH_HAS_PREFETCH @@ -595,11 +595,13 @@ refill_inactive_zone(struct zone *zone, int pgmoved; int pgdeactivate = 0; int nr_pages = nr_pages_in; + int pg_size; LIST_HEAD(l_hold); /* The pages which were snipped off */ LIST_HEAD(l_inactive); /* Pages to go onto the inactive_list */ LIST_HEAD(l_active); /* Pages to go onto the active_list */ struct page *page; struct pagevec pvec; + struct sysinfo i; int reclaim_mapped = 0; long mapped_ratio; long distress; @@ -642,6 +644,16 @@ refill_inactive_zone(struct zone *zone, mapped_ratio = (ps->nr_mapped * 100) / total_memory; /* + * Autoregulate vm_swappiness to be application pages % -ck. + */ + si_meminfo(&i); + si_swapinfo(&i); + pg_size = get_page_cache_size() - i.bufferram ; + vm_swappiness = 100 - (((i.freeram + i.bufferram + + (pg_size - swapper_space.nrpages)) * 100) / + (i.totalram ? i.totalram : 1)); + + /* * Now decide how much we really want to unmap some pages. The mapped * ratio is downgraded - just because there's a lot of mapped memory * doesn't necessarily mean that page reclaim isn't succeeding. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness 2.6.0-test8 2003-10-23 13:37 [PATCH] Autoregulate vm swappiness 2.6.0-test8 Con Kolivas @ 2003-10-23 14:42 ` Martin J. Bligh 2003-10-23 15:03 ` Con Kolivas 0 siblings, 1 reply; 9+ messages in thread From: Martin J. Bligh @ 2003-10-23 14:42 UTC (permalink / raw) To: Con Kolivas, linux kernel mailing list; +Cc: Andrew Morton > + * Autoregulate vm_swappiness to be application pages % -ck. > + */ > + si_meminfo(&i); > + si_swapinfo(&i); > + pg_size = get_page_cache_size() - i.bufferram ; > + vm_swappiness = 100 - (((i.freeram + i.bufferram + > + (pg_size - swapper_space.nrpages)) * 100) / > + (i.totalram ? i.totalram : 1)); > + > + /* It seems that you don't need si_swapinfo here, do you? i.freeram, i.bufferram, and i.totalram all come from meminfo, as far as I can see? Maybe I'm missing a bit ... M. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness 2.6.0-test8 2003-10-23 14:42 ` Martin J. Bligh @ 2003-10-23 15:03 ` Con Kolivas 2003-10-25 6:58 ` [PATCH] Autoregulate vm swappiness cleanup Con Kolivas 0 siblings, 1 reply; 9+ messages in thread From: Con Kolivas @ 2003-10-23 15:03 UTC (permalink / raw) To: Martin J. Bligh, linux kernel mailing list; +Cc: Andrew Morton [-- Attachment #1: Type: text/plain, Size: 727 bytes --] On Friday 24 October 2003 00:42, Martin J. Bligh wrote: > > + * Autoregulate vm_swappiness to be application pages % -ck. > > + */ > > + si_meminfo(&i); > > + si_swapinfo(&i); > > + pg_size = get_page_cache_size() - i.bufferram ; > > + vm_swappiness = 100 - (((i.freeram + i.bufferram + > > + (pg_size - swapper_space.nrpages)) * 100) / > > + (i.totalram ? i.totalram : 1)); > > + > > + /* > > It seems that you don't need si_swapinfo here, do you? i.freeram, > i.bufferram, and i.totalram all come from meminfo, as far as I can > see? Maybe I'm missing a bit ... Well I did do it a while ago and it seems I got carried away adding and subtracting info indeed. :-) Here's a simpler patch that does the same thing. Con [-- Attachment #2: patch-test8-am-2 --] [-- Type: text/x-diff, Size: 1132 bytes --] --- linux-2.6.0-test8-base/mm/vmscan.c 2003-10-19 20:24:36.000000000 +1000 +++ linux-2.6.0-test8-am/mm/vmscan.c 2003-10-24 00:46:52.000000000 +1000 @@ -47,7 +47,7 @@ /* * From 0 .. 100. Higher means more swappy. */ -int vm_swappiness = 60; +int vm_swappiness = 0; static long total_memory; #ifdef ARCH_HAS_PREFETCH @@ -600,6 +600,7 @@ refill_inactive_zone(struct zone *zone, LIST_HEAD(l_active); /* Pages to go onto the active_list */ struct page *page; struct pagevec pvec; + struct sysinfo i; int reclaim_mapped = 0; long mapped_ratio; long distress; @@ -642,6 +643,13 @@ refill_inactive_zone(struct zone *zone, mapped_ratio = (ps->nr_mapped * 100) / total_memory; /* + * Autoregulate vm_swappiness to be application pages% -ck + */ + si_meminfo(&i); + vm_swappiness = 100 - (((i.freeram + get_page_cache_size() - + swapper_space.nrpages) * 100) / (i.totalram ? i.totalram : 1)); + + /* * Now decide how much we really want to unmap some pages. The mapped * ratio is downgraded - just because there's a lot of mapped memory * doesn't necessarily mean that page reclaim isn't succeeding. ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH] Autoregulate vm swappiness cleanup 2003-10-23 15:03 ` Con Kolivas @ 2003-10-25 6:58 ` Con Kolivas 2003-10-26 11:22 ` Nick Piggin 2003-10-28 11:04 ` Pavel Machek 0 siblings, 2 replies; 9+ messages in thread From: Con Kolivas @ 2003-10-25 6:58 UTC (permalink / raw) To: Martin J. Bligh, linux kernel mailing list; +Cc: Andrew Morton [-- Attachment #1: Type: text/plain, Size: 894 bytes --] On Fri, 24 Oct 2003 01:03, Con Kolivas wrote: > On Friday 24 October 2003 00:42, Martin J. Bligh wrote: > > It seems that you don't need si_swapinfo here, do you? i.freeram, > > i.bufferram, and i.totalram all come from meminfo, as far as I can > > see? Maybe I'm missing a bit ... > > Well I did do it a while ago and it seems I got carried away adding and > subtracting info indeed. :-) Here's a simpler patch that does the same > thing. The off-list enthusiasm has been rather strong so here is a patch done the right way (tm). There is no need for the check of totalram being zero (the original version of this patch modified the swappiness every tick which was wasteful and had a divide by zero on init). Adjusting vm_swappiness only when there is pressure to swap means totalram shouldn't be ever be zero. The sysctl is made read only since writing to it would be ignored now. Con [-- Attachment #2: patch-test8-am-3 --] [-- Type: text/x-diff, Size: 1725 bytes --] --- linux-2.6.0-test8-base/kernel/sysctl.c 2003-10-19 20:24:49.000000000 +1000 +++ linux-2.6.0-test8-am/kernel/sysctl.c 2003-10-25 16:37:44.384824976 +1000 @@ -664,11 +664,8 @@ static ctl_table vm_table[] = { .procname = "swappiness", .data = &vm_swappiness, .maxlen = sizeof(vm_swappiness), - .mode = 0644, - .proc_handler = &proc_dointvec_minmax, - .strategy = &sysctl_intvec, - .extra1 = &zero, - .extra2 = &one_hundred, + .mode = 0444 /* read-only*/, + .proc_handler = &proc_dointvec, }, #ifdef CONFIG_HUGETLB_PAGE { --- linux-2.6.0-test8-base/mm/vmscan.c 2003-10-19 20:24:36.000000000 +1000 +++ linux-2.6.0-test8-am/mm/vmscan.c 2003-10-25 16:40:33.099176496 +1000 @@ -47,7 +47,7 @@ /* * From 0 .. 100. Higher means more swappy. */ -int vm_swappiness = 60; +int vm_swappiness = 0; static long total_memory; #ifdef ARCH_HAS_PREFETCH @@ -600,6 +600,7 @@ refill_inactive_zone(struct zone *zone, LIST_HEAD(l_active); /* Pages to go onto the active_list */ struct page *page; struct pagevec pvec; + struct sysinfo i; int reclaim_mapped = 0; long mapped_ratio; long distress; @@ -642,6 +643,14 @@ refill_inactive_zone(struct zone *zone, mapped_ratio = (ps->nr_mapped * 100) / total_memory; /* + * Autoregulate vm_swappiness to be equal to the percentage of + * pages in physical ram that are application pages. -ck + */ + si_meminfo(&i); + vm_swappiness = 100 - (((i.freeram + get_page_cache_size() - + swapper_space.nrpages) * 100) / i.totalram); + + /* * Now decide how much we really want to unmap some pages. The mapped * ratio is downgraded - just because there's a lot of mapped memory * doesn't necessarily mean that page reclaim isn't succeeding. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness cleanup 2003-10-25 6:58 ` [PATCH] Autoregulate vm swappiness cleanup Con Kolivas @ 2003-10-26 11:22 ` Nick Piggin 2003-10-26 10:36 ` Con Kolivas 2003-10-28 11:04 ` Pavel Machek 1 sibling, 1 reply; 9+ messages in thread From: Nick Piggin @ 2003-10-26 11:22 UTC (permalink / raw) To: Con Kolivas; +Cc: Martin J. Bligh, linux kernel mailing list, Andrew Morton Con Kolivas wrote: >On Fri, 24 Oct 2003 01:03, Con Kolivas wrote: > >>On Friday 24 October 2003 00:42, Martin J. Bligh wrote: >> >>>It seems that you don't need si_swapinfo here, do you? i.freeram, >>>i.bufferram, and i.totalram all come from meminfo, as far as I can >>>see? Maybe I'm missing a bit ... >>> >>Well I did do it a while ago and it seems I got carried away adding and >>subtracting info indeed. :-) Here's a simpler patch that does the same >>thing. >> > >The off-list enthusiasm has been rather strong so here is a patch done the >right way (tm). There is no need for the check of totalram being zero (the >original version of this patch modified the swappiness every tick which was >wasteful and had a divide by zero on init). Adjusting vm_swappiness only when >there is pressure to swap means totalram shouldn't be ever be zero. The >sysctl is made read only since writing to it would be ignored now. > >Con > > > >------------------------------------------------------------------------ > >--- linux-2.6.0-test8-base/kernel/sysctl.c 2003-10-19 20:24:49.000000000 +1000 >+++ linux-2.6.0-test8-am/kernel/sysctl.c 2003-10-25 16:37:44.384824976 +1000 >@@ -664,11 +664,8 @@ static ctl_table vm_table[] = { > .procname = "swappiness", > .data = &vm_swappiness, > .maxlen = sizeof(vm_swappiness), >- .mode = 0644, >- .proc_handler = &proc_dointvec_minmax, >- .strategy = &sysctl_intvec, >- .extra1 = &zero, >- .extra2 = &one_hundred, >+ .mode = 0444 /* read-only*/, >+ .proc_handler = &proc_dointvec, > }, > #ifdef CONFIG_HUGETLB_PAGE > { >--- linux-2.6.0-test8-base/mm/vmscan.c 2003-10-19 20:24:36.000000000 +1000 >+++ linux-2.6.0-test8-am/mm/vmscan.c 2003-10-25 16:40:33.099176496 +1000 >@@ -47,7 +47,7 @@ > /* > * From 0 .. 100. Higher means more swappy. > */ >-int vm_swappiness = 60; >+int vm_swappiness = 0; > static long total_memory; > > #ifdef ARCH_HAS_PREFETCH >@@ -600,6 +600,7 @@ refill_inactive_zone(struct zone *zone, > LIST_HEAD(l_active); /* Pages to go onto the active_list */ > struct page *page; > struct pagevec pvec; >+ struct sysinfo i; > int reclaim_mapped = 0; > long mapped_ratio; > long distress; >@@ -642,6 +643,14 @@ refill_inactive_zone(struct zone *zone, > mapped_ratio = (ps->nr_mapped * 100) / total_memory; > > /* >+ * Autoregulate vm_swappiness to be equal to the percentage of >+ * pages in physical ram that are application pages. -ck >+ */ >+ si_meminfo(&i); >+ vm_swappiness = 100 - (((i.freeram + get_page_cache_size() - >+ swapper_space.nrpages) * 100) / i.totalram); >+ >+ /* > * Now decide how much we really want to unmap some pages. The mapped > * ratio is downgraded - just because there's a lot of mapped memory > * doesn't necessarily mean that page reclaim isn't succeeding. > Hi Con, If this indeed makes VM behaviour better, why not just merge the calculation with the swap_tendancy calculation and leave vm_swappiness there as a tunable? ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness cleanup 2003-10-26 11:22 ` Nick Piggin @ 2003-10-26 10:36 ` Con Kolivas 2003-10-26 11:42 ` Nick Piggin 0 siblings, 1 reply; 9+ messages in thread From: Con Kolivas @ 2003-10-26 10:36 UTC (permalink / raw) To: Nick Piggin Cc: Con Kolivas, Martin J. Bligh, linux kernel mailing list, Andrew Morton Nick Piggin wrote: > > Hi Con, > If this indeed makes VM behaviour better, why not just merge the > calculation > with the swap_tendancy calculation and leave vm_swappiness there as a > tunable? Because the whole point of it is to remove the tunable and make it auto tuning. We could do away with the vm_swappiness variable altogether too (which I would actually prefer to do) but this leaves it intact to see what the vm is doing. Con ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness cleanup 2003-10-26 10:36 ` Con Kolivas @ 2003-10-26 11:42 ` Nick Piggin 0 siblings, 0 replies; 9+ messages in thread From: Nick Piggin @ 2003-10-26 11:42 UTC (permalink / raw) To: Con Kolivas Cc: Con Kolivas, Martin J. Bligh, linux kernel mailing list, Andrew Morton Con Kolivas wrote: > Nick Piggin wrote: > >> >> Hi Con, >> If this indeed makes VM behaviour better, why not just merge the >> calculation >> with the swap_tendancy calculation and leave vm_swappiness there as a >> tunable? > > > Because the whole point of it is to remove the tunable and make it > auto tuning. We could do away with the vm_swappiness variable > altogether too (which I would actually prefer to do) but this leaves > it intact to see what the vm is doing. Right. This just had me a bit confused. No worries. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness cleanup 2003-10-25 6:58 ` [PATCH] Autoregulate vm swappiness cleanup Con Kolivas 2003-10-26 11:22 ` Nick Piggin @ 2003-10-28 11:04 ` Pavel Machek 2003-10-28 12:40 ` Con Kolivas 1 sibling, 1 reply; 9+ messages in thread From: Pavel Machek @ 2003-10-28 11:04 UTC (permalink / raw) To: Con Kolivas; +Cc: Martin J. Bligh, linux kernel mailing list, Andrew Morton Hi! > > > It seems that you don't need si_swapinfo here, do you? i.freeram, > > > i.bufferram, and i.totalram all come from meminfo, as far as I can > > > see? Maybe I'm missing a bit ... > > > > Well I did do it a while ago and it seems I got carried away adding and > > subtracting info indeed. :-) Here's a simpler patch that does the same > > thing. > > The off-list enthusiasm has been rather strong so here is a patch done the > right way (tm). There is no need for the check of totalram being zero (the > original version of this patch modified the swappiness every tick which was > wasteful and had a divide by zero on init). Adjusting vm_swappiness only when > there is pressure to swap means totalram shouldn't be ever be zero. The > sysctl is made read only since writing to it would be ignored now. I believe swappiness == 100 was "I want max throughput, I don't care about latency going through roof", while swappiness == 0 was "I don't want you to swap too much, behave reasonably". As you don't know if user cares about latency or not, I don't see how you can autotune this. -- When do you have a heart between your knees? [Johanka's followup: and *two* hearts?] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Autoregulate vm swappiness cleanup 2003-10-28 11:04 ` Pavel Machek @ 2003-10-28 12:40 ` Con Kolivas 0 siblings, 0 replies; 9+ messages in thread From: Con Kolivas @ 2003-10-28 12:40 UTC (permalink / raw) To: Pavel Machek; +Cc: Martin J. Bligh, linux kernel mailing list, Andrew Morton On Tue, 28 Oct 2003 22:04, Pavel Machek wrote: > Hi! Hello! > I believe swappiness == 100 was "I want max throughput, I don't care > about latency going through roof", while swappiness == 0 was "I don't > want you to swap too much, behave reasonably". > > As you don't know if user cares about latency or not, I don't see how > you can autotune this. Well I guess you either see merit in what my patch does based on what I said, or you don't... so I guess you don't. That's fine; I just offered why I felt this helped in my varied workloads more than a static value did. Con ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2003-10-28 12:40 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2003-10-23 13:37 [PATCH] Autoregulate vm swappiness 2.6.0-test8 Con Kolivas 2003-10-23 14:42 ` Martin J. Bligh 2003-10-23 15:03 ` Con Kolivas 2003-10-25 6:58 ` [PATCH] Autoregulate vm swappiness cleanup Con Kolivas 2003-10-26 11:22 ` Nick Piggin 2003-10-26 10:36 ` Con Kolivas 2003-10-26 11:42 ` Nick Piggin 2003-10-28 11:04 ` Pavel Machek 2003-10-28 12:40 ` Con Kolivas
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).