* [PATCH NG] alloc_pages_limit & pages_min
@ 2001-08-22 23:46 Roger Larsson
2001-08-23 0:40 ` Rik van Riel
0 siblings, 1 reply; 13+ messages in thread
From: Roger Larsson @ 2001-08-22 23:46 UTC (permalink / raw)
To: linux-mm
Hi,
[Note: not tested yet... but it might be hard to trigger]
After discussions with Riel and some additions...
* The original code had a little bug in those cases when
- kreclaimd were not allowed to run for a LONG time...
Lots of kernel activity, RT tasks, or code running
around for a long time allocating memory.
- there were lots of inactive clean pages
- pages were allocated without direct_reclaim (higher order)
(networking might be one candidate)
it could result in using up ALL free pages!
This patch tries to prevent this situation in several ways:
1) Do not allow to alloc a free page when they are critically low.
[last line of patch]
2) If direct reclaims are allowed do some additional work.
reclaim & free until pages_min,
alloc one page,
reclaim and free until pages_low
Q) Nothing is done to force execution of kreclaimd, if no process
that can direct_reclaim allocs a page - what will happen then?
[unlikely but...]
/RogerL
--
Roger Larsson
Skelleftea
Sweden
*******************************************
Patch prepared by: roger.larsson@norran.net with comments from Riel
--- linux/mm/page_alloc.c.orig Wed Aug 22 13:36:57 2001
+++ linux/mm/page_alloc.c Thu Aug 23 01:15:17 2001
@@ -253,11 +253,35 @@
if (z->free_pages + z->inactive_clean_pages >= water_mark) {
struct page *page = NULL;
- /* If possible, reclaim a page directly. */
- if (direct_reclaim)
+
+ /*
+ * Reclaim a page from the inactive_clean list.
+ * If needed, refill the free list up to the
+ * low water mark.
+ */
+ if (direct_reclaim) {
page = reclaim_page(z);
- /* If that fails, fall back to rmqueue. */
- if (!page)
+
+ while (page && z->free_pages < z->pages_min) {
+ __free_page(page);
+ page = reclaim_page(z);
+ }
+
+ if (page) {
+ while (z->free_pages < z->pages_low) {
+ struct page *extra = reclaim_page(z);
+ if (!extra)
+ break;
+ __free_page(extra);
+ }
+ }
+
+ /* let kreclaimd handle up to pages_high */
+ }
+ /* If that fails, fall back to rmqueue, but never let
+ * free_pages go below pages_min...
+ */
+ if (!page && z->free_pages >= z->pages_min)
page = rmqueue(z, order);
if (page)
return page;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH NG] alloc_pages_limit & pages_min
2001-08-22 23:46 [PATCH NG] alloc_pages_limit & pages_min Roger Larsson
@ 2001-08-23 0:40 ` Rik van Riel
2001-08-23 18:36 ` Roger Larsson
2001-08-23 18:45 ` Roger Larsson
0 siblings, 2 replies; 13+ messages in thread
From: Rik van Riel @ 2001-08-23 0:40 UTC (permalink / raw)
To: Roger Larsson; +Cc: linux-mm
On Thu, 23 Aug 2001, Roger Larsson wrote:
> + if (page) {
> + while (z->free_pages < z->pages_low) {
> + struct page *extra = reclaim_page(z);
> + if (!extra)
> + break;
> + __free_page(extra);
> + }
> + }
This is a surprise ;)
Why did you introduce this piece of code?
What is it supposed to achieve ?
Rik
--
IA64: a worthy successor to i860.
http://www.surriel.com/ http://distro.conectiva.com/
Send all your spam to aardvark@nl.linux.org (spam digging piggy)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH NG] alloc_pages_limit & pages_min
2001-08-23 0:40 ` Rik van Riel
@ 2001-08-23 18:36 ` Roger Larsson
2001-08-23 18:44 ` Rik van Riel
2001-08-23 18:45 ` Roger Larsson
1 sibling, 1 reply; 13+ messages in thread
From: Roger Larsson @ 2001-08-23 18:36 UTC (permalink / raw)
To: Rik van Riel; +Cc: linux-mm
On Thursdayen den 23 August 2001 02:40, Rik van Riel wrote:
> On Thu, 23 Aug 2001, Roger Larsson wrote:
> > + if (page) {
> > + while (z->free_pages < z->pages_low) {
> > + struct page *extra = reclaim_page(z);
> > + if (!extra)
> > + break;
> > + __free_page(extra);
> > + }
> > + }
>
> This is a surprise ;)
>
> Why did you introduce this piece of code?
> What is it supposed to achieve ?
>
f we did get one page => we are above pages_min
try to reach pages_low too.
But holding on to the page we got.
It is possible that it should only be done if we started under
pages_min. But it will be faster the closer we start to pages_low
/RogerL
--
Roger Larsson
Skelleftea
Sweden
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH NG] alloc_pages_limit & pages_min
2001-08-23 18:36 ` Roger Larsson
@ 2001-08-23 18:44 ` Rik van Riel
2001-08-23 18:52 ` Roger Larsson
0 siblings, 1 reply; 13+ messages in thread
From: Rik van Riel @ 2001-08-23 18:44 UTC (permalink / raw)
To: Roger Larsson; +Cc: linux-mm
On Thu, 23 Aug 2001, Roger Larsson wrote:
> f we did get one page => we are above pages_min
> try to reach pages_low too.
Yeah, but WHY ?
Rik
--
IA64: a worthy successor to the i860.
http://www.surriel.com/
http://www.conectiva.com/ http://distro.conectiva.com/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH NG] alloc_pages_limit & pages_min
2001-08-23 0:40 ` Rik van Riel
2001-08-23 18:36 ` Roger Larsson
@ 2001-08-23 18:45 ` Roger Larsson
2001-08-23 18:55 ` Rik van Riel
1 sibling, 1 reply; 13+ messages in thread
From: Roger Larsson @ 2001-08-23 18:45 UTC (permalink / raw)
To: Rik van Riel; +Cc: linux-mm
On Thursdayen den 23 August 2001 02:40, Rik van Riel wrote:
> On Thu, 23 Aug 2001, Roger Larsson wrote:
> > + if (page) {
> > + while (z->free_pages < z->pages_low) {
> > + struct page *extra = reclaim_page(z);
> > + if (!extra)
> > + break;
> > + __free_page(extra);
> > + }
> > + }
>
> This is a surprise ;)
>
> Why did you introduce this piece of code?
> What is it supposed to achieve ?
>
A lighter alternative would be to reclaim just one extra page...
Then it will move in the right direction but not more, quite
nice actually!
/RogerL
--
Roger Larsson
Skelleftea
Sweden
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH NG] alloc_pages_limit & pages_min
2001-08-23 18:44 ` Rik van Riel
@ 2001-08-23 18:52 ` Roger Larsson
2001-08-23 19:03 ` Rik van Riel
0 siblings, 1 reply; 13+ messages in thread
From: Roger Larsson @ 2001-08-23 18:52 UTC (permalink / raw)
To: Rik van Riel; +Cc: linux-mm
On Thursdayen den 23 August 2001 20:44, Rik van Riel wrote:
> On Thu, 23 Aug 2001, Roger Larsson wrote:
> > f we did get one page => we are above pages_min
> > try to reach pages_low too.
>
> Yeah, but WHY ?
>
* Historic reasons - I feel good at that limit... :-)
MIN the limit never crossed
LOW center, our target of free pages - when all zones time to free.
HIGH limit were to stop the freeing.
--
Roger Larsson
Skelleftea
Sweden
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH NG] alloc_pages_limit & pages_min
2001-08-23 18:45 ` Roger Larsson
@ 2001-08-23 18:55 ` Rik van Riel
0 siblings, 0 replies; 13+ messages in thread
From: Rik van Riel @ 2001-08-23 18:55 UTC (permalink / raw)
To: Roger Larsson; +Cc: linux-mm
On Thu, 23 Aug 2001, Roger Larsson wrote:
> > Why did you introduce this piece of code?
> > What is it supposed to achieve ?
>
> A lighter alternative would be to reclaim just one extra page...
> Then it will move in the right direction but not more, quite
> nice actually!
Why ?
Or, to be more specific, why would we want to throw away
data from the cache all the way up to pages_min when we
know we're running a workload with allocations which can
eat directly from the cache ?
Rik
--
IA64: a worthy successor to the i860.
http://www.surriel.com/
http://www.conectiva.com/ http://distro.conectiva.com/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH NG] alloc_pages_limit & pages_min
2001-08-23 18:52 ` Roger Larsson
@ 2001-08-23 19:03 ` Rik van Riel
2001-08-23 19:28 ` Roger Larsson
0 siblings, 1 reply; 13+ messages in thread
From: Rik van Riel @ 2001-08-23 19:03 UTC (permalink / raw)
To: Roger Larsson; +Cc: linux-mm
On Thu, 23 Aug 2001, Roger Larsson wrote:
> On Thursdayen den 23 August 2001 20:44, Rik van Riel wrote:
> > On Thu, 23 Aug 2001, Roger Larsson wrote:
> > > f we did get one page => we are above pages_min
> > > try to reach pages_low too.
> >
> > Yeah, but WHY ?
>
> * Historic reasons - I feel good at that limit... :-)
> MIN the limit never crossed
Never crossed by (free + clean) pages. I see no reason why
we couldn't leave the free-only target at this limit...
> LOW center, our target of free pages - when all zones time to free.
Meaning we'll usually not have any clean pages around but
only free pages if your patch gets applied ;)
> HIGH limit were to stop the freeing.
Rik
--
IA64: a worthy successor to the i860.
http://www.surriel.com/
http://www.conectiva.com/ http://distro.conectiva.com/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Upd: [PATCH NG] alloc_pages_limit & pages_min
2001-08-23 19:03 ` Rik van Riel
@ 2001-08-23 19:28 ` Roger Larsson
0 siblings, 0 replies; 13+ messages in thread
From: Roger Larsson @ 2001-08-23 19:28 UTC (permalink / raw)
To: Rik van Riel, Linus Torvalds, alan.cox
Cc: linux-mm, Linux Kernel Mailing List
Riel convinced be to back off a part of the patch.
Here comes an updated one.
--
Roger Larsson
Skellefteå
Sweden
*******************************************
Patch prepared by: roger.larsson@norran.net
Name of file: linux-2.4.8-pre3-pages_min-R3
--- linux/mm/page_alloc.c.orig Thu Aug 23 19:58:55 2001
+++ linux/mm/page_alloc.c Thu Aug 23 21:19:20 2001
@@ -253,11 +253,26 @@
if (z->free_pages + z->inactive_clean_pages >= water_mark) {
struct page *page = NULL;
- /* If possible, reclaim a page directly. */
- if (direct_reclaim)
+
+ /*
+ * Reclaim a page from the inactive_clean list.
+ * If needed, refill the free list up to the
+ * low water mark.
+ */
+ if (direct_reclaim) {
page = reclaim_page(z);
- /* If that fails, fall back to rmqueue. */
- if (!page)
+
+ while (page && z->free_pages < z->pages_min) {
+ __free_page(page);
+ page = reclaim_page(z);
+ }
+
+ /* let kreclaimd handle up to pages_high */
+ }
+ /* If that fails, fall back to rmqueue, but never let
+ * free_pages go below pages_min...
+ */
+ if (!page && z->free_pages >= z->pages_min)
page = rmqueue(z, order);
if (page)
return page;
^ permalink raw reply [flat|nested] 13+ messages in thread
* Upd: [PATCH NG] alloc_pages_limit & pages_min
@ 2001-08-23 19:28 ` Roger Larsson
0 siblings, 0 replies; 13+ messages in thread
From: Roger Larsson @ 2001-08-23 19:28 UTC (permalink / raw)
To: Rik van Riel, Linus Torvalds, alan.cox
Cc: linux-mm, Linux Kernel Mailing List
Riel convinced be to back off a part of the patch.
Here comes an updated one.
--
Roger Larsson
Skelleftea
Sweden
*******************************************
Patch prepared by: roger.larsson@norran.net
Name of file: linux-2.4.8-pre3-pages_min-R3
--- linux/mm/page_alloc.c.orig Thu Aug 23 19:58:55 2001
+++ linux/mm/page_alloc.c Thu Aug 23 21:19:20 2001
@@ -253,11 +253,26 @@
if (z->free_pages + z->inactive_clean_pages >= water_mark) {
struct page *page = NULL;
- /* If possible, reclaim a page directly. */
- if (direct_reclaim)
+
+ /*
+ * Reclaim a page from the inactive_clean list.
+ * If needed, refill the free list up to the
+ * low water mark.
+ */
+ if (direct_reclaim) {
page = reclaim_page(z);
- /* If that fails, fall back to rmqueue. */
- if (!page)
+
+ while (page && z->free_pages < z->pages_min) {
+ __free_page(page);
+ page = reclaim_page(z);
+ }
+
+ /* let kreclaimd handle up to pages_high */
+ }
+ /* If that fails, fall back to rmqueue, but never let
+ * free_pages go below pages_min...
+ */
+ if (!page && z->free_pages >= z->pages_min)
page = rmqueue(z, order);
if (page)
return page;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Upd: [PATCH NG] alloc_pages_limit & pages_min
2001-08-23 19:28 ` Roger Larsson
(?)
@ 2001-08-24 9:25 ` Stephan von Krawczynski
2001-08-24 10:28 ` Roger Larsson
-1 siblings, 1 reply; 13+ messages in thread
From: Stephan von Krawczynski @ 2001-08-24 9:25 UTC (permalink / raw)
To: Roger Larsson; +Cc: linux-kernel
On Thu, 23 Aug 2001 21:28:44 +0200
Roger Larsson <roger.larsson@norran.net> wrote:
> Riel convinced be to back off a part of the patch.
> Here comes an updated one.
Hello Roger,
this does not solve my problem with NFS-copies. Just for information. I tried and did not work.
Besides I expected the patch to make the free pages pool somehow bigger during file-copies, but ended up with this situation:
total: used: free: shared: buffers: cached:
Mem: 921726976 918573056 3153920 0 12337152 816504832
Swap: 271392768 0 271392768
MemTotal: 900124 kB
MemFree: 3080 kB
MemShared: 0 kB
Buffers: 12048 kB
Cached: 797368 kB
SwapCached: 0 kB
Active: 473628 kB
Inact_dirty: 331796 kB
Inact_clean: 3992 kB
Inact_target: 1940 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 900124 kB
LowFree: 3080 kB
SwapTotal: 265032 kB
SwapFree: 265032 kB
The MemFree isn't really a lot compared to inact_dirty. knfsd fails at least.
Regards, Stephan
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Upd: [PATCH NG] alloc_pages_limit & pages_min
2001-08-24 9:25 ` Stephan von Krawczynski
@ 2001-08-24 10:28 ` Roger Larsson
2001-08-24 12:50 ` Stephan von Krawczynski
0 siblings, 1 reply; 13+ messages in thread
From: Roger Larsson @ 2001-08-24 10:28 UTC (permalink / raw)
To: Stephan von Krawczynski; +Cc: linux-kernel
On Fridayen den 24 August 2001 11:25, Stephan von Krawczynski wrote:
> On Thu, 23 Aug 2001 21:28:44 +0200
>
> Roger Larsson <roger.larsson@norran.net> wrote:
> > Riel convinced be to back off a part of the patch.
> > Here comes an updated one.
>
> Hello Roger,
>
> this does not solve my problem with NFS-copies. Just for information. I
> tried and did not work. Besides I expected the patch to make the free pages
> pool somehow bigger during file-copies, but ended up with this situation:
>
> [snip]
>
> The MemFree isn't really a lot compared to inact_dirty. knfsd fails at
> least.
Wait a minute - knfsd... hmm...
Suppose knfsd allock without the wait flag - then it could cause this problems
by itself...
> Aug 21 20:14:51 admin kernel: __alloc_pages: 3-order allocation failed
> (gfp=0x20/0).
Who is doing thise allocs?
Add a printout of current->pid with format %d
I bet it is knfsd itself (or a driver it uses).
Another thing to try is to run with non kernel nfs...
/RogerL
--
Roger Larsson
Skellefteå
Sweden
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Upd: [PATCH NG] alloc_pages_limit & pages_min
2001-08-24 10:28 ` Roger Larsson
@ 2001-08-24 12:50 ` Stephan von Krawczynski
0 siblings, 0 replies; 13+ messages in thread
From: Stephan von Krawczynski @ 2001-08-24 12:50 UTC (permalink / raw)
To: Roger Larsson; +Cc: linux-kernel
On Fri, 24 Aug 2001 12:28:13 +0200
Roger Larsson <roger.larsson@skelleftea.mail.telia.com> wrote:
> Another thing to try is to run with non kernel nfs...
Sorry, probably can't do that. As far as I read the docs from reiser there is a problem with unfs on large reiser disks. If I got that right, it will not work anyway, even without this specific problem.
Regards, Stephan
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2001-08-24 12:51 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2001-08-22 23:46 [PATCH NG] alloc_pages_limit & pages_min Roger Larsson
2001-08-23 0:40 ` Rik van Riel
2001-08-23 18:36 ` Roger Larsson
2001-08-23 18:44 ` Rik van Riel
2001-08-23 18:52 ` Roger Larsson
2001-08-23 19:03 ` Rik van Riel
2001-08-23 19:28 ` Upd: " Roger Larsson
2001-08-23 19:28 ` Roger Larsson
2001-08-24 9:25 ` Stephan von Krawczynski
2001-08-24 10:28 ` Roger Larsson
2001-08-24 12:50 ` Stephan von Krawczynski
2001-08-23 18:45 ` Roger Larsson
2001-08-23 18:55 ` Rik van Riel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.