All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: Mel Gorman <mgorman@suse.de>, Rik van Riel <riel@redhat.com>,
	Minchan Kim <minchan.kim@gmail.com>,
	Michal Hocko <mhocko@suse.cz>,
	Christoph Hellwig <hch@infradead.org>,
	Wu Fengguang <fengguang.wu@intel.com>,
	Dave Chinner <david@fromorbit.com>, Jan Kara <jack@suse.cz>,
	Shaohua Li <shaohua.li@intel.com>,
	linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [patch 1/5] mm: exclude reserved pages from dirtyable memory
Date: Tue, 29 Nov 2011 16:20:14 -0800	[thread overview]
Message-ID: <20111129162014.aa290174.akpm@linux-foundation.org> (raw)
In-Reply-To: <1322055258-3254-2-git-send-email-hannes@cmpxchg.org>

On Wed, 23 Nov 2011 14:34:14 +0100
Johannes Weiner <hannes@cmpxchg.org> wrote:

> From: Johannes Weiner <jweiner@redhat.com>
> 
> The amount of dirtyable pages should not include the full number of
> free pages: there is a number of reserved pages that the page
> allocator and kswapd always try to keep free.
> 
> The closer (reclaimable pages - dirty pages) is to the number of
> reserved pages, the more likely it becomes for reclaim to run into
> dirty pages:
> 
>        +----------+ ---
>        |   anon   |  |
>        +----------+  |
>        |          |  |
>        |          |  -- dirty limit new    -- flusher new
>        |   file   |  |                     |
>        |          |  |                     |
>        |          |  -- dirty limit old    -- flusher old
>        |          |                        |
>        +----------+                       --- reclaim
>        | reserved |
>        +----------+
>        |  kernel  |
>        +----------+
> 
> This patch introduces a per-zone dirty reserve that takes both the
> lowmem reserve as well as the high watermark of the zone into account,
> and a global sum of those per-zone values that is subtracted from the
> global amount of dirtyable pages.  The lowmem reserve is unavailable
> to page cache allocations and kswapd tries to keep the high watermark
> free.  We don't want to end up in a situation where reclaim has to
> clean pages in order to balance zones.
> 
> Not treating reserved pages as dirtyable on a global level is only a
> conceptual fix.  In reality, dirty pages are not distributed equally
> across zones and reclaim runs into dirty pages on a regular basis.
> 
> But it is important to get this right before tackling the problem on a
> per-zone level, where the distance between reclaim and the dirty pages
> is mostly much smaller in absolute numbers.
> 
> ...
>
> --- a/mm/page-writeback.c
> +++ b/mm/page-writeback.c
> @@ -327,7 +327,8 @@ static unsigned long highmem_dirtyable_memory(unsigned long total)
>  			&NODE_DATA(node)->node_zones[ZONE_HIGHMEM];
>  
>  		x += zone_page_state(z, NR_FREE_PAGES) +
> -		     zone_reclaimable_pages(z);
> +		     zone_reclaimable_pages(z) -
> +		     zone->dirty_balance_reserve;

Doesn't compile.  s/zone/z/.

Which makes me suspect it wasn't tested on a highmem box.  This is
rather worrisome, as highmem machines tend to have acute and unique
zone balancing issues.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Andrew Morton <akpm@linux-foundation.org>
To: Johannes Weiner <hannes@cmpxchg.org>
Cc: Mel Gorman <mgorman@suse.de>, Rik van Riel <riel@redhat.com>,
	Minchan Kim <minchan.kim@gmail.com>,
	Michal Hocko <mhocko@suse.cz>,
	Christoph Hellwig <hch@infradead.org>,
	Wu Fengguang <fengguang.wu@intel.com>,
	Dave Chinner <david@fromorbit.com>, Jan Kara <jack@suse.cz>,
	Shaohua Li <shaohua.li@intel.com>,
	linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [patch 1/5] mm: exclude reserved pages from dirtyable memory
Date: Tue, 29 Nov 2011 16:20:14 -0800	[thread overview]
Message-ID: <20111129162014.aa290174.akpm@linux-foundation.org> (raw)
In-Reply-To: <1322055258-3254-2-git-send-email-hannes@cmpxchg.org>

On Wed, 23 Nov 2011 14:34:14 +0100
Johannes Weiner <hannes@cmpxchg.org> wrote:

> From: Johannes Weiner <jweiner@redhat.com>
> 
> The amount of dirtyable pages should not include the full number of
> free pages: there is a number of reserved pages that the page
> allocator and kswapd always try to keep free.
> 
> The closer (reclaimable pages - dirty pages) is to the number of
> reserved pages, the more likely it becomes for reclaim to run into
> dirty pages:
> 
>        +----------+ ---
>        |   anon   |  |
>        +----------+  |
>        |          |  |
>        |          |  -- dirty limit new    -- flusher new
>        |   file   |  |                     |
>        |          |  |                     |
>        |          |  -- dirty limit old    -- flusher old
>        |          |                        |
>        +----------+                       --- reclaim
>        | reserved |
>        +----------+
>        |  kernel  |
>        +----------+
> 
> This patch introduces a per-zone dirty reserve that takes both the
> lowmem reserve as well as the high watermark of the zone into account,
> and a global sum of those per-zone values that is subtracted from the
> global amount of dirtyable pages.  The lowmem reserve is unavailable
> to page cache allocations and kswapd tries to keep the high watermark
> free.  We don't want to end up in a situation where reclaim has to
> clean pages in order to balance zones.
> 
> Not treating reserved pages as dirtyable on a global level is only a
> conceptual fix.  In reality, dirty pages are not distributed equally
> across zones and reclaim runs into dirty pages on a regular basis.
> 
> But it is important to get this right before tackling the problem on a
> per-zone level, where the distance between reclaim and the dirty pages
> is mostly much smaller in absolute numbers.
> 
> ...
>
> --- a/mm/page-writeback.c
> +++ b/mm/page-writeback.c
> @@ -327,7 +327,8 @@ static unsigned long highmem_dirtyable_memory(unsigned long total)
>  			&NODE_DATA(node)->node_zones[ZONE_HIGHMEM];
>  
>  		x += zone_page_state(z, NR_FREE_PAGES) +
> -		     zone_reclaimable_pages(z);
> +		     zone_reclaimable_pages(z) -
> +		     zone->dirty_balance_reserve;

Doesn't compile.  s/zone/z/.

Which makes me suspect it wasn't tested on a highmem box.  This is
rather worrisome, as highmem machines tend to have acute and unique
zone balancing issues.


  reply	other threads:[~2011-11-30  0:20 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-11-23 13:34 [patch 0/5] mm: per-zone dirty limits v3-resend Johannes Weiner
2011-11-23 13:34 ` Johannes Weiner
2011-11-23 13:34 ` [patch 1/5] mm: exclude reserved pages from dirtyable memory Johannes Weiner
2011-11-23 13:34   ` Johannes Weiner
2011-11-30  0:20   ` Andrew Morton [this message]
2011-11-30  0:20     ` Andrew Morton
2011-12-07 13:58     ` Johannes Weiner
2011-12-07 13:58       ` Johannes Weiner
2011-11-23 13:34 ` [patch 2/5] mm: writeback: cleanups in preparation for per-zone dirty limits Johannes Weiner
2011-11-23 13:34   ` Johannes Weiner
2011-11-23 13:34 ` [patch 3/5] mm: try to distribute dirty pages fairly across zones Johannes Weiner
2011-11-23 13:34   ` Johannes Weiner
2011-11-24  1:07   ` KAMEZAWA Hiroyuki
2011-11-24  1:07     ` KAMEZAWA Hiroyuki
2011-11-24 13:11     ` Johannes Weiner
2011-11-24 13:11       ` Johannes Weiner
2011-11-25  1:00       ` KAMEZAWA Hiroyuki
2011-11-25  1:00         ` KAMEZAWA Hiroyuki
2011-11-23 13:34 ` [patch 4/5] mm: filemap: pass __GFP_WRITE from grab_cache_page_write_begin() Johannes Weiner
2011-11-23 13:34   ` Johannes Weiner
2011-11-23 13:34 ` [patch 5/5] Btrfs: pass __GFP_WRITE for buffered write page allocations Johannes Weiner
2011-11-23 13:34   ` Johannes Weiner
  -- strict thread matches above, loose matches on Subject: below --
2011-09-30  7:17 [patch 0/5] per-zone dirty limits v3 Johannes Weiner
2011-09-30  7:17 ` [patch 1/5] mm: exclude reserved pages from dirtyable memory Johannes Weiner
2011-09-30  7:17   ` Johannes Weiner
2011-09-30  7:17   ` Johannes Weiner
2011-09-30 13:53   ` Michal Hocko
2011-09-30 13:53     ` Michal Hocko
2011-09-30 13:53     ` Michal Hocko
2011-10-01  7:10   ` Minchan Kim
2011-10-01  7:10     ` Minchan Kim
2011-10-01  7:10     ` Minchan Kim
2011-10-03 11:22   ` Mel Gorman
2011-10-03 11:22     ` Mel Gorman
2011-10-03 11:22     ` Mel Gorman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20111129162014.aa290174.akpm@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=david@fromorbit.com \
    --cc=fengguang.wu@intel.com \
    --cc=hannes@cmpxchg.org \
    --cc=hch@infradead.org \
    --cc=jack@suse.cz \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@suse.de \
    --cc=mhocko@suse.cz \
    --cc=minchan.kim@gmail.com \
    --cc=riel@redhat.com \
    --cc=shaohua.li@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.