All of lore.kernel.org
 help / color / mirror / Atom feed
From: Fengguang Wu <fengguang.wu@intel.com>
To: Sonny Rao <sonnyrao@chromium.org>
Cc: linux-kernel@vger.kernel.org,
	Peter Zijlstra <a.p.zijlstra@chello.nl>,
	Andrew Morton <akpm@linux-foundation.org>,
	Michal Hocko <mhocko@suse.cz>,
	linux-mm@kvack.org, Mandeep Singh Baines <msb@chromium.org>,
	Johannes Weiner <jweiner@redhat.com>,
	Olof Johansson <olofj@chromium.org>,
	Will Drewry <wad@chromium.org>, Kees Cook <keescook@chromium.org>,
	Aaron Durbin <adurbin@chromium.org>,
	Puneet Kumar <puneetster@chromium.org>
Subject: Re: [PATCHv2] mm: Fix calculation of dirtyable memory
Date: Fri, 9 Nov 2012 10:36:38 +0800	[thread overview]
Message-ID: <20121109023638.GA11105@localhost> (raw)
In-Reply-To: <1352422353-11229-1-git-send-email-sonnyrao@chromium.org>

On Thu, Nov 08, 2012 at 04:52:33PM -0800, Sonny Rao wrote:
> The system uses global_dirtyable_memory() to calculate
> number of dirtyable pages/pages that can be allocated
> to the page cache.  A bug causes an underflow thus making
> the page count look like a big unsigned number.  This in turn
> confuses the dirty writeback throttling to aggressively write
> back pages as they become dirty (usually 1 page at a time).
> 
> Fix is to ensure there is no underflow while doing the math.

Good catch, thanks!

> Signed-off-by: Sonny Rao <sonnyrao@chromium.org>
> Signed-off-by: Puneet Kumar <puneetster@chromium.org>
> ---
>  v2: added apkm's suggestion to make the highmem calculation better
>  mm/page-writeback.c |   17 +++++++++++++++--
>  1 files changed, 15 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/page-writeback.c b/mm/page-writeback.c
> index 830893b..ce62442 100644
> --- a/mm/page-writeback.c
> +++ b/mm/page-writeback.c
> @@ -201,6 +201,18 @@ static unsigned long highmem_dirtyable_memory(unsigned long total)
>  		     zone_reclaimable_pages(z) - z->dirty_balance_reserve;
>  	}
>  	/*
> +	 * Unreclaimable memory (kernel memory or anonymous memory
> +	 * without swap) can bring down the dirtyable pages below
> +	 * the zone's dirty balance reserve and the above calculation
> +	 * will underflow.  However we still want to add in nodes
> +	 * which are below threshold (negative values) to get a more
> +	 * accurate calculation but make sure that the total never
> +	 * underflows.
> +	 */
> +	if ((long)x < 0)
> +		x = 0;
> +
> +	/*
>  	 * Make sure that the number of highmem pages is never larger
>  	 * than the number of the total dirtyable memory. This can only
>  	 * occur in very strange VM situations but we want to make sure
> @@ -222,8 +234,9 @@ static unsigned long global_dirtyable_memory(void)
>  {
>  	unsigned long x;
>  
> -	x = global_page_state(NR_FREE_PAGES) + global_reclaimable_pages() -
> -	    dirty_balance_reserve;
> +	x = global_page_state(NR_FREE_PAGES) + global_reclaimable_pages();
> +	if (x >= dirty_balance_reserve)
> +		x -= dirty_balance_reserve;

That can be converted to "if ((long)x < 0) x = 0;", too.

And I suspect zone_dirtyable_memory() needs similar fix, too.

Thanks,
Fengguang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Fengguang Wu <fengguang.wu@intel.com>
To: Sonny Rao <sonnyrao@chromium.org>
Cc: linux-kernel@vger.kernel.org,
	Peter Zijlstra <a.p.zijlstra@chello.nl>,
	Andrew Morton <akpm@linux-foundation.org>,
	Michal Hocko <mhocko@suse.cz>,
	linux-mm@kvack.org, Mandeep Singh Baines <msb@chromium.org>,
	Johannes Weiner <jweiner@redhat.com>,
	Olof Johansson <olofj@chromium.org>,
	Will Drewry <wad@chromium.org>, Kees Cook <keescook@chromium.org>,
	Aaron Durbin <adurbin@chromium.org>,
	Puneet Kumar <puneetster@chromium.org>
Subject: Re: [PATCHv2] mm: Fix calculation of dirtyable memory
Date: Fri, 9 Nov 2012 10:36:38 +0800	[thread overview]
Message-ID: <20121109023638.GA11105@localhost> (raw)
In-Reply-To: <1352422353-11229-1-git-send-email-sonnyrao@chromium.org>

On Thu, Nov 08, 2012 at 04:52:33PM -0800, Sonny Rao wrote:
> The system uses global_dirtyable_memory() to calculate
> number of dirtyable pages/pages that can be allocated
> to the page cache.  A bug causes an underflow thus making
> the page count look like a big unsigned number.  This in turn
> confuses the dirty writeback throttling to aggressively write
> back pages as they become dirty (usually 1 page at a time).
> 
> Fix is to ensure there is no underflow while doing the math.

Good catch, thanks!

> Signed-off-by: Sonny Rao <sonnyrao@chromium.org>
> Signed-off-by: Puneet Kumar <puneetster@chromium.org>
> ---
>  v2: added apkm's suggestion to make the highmem calculation better
>  mm/page-writeback.c |   17 +++++++++++++++--
>  1 files changed, 15 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/page-writeback.c b/mm/page-writeback.c
> index 830893b..ce62442 100644
> --- a/mm/page-writeback.c
> +++ b/mm/page-writeback.c
> @@ -201,6 +201,18 @@ static unsigned long highmem_dirtyable_memory(unsigned long total)
>  		     zone_reclaimable_pages(z) - z->dirty_balance_reserve;
>  	}
>  	/*
> +	 * Unreclaimable memory (kernel memory or anonymous memory
> +	 * without swap) can bring down the dirtyable pages below
> +	 * the zone's dirty balance reserve and the above calculation
> +	 * will underflow.  However we still want to add in nodes
> +	 * which are below threshold (negative values) to get a more
> +	 * accurate calculation but make sure that the total never
> +	 * underflows.
> +	 */
> +	if ((long)x < 0)
> +		x = 0;
> +
> +	/*
>  	 * Make sure that the number of highmem pages is never larger
>  	 * than the number of the total dirtyable memory. This can only
>  	 * occur in very strange VM situations but we want to make sure
> @@ -222,8 +234,9 @@ static unsigned long global_dirtyable_memory(void)
>  {
>  	unsigned long x;
>  
> -	x = global_page_state(NR_FREE_PAGES) + global_reclaimable_pages() -
> -	    dirty_balance_reserve;
> +	x = global_page_state(NR_FREE_PAGES) + global_reclaimable_pages();
> +	if (x >= dirty_balance_reserve)
> +		x -= dirty_balance_reserve;

That can be converted to "if ((long)x < 0) x = 0;", too.

And I suspect zone_dirtyable_memory() needs similar fix, too.

Thanks,
Fengguang

  reply	other threads:[~2012-11-09  2:36 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-11-08 23:25 [PATCH] mm: Fix calculation of dirtyable memory Sonny Rao
2012-11-08 23:25 ` Sonny Rao
2012-11-08 23:37 ` Andrew Morton
2012-11-08 23:37   ` Andrew Morton
2012-11-09  0:42   ` [PATCHv2] " Sonny Rao
2012-11-09  0:42     ` Sonny Rao
2012-11-09  0:45     ` Sonny Rao
2012-11-09  0:45       ` Sonny Rao
2012-11-09  0:52       ` Sonny Rao
2012-11-09  0:52         ` Sonny Rao
2012-11-09  2:36         ` Fengguang Wu [this message]
2012-11-09  2:36           ` Fengguang Wu
2012-11-12 19:35           ` [PATCH] " Sonny Rao
2012-11-12 19:35             ` Sonny Rao
2012-11-12 20:32             ` Johannes Weiner
2012-11-12 20:32               ` Johannes Weiner
2012-11-12 21:35               ` [PATCHv4] " Sonny Rao
2012-11-12 21:35                 ` Sonny Rao
2012-11-17 20:41                 ` Damien Wyart
2012-11-19 13:40                 ` Johannes Weiner
2012-11-19 13:40                   ` Johannes Weiner
2012-11-19 18:41                   ` [PATCHv5] " Sonny Rao
2012-11-19 18:41                     ` Sonny Rao
2012-11-19 18:44                     ` Sonny Rao
2012-11-19 18:44                       ` Sonny Rao
2012-11-19 19:22                       ` Damien Wyart
2012-11-19 19:22                         ` Damien Wyart

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20121109023638.GA11105@localhost \
    --to=fengguang.wu@intel.com \
    --cc=a.p.zijlstra@chello.nl \
    --cc=adurbin@chromium.org \
    --cc=akpm@linux-foundation.org \
    --cc=jweiner@redhat.com \
    --cc=keescook@chromium.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@suse.cz \
    --cc=msb@chromium.org \
    --cc=olofj@chromium.org \
    --cc=puneetster@chromium.org \
    --cc=sonnyrao@chromium.org \
    --cc=wad@chromium.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.