All of lore.kernel.org
 help / color / mirror / Atom feed
From: Vlastimil Babka <vbabka@suse.cz>
To: Michal Hocko <mhocko@kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Mel Gorman <mgorman@suse.de>,
	David Rientjes <rientjes@google.com>,
	Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>,
	Joonsoo Kim <js1304@gmail.com>,
	Hillf Danton <hillf.zj@alibaba-inc.com>,
	linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org>,
	Michal Hocko <mhocko@suse.com>
Subject: Re: [PATCH 10/11] mm, oom: protect !costly allocations some more
Date: Mon, 11 Apr 2016 16:48:40 +0200	[thread overview]
Message-ID: <570BB948.6000900@suse.cz> (raw)
In-Reply-To: <1459855533-4600-11-git-send-email-mhocko@kernel.org>

On 04/05/2016 01:25 PM, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
>
> should_reclaim_retry will give up retries for higher order allocations
> if none of the eligible zones has any requested or higher order pages
> available even if we pass the watermak check for order-0. This is done
> because there is no guarantee that the reclaimable and currently free
> pages will form the required order.
>
> This can, however, lead to situations were the high-order request (e.g.
> order-2 required for the stack allocation during fork) will trigger
> OOM too early - e.g. after the first reclaim/compaction round. Such a
> system would have to be highly fragmented and there is no guarantee
> further reclaim/compaction attempts would help but at least make sure
> that the compaction was active before we go OOM and keep retrying even
> if should_reclaim_retry tells us to oom if
> 	- the last compaction round backed off or
> 	- we haven't completed at least MAX_COMPACT_RETRIES active
> 	  compaction rounds.
>
> The first rule ensures that the very last attempt for compaction
> was not ignored while the second guarantees that the compaction has done
> some work. Multiple retries might be needed to prevent occasional
> pigggy packing of other contexts to steal the compacted pages before
> the current context manages to retry to allocate them.
>
> compaction_failed() is taken as a final word from the compaction that
> the retry doesn't make much sense. We have to be careful though because
> the first compaction round is MIGRATE_ASYNC which is rather weak as it
> ignores pages under writeback and gives up too easily in other
> situations. We therefore have to make sure that MIGRATE_SYNC_LIGHT mode
> has been used before we give up. With this logic in place we do not have
> to increase the migration mode unconditionally and rather do it only if
> the compaction failed for the weaker mode. A nice side effect is that
> the stronger migration mode is used only when really needed so this has
> a potential of smaller latencies in some cases.
>
> Please note that the compaction doesn't tell us much about how
> successful it was when returning compaction_made_progress so we just
> have to blindly trust that another retry is worthwhile and cap the
> number to something reasonable to guarantee a convergence.
>
> If the given number of successful retries is not sufficient for a
> reasonable workloads we should focus on the collected compaction
> tracepoints data and try to address the issue in the compaction code.
> If this is not feasible we can increase the retries limit.
>
> Signed-off-by: Michal Hocko <mhocko@suse.com>

Looks good.

Acked-by: Vlastimil Babka <vbabka@suse.cz>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Vlastimil Babka <vbabka@suse.cz>
To: Michal Hocko <mhocko@kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Mel Gorman <mgorman@suse.de>,
	David Rientjes <rientjes@google.com>,
	Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>,
	Joonsoo Kim <js1304@gmail.com>,
	Hillf Danton <hillf.zj@alibaba-inc.com>,
	linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org>,
	Michal Hocko <mhocko@suse.com>
Subject: Re: [PATCH 10/11] mm, oom: protect !costly allocations some more
Date: Mon, 11 Apr 2016 16:48:40 +0200	[thread overview]
Message-ID: <570BB948.6000900@suse.cz> (raw)
In-Reply-To: <1459855533-4600-11-git-send-email-mhocko@kernel.org>

On 04/05/2016 01:25 PM, Michal Hocko wrote:
> From: Michal Hocko <mhocko@suse.com>
>
> should_reclaim_retry will give up retries for higher order allocations
> if none of the eligible zones has any requested or higher order pages
> available even if we pass the watermak check for order-0. This is done
> because there is no guarantee that the reclaimable and currently free
> pages will form the required order.
>
> This can, however, lead to situations were the high-order request (e.g.
> order-2 required for the stack allocation during fork) will trigger
> OOM too early - e.g. after the first reclaim/compaction round. Such a
> system would have to be highly fragmented and there is no guarantee
> further reclaim/compaction attempts would help but at least make sure
> that the compaction was active before we go OOM and keep retrying even
> if should_reclaim_retry tells us to oom if
> 	- the last compaction round backed off or
> 	- we haven't completed at least MAX_COMPACT_RETRIES active
> 	  compaction rounds.
>
> The first rule ensures that the very last attempt for compaction
> was not ignored while the second guarantees that the compaction has done
> some work. Multiple retries might be needed to prevent occasional
> pigggy packing of other contexts to steal the compacted pages before
> the current context manages to retry to allocate them.
>
> compaction_failed() is taken as a final word from the compaction that
> the retry doesn't make much sense. We have to be careful though because
> the first compaction round is MIGRATE_ASYNC which is rather weak as it
> ignores pages under writeback and gives up too easily in other
> situations. We therefore have to make sure that MIGRATE_SYNC_LIGHT mode
> has been used before we give up. With this logic in place we do not have
> to increase the migration mode unconditionally and rather do it only if
> the compaction failed for the weaker mode. A nice side effect is that
> the stronger migration mode is used only when really needed so this has
> a potential of smaller latencies in some cases.
>
> Please note that the compaction doesn't tell us much about how
> successful it was when returning compaction_made_progress so we just
> have to blindly trust that another retry is worthwhile and cap the
> number to something reasonable to guarantee a convergence.
>
> If the given number of successful retries is not sufficient for a
> reasonable workloads we should focus on the collected compaction
> tracepoints data and try to address the issue in the compaction code.
> If this is not feasible we can increase the retries limit.
>
> Signed-off-by: Michal Hocko <mhocko@suse.com>

Looks good.

Acked-by: Vlastimil Babka <vbabka@suse.cz>

  parent reply	other threads:[~2016-04-11 14:48 UTC|newest]

Thread overview: 79+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-04-05 11:25 [PATCH 00/11] oom detection rework v5 Michal Hocko
2016-04-05 11:25 ` Michal Hocko
2016-04-05 11:25 ` [PATCH 01/11] mm, oom: rework oom detection Michal Hocko
2016-04-05 11:25   ` Michal Hocko
2016-04-05 11:25 ` [PATCH 02/11] mm: throttle on IO only when there are too many dirty and writeback pages Michal Hocko
2016-04-05 11:25   ` Michal Hocko
2016-04-05 11:25 ` [PATCH 03/11] mm, compaction: change COMPACT_ constants into enum Michal Hocko
2016-04-05 11:25   ` Michal Hocko
2016-04-05 11:25 ` [PATCH 04/11] mm, compaction: cover all compaction mode in compact_zone Michal Hocko
2016-04-05 11:25   ` Michal Hocko
2016-04-05 11:25 ` [PATCH 05/11] mm, compaction: distinguish COMPACT_DEFERRED from COMPACT_SKIPPED Michal Hocko
2016-04-05 11:25   ` Michal Hocko
2016-04-11 11:02   ` Vlastimil Babka
2016-04-11 11:02     ` Vlastimil Babka
2016-04-11 11:24     ` Michal Hocko
2016-04-11 11:24       ` Michal Hocko
2016-04-05 11:25 ` [PATCH 06/11] mm, compaction: distinguish between full and partial COMPACT_COMPLETE Michal Hocko
2016-04-05 11:25   ` Michal Hocko
2016-04-11 12:10   ` Vlastimil Babka
2016-04-11 12:10     ` Vlastimil Babka
2016-04-11 12:46     ` Michal Hocko
2016-04-11 12:46       ` Michal Hocko
2016-04-11 12:53       ` Vlastimil Babka
2016-04-11 12:53         ` Vlastimil Babka
2016-04-11 13:27         ` Michal Hocko
2016-04-11 13:27           ` Michal Hocko
2016-04-11 13:42           ` Vlastimil Babka
2016-04-11 13:42             ` Vlastimil Babka
2016-04-11 13:46             ` Michal Hocko
2016-04-11 13:46               ` Michal Hocko
2016-04-05 11:25 ` [PATCH 07/11] mm, compaction: Update compaction_result ordering Michal Hocko
2016-04-05 11:25   ` Michal Hocko
2016-04-11 12:16   ` Vlastimil Babka
2016-04-11 12:16     ` Vlastimil Babka
2016-04-05 11:25 ` [PATCH 08/11] mm, compaction: Simplify __alloc_pages_direct_compact feedback interface Michal Hocko
2016-04-05 11:25   ` Michal Hocko
2016-04-11 13:48   ` Vlastimil Babka
2016-04-11 13:48     ` Vlastimil Babka
2016-04-05 11:25 ` [PATCH 09/11] mm, compaction: Abstract compaction feedback to helpers Michal Hocko
2016-04-05 11:25   ` Michal Hocko
2016-04-05 23:58   ` Andrew Morton
2016-04-05 23:58     ` Andrew Morton
2016-04-06  0:55     ` Hugh Dickins
2016-04-06  0:55       ` Hugh Dickins
2016-04-06  9:26       ` Michal Hocko
2016-04-06  9:26         ` Michal Hocko
2016-04-06 17:46       ` Andrew Morton
2016-04-06 17:46         ` Andrew Morton
2016-04-11 14:39   ` Vlastimil Babka
2016-04-11 14:39     ` Vlastimil Babka
2016-04-11 15:14     ` Michal Hocko
2016-04-11 15:14       ` Michal Hocko
2016-04-11 15:33       ` Michal Hocko
2016-04-11 15:33         ` Michal Hocko
2016-04-12 11:53       ` Vlastimil Babka
2016-04-12 11:53         ` Vlastimil Babka
2016-04-12 12:23         ` Michal Hocko
2016-04-12 12:23           ` Michal Hocko
2016-04-11 15:40   ` Michal Hocko
2016-04-11 15:40     ` Michal Hocko
2016-04-11 16:07     ` [RFC PATCH] mm: use compaction feedback for thp backoff conditions Michal Hocko
2016-04-11 16:07       ` Michal Hocko
2016-04-12 11:54     ` [PATCH 09/11] mm, compaction: Abstract compaction feedback to helpers Vlastimil Babka
2016-04-12 11:54       ` Vlastimil Babka
2016-04-05 11:25 ` [PATCH 10/11] mm, oom: protect !costly allocations some more Michal Hocko
2016-04-05 11:25   ` Michal Hocko
2016-04-06  0:06   ` Andrew Morton
2016-04-06  0:06     ` Andrew Morton
2016-04-06  9:28     ` Michal Hocko
2016-04-06  9:28       ` Michal Hocko
2016-04-11 14:48   ` Vlastimil Babka [this message]
2016-04-11 14:48     ` Vlastimil Babka
2016-04-05 11:25 ` [PATCH 11/11] mm: consider compaction feedback also for costly allocation Michal Hocko
2016-04-05 11:25   ` Michal Hocko
2016-04-05 12:46   ` Michal Hocko
2016-04-11 15:07   ` Vlastimil Babka
2016-04-11 15:07     ` Vlastimil Babka
2016-04-05 12:47 ` [PATCH 00/11] oom detection rework v5 Michal Hocko
2016-04-05 12:47   ` Michal Hocko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=570BB948.6000900@suse.cz \
    --to=vbabka@suse.cz \
    --cc=akpm@linux-foundation.org \
    --cc=hannes@cmpxchg.org \
    --cc=hillf.zj@alibaba-inc.com \
    --cc=js1304@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@suse.de \
    --cc=mhocko@kernel.org \
    --cc=mhocko@suse.com \
    --cc=penguin-kernel@I-love.SAKURA.ne.jp \
    --cc=rientjes@google.com \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.