From: Mel Gorman <mgorman@techsingularity.net>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Linux-MM <linux-mm@kvack.org>,
David Rientjes <rientjes@google.com>,
Andrea Arcangeli <aarcange@redhat.com>,
Vlastimil Babka <vbabka@suse.cz>,
ying.huang@intel.com, kirill@shutemov.name,
Linux List Kernel Mailing <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 00/25] Increase success rates and reduce latency of compaction v2
Date: Tue, 8 Jan 2019 09:12:17 +0000 [thread overview]
Message-ID: <20190108091217.GL31517@techsingularity.net> (raw)
In-Reply-To: <20190107154354.b0805ca15767fc7ea9e37545@linux-foundation.org>
On Mon, Jan 07, 2019 at 03:43:54PM -0800, Andrew Morton wrote:
> On Fri, 4 Jan 2019 12:49:46 +0000 Mel Gorman <mgorman@techsingularity.net> wrote:
>
> > This series reduces scan rates and success rates of compaction, primarily
> > by using the free lists to shorten scans, better controlling of skip
> > information and whether multiple scanners can target the same block and
> > capturing pageblocks before being stolen by parallel requests. The series
> > is based on the 4.21/5.0 merge window after Andrew's tree had been merged.
> > It's known to rebase cleanly.
> >
> > ...
> >
> > include/linux/compaction.h | 3 +-
> > include/linux/gfp.h | 7 +-
> > include/linux/mmzone.h | 2 +
> > include/linux/sched.h | 4 +
> > kernel/sched/core.c | 3 +
> > mm/compaction.c | 1031 ++++++++++++++++++++++++++++++++++----------
> > mm/internal.h | 23 +-
> > mm/migrate.c | 2 +-
> > mm/page_alloc.c | 70 ++-
> > 9 files changed, 908 insertions(+), 237 deletions(-)
>
> Boy that's a lot of material.
It's unfortunate I know. It just turned out that there is a lot that had
to change to make the most important patches in the series work without
obvious side-effects.
> I just tossed it in there unread for
> now. Do you have any suggestions as to how we can move ahead with
> getting this appropriately reviewed and tested?
>
The main workloads that should see a difference are those that use
MADV_HUGEPAGE or change /sys/kernel/mm/transparent_hugepage/defrag. I'm
expecting MADV_HUGEPAGE is more common in practice. By default, there
should be little change as direct compaction is not used heavily for THP.
Although SLUB workloads might see a difference given a long enough uptime,
it will be relatively difficult to detect.
As this was partially motivated by the __GFP_THISNODE discussion, I
would like to hear from David if this series makes an impact, if any,
when starting Google workloads on a fragmented system.
Similarly, I would be interested in hearing if Andrea's KVM startup times
see any benefit. I'm expecting less here as I expect that workload is
still bound by reclaim thrashing the local node in reclaim. Still, a
confirmation would be nice and if there is any benefit then it's a plus
even if the workload gets reclaimed excessively.
Local tests didn't show up anything interesting *other* than what is
already in the changelogs as those workloads are specifically targetting
those paths. Intel LKP has not reported any regressions (functional or
performance) despite being on git.kernel.org for a few weeks. However,
as they are using default configurations, this is not much of a surprise.
Review is harder. Vlastimil would normally be the best fit as he has
worked on compaction but for him or for anyone else, I'm expecting they're
dealing with a backlog after the holidays. I know I still have to get
to Vlastimil's recent series on THP allocations so I'm guilty of the same
crime with respect to review.
--
Mel Gorman
SUSE Labs
next prev parent reply other threads:[~2019-01-08 9:12 UTC|newest]
Thread overview: 75+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-04 12:49 [PATCH 00/25] Increase success rates and reduce latency of compaction v2 Mel Gorman
2019-01-04 12:49 ` [PATCH 01/25] mm, compaction: Shrink compact_control Mel Gorman
2019-01-04 12:49 ` [PATCH 02/25] mm, compaction: Rearrange compact_control Mel Gorman
2019-01-04 12:49 ` [PATCH 03/25] mm, compaction: Remove last_migrated_pfn from compact_control Mel Gorman
2019-01-04 12:49 ` [PATCH 04/25] mm, compaction: Remove unnecessary zone parameter in some instances Mel Gorman
2019-01-15 11:43 ` Vlastimil Babka
2019-01-04 12:49 ` [PATCH 05/25] mm, compaction: Rename map_pages to split_map_pages Mel Gorman
2019-01-15 11:59 ` Vlastimil Babka
2019-01-04 12:49 ` [PATCH 06/25] mm, compaction: Skip pageblocks with reserved pages Mel Gorman
2019-01-15 12:10 ` Vlastimil Babka
2019-01-15 12:50 ` Mel Gorman
2019-01-16 9:42 ` Mel Gorman
2019-01-04 12:49 ` [PATCH 07/25] mm, migrate: Immediately fail migration of a page with no migration handler Mel Gorman
2019-01-04 12:49 ` [PATCH 08/25] mm, compaction: Always finish scanning of a full pageblock Mel Gorman
2019-01-04 12:49 ` [PATCH 09/25] mm, compaction: Use the page allocator bulk-free helper for lists of pages Mel Gorman
2019-01-15 12:39 ` Vlastimil Babka
2019-01-16 9:46 ` Mel Gorman
2019-01-04 12:49 ` [PATCH 10/25] mm, compaction: Ignore the fragmentation avoidance boost for isolation and compaction Mel Gorman
2019-01-15 13:18 ` Vlastimil Babka
2019-01-04 12:49 ` [PATCH 11/25] mm, compaction: Use free lists to quickly locate a migration source Mel Gorman
2019-01-16 13:15 ` Vlastimil Babka
2019-01-16 14:33 ` Mel Gorman
2019-01-16 15:00 ` Vlastimil Babka
2019-01-16 15:43 ` Mel Gorman
2019-01-04 12:49 ` [PATCH 12/25] mm, compaction: Keep migration source private to a single compaction instance Mel Gorman
2019-01-16 15:45 ` Vlastimil Babka
2019-01-16 16:15 ` Mel Gorman
2019-01-17 9:29 ` Vlastimil Babka
2019-01-17 9:40 ` Vlastimil Babka
2019-01-04 12:49 ` [PATCH 13/25] mm, compaction: Use free lists to quickly locate a migration target Mel Gorman
2019-01-17 14:36 ` Vlastimil Babka
2019-01-17 15:51 ` Mel Gorman
2019-01-04 12:50 ` [PATCH 14/25] mm, compaction: Avoid rescanning the same pageblock multiple times Mel Gorman
2019-01-17 15:16 ` Vlastimil Babka
2019-01-17 16:00 ` Mel Gorman
2019-01-04 12:50 ` [PATCH 15/25] mm, compaction: Finish pageblock scanning on contention Mel Gorman
2019-01-17 16:38 ` Vlastimil Babka
2019-01-17 17:11 ` Mel Gorman
2019-01-18 8:57 ` Vlastimil Babka
2019-01-04 12:50 ` [PATCH 16/25] mm, compaction: Check early for huge pages encountered by the migration scanner Mel Gorman
2019-01-17 17:01 ` Vlastimil Babka
2019-01-17 17:35 ` Mel Gorman
2019-01-04 12:50 ` [PATCH 17/25] mm, compaction: Keep cached migration PFNs synced for unusable pageblocks Mel Gorman
2019-01-17 17:17 ` Vlastimil Babka
2019-01-17 17:37 ` Mel Gorman
2019-01-04 12:50 ` [PATCH 18/25] mm, compaction: Rework compact_should_abort as compact_check_resched Mel Gorman
2019-01-17 17:27 ` Vlastimil Babka
2019-01-04 12:50 ` [PATCH 19/25] mm, compaction: Do not consider a need to reschedule as contention Mel Gorman
2019-01-17 17:33 ` Vlastimil Babka
2019-01-17 18:05 ` Mel Gorman
2019-01-04 12:50 ` [PATCH 20/25] mm, compaction: Reduce unnecessary skipping of migration target scanner Mel Gorman
2019-01-17 17:58 ` Vlastimil Babka
2019-01-17 19:39 ` Mel Gorman
2019-01-18 9:09 ` Vlastimil Babka
2019-01-04 12:50 ` [PATCH 21/25] mm, compaction: Round-robin the order while searching the free lists for a target Mel Gorman
2019-01-18 9:17 ` Vlastimil Babka
2019-01-04 12:50 ` [PATCH 22/25] mm, compaction: Sample pageblocks for free pages Mel Gorman
2019-01-18 10:38 ` Vlastimil Babka
2019-01-18 13:44 ` Mel Gorman
2019-01-04 12:50 ` [PATCH 23/25] mm, compaction: Be selective about what pageblocks to clear skip hints Mel Gorman
2019-01-18 12:55 ` Vlastimil Babka
2019-01-18 14:10 ` Mel Gorman
2019-01-04 12:50 ` [PATCH 24/25] mm, compaction: Capture a page under direct compaction Mel Gorman
2019-01-18 13:40 ` Vlastimil Babka
2019-01-18 14:39 ` Mel Gorman
2019-01-04 12:50 ` [PATCH 25/25] mm, compaction: Do not direct compact remote memory Mel Gorman
2019-01-18 13:51 ` Vlastimil Babka
2019-01-18 14:46 ` Mel Gorman
2019-01-07 23:43 ` [PATCH 00/25] Increase success rates and reduce latency of compaction v2 Andrew Morton
2019-01-08 9:12 ` Mel Gorman [this message]
2019-01-09 11:13 ` [PATCH] mm, compaction: Use free lists to quickly locate a migration target -fix Mel Gorman
2019-01-09 19:27 ` Andrew Morton
2019-01-09 21:26 ` Mel Gorman
2019-01-09 11:15 ` [PATCH] mm, compaction: Finish pageblock scanning on contention -fix Mel Gorman
2019-01-09 11:16 ` [PATCH] mm, compaction: Round-robin the order while searching the free lists for a target -fix Mel Gorman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190108091217.GL31517@techsingularity.net \
--to=mgorman@techsingularity.net \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=kirill@shutemov.name \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=rientjes@google.com \
--cc=vbabka@suse.cz \
--cc=ying.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.