* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Benjamin Herrenschmidt @ 2017-05-05 15:57 UTC (permalink / raw)
To: Michal Hocko
Cc: Balbir Singh, linux-mm, akpm, khandual, aneesh.kumar, paulmck,
srikar, haren, jglisse, mgorman, arbab, vbabka, cl
In-Reply-To: <20170505145238.GE31461@dhcp22.suse.cz>
On Fri, 2017-05-05 at 16:52 +0200, Michal Hocko wrote:
>
> This sounds pretty much like a HW specific details which is not the
> right criterion to design general CDM around.
Which is why I don't see what's the problem with simply making this
a hot-plugged NUMA node, since it's basically what it is with a
"different" kind of CPU, possibly covered with a CMA, which provides
both some isolation and the ability to do large physical allocations
for applications who chose to use the legacy programming interfaces and
manually control the memory.
Then, the "issues" with things like reclaim, autonuma can be handled
with policy tunables. Possibly node attributes.
It seems to me that such a model fits well in the picture where we are
heading not just with GPUs, but with OpenCAPI based memory, CCIX or
other similar technologies that can provide memory possibly with co-
located acceleration devices.
It also mostly already just work.
> So let me repeat the fundamental question. Is the only difference from
> cpuless nodes the fact that the node should be invisible to processes
> unless they specify an explicit node mask?
It would be *preferable* that it is.
It's not necessarily an absolute requirement as long as what lands
there can be kicked out. However the system would potentially be
performing poorly if too much unrelated stuff lands on the GPU memory
as it has a much higher latency.
Due to the nature of GPUs (and possibly other such accelerators but not
necessarily all of them), that memory is also more likely to fail. GPUs
crash often. However that isn't necessarily true of OpenCAPI devices or
CCIX.
This is the kind of attributes of the memory (quality ?) that can be
provided by the driver that is putting it online. We can then
orthogonally decide how we chose (or not) to take those into account,
either in the default mm algorithms or from explicit policy mechanisms
set from userspace, but the latter is often awkward and never done
right.
> If yes then we are talking
> about policy in the kernel and that sounds like a big no-no to me.
It makes sense to expose a concept of "characteristics" of a given
memory node that affect the various policies the user can set.
It makes sense to haveA "default" policy models selected.
Policies aren't always decided in the kernel indeed (though they are
more often than not, face it, most of the time, leaving it to userspace
results in things simply not working). However the mechanisms by which
the policy is applied are in the kernel.
> Moreover cpusets already support exclusive numa nodes AFAIR.
Which implies that the user would have to do epxlciit cpuset
manipulations for the system to work right ? Most user wouldn't and the
rsult is that most user would have badly working systems. That's almost
always what happens when we chose to bounce *all* policy decision to
the user without the kernel attempting to have some kind of semi-sane
default.
> I am either missing something important here, and the discussion so
far
> hasn't helped to be honest, or this whole CDM effort tries to build a
> generic interface around a _specific_ piece of HW.
No. You guys have just been sticking your head in the sand for month
for reasons I can't quite understand completely :-)
There is a definite direction out there for devices to participate in
cache coherency and to operate within user process MMU contexts. This
is what the GPUs on P9 will be doing via nvlink, but this will also be
possible with technologies like OpenCAPI, I believe CCIX, etc...
This is by no mean a special case.
> The matter is worse
> by the fact that the described usecases are so vague that it is hard to
> build a good picture whether this is generic enough that a new/different
> HW will still fit into this picture.
The GPU use case is rather trivial.
The end goal is to simply have accelerators transparently operate in
userspace context, along with the ability to migrate page to the memory
that is the most efficient for a given operation.
Thus for example, mmap a large file (page cache) and have the program
pass a pointer to that mmap to a GPU program that starts churning on
it.
In the specific GPU case, we have HW on the link telling us the pages
are pounded on remotely, allowing us to trigger migration toward GPU
memory (but the other way works too).
The problem with the HMM based approach is that it is based on
ZONE_DEVICE. This means "special" struct pages that aren't in LRU and
implies, at least that's my understanding, piles of special cases all
over the place to deal with them, along with various APIs etc... that
don't work with such pages.
So it makes it difficult to be able to pickup anything mapped into a
process address space, whether it is page cache pages, shared memory,
etc... and migrate it to GPU pages.
At least, that's my understanding and Jerome somewhat confirmed it,
we'd end up fighting an uphill battle dealing with all those special
cases. HMM is well suited for non-coherent systems with a distinct MMU
translation on the remote device.
This is why we think a NUMA based approach is a lot simpler. We start
by having the GPU memory be "normal" memory, and then we look at what
needs to be done to improve the default system behaviour and policies
to take into account it slightly different characteristics.
Ben.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v3 4/4] mm: Adaptive hash table scaling
From: Pasha Tatashin @ 2017-05-05 15:33 UTC (permalink / raw)
To: Michal Hocko; +Cc: Andrew Morton, linux-mm, sparclinux, linux-fsdevel, Al Viro
In-Reply-To: <20170505133029.GC31461@dhcp22.suse.cz>
On 05/05/2017 09:30 AM, Michal Hocko wrote:
> On Thu 04-05-17 14:28:51, Pasha Tatashin wrote:
>> BTW, I am OK with your patch on top of this "Adaptive hash table" patch, but
>> I do not know what high_limit should be from where HASH_ADAPT will kick in.
>> 128M sound reasonable to you?
>
> For simplicity I would just use it unconditionally when no high_limit is
> set. What would be the problem with that?
Sure, that sounds good.
If you look at current users
> (and there no new users emerging too often) then most of them just want
> _some_ scaling. The original one obviously doesn't scale with large
> machines. Are you OK to fold my change to your patch or you want me to
> send a separate patch? AFAIK Andrew hasn't posted this patch to Linus
> yet.
>
I would like a separate patch because mine has soaked in mm tree for a
while now.
Thank you,
Pasha
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Michal Hocko @ 2017-05-05 14:52 UTC (permalink / raw)
To: Benjamin Herrenschmidt
Cc: Balbir Singh, linux-mm, akpm, khandual, aneesh.kumar, paulmck,
srikar, haren, jglisse, mgorman, arbab, vbabka, cl
In-Reply-To: <1493912961.25766.379.camel@kernel.crashing.org>
On Thu 04-05-17 17:49:21, Benjamin Herrenschmidt wrote:
> On Thu, 2017-05-04 at 14:52 +0200, Michal Hocko wrote:
> > But the direct reclaim would be effective only _after_ all other nodes
> > are full.
> >
> > I thought that kswapd reclaim is a problem because the HW doesn't
> > support aging properly but as the direct reclaim works then what is the
> > actual problem?
>
> Ageing isn't isn't completely broken. The ATS MMU supports
> dirty/accessed just fine.
>
> However the TLB invalidations are quite expensive with a GPU so too
> much harvesting is detrimental, and the GPU tends to check pages out
> using a special "read with intend to write" mode, which means it almost
> always set the dirty bit if the page is writable to begin with.
This sounds pretty much like a HW specific details which is not the
right criterion to design general CDM around.
So let me repeat the fundamental question. Is the only difference from
cpuless nodes the fact that the node should be invisible to processes
unless they specify an explicit node mask? If yes then we are talking
about policy in the kernel and that sounds like a big no-no to me.
Moreover cpusets already support exclusive numa nodes AFAIR.
I am either missing something important here, and the discussion so far
hasn't helped to be honest, or this whole CDM effort tries to build a
generic interface around a _specific_ piece of HW. The matter is worse
by the fact that the described usecases are so vague that it is hard to
build a good picture whether this is generic enough that a new/different
HW will still fit into this picture.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Dave Hansen @ 2017-05-05 14:51 UTC (permalink / raw)
To: Balbir Singh, Benjamin Herrenschmidt, Michal Hocko
Cc: linux-mm, akpm, khandual, aneesh.kumar, paulmck, srikar, haren,
jglisse, mgorman, arbab, vbabka, cl
In-Reply-To: <1493954234.4227.1.camel@gmail.com>
On 05/04/2017 08:17 PM, Balbir Singh wrote:
>>> However the TLB invalidations are quite expensive with a GPU so too
>>> much harvesting is detrimental, and the GPU tends to check pages out
>>> using a special "read with intend to write" mode, which means it almost
>>> always set the dirty bit if the page is writable to begin with.
>> Why do you have to invalidate the TLB? Does the GPU have a TLB so large
>> that it can keep thing in the TLB for super-long periods of time?
>>
>> We don't flush the TLB on clearing Accessed on x86 normally.
> Isn't that mostly because x86 relies on non-global pages to be flushed
> on context switch?
Well, that's not the case with Process Context Identifiers. Somebody
will enable those some day. It also isn't true for a long-lived process
camping on a CPU core.
I don't know about "mostly", but it's certainly a combination of stuff
having to be reloaded in the TLB and flushed at context switch today.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH][RFC][V2] mm: make kswapd try harder to keep active pages in cache
From: Josef Bacik @ 2017-05-05 14:20 UTC (permalink / raw)
To: linux-mm, hannes, kernel-team, riel
When testing a slab heavy workload I noticed that we often would barely
reclaim anything at all from slab when kswapd started doing reclaim.
This is because we use the ratio of nr_scanned / nr_lru to determine how
much of slab we should reclaim. But in a slab only/mostly workload we
will not have much page cache to reclaim, and thus our ratio will be
really low and not at all related to where the memory on the system is.
Instead we want to use a ratio of the reclaimable slab to the actual
reclaimable space on the system. That way if we are slab heavy we work
harder to reclaim slab.
The other part of this that hurts is when we are running close to full
memory with our working set. If we start putting a lot of reclaimable
slab pressure on the system (think find /, or some other silliness), we
will happily evict the active pages over the slab cache. This is kind
of backwards as we want to do all that we can to keep the active working
set in memory, and instead evict these short lived objects. The same
thing occurs when say you do a yum update of a few packages while your
working set takes up most of RAM, you end up with inactive lists being
relatively small and so we reclaim active pages even though we could
reclaim these short lived inactive pages.
My approach here is twofold. First, keep track of the difference in
inactive and slab pages since the last time kswapd ran. In the first
run this will just be the overall counts of inactive and slab, but for
each subsequent run we'll have a good idea of where the memory pressure
is coming from. Then we use this information to put pressure on either
the inactive lists or the slab caches, depending on where the pressure
is coming from.
If this optimization does not work, then we fall back to the previous
methods of reclaiming space with a slight adjustment. Instead of using
the overall scan rate of page cache to determine the scan rate for slab,
we instead use the total usage of slab compared to the reclaimable page
cache on the box. This will allow us to put an appropriate amount of
pressure on the slab shrinkers if we are a mostly slab workload.
I have two tests I was using to watch either side of this problem. The
first test kept 2 files that took up 3/4 of the memory, and then started
creating a bunch of empty files. Without this patch we would have to
re-read both files in their entirety at least 3 times during the run.
With this patch the active pages are never evicted.
The second test was a test that would read and stat all the files in a
directory, which again would take up about 3/4 of the memory with slab
cache. Then I cat'ed a 100gib file into /dev/null and checked to see if
any of the files were evicted and verified that none of the files were
evicted.
Signed-off-by: Josef Bacik <jbacik@fb.com>
---
V1->V2:
- brought back an earlier idea I had about tracking where the pressure was
coming from and applying pressure to the shrinker based on where we were
adding new pages. This makes us less hard and fast about only shrinking the
inactive list the first go around and instead is based on the workload.
mm/vmscan.c | 194 +++++++++++++++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 171 insertions(+), 23 deletions(-)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index bc8031e..87a5cc4 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -105,11 +105,20 @@ struct scan_control {
/* One of the zones is ready for compaction */
unsigned int compaction_ready:1;
+ /* Only reclaim inactive page cache or slab. */
+ unsigned int inactive_only:1;
+
/* Incremented by the number of inactive pages that were scanned */
unsigned long nr_scanned;
/* Number of pages freed so far during a call to shrink_zones() */
unsigned long nr_reclaimed;
+
+ /* Number of inactive pages added since last kswapd run. */
+ unsigned long inactive_diff;
+
+ /* Number of slab pages added since last kswapd run. */
+ unsigned long slab_diff;
};
#ifdef ARCH_HAS_PREFETCH
@@ -309,7 +318,8 @@ EXPORT_SYMBOL(unregister_shrinker);
static unsigned long do_shrink_slab(struct shrink_control *shrinkctl,
struct shrinker *shrinker,
unsigned long nr_scanned,
- unsigned long nr_eligible)
+ unsigned long nr_eligible,
+ unsigned long *slab_scanned)
{
unsigned long freed = 0;
unsigned long long delta;
@@ -410,6 +420,9 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl,
next_deferred -= scanned;
else
next_deferred = 0;
+ if (slab_scanned)
+ (*slab_scanned) += scanned;
+
/*
* move the unused scan count back into the shrinker in a
* manner that handles concurrent updates. If we exhausted the
@@ -456,7 +469,8 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl,
static unsigned long shrink_slab(gfp_t gfp_mask, int nid,
struct mem_cgroup *memcg,
unsigned long nr_scanned,
- unsigned long nr_eligible)
+ unsigned long nr_eligible,
+ unsigned long *slab_scanned)
{
struct shrinker *shrinker;
unsigned long freed = 0;
@@ -464,9 +478,6 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int nid,
if (memcg && (!memcg_kmem_enabled() || !mem_cgroup_online(memcg)))
return 0;
- if (nr_scanned == 0)
- nr_scanned = SWAP_CLUSTER_MAX;
-
if (!down_read_trylock(&shrinker_rwsem)) {
/*
* If we would return 0, our callers would understand that we
@@ -497,7 +508,8 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int nid,
if (!(shrinker->flags & SHRINKER_NUMA_AWARE))
sc.nid = 0;
- freed += do_shrink_slab(&sc, shrinker, nr_scanned, nr_eligible);
+ freed += do_shrink_slab(&sc, shrinker, nr_scanned, nr_eligible,
+ slab_scanned);
}
up_read(&shrinker_rwsem);
@@ -516,7 +528,7 @@ void drop_slab_node(int nid)
freed = 0;
do {
freed += shrink_slab(GFP_KERNEL, nid, memcg,
- 1000, 1000);
+ 1000, 1000, NULL);
} while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)) != NULL);
} while (freed > 10);
}
@@ -2100,6 +2112,7 @@ enum scan_balance {
SCAN_FRACT,
SCAN_ANON,
SCAN_FILE,
+ SCAN_INACTIVE,
};
/*
@@ -2148,6 +2161,11 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
if (!global_reclaim(sc))
force_scan = true;
+ if (sc->inactive_only) {
+ scan_balance = SCAN_INACTIVE;
+ goto out;
+ }
+
/* If we have no swap space, do not bother scanning anon pages. */
if (!sc->may_swap || mem_cgroup_get_nr_swap_pages(memcg) <= 0) {
scan_balance = SCAN_FILE;
@@ -2312,6 +2330,15 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
scan = 0;
}
break;
+ case SCAN_INACTIVE:
+ if (file && !is_active_lru(lru)) {
+ if (scan && size > sc->nr_to_reclaim)
+ scan = sc->nr_to_reclaim;
+ } else {
+ size = 0;
+ scan = 0;
+ }
+ break;
default:
/* Look ma, no brain */
BUG();
@@ -2536,8 +2563,62 @@ static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc)
{
struct reclaim_state *reclaim_state = current->reclaim_state;
unsigned long nr_reclaimed, nr_scanned;
+ unsigned long nr_reclaim, nr_slab, total_high_wmark = 0, nr_inactive;
+ int z;
bool reclaimable = false;
+ bool skip_slab = false;
+
+ nr_slab = sum_zone_node_page_state(pgdat->node_id,
+ NR_SLAB_RECLAIMABLE);
+ nr_inactive = node_page_state(pgdat, NR_INACTIVE_FILE);
+ nr_reclaim = pgdat_reclaimable_pages(pgdat);
+
+ for (z = 0; z < MAX_NR_ZONES; z++) {
+ struct zone *zone = &pgdat->node_zones[z];
+ if (!managed_zone(zone))
+ continue;
+ total_high_wmark += high_wmark_pages(zone);
+ }
+
+ /*
+ * If we don't have a lot of inactive or slab pages then there's no
+ * point in trying to free them exclusively, do the normal scan stuff.
+ */
+ if (nr_inactive < total_high_wmark && nr_slab < total_high_wmark)
+ sc->inactive_only = 0;
+
+ /*
+ * We don't have historical information, we can't make good decisions
+ * about ratio's and where we should put pressure, so just apply
+ * pressure based on overall consumption ratios.
+ */
+ if (!sc->slab_diff && !sc->inactive_diff)
+ sc->inactive_only = 0;
+
+ /*
+ * We still want to slightly prefer slab over inactive, so if the
+ * inactive on this node is large enough and what is pushing us into
+ * reclaim terretitory then limit our flushing to the inactive list for
+ * the first go around.
+ *
+ * The idea is that with a memcg configured system we will still reclaim
+ * memcg aware shrinkers, which includes the super block shrinkers. So
+ * if our steady state is keeping fs objects in cache for our workload
+ * we'll still put a certain amount of pressure on them anyway. To
+ * avoid evicting things we actually care about we want to skip slab
+ * reclaim altogether.
+ *
+ * However we still want to account for slab and inactive growing at the
+ * same rate, so if that is the case just carry on shrinking inactive
+ * and slab together.
+ */
+ if (nr_inactive > total_high_wmark &&
+ sc->inactive_diff > sc->slab_diff) {
+ unsigned long tmp = sc->inactive_diff >> 1;
+ if (tmp >= sc->slab_diff)
+ skip_slab = true;
+ }
do {
struct mem_cgroup *root = sc->target_mem_cgroup;
struct mem_cgroup_reclaim_cookie reclaim = {
@@ -2545,6 +2626,8 @@ static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc)
.priority = sc->priority,
};
unsigned long node_lru_pages = 0;
+ unsigned long slab_reclaimed = 0;
+ unsigned long slab_scanned = 0;
struct mem_cgroup *memcg;
nr_reclaimed = sc->nr_reclaimed;
@@ -2568,10 +2651,27 @@ static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc)
shrink_node_memcg(pgdat, memcg, sc, &lru_pages);
node_lru_pages += lru_pages;
- if (memcg)
- shrink_slab(sc->gfp_mask, pgdat->node_id,
- memcg, sc->nr_scanned - scanned,
- lru_pages);
+ /*
+ * We don't want to put a lot of pressure on all of the
+ * slabs if a memcg is mostly full, so use the ratio of
+ * the lru size to the total reclaimable space on the
+ * system. If we have sc->inactive_only set then we
+ * want to use the ratio of the difference between the
+ * two since the last kswapd run so we apply pressure to
+ * the consumer appropriately.
+ */
+ if (memcg && !skip_slab) {
+ unsigned long numerator = lru_pages;
+ unsigned long denominator = nr_reclaim;
+ if (sc->inactive_only) {
+ numerator = sc->slab_diff;
+ denominator = sc->inactive_diff;
+ }
+ slab_reclaimed +=
+ shrink_slab(sc->gfp_mask, pgdat->node_id,
+ memcg, numerator, denominator,
+ &slab_scanned);
+ }
/* Record the group's reclaim efficiency */
vmpressure(sc->gfp_mask, memcg, false,
@@ -2595,14 +2695,18 @@ static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc)
}
} while ((memcg = mem_cgroup_iter(root, memcg, &reclaim)));
- /*
- * Shrink the slab caches in the same proportion that
- * the eligible LRU pages were scanned.
- */
- if (global_reclaim(sc))
- shrink_slab(sc->gfp_mask, pgdat->node_id, NULL,
- sc->nr_scanned - nr_scanned,
- node_lru_pages);
+ if (!skip_slab && global_reclaim(sc)) {
+ unsigned long numerator = nr_slab;
+ unsigned long denominator = nr_reclaim;
+ if (sc->inactive_only) {
+ numerator = sc->slab_diff;
+ denominator = sc->inactive_diff;
+ }
+ slab_reclaimed += shrink_slab(sc->gfp_mask,
+ pgdat->node_id, NULL,
+ numerator, denominator,
+ &slab_scanned);
+ }
if (reclaim_state) {
sc->nr_reclaimed += reclaim_state->reclaimed_slab;
@@ -2614,9 +2718,27 @@ static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc)
sc->nr_scanned - nr_scanned,
sc->nr_reclaimed - nr_reclaimed);
- if (sc->nr_reclaimed - nr_reclaimed)
+ if (sc->nr_reclaimed - nr_reclaimed) {
reclaimable = true;
+ } else if (sc->inactive_only && !skip_slab) {
+ unsigned long percent;
+ /*
+ * We didn't reclaim anything this go around, so the
+ * inactive list is likely spent. If we're reclaiming
+ * less than half of the objects in slab that we're
+ * scanning then just stop doing the inactive only scan.
+ * Otherwise ramp up the pressure on the slab caches
+ * hoping that eventually we'll start freeing enough
+ * objects to reclaim space.
+ */
+ percent = (slab_reclaimed * 100 / slab_scanned);
+ if (percent < 50)
+ sc->inactive_only = 0;
+ else
+ nr_slab <<= 1;
+ }
+ skip_slab = false;
} while (should_continue_reclaim(pgdat, sc->nr_reclaimed - nr_reclaimed,
sc->nr_scanned - nr_scanned, sc));
@@ -3197,7 +3319,8 @@ static bool kswapd_shrink_node(pg_data_t *pgdat,
* or lower is eligible for reclaim until at least one usable zone is
* balanced.
*/
-static int balance_pgdat(pg_data_t *pgdat, int order, int classzone_idx)
+static int balance_pgdat(pg_data_t *pgdat, int order, int classzone_idx,
+ unsigned long inactive_diff, unsigned long slab_diff)
{
int i;
unsigned long nr_soft_reclaimed;
@@ -3210,6 +3333,9 @@ static int balance_pgdat(pg_data_t *pgdat, int order, int classzone_idx)
.may_writepage = !laptop_mode,
.may_unmap = 1,
.may_swap = 1,
+ .inactive_only = 1,
+ .inactive_diff = inactive_diff,
+ .slab_diff = slab_diff,
};
count_vm_event(PAGEOUTRUN);
@@ -3412,7 +3538,7 @@ static int kswapd(void *p)
unsigned int alloc_order, reclaim_order, classzone_idx;
pg_data_t *pgdat = (pg_data_t*)p;
struct task_struct *tsk = current;
-
+ unsigned long nr_slab = 0, nr_inactive = 0;
struct reclaim_state reclaim_state = {
.reclaimed_slab = 0,
};
@@ -3442,6 +3568,7 @@ static int kswapd(void *p)
pgdat->kswapd_order = alloc_order = reclaim_order = 0;
pgdat->kswapd_classzone_idx = classzone_idx = 0;
for ( ; ; ) {
+ unsigned long slab_diff, inactive_diff;
bool ret;
kswapd_try_sleep:
@@ -3466,6 +3593,23 @@ static int kswapd(void *p)
continue;
/*
+ * We want to know where we're adding pages so we can make
+ * smarter decisions about where we're going to put pressure
+ * when shrinking.
+ */
+ slab_diff = sum_zone_node_page_state(pgdat->node_id,
+ NR_SLAB_RECLAIMABLE);
+ inactive_diff = node_page_state(pgdat, NR_INACTIVE_FILE);
+ if (nr_slab > slab_diff)
+ slab_diff = 0;
+ else
+ slab_diff -= nr_slab;
+ if (inactive_diff < nr_inactive)
+ inactive_diff = 0;
+ else
+ inactive_diff -= nr_inactive;
+
+ /*
* Reclaim begins at the requested order but if a high-order
* reclaim fails then kswapd falls back to reclaiming for
* order-0. If that happens, kswapd will consider sleeping
@@ -3475,7 +3619,11 @@ static int kswapd(void *p)
*/
trace_mm_vmscan_kswapd_wake(pgdat->node_id, classzone_idx,
alloc_order);
- reclaim_order = balance_pgdat(pgdat, alloc_order, classzone_idx);
+ reclaim_order = balance_pgdat(pgdat, alloc_order, classzone_idx,
+ inactive_diff, slab_diff);
+ nr_inactive = node_page_state(pgdat, NR_INACTIVE_FILE);
+ nr_slab = sum_zone_node_page_state(pgdat->node_id,
+ NR_SLAB_RECLAIMABLE);
if (reclaim_order < alloc_order)
goto kswapd_try_sleep;
--
2.7.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [PATCH] swap: add block io poll in swapin path
From: Jens Axboe @ 2017-05-05 13:47 UTC (permalink / raw)
To: Huang, Ying, Shaohua Li; +Cc: linux-mm, Andrew Morton, Kernel-team, Tim Chen
In-Reply-To: <878tmb265d.fsf@yhuang-dev.intel.com>
On 05/05/2017 12:02 AM, Huang, Ying wrote:
>> The hybrid polling could help. The default hybrid polling poll half
>> the average IO latency. But it will not work very well if the latency
>> becomes very big. The hybrid polling has an interface to allow
>> userspace to configure the poll threshold, but since the latency
>> varies from time to time, it would be very hard to set a single
>> threshold for all workloads.
>
> If my understanding were correct, the hybrid polling will insert some
> sleep before the polling, but will not restrict the duration of the
> polling itself. This helps CPU usage, but may not help much for very
> long latency. How about add another threshold to restrict the max
> polling time? For example, the sleep time + max polling time could be
> 1.5 * mean latency. So that most IO requests could be serviced by
> polling, and for very long latency, polling could be restricted to
> reduce CPU usage.
I don't think that's a bad idea at all, there's definitely room for
improvement on how long to sleep and when to completely stop. The stats
track min/avg/max for a given window of time, so it would not be too
hard to implement an appropriate backoff as well.
--
Jens Axboe
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v3 4/4] mm: Adaptive hash table scaling
From: Michal Hocko @ 2017-05-05 13:30 UTC (permalink / raw)
To: Pasha Tatashin
Cc: Andrew Morton, linux-mm, sparclinux, linux-fsdevel, Al Viro
In-Reply-To: <429c8506-c498-0599-4258-7bac947fe29c@oracle.com>
On Thu 04-05-17 14:28:51, Pasha Tatashin wrote:
> BTW, I am OK with your patch on top of this "Adaptive hash table" patch, but
> I do not know what high_limit should be from where HASH_ADAPT will kick in.
> 128M sound reasonable to you?
For simplicity I would just use it unconditionally when no high_limit is
set. What would be the problem with that? If you look at current users
(and there no new users emerging too often) then most of them just want
_some_ scaling. The original one obviously doesn't scale with large
machines. Are you OK to fold my change to your patch or you want me to
send a separate patch? AFAIK Andrew hasn't posted this patch to Linus
yet.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v3 4/4] mm: Adaptive hash table scaling
From: Michal Hocko @ 2017-05-05 13:29 UTC (permalink / raw)
To: Pasha Tatashin
Cc: Andrew Morton, linux-mm, sparclinux, linux-fsdevel, Al Viro
In-Reply-To: <40f72efa-3928-b3c6-acca-0740f1a15ba4@oracle.com>
On Thu 04-05-17 14:23:24, Pasha Tatashin wrote:
> Hi Michal,
>
> I do not really want to impose any hard limit, because I do not know what it
> should be.
>
> The owners of the subsystems that use these large hash table should make a
> call, and perhaps pass high_limit, if needed into alloc_large_system_hash().
Some of surely should. E.g. mount_hashtable resp. mountpoint_hashtable
really do not need a large hash AFAIU. On the other hand it is somehow
handy to scale dentry and inode hashes according to the amount of
memory. But the scale factor should be much slower than the current
upstream implementation. As I've said I do not want to judge your
scaling change. All I am saying that making it explicit is just _wrong_
because it a) doesn't cover all cases just the two you have noticed and
b) new users will most probably just copy&paste existing users so
chances are they will introduce the same large hashtables without a good
reason. I would even say that user shouldn't care about how the scaling
is implemented. There is a way to limit it and if there is no limit set
then just do whatever is appropriate.
>
> Previous growth rate was unacceptable, because in addition to allocating
> large tables (which is acceptable if we take a total system memory size), we
> also needed to zero that, and zeroing while we have only one CPU available
> was significantly reducing the boot time.
>
> Now, on 32T the hash table is 1G instead of 32G, so the call is 32 times
> faster to finish. While it is not a good idea to waste memory, both 1G and
> 32G is insignificant amount of memory compared to the total amount of such
> 32T systems (0.09% and 0.003% accordingly).
Try to think in terms of hashed objects. How many objects would we need
to hash? Also this might be not a significant portion of the memory but
it is still a memory which can be used for other purposes.
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] mm, sparsemem: break out of loops early
From: Kirill A. Shutemov @ 2017-05-05 13:16 UTC (permalink / raw)
To: Dave Hansen; +Cc: linux-kernel, akpm, linux-mm, kirill.shutemov
In-Reply-To: <20170504174434.C45A4735@viggo.jf.intel.com>
On Thu, May 04, 2017 at 10:44:34AM -0700, Dave Hansen wrote:
>
> From: Dave Hansen <dave.hansen@linux.intel.com>
>
> There are a number of times that we loop over NR_MEM_SECTIONS,
> looking for section_present() on each section. But, when we have
> very large physical address spaces (large MAX_PHYSMEM_BITS),
> NR_MEM_SECTIONS becomes very large, making the loops quite long.
>
> With MAX_PHYSMEM_BITS=46 and a section size of 128MB, the current
> loops are 512k iterations, which we barely notice on modern
> hardware. But, raising MAX_PHYSMEM_BITS higher (like we will see
> on systems that support 5-level paging) makes this 64x longer and
> we start to notice, especially on slower systems like simulators.
> A 10-second delay for 512k iterations is annoying. But, a 640-
> second delay is crippling.
>
> This does not help if we have extremely sparse physical address
> spaces, but those are quite rare. We expect that most of the
> "slow" systems where this matters will also be quite small and
> non-sparse.
>
> To fix this, we track the highest section we've ever encountered.
> This lets us know when we will *never* see another
> section_present(), and lets us break out of the loops earlier.
>
> Doing the whole for_each_present_section_nr() macro is probably
> overkill, but it will ensure that any future loop iterations that
> we grow are more likely to be correct.
>
> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Tested-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
It shaved almost 40 seconds from boot time in qemu with 5-level paging
enabled for me :)
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: RFC v2: post-init-read-only protection for data allocated dynamically
From: Igor Stoppa @ 2017-05-05 12:19 UTC (permalink / raw)
To: Michal Hocko; +Cc: linux-mm, linux-kernel, Dave Hansen
In-Reply-To: <20170504140126.GJ31540@dhcp22.suse.cz>
On 04/05/17 17:01, Michal Hocko wrote:
> On Thu 04-05-17 16:37:55, Igor Stoppa wrote:
[...]
>> The disadvantage is that anything can happen, undetected, while the seal
>> is lifted.
>
> Yes and I think this makes it basically pointless
ok, this goes a bit beyond what I had in mind initially, but I see your
point
[...]
> Just to make my proposal more clear. I suggest the following workflow
>
> cache = kmem_cache_create(foo, object_size, ..., SLAB_SEAL);
>
> obj = kmem_cache_alloc(cache, gfp_mask);
> init_obj(obj)
> [more allocations]
> kmem_cache_seal(cache);
In case one doesn't want the feature, at which point would it be disabled?
* not creating the slab
* not sealing it
* something else?
> All slab pages belonging to the cache would get write protection. All
> new allocations from this cache would go to new slab pages. Later
> kmem_cache_seal will write protect only those new pages.
ok
> The main discomfort with this approach is that you have to create those
> caches in advance, obviously. We could help by creating some general
> purpose caches for common sizes but this sound like an overkill to me.
> The caller will know which objects will need the protection so the
> appropriate cache can be created on demand. But this reall depends on
> potential users...
Yes, I provided a more detailed answer in another branch of this thread.
Right now I can answer only for what I have already looked into: SE
Linux policy DB and LSM Hooks, and they do not seem very large.
I do not expect a large footprint, overall, although there might be some
exception.
--
igor
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: RFC v2: post-init-read-only protection for data allocated dynamically
From: Igor Stoppa @ 2017-05-05 12:08 UTC (permalink / raw)
To: Dave Hansen, Michal Hocko; +Cc: linux-mm, linux-kernel
In-Reply-To: <361e39e9-517a-2fc2-016c-23f9359fef0a@intel.com>
On 04/05/17 20:24, Dave Hansen wrote:
> On 05/04/2017 07:01 AM, Michal Hocko wrote:
>> Just to make my proposal more clear. I suggest the following workflow
>>
>> cache = kmem_cache_create(foo, object_size, ..., SLAB_SEAL);
>>
>> obj = kmem_cache_alloc(cache, gfp_mask);
>> init_obj(obj)
>> [more allocations]
>> kmem_cache_seal(cache);
>>
>> All slab pages belonging to the cache would get write protection. All
>> new allocations from this cache would go to new slab pages. Later
>> kmem_cache_seal will write protect only those new pages.
>
> Igor, what sizes of objects are you after here, mostly?
Theoretically, anything, since I have not really looked in details into
all the various subsystems, however, taking a more pragmatical approach
and referring to SE Linux and LSM Hooks, which were my initial target,
For SE Linux, I'm taking as example the policy db [1]:
The sizes are mostly small-ish: from 4-6 bytes to 16-32, overall.
There are some exceptions: the main policydb structure is way larger,
but it's not supposed to be instantiated repeatedly.
For LSM Hooks, the sublists in that hydra which goes under the name of
struct security_hook_heads, which are of type struct security_hook_list,
so a handful of bytes for the generic element [2].
> I ask because slub, at least, doesn't work at all for objects
>> PAGE_SIZE. It just punts those to the page allocator. But, you
> _could_ still use vmalloc() for those.
I would be surprised to find many objects that are larger than PAGE_SIZE
and qqualify for post-init-read-only protection, even if the page size
was only 4kB.
>From that perspective, I'm more concerned about avoiding taking a lot of
pages and leaving them mostly unused.
[1] security/selinux/ss/policydb.h
[2] include/linux/lsm_hooks.h
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: RFC v2: post-init-read-only protection for data allocated dynamically
From: Igor Stoppa @ 2017-05-05 10:42 UTC (permalink / raw)
To: Laura Abbott, Michal Hocko
Cc: linux-mm, linux-kernel, kernel-hardening@lists.openwall.com
In-Reply-To: <a445774f-a307-25aa-d44e-c523a7a42da6@redhat.com>
On 04/05/17 19:49, Laura Abbott wrote:
> [adding kernel-hardening since I think there would be interest]
thank you, I overlooked this
> BPF takes the approach of calling set_memory_ro to mark regions as
> read only. I'm certainly over simplifying but it sounds like this
> is mostly a mechanism to have this happen mostly automatically.
> Can you provide any more details about tradeoffs of the two approaches?
I am not sure I understand the question ...
For what I can understand, the bpf is marking as read only something
that spans across various pages, which is fine.
The payload to be protected is already organized in such pages.
But in the case I have in mind, I have various, heterogeneous chunks of
data, coming from various subsystems, not necessarily page aligned.
And, even if they were page aligned, most likely they would be far
smaller than a page, even a 4k page.
The first problem I see, is how to compact them into pages, ensuring
that no rwdata manages to infiltrate the range.
The actual mechanism for marking pages as read only is not relevant at
this point, if I understand your question correctly, since set_memory_ro
is walking the pages it receives as parameter.
> arm and arm64 have the added complexity of using larger
> page sizes on the linear map so dynamic mapping/unmapping generally
> doesn't work.
Do you mean that a page could be 16MB and therefore it would not be
possible to get a smaller chunk?
> arm64 supports DEBUG_PAGEALLOC by mapping with only
> pages but this is generally only wanted as a debug mechanism.
> I don't know if you've given this any thought at all.
Since the beginning I have thought about this feature as an opt-in
feature. I am aware that it can have drawbacks, but I think it would be
valuable as debugging tool even where it's not feasible to keep it
always-on.
OTOH on certain systems it can be sufficiently appealing to be kept on,
even if it eats up some more memory.
If this doesn't answer your question, could you please detail it more?
---
thanks, igor
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: RFC v2: post-init-read-only protection for data allocated dynamically
From: Igor Stoppa @ 2017-05-05 8:53 UTC (permalink / raw)
To: Dave Hansen, Michal Hocko; +Cc: linux-mm, linux-kernel
In-Reply-To: <210752b7-1cbf-2ac3-9f9a-62536dfd24d8@intel.com>
On 04/05/17 17:30, Dave Hansen wrote:
> On 05/04/2017 01:17 AM, Igor Stoppa wrote:
>> Or, let me put it differently: my goal is to not fracture more pages
>> than needed.
>> It will probably require some profiling to figure out what is the
>> ballpark of the memory footprint.
>
> This is easy to say, but hard to do. What if someone loads a different
> set of LSMs, or uses a very different configuration? How could this
> possibly work generally without vastly over-reserving in most cases?
I am probably making some implicit assumptions.
Let me try to make them explicit and let's see if they survive public
scrutiny, and btw while writing it, I see that it probably won't :-S
Observations
------------
* The memory that might need sealing is less or equal to the total
r/w memory - whatever that might be.
* In practice only a subset of the r/w memory will qualify for sealing.
* The over-reserving might be abysmal, in terms of percentage of
actually used memory, but it might not affect too much the system, in
absolute terms.
* On my machine (Ubuntu 16.10 64bit):
~$ dmesg |grep Memory
[ 0.000000] Memory: 32662956K/33474640K available (8848K kernel
code, 1441K rwdata, 3828K rodata, 1552K init, 1296K bss, 811684K
reserved, 0K cma-reserved)
* This is the memory at boot, I am not sure what would be the right way
to get the same info at runtime.
Speculations
------------
* after loading enough modules, the rwdata is 2-3 times larger
* the amount of rwdata that can be converted to rodata is 50%;
this is purely a working assumption I am making, as I have no
measurement yet and needs to be revised.
* on a system like mine, it would mean 2-3 MB
Conclusions
-----------
* 2-3 MB with possibly 50% of utilization might be an acceptable
compromise for a distro - as user I probably wouldn't mind too much.
* if someone is not happy with the distro defaults, every major distro
provides means to reconfigure and rebuild its kernel (the expectation is
that the only distro users who are not happy are those who would
probably reconfigure the kernel anyway, like a data center)
* non distro-users, like mobile, embedded, IoT, etc would do
optimizations and tweaking also without this feature mandating it.
--
In my defense, I can only say that my idea for this feature was to make
it as opt-in, where if one chooses to enable it, it is known upfront
what it will entail.
Now we are talking about distros, with the feature enabled by default.
TBH I am not sure there even is a truly generic solution, because we are
talking about dynamically allocated data, where the amount is not known
upfront (if it was, probably the data would be static).
I have the impression that it's a situation like:
- efficient memory occupation
- no need for profiling
- non fragmented pages
Choose 2 of them.
Of course, there might be a better way, but I haven't found it yet,
other than the usual way out: make it a command line option and let the
unhappy user modify the command line that the bootloader passes to the
kernel.
[...]
> I'm starting with the assumption that a new zone isn't feasible. :)
I really have no bias: I have a problem and I am trying to solve it.
I think the improvement could be useful also for others.
If the problem can be solved in a better way than what I proposed, it is
still good for me.
---
igor
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Benjamin Herrenschmidt @ 2017-05-05 7:49 UTC (permalink / raw)
To: Dave Hansen, Michal Hocko, Balbir Singh
Cc: linux-mm, akpm, khandual, aneesh.kumar, paulmck, srikar, haren,
jglisse, mgorman, arbab, vbabka, cl
In-Reply-To: <ac379467-0f9b-bbb2-955e-e677d94200f1@intel.com>
On Thu, 2017-05-04 at 10:33 -0700, Dave Hansen wrote:
> > However the TLB invalidations are quite expensive with a GPU so too
> > much harvesting is detrimental, and the GPU tends to check pages out
> > using a special "read with intend to write" mode, which means it almost
> > always set the dirty bit if the page is writable to begin with.
>
> Why do you have to invalidate the TLB?A Does the GPU have a TLB so large
> that it can keep thing in the TLB for super-long periods of time?
>
> We don't flush the TLB on clearing Accessed on x86 normally.
We don't *have* to but there is no telling when it will get set again.
I always found the non-invalidation of the TLB for harvesting
"Accessed" on x86 chancy ... if a process pounds on a handful of pages
heavily, they never get seen as accessed, which is just plain weird.
But yes, we can do the same thing.
Cheers,
Ben.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v2 1/2] dax: prevent invalidation of mapped DAX entries
From: Jan Kara @ 2017-05-05 7:29 UTC (permalink / raw)
To: Ross Zwisler
Cc: Andrew Morton, linux-kernel, Alexander Viro, Alexey Kuznetsov,
Andrey Ryabinin, Anna Schumaker, Christoph Hellwig, Dan Williams,
Darrick J. Wong, Eric Van Hensbergen, Jan Kara, Jens Axboe,
Johannes Weiner, Konrad Rzeszutek Wilk, Latchesar Ionkov,
linux-cifs, linux-fsdevel, linux-mm, linux-nfs, linux-nvdimm,
Matthew Wilcox, Ron Minnich, samba-technical, Steve French,
Trond Myklebust, v9fs-developer
In-Reply-To: <20170504195910.11579-1-ross.zwisler@linux.intel.com>
On Thu 04-05-17 13:59:09, Ross Zwisler wrote:
> dax_invalidate_mapping_entry() currently removes DAX exceptional entries
> only if they are clean and unlocked. This is done via:
>
> invalidate_mapping_pages()
> invalidate_exceptional_entry()
> dax_invalidate_mapping_entry()
>
> However, for page cache pages removed in invalidate_mapping_pages() there
> is an additional criteria which is that the page must not be mapped. This
> is noted in the comments above invalidate_mapping_pages() and is checked in
> invalidate_inode_page().
>
> For DAX entries this means that we can can end up in a situation where a
> DAX exceptional entry, either a huge zero page or a regular DAX entry,
> could end up mapped but without an associated radix tree entry. This is
> inconsistent with the rest of the DAX code and with what happens in the
> page cache case.
>
> We aren't able to unmap the DAX exceptional entry because according to its
> comments invalidate_mapping_pages() isn't allowed to block, and
> unmap_mapping_range() takes a write lock on the mapping->i_mmap_rwsem.
>
> We could potentially do an rmap walk to see if each of the entries actually
> has any active mappings before we remove it, but this might end up being
> very expensive and doesn't currently look to be worth it.
>
> So, just remove dax_invalidate_mapping_entry() and leave the DAX entries in
> the radix tree.
>
> Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
> Fixes: c6dcf52c23d2 ("mm: Invalidate DAX radix tree entries only if appropriate")
> Reported-by: Jan Kara <jack@suse.cz>
> Reviewed-by: Jan Kara <jack@suse.cz>
> Cc: <stable@vger.kernel.org> [4.10+]
Ah, I've just sent out a series which contains these two patches and
another two patches which change the entry locking to fix the last spotted
race... So either just take my last two patches on top of these two or
take my series as a whole.
Honza
> ---
> fs/dax.c | 29 -----------------------------
> include/linux/dax.h | 1 -
> mm/truncate.c | 9 +++------
> 3 files changed, 3 insertions(+), 36 deletions(-)
>
> diff --git a/fs/dax.c b/fs/dax.c
> index 85abd74..166504c 100644
> --- a/fs/dax.c
> +++ b/fs/dax.c
> @@ -507,35 +507,6 @@ int dax_delete_mapping_entry(struct address_space *mapping, pgoff_t index)
> }
>
> /*
> - * Invalidate exceptional DAX entry if easily possible. This handles DAX
> - * entries for invalidate_inode_pages() so we evict the entry only if we can
> - * do so without blocking.
> - */
> -int dax_invalidate_mapping_entry(struct address_space *mapping, pgoff_t index)
> -{
> - int ret = 0;
> - void *entry, **slot;
> - struct radix_tree_root *page_tree = &mapping->page_tree;
> -
> - spin_lock_irq(&mapping->tree_lock);
> - entry = __radix_tree_lookup(page_tree, index, NULL, &slot);
> - if (!entry || !radix_tree_exceptional_entry(entry) ||
> - slot_locked(mapping, slot))
> - goto out;
> - if (radix_tree_tag_get(page_tree, index, PAGECACHE_TAG_DIRTY) ||
> - radix_tree_tag_get(page_tree, index, PAGECACHE_TAG_TOWRITE))
> - goto out;
> - radix_tree_delete(page_tree, index);
> - mapping->nrexceptional--;
> - ret = 1;
> -out:
> - spin_unlock_irq(&mapping->tree_lock);
> - if (ret)
> - dax_wake_mapping_entry_waiter(mapping, index, entry, true);
> - return ret;
> -}
> -
> -/*
> * Invalidate exceptional DAX entry if it is clean.
> */
> int dax_invalidate_mapping_entry_sync(struct address_space *mapping,
> diff --git a/include/linux/dax.h b/include/linux/dax.h
> index d8a3dc0..f8e1833 100644
> --- a/include/linux/dax.h
> +++ b/include/linux/dax.h
> @@ -41,7 +41,6 @@ ssize_t dax_iomap_rw(struct kiocb *iocb, struct iov_iter *iter,
> int dax_iomap_fault(struct vm_fault *vmf, enum page_entry_size pe_size,
> const struct iomap_ops *ops);
> int dax_delete_mapping_entry(struct address_space *mapping, pgoff_t index);
> -int dax_invalidate_mapping_entry(struct address_space *mapping, pgoff_t index);
> int dax_invalidate_mapping_entry_sync(struct address_space *mapping,
> pgoff_t index);
> void dax_wake_mapping_entry_waiter(struct address_space *mapping,
> diff --git a/mm/truncate.c b/mm/truncate.c
> index 6263aff..c537184 100644
> --- a/mm/truncate.c
> +++ b/mm/truncate.c
> @@ -67,17 +67,14 @@ static void truncate_exceptional_entry(struct address_space *mapping,
>
> /*
> * Invalidate exceptional entry if easily possible. This handles exceptional
> - * entries for invalidate_inode_pages() so for DAX it evicts only unlocked and
> - * clean entries.
> + * entries for invalidate_inode_pages().
> */
> static int invalidate_exceptional_entry(struct address_space *mapping,
> pgoff_t index, void *entry)
> {
> - /* Handled by shmem itself */
> - if (shmem_mapping(mapping))
> + /* Handled by shmem itself, or for DAX we do nothing. */
> + if (shmem_mapping(mapping) || dax_mapping(mapping))
> return 1;
> - if (dax_mapping(mapping))
> - return dax_invalidate_mapping_entry(mapping, index);
> clear_shadow_entry(mapping, index, entry);
> return 1;
> }
> --
> 2.9.3
>
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] swap: add block io poll in swapin path
From: Huang, Ying @ 2017-05-05 6:02 UTC (permalink / raw)
To: Shaohua Li
Cc: Huang, Ying, linux-mm, Andrew Morton, Kernel-team, Tim Chen,
Jens Axboe
In-Reply-To: <20170505051218.GA50755@dhcp-172-20-191-107.dhcp.thefacebook.com>
Shaohua Li <shli@fb.com> writes:
> On Fri, May 05, 2017 at 11:07:54AM +0800, Huang, Ying wrote:
>> Hi, Shaohua,
>>
>> Shaohua Li <shli@fb.com> writes:
>>
>> > For fast flash disk, async IO could introduce overhead because of
>> > context switch. block-mq now supports IO poll, which improves
>> > performance and latency a lot. swapin is a good place to use this
>> > technique, because the task is waitting for the swapin page to continue
>> > execution.
>> >
>> > In my virtual machine, directly read 4k data from a NVMe with iopoll is
>> > about 60% better than that without poll. With iopoll support in swapin
>> > patch, my microbenchmark (a task does random memory write) is about 10%
>> > ~ 25% faster.
>>
>> How many concurrent processes/threads for memory writing in your test?
>
> I tried 1 thread and 8 threads
If we can measure the distribution of the latency during the test, it
may be helpful for us to evaluate whether my question below is important
or not.
pmbench (https://sourceforge.net/projects/pmbench/) may be helpful here.
>> In general, I think polling is a good way to reduce swap in latency for
>> high speed NVMe disk.
>>
>> I have a question. If the load of NVMe disk is high, for example, there
>> is quite some swap out occurs at the same time of swap in, the latency
>> of swap in may be much higher too. Then, under the max swap in latency,
>> the overhead of polling may be high. For example, it may be not an
>> issue to pool for 10us, but it is more serious if we poll for 500us or
>> 1ms. Is there some way to resolve this? Can we set a threshold for
>> polling? Then if we poll for more than the threshold, we will go to
>> sleep.
>
> The hybrid polling could help. The default hybrid polling poll half the average
> IO latency. But it will not work very well if the latency becomes very big. The
> hybrid polling has an interface to allow userspace to configure the poll
> threshold, but since the latency varies from time to time, it would be very
> hard to set a single threshold for all workloads.
If my understanding were correct, the hybrid polling will insert some
sleep before the polling, but will not restrict the duration of the
polling itself. This helps CPU usage, but may not help much for very
long latency. How about add another threshold to restrict the max
polling time? For example, the sleep time + max polling time could be
1.5 * mean latency. So that most IO requests could be serviced by
polling, and for very long latency, polling could be restricted to
reduce CPU usage.
Best Regards,
Huang, Ying
> Thanks,
> Shaohua
>
>>
>> Best Regards,
>> Huang, Ying
>>
>> > CPU utilization increases a lot though, 2x and even 3x CPU
>> > utilization. This will depend on disk speed though. While iopoll in
>> > swapin isn't intended for all usage cases, it's a win for latency
>> > sensistive workloads with high speed swap disk. block layer has knob to
>> > control poll in runtime. If poll isn't enabled in block layer, there
>> > should be no noticeable change in swapin.
>> >
>> > The swapin readahead might read several pages in in the same time and
>> > form a big IO request. Since the IO will take longer time, it doesn't
>> > make sense to do poll, so the patch only does iopoll for single page
>> > swapin.
>> >
>> > Cc: Tim Chen <tim.c.chen@intel.com>
>> > Cc: Huang Ying <ying.huang@intel.com>
>> > Cc: Jens Axboe <axboe@fb.com>
>> > Signed-off-by: Shaohua Li <shli@fb.com>
>> > ---
>> > include/linux/swap.h | 5 +++--
>> > mm/madvise.c | 4 ++--
>> > mm/page_io.c | 20 ++++++++++++++++++--
>> > mm/swap_state.c | 10 ++++++----
>> > mm/swapfile.c | 2 +-
>> > 5 files changed, 30 insertions(+), 11 deletions(-)
>> >
>> > diff --git a/include/linux/swap.h b/include/linux/swap.h
>> > index ba58824..c589e6c 100644
>> > --- a/include/linux/swap.h
>> > +++ b/include/linux/swap.h
>> > @@ -331,7 +331,7 @@ extern void kswapd_stop(int nid);
>> > #include <linux/blk_types.h> /* for bio_end_io_t */
>> >
>> > /* linux/mm/page_io.c */
>> > -extern int swap_readpage(struct page *);
>> > +extern int swap_readpage(struct page *, bool do_poll);
>> > extern int swap_writepage(struct page *page, struct writeback_control *wbc);
>> > extern void end_swap_bio_write(struct bio *bio);
>> > extern int __swap_writepage(struct page *page, struct writeback_control *wbc,
>> > @@ -362,7 +362,8 @@ extern void free_page_and_swap_cache(struct page *);
>> > extern void free_pages_and_swap_cache(struct page **, int);
>> > extern struct page *lookup_swap_cache(swp_entry_t);
>> > extern struct page *read_swap_cache_async(swp_entry_t, gfp_t,
>> > - struct vm_area_struct *vma, unsigned long addr);
>> > + struct vm_area_struct *vma, unsigned long addr,
>> > + bool do_poll);
>> > extern struct page *__read_swap_cache_async(swp_entry_t, gfp_t,
>> > struct vm_area_struct *vma, unsigned long addr,
>> > bool *new_page_allocated);
>> > diff --git a/mm/madvise.c b/mm/madvise.c
>> > index 25b78ee..8eda184 100644
>> > --- a/mm/madvise.c
>> > +++ b/mm/madvise.c
>> > @@ -205,7 +205,7 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned long start,
>> > continue;
>> >
>> > page = read_swap_cache_async(entry, GFP_HIGHUSER_MOVABLE,
>> > - vma, index);
>> > + vma, index, false);
>> > if (page)
>> > put_page(page);
>> > }
>> > @@ -246,7 +246,7 @@ static void force_shm_swapin_readahead(struct vm_area_struct *vma,
>> > }
>> > swap = radix_to_swp_entry(page);
>> > page = read_swap_cache_async(swap, GFP_HIGHUSER_MOVABLE,
>> > - NULL, 0);
>> > + NULL, 0, false);
>> > if (page)
>> > put_page(page);
>> > }
>> > diff --git a/mm/page_io.c b/mm/page_io.c
>> > index 23f6d0d..464cf16 100644
>> > --- a/mm/page_io.c
>> > +++ b/mm/page_io.c
>> > @@ -117,6 +117,7 @@ static void swap_slot_free_notify(struct page *page)
>> > static void end_swap_bio_read(struct bio *bio)
>> > {
>> > struct page *page = bio->bi_io_vec[0].bv_page;
>> > + struct task_struct *waiter = bio->bi_private;
>> >
>> > if (bio->bi_error) {
>> > SetPageError(page);
>> > @@ -133,6 +134,7 @@ static void end_swap_bio_read(struct bio *bio)
>> > out:
>> > unlock_page(page);
>> > bio_put(bio);
>> > + wake_up_process(waiter);
>> > }
>> >
>> > int generic_swapfile_activate(struct swap_info_struct *sis,
>> > @@ -329,11 +331,13 @@ int __swap_writepage(struct page *page, struct writeback_control *wbc,
>> > return ret;
>> > }
>> >
>> > -int swap_readpage(struct page *page)
>> > +int swap_readpage(struct page *page, bool do_poll)
>> > {
>> > struct bio *bio;
>> > int ret = 0;
>> > struct swap_info_struct *sis = page_swap_info(page);
>> > + blk_qc_t qc;
>> > + struct block_device *bdev;
>> >
>> > VM_BUG_ON_PAGE(!PageSwapCache(page), page);
>> > VM_BUG_ON_PAGE(!PageLocked(page), page);
>> > @@ -372,9 +376,21 @@ int swap_readpage(struct page *page)
>> > ret = -ENOMEM;
>> > goto out;
>> > }
>> > + bdev = bio->bi_bdev;
>> > + bio->bi_private = current;
>> > bio_set_op_attrs(bio, REQ_OP_READ, 0);
>> > count_vm_event(PSWPIN);
>> > - submit_bio(bio);
>> > + qc = submit_bio(bio);
>> > + while (do_poll) {
>> > + set_current_state(TASK_UNINTERRUPTIBLE);
>> > + if (!PageLocked(page))
>> > + break;
>> > +
>> > + if (!blk_mq_poll(bdev_get_queue(bdev), qc))
>> > + break;
>> > + }
>> > + __set_current_state(TASK_RUNNING);
>> > +
>> > out:
>> > return ret;
>> > }
>> > diff --git a/mm/swap_state.c b/mm/swap_state.c
>> > index 539b888..7c0a66c 100644
>> > --- a/mm/swap_state.c
>> > +++ b/mm/swap_state.c
>> > @@ -404,14 +404,14 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
>> > * the swap entry is no longer in use.
>> > */
>> > struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
>> > - struct vm_area_struct *vma, unsigned long addr)
>> > + struct vm_area_struct *vma, unsigned long addr, bool do_poll)
>> > {
>> > bool page_was_allocated;
>> > struct page *retpage = __read_swap_cache_async(entry, gfp_mask,
>> > vma, addr, &page_was_allocated);
>> >
>> > if (page_was_allocated)
>> > - swap_readpage(retpage);
>> > + swap_readpage(retpage, do_poll);
>> >
>> > return retpage;
>> > }
>> > @@ -488,11 +488,13 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask,
>> > unsigned long start_offset, end_offset;
>> > unsigned long mask;
>> > struct blk_plug plug;
>> > + bool do_poll = true;
>> >
>> > mask = swapin_nr_pages(offset) - 1;
>> > if (!mask)
>> > goto skip;
>> >
>> > + do_poll = false;
>> > /* Read a page_cluster sized and aligned cluster around offset. */
>> > start_offset = offset & ~mask;
>> > end_offset = offset | mask;
>> > @@ -503,7 +505,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask,
>> > for (offset = start_offset; offset <= end_offset ; offset++) {
>> > /* Ok, do the async read-ahead now */
>> > page = read_swap_cache_async(swp_entry(swp_type(entry), offset),
>> > - gfp_mask, vma, addr);
>> > + gfp_mask, vma, addr, false);
>> > if (!page)
>> > continue;
>> > if (offset != entry_offset)
>> > @@ -514,7 +516,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask,
>> >
>> > lru_add_drain(); /* Push any new pages onto the LRU now */
>> > skip:
>> > - return read_swap_cache_async(entry, gfp_mask, vma, addr);
>> > + return read_swap_cache_async(entry, gfp_mask, vma, addr, do_poll);
>> > }
>> >
>> > int init_swap_address_space(unsigned int type, unsigned long nr_pages)
>> > diff --git a/mm/swapfile.c b/mm/swapfile.c
>> > index 4f6cba1..04516c1 100644
>> > --- a/mm/swapfile.c
>> > +++ b/mm/swapfile.c
>> > @@ -1719,7 +1719,7 @@ int try_to_unuse(unsigned int type, bool frontswap,
>> > swap_map = &si->swap_map[i];
>> > entry = swp_entry(type, i);
>> > page = read_swap_cache_async(entry,
>> > - GFP_HIGHUSER_MOVABLE, NULL, 0);
>> > + GFP_HIGHUSER_MOVABLE, NULL, 0, false);
>> > if (!page) {
>> > /*
>> > * Either swap_duplicate() failed because entry
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] swap: add block io poll in swapin path
From: Shaohua Li @ 2017-05-05 5:12 UTC (permalink / raw)
To: Huang, Ying; +Cc: linux-mm, Andrew Morton, Kernel-team, Tim Chen, Jens Axboe
In-Reply-To: <87shkk0zn9.fsf@yhuang-dev.intel.com>
On Fri, May 05, 2017 at 11:07:54AM +0800, Huang, Ying wrote:
> Hi, Shaohua,
>
> Shaohua Li <shli@fb.com> writes:
>
> > For fast flash disk, async IO could introduce overhead because of
> > context switch. block-mq now supports IO poll, which improves
> > performance and latency a lot. swapin is a good place to use this
> > technique, because the task is waitting for the swapin page to continue
> > execution.
> >
> > In my virtual machine, directly read 4k data from a NVMe with iopoll is
> > about 60% better than that without poll. With iopoll support in swapin
> > patch, my microbenchmark (a task does random memory write) is about 10%
> > ~ 25% faster.
>
> How many concurrent processes/threads for memory writing in your test?
I tried 1 thread and 8 threads
> In general, I think polling is a good way to reduce swap in latency for
> high speed NVMe disk.
>
> I have a question. If the load of NVMe disk is high, for example, there
> is quite some swap out occurs at the same time of swap in, the latency
> of swap in may be much higher too. Then, under the max swap in latency,
> the overhead of polling may be high. For example, it may be not an
> issue to pool for 10us, but it is more serious if we poll for 500us or
> 1ms. Is there some way to resolve this? Can we set a threshold for
> polling? Then if we poll for more than the threshold, we will go to
> sleep.
The hybrid polling could help. The default hybrid polling poll half the average
IO latency. But it will not work very well if the latency becomes very big. The
hybrid polling has an interface to allow userspace to configure the poll
threshold, but since the latency varies from time to time, it would be very
hard to set a single threshold for all workloads.
Thanks,
Shaohua
>
> Best Regards,
> Huang, Ying
>
> > CPU utilization increases a lot though, 2x and even 3x CPU
> > utilization. This will depend on disk speed though. While iopoll in
> > swapin isn't intended for all usage cases, it's a win for latency
> > sensistive workloads with high speed swap disk. block layer has knob to
> > control poll in runtime. If poll isn't enabled in block layer, there
> > should be no noticeable change in swapin.
> >
> > The swapin readahead might read several pages in in the same time and
> > form a big IO request. Since the IO will take longer time, it doesn't
> > make sense to do poll, so the patch only does iopoll for single page
> > swapin.
> >
> > Cc: Tim Chen <tim.c.chen@intel.com>
> > Cc: Huang Ying <ying.huang@intel.com>
> > Cc: Jens Axboe <axboe@fb.com>
> > Signed-off-by: Shaohua Li <shli@fb.com>
> > ---
> > include/linux/swap.h | 5 +++--
> > mm/madvise.c | 4 ++--
> > mm/page_io.c | 20 ++++++++++++++++++--
> > mm/swap_state.c | 10 ++++++----
> > mm/swapfile.c | 2 +-
> > 5 files changed, 30 insertions(+), 11 deletions(-)
> >
> > diff --git a/include/linux/swap.h b/include/linux/swap.h
> > index ba58824..c589e6c 100644
> > --- a/include/linux/swap.h
> > +++ b/include/linux/swap.h
> > @@ -331,7 +331,7 @@ extern void kswapd_stop(int nid);
> > #include <linux/blk_types.h> /* for bio_end_io_t */
> >
> > /* linux/mm/page_io.c */
> > -extern int swap_readpage(struct page *);
> > +extern int swap_readpage(struct page *, bool do_poll);
> > extern int swap_writepage(struct page *page, struct writeback_control *wbc);
> > extern void end_swap_bio_write(struct bio *bio);
> > extern int __swap_writepage(struct page *page, struct writeback_control *wbc,
> > @@ -362,7 +362,8 @@ extern void free_page_and_swap_cache(struct page *);
> > extern void free_pages_and_swap_cache(struct page **, int);
> > extern struct page *lookup_swap_cache(swp_entry_t);
> > extern struct page *read_swap_cache_async(swp_entry_t, gfp_t,
> > - struct vm_area_struct *vma, unsigned long addr);
> > + struct vm_area_struct *vma, unsigned long addr,
> > + bool do_poll);
> > extern struct page *__read_swap_cache_async(swp_entry_t, gfp_t,
> > struct vm_area_struct *vma, unsigned long addr,
> > bool *new_page_allocated);
> > diff --git a/mm/madvise.c b/mm/madvise.c
> > index 25b78ee..8eda184 100644
> > --- a/mm/madvise.c
> > +++ b/mm/madvise.c
> > @@ -205,7 +205,7 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned long start,
> > continue;
> >
> > page = read_swap_cache_async(entry, GFP_HIGHUSER_MOVABLE,
> > - vma, index);
> > + vma, index, false);
> > if (page)
> > put_page(page);
> > }
> > @@ -246,7 +246,7 @@ static void force_shm_swapin_readahead(struct vm_area_struct *vma,
> > }
> > swap = radix_to_swp_entry(page);
> > page = read_swap_cache_async(swap, GFP_HIGHUSER_MOVABLE,
> > - NULL, 0);
> > + NULL, 0, false);
> > if (page)
> > put_page(page);
> > }
> > diff --git a/mm/page_io.c b/mm/page_io.c
> > index 23f6d0d..464cf16 100644
> > --- a/mm/page_io.c
> > +++ b/mm/page_io.c
> > @@ -117,6 +117,7 @@ static void swap_slot_free_notify(struct page *page)
> > static void end_swap_bio_read(struct bio *bio)
> > {
> > struct page *page = bio->bi_io_vec[0].bv_page;
> > + struct task_struct *waiter = bio->bi_private;
> >
> > if (bio->bi_error) {
> > SetPageError(page);
> > @@ -133,6 +134,7 @@ static void end_swap_bio_read(struct bio *bio)
> > out:
> > unlock_page(page);
> > bio_put(bio);
> > + wake_up_process(waiter);
> > }
> >
> > int generic_swapfile_activate(struct swap_info_struct *sis,
> > @@ -329,11 +331,13 @@ int __swap_writepage(struct page *page, struct writeback_control *wbc,
> > return ret;
> > }
> >
> > -int swap_readpage(struct page *page)
> > +int swap_readpage(struct page *page, bool do_poll)
> > {
> > struct bio *bio;
> > int ret = 0;
> > struct swap_info_struct *sis = page_swap_info(page);
> > + blk_qc_t qc;
> > + struct block_device *bdev;
> >
> > VM_BUG_ON_PAGE(!PageSwapCache(page), page);
> > VM_BUG_ON_PAGE(!PageLocked(page), page);
> > @@ -372,9 +376,21 @@ int swap_readpage(struct page *page)
> > ret = -ENOMEM;
> > goto out;
> > }
> > + bdev = bio->bi_bdev;
> > + bio->bi_private = current;
> > bio_set_op_attrs(bio, REQ_OP_READ, 0);
> > count_vm_event(PSWPIN);
> > - submit_bio(bio);
> > + qc = submit_bio(bio);
> > + while (do_poll) {
> > + set_current_state(TASK_UNINTERRUPTIBLE);
> > + if (!PageLocked(page))
> > + break;
> > +
> > + if (!blk_mq_poll(bdev_get_queue(bdev), qc))
> > + break;
> > + }
> > + __set_current_state(TASK_RUNNING);
> > +
> > out:
> > return ret;
> > }
> > diff --git a/mm/swap_state.c b/mm/swap_state.c
> > index 539b888..7c0a66c 100644
> > --- a/mm/swap_state.c
> > +++ b/mm/swap_state.c
> > @@ -404,14 +404,14 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
> > * the swap entry is no longer in use.
> > */
> > struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
> > - struct vm_area_struct *vma, unsigned long addr)
> > + struct vm_area_struct *vma, unsigned long addr, bool do_poll)
> > {
> > bool page_was_allocated;
> > struct page *retpage = __read_swap_cache_async(entry, gfp_mask,
> > vma, addr, &page_was_allocated);
> >
> > if (page_was_allocated)
> > - swap_readpage(retpage);
> > + swap_readpage(retpage, do_poll);
> >
> > return retpage;
> > }
> > @@ -488,11 +488,13 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask,
> > unsigned long start_offset, end_offset;
> > unsigned long mask;
> > struct blk_plug plug;
> > + bool do_poll = true;
> >
> > mask = swapin_nr_pages(offset) - 1;
> > if (!mask)
> > goto skip;
> >
> > + do_poll = false;
> > /* Read a page_cluster sized and aligned cluster around offset. */
> > start_offset = offset & ~mask;
> > end_offset = offset | mask;
> > @@ -503,7 +505,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask,
> > for (offset = start_offset; offset <= end_offset ; offset++) {
> > /* Ok, do the async read-ahead now */
> > page = read_swap_cache_async(swp_entry(swp_type(entry), offset),
> > - gfp_mask, vma, addr);
> > + gfp_mask, vma, addr, false);
> > if (!page)
> > continue;
> > if (offset != entry_offset)
> > @@ -514,7 +516,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask,
> >
> > lru_add_drain(); /* Push any new pages onto the LRU now */
> > skip:
> > - return read_swap_cache_async(entry, gfp_mask, vma, addr);
> > + return read_swap_cache_async(entry, gfp_mask, vma, addr, do_poll);
> > }
> >
> > int init_swap_address_space(unsigned int type, unsigned long nr_pages)
> > diff --git a/mm/swapfile.c b/mm/swapfile.c
> > index 4f6cba1..04516c1 100644
> > --- a/mm/swapfile.c
> > +++ b/mm/swapfile.c
> > @@ -1719,7 +1719,7 @@ int try_to_unuse(unsigned int type, bool frontswap,
> > swap_map = &si->swap_map[i];
> > entry = swp_entry(type, i);
> > page = read_swap_cache_async(entry,
> > - GFP_HIGHUSER_MOVABLE, NULL, 0);
> > + GFP_HIGHUSER_MOVABLE, NULL, 0, false);
> > if (!page) {
> > /*
> > * Either swap_duplicate() failed because entry
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Balbir Singh @ 2017-05-05 3:17 UTC (permalink / raw)
To: Dave Hansen, Benjamin Herrenschmidt, Michal Hocko
Cc: linux-mm, akpm, khandual, aneesh.kumar, paulmck, srikar, haren,
jglisse, mgorman, arbab, vbabka, cl
In-Reply-To: <ac379467-0f9b-bbb2-955e-e677d94200f1@intel.com>
On Thu, 2017-05-04 at 10:33 -0700, Dave Hansen wrote:
> On 05/04/2017 08:49 AM, Benjamin Herrenschmidt wrote:
> > On Thu, 2017-05-04 at 14:52 +0200, Michal Hocko wrote:
> > > But the direct reclaim would be effective only _after_ all other nodes
> > > are full.
> > >
> > > I thought that kswapd reclaim is a problem because the HW doesn't
> > > support aging properly but as the direct reclaim works then what is the
> > > actual problem?
> >
> > Ageing isn't isn't completely broken. The ATS MMU supports
> > dirty/accessed just fine.
> >
> > However the TLB invalidations are quite expensive with a GPU so too
> > much harvesting is detrimental, and the GPU tends to check pages out
> > using a special "read with intend to write" mode, which means it almost
> > always set the dirty bit if the page is writable to begin with.
>
> Why do you have to invalidate the TLB? Does the GPU have a TLB so large
> that it can keep thing in the TLB for super-long periods of time?
>
> We don't flush the TLB on clearing Accessed on x86 normally.
Isn't that mostly because x86 relies on non-global pages to be flushed
on context switch?
Balbir Singh.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] swap: add block io poll in swapin path
From: Huang, Ying @ 2017-05-05 3:07 UTC (permalink / raw)
To: Shaohua Li
Cc: linux-mm, Andrew Morton, Kernel-team, Tim Chen, Huang Ying,
Jens Axboe
In-Reply-To: <7dd0349ba5d321af557d7a09e08610f2486ea29e.1493930299.git.shli@fb.com>
Hi, Shaohua,
Shaohua Li <shli@fb.com> writes:
> For fast flash disk, async IO could introduce overhead because of
> context switch. block-mq now supports IO poll, which improves
> performance and latency a lot. swapin is a good place to use this
> technique, because the task is waitting for the swapin page to continue
> execution.
>
> In my virtual machine, directly read 4k data from a NVMe with iopoll is
> about 60% better than that without poll. With iopoll support in swapin
> patch, my microbenchmark (a task does random memory write) is about 10%
> ~ 25% faster.
How many concurrent processes/threads for memory writing in your test?
In general, I think polling is a good way to reduce swap in latency for
high speed NVMe disk.
I have a question. If the load of NVMe disk is high, for example, there
is quite some swap out occurs at the same time of swap in, the latency
of swap in may be much higher too. Then, under the max swap in latency,
the overhead of polling may be high. For example, it may be not an
issue to pool for 10us, but it is more serious if we poll for 500us or
1ms. Is there some way to resolve this? Can we set a threshold for
polling? Then if we poll for more than the threshold, we will go to
sleep.
Best Regards,
Huang, Ying
> CPU utilization increases a lot though, 2x and even 3x CPU
> utilization. This will depend on disk speed though. While iopoll in
> swapin isn't intended for all usage cases, it's a win for latency
> sensistive workloads with high speed swap disk. block layer has knob to
> control poll in runtime. If poll isn't enabled in block layer, there
> should be no noticeable change in swapin.
>
> The swapin readahead might read several pages in in the same time and
> form a big IO request. Since the IO will take longer time, it doesn't
> make sense to do poll, so the patch only does iopoll for single page
> swapin.
>
> Cc: Tim Chen <tim.c.chen@intel.com>
> Cc: Huang Ying <ying.huang@intel.com>
> Cc: Jens Axboe <axboe@fb.com>
> Signed-off-by: Shaohua Li <shli@fb.com>
> ---
> include/linux/swap.h | 5 +++--
> mm/madvise.c | 4 ++--
> mm/page_io.c | 20 ++++++++++++++++++--
> mm/swap_state.c | 10 ++++++----
> mm/swapfile.c | 2 +-
> 5 files changed, 30 insertions(+), 11 deletions(-)
>
> diff --git a/include/linux/swap.h b/include/linux/swap.h
> index ba58824..c589e6c 100644
> --- a/include/linux/swap.h
> +++ b/include/linux/swap.h
> @@ -331,7 +331,7 @@ extern void kswapd_stop(int nid);
> #include <linux/blk_types.h> /* for bio_end_io_t */
>
> /* linux/mm/page_io.c */
> -extern int swap_readpage(struct page *);
> +extern int swap_readpage(struct page *, bool do_poll);
> extern int swap_writepage(struct page *page, struct writeback_control *wbc);
> extern void end_swap_bio_write(struct bio *bio);
> extern int __swap_writepage(struct page *page, struct writeback_control *wbc,
> @@ -362,7 +362,8 @@ extern void free_page_and_swap_cache(struct page *);
> extern void free_pages_and_swap_cache(struct page **, int);
> extern struct page *lookup_swap_cache(swp_entry_t);
> extern struct page *read_swap_cache_async(swp_entry_t, gfp_t,
> - struct vm_area_struct *vma, unsigned long addr);
> + struct vm_area_struct *vma, unsigned long addr,
> + bool do_poll);
> extern struct page *__read_swap_cache_async(swp_entry_t, gfp_t,
> struct vm_area_struct *vma, unsigned long addr,
> bool *new_page_allocated);
> diff --git a/mm/madvise.c b/mm/madvise.c
> index 25b78ee..8eda184 100644
> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -205,7 +205,7 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned long start,
> continue;
>
> page = read_swap_cache_async(entry, GFP_HIGHUSER_MOVABLE,
> - vma, index);
> + vma, index, false);
> if (page)
> put_page(page);
> }
> @@ -246,7 +246,7 @@ static void force_shm_swapin_readahead(struct vm_area_struct *vma,
> }
> swap = radix_to_swp_entry(page);
> page = read_swap_cache_async(swap, GFP_HIGHUSER_MOVABLE,
> - NULL, 0);
> + NULL, 0, false);
> if (page)
> put_page(page);
> }
> diff --git a/mm/page_io.c b/mm/page_io.c
> index 23f6d0d..464cf16 100644
> --- a/mm/page_io.c
> +++ b/mm/page_io.c
> @@ -117,6 +117,7 @@ static void swap_slot_free_notify(struct page *page)
> static void end_swap_bio_read(struct bio *bio)
> {
> struct page *page = bio->bi_io_vec[0].bv_page;
> + struct task_struct *waiter = bio->bi_private;
>
> if (bio->bi_error) {
> SetPageError(page);
> @@ -133,6 +134,7 @@ static void end_swap_bio_read(struct bio *bio)
> out:
> unlock_page(page);
> bio_put(bio);
> + wake_up_process(waiter);
> }
>
> int generic_swapfile_activate(struct swap_info_struct *sis,
> @@ -329,11 +331,13 @@ int __swap_writepage(struct page *page, struct writeback_control *wbc,
> return ret;
> }
>
> -int swap_readpage(struct page *page)
> +int swap_readpage(struct page *page, bool do_poll)
> {
> struct bio *bio;
> int ret = 0;
> struct swap_info_struct *sis = page_swap_info(page);
> + blk_qc_t qc;
> + struct block_device *bdev;
>
> VM_BUG_ON_PAGE(!PageSwapCache(page), page);
> VM_BUG_ON_PAGE(!PageLocked(page), page);
> @@ -372,9 +376,21 @@ int swap_readpage(struct page *page)
> ret = -ENOMEM;
> goto out;
> }
> + bdev = bio->bi_bdev;
> + bio->bi_private = current;
> bio_set_op_attrs(bio, REQ_OP_READ, 0);
> count_vm_event(PSWPIN);
> - submit_bio(bio);
> + qc = submit_bio(bio);
> + while (do_poll) {
> + set_current_state(TASK_UNINTERRUPTIBLE);
> + if (!PageLocked(page))
> + break;
> +
> + if (!blk_mq_poll(bdev_get_queue(bdev), qc))
> + break;
> + }
> + __set_current_state(TASK_RUNNING);
> +
> out:
> return ret;
> }
> diff --git a/mm/swap_state.c b/mm/swap_state.c
> index 539b888..7c0a66c 100644
> --- a/mm/swap_state.c
> +++ b/mm/swap_state.c
> @@ -404,14 +404,14 @@ struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
> * the swap entry is no longer in use.
> */
> struct page *read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask,
> - struct vm_area_struct *vma, unsigned long addr)
> + struct vm_area_struct *vma, unsigned long addr, bool do_poll)
> {
> bool page_was_allocated;
> struct page *retpage = __read_swap_cache_async(entry, gfp_mask,
> vma, addr, &page_was_allocated);
>
> if (page_was_allocated)
> - swap_readpage(retpage);
> + swap_readpage(retpage, do_poll);
>
> return retpage;
> }
> @@ -488,11 +488,13 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask,
> unsigned long start_offset, end_offset;
> unsigned long mask;
> struct blk_plug plug;
> + bool do_poll = true;
>
> mask = swapin_nr_pages(offset) - 1;
> if (!mask)
> goto skip;
>
> + do_poll = false;
> /* Read a page_cluster sized and aligned cluster around offset. */
> start_offset = offset & ~mask;
> end_offset = offset | mask;
> @@ -503,7 +505,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask,
> for (offset = start_offset; offset <= end_offset ; offset++) {
> /* Ok, do the async read-ahead now */
> page = read_swap_cache_async(swp_entry(swp_type(entry), offset),
> - gfp_mask, vma, addr);
> + gfp_mask, vma, addr, false);
> if (!page)
> continue;
> if (offset != entry_offset)
> @@ -514,7 +516,7 @@ struct page *swapin_readahead(swp_entry_t entry, gfp_t gfp_mask,
>
> lru_add_drain(); /* Push any new pages onto the LRU now */
> skip:
> - return read_swap_cache_async(entry, gfp_mask, vma, addr);
> + return read_swap_cache_async(entry, gfp_mask, vma, addr, do_poll);
> }
>
> int init_swap_address_space(unsigned int type, unsigned long nr_pages)
> diff --git a/mm/swapfile.c b/mm/swapfile.c
> index 4f6cba1..04516c1 100644
> --- a/mm/swapfile.c
> +++ b/mm/swapfile.c
> @@ -1719,7 +1719,7 @@ int try_to_unuse(unsigned int type, bool frontswap,
> swap_map = &si->swap_map[i];
> entry = swp_entry(type, i);
> page = read_swap_cache_async(entry,
> - GFP_HIGHUSER_MOVABLE, NULL, 0);
> + GFP_HIGHUSER_MOVABLE, NULL, 0, false);
> if (!page) {
> /*
> * Either swap_duplicate() failed because entry
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] swap: add block io poll in swapin path
From: Jens Axboe @ 2017-05-05 2:24 UTC (permalink / raw)
To: Chen, Tim C, Shaohua Li
Cc: linux-mm@kvack.org, Andrew Morton, Kernel-team@fb.com,
Huang, Ying
In-Reply-To: <045D8A5597B93E4EBEDDCBF1FC15F50935CE8E53@fmsmsx104.amr.corp.intel.com>
On 05/04/2017 05:23 PM, Chen, Tim C wrote:
>> -----Original Message-----
>> From: Jens Axboe [mailto:axboe@fb.com]
>> Sent: Thursday, May 04, 2017 2:29 PM
>> To: Shaohua Li
>> Cc: linux-mm@kvack.org; Andrew Morton; Kernel-team@fb.com; Chen, Tim C;
>> Huang, Ying
>> Subject: Re: [PATCH] swap: add block io poll in swapin path
>>
>> On 05/04/2017 03:27 PM, Shaohua Li wrote:
>>> On Thu, May 04, 2017 at 02:53:59PM -0600, Jens Axboe wrote:
>>>> On 05/04/2017 02:42 PM, Shaohua Li wrote:
>>>>> For fast flash disk, async IO could introduce overhead because of
>>>>> context switch. block-mq now supports IO poll, which improves
>>>>> performance and latency a lot. swapin is a good place to use this
>>>>> technique, because the task is waitting for the swapin page to
>>>>> continue execution.
>>>>
>>>> Nitfy!
>>>>
>>>>> In my virtual machine, directly read 4k data from a NVMe with iopoll
>>>>> is about 60% better than that without poll. With iopoll support in
>>>>> swapin patch, my microbenchmark (a task does random memory write) is
>>>>> about 10% ~ 25% faster. CPU utilization increases a lot though, 2x
>>>>> and even 3x CPU utilization. This will depend on disk speed though.
>>>>> While iopoll in swapin isn't intended for all usage cases, it's a
>>>>> win for latency sensistive workloads with high speed swap disk.
>>>>> block layer has knob to control poll in runtime. If poll isn't
>>>>> enabled in block layer, there should be no noticeable change in swapin.
>>>>
>>>> Did you try with hybrid polling enabled? We should be able to achieve
>>>> most of the latency win at much less CPU cost with that.
>>>
>>> Hybrid poll is much slower than classic in my test, I tried different settings.
>>> maybe because this is a vm though.
>>
>> It's probably a vm issue, I bet the timed sleep are just too slow to be useful in a
>> vm.
>>
>
> The speedup is quite nice. The high CPU utilization is somewhat of a
> concern. But this is directly proportional to the poll time or
> latency of the drive's response. The latest generation of SSD drive's
> latency is a factor of 7 or more compared to the previous one, so the
> poll time could go down quite a bit, depending on what drive you were
> using in your test.
That was my point with the hybrid comment. In hybrid mode, there's no
reason why we can't get the same latencies as pure polling, at a
drastically reduced overhead. The latencies of the drive should not
matter, as we use the actual completion times to decide how long to
sleep and spin.
There's room for a bit of improvement, though. We should be tracking the
time it takes to do sleep+wakeup, and factor that into our wait cycle.
Currently we just blindly use half the average completion time. But even
with that, testing by others have shown basically identical latencies
with hybrid polling, burning only half a core instead of a full one.
Compared to strict sync irq driven mode, that's still a bit higher in
terms of CPU, but not really that much.
--
Jens Axboe
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v10 4/6] mm: function to offer a page block on the free list
From: kbuild test robot @ 2017-05-05 0:21 UTC (permalink / raw)
To: Wei Wang
Cc: kbuild-all, virtio-dev, linux-kernel, qemu-devel, virtualization,
kvm, linux-mm, mst, david, dave.hansen, cornelia.huck, akpm,
mgorman, aarcange, amit.shah, pbonzini, liliang.opensource
In-Reply-To: <1493887815-6070-5-git-send-email-wei.w.wang@intel.com>
[-- Attachment #1: Type: text/plain, Size: 13834 bytes --]
Hi Wei,
[auto build test WARNING on linus/master]
[also build test WARNING on v4.11 next-20170504]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Wei-Wang/Extend-virtio-balloon-for-fast-de-inflating-fast-live-migration/20170505-052958
reproduce: make htmldocs
All warnings (new ones prefixed by >>):
WARNING: convert(1) not found, for SVG to PDF conversion install ImageMagick (https://www.imagemagick.org)
arch/x86/include/asm/uaccess_32.h:1: warning: no structured comments found
>> mm/page_alloc.c:4663: warning: No description found for parameter 'zone'
>> mm/page_alloc.c:4663: warning: No description found for parameter 'order'
>> mm/page_alloc.c:4663: warning: No description found for parameter 'migratetype'
>> mm/page_alloc.c:4663: warning: No description found for parameter 'page'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
include/net/cfg80211.h:1738: warning: No description found for parameter 'report_results'
include/net/cfg80211.h:1738: warning: Excess struct/union/enum/typedef member 'results_wk' description in 'cfg80211_sched_scan_request'
vim +/zone +4663 mm/page_alloc.c
4647 * Heuristically get a page block in the system that is unused.
4648 * It is possible that pages from the page block are used immediately after
4649 * report_unused_page_block() returns. It is the caller's responsibility
4650 * to either detect or prevent the use of such pages.
4651 *
4652 * The free list to check: zone->free_area[order].free_list[migratetype].
4653 *
4654 * If the caller supplied page block (i.e. **page) is on the free list, offer
4655 * the next page block on the list to the caller. Otherwise, offer the first
4656 * page block on the list.
4657 *
4658 * Return 0 when a page block is found on the caller specified free list.
4659 */
4660 int report_unused_page_block(struct zone *zone, unsigned int order,
4661 unsigned int migratetype, struct page **page)
4662 {
> 4663 struct zone *this_zone;
4664 struct list_head *this_list;
4665 int ret = 0;
4666 unsigned long flags;
4667
4668 /* Sanity check */
4669 if (zone == NULL || page == NULL || order >= MAX_ORDER ||
4670 migratetype >= MIGRATE_TYPES)
4671 return -EINVAL;
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 6564 bytes --]
^ permalink raw reply
* RE: [PATCH] swap: add block io poll in swapin path
From: Chen, Tim C @ 2017-05-04 23:23 UTC (permalink / raw)
To: Jens Axboe, Shaohua Li
Cc: linux-mm@kvack.org, Andrew Morton, Kernel-team@fb.com,
Huang, Ying
In-Reply-To: <196d941b-39cb-4526-1763-e480ba326a98@fb.com>
>-----Original Message-----
>From: Jens Axboe [mailto:axboe@fb.com]
>Sent: Thursday, May 04, 2017 2:29 PM
>To: Shaohua Li
>Cc: linux-mm@kvack.org; Andrew Morton; Kernel-team@fb.com; Chen, Tim C;
>Huang, Ying
>Subject: Re: [PATCH] swap: add block io poll in swapin path
>
>On 05/04/2017 03:27 PM, Shaohua Li wrote:
>> On Thu, May 04, 2017 at 02:53:59PM -0600, Jens Axboe wrote:
>>> On 05/04/2017 02:42 PM, Shaohua Li wrote:
>>>> For fast flash disk, async IO could introduce overhead because of
>>>> context switch. block-mq now supports IO poll, which improves
>>>> performance and latency a lot. swapin is a good place to use this
>>>> technique, because the task is waitting for the swapin page to
>>>> continue execution.
>>>
>>> Nitfy!
>>>
>>>> In my virtual machine, directly read 4k data from a NVMe with iopoll
>>>> is about 60% better than that without poll. With iopoll support in
>>>> swapin patch, my microbenchmark (a task does random memory write) is
>>>> about 10% ~ 25% faster. CPU utilization increases a lot though, 2x
>>>> and even 3x CPU utilization. This will depend on disk speed though.
>>>> While iopoll in swapin isn't intended for all usage cases, it's a
>>>> win for latency sensistive workloads with high speed swap disk.
>>>> block layer has knob to control poll in runtime. If poll isn't
>>>> enabled in block layer, there should be no noticeable change in swapin.
>>>
>>> Did you try with hybrid polling enabled? We should be able to achieve
>>> most of the latency win at much less CPU cost with that.
>>
>> Hybrid poll is much slower than classic in my test, I tried different settings.
>> maybe because this is a vm though.
>
>It's probably a vm issue, I bet the timed sleep are just too slow to be useful in a
>vm.
>
The speedup is quite nice.
The high CPU utilization is somewhat of a concern. But this is directly
proportional to the poll time or latency of the drive's response. The latest generation of
SSD drive's latency is a factor of 7 or more compared to the previous one, so the poll time
could go down quite a bit, depending on what drive you were using in your test.
What is the latency and the kind of drive you're using?
Tim
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] swap: add block io poll in swapin path
From: Jens Axboe @ 2017-05-04 21:29 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-mm, Andrew Morton, Kernel-team, Tim Chen, Huang Ying
In-Reply-To: <20170504212725.GA26681@MacBook-Pro.dhcp.thefacebook.com>
On 05/04/2017 03:27 PM, Shaohua Li wrote:
> On Thu, May 04, 2017 at 02:53:59PM -0600, Jens Axboe wrote:
>> On 05/04/2017 02:42 PM, Shaohua Li wrote:
>>> For fast flash disk, async IO could introduce overhead because of
>>> context switch. block-mq now supports IO poll, which improves
>>> performance and latency a lot. swapin is a good place to use this
>>> technique, because the task is waitting for the swapin page to continue
>>> execution.
>>
>> Nitfy!
>>
>>> In my virtual machine, directly read 4k data from a NVMe with iopoll is
>>> about 60% better than that without poll. With iopoll support in swapin
>>> patch, my microbenchmark (a task does random memory write) is about 10%
>>> ~ 25% faster. CPU utilization increases a lot though, 2x and even 3x CPU
>>> utilization. This will depend on disk speed though. While iopoll in
>>> swapin isn't intended for all usage cases, it's a win for latency
>>> sensistive workloads with high speed swap disk. block layer has knob to
>>> control poll in runtime. If poll isn't enabled in block layer, there
>>> should be no noticeable change in swapin.
>>
>> Did you try with hybrid polling enabled? We should be able to achieve
>> most of the latency win at much less CPU cost with that.
>
> Hybrid poll is much slower than classic in my test, I tried different settings.
> maybe because this is a vm though.
It's probably a vm issue, I bet the timed sleep are just too slow to be
useful in a vm.
--
Jens Axboe
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] swap: add block io poll in swapin path
From: Shaohua Li @ 2017-05-04 21:27 UTC (permalink / raw)
To: Jens Axboe; +Cc: linux-mm, Andrew Morton, Kernel-team, Tim Chen, Huang Ying
In-Reply-To: <b1fec49f-5e22-3d0c-1725-09625b3047b0@fb.com>
On Thu, May 04, 2017 at 02:53:59PM -0600, Jens Axboe wrote:
> On 05/04/2017 02:42 PM, Shaohua Li wrote:
> > For fast flash disk, async IO could introduce overhead because of
> > context switch. block-mq now supports IO poll, which improves
> > performance and latency a lot. swapin is a good place to use this
> > technique, because the task is waitting for the swapin page to continue
> > execution.
>
> Nitfy!
>
> > In my virtual machine, directly read 4k data from a NVMe with iopoll is
> > about 60% better than that without poll. With iopoll support in swapin
> > patch, my microbenchmark (a task does random memory write) is about 10%
> > ~ 25% faster. CPU utilization increases a lot though, 2x and even 3x CPU
> > utilization. This will depend on disk speed though. While iopoll in
> > swapin isn't intended for all usage cases, it's a win for latency
> > sensistive workloads with high speed swap disk. block layer has knob to
> > control poll in runtime. If poll isn't enabled in block layer, there
> > should be no noticeable change in swapin.
>
> Did you try with hybrid polling enabled? We should be able to achieve
> most of the latency win at much less CPU cost with that.
Hybrid poll is much slower than classic in my test, I tried different settings.
maybe because this is a vm though.
Thanks,
Shaohua
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] swap: add block io poll in swapin path
From: Jens Axboe @ 2017-05-04 20:53 UTC (permalink / raw)
To: Shaohua Li, linux-mm; +Cc: Andrew Morton, Kernel-team, Tim Chen, Huang Ying
In-Reply-To: <7dd0349ba5d321af557d7a09e08610f2486ea29e.1493930299.git.shli@fb.com>
On 05/04/2017 02:42 PM, Shaohua Li wrote:
> For fast flash disk, async IO could introduce overhead because of
> context switch. block-mq now supports IO poll, which improves
> performance and latency a lot. swapin is a good place to use this
> technique, because the task is waitting for the swapin page to continue
> execution.
Nitfy!
> In my virtual machine, directly read 4k data from a NVMe with iopoll is
> about 60% better than that without poll. With iopoll support in swapin
> patch, my microbenchmark (a task does random memory write) is about 10%
> ~ 25% faster. CPU utilization increases a lot though, 2x and even 3x CPU
> utilization. This will depend on disk speed though. While iopoll in
> swapin isn't intended for all usage cases, it's a win for latency
> sensistive workloads with high speed swap disk. block layer has knob to
> control poll in runtime. If poll isn't enabled in block layer, there
> should be no noticeable change in swapin.
Did you try with hybrid polling enabled? We should be able to achieve
most of the latency win at much less CPU cost with that.
--
Jens Axboe
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox