From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
To: Minchan Kim <minchan@kernel.org>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
Subject: Re: [RFC][PATCHv2 8/8] zsmalloc: register a shrinker to trigger auto-compaction
Date: Wed, 17 Jun 2015 16:11:02 +0900 [thread overview]
Message-ID: <20150617071102.GA3517@swordfish> (raw)
In-Reply-To: <20150616144730.GD31387@blaptop>
Hello,
On (06/16/15 23:47), Minchan Kim wrote:
[..]
>
> I like the idea but still have a concern to lack of fragmented zspages
> during memory pressure because auto-compaction will prevent fragment
> most of time. Surely, using fragment space as buffer in heavy memory
> pressure is not intened design so it could be fragile but I'm afraid
> this feature might accelrate it and it ends up having a problem and
> change current behavior in zram as swap.
>
> I hope you test this feature with considering my concern.
> Of course, I will test it with enough time.
>
OK, to explore "compaction leaves no fragmentation in classes" I did some
heavy testing today -- parallel copy/remove of the linux kernel, git, glibc;
parallel builds (-j4), parallel clean ups (make clean); git gc, etc.
device's IO stats:
cat /sys/block/zram0/stata??
277050 0 2216400 1463 8442846 0 67542768 106536 0 107810 108146
device's MM stats:
cat /sys/block/zram0/mm_stata??
3095515136 2020518768 2057990144 0 2645716992 2030 182119
We auto-compacted (mostly auto-compacted, because I triggered manual compaction
less than 5 times) 182119 objects.
Now, during the compaction I also accounted the number of classes that ended up to
be 'fully compacted' (class->OBJ_ALLOCATED) == class->OBJ_USED) and 'partially
compacted'.
And the results (after 1377 compactions) are:
pool compaction nr:1377 (full:487 part:35498)
So, we 'fully compact'-ed only 487/(35498 + 487) == 0.0135
roughtly ~1.35%
This argument does not stand anymore. We leave 'holes' in classes in ~98% of the
cases.
code that I used to gather those stats:
---
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 55cfda8..894773a 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -253,6 +253,9 @@ struct zs_pool {
#ifdef CONFIG_ZSMALLOC_STAT
struct dentry *stat_dentry;
#endif
+ int compaction_nr;
+ long full_compact;
+ long part_compact;
};
/*
@@ -1717,6 +1720,7 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
struct zs_compact_control cc;
struct page *src_page;
struct page *dst_page = NULL;
+ bool compacted = false;
spin_lock(&class->lock);
while ((src_page = isolate_source_page(class))) {
@@ -1726,6 +1730,8 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
if (!zs_can_compact(class))
break;
+ compacted = true;
+
cc.index = 0;
cc.s_page = src_page;
@@ -1751,6 +1757,13 @@ out:
if (src_page)
putback_zspage(pool, class, src_page);
+ if (compacted) {
+ if (zs_stat_get(class, OBJ_ALLOCATED) == zs_stat_get(class, OBJ_USED))
+ pool->full_compact++;
+ else
+ pool->part_compact++;
+ }
+
spin_unlock(&class->lock);
}
@@ -1767,6 +1780,11 @@ unsigned long zs_compact(struct zs_pool *pool)
continue;
__zs_compact(pool, class);
}
+
+ pool->compaction_nr++;
+ pr_err("pool compaction nr:%d (full:%ld part:%ld)\n", pool->compaction_nr,
+ pool->full_compact, pool->part_compact);
+
return pool->num_migrated;
}
EXPORT_SYMBOL_GPL(zs_compact);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2015-06-17 7:10 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-05 12:03 [RFC][PATCHv2 0/8] introduce automatic pool compaction Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 1/8] zsmalloc: drop unused variable `nr_to_migrate' Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 2/8] zsmalloc: partial page ordering within a fullness_list Sergey Senozhatsky
2015-06-16 13:19 ` Minchan Kim
2015-06-16 14:30 ` Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 3/8] zsmalloc: lower ZS_ALMOST_FULL waterline Sergey Senozhatsky
2015-06-16 13:37 ` Minchan Kim
2015-06-16 14:35 ` Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 4/8] zsmalloc: always keep per-class stats Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 5/8] zsmalloc: introduce zs_can_compact() function Sergey Senozhatsky
2015-06-16 14:19 ` Minchan Kim
2015-06-16 14:41 ` Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 6/8] zsmalloc: cosmetic compaction code adjustments Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 7/8] zsmalloc/zram: move `num_migrated' to zs_pool Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 8/8] zsmalloc: register a shrinker to trigger auto-compaction Sergey Senozhatsky
2015-06-16 14:47 ` Minchan Kim
2015-06-16 15:45 ` Sergey Senozhatsky
2015-06-18 1:50 ` Minchan Kim
2015-06-18 2:41 ` Sergey Senozhatsky
2015-06-18 3:01 ` Sergey Senozhatsky
2015-06-18 3:46 ` Minchan Kim
2015-06-18 3:39 ` Minchan Kim
2015-06-18 3:58 ` Sergey Senozhatsky
2015-06-17 7:11 ` Sergey Senozhatsky [this message]
2015-06-10 0:04 ` [RFC][PATCHv2 0/8] introduce automatic pool compaction Minchan Kim
2015-06-10 0:07 ` Sergey Senozhatsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150617071102.GA3517@swordfish \
--to=sergey.senozhatsky.work@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=minchan@kernel.org \
--cc=sergey.senozhatsky@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).