linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
To: Minchan Kim <minchan@kernel.org>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
Subject: Re: [RFC][PATCHv2 8/8] zsmalloc: register a shrinker to trigger auto-compaction
Date: Wed, 17 Jun 2015 16:11:02 +0900	[thread overview]
Message-ID: <20150617071102.GA3517@swordfish> (raw)
In-Reply-To: <20150616144730.GD31387@blaptop>

Hello,

On (06/16/15 23:47), Minchan Kim wrote:
[..]
> 
> I like the idea but still have a concern to lack of fragmented zspages
> during memory pressure because auto-compaction will prevent fragment
> most of time. Surely, using fragment space as buffer in heavy memory
> pressure is not intened design so it could be fragile but I'm afraid
> this feature might accelrate it and it ends up having a problem and
> change current behavior in zram as swap.
> 
> I hope you test this feature with considering my concern.
> Of course, I will test it with enough time.
> 

OK, to explore "compaction leaves no fragmentation in classes" I did some
heavy testing today -- parallel copy/remove of the linux kernel, git, glibc;
parallel builds (-j4), parallel clean ups (make clean); git gc, etc.


device's IO stats:
cat /sys/block/zram0/stata??
   277050        0  2216400     1463  8442846        0 67542768   106536        0   107810   108146

device's MM stats:
cat /sys/block/zram0/mm_stata??
 3095515136 2020518768 2057990144        0 2645716992     2030   182119


We auto-compacted (mostly auto-compacted, because I triggered manual compaction
less than 5 times)    182119  objects.


Now, during the compaction I also accounted the number of classes that ended up to
be 'fully compacted' (class->OBJ_ALLOCATED) == class->OBJ_USED) and 'partially
compacted'.


And the results (after 1377 compactions) are:

 pool compaction nr:1377 (full:487 part:35498)



So, we 'fully compact'-ed only 487/(35498 + 487) == 0.0135

roughtly ~1.35%

This argument does not stand anymore. We leave 'holes' in classes in ~98% of the
cases.



code that I used to gather those stats:

---

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 55cfda8..894773a 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -253,6 +253,9 @@ struct zs_pool {
 #ifdef CONFIG_ZSMALLOC_STAT
 	struct dentry		*stat_dentry;
 #endif
+	int			compaction_nr;
+	long			full_compact;
+	long			part_compact;
 };
 
 /*
@@ -1717,6 +1720,7 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
 	struct zs_compact_control cc;
 	struct page *src_page;
 	struct page *dst_page = NULL;
+	bool compacted = false;
 
 	spin_lock(&class->lock);
 	while ((src_page = isolate_source_page(class))) {
@@ -1726,6 +1730,8 @@ static void __zs_compact(struct zs_pool *pool, struct size_class *class)
 		if (!zs_can_compact(class))
 			break;
 
+		compacted = true;
+
 		cc.index = 0;
 		cc.s_page = src_page;
 
@@ -1751,6 +1757,13 @@ out:
 	if (src_page)
 		putback_zspage(pool, class, src_page);
 
+	if (compacted) {
+		if (zs_stat_get(class, OBJ_ALLOCATED) == zs_stat_get(class, OBJ_USED))
+			pool->full_compact++;
+		else
+			pool->part_compact++;
+	}
+
 	spin_unlock(&class->lock);
 }
 
@@ -1767,6 +1780,11 @@ unsigned long zs_compact(struct zs_pool *pool)
 			continue;
 		__zs_compact(pool, class);
 	}
+
+	pool->compaction_nr++;
+	pr_err("pool compaction nr:%d (full:%ld part:%ld)\n", pool->compaction_nr,
+			pool->full_compact, pool->part_compact);
+
 	return pool->num_migrated;
 }
 EXPORT_SYMBOL_GPL(zs_compact);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2015-06-17  7:10 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-06-05 12:03 [RFC][PATCHv2 0/8] introduce automatic pool compaction Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 1/8] zsmalloc: drop unused variable `nr_to_migrate' Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 2/8] zsmalloc: partial page ordering within a fullness_list Sergey Senozhatsky
2015-06-16 13:19   ` Minchan Kim
2015-06-16 14:30     ` Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 3/8] zsmalloc: lower ZS_ALMOST_FULL waterline Sergey Senozhatsky
2015-06-16 13:37   ` Minchan Kim
2015-06-16 14:35     ` Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 4/8] zsmalloc: always keep per-class stats Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 5/8] zsmalloc: introduce zs_can_compact() function Sergey Senozhatsky
2015-06-16 14:19   ` Minchan Kim
2015-06-16 14:41     ` Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 6/8] zsmalloc: cosmetic compaction code adjustments Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 7/8] zsmalloc/zram: move `num_migrated' to zs_pool Sergey Senozhatsky
2015-06-05 12:03 ` [RFC][PATCHv2 8/8] zsmalloc: register a shrinker to trigger auto-compaction Sergey Senozhatsky
2015-06-16 14:47   ` Minchan Kim
2015-06-16 15:45     ` Sergey Senozhatsky
2015-06-18  1:50       ` Minchan Kim
2015-06-18  2:41         ` Sergey Senozhatsky
2015-06-18  3:01           ` Sergey Senozhatsky
2015-06-18  3:46             ` Minchan Kim
2015-06-18  3:39           ` Minchan Kim
2015-06-18  3:58             ` Sergey Senozhatsky
2015-06-17  7:11     ` Sergey Senozhatsky [this message]
2015-06-10  0:04 ` [RFC][PATCHv2 0/8] introduce automatic pool compaction Minchan Kim
2015-06-10  0:07   ` Sergey Senozhatsky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150617071102.GA3517@swordfish \
    --to=sergey.senozhatsky.work@gmail.com \
    --cc=akpm@linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=minchan@kernel.org \
    --cc=sergey.senozhatsky@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).