From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
To: Minchan Kim <minchan@kernel.org>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>,
Joonsoo Kim <js1304@gmail.com>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
Subject: Re: [RFC][PATCH v3 1/5] mm/zsmalloc: introduce class auto-compaction
Date: Mon, 14 Mar 2016 16:41:59 +0900 [thread overview]
Message-ID: <20160314074159.GA542@swordfish> (raw)
In-Reply-To: <20160314061759.GC10675@bbox>
Hello Minchan,
On (03/14/16 15:17), Minchan Kim wrote:
[..]
> > demonstrates that class-896 has 12/26=46% of unused pages, class-2336 has
> > 1648/4932=33% of unused pages, etc. And the more classes we will have as
> > 'normal' classes (more than one object per-zspage) the bigger this problem
> > will grow. The existing compaction relies on a user space (user can trigger
> > compaction via `compact' zram's sysfs attr) or a shrinker; it does not
> > happen automatically.
> >
> > This patch introduces a 'watermark' value of unused pages and schedules a
> > compaction work on a per-class basis once class's fragmentation becomes
> > too big. So compaction is not performed in current I/O operation context,
> > but in workqueue workers later.
> >
> > The current watermark is set to 40% -- if class has 40+% of `freeable'
> > pages then compaction work will be scheduled.
>
> Could you explain why you select per-class watermark?
yes,
we do less work this way - scan and compact only one class, instead
of locking and compacting all of them; which sounds reasonable.
> Because my plan was we kick background work based on total fragmented memory
> (i.e., considering used_pages/allocated_pages < some threshold).
if we know that a particular class B is fragmented and the rest of them
are just fine, then we can compact only that class B, skipping extra job.
> IOW, if used_pages/allocated_pages is less than some ratio,
> we kick background job with marking index of size class just freed
> and then the job scans size_class from the index circulary.
>
> As well, we should put a upper bound to scan zspages to make it
> deterministic.
you mean that __zs_compact() instead of just checking per-class
zs_can_compact() should check global pool ratio and bail out if
compaction of class Z has dropped the overall fragmentation ratio
below some watermark?
my logic was that
-- suppose we have class A with fragmentation ratio 49% and class B
with 8% of wasted pages, so the overall pool fragmentation is
(50 + 10)/ 2 < 30%, while we still have almost 50% fragmented class.
if the aim is to reduce the memory wastage then per-class watermarks
seem to be more flexible.
> What do you think about it?
-ss
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
To: Minchan Kim <minchan@kernel.org>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>,
Joonsoo Kim <js1304@gmail.com>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
Subject: Re: [RFC][PATCH v3 1/5] mm/zsmalloc: introduce class auto-compaction
Date: Mon, 14 Mar 2016 16:41:59 +0900 [thread overview]
Message-ID: <20160314074159.GA542@swordfish> (raw)
In-Reply-To: <20160314061759.GC10675@bbox>
Hello Minchan,
On (03/14/16 15:17), Minchan Kim wrote:
[..]
> > demonstrates that class-896 has 12/26=46% of unused pages, class-2336 has
> > 1648/4932=33% of unused pages, etc. And the more classes we will have as
> > 'normal' classes (more than one object per-zspage) the bigger this problem
> > will grow. The existing compaction relies on a user space (user can trigger
> > compaction via `compact' zram's sysfs attr) or a shrinker; it does not
> > happen automatically.
> >
> > This patch introduces a 'watermark' value of unused pages and schedules a
> > compaction work on a per-class basis once class's fragmentation becomes
> > too big. So compaction is not performed in current I/O operation context,
> > but in workqueue workers later.
> >
> > The current watermark is set to 40% -- if class has 40+% of `freeable'
> > pages then compaction work will be scheduled.
>
> Could you explain why you select per-class watermark?
yes,
we do less work this way - scan and compact only one class, instead
of locking and compacting all of them; which sounds reasonable.
> Because my plan was we kick background work based on total fragmented memory
> (i.e., considering used_pages/allocated_pages < some threshold).
if we know that a particular class B is fragmented and the rest of them
are just fine, then we can compact only that class B, skipping extra job.
> IOW, if used_pages/allocated_pages is less than some ratio,
> we kick background job with marking index of size class just freed
> and then the job scans size_class from the index circulary.
>
> As well, we should put a upper bound to scan zspages to make it
> deterministic.
you mean that __zs_compact() instead of just checking per-class
zs_can_compact() should check global pool ratio and bail out if
compaction of class Z has dropped the overall fragmentation ratio
below some watermark?
my logic was that
-- suppose we have class A with fragmentation ratio 49% and class B
with 8% of wasted pages, so the overall pool fragmentation is
(50 + 10)/ 2 < 30%, while we still have almost 50% fragmented class.
if the aim is to reduce the memory wastage then per-class watermarks
seem to be more flexible.
> What do you think about it?
-ss
next prev parent reply other threads:[~2016-03-14 7:40 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-03-03 14:45 [RFC][PATCH v3 0/5] mm/zsmalloc: rework compaction and increase density Sergey Senozhatsky
2016-03-03 14:45 ` Sergey Senozhatsky
2016-03-03 14:45 ` [RFC][PATCH v3 1/5] mm/zsmalloc: introduce class auto-compaction Sergey Senozhatsky
2016-03-03 14:45 ` Sergey Senozhatsky
2016-03-14 6:17 ` Minchan Kim
2016-03-14 6:17 ` Minchan Kim
2016-03-14 7:41 ` Sergey Senozhatsky [this message]
2016-03-14 7:41 ` Sergey Senozhatsky
2016-03-14 8:20 ` Sergey Senozhatsky
2016-03-14 8:20 ` Sergey Senozhatsky
2016-03-15 0:46 ` Minchan Kim
2016-03-15 0:46 ` Minchan Kim
2016-03-15 1:33 ` Sergey Senozhatsky
2016-03-15 1:33 ` Sergey Senozhatsky
2016-03-15 6:17 ` Minchan Kim
2016-03-15 6:17 ` Minchan Kim
2016-03-17 1:29 ` Sergey Senozhatsky
2016-03-17 1:29 ` Sergey Senozhatsky
2016-03-18 1:17 ` Minchan Kim
2016-03-18 1:17 ` Minchan Kim
2016-03-18 2:00 ` Sergey Senozhatsky
2016-03-18 2:00 ` Sergey Senozhatsky
2016-03-18 4:03 ` Minchan Kim
2016-03-18 4:03 ` Minchan Kim
2016-03-18 4:10 ` Sergey Senozhatsky
2016-03-18 4:10 ` Sergey Senozhatsky
2016-03-03 14:46 ` [RFC][PATCH v3 2/5] mm/zsmalloc: remove shrinker compaction callbacks Sergey Senozhatsky
2016-03-03 14:46 ` Sergey Senozhatsky
2016-03-14 6:32 ` Minchan Kim
2016-03-14 6:32 ` Minchan Kim
2016-03-14 7:45 ` Sergey Senozhatsky
2016-03-14 7:45 ` Sergey Senozhatsky
2016-03-15 0:52 ` Minchan Kim
2016-03-15 0:52 ` Minchan Kim
2016-03-15 1:05 ` Sergey Senozhatsky
2016-03-15 1:05 ` Sergey Senozhatsky
2016-03-15 2:19 ` Minchan Kim
2016-03-15 2:19 ` Minchan Kim
2016-03-03 14:46 ` [RFC][PATCH v3 3/5] mm/zsmalloc: introduce zs_huge_object() Sergey Senozhatsky
2016-03-03 14:46 ` Sergey Senozhatsky
2016-03-14 6:53 ` Minchan Kim
2016-03-14 6:53 ` Minchan Kim
2016-03-14 8:08 ` Sergey Senozhatsky
2016-03-14 8:08 ` Sergey Senozhatsky
2016-03-15 0:54 ` Minchan Kim
2016-03-15 0:54 ` Minchan Kim
2016-03-03 14:46 ` [RFC][PATCH v3 4/5] zram: use zs_huge_object() Sergey Senozhatsky
2016-03-03 14:46 ` Sergey Senozhatsky
2016-03-03 14:46 ` [RFC][PATCH v3 5/5] mm/zsmalloc: reduce the number of huge classes Sergey Senozhatsky
2016-03-03 14:46 ` Sergey Senozhatsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160314074159.GA542@swordfish \
--to=sergey.senozhatsky.work@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=js1304@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=minchan@kernel.org \
--cc=sergey.senozhatsky@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.