[RFC PATCH 0/3] mm/zsmalloc: increase density and reduce memory wastage

[RFC PATCH 0/3] mm/zsmalloc: increase density and reduce memory wastage

[Sergey Senozhatsky]

Hello,

 RFC

 ->huge classes are evil, and zsmalloc knows the watermark after which classes
are considered to be ->huge -- every object stored consumes the entire zspage
(which consist of a single order-0 page). zram, however, has its own statically
defined watermark for `bad' compression and stores every object larger than
this watermark as a PAGE_SIZE, object, IOW, to a ->huge class, this results
in increased memory consumption and memory wastage. And zram's 'bad' watermark
is much lower than zsmalloc. Apart from that, 'bad' compressions are not so rare,
on some of my tests 41% of writes result in 'bad' compressions.

This patch set inverts this 'huge class watermark' enforcement, it's zsmalloc
that knows better, not zram.

I did a number of tests (see 0003 commit message) and memory savings were around
36MB and 51MB (depending on zsmalloc configuration).

I also copied a linux-next directory (with object files, du -sh  2.5G)
and (ZS_MAX_PAGES_PER_ZSPAGE=5) memory saving were around 17-20MB.



Sergey Senozhatsky (3):
  mm/zsmalloc: introduce zs_get_huge_class_size_watermark()
  zram: use zs_get_huge_class_size_watermark()
  mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

 drivers/block/zram/zram_drv.c |  2 +-
 drivers/block/zram/zram_drv.h |  6 ------
 include/linux/zsmalloc.h      |  2 ++
 mm/zsmalloc.c                 | 21 +++++++++++++++++----
 4 files changed, 20 insertions(+), 11 deletions(-)

-- 
2.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC PATCH 1/3] mm/zsmalloc: introduce zs_get_huge_class_size_watermark()

[Sergey Senozhatsky]

zsmalloc knows the watermark after which classes are considered
to be ->huge -- every object stored consumes the entire zspage (which
consist of a single order-0 page). On x86_64, PAGE_SHIFT 12 box, the
first non-huge class size is 3264, so starting down from size 3264,
objects share page(-s) and thus minimize memory wastage.

zram, however, has its own statically defined watermark for `bad'
compression "3 * PAGE_SIZE / 4 = 3072", and stores every object
larger than this watermark (3072) as a PAGE_SIZE, object, IOW,
to a ->huge class, this results in increased memory consumption and
memory wastage.

Introduce a zs_get_huge_class_size_watermark() function which tells
the size of a first non-huge class; so zram now can store objects
to ->huge clases only when those objects have sizes greater than
huge_class_size_watermark.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 include/linux/zsmalloc.h |  2 ++
 mm/zsmalloc.c            | 14 ++++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h
index 34eb160..45dcb51 100644
--- a/include/linux/zsmalloc.h
+++ b/include/linux/zsmalloc.h
@@ -55,4 +55,6 @@ unsigned long zs_get_total_pages(struct zs_pool *pool);
 unsigned long zs_compact(struct zs_pool *pool);
 
 void zs_pool_stats(struct zs_pool *pool, struct zs_pool_stats *stats);
+
+int zs_get_huge_class_size_watermark(void);
 #endif
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 43e4cbc..61b1b35 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -188,6 +188,11 @@ static struct dentry *zs_stat_root;
 static int zs_size_classes;
 
 /*
+ * All classes above this class_size are huge classes
+ */
+static int huge_class_size_watermark;
+
+/*
  * We assign a page to ZS_ALMOST_EMPTY fullness group when:
  *	n <= N / f, where
  * n = number of allocated objects
@@ -1241,6 +1246,12 @@ unsigned long zs_get_total_pages(struct zs_pool *pool)
 }
 EXPORT_SYMBOL_GPL(zs_get_total_pages);
 
+int zs_get_huge_class_size_watermark(void)
+{
+	return huge_class_size_watermark;
+}
+EXPORT_SYMBOL_GPL(zs_get_huge_class_size_watermark);
+
 /**
  * zs_map_object - get address of allocated object from handle.
  * @pool: pool from which the object was allocated
@@ -1942,10 +1953,13 @@ struct zs_pool *zs_create_pool(const char *name, gfp_t flags)
 		if (pages_per_zspage == 1 &&
 			get_maxobj_per_zspage(size, pages_per_zspage) == 1)
 			class->huge = true;
+
 		spin_lock_init(&class->lock);
 		pool->size_class[i] = class;
 
 		prev_class = class;
+		if (!class->huge && !huge_class_size_watermark)
+			huge_class_size_watermark = size;
 	}
 
 	pool->flags = flags;
-- 
2.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC PATCH 2/3] zram: use zs_get_huge_class_size_watermark()

[Sergey Senozhatsky]

zram should stop enforcing its own 'bad' object size watermark,
and start using zs_get_huge_class_size_watermark(). zsmalloc
really knows better.

Drop `max_zpage_size' and use zs_get_huge_class_size_watermark()
instead.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 drivers/block/zram/zram_drv.c | 2 +-
 drivers/block/zram/zram_drv.h | 6 ------
 2 files changed, 1 insertion(+), 7 deletions(-)

diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 46055db..2621564 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -714,7 +714,7 @@ static int zram_bvec_write(struct zram *zram, struct bio_vec *bvec, u32 index,
 		goto out;
 	}
 	src = zstrm->buffer;
-	if (unlikely(clen > max_zpage_size)) {
+	if (unlikely(clen > zs_get_huge_class_size_watermark())) {
 		clen = PAGE_SIZE;
 		if (is_partial_io(bvec))
 			src = uncmem;
diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h
index 8e92339..8879161 100644
--- a/drivers/block/zram/zram_drv.h
+++ b/drivers/block/zram/zram_drv.h
@@ -23,12 +23,6 @@
 /*-- Configurable parameters */
 
 /*
- * Pages that compress to size greater than this are stored
- * uncompressed in memory.
- */
-static const size_t max_zpage_size = PAGE_SIZE / 4 * 3;
-
-/*
  * NOTE: max_zpage_size must be less than or equal to:
  *   ZS_MAX_ALLOC_SIZE. Otherwise, zs_malloc() would
  * always return failure.
-- 
2.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Sergey Senozhatsky]

ZS_MAX_PAGES_PER_ZSPAGE does not have to be order or 2. The existing
limit of 4 pages per zspage sets a tight limit on ->huge classes, which
results in increased memory wastage and consumption.

For example, on x86_64, PAGE_SHIFT 12, ->huge class_size range is

ZS_MAX_PAGES_PER_ZSPAGE		->huge classes size range
4					3280-4096
5					3424-4096
6					3520-4096

With bigger ZS_MAX_PAGES_PER_ZSPAGE we have less ->huge classes, because
some of the previously known as ->huge classes now have better chances to
form zspages that will waste less memory. This increases the density and
improves memory efficiency.

Example,

class_size 3328 with ZS_MAX_PAGES_PER_ZSPAGE=5 has pages_per_zspage 5
and max_objects 6, while with ZS_MAX_PAGES_PER_ZSPAGE=1 it had
pages_per_zspage 1 and max_objects 1. So now every 6th 3328-bytes object
stored by zram will not consume a new zspage (and order-0 page), but will
share an already allocated one.

TEST
====

Create a text file and do rounds of dd (one process). The amount of
copied data, its content and order are stable.

test script:

rm /tmp/test-file
for i in {1..200}; do
	cat /media/dev/linux-mmots/mm/zsmalloc.c >> /tmp/test-file;
done

for i in {1..5}; do
        umount /zram
        rmmod zram

        # create a 4G zram device, LZ0, multi stream, ext4 fs
        ./create-zram 4g

        for k in {1..3}; do
                j=1;
                while [ $j -lt $((1024*1024)) ]; do
                        dd if=/tmp/test-file of=/zram/file-$k-$j bs=$j count=1 \
                                oflag=sync > /dev/null 2>&1
                        let j=$j+512
                done
        done

        sync
        cat /sys/block/zram0/mm_stat >> /tmp/zram-stat
        umount /zram
        rmmod zram
done

RESULTS
=======
cat /sys/block/zram0/mm_stat column 3 is zs_get_total_pages() << PAGE_SHIFT

BASE
3371106304 1714719722 1842778112        0 1842778112       16        0        1
3371098112 1714667024 1842831360        0 1842831360       16        0        1
3371110400 1714767329 1842716672        0 1842716672       16        0        1
3371110400 1714717615 1842601984        0 1842601984       16        0        1
3371106304 1714744207 1842135040        0 1842135040       16        0        1

ZS_MAX_PAGES_PER_ZSPAGE=5
3371094016 1714584459 1804095488        0 1804095488       16        0        1
3371102208 1714619140 1804660736        0 1804660736       16        0        1
3371114496 1714755452 1804316672        0 1804316672       16        0        1
3371081728 1714606179 1804800000        0 1804800000       16        0        1
3371122688 1714871507 1804361728        0 1804361728       16        0        1

ZS_MAX_PAGES_PER_ZSPAGE=6
3371114496 1714704275 1789206528        0 1789206528       16        0        1
3371102208 1714740225 1789259776        0 1789259776       16        0        1
3371102208 1714717465 1789071360        0 1789071360       16        0        1
3371110400 1714704079 1789194240        0 1789194240       16        0        1
3371085824 1714792954 1789308928        0 1789308928       16        0        1

So that's
 around 36MB of saved space between BASE and ZS_MAX_PAGES_PER_ZSPAGE=5
and
 around 51MB of saved space between BASE and ZS_MAX_PAGES_PER_ZSPAGE=6.

Set ZS_MAX_PAGES_PER_ZSPAGE to 6 for now.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 mm/zsmalloc.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 61b1b35..0c9f117 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -74,11 +74,10 @@
 #define ZS_ALIGN		8
 
 /*
- * A single 'zspage' is composed of up to 2^N discontiguous 0-order (single)
- * pages. ZS_MAX_ZSPAGE_ORDER defines upper limit on N.
+ * A single 'zspage' is composed of up ZS_MAX_PAGES_PER_ZSPAGE discontiguous
+ * 0-order (single) pages.
  */
-#define ZS_MAX_ZSPAGE_ORDER 2
-#define ZS_MAX_PAGES_PER_ZSPAGE (_AC(1, UL) << ZS_MAX_ZSPAGE_ORDER)
+#define ZS_MAX_PAGES_PER_ZSPAGE	6
 
 #define ZS_HANDLE_SIZE (sizeof(unsigned long))
 
-- 
2.7.1

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Sergey Senozhatsky]

On (02/18/16 12:02), Sergey Senozhatsky wrote:
[..]
> ---
>  mm/zsmalloc.c | 7 +++----
>  1 file changed, 3 insertions(+), 4 deletions(-)
> 
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 61b1b35..0c9f117 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -74,11 +74,10 @@
>  #define ZS_ALIGN		8
>  
>  /*
> - * A single 'zspage' is composed of up to 2^N discontiguous 0-order (single)
> - * pages. ZS_MAX_ZSPAGE_ORDER defines upper limit on N.
> + * A single 'zspage' is composed of up ZS_MAX_PAGES_PER_ZSPAGE discontiguous
> + * 0-order (single) pages.
>   */
> -#define ZS_MAX_ZSPAGE_ORDER 2
> -#define ZS_MAX_PAGES_PER_ZSPAGE (_AC(1, UL) << ZS_MAX_ZSPAGE_ORDER)
> +#define ZS_MAX_PAGES_PER_ZSPAGE	6
>  
>  #define ZS_HANDLE_SIZE (sizeof(unsigned long))
>  


I think we better switch to different logic here -- specify how many ->huge
classes we want to have, and let zsmalloc to calculate ZS_MAX_PAGES_PER_ZSPAGE.


For example, if we want to have 20 ->huge classes, the 'smallest' (or the last)
non-huge class will have CLASS_SIZE * SIZE_CLASS_DELTA of spare (wasted) space,
so  PAGE_SIZE / (CLASS_SIZE * SIZE_CLASS_DELTA)  will give us the number of pages
we need to form into a zspage to make it the last huge class.


setting ZS_MIN_HUGE_CLASSES_NUM to 32 gives us (on x86_64, PAGE_SHIFT 12) ->huge
class size range of [3648, 4096]. so all objects smaller than 3648 will not waste
an entire zspage (and order-0 page), but will share the page with another objects
of that size.


something like this:

---

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 0c9f117..d5252d1 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -73,12 +73,6 @@
  */
 #define ZS_ALIGN               8
 
-/*
- * A single 'zspage' is composed of up ZS_MAX_PAGES_PER_ZSPAGE discontiguous
- * 0-order (single) pages.
- */
-#define ZS_MAX_PAGES_PER_ZSPAGE        6
-
 #define ZS_HANDLE_SIZE (sizeof(unsigned long))
 
 /*
@@ -149,6 +143,21 @@
 #define ZS_SIZE_CLASS_DELTA    (PAGE_SIZE >> 8)
 
 /*
+ * We want to have at least this number of ->huge classes.
+ */
+#define ZS_MIN_HUGE_CLASSES_NUM        32
+/*
+ * A single 'zspage' is composed of up ZS_MAX_PAGES_PER_ZSPAGE discontiguous
+ * 0-order (single) pages.
+ *
+ * The smallest huge class will have CLASS_SIZE * SIZE_CLASS_DELTA of
+ * wasted space, calculate how many pages we need to fit a CLASS_SIZE
+ * object there and, thus, to save a additional zspage.
+ */
+#define ZS_MAX_PAGES_PER_ZSPAGE        \
+       (PAGE_SIZE / (ZS_MIN_HUGE_CLASSES_NUM * ZS_SIZE_CLASS_DELTA))
+
+/*
  * We do not maintain any list for completely empty or full pages
  */
 enum fullness_group {

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Sergey Senozhatsky]

On (02/18/16 13:41), Sergey Senozhatsky wrote:
> I think we better switch to different logic here -- specify how many ->huge
> classes we want to have, and let zsmalloc to calculate ZS_MAX_PAGES_PER_ZSPAGE.
> 
> 
> For example, if we want to have 20 ->huge classes, the 'smallest' (or the last)
							^^^ largest
> non-huge class will have CLASS_SIZE * SIZE_CLASS_DELTA of spare (wasted) space,
> so  PAGE_SIZE / (CLASS_SIZE * SIZE_CLASS_DELTA)  will give us the number of pages
> we need to form into a zspage to make it the last huge class.
						    ^^ 'non-huge'... what is the
						    correct english word I'm looking
						    for...

	-ss

> setting ZS_MIN_HUGE_CLASSES_NUM to 32 gives us (on x86_64, PAGE_SHIFT 12) ->huge
> class size range of [3648, 4096]. so all objects smaller than 3648 will not waste
> an entire zspage (and order-0 page), but will share the page with another objects
> of that size.
> 
> 
> something like this:
> 
> ---
> 
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 0c9f117..d5252d1 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -73,12 +73,6 @@
>   */
>  #define ZS_ALIGN               8
>  
> -/*
> - * A single 'zspage' is composed of up ZS_MAX_PAGES_PER_ZSPAGE discontiguous
> - * 0-order (single) pages.
> - */
> -#define ZS_MAX_PAGES_PER_ZSPAGE        6
> -
>  #define ZS_HANDLE_SIZE (sizeof(unsigned long))
>  
>  /*
> @@ -149,6 +143,21 @@
>  #define ZS_SIZE_CLASS_DELTA    (PAGE_SIZE >> 8)
>  
>  /*
> + * We want to have at least this number of ->huge classes.
> + */
> +#define ZS_MIN_HUGE_CLASSES_NUM        32
> +/*
> + * A single 'zspage' is composed of up ZS_MAX_PAGES_PER_ZSPAGE discontiguous
> + * 0-order (single) pages.
> + *
> + * The smallest huge class will have CLASS_SIZE * SIZE_CLASS_DELTA of
> + * wasted space, calculate how many pages we need to fit a CLASS_SIZE
> + * object there and, thus, to save a additional zspage.
> + */
> +#define ZS_MAX_PAGES_PER_ZSPAGE        \
> +       (PAGE_SIZE / (ZS_MIN_HUGE_CLASSES_NUM * ZS_SIZE_CLASS_DELTA))
> +
> +/*
>   * We do not maintain any list for completely empty or full pages
>   */
>  enum fullness_group {
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Sergey Senozhatsky]

On (02/18/16 13:41), Sergey Senozhatsky wrote:
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index 0c9f117..d5252d1 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -73,12 +73,6 @@
>   */
>  #define ZS_ALIGN               8
>  
> -/*
> - * A single 'zspage' is composed of up ZS_MAX_PAGES_PER_ZSPAGE discontiguous
> - * 0-order (single) pages.
> - */
> -#define ZS_MAX_PAGES_PER_ZSPAGE        6
> -
>  #define ZS_HANDLE_SIZE (sizeof(unsigned long))
>  
>  /*
> @@ -149,6 +143,21 @@
>  #define ZS_SIZE_CLASS_DELTA    (PAGE_SIZE >> 8)
>  
>  /*
> + * We want to have at least this number of ->huge classes.
> + */
> +#define ZS_MIN_HUGE_CLASSES_NUM        32
> +/*
> + * A single 'zspage' is composed of up ZS_MAX_PAGES_PER_ZSPAGE discontiguous
> + * 0-order (single) pages.
> + *
> + * The smallest huge class will have CLASS_SIZE * SIZE_CLASS_DELTA of
> + * wasted space, calculate how many pages we need to fit a CLASS_SIZE
> + * object there and, thus, to save a additional zspage.
> + */
> +#define ZS_MAX_PAGES_PER_ZSPAGE        \
> +       (PAGE_SIZE / (ZS_MIN_HUGE_CLASSES_NUM * ZS_SIZE_CLASS_DELTA))
> +
> +/*
>   * We do not maintain any list for completely empty or full pages
>   */
>  enum fullness_group {

and the difference between ZS_MIN_HUGE_CLASSES_NUM 57 (BASE number of
->huge classes) and ZS_MIN_HUGE_CLASSES_NUM 32 is:

ZS_MIN_HUGE_CLASSES_NUM 57 (BASE)
1151078400 575501415 621494272        0 621494272       17        0        1
1151074304 575499905 621477888        0 621477888       17        0        1
1151074304 575516318 621363200        0 621363200       17        0        1
1151078400 575558182 621346816        0 621346816       17        0        1
1151078400 575599320 621531136        0 621531136       17        0        1

ZS_MIN_HUGE_CLASSES_NUM 32
1151074304 575483112 594194432        0 594194432       17        0        1
1151074304 575521895 593678336        0 593678336       17        0        1
1151074304 575570453 594173952        0 594173952       17        0        1
1151074304 575461842 594010112        0 594010112       17        0        1
1151074304 575537116 593879040        0 593879040       17        0        1


around 26MB of order-0 pages.

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Joonsoo Kim]

Hello,

2016-02-18 12:02 GMT+09:00 Sergey Senozhatsky
<sergey.senozhatsky.work@gmail.com>:
> ZS_MAX_PAGES_PER_ZSPAGE does not have to be order or 2. The existing
> limit of 4 pages per zspage sets a tight limit on ->huge classes, which
> results in increased memory wastage and consumption.

There is a reason that it is order of 2. Increasing ZS_MAX_PAGES_PER_ZSPAGE
is related to ZS_MIN_ALLOC_SIZE. If we don't have enough OBJ_INDEX_BITS,
ZS_MIN_ALLOC_SIZE would be increase and it causes regression on some
system. Using one more bit on OBJ_INDEX_BITS means that we can increase
ZS_MAX_PAGES_PER_ZSPAGE to double times. AFAIK, there is no bit left
so you need to steal one from somewhere to increase
ZS_MAX_PAGES_PER_ZSPAGE.

> For example, on x86_64, PAGE_SHIFT 12, ->huge class_size range is
>
> ZS_MAX_PAGES_PER_ZSPAGE         ->huge classes size range
> 4                                       3280-4096
> 5                                       3424-4096
> 6                                       3520-4096
>
> With bigger ZS_MAX_PAGES_PER_ZSPAGE we have less ->huge classes, because
> some of the previously known as ->huge classes now have better chances to
> form zspages that will waste less memory. This increases the density and
> improves memory efficiency.
>
> Example,
>
> class_size 3328 with ZS_MAX_PAGES_PER_ZSPAGE=5 has pages_per_zspage 5
> and max_objects 6, while with ZS_MAX_PAGES_PER_ZSPAGE=1 it had
> pages_per_zspage 1 and max_objects 1. So now every 6th 3328-bytes object
> stored by zram will not consume a new zspage (and order-0 page), but will
> share an already allocated one.
>
> TEST
> ====
>
> Create a text file and do rounds of dd (one process). The amount of
> copied data, its content and order are stable.
>
> test script:
>
> rm /tmp/test-file
> for i in {1..200}; do
>         cat /media/dev/linux-mmots/mm/zsmalloc.c >> /tmp/test-file;
> done
>
> for i in {1..5}; do
>         umount /zram
>         rmmod zram
>
>         # create a 4G zram device, LZ0, multi stream, ext4 fs
>         ./create-zram 4g
>
>         for k in {1..3}; do
>                 j=1;
>                 while [ $j -lt $((1024*1024)) ]; do
>                         dd if=/tmp/test-file of=/zram/file-$k-$j bs=$j count=1 \
>                                 oflag=sync > /dev/null 2>&1
>                         let j=$j+512
>                 done
>         done
>
>         sync
>         cat /sys/block/zram0/mm_stat >> /tmp/zram-stat
>         umount /zram
>         rmmod zram
> done
>
> RESULTS
> =======
> cat /sys/block/zram0/mm_stat column 3 is zs_get_total_pages() << PAGE_SHIFT
>
> BASE
> 3371106304 1714719722 1842778112        0 1842778112       16        0        1
> 3371098112 1714667024 1842831360        0 1842831360       16        0        1
> 3371110400 1714767329 1842716672        0 1842716672       16        0        1
> 3371110400 1714717615 1842601984        0 1842601984       16        0        1
> 3371106304 1714744207 1842135040        0 1842135040       16        0        1
>
> ZS_MAX_PAGES_PER_ZSPAGE=5
> 3371094016 1714584459 1804095488        0 1804095488       16        0        1
> 3371102208 1714619140 1804660736        0 1804660736       16        0        1
> 3371114496 1714755452 1804316672        0 1804316672       16        0        1
> 3371081728 1714606179 1804800000        0 1804800000       16        0        1
> 3371122688 1714871507 1804361728        0 1804361728       16        0        1
>
> ZS_MAX_PAGES_PER_ZSPAGE=6
> 3371114496 1714704275 1789206528        0 1789206528       16        0        1
> 3371102208 1714740225 1789259776        0 1789259776       16        0        1
> 3371102208 1714717465 1789071360        0 1789071360       16        0        1
> 3371110400 1714704079 1789194240        0 1789194240       16        0        1
> 3371085824 1714792954 1789308928        0 1789308928       16        0        1
>
> So that's
>  around 36MB of saved space between BASE and ZS_MAX_PAGES_PER_ZSPAGE=5
> and
>  around 51MB of saved space between BASE and ZS_MAX_PAGES_PER_ZSPAGE=6.

Looks nice.

> Set ZS_MAX_PAGES_PER_ZSPAGE to 6 for now.

Why not just set it to 8? It could save more because some classes can fit
better to 8 pages zspage.

I have a concern that increasing ZS_MAX_PAGES_PER_ZSPAGE would
cause more pressure on memory at some moment because it requires
more pages to compress and store just 1 pages. What do you think
about it?

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Sergey Senozhatsky]

Hello Joonsoo,

On (02/18/16 17:28), Joonsoo Kim wrote:
> 2016-02-18 12:02 GMT+09:00 Sergey Senozhatsky
> <sergey.senozhatsky.work@gmail.com>:
> > ZS_MAX_PAGES_PER_ZSPAGE does not have to be order or 2. The existing
> > limit of 4 pages per zspage sets a tight limit on ->huge classes, which
> > results in increased memory wastage and consumption.
> 
> There is a reason that it is order of 2. Increasing ZS_MAX_PAGES_PER_ZSPAGE
> is related to ZS_MIN_ALLOC_SIZE. If we don't have enough OBJ_INDEX_BITS,
> ZS_MIN_ALLOC_SIZE would be increase and it causes regression on some
> system.

Thanks!

do you mean PHYSMEM_BITS != BITS_PER_LONG systems? PAE/LPAE? isn't it
the case that on those systems ZS_MIN_ALLOC_SIZE already bigger than 32?

MAX_PHYSMEM_BITS	36
_PFN_BITS		36 - 12
OBJ_INDEX_BITS		(32 - (36 - 12) - 1)
ZS_MIN_ALLOC_SIZE	MAX(32, 4 << 12 >> (32 - (36 - 12) - 1))  !=  32

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Sergey Senozhatsky]

On (02/18/16 18:55), Sergey Senozhatsky wrote:
> > There is a reason that it is order of 2. Increasing ZS_MAX_PAGES_PER_ZSPAGE
> > is related to ZS_MIN_ALLOC_SIZE. If we don't have enough OBJ_INDEX_BITS,
> > ZS_MIN_ALLOC_SIZE would be increase and it causes regression on some
> > system.
> 
> Thanks!
> 
> do you mean PHYSMEM_BITS != BITS_PER_LONG systems? PAE/LPAE? isn't it
> the case that on those systems ZS_MIN_ALLOC_SIZE already bigger than 32?

I mean, yes, there are ZS_ALIGN requirements that I completely ignored,
thanks for pointing that out.

just saying, not insisting on anything, theoretically, trading 32 bit size
objects in exchange of reducing a much bigger memory wastage is sort of
interesting. zram stores objects bigger than 3072 as huge objects, leaving
4096-3072 bytes unused, and it'll take 4096-3072/32 = 4000  32 bit objects
to beat that single 'bad' compression object in storing inefficiency...

well, patches 0001/0002 are trying to address this a bit, but the biggest
problem is still there: we have too many ->huge classes and they are a bit
far from good.

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Joonsoo Kim]

2016-02-18 19:19 GMT+09:00 Sergey Senozhatsky
<sergey.senozhatsky.work@gmail.com>:
> On (02/18/16 18:55), Sergey Senozhatsky wrote:
>> > There is a reason that it is order of 2. Increasing ZS_MAX_PAGES_PER_ZSPAGE
>> > is related to ZS_MIN_ALLOC_SIZE. If we don't have enough OBJ_INDEX_BITS,
>> > ZS_MIN_ALLOC_SIZE would be increase and it causes regression on some
>> > system.
>>
>> Thanks!
>>
>> do you mean PHYSMEM_BITS != BITS_PER_LONG systems? PAE/LPAE? isn't it
>> the case that on those systems ZS_MIN_ALLOC_SIZE already bigger than 32?

Indeed.

> I mean, yes, there are ZS_ALIGN requirements that I completely ignored,
> thanks for pointing that out.
>
> just saying, not insisting on anything, theoretically, trading 32 bit size
> objects in exchange of reducing a much bigger memory wastage is sort of
> interesting. zram stores objects bigger than 3072 as huge objects, leaving

I'm also just saying. :)
On the above example system which already uses 128 byte min class,
your change makes it to 160 or 192. It could make a more trouble than
you thought.

> 4096-3072 bytes unused, and it'll take 4096-3072/32 = 4000  32 bit objects
> to beat that single 'bad' compression object in storing inefficiency...

Where does 4096-3072/32 calculation comes from? I'm not familiar to recent
change on zsmalloc such as huge class so can't understand this calculation.

> well, patches 0001/0002 are trying to address this a bit, but the biggest
> problem is still there: we have too many ->huge classes and they are a bit
> far from good.

Agreed. And I agree your patchset, too.

Anyway, could you answer my other questions on original reply?

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Sergey Senozhatsky]

Hello Joonsoo,


On (02/19/16 10:19), Joonsoo Kim wrote:
[..]
> Where does 4096-3072/32 calculation comes from? I'm not familiar to recent
> change on zsmalloc such as huge class so can't understand this calculation.

I'm very sorry, it's utterly wrong. I got lost in translations,
0001/0002 zram-zsmalloc changes, and so on. no excuse, my bad.


> > well, patches 0001/0002 are trying to address this a bit, but the biggest
> > problem is still there: we have too many ->huge classes and they are a bit
> > far from good.
> 
> Agreed. And I agree your patchset, too.

thanks. I'll scratch 0003 for now and do more tests. the observation so far is that 'bad'
(from zram point of view, and zram should stop having opinion on that) compressions are
happening way more often than I'd though, and zram is taking extra step to make zsmalloc
worse.

so there are some questions now. why 32 byte was considered to be good enough as
MIN alloc, were "32 - 2 * ZS_SIZE_CLASS_DELTA" or "32 - 1 * ZS_SIZE_CLASS_DELTA" way
to rare to consider the K * ZS_SIZE_CLASS_DELTA memory waste there? etc., etc.


every "unleashed" ->huge class_size will give us a PAGE_SIZE memory saving every time
we store (class_size / (PAGE_SIZE - class_size_idx * ZS_SIZE_CLASS_DELTA) objects, and
every abandoned MIN alloc size will give us ZS_SIZE_CLASS_DELTA memory wastage every
time we store an object of that class. so the question is what's the ratio, X huge objects
that result in PAGE_SIZE saved or Y small objects that result in PAGE_SIZE wasted, because
we abandoned the class_size those objects used to belong to and now zsmalloc stores
those objects in available MIN class (which can be size + K * ZS_SIZE_CLASS_DELTA away).
well, object of smaller than 32 bytes sizes do this already.


I agree that having a MIN alloc of 128 or 192 or more bytes is a bad thing to do, sure.
... so we need to have some limit here. but that'll will take more time to settle down.


> Anyway, could you answer my other questions on original reply?

sorry, what questions? I certainly would love to answer. there is only one email in my
inbox, with the message-id CAAmzW4O-yQ5GBTE-6WvCL-hZeqyW=k3Fzn4_9G2qkMmp=ceuJg@mail.gmail.com
is there an email that I missed or I misread your previous email and overlooked some
questions in there? I think we don't have too many spare bits left, so raising MIN alloc
seems to be the easiest option here.


... and *may be* losing 32 byte size is not so bad. we don't deal with random data size
compressions, it's PAGE_SIZE buffers that we compress (excluding partial IO case, which
is rarely seen, I believe, well unless you use FAT fs and may be some other). so can we
generally expect PAGE_SIZE buffer to be compressed to 32 bytes? taking a look at compression
ratios
http://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO

I'd say that more likely it'll be in XXX-XXXX bytes range. not in 32-XX range, probably.
(well, it's questionable of course, depends on the data, etc. etc.)

more over, isn't it the case, that compression algorithm store headers in the compression
buffers (like the number of mathces, etc.), checksums, etc. etc.. looking at lzo1x_1_do_compress.

 20 static noinline size_t
 21 lzo1x_1_do_compress(const unsigned char *in, size_t in_len,
 22                     unsigned char *out, size_t *out_len,
 23                     size_t ti, void *wrkmem)
 24 {
 25         const unsigned char *ip;
 26         unsigned char *op;
 27         const unsigned char * const in_end = in + in_len;
 28         const unsigned char * const ip_end = in + in_len - 20;
 29         const unsigned char *ii;
 30         lzo_dict_t * const dict = (lzo_dict_t *) wrkmem;


what's that "in_len - 20"? a header size of 20 bytes? so to hit a class_size of 32 we
need to compress 4096 bytes to 12 bytes? that's quite unlikely...

hm...
http://cyan4973.github.io/lz4/lz4_Block_format.html
http://cyan4973.github.io/lz4/lz4_Frame_format.html



I extended zsmalloc stats reporting with "huge" column (jsut in case), and did some tests.
copy linux, run iozone, copy text files, etc. etc.

and the results are...

# cat /sys/kernel/debug/zsmalloc/zram0/classes 
  class  size  huge almost_full almost_empty obj_allocated   obj_used pages_used pages_per_zspage
     0    32             0            0             0          0          0                1
     1    48             0            0             0          0          0                3
     2    64             3            0         31360      31355        490                1
     3    80             1            1         62781      62753       1231                1
     4    96             2            0          2176       2142         51                3
     5   112             0            2          1460       1413         40                2
     6   128             1            0          1312       1311         41                1
     7   144             2            2          1275       1191         45                3
     8   160             0            2          1020        991         40                2
     9   176             4            1          1209       1156         52                4
    10   192             2            0           960        938         45                3
    11   208             1            0           858        853         44                2
    12   224             3            0           949        940         52                4
    13   240             1            1           799        793         47                1
    14   256             1            0           944        941         59                1
    15   272             0            1          1005        994         67                1
    16   288             0            0          1036       1036         74                1
    17   304             1            0          1160       1156         87                3
    18   320             2            0          1479       1468        116                4
    19   336             0            1          1728       1720        144                1
    20   352             2            1          1863       1851        162                2
    21   368             1            1          2079       2072        189                1
    22   384             1            0          2240       2237        210                3
    23   400             1            1          2030       2022        203                1
    24   416             3            0          2028       2019        208                4
    25   432             2            0          1960       1955        210                3
    26   448             0            1          1881       1873        209                1
    27   464             2            0          1820       1813        208                4
    28   480             2            2          2023       1994        238                2
    29   496             3            2          1650       1620        200                4
    30   512             1            1          1544       1536        193                1
    31   528             0            1          1426       1416        184                4
    32   544             2            0          1200       1198        160                2
    33   560             1            0          1218       1217        168                4
    34   576             1            1          1197       1191        171                1
    35   592             0            1          1080       1069        160                4
    36   608             0            2          1220       1207        183                3
    37   624             0            1          1326       1316        204                2
    38   640             0            3          1520       1493        240                3
    40   672             1            1          2790       2785        465                1
    42   704             1            0          2852       2849        496                4
    43   720             1            0          1530       1529        270                3
    44   736             0            1          1463       1459        266                2
    46   768             0            1          2832       2827        531                3
    49   816             0            1          3820       3818        764                1
    51   848             1            1          2527       2521        532                4
    52   864             2            1          1316       1310        282                3
    54   896             3            1          2781       2775        618                2
    57   944             1            0          4381       4380       1011                3
    58   960             1            2          1530       1514        360                4
    62  1024             0            3          6576       6573       1644                1
    66  1088             1            1          7140       7131       1904                4
    67  1104             2            1          1859       1851        507                3
    71  1168             1            1          7245       7239       2070                2
    74  1216             1            0          5970       5969       1791                3
    76  1248             1            2          3796       3785       1168                4
    83  1360             0            3         18828      18824       6276                1
    91  1488             1            1         16456      16451       5984                4
    94  1536             0            0          4840       4840       1815                3
   100  1632             2            0          9665       9663       3866                2
   107  1744             2            4         12201      12187       5229                3
   111  1808             1            1          6822       6816       3032                4
   126  2048             0           13         29624      29611      14812                1
   144  2336             6           12         29302      29241      16744                4
   151  2448             1            9          7115       7087       4269                3
   168  2720             0           34          8547       8495       5698                2
   190  3072             0            9          6344       6332       4758                3
   202  3264             0            3           155        146        124                4
   254  4096 Y           0            0        632586     632586     632586                1



so BAD classes are 10 times more often than 64 bytes objects for example. and not all of 4096
objects actually deserve a full PAGE_SIZE (the test is with out 0001/0002 applied).



ok, this sets us on a  "do we need 32 and 48 bytes classes at all"  track?


	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Sergey Senozhatsky]

On (02/19/16 13:16), Sergey Senozhatsky wrote:
> # cat /sys/kernel/debug/zsmalloc/zram0/classes 
>   class  size  huge almost_full almost_empty obj_allocated   obj_used pages_used pages_per_zspage
>      0    32             0            0             0          0          0                1
>      1    48             0            0             0          0          0                3
>      2    64             3            0         31360      31355        490                1
>      3    80             1            1         62781      62753       1231                1
[..]
>    254  4096 Y           0            0        632586     632586     632586                1
> 

> so BAD classes are 10 times more often than 64 bytes objects for example. and not all of 4096
a typo, 				^^^^^ 80 bytes. and 20 times class 64.

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Sergey Senozhatsky]

On (02/19/16 13:16), Sergey Senozhatsky wrote:
> ok, this sets us on a  "do we need 32 and 48 bytes classes at all"  track?
> 

seems that lz4 defines a minimum length to be at least

 61 #define COPYLENGTH 8
 67 #define MINMATCH        4
 70 #define MFLIMIT         (COPYLENGTH + MINMATCH)
 71 #define MINLENGTH       (MFLIMIT + 1)

bytes.

and 32 bytes class still looks unreachable.

# cat /sys/kernel/debug/zsmalloc/zram0/classes 
  class  size  huge almost_full almost_empty obj_allocated   obj_used pages_used pages_per_zspage
     0    32             0            0             0          0          0                1
     1    48            26            0         31488      31259        369                3
     2    64             1            0         61760      61756        965                1
     3    80             2            1          2295       2253         45                1
     4    96             4            0          2176       2127         51                3
     5   112             1            1          1387       1358         38                2
     6   128             0            1          1312       1281         41                1
     7   144             1            0          1105       1086         39                3
     8   160             2            2          1173       1137         46                2
     9   176             4            0          1023        995         44                4
    10   192             6            1           960        913         45                3
    11   208             1            1           897        883         46                2
    12   224             1            2           803        735         44                4
    13   240             2            1           799        785         47                1
    14   256             1            1           816        807         51                1
    15   272             1            1           765        758         51                1
    16   288             3            1           840        831         60                1
    17   304             2            1           960        940         72                3
    18   320             1            1          1020        999         80                4
    19   336             2            0          1104       1102         92                1
    20   352             4            1          1265       1247        110                2
    21   368             1            1          1287       1280        117                1
    22   384             2            0          1248       1242        117                3
    23   400             0            0          1380       1380        138                1
    24   416             1            0          1404       1403        144                4
    25   432             0            0          1400       1400        150                3
    26   448             1            0          1278       1277        142                1
    27   464             0            2          1295       1263        148                4
    28   480             4            0          1326       1319        156                2
    29   496             2            2          2343       2311        284                4
    30   512             0            0          1360       1360        170                1
    31   528             0            3          1395       1365        180                4
    32   544             2            1          1320       1306        176                2
    33   560             0            1          1218       1203        168                4
    34   576             0            0          1162       1162        166                1
    35   592             2            1          1053       1033        156                4
    36   608             2            2          1440       1424        216                3
    37   624             0            1          1664       1659        256                2
    38   640             1            1          1197       1186        189                3
    40   672             0            2          2292       2287        382                1
    42   704             3            0          2369       2365        412                4
    43   720             1            1          1207       1198        213                3
    44   736             1            1          1232       1227        224                2
    46   768             1            2          2336       2323        438                3
    49   816             0            0          3615       3615        723                1
    51   848             3            1          2185       2174        460                4
    52   864             1            1          1148       1141        246                3
    54   896             2            2          2889       2881        642                2
    57   944             2            0          3796       3794        876                3
    58   960             2            0          1428       1423        336                4
    62  1024             0            1          5604       5603       1401                1
    66  1088             0            1          6060       6047       1616                4
    67  1104             1            0          1661       1659        453                3
    71  1168             2            0          6440       6438       1840                2
    74  1216             4            0          5120       5115       1536                3
    76  1248             4            0          3536       3531       1088                4
    83  1360             0            1         15282      15281       5094                1
    91  1488             3            1         17897      17887       6508                4
    94  1536             3            1          5768       5762       2163                3
   100  1632             3            1         10275      10270       4110                2
   107  1744             1            1         11676      11673       5004                3
   111  1808             3            0          6714       6711       2984                4
   126  2048             0            2         27758      27756      13879                1
   144  2336             0            5         32823      32807      18756                4
   151  2448             3            2          9650       9642       5790                3
   168  2720             0            8         13341      13326       8894                2
   190  3072             0            3          7804       7799       5853                3
   202  3264             2            0           255        253        204                4
   254  4096 Y           0            0        636960     636960     636960                1

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Sergey Senozhatsky]

On (02/19/16 13:46), Sergey Senozhatsky wrote:
> On (02/19/16 13:16), Sergey Senozhatsky wrote:
> > ok, this sets us on a  "do we need 32 and 48 bytes classes at all"  track?
> > 
> 
> seems that lz4 defines a minimum length to be at least
> 
>  61 #define COPYLENGTH 8
>  67 #define MINMATCH        4
>  70 #define MFLIMIT         (COPYLENGTH + MINMATCH)
>  71 #define MINLENGTH       (MFLIMIT + 1)
> 
> bytes.

hm, on a second look, zsmalloc defines the following macros:

#define ZS_MAX_ZSPAGE_ORDER 2
#define ZS_MAX_PAGES_PER_ZSPAGE (_AC(1, UL) << ZS_MAX_ZSPAGE_ORDER)

#ifndef MAX_PHYSMEM_BITS
#ifdef CONFIG_HIGHMEM64G
#define MAX_PHYSMEM_BITS 36
#else /* !CONFIG_HIGHMEM64G */
/*
 * If this definition of MAX_PHYSMEM_BITS is used, OBJ_INDEX_BITS will just
 * be PAGE_SHIFT
 */
#define MAX_PHYSMEM_BITS BITS_PER_LONG
#endif
#endif

#define _PFN_BITS		(MAX_PHYSMEM_BITS - PAGE_SHIFT)

#define OBJ_ALLOCATED_TAG 1
#define OBJ_TAG_BITS 1
#define OBJ_INDEX_BITS	(BITS_PER_LONG - _PFN_BITS - OBJ_TAG_BITS)
#define OBJ_INDEX_MASK	((_AC(1, UL) << OBJ_INDEX_BITS) - 1)

#define ZS_MIN_ALLOC_SIZE \
	MAX(32, (ZS_MAX_PAGES_PER_ZSPAGE << PAGE_SHIFT >> OBJ_INDEX_BITS))





so let's do some calculations, hopefuly I'm not mistaken anywhere.

with ZS_MAX_ZSPAGE_ORDER 4

-- on 32 bit system, PAGE_SHIFT 12

ZS_MAX_PAGES_PER_ZSPAGE 1 << 4						16
OBJ_INDEX_BITS (32 - (32 - 12) - 1)					11
OBJ_INDEX_MASK ((1 << (32 - (32 - 12) - 1)) - 1)			2047
ZS_MIN_ALLOC_SIZE MAX(32, ((1 << 4) << 12 >> (32 - (32 - 12) - 1)))	32

-- on 64 bit system, PAGE_SHIFT 12

ZS_MAX_PAGES_PER_ZSPAGE 1 << 4						16
OBJ_INDEX_BITS (64 - (64 - 12) - 1)					11
OBJ_INDEX_MASK ((1 << (64 - (64 - 12) - 1)) - 1)			2047
ZS_MIN_ALLOC_SIZE MAX(32, ((1 << 4) << 12 >> (64 - (64 - 12) - 1)))	32

-- on 64 bit system, PAGE_SHIFT 14

ZS_MAX_PAGES_PER_ZSPAGE 1 << 4						16
OBJ_INDEX_BITS (64 - (64 - 14) - 1)					13
OBJ_INDEX_MASK ((1 << (64 - (64 - 14) - 1)) - 1)			8191
ZS_MIN_ALLOC_SIZE MAX(32, ((1 << 4) << 14 >> (64 - (64 - 14) - 1)))	32

-- on 64 bit system, PAGE_SHIFT 16

ZS_MAX_PAGES_PER_ZSPAGE 1 << 4						16
OBJ_INDEX_BITS (64 - (64 - 16) - 1)					15
OBJ_INDEX_MASK ((1 << (64 - (64 - 16) - 1)) - 1)			32767
ZS_MIN_ALLOC_SIZE MAX(32, ((1 << 4) << 16 >> (64 - (64 - 14) - 1)))	128     << bad


so, isn't it enough OBJ_INDEX_BITS bits to even keep 32 bytes class around?

we probably would prefer to lower ZS_MAX_ZSPAGE_ORDER on PAGE_SHIFT 16 systems.
for example to ZS_MAX_PAGES_PER_ZSPAGE 1 << 3, or 1 << 2.
and of course LPAE/PAE enabled systems -- leave ZS_MAX_ZSPAGE_ORDER 2 there.




ZS_MAX_PAGES_PER_ZSPAGE 1 << 4  gives us

# cat /sys/kernel/debug/zsmalloc/zram0/classes 
  class  size  huge almost_full almost_empty obj_allocated   obj_used pages_used pages_per_zspage
     0    32             0            0             0          0          0                1
...
   238  3840             0            0             0          0          0               15
   254  4096 Y           0            0             0          0          0                1


so starting from 3840+ we have huge classes, the rest are 'normal' classes and will
save memory there in theory.

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC PATCH 3/3] mm/zsmalloc: change ZS_MAX_PAGES_PER_ZSPAGE

[Sergey Senozhatsky]

On (02/19/16 14:38), Sergey Senozhatsky wrote:
[..]
> #define OBJ_ALLOCATED_TAG 1
> #define OBJ_TAG_BITS 1
> #define OBJ_INDEX_BITS	(BITS_PER_LONG - _PFN_BITS - OBJ_TAG_BITS)
> #define OBJ_INDEX_MASK	((_AC(1, UL) << OBJ_INDEX_BITS) - 1)
> 
> #define ZS_MIN_ALLOC_SIZE \
> 	MAX(32, (ZS_MAX_PAGES_PER_ZSPAGE << PAGE_SHIFT >> OBJ_INDEX_BITS))
[..]

> -- on 32 bit system, PAGE_SHIFT 12
> 
> ZS_MAX_PAGES_PER_ZSPAGE 1 << 4						16
> OBJ_INDEX_BITS (32 - (32 - 12) - 1)					11
> OBJ_INDEX_MASK ((1 << (32 - (32 - 12) - 1)) - 1)			2047
> ZS_MIN_ALLOC_SIZE MAX(32, ((1 << 4) << 12 >> (32 - (32 - 12) - 1)))	32
> 
> -- on 64 bit system, PAGE_SHIFT 12
> 
> ZS_MAX_PAGES_PER_ZSPAGE 1 << 4						16
> OBJ_INDEX_BITS (64 - (64 - 12) - 1)					11
> OBJ_INDEX_MASK ((1 << (64 - (64 - 12) - 1)) - 1)			2047
> ZS_MIN_ALLOC_SIZE MAX(32, ((1 << 4) << 12 >> (64 - (64 - 12) - 1)))	32

even if it's missing  "HANDLE_PIN_BIT 0", it's still OBJ_INDEX_BITS 10,
2<<10 should be enough to keep 32 bytes class around.


> -- on 64 bit system, PAGE_SHIFT 14
> 
> ZS_MAX_PAGES_PER_ZSPAGE 1 << 4						16
> OBJ_INDEX_BITS (64 - (64 - 14) - 1)					13
> OBJ_INDEX_MASK ((1 << (64 - (64 - 14) - 1)) - 1)			8191
> ZS_MIN_ALLOC_SIZE MAX(32, ((1 << 4) << 14 >> (64 - (64 - 14) - 1)))	32

OBJ_INDEX_BITS 2<<12 still looks to be good enough.

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>