[PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[Wanpeng Li]

Changelog:
 v5 -> v6:
  * shove variables in debug.c and in debug.h just have an extern, spotted by Konrad
  * update patch description, spotted by Konrad
 v4 -> v5:
  * fix compile error, reported by Fengguang, Geert
  * add check for !is_ephemeral(pool), spotted by Bob
 v3 -> v4:
  * handle duplication in page_is_zero_filled, spotted by Bob
  * fix zcache writeback in dubugfs
  * fix pers_pageframes|_max isn't exported in debugfs
  * fix static variable defined in debug.h but used in multiple C files
  * rebase on Greg's staging-next
 v2 -> v3:
  * increment/decrement zcache_[eph|pers]_zpages for zero-filled pages, spotted by Dan
  * replace "zero" or "zero page" by "zero_filled_page", spotted by Dan
 v1 -> v2:
  * avoid changing tmem.[ch] entirely, spotted by Dan.
  * don't accumulate [eph|pers]pageframe and [eph|pers]zpages for
    zero-filled pages, spotted by Dan
  * cleanup TODO list
  * add Dan Acked-by.

Motivation:

- Seth Jennings points out compress zero-filled pages with LZO(a lossless
  data compression algorithm) will waste memory and result in fragmentation.
  https://lkml.org/lkml/2012/8/14/347
- Dan Magenheimer add "Support zero-filled pages more efficiently" feature
  in zcache TODO list https://lkml.org/lkml/2013/2/13/503

Design:

- For store page, capture zero-filled pages(evicted clean page cache pages and
  swap pages), but don't compress them, set pampd which store zpage address to
  0x2(since 0x0 and 0x1 has already been ocuppied) to mark special zero-filled
  case and take advantage of tmem infrastructure to transform handle-tuple(pool
  id, object id, and an index) to a pampd. Twice compress zero-filled pages will
  contribute to one zcache_[eph|pers]_pageframes count accumulated.
- For load page, traverse tmem hierachical to transform handle-tuple to pampd
  and identify zero-filled case by pampd equal to 0x2 when filesystem reads
  file pages or a page needs to be swapped in, then refill the page to zero
  and return.

Test:

dd if=/dev/zero of=zerofile bs=1MB count=500
vmtouch -t zerofile
vmtouch -e zerofile

formula:
- fragmentation level = (zcache_[eph|pers]_pageframes * PAGE_SIZE - zcache_[eph|pers]_zbytes)
  * 100 / (zcache_[eph|pers]_pageframes * PAGE_SIZE)
- memory zcache occupy = zcache_[eph|pers]_zbytes

Result:

without zero-filled awareness:
- fragmentation level: 98%
- memory zcache occupy: 238MB
with zero-filled awareness:
- fragmentation level: 0%
- memory zcache occupy: 0MB

Wanpeng Li (3):
  staging: zcache: fix static variables defined in debug.h but used in
    mutiple C files
  staging: zcache: introduce zero-filled page stat count
  staging: zcache: clean TODO list

 drivers/staging/zcache/TODO          |    3 +-
 drivers/staging/zcache/debug.c       |   35 +++++++++++++++
 drivers/staging/zcache/debug.h       |   79 ++++++++++++++++++++-------------
 drivers/staging/zcache/zcache-main.c |    4 ++
 4 files changed, 88 insertions(+), 33 deletions(-)

-- 
1.7.5.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v6 1/3] staging: zcache: fix static variables defined in debug.h but used in mutiple C files

[Wanpeng Li]

After commit 95bdaee214 ("zcache: Move debugfs code out of zcache-main.c file") 
be merged, most of knods in zcache debugfs just export zero since these variables
are defined in debug.h but are in use in multiple C files zcache-main.c and debug.c, 
in this case variables can't be treated as shared variables.

Signed-off-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
---
 drivers/staging/zcache/debug.c |   32 ++++++++++++++++++++
 drivers/staging/zcache/debug.h |   62 ++++++++++++++++++++--------------------
 2 files changed, 63 insertions(+), 31 deletions(-)

diff --git a/drivers/staging/zcache/debug.c b/drivers/staging/zcache/debug.c
index d2d1fdf..faab2a9 100644
--- a/drivers/staging/zcache/debug.c
+++ b/drivers/staging/zcache/debug.c
@@ -4,6 +4,38 @@
 #ifdef CONFIG_ZCACHE_DEBUG
 #include <linux/debugfs.h>
 
+ssize_t zcache_obj_count;
+ssize_t zcache_obj_count_max;
+ssize_t zcache_objnode_count;
+ssize_t zcache_objnode_count_max;
+u64 zcache_eph_zbytes;
+u64 zcache_eph_zbytes_max;
+u64 zcache_pers_zbytes_max;
+ssize_t zcache_eph_pageframes_max;
+ssize_t zcache_pers_pageframes_max;
+ssize_t zcache_pageframes_alloced;
+ssize_t zcache_pageframes_freed;
+ssize_t zcache_eph_zpages;
+ssize_t zcache_eph_zpages_max;
+ssize_t zcache_pers_zpages_max;
+ssize_t zcache_flush_total;
+ssize_t zcache_flush_found;
+ssize_t zcache_flobj_total;
+ssize_t zcache_flobj_found;
+ssize_t zcache_failed_eph_puts;
+ssize_t zcache_failed_pers_puts;
+ssize_t zcache_failed_getfreepages;
+ssize_t zcache_failed_alloc;
+ssize_t zcache_put_to_flush;
+ssize_t zcache_compress_poor;
+ssize_t zcache_mean_compress_poor;
+ssize_t zcache_eph_ate_tail;
+ssize_t zcache_eph_ate_tail_failed;
+ssize_t zcache_pers_ate_eph;
+ssize_t zcache_pers_ate_eph_failed;
+ssize_t zcache_evicted_eph_zpages;
+ssize_t zcache_evicted_eph_pageframes;
+
 #define ATTR(x)  { .name = #x, .val = &zcache_##x, }
 static struct debug_entry {
 	const char *name;
diff --git a/drivers/staging/zcache/debug.h b/drivers/staging/zcache/debug.h
index 4bbe49b..8ec82d4 100644
--- a/drivers/staging/zcache/debug.h
+++ b/drivers/staging/zcache/debug.h
@@ -3,9 +3,9 @@
 #ifdef CONFIG_ZCACHE_DEBUG
 
 /* we try to keep these statistics SMP-consistent */
-static ssize_t zcache_obj_count;
+extern ssize_t zcache_obj_count;
 static atomic_t zcache_obj_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_obj_count_max;
+extern ssize_t zcache_obj_count_max;
 static inline void inc_zcache_obj_count(void)
 {
 	zcache_obj_count = atomic_inc_return(&zcache_obj_atomic);
@@ -17,9 +17,9 @@ static inline void dec_zcache_obj_count(void)
 	zcache_obj_count = atomic_dec_return(&zcache_obj_atomic);
 	BUG_ON(zcache_obj_count < 0);
 };
-static ssize_t zcache_objnode_count;
+extern ssize_t zcache_objnode_count;
 static atomic_t zcache_objnode_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_objnode_count_max;
+extern ssize_t zcache_objnode_count_max;
 static inline void inc_zcache_objnode_count(void)
 {
 	zcache_objnode_count = atomic_inc_return(&zcache_objnode_atomic);
@@ -31,9 +31,9 @@ static inline void dec_zcache_objnode_count(void)
 	zcache_objnode_count = atomic_dec_return(&zcache_objnode_atomic);
 	BUG_ON(zcache_objnode_count < 0);
 };
-static u64 zcache_eph_zbytes;
+extern u64 zcache_eph_zbytes;
 static atomic_long_t zcache_eph_zbytes_atomic = ATOMIC_INIT(0);
-static u64 zcache_eph_zbytes_max;
+extern u64 zcache_eph_zbytes_max;
 static inline void inc_zcache_eph_zbytes(unsigned clen)
 {
 	zcache_eph_zbytes = atomic_long_add_return(clen, &zcache_eph_zbytes_atomic);
@@ -46,7 +46,7 @@ static inline void dec_zcache_eph_zbytes(unsigned zsize)
 };
 extern  u64 zcache_pers_zbytes;
 static atomic_long_t zcache_pers_zbytes_atomic = ATOMIC_INIT(0);
-static u64 zcache_pers_zbytes_max;
+extern u64 zcache_pers_zbytes_max;
 static inline void inc_zcache_pers_zbytes(unsigned clen)
 {
 	zcache_pers_zbytes = atomic_long_add_return(clen, &zcache_pers_zbytes_atomic);
@@ -59,7 +59,7 @@ static inline void dec_zcache_pers_zbytes(unsigned zsize)
 }
 extern ssize_t zcache_eph_pageframes;
 static atomic_t zcache_eph_pageframes_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_eph_pageframes_max;
+extern ssize_t zcache_eph_pageframes_max;
 static inline void inc_zcache_eph_pageframes(void)
 {
 	zcache_eph_pageframes = atomic_inc_return(&zcache_eph_pageframes_atomic);
@@ -72,7 +72,7 @@ static inline void dec_zcache_eph_pageframes(void)
 };
 extern ssize_t zcache_pers_pageframes;
 static atomic_t zcache_pers_pageframes_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_pers_pageframes_max;
+extern ssize_t zcache_pers_pageframes_max;
 static inline void inc_zcache_pers_pageframes(void)
 {
 	zcache_pers_pageframes = atomic_inc_return(&zcache_pers_pageframes_atomic);
@@ -83,21 +83,21 @@ static inline void dec_zcache_pers_pageframes(void)
 {
 	zcache_pers_pageframes = atomic_dec_return(&zcache_pers_pageframes_atomic);
 }
-static ssize_t zcache_pageframes_alloced;
+extern ssize_t zcache_pageframes_alloced;
 static atomic_t zcache_pageframes_alloced_atomic = ATOMIC_INIT(0);
 static inline void inc_zcache_pageframes_alloced(void)
 {
 	zcache_pageframes_alloced = atomic_inc_return(&zcache_pageframes_alloced_atomic);
 };
-static ssize_t zcache_pageframes_freed;
+extern ssize_t zcache_pageframes_freed;
 static atomic_t zcache_pageframes_freed_atomic = ATOMIC_INIT(0);
 static inline void inc_zcache_pageframes_freed(void)
 {
 	zcache_pageframes_freed = atomic_inc_return(&zcache_pageframes_freed_atomic);
 }
-static ssize_t zcache_eph_zpages;
+extern ssize_t zcache_eph_zpages;
 static atomic_t zcache_eph_zpages_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_eph_zpages_max;
+extern ssize_t zcache_eph_zpages_max;
 static inline void inc_zcache_eph_zpages(void)
 {
 	zcache_eph_zpages = atomic_inc_return(&zcache_eph_zpages_atomic);
@@ -110,7 +110,7 @@ static inline void dec_zcache_eph_zpages(unsigned zpages)
 }
 extern ssize_t zcache_pers_zpages;
 static atomic_t zcache_pers_zpages_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_pers_zpages_max;
+extern ssize_t zcache_pers_zpages_max;
 static inline void inc_zcache_pers_zpages(void)
 {
 	zcache_pers_zpages = atomic_inc_return(&zcache_pers_zpages_atomic);
@@ -130,23 +130,23 @@ static inline unsigned long curr_pageframes_count(void)
 		atomic_read(&zcache_pers_pageframes_atomic);
 };
 /* but for the rest of these, counting races are ok */
-static ssize_t zcache_flush_total;
-static ssize_t zcache_flush_found;
-static ssize_t zcache_flobj_total;
-static ssize_t zcache_flobj_found;
-static ssize_t zcache_failed_eph_puts;
-static ssize_t zcache_failed_pers_puts;
-static ssize_t zcache_failed_getfreepages;
-static ssize_t zcache_failed_alloc;
-static ssize_t zcache_put_to_flush;
-static ssize_t zcache_compress_poor;
-static ssize_t zcache_mean_compress_poor;
-static ssize_t zcache_eph_ate_tail;
-static ssize_t zcache_eph_ate_tail_failed;
-static ssize_t zcache_pers_ate_eph;
-static ssize_t zcache_pers_ate_eph_failed;
-static ssize_t zcache_evicted_eph_zpages;
-static ssize_t zcache_evicted_eph_pageframes;
+extern ssize_t zcache_flush_total;
+extern ssize_t zcache_flush_found;
+extern ssize_t zcache_flobj_total;
+extern ssize_t zcache_flobj_found;
+extern ssize_t zcache_failed_eph_puts;
+extern ssize_t zcache_failed_pers_puts;
+extern ssize_t zcache_failed_getfreepages;
+extern ssize_t zcache_failed_alloc;
+extern ssize_t zcache_put_to_flush;
+extern ssize_t zcache_compress_poor;
+extern ssize_t zcache_mean_compress_poor;
+extern ssize_t zcache_eph_ate_tail;
+extern ssize_t zcache_eph_ate_tail_failed;
+extern ssize_t zcache_pers_ate_eph;
+extern ssize_t zcache_pers_ate_eph_failed;
+extern ssize_t zcache_evicted_eph_zpages;
+extern ssize_t zcache_evicted_eph_pageframes;
 
 extern ssize_t zcache_last_active_file_pageframes;
 extern ssize_t zcache_last_inactive_file_pageframes;
-- 
1.7.5.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v6 1/3] staging: zcache: fix static variables defined in debug.h but used in mutiple C files

[Konrad Rzeszutek Wilk]

On Wed, Apr 3, 2013 at 6:16 AM, Wanpeng Li <liwanp@linux.vnet.ibm.com> wrote:
> After commit 95bdaee214 ("zcache: Move debugfs code out of zcache-main.c file")
> be merged, most of knods in zcache debugfs just export zero since these variables
> are defined in debug.h but are in use in multiple C files zcache-main.c and debug.c,
> in this case variables can't be treated as shared variables.
>
> Signed-off-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v6 2/3] staging: zcache: introduce zero-filled page stat count

[Wanpeng Li]

Introduce zero-filled page statistics to monitor the number of
zero-filled pages.

Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
---
 drivers/staging/zcache/debug.c       |    3 +++
 drivers/staging/zcache/debug.h       |   17 +++++++++++++++++
 drivers/staging/zcache/zcache-main.c |    4 ++++
 3 files changed, 24 insertions(+), 0 deletions(-)

diff --git a/drivers/staging/zcache/debug.c b/drivers/staging/zcache/debug.c
index faab2a9..daa2691 100644
--- a/drivers/staging/zcache/debug.c
+++ b/drivers/staging/zcache/debug.c
@@ -35,6 +35,8 @@ ssize_t zcache_pers_ate_eph;
 ssize_t zcache_pers_ate_eph_failed;
 ssize_t zcache_evicted_eph_zpages;
 ssize_t zcache_evicted_eph_pageframes;
+ssize_t zcache_zero_filled_pages;
+ssize_t zcache_zero_filled_pages_max;
 
 #define ATTR(x)  { .name = #x, .val = &zcache_##x, }
 static struct debug_entry {
@@ -62,6 +64,7 @@ static struct debug_entry {
 	ATTR(last_inactive_anon_pageframes),
 	ATTR(eph_nonactive_puts_ignored),
 	ATTR(pers_nonactive_puts_ignored),
+	ATTR(zero_filled_pages),
 #ifdef CONFIG_ZCACHE_WRITEBACK
 	ATTR(outstanding_writeback_pages),
 	ATTR(writtenback_pages),
diff --git a/drivers/staging/zcache/debug.h b/drivers/staging/zcache/debug.h
index 8ec82d4..ddad92f 100644
--- a/drivers/staging/zcache/debug.h
+++ b/drivers/staging/zcache/debug.h
@@ -122,6 +122,21 @@ static inline void dec_zcache_pers_zpages(unsigned zpages)
 	zcache_pers_zpages = atomic_sub_return(zpages, &zcache_pers_zpages_atomic);
 }
 
+extern ssize_t zcache_zero_filled_pages;
+static atomic_t zcache_zero_filled_pages_atomic = ATOMIC_INIT(0);
+extern ssize_t zcache_zero_filled_pages_max;
+static inline void inc_zcache_zero_filled_pages(void)
+{
+	zcache_zero_filled_pages = atomic_inc_return(
+					&zcache_zero_filled_pages_atomic);
+	if (zcache_zero_filled_pages > zcache_zero_filled_pages_max)
+		zcache_zero_filled_pages_max = zcache_zero_filled_pages;
+}
+static inline void dec_zcache_zero_filled_pages(void)
+{
+	zcache_zero_filled_pages = atomic_dec_return(
+					&zcache_zero_filled_pages_atomic);
+}
 static inline unsigned long curr_pageframes_count(void)
 {
 	return zcache_pageframes_alloced -
@@ -200,6 +215,8 @@ static inline void inc_zcache_eph_zpages(void) { };
 static inline void dec_zcache_eph_zpages(unsigned zpages) { };
 static inline void inc_zcache_pers_zpages(void) { };
 static inline void dec_zcache_pers_zpages(unsigned zpages) { };
+static inline void inc_zcache_zero_filled_pages(void) { };
+static inline void dec_zcache_zero_filled_pages(void) { };
 static inline unsigned long curr_pageframes_count(void)
 {
 	return 0;
diff --git a/drivers/staging/zcache/zcache-main.c b/drivers/staging/zcache/zcache-main.c
index 1994cab..f3de76d 100644
--- a/drivers/staging/zcache/zcache-main.c
+++ b/drivers/staging/zcache/zcache-main.c
@@ -374,6 +374,7 @@ static void *zcache_pampd_eph_create(char *data, size_t size, bool raw,
 	if (page_is_zero_filled(page)) {
 		clen = 0;
 		zero_filled = true;
+		inc_zcache_zero_filled_pages();
 		goto got_pampd;
 	}
 
@@ -440,6 +441,7 @@ static void *zcache_pampd_pers_create(char *data, size_t size, bool raw,
 	if (page_is_zero_filled(page)) {
 		clen = 0;
 		zero_filled = true;
+		inc_zcache_zero_filled_pages();
 		goto got_pampd;
 	}
 
@@ -652,6 +654,7 @@ static int zcache_pampd_get_data_and_free(char *data, size_t *sizep, bool raw,
 		zpages = 1;
 		if (!raw)
 			*sizep = PAGE_SIZE;
+		dec_zcache_zero_filled_pages();
 		goto zero_fill;
 	}
 
@@ -702,6 +705,7 @@ static void zcache_pampd_free(void *pampd, struct tmem_pool *pool,
 		zero_filled = true;
 		zsize = 0;
 		zpages = 1;
+		dec_zcache_zero_filled_pages();
 	}
 
 	if (pampd_is_remote(pampd) && !zero_filled) {
-- 
1.7.5.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v6 2/3] staging: zcache: introduce zero-filled page stat count

[Konrad Rzeszutek Wilk]

On Wed, Apr 3, 2013 at 6:16 AM, Wanpeng Li <liwanp@linux.vnet.ibm.com> wrote:
> Introduce zero-filled page statistics to monitor the number of
> zero-filled pages.
>
> Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
> Signed-off-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>

Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  drivers/staging/zcache/debug.c       |    3 +++
>  drivers/staging/zcache/debug.h       |   17 +++++++++++++++++
>  drivers/staging/zcache/zcache-main.c |    4 ++++
>  3 files changed, 24 insertions(+), 0 deletions(-)
>
> diff --git a/drivers/staging/zcache/debug.c b/drivers/staging/zcache/debug.c
> index faab2a9..daa2691 100644
> --- a/drivers/staging/zcache/debug.c
> +++ b/drivers/staging/zcache/debug.c
> @@ -35,6 +35,8 @@ ssize_t zcache_pers_ate_eph;
>  ssize_t zcache_pers_ate_eph_failed;
>  ssize_t zcache_evicted_eph_zpages;
>  ssize_t zcache_evicted_eph_pageframes;
> +ssize_t zcache_zero_filled_pages;
> +ssize_t zcache_zero_filled_pages_max;
>
>  #define ATTR(x)  { .name = #x, .val = &zcache_##x, }
>  static struct debug_entry {
> @@ -62,6 +64,7 @@ static struct debug_entry {
>         ATTR(last_inactive_anon_pageframes),
>         ATTR(eph_nonactive_puts_ignored),
>         ATTR(pers_nonactive_puts_ignored),
> +       ATTR(zero_filled_pages),
>  #ifdef CONFIG_ZCACHE_WRITEBACK
>         ATTR(outstanding_writeback_pages),
>         ATTR(writtenback_pages),
> diff --git a/drivers/staging/zcache/debug.h b/drivers/staging/zcache/debug.h
> index 8ec82d4..ddad92f 100644
> --- a/drivers/staging/zcache/debug.h
> +++ b/drivers/staging/zcache/debug.h
> @@ -122,6 +122,21 @@ static inline void dec_zcache_pers_zpages(unsigned zpages)
>         zcache_pers_zpages = atomic_sub_return(zpages, &zcache_pers_zpages_atomic);
>  }
>
> +extern ssize_t zcache_zero_filled_pages;
> +static atomic_t zcache_zero_filled_pages_atomic = ATOMIC_INIT(0);
> +extern ssize_t zcache_zero_filled_pages_max;
> +static inline void inc_zcache_zero_filled_pages(void)
> +{
> +       zcache_zero_filled_pages = atomic_inc_return(
> +                                       &zcache_zero_filled_pages_atomic);
> +       if (zcache_zero_filled_pages > zcache_zero_filled_pages_max)
> +               zcache_zero_filled_pages_max = zcache_zero_filled_pages;
> +}
> +static inline void dec_zcache_zero_filled_pages(void)
> +{
> +       zcache_zero_filled_pages = atomic_dec_return(
> +                                       &zcache_zero_filled_pages_atomic);
> +}
>  static inline unsigned long curr_pageframes_count(void)
>  {
>         return zcache_pageframes_alloced -
> @@ -200,6 +215,8 @@ static inline void inc_zcache_eph_zpages(void) { };
>  static inline void dec_zcache_eph_zpages(unsigned zpages) { };
>  static inline void inc_zcache_pers_zpages(void) { };
>  static inline void dec_zcache_pers_zpages(unsigned zpages) { };
> +static inline void inc_zcache_zero_filled_pages(void) { };
> +static inline void dec_zcache_zero_filled_pages(void) { };
>  static inline unsigned long curr_pageframes_count(void)
>  {
>         return 0;
> diff --git a/drivers/staging/zcache/zcache-main.c b/drivers/staging/zcache/zcache-main.c
> index 1994cab..f3de76d 100644
> --- a/drivers/staging/zcache/zcache-main.c
> +++ b/drivers/staging/zcache/zcache-main.c
> @@ -374,6 +374,7 @@ static void *zcache_pampd_eph_create(char *data, size_t size, bool raw,
>         if (page_is_zero_filled(page)) {
>                 clen = 0;
>                 zero_filled = true;
> +               inc_zcache_zero_filled_pages();
>                 goto got_pampd;
>         }
>
> @@ -440,6 +441,7 @@ static void *zcache_pampd_pers_create(char *data, size_t size, bool raw,
>         if (page_is_zero_filled(page)) {
>                 clen = 0;
>                 zero_filled = true;
> +               inc_zcache_zero_filled_pages();
>                 goto got_pampd;
>         }
>
> @@ -652,6 +654,7 @@ static int zcache_pampd_get_data_and_free(char *data, size_t *sizep, bool raw,
>                 zpages = 1;
>                 if (!raw)
>                         *sizep = PAGE_SIZE;
> +               dec_zcache_zero_filled_pages();
>                 goto zero_fill;
>         }
>
> @@ -702,6 +705,7 @@ static void zcache_pampd_free(void *pampd, struct tmem_pool *pool,
>                 zero_filled = true;
>                 zsize = 0;
>                 zpages = 1;
> +               dec_zcache_zero_filled_pages();
>         }
>
>         if (pampd_is_remote(pampd) && !zero_filled) {
> --
> 1.7.5.4
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH v2 3/3] staging: zcache: clean TODO list

[Wanpeng Li]

Cleanup TODO list since support zero-filled pages more efficiently has 
already done by this patchset.

Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Signed-off-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
---
 drivers/staging/zcache/TODO |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/zcache/TODO b/drivers/staging/zcache/TODO
index ec9aa11..d0c18fa 100644
--- a/drivers/staging/zcache/TODO
+++ b/drivers/staging/zcache/TODO
@@ -61,5 +61,4 @@ ZCACHE FUTURE NEW FUNCTIONALITY
 
 A. Support zsmalloc as an alternative high-density allocator
     (See https://lkml.org/lkml/2013/1/23/511)
-B. Support zero-filled pages more efficiently
-C. Possibly support three zbuds per pageframe when space allows
+B. Possibly support three zbuds per pageframe when space allows
-- 
1.7.5.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[Wanpeng Li]

On Wed, Apr 03, 2013 at 06:16:20PM +0800, Wanpeng Li wrote:
>Changelog:
> v5 -> v6:
>  * shove variables in debug.c and in debug.h just have an extern, spotted by Konrad
>  * update patch description, spotted by Konrad
> v4 -> v5:
>  * fix compile error, reported by Fengguang, Geert
>  * add check for !is_ephemeral(pool), spotted by Bob
> v3 -> v4:
>  * handle duplication in page_is_zero_filled, spotted by Bob
>  * fix zcache writeback in dubugfs
>  * fix pers_pageframes|_max isn't exported in debugfs
>  * fix static variable defined in debug.h but used in multiple C files
>  * rebase on Greg's staging-next
> v2 -> v3:
>  * increment/decrement zcache_[eph|pers]_zpages for zero-filled pages, spotted by Dan
>  * replace "zero" or "zero page" by "zero_filled_page", spotted by Dan
> v1 -> v2:
>  * avoid changing tmem.[ch] entirely, spotted by Dan.
>  * don't accumulate [eph|pers]pageframe and [eph|pers]zpages for
>    zero-filled pages, spotted by Dan
>  * cleanup TODO list
>  * add Dan Acked-by.
>

Hi Dan,

Some issues against Ramster:

- Ramster who takes advantage of zcache also should support zero-filled 
  pages more efficiently, correct? It doesn't handle zero-filled pages well
  currently.
- Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache
  all are exported in /sys/kernel/debug/, should we unify them?
- If ramster also should move DebugFS counters to a single file like
  zcache do?

If you confirm these issues are make sense to fix, I will start coding. ;-)

Regards,
Wanpeng Li 

>Motivation:
>
>- Seth Jennings points out compress zero-filled pages with LZO(a lossless
>  data compression algorithm) will waste memory and result in fragmentation.
>  https://lkml.org/lkml/2012/8/14/347
>- Dan Magenheimer add "Support zero-filled pages more efficiently" feature
>  in zcache TODO list https://lkml.org/lkml/2013/2/13/503
>
>Design:
>
>- For store page, capture zero-filled pages(evicted clean page cache pages and
>  swap pages), but don't compress them, set pampd which store zpage address to
>  0x2(since 0x0 and 0x1 has already been ocuppied) to mark special zero-filled
>  case and take advantage of tmem infrastructure to transform handle-tuple(pool
>  id, object id, and an index) to a pampd. Twice compress zero-filled pages will
>  contribute to one zcache_[eph|pers]_pageframes count accumulated.
>- For load page, traverse tmem hierachical to transform handle-tuple to pampd
>  and identify zero-filled case by pampd equal to 0x2 when filesystem reads
>  file pages or a page needs to be swapped in, then refill the page to zero
>  and return.
>
>Test:
>
>dd if=/dev/zero of=zerofile bs=1MB count=500
>vmtouch -t zerofile
>vmtouch -e zerofile
>
>formula:
>- fragmentation level = (zcache_[eph|pers]_pageframes * PAGE_SIZE - zcache_[eph|pers]_zbytes)
>  * 100 / (zcache_[eph|pers]_pageframes * PAGE_SIZE)
>- memory zcache occupy = zcache_[eph|pers]_zbytes
>
>Result:
>
>without zero-filled awareness:
>- fragmentation level: 98%
>- memory zcache occupy: 238MB
>with zero-filled awareness:
>- fragmentation level: 0%
>- memory zcache occupy: 0MB
>
>Wanpeng Li (3):
>  staging: zcache: fix static variables defined in debug.h but used in
>    mutiple C files
>  staging: zcache: introduce zero-filled page stat count
>  staging: zcache: clean TODO list
>
> drivers/staging/zcache/TODO          |    3 +-
> drivers/staging/zcache/debug.c       |   35 +++++++++++++++
> drivers/staging/zcache/debug.h       |   79 ++++++++++++++++++++-------------
> drivers/staging/zcache/zcache-main.c |    4 ++
> 4 files changed, 88 insertions(+), 33 deletions(-)
>
>-- 
>1.7.5.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[Wanpeng Li]

On Wed, Apr 03, 2013 at 06:16:20PM +0800, Wanpeng Li wrote:
>Changelog:
> v5 -> v6:
>  * shove variables in debug.c and in debug.h just have an extern, spotted by Konrad
>  * update patch description, spotted by Konrad
> v4 -> v5:
>  * fix compile error, reported by Fengguang, Geert
>  * add check for !is_ephemeral(pool), spotted by Bob
> v3 -> v4:
>  * handle duplication in page_is_zero_filled, spotted by Bob
>  * fix zcache writeback in dubugfs
>  * fix pers_pageframes|_max isn't exported in debugfs
>  * fix static variable defined in debug.h but used in multiple C files
>  * rebase on Greg's staging-next
> v2 -> v3:
>  * increment/decrement zcache_[eph|pers]_zpages for zero-filled pages, spotted by Dan
>  * replace "zero" or "zero page" by "zero_filled_page", spotted by Dan
> v1 -> v2:
>  * avoid changing tmem.[ch] entirely, spotted by Dan.
>  * don't accumulate [eph|pers]pageframe and [eph|pers]zpages for
>    zero-filled pages, spotted by Dan
>  * cleanup TODO list
>  * add Dan Acked-by.
>

Hi Dan,

Some issues against Ramster:

- Ramster who takes advantage of zcache also should support zero-filled 
  pages more efficiently, correct? It doesn't handle zero-filled pages well
  currently.
- Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache
  all are exported in /sys/kernel/debug/, should we unify them?
- If ramster also should move DebugFS counters to a single file like
  zcache do?

If you confirm these issues are make sense to fix, I will start coding. ;-)

Regards,
Wanpeng Li 

>Motivation:
>
>- Seth Jennings points out compress zero-filled pages with LZO(a lossless
>  data compression algorithm) will waste memory and result in fragmentation.
>  https://lkml.org/lkml/2012/8/14/347
>- Dan Magenheimer add "Support zero-filled pages more efficiently" feature
>  in zcache TODO list https://lkml.org/lkml/2013/2/13/503
>
>Design:
>
>- For store page, capture zero-filled pages(evicted clean page cache pages and
>  swap pages), but don't compress them, set pampd which store zpage address to
>  0x2(since 0x0 and 0x1 has already been ocuppied) to mark special zero-filled
>  case and take advantage of tmem infrastructure to transform handle-tuple(pool
>  id, object id, and an index) to a pampd. Twice compress zero-filled pages will
>  contribute to one zcache_[eph|pers]_pageframes count accumulated.
>- For load page, traverse tmem hierachical to transform handle-tuple to pampd
>  and identify zero-filled case by pampd equal to 0x2 when filesystem reads
>  file pages or a page needs to be swapped in, then refill the page to zero
>  and return.
>
>Test:
>
>dd if=/dev/zero of=zerofile bs=1MB count=500
>vmtouch -t zerofile
>vmtouch -e zerofile
>
>formula:
>- fragmentation level = (zcache_[eph|pers]_pageframes * PAGE_SIZE - zcache_[eph|pers]_zbytes)
>  * 100 / (zcache_[eph|pers]_pageframes * PAGE_SIZE)
>- memory zcache occupy = zcache_[eph|pers]_zbytes
>
>Result:
>
>without zero-filled awareness:
>- fragmentation level: 98%
>- memory zcache occupy: 238MB
>with zero-filled awareness:
>- fragmentation level: 0%
>- memory zcache occupy: 0MB
>
>Wanpeng Li (3):
>  staging: zcache: fix static variables defined in debug.h but used in
>    mutiple C files
>  staging: zcache: introduce zero-filled page stat count
>  staging: zcache: clean TODO list
>
> drivers/staging/zcache/TODO          |    3 +-
> drivers/staging/zcache/debug.c       |   35 +++++++++++++++
> drivers/staging/zcache/debug.h       |   79 ++++++++++++++++++++-------------
> drivers/staging/zcache/zcache-main.c |    4 ++
> 4 files changed, 88 insertions(+), 33 deletions(-)
>
>-- 
>1.7.5.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[Dan Magenheimer]

> From: Wanpeng Li [mailto:liwanp@linux.vnet.ibm.com]
> Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
> 
> Hi Dan,
> 
> Some issues against Ramster:
> 
> - Ramster who takes advantage of zcache also should support zero-filled
>   pages more efficiently, correct? It doesn't handle zero-filled pages well
>   currently.

When you first posted your patchset I took a quick look at ramster
and it looked like your patchset should work for ramster also.
However I didn't actually run ramster to try it so there may
be a bug.  If it doesn't work, I would very much appreciate a patch.

> - Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache
>   all are exported in /sys/kernel/debug/, should we unify them?

That would be great.

> - If ramster also should move DebugFS counters to a single file like
>   zcache do?

Sure!  I am concerned about Konrad's patches adding debug.c as they
add many global variables.  They are only required when ZCACHE_DEBUG
is enabled so they may be ok.  If not, adding ramster variables
to debug.c may make the problem worse.

> If you confirm these issues are make sense to fix, I will start coding. ;-)

That would be great.  Note that I have a how-to for ramster here:

https://oss.oracle.com/projects/tmem/dist/files/RAMster/HOWTO-120817 

If when you are testing you find that this how-to has mistakes,
please let me know.  Or feel free to add the (corrected) how-to file
as a patch in your patchset.

Thanks very much, Wanpeng, for your great contributions!

(Ric, since you have expressed interest in ramster, if you try it and
find corrections to the how-to file above, your input would be
very much appreciated also!)

Dan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[Dan Magenheimer]

> From: Dan Magenheimer
> Subject: RE: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
> 
> > From: Wanpeng Li [mailto:liwanp@linux.vnet.ibm.com]
> > Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
> >
> > Hi Dan,
> >
> > Some issues against Ramster:
> >
> 
> Sure!  I am concerned about Konrad's patches adding debug.c as they
> add many global variables.  They are only required when ZCACHE_DEBUG
> is enabled so they may be ok.  If not, adding ramster variables
> to debug.c may make the problem worse.

Oops, I just noticed/remembered that ramster uses BOTH debugfs and sysfs.
The sysfs variables are all currently required, i.e. for configuration
so should not be tied to debugfs or a DEBUG config option.  However,
if there is a more acceptable way to implement the function of
those sysfs variables, that would be fine.

Thanks,
Dan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[Wanpeng Li]

On Sun, Apr 07, 2013 at 10:59:18AM -0700, Dan Magenheimer wrote:
>> From: Dan Magenheimer
>> Subject: RE: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
>> 
>> > From: Wanpeng Li [mailto:liwanp@linux.vnet.ibm.com]
>> > Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
>> >
>> > Hi Dan,
>> >
>> > Some issues against Ramster:
>> >
>> 
>> Sure!  I am concerned about Konrad's patches adding debug.c as they
>> add many global variables.  They are only required when ZCACHE_DEBUG
>> is enabled so they may be ok.  If not, adding ramster variables
>> to debug.c may make the problem worse.
>
>Oops, I just noticed/remembered that ramster uses BOTH debugfs and sysfs.
>The sysfs variables are all currently required, i.e. for configuration
>so should not be tied to debugfs or a DEBUG config option.  However,
>if there is a more acceptable way to implement the function of
>those sysfs variables, that would be fine.

So if we need move debugfs counters to a single debug.c in
zcache/ramster/ and introduce RAMSTER_DEBUG? The work similiar 
as Konrad done against zcache. ;-)

Regards,
Wanpeng Li 

>
>Thanks,
>Dan
>
>--
>To unsubscribe, send a message with 'unsubscribe linux-mm' in
>the body to majordomo@kvack.org.  For more info on Linux MM,
>see: http://www.linux-mm.org/ .
>Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[Wanpeng Li]

On Sun, Apr 07, 2013 at 10:59:18AM -0700, Dan Magenheimer wrote:
>> From: Dan Magenheimer
>> Subject: RE: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
>> 
>> > From: Wanpeng Li [mailto:liwanp@linux.vnet.ibm.com]
>> > Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
>> >
>> > Hi Dan,
>> >
>> > Some issues against Ramster:
>> >
>> 
>> Sure!  I am concerned about Konrad's patches adding debug.c as they
>> add many global variables.  They are only required when ZCACHE_DEBUG
>> is enabled so they may be ok.  If not, adding ramster variables
>> to debug.c may make the problem worse.
>
>Oops, I just noticed/remembered that ramster uses BOTH debugfs and sysfs.
>The sysfs variables are all currently required, i.e. for configuration
>so should not be tied to debugfs or a DEBUG config option.  However,
>if there is a more acceptable way to implement the function of
>those sysfs variables, that would be fine.

So if we need move debugfs counters to a single debug.c in
zcache/ramster/ and introduce RAMSTER_DEBUG? The work similiar 
as Konrad done against zcache. ;-)

Regards,
Wanpeng Li 

>
>Thanks,
>Dan
>
>--
>To unsubscribe, send a message with 'unsubscribe linux-mm' in
>the body to majordomo@kvack.org.  For more info on Linux MM,
>see: http://www.linux-mm.org/ .
>Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[Wanpeng Li]

On Sun, Apr 07, 2013 at 10:51:27AM -0700, Dan Magenheimer wrote:
>> From: Wanpeng Li [mailto:liwanp@linux.vnet.ibm.com]
>> Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
>> 
>> Hi Dan,
>> 
>> Some issues against Ramster:
>> 
>> - Ramster who takes advantage of zcache also should support zero-filled
>>   pages more efficiently, correct? It doesn't handle zero-filled pages well
>>   currently.
>
>When you first posted your patchset I took a quick look at ramster
>and it looked like your patchset should work for ramster also.
>However I didn't actually run ramster to try it so there may
>be a bug.  If it doesn't work, I would very much appreciate a patch.
>

I have already fix it. ;-)

>> - Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache
>>   all are exported in /sys/kernel/debug/, should we unify them?
>
>That would be great.
>
>> - If ramster also should move DebugFS counters to a single file like
>>   zcache do?
>
>Sure!  I am concerned about Konrad's patches adding debug.c as they
>add many global variables.  They are only required when ZCACHE_DEBUG
>is enabled so they may be ok.  If not, adding ramster variables
>to debug.c may make the problem worse.

I move counters which use dubugfs to single file in zcache/ramster/ and 
introduce CONFIG_RAMSTER_DEBUG, patchset has already done and under test.

>
>> If you confirm these issues are make sense to fix, I will start coding. ;-)
>
>That would be great.  Note that I have a how-to for ramster here:
>
>https://oss.oracle.com/projects/tmem/dist/files/RAMster/HOWTO-120817 
>
>If when you are testing you find that this how-to has mistakes,
>please let me know.  Or feel free to add the (corrected) how-to file
>as a patch in your patchset.
>

Ok, I will add it to my patchset. ;-)

Regards,
Wanpeng Li 

>Thanks very much, Wanpeng, for your great contributions!
>
>(Ric, since you have expressed interest in ramster, if you try it and
>find corrections to the how-to file above, your input would be
>very much appreciated also!)
>
>Dan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[Wanpeng Li]

On Sun, Apr 07, 2013 at 10:51:27AM -0700, Dan Magenheimer wrote:
>> From: Wanpeng Li [mailto:liwanp@linux.vnet.ibm.com]
>> Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
>> 
>> Hi Dan,
>> 
>> Some issues against Ramster:
>> 
>> - Ramster who takes advantage of zcache also should support zero-filled
>>   pages more efficiently, correct? It doesn't handle zero-filled pages well
>>   currently.
>
>When you first posted your patchset I took a quick look at ramster
>and it looked like your patchset should work for ramster also.
>However I didn't actually run ramster to try it so there may
>be a bug.  If it doesn't work, I would very much appreciate a patch.
>

I have already fix it. ;-)

>> - Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache
>>   all are exported in /sys/kernel/debug/, should we unify them?
>
>That would be great.
>
>> - If ramster also should move DebugFS counters to a single file like
>>   zcache do?
>
>Sure!  I am concerned about Konrad's patches adding debug.c as they
>add many global variables.  They are only required when ZCACHE_DEBUG
>is enabled so they may be ok.  If not, adding ramster variables
>to debug.c may make the problem worse.

I move counters which use dubugfs to single file in zcache/ramster/ and 
introduce CONFIG_RAMSTER_DEBUG, patchset has already done and under test.

>
>> If you confirm these issues are make sense to fix, I will start coding. ;-)
>
>That would be great.  Note that I have a how-to for ramster here:
>
>https://oss.oracle.com/projects/tmem/dist/files/RAMster/HOWTO-120817 
>
>If when you are testing you find that this how-to has mistakes,
>please let me know.  Or feel free to add the (corrected) how-to file
>as a patch in your patchset.
>

Ok, I will add it to my patchset. ;-)

Regards,
Wanpeng Li 

>Thanks very much, Wanpeng, for your great contributions!
>
>(Ric, since you have expressed interest in ramster, if you try it and
>find corrections to the how-to file above, your input would be
>very much appreciated also!)
>
>Dan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[Wanpeng Li]

On Sun, Apr 07, 2013 at 10:51:27AM -0700, Dan Magenheimer wrote:
>> From: Wanpeng Li [mailto:liwanp@linux.vnet.ibm.com]
>> Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
>> 
>> Hi Dan,
>> 
>> Some issues against Ramster:
>> 
>> - Ramster who takes advantage of zcache also should support zero-filled
>>   pages more efficiently, correct? It doesn't handle zero-filled pages well
>>   currently.
>
>When you first posted your patchset I took a quick look at ramster
>and it looked like your patchset should work for ramster also.
>However I didn't actually run ramster to try it so there may
>be a bug.  If it doesn't work, I would very much appreciate a patch.
>
>> - Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache
>>   all are exported in /sys/kernel/debug/, should we unify them?
>
>That would be great.
>
>> - If ramster also should move DebugFS counters to a single file like
>>   zcache do?
>
>Sure!  I am concerned about Konrad's patches adding debug.c as they
>add many global variables.  They are only required when ZCACHE_DEBUG
>is enabled so they may be ok.  If not, adding ramster variables
>to debug.c may make the problem worse.
>
>> If you confirm these issues are make sense to fix, I will start coding. ;-)

Hi Dan,

>
>That would be great.  Note that I have a how-to for ramster here:
>
>https://oss.oracle.com/projects/tmem/dist/files/RAMster/HOWTO-120817 
>
>If when you are testing you find that this how-to has mistakes,
>please let me know.  Or feel free to add the (corrected) how-to file
>as a patch in your patchset.

You can review my patchset [PATCH 00/10] staging: zcache/ramster: fix
and ramster/debugfs improvement

Just update TESTING RAMSTER part since some knobs are eported in 
/sys/kernel/debug/ramster/ instead of /sys/kernel/mm/ramster/ for 
your HOWTO file. 

D. TESTING RAMSTER 
+2) To see if RAMster is working, on the remote system

Regards,
Wanpeng Li 

>
>Thanks very much, Wanpeng, for your great contributions!
>
>(Ric, since you have expressed interest in ramster, if you try it and
>find corrections to the how-to file above, your input would be
>very much appreciated also!)
>
>Dan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[Wanpeng Li]

On Sun, Apr 07, 2013 at 10:51:27AM -0700, Dan Magenheimer wrote:
>> From: Wanpeng Li [mailto:liwanp@linux.vnet.ibm.com]
>> Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
>> 
>> Hi Dan,
>> 
>> Some issues against Ramster:
>> 
>> - Ramster who takes advantage of zcache also should support zero-filled
>>   pages more efficiently, correct? It doesn't handle zero-filled pages well
>>   currently.
>
>When you first posted your patchset I took a quick look at ramster
>and it looked like your patchset should work for ramster also.
>However I didn't actually run ramster to try it so there may
>be a bug.  If it doesn't work, I would very much appreciate a patch.
>
>> - Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache
>>   all are exported in /sys/kernel/debug/, should we unify them?
>
>That would be great.
>
>> - If ramster also should move DebugFS counters to a single file like
>>   zcache do?
>
>Sure!  I am concerned about Konrad's patches adding debug.c as they
>add many global variables.  They are only required when ZCACHE_DEBUG
>is enabled so they may be ok.  If not, adding ramster variables
>to debug.c may make the problem worse.
>
>> If you confirm these issues are make sense to fix, I will start coding. ;-)

Hi Dan,

>
>That would be great.  Note that I have a how-to for ramster here:
>
>https://oss.oracle.com/projects/tmem/dist/files/RAMster/HOWTO-120817 
>
>If when you are testing you find that this how-to has mistakes,
>please let me know.  Or feel free to add the (corrected) how-to file
>as a patch in your patchset.

You can review my patchset [PATCH 00/10] staging: zcache/ramster: fix
and ramster/debugfs improvement

Just update TESTING RAMSTER part since some knobs are eported in 
/sys/kernel/debug/ramster/ instead of /sys/kernel/mm/ramster/ for 
your HOWTO file. 

D. TESTING RAMSTER 
+2) To see if RAMster is working, on the remote system

Regards,
Wanpeng Li 

>
>Thanks very much, Wanpeng, for your great contributions!
>
>(Ric, since you have expressed interest in ramster, if you try it and
>find corrections to the how-to file above, your input would be
>very much appreciated also!)
>
>Dan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently

[Ric Mason]

cc Bob
On 04/07/2013 05:03 PM, Wanpeng Li wrote:
> On Wed, Apr 03, 2013 at 06:16:20PM +0800, Wanpeng Li wrote:
>> Changelog:
>> v5 -> v6:
>>   * shove variables in debug.c and in debug.h just have an extern, spotted by Konrad
>>   * update patch description, spotted by Konrad
>> v4 -> v5:
>>   * fix compile error, reported by Fengguang, Geert
>>   * add check for !is_ephemeral(pool), spotted by Bob
>> v3 -> v4:
>>   * handle duplication in page_is_zero_filled, spotted by Bob
>>   * fix zcache writeback in dubugfs
>>   * fix pers_pageframes|_max isn't exported in debugfs
>>   * fix static variable defined in debug.h but used in multiple C files
>>   * rebase on Greg's staging-next
>> v2 -> v3:
>>   * increment/decrement zcache_[eph|pers]_zpages for zero-filled pages, spotted by Dan
>>   * replace "zero" or "zero page" by "zero_filled_page", spotted by Dan
>> v1 -> v2:
>>   * avoid changing tmem.[ch] entirely, spotted by Dan.
>>   * don't accumulate [eph|pers]pageframe and [eph|pers]zpages for
>>     zero-filled pages, spotted by Dan
>>   * cleanup TODO list
>>   * add Dan Acked-by.
>>
> Hi Dan,
>
> Some issues against Ramster:
>
> - Ramster who takes advantage of zcache also should support zero-filled
>    pages more efficiently, correct? It doesn't handle zero-filled pages well
>    currently.
> - Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache
>    all are exported in /sys/kernel/debug/, should we unify them?
> - If ramster also should move DebugFS counters to a single file like
>    zcache do?
>
> If you confirm these issues are make sense to fix, I will start coding. ;-)
>
> Regards,
> Wanpeng Li
>
>> Motivation:
>>
>> - Seth Jennings points out compress zero-filled pages with LZO(a lossless
>>   data compression algorithm) will waste memory and result in fragmentation.
>>   https://lkml.org/lkml/2012/8/14/347
>> - Dan Magenheimer add "Support zero-filled pages more efficiently" feature
>>   in zcache TODO list https://lkml.org/lkml/2013/2/13/503
>>
>> Design:
>>
>> - For store page, capture zero-filled pages(evicted clean page cache pages and
>>   swap pages), but don't compress them, set pampd which store zpage address to
>>   0x2(since 0x0 and 0x1 has already been ocuppied) to mark special zero-filled
>>   case and take advantage of tmem infrastructure to transform handle-tuple(pool
>>   id, object id, and an index) to a pampd. Twice compress zero-filled pages will
>>   contribute to one zcache_[eph|pers]_pageframes count accumulated.
>> - For load page, traverse tmem hierachical to transform handle-tuple to pampd
>>   and identify zero-filled case by pampd equal to 0x2 when filesystem reads
>>   file pages or a page needs to be swapped in, then refill the page to zero
>>   and return.
>>
>> Test:
>>
>> dd if=/dev/zero of=zerofile bs=1MB count=500
>> vmtouch -t zerofile
>> vmtouch -e zerofile
>>
>> formula:
>> - fragmentation level = (zcache_[eph|pers]_pageframes * PAGE_SIZE - zcache_[eph|pers]_zbytes)
>>   * 100 / (zcache_[eph|pers]_pageframes * PAGE_SIZE)
>> - memory zcache occupy = zcache_[eph|pers]_zbytes
>>
>> Result:
>>
>> without zero-filled awareness:
>> - fragmentation level: 98%
>> - memory zcache occupy: 238MB
>> with zero-filled awareness:
>> - fragmentation level: 0%
>> - memory zcache occupy: 0MB
>>
>> Wanpeng Li (3):
>>   staging: zcache: fix static variables defined in debug.h but used in
>>     mutiple C files
>>   staging: zcache: introduce zero-filled page stat count
>>   staging: zcache: clean TODO list
>>
>> drivers/staging/zcache/TODO          |    3 +-
>> drivers/staging/zcache/debug.c       |   35 +++++++++++++++
>> drivers/staging/zcache/debug.h       |   79 ++++++++++++++++++++-------------
>> drivers/staging/zcache/zcache-main.c |    4 ++
>> 4 files changed, 88 insertions(+), 33 deletions(-)
>>
>> -- 
>> 1.7.5.4
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>