linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
* [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication
@ 2014-11-13 21:01 Timofey Titovets
  2014-11-13 21:01 ` [RESEND][PATCH V3 1/4] KSM: Add auto flag new VMA as VM_MERGEABLE Timofey Titovets
                   ` (4 more replies)
  0 siblings, 5 replies; 7+ messages in thread
From: Timofey Titovets @ 2014-11-13 21:01 UTC (permalink / raw)
  To: linux-mm; +Cc: akpm, Timofey Titovets

Good time of day List,
this tiny series of patches implement feature for auto deduping all anonymous memory.
mark_new_vma - new ksm sysfs interface
Every time then new vma created and mark_new_vma set to 1, then will be vma marked as VM_MERGEABLE and added to ksm queue.
This can produce small overheads
(I have not catch any problems or slowdown)

This is useful for:
Android (CM) devs which implement ksm support with patching system.
Users of tiny pc.
Servers what not use KVM but use something very releated, like containers.

Can be pulled from:
https://github.com/Nefelim4ag/linux.git ksm_improvements

For tests:
I have tested it and it working very good. For testing apply it and enable ksm:
echo 1 | sudo tee /sys/kernel/mm/ksm/run
This show how much memory saved:
echo $[$(cat /sys/kernel/mm/ksm/pages_shared)*$(getconf PAGE_SIZE)/1024 ]KB

On my system i save ~1% of memory 26 Mb/2100 Mb (deduped)/(used)

v2:
	Added Kconfig for control default value of mark_new_vma
	Added sysfs interface for control mark_new_vma
	Splitted in several patches

v3:
	Documentation for ksm changed for clarify new cha

Timofey Titovets (4):
  KSM: Add auto flag new VMA as VM_MERGEABLE
  KSM: Add to sysfs - mark_new_vma
  KSM: Add config to control mark_new_vma
  KSM: mark_new_vma added to Documentation.

 Documentation/vm/ksm.txt |  7 +++++++
 include/linux/ksm.h      | 39 +++++++++++++++++++++++++++++++++++++++
 mm/Kconfig               |  7 +++++++
 mm/ksm.c                 | 39 ++++++++++++++++++++++++++++++++++++++-
 mm/mmap.c                | 17 +++++++++++++++++
 5 files changed, 108 insertions(+), 1 deletion(-)

-- 
2.1.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [RESEND][PATCH V3 1/4] KSM: Add auto flag new VMA as VM_MERGEABLE
  2014-11-13 21:01 [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication Timofey Titovets
@ 2014-11-13 21:01 ` Timofey Titovets
  2014-11-13 21:01 ` [RESEND][PATCH V3 2/4] KSM: Add to sysfs - mark_new_vma Timofey Titovets
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 7+ messages in thread
From: Timofey Titovets @ 2014-11-13 21:01 UTC (permalink / raw)
  To: linux-mm; +Cc: akpm, Timofey Titovets

Implement two functions:
ksm_vm_flags_mod() - if existing flags supported by ksm - then mark like VM_MERGEABLE
ksm_vma_add_new() - If vma marked as VM_MERGEABLE add it to ksm page queue

Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
---
 include/linux/ksm.h | 31 +++++++++++++++++++++++++++++++
 mm/mmap.c           | 17 +++++++++++++++++
 2 files changed, 48 insertions(+)

diff --git a/include/linux/ksm.h b/include/linux/ksm.h
index 3be6bb1..c3fff64 100644
--- a/include/linux/ksm.h
+++ b/include/linux/ksm.h
@@ -76,6 +76,29 @@ struct page *ksm_might_need_to_copy(struct page *page,
 int rmap_walk_ksm(struct page *page, struct rmap_walk_control *rwc);
 void ksm_migrate_page(struct page *newpage, struct page *oldpage);
 
+/*
+ * Allow to mark new vma as VM_MERGEABLE
+ */
+#ifndef VM_SAO
+#define VM_SAO 0
+#endif
+static inline void ksm_vm_flags_mod(unsigned long *vm_flags)
+{
+	if (*vm_flags & (VM_MERGEABLE | VM_SHARED  | VM_MAYSHARE   |
+			 VM_PFNMAP    | VM_IO      | VM_DONTEXPAND |
+			 VM_HUGETLB | VM_NONLINEAR | VM_MIXEDMAP   | VM_SAO) )
+		return;
+	*vm_flags |= VM_MERGEABLE;
+}
+
+static inline void ksm_vma_add_new(struct vm_area_struct *vma)
+{
+	struct mm_struct *mm = vma->vm_mm;
+	if (!test_bit(MMF_VM_MERGEABLE, &mm->flags)) {
+		__ksm_enter(mm);
+	}
+}
+
 #else  /* !CONFIG_KSM */
 
 static inline int ksm_fork(struct mm_struct *mm, struct mm_struct *oldmm)
@@ -92,6 +115,14 @@ static inline int PageKsm(struct page *page)
 	return 0;
 }
 
+static inline void ksm_vm_flags_mod(unsigned long *vm_flags_p)
+{
+}
+
+void ksm_vma_add_new(struct vm_area_struct *vma)
+{
+}
+
 #ifdef CONFIG_MMU
 static inline int ksm_madvise(struct vm_area_struct *vma, unsigned long start,
 		unsigned long end, int advice, unsigned long *vm_flags)
diff --git a/mm/mmap.c b/mm/mmap.c
index 7f85520..ce0073e 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -41,6 +41,7 @@
 #include <linux/notifier.h>
 #include <linux/memory.h>
 #include <linux/printk.h>
+#include <linux/ksm.h>
 
 #include <asm/uaccess.h>
 #include <asm/cacheflush.h>
@@ -911,10 +912,14 @@ again:			remove_next = 1 + (end > next->vm_end);
 			vma_gap_update(next);
 		else
 			mm->highest_vm_end = end;
+	} else {
+		if (next && !insert)
+			ksm_vma_add_new(next);
 	}
 	if (insert && file)
 		uprobe_mmap(insert);
 
+	ksm_vma_add_new(vma);
 	validate_mm(mm);
 
 	return 0;
@@ -1307,6 +1312,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 	vm_flags = calc_vm_prot_bits(prot) | calc_vm_flag_bits(flags) |
 			mm->def_flags | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC;
 
+	/* If ksm is enabled, we add VM_MERGABLE to new VMAs. */
+	ksm_vm_flags_mod(&vm_flags);
+
 	if (flags & MAP_LOCKED)
 		if (!can_do_mlock())
 			return -EPERM;
@@ -1648,6 +1656,7 @@ munmap_back:
 			allow_write_access(file);
 	}
 	file = vma->vm_file;
+	ksm_vma_add_new(vma);
 out:
 	perf_event_mmap(vma);
 
@@ -2484,6 +2493,8 @@ static int __split_vma(struct mm_struct *mm, struct vm_area_struct *vma,
 	else
 		err = vma_adjust(vma, vma->vm_start, addr, vma->vm_pgoff, new);
 
+	ksm_vma_add_new(vma);
+
 	/* Success. */
 	if (!err)
 		return 0;
@@ -2659,6 +2670,9 @@ static unsigned long do_brk(unsigned long addr, unsigned long len)
 	if (error)
 		return error;
 
+	/* If ksm is enabled, we add VM_MERGABLE to new VMAs. */
+	ksm_vm_flags_mod(&flags);
+
 	/*
 	 * mm->mmap_sem is required to protect against another thread
 	 * changing the mappings in case we sleep.
@@ -2708,6 +2722,7 @@ static unsigned long do_brk(unsigned long addr, unsigned long len)
 	vma->vm_flags = flags;
 	vma->vm_page_prot = vm_get_page_prot(flags);
 	vma_link(mm, vma, prev, rb_link, rb_parent);
+	ksm_vma_add_new(vma);
 out:
 	perf_event_mmap(vma);
 	mm->total_vm += len >> PAGE_SHIFT;
@@ -2887,6 +2902,7 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,
 				new_vma->vm_ops->open(new_vma);
 			vma_link(mm, new_vma, prev, rb_link, rb_parent);
 			*need_rmap_locks = false;
+			ksm_vma_add_new(new_vma);
 		}
 	}
 	return new_vma;
@@ -3004,6 +3020,7 @@ static struct vm_area_struct *__install_special_mapping(
 	mm->total_vm += len >> PAGE_SHIFT;
 
 	perf_event_mmap(vma);
+	ksm_vma_add_new(vma);
 
 	return vma;
 
-- 
2.1.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [RESEND][PATCH V3 2/4] KSM: Add to sysfs - mark_new_vma
  2014-11-13 21:01 [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication Timofey Titovets
  2014-11-13 21:01 ` [RESEND][PATCH V3 1/4] KSM: Add auto flag new VMA as VM_MERGEABLE Timofey Titovets
@ 2014-11-13 21:01 ` Timofey Titovets
  2014-11-13 21:01 ` [RESEND][PATCH V3 3/4] KSM: Add config to control mark_new_vma Timofey Titovets
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 7+ messages in thread
From: Timofey Titovets @ 2014-11-13 21:01 UTC (permalink / raw)
  To: linux-mm; +Cc: akpm, Timofey Titovets

It allow to control user mark new vma as VM_MERGEABLE or not
Create new sysfs interface /sys/kernel/mm/ksm/mark_new_vma
1 - enabled - mark new allocated vma as VM_MERGEABLE and add it to ksm queue
0 - disable it

Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
---
 include/linux/ksm.h | 10 +++++++++-
 mm/ksm.c            | 39 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 47 insertions(+), 2 deletions(-)

diff --git a/include/linux/ksm.h b/include/linux/ksm.h
index c3fff64..deb60fc 100644
--- a/include/linux/ksm.h
+++ b/include/linux/ksm.h
@@ -76,6 +76,12 @@ struct page *ksm_might_need_to_copy(struct page *page,
 int rmap_walk_ksm(struct page *page, struct rmap_walk_control *rwc);
 void ksm_migrate_page(struct page *newpage, struct page *oldpage);
 
+
+/* Mark new vma as mergeable */
+#define ALLOW	1
+#define DENY	0
+extern unsigned mark_new_vma __read_mostly;
+
 /*
  * Allow to mark new vma as VM_MERGEABLE
  */
@@ -84,6 +90,8 @@ void ksm_migrate_page(struct page *newpage, struct page *oldpage);
 #endif
 static inline void ksm_vm_flags_mod(unsigned long *vm_flags)
 {
+	if (!mark_new_vma)
+		return;
 	if (*vm_flags & (VM_MERGEABLE | VM_SHARED  | VM_MAYSHARE   |
 			 VM_PFNMAP    | VM_IO      | VM_DONTEXPAND |
 			 VM_HUGETLB | VM_NONLINEAR | VM_MIXEDMAP   | VM_SAO) )
@@ -94,7 +102,7 @@ static inline void ksm_vm_flags_mod(unsigned long *vm_flags)
 static inline void ksm_vma_add_new(struct vm_area_struct *vma)
 {
 	struct mm_struct *mm = vma->vm_mm;
-	if (!test_bit(MMF_VM_MERGEABLE, &mm->flags)) {
+	if (mark_new_vma && !test_bit(MMF_VM_MERGEABLE, &mm->flags)) {
 		__ksm_enter(mm);
 	}
 }
diff --git a/mm/ksm.c b/mm/ksm.c
index 6b2e337..7bfb616 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -223,6 +223,12 @@ static unsigned int ksm_thread_pages_to_scan = 100;
 /* Milliseconds ksmd should sleep between batches */
 static unsigned int ksm_thread_sleep_millisecs = 20;
 
+/* Mark new vma as mergeable */
+#ifndef CONFIG_KSM_MARK_NEW_VMA
+#define CONFIG_KSM_MARK_NEW_VMA DENY
+#endif
+unsigned mark_new_vma = CONFIG_KSM_MARK_NEW_VMA;
+
 #ifdef CONFIG_NUMA
 /* Zeroed when merging across nodes is not allowed */
 static unsigned int ksm_merge_across_nodes = 1;
@@ -2127,6 +2133,34 @@ static ssize_t pages_to_scan_store(struct kobject *kobj,
 }
 KSM_ATTR(pages_to_scan);
 
+static ssize_t mark_new_vma_show(struct kobject *kobj, struct kobj_attribute *attr,
+			char *buf)
+{
+	return sprintf(buf, "%u\n", mark_new_vma);
+}
+static ssize_t mark_new_vma_store(struct kobject *kobj,
+				  struct kobj_attribute *attr,
+				  const char *buf, size_t count)
+{
+	int err;
+	unsigned long flags;
+
+	err = kstrtoul(buf, 10, &flags);
+	if (err || flags > UINT_MAX || flags > ALLOW)
+		return -EINVAL;
+
+	/*
+	 * ALLOW = 1 - sets allow for mark new vma as
+	 * VM_MERGEABLE and adding it to ksm
+	 * DENY = 0 - disable it
+	 */
+	if (mark_new_vma != flags) {
+		mark_new_vma = flags;
+	}
+	return count;
+}
+KSM_ATTR(mark_new_vma);
+
 static ssize_t run_show(struct kobject *kobj, struct kobj_attribute *attr,
 			char *buf)
 {
@@ -2287,6 +2321,7 @@ static struct attribute *ksm_attrs[] = {
 	&pages_unshared_attr.attr,
 	&pages_volatile_attr.attr,
 	&full_scans_attr.attr,
+	&mark_new_vma_attr.attr,
 #ifdef CONFIG_NUMA
 	&merge_across_nodes_attr.attr,
 #endif
@@ -2323,7 +2358,9 @@ static int __init ksm_init(void)
 		goto out_free;
 	}
 #else
-	ksm_run = KSM_RUN_MERGE;	/* no way for user to start it */
+	/* no way for user to start it */
+	mark_new_vma = ALLOW;
+	ksm_run = KSM_RUN_MERGE;
 
 #endif /* CONFIG_SYSFS */
 
-- 
2.1.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [RESEND][PATCH V3 3/4] KSM: Add config to control mark_new_vma
  2014-11-13 21:01 [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication Timofey Titovets
  2014-11-13 21:01 ` [RESEND][PATCH V3 1/4] KSM: Add auto flag new VMA as VM_MERGEABLE Timofey Titovets
  2014-11-13 21:01 ` [RESEND][PATCH V3 2/4] KSM: Add to sysfs - mark_new_vma Timofey Titovets
@ 2014-11-13 21:01 ` Timofey Titovets
  2014-11-13 21:01 ` [RESEND][PATCH V3 4/4] KSM: mark_new_vma added to Documentation Timofey Titovets
  2014-12-01 10:35 ` [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication Hugh Dickins
  4 siblings, 0 replies; 7+ messages in thread
From: Timofey Titovets @ 2014-11-13 21:01 UTC (permalink / raw)
  To: linux-mm; +Cc: akpm, Timofey Titovets

Allowing to control mark_new_vma default value
Allowing work ksm on early allocated vmas

Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
---
 mm/Kconfig | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/mm/Kconfig b/mm/Kconfig
index 1d1ae6b..90f40a6 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -340,6 +340,13 @@ config KSM
 	  until a program has madvised that an area is MADV_MERGEABLE, and
 	  root has set /sys/kernel/mm/ksm/run to 1 (if CONFIG_SYSFS is set).
 
+config KSM_MARK_NEW_VMA
+	int "Marking new vma pages as VM_MERGEABLE"
+	depends on KSM
+	default 0
+	range 0 1
+	help
+
 config DEFAULT_MMAP_MIN_ADDR
         int "Low address space to protect from user allocation"
 	depends on MMU
-- 
2.1.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [RESEND][PATCH V3 4/4] KSM: mark_new_vma added to Documentation.
  2014-11-13 21:01 [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication Timofey Titovets
                   ` (2 preceding siblings ...)
  2014-11-13 21:01 ` [RESEND][PATCH V3 3/4] KSM: Add config to control mark_new_vma Timofey Titovets
@ 2014-11-13 21:01 ` Timofey Titovets
  2014-12-01 10:35 ` [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication Hugh Dickins
  4 siblings, 0 replies; 7+ messages in thread
From: Timofey Titovets @ 2014-11-13 21:01 UTC (permalink / raw)
  To: linux-mm; +Cc: akpm, Timofey Titovets

Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
---
 Documentation/vm/ksm.txt | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/Documentation/vm/ksm.txt b/Documentation/vm/ksm.txt
index f34a8ee..880fdbf 100644
--- a/Documentation/vm/ksm.txt
+++ b/Documentation/vm/ksm.txt
@@ -24,6 +24,8 @@ KSM only operates on those areas of address space which an application
 has advised to be likely candidates for merging, by using the madvise(2)
 system call: int madvise(addr, length, MADV_MERGEABLE).
 
+Also KSM can mark anonymous memory as mergeable, see below.
+
 The app may call int madvise(addr, length, MADV_UNMERGEABLE) to cancel
 that advice and restore unshared pages: whereupon KSM unmerges whatever
 it merged in that range.  Note: this unmerging call may suddenly require
@@ -73,6 +75,11 @@ merge_across_nodes - specifies if pages from different numa nodes can be merged.
                    merge_across_nodes, to remerge according to the new setting.
                    Default: 1 (merging across nodes as in earlier releases)
 
+mark_new_vma     - set 0 to disallow ksm marking every new allocated anonymous
+                   memory as mergeable.
+                   set 1 to allow ksm mark every new allocated anonymous memory
+                   as mergeable
+
 run              - set 0 to stop ksmd from running but keep merged pages,
                    set 1 to run ksmd e.g. "echo 1 > /sys/kernel/mm/ksm/run",
                    set 2 to stop ksmd and unmerge all pages currently merged,
-- 
2.1.3

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication
  2014-11-13 21:01 [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication Timofey Titovets
                   ` (3 preceding siblings ...)
  2014-11-13 21:01 ` [RESEND][PATCH V3 4/4] KSM: mark_new_vma added to Documentation Timofey Titovets
@ 2014-12-01 10:35 ` Hugh Dickins
  2014-12-07 15:20   ` Timofey Titovets
  4 siblings, 1 reply; 7+ messages in thread
From: Hugh Dickins @ 2014-12-01 10:35 UTC (permalink / raw)
  To: Timofey Titovets; +Cc: Andrew Morton, linux-mm

On Fri, 14 Nov 2014, Timofey Titovets wrote:

> Good time of day List,
> this tiny series of patches implement feature for auto deduping all anonymous memory.
> mark_new_vma - new ksm sysfs interface
> Every time then new vma created and mark_new_vma set to 1, then will be vma marked as VM_MERGEABLE and added to ksm queue.
> This can produce small overheads
> (I have not catch any problems or slowdown)
> 
> This is useful for:
> Android (CM) devs which implement ksm support with patching system.
> Users of tiny pc.
> Servers what not use KVM but use something very releated, like containers.
> 
> Can be pulled from:
> https://github.com/Nefelim4ag/linux.git ksm_improvements
> 
> For tests:
> I have tested it and it working very good. For testing apply it and enable ksm:
> echo 1 | sudo tee /sys/kernel/mm/ksm/run
> This show how much memory saved:
> echo $[$(cat /sys/kernel/mm/ksm/pages_shared)*$(getconf PAGE_SIZE)/1024 ]KB
> 
> On my system i save ~1% of memory 26 Mb/2100 Mb (deduped)/(used)
> 
> v2:
> 	Added Kconfig for control default value of mark_new_vma
> 	Added sysfs interface for control mark_new_vma
> 	Splitted in several patches
> 
> v3:
> 	Documentation for ksm changed for clarify new cha
> 
> Timofey Titovets (4):
>   KSM: Add auto flag new VMA as VM_MERGEABLE
>   KSM: Add to sysfs - mark_new_vma
>   KSM: Add config to control mark_new_vma
>   KSM: mark_new_vma added to Documentation.
> 
>  Documentation/vm/ksm.txt |  7 +++++++
>  include/linux/ksm.h      | 39 +++++++++++++++++++++++++++++++++++++++
>  mm/Kconfig               |  7 +++++++
>  mm/ksm.c                 | 39 ++++++++++++++++++++++++++++++++++++++-
>  mm/mmap.c                | 17 +++++++++++++++++
>  5 files changed, 108 insertions(+), 1 deletion(-)

I welcome what you're trying to achieve with this,
but have a lot of issues with how you have gone about it.

(I have my own patch to mm/mmap.c to achieve a similar effect, for
testing KSM: but looking through it again now, conclude that it's just
too hacky for others' eyes: I'd be glad to have something in the tree
that saves me from having to apply that hack in the future.)

Please reduce this to a single patch: I don't see anything gained,
but both our time wasted, from having it split into four.

Please run scripts/checkpatch.pl over the result: it has quite a
few complaints; or you might prefer to do that afterwards...

Please throw away all(?) of your changes to mm/mmap.c, and those
inlines in include/linux/ksm.h.  KSM has a tradition of invading
the rest of mm as little as possible, and I'd like to keep to that
so far as we can (and others will want us to keep to that even more
than I do).

   Instead of trying to apply VM_MERGEABLE in assorted places
   throughout mmap.c, you should instead be modifying ksm.c to
   ignore VM_MERGEABLE and respect only its conditioning flags
   when your option is set.  (Looking through those conditioning
   flags, I think the only one that actually matters internally
   is probably VM_HUGETLB - having an anon_vma is in practice the
   important criterion - but better that you continue to respect
   those flags for now, rather than experimenting without them.)

   Then I think all you need is one hook somewhere to ensure that
   every mm gets on KSM's mm_list (while your option is set),
   instead of just those which call madvise(,,MADV_MERGEABLE).
   Or does this approach not work for some reason?

Please find some better naming: I suggest "all_mergeable" for
the tunable, and KSM_ALL_MERGEABLE for the config option (my hack
has an "allksm" boot option, but I think we can do without that).
Poor naming ("ksm_vma_add_new" was an mm operation) misled you into
putting hooks into quite unnecessary places (how would splitting a
vma require a change to the mm's MMF_MERGEABLE?).

Then we can look at the result of that; but be warned, as you can
see from the "speed" of my reply, I have no time to spare at present,
and few others are interested in KSM (perhaps that's an unfortunate
side-effect of our success in keeping it isolated from the rest of mm).

Thanks,
Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication
  2014-12-01 10:35 ` [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication Hugh Dickins
@ 2014-12-07 15:20   ` Timofey Titovets
  0 siblings, 0 replies; 7+ messages in thread
From: Timofey Titovets @ 2014-12-07 15:20 UTC (permalink / raw)
  To: Hugh Dickins; +Cc: Andrew Morton, linux-mm

2014-12-01 13:35 GMT+03:00 Hugh Dickins <hughd@google.com>:
> On Fri, 14 Nov 2014, Timofey Titovets wrote:
>
>> Good time of day List,
>> this tiny series of patches implement feature for auto deduping all anonymous memory.
>> mark_new_vma - new ksm sysfs interface
>> Every time then new vma created and mark_new_vma set to 1, then will be vma marked as VM_MERGEABLE and added to ksm queue.
>> This can produce small overheads
>> (I have not catch any problems or slowdown)
>>
>> This is useful for:
>> Android (CM) devs which implement ksm support with patching system.
>> Users of tiny pc.
>> Servers what not use KVM but use something very releated, like containers.
>>
>> Can be pulled from:
>> https://github.com/Nefelim4ag/linux.git ksm_improvements
>>
>> For tests:
>> I have tested it and it working very good. For testing apply it and enable ksm:
>> echo 1 | sudo tee /sys/kernel/mm/ksm/run
>> This show how much memory saved:
>> echo $[$(cat /sys/kernel/mm/ksm/pages_shared)*$(getconf PAGE_SIZE)/1024 ]KB
>>
>> On my system i save ~1% of memory 26 Mb/2100 Mb (deduped)/(used)
>>
>> v2:
>>       Added Kconfig for control default value of mark_new_vma
>>       Added sysfs interface for control mark_new_vma
>>       Splitted in several patches
>>
>> v3:
>>       Documentation for ksm changed for clarify new cha
>>
>> Timofey Titovets (4):
>>   KSM: Add auto flag new VMA as VM_MERGEABLE
>>   KSM: Add to sysfs - mark_new_vma
>>   KSM: Add config to control mark_new_vma
>>   KSM: mark_new_vma added to Documentation.
>>
>>  Documentation/vm/ksm.txt |  7 +++++++
>>  include/linux/ksm.h      | 39 +++++++++++++++++++++++++++++++++++++++
>>  mm/Kconfig               |  7 +++++++
>>  mm/ksm.c                 | 39 ++++++++++++++++++++++++++++++++++++++-
>>  mm/mmap.c                | 17 +++++++++++++++++
>>  5 files changed, 108 insertions(+), 1 deletion(-)
>
> I welcome what you're trying to achieve with this,
> but have a lot of issues with how you have gone about it.
>
> (I have my own patch to mm/mmap.c to achieve a similar effect, for
> testing KSM: but looking through it again now, conclude that it's just
> too hacky for others' eyes: I'd be glad to have something in the tree
> that saves me from having to apply that hack in the future.)

Where i can see your code? =)

> Please reduce this to a single patch: I don't see anything gained,
> but both our time wasted, from having it split into four.

Sorry, fixed =-=

> Please run scripts/checkpatch.pl over the result: it has quite a
> few complaints; or you might prefer to do that afterwards...
>
> Please throw away all(?) of your changes to mm/mmap.c, and those
> inlines in include/linux/ksm.h.  KSM has a tradition of invading
> the rest of mm as little as possible, and I'd like to keep to that
> so far as we can (and others will want us to keep to that even more
> than I do).
>
>    Instead of trying to apply VM_MERGEABLE in assorted places
>    throughout mmap.c, you should instead be modifying ksm.c to
>    ignore VM_MERGEABLE and respect only its conditioning flags
>    when your option is set.  (Looking through those conditioning
>    flags, I think the only one that actually matters internally
>    is probably VM_HUGETLB - having an anon_vma is in practice the
>    important criterion - but better that you continue to respect
>    those flags for now, rather than experimenting without them.)
>
>    Then I think all you need is one hook somewhere to ensure that
>    every mm gets on KSM's mm_list (while your option is set),
>    instead of just those which call madvise(,,MADV_MERGEABLE).
>    Or does this approach not work for some reason?

First of all, I am a kernel newbie and i can miss something, sorry if
it happens.

Ksm can't scan memory by itself. I'm not only mark page as
VM_MERGEABLE, i call __ksm_enter for adding it to ksm mm_list.
I already think about case where ksm scan all mm find marked with
VM_MERGEABLE and add it to own mm_list:
First its can slow down entire system as i think. But at now it does not matter.
Second, really, i can't find global mm tree =_='' or something like it
for add one to ksm.
Can you specify where i can found it or may be i mistaking in somewhere?

i must fix issue above before try modify ksm for ignore VM_MERGEABLE =\

> Please find some better naming: I suggest "all_mergeable" for
> the tunable, and KSM_ALL_MERGEABLE for the config option (my hack
> has an "allksm" boot option, but I think we can do without that).
> Poor naming ("ksm_vma_add_new" was an mm operation) misled you into
> putting hooks into quite unnecessary places (how would splitting a
> vma require a change to the mm's MMF_MERGEABLE?).

Thanks for idea, i renamed it.

> Then we can look at the result of that; but be warned, as you can
> see from the "speed" of my reply, I have no time to spare at present,
> and few others are interested in KSM (perhaps that's an unfortunate
> side-effect of our success in keeping it isolated from the rest of mm).
>
> Thanks,
> Hugh

Not a problem Hugh, thanks for reply.

-- 
Have a nice day,
Timofey.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2014-12-07 15:21 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-11-13 21:01 [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication Timofey Titovets
2014-11-13 21:01 ` [RESEND][PATCH V3 1/4] KSM: Add auto flag new VMA as VM_MERGEABLE Timofey Titovets
2014-11-13 21:01 ` [RESEND][PATCH V3 2/4] KSM: Add to sysfs - mark_new_vma Timofey Titovets
2014-11-13 21:01 ` [RESEND][PATCH V3 3/4] KSM: Add config to control mark_new_vma Timofey Titovets
2014-11-13 21:01 ` [RESEND][PATCH V3 4/4] KSM: mark_new_vma added to Documentation Timofey Titovets
2014-12-01 10:35 ` [RESEND][PATCH V3 0/4] KSM: Mark new vma for deduplication Hugh Dickins
2014-12-07 15:20   ` Timofey Titovets

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).