[PATCH 0/2 V2] Add trace points to [m|mun|mre]map and brk

[PATCH 0/2 V2] Add trace points to [m|mun|mre]map and brk

[Eric B Munson]

This patch set is a resubmit of several patches I sent out earlier that adds
trace points to the mmap family.  These events report addresses and sizes when
each function returns success.  They will be used by a userspace tool that will model memory usage.

Changes from V1:
- Group mmap, munmap, and brk into first patch (all in mmap.c) and mremap into
  second (in mremap.c)
- Use DEFINE_EVENT_CLASS and DEFINE_EVENT for mmap and brk events

Eric B Munson (2):
  Add trace points to mmap, munmap, and brk
  Add mremap trace point

 include/trace/events/mm.h |   97 +++++++++++++++++++++++++++++++++++++++++++++
 mm/mmap.c                 |   15 ++++++-
 mm/mremap.c               |    4 ++
 3 files changed, 115 insertions(+), 1 deletions(-)
 create mode 100644 include/trace/events/mm.h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 1/2] Add trace points to mmap, munmap, and brk

[Eric B Munson]

This patch adds trace points to mmap, munmap, and brk that will report
relevant addresses and sizes before each function exits successfully.

Signed-off-by: Eric B Munson <emunson@mgebm.net>
---
 include/trace/events/mm.h |   75 +++++++++++++++++++++++++++++++++++++++++++++
 mm/mmap.c                 |   15 ++++++++-
 2 files changed, 89 insertions(+), 1 deletions(-)
 create mode 100644 include/trace/events/mm.h

diff --git a/include/trace/events/mm.h b/include/trace/events/mm.h
new file mode 100644
index 0000000..892bbe3
--- /dev/null
+++ b/include/trace/events/mm.h
@@ -0,0 +1,75 @@
+/*
+ * Copyright (c) 2010, Eric Munson
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+ */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM mm
+
+#if !defined(_TRACE_MM_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_MM_H
+
+#include <linux/tracepoint.h>
+
+DECLARE_EVENT_CLASS(
+		mm_mmap_class,
+		TP_PROTO(unsigned long addr, unsigned long len),
+		TP_ARGS(addr, len),
+		TP_STRUCT__entry(
+			__field(unsigned long, addr)
+			__field(unsigned long, len)
+		),
+		TP_fast_assign(
+			__entry->addr = addr;
+			__entry->len = len;
+		),
+		TP_printk("%lu bytes at 0x%lx\n", __entry->len, __entry->addr)
+);
+
+DEFINE_EVENT(
+		mm_mmap_class,
+		mmap,
+		TP_PROTO(unsigned long addr, unsigned long len),
+		TP_ARGS(addr, len)
+);
+
+
+DEFINE_EVENT(
+		mm_mmap_class,
+		brk,
+		TP_PROTO(unsigned long addr, unsigned long len),
+		TP_ARGS(addr, len)
+);
+
+TRACE_EVENT(
+		munmap,
+		TP_PROTO(unsigned long start, size_t len),
+		TP_ARGS(start, len),
+		TP_STRUCT__entry(
+			__field(unsigned long, start)
+			__field(size_t, len)
+		),
+		TP_fast_assign(
+			__entry->start = start;
+			__entry->len = len;
+		),
+
+		TP_printk("%u bytes at 0x%lx\n", __entry->len, __entry->start)
+);
+
+#endif /* _TRACE_MM_H */
+
+#include <trace/define_trace.h>
+
diff --git a/mm/mmap.c b/mm/mmap.c
index 456ec6f..430ce46 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -29,6 +29,9 @@
 #include <linux/mmu_notifier.h>
 #include <linux/perf_event.h>
 
+#define CREATE_TRACE_POINTS
+#include <trace/events/mm.h>
+
 #include <asm/uaccess.h>
 #include <asm/cacheflush.h>
 #include <asm/tlb.h>
@@ -949,6 +952,7 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 	unsigned int vm_flags;
 	int error;
 	unsigned long reqprot = prot;
+	unsigned long ret;
 
 	/*
 	 * Does the application expect PROT_READ to imply PROT_EXEC?
@@ -1074,7 +1078,12 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 	if (error)
 		return error;
 
-	return mmap_region(file, addr, len, flags, vm_flags, pgoff);
+	ret = mmap_region(file, addr, len, flags, vm_flags, pgoff);
+
+	if(!(ret & ~PAGE_MASK))
+		trace_mmap(addr,len);
+
+	return ret;
 }
 EXPORT_SYMBOL(do_mmap_pgoff);
 
@@ -2079,6 +2088,8 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len)
 		}
 	}
 
+	trace_munmap(start, len);
+
 	/*
 	 * Remove the vma's, and unmap the actual pages
 	 */
@@ -2213,6 +2224,8 @@ out:
 		if (!mlock_vma_pages_range(vma, addr, addr + len))
 			mm->locked_vm += (len >> PAGE_SHIFT);
 	}
+
+	trace_brk(addr, len);
 	return addr;
 }
 
-- 
1.7.0.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 2/2] Add mremap trace point

[Eric B Munson]

This patch adds the trace point for mremap which reports relevant addresses
and sizes when mremap exits successfully.

Signed-off-by: Eric B Munson <emunson@mgebm.net>
---
 include/trace/events/mm.h |   22 ++++++++++++++++++++++
 mm/mremap.c               |    4 ++++
 2 files changed, 26 insertions(+), 0 deletions(-)

diff --git a/include/trace/events/mm.h b/include/trace/events/mm.h
index 892bbe3..16f8c36 100644
--- a/include/trace/events/mm.h
+++ b/include/trace/events/mm.h
@@ -69,6 +69,28 @@ TRACE_EVENT(
 		TP_printk("%u bytes at 0x%lx\n", __entry->len, __entry->start)
 );
 
+TRACE_EVENT(
+		mremap,
+		TP_PROTO(unsigned long addr, unsigned long old_len,
+			 unsigned long new_addr, unsigned long new_len),
+		TP_ARGS(addr, old_len, new_addr, new_len),
+		TP_STRUCT__entry(
+			__field(unsigned long, addr)
+			__field(unsigned long, old_len)
+			__field(unsigned long, new_addr)
+			__field(unsigned long, new_len)
+		),
+		TP_fast_assign(
+			__entry->addr = addr;
+			__entry->old_len = old_len;
+			__entry->new_addr = new_addr;
+			__entry->new_len = new_len;
+		),
+		TP_printk("%lu bytes from 0x%lx to %lu bytes at 0x%lx\n",
+			__entry->old_len, __entry->addr, __entry->new_len,
+			__entry->new_addr)
+);
+
 #endif /* _TRACE_MM_H */
 
 #include <trace/define_trace.h>
diff --git a/mm/mremap.c b/mm/mremap.c
index cde56ee..4ef1dd3 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -20,6 +20,8 @@
 #include <linux/syscalls.h>
 #include <linux/mmu_notifier.h>
 
+#include <trace/events/mm.h>
+
 #include <asm/uaccess.h>
 #include <asm/cacheflush.h>
 #include <asm/tlbflush.h>
@@ -504,6 +506,8 @@ unsigned long do_mremap(unsigned long addr,
 out:
 	if (ret & ~PAGE_MASK)
 		vm_unacct_memory(charged);
+	else
+		trace_mremap(addr, old_len, new_addr, new_len);
 	return ret;
 }
 
-- 
1.7.0.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/2] Add trace points to mmap, munmap, and brk

[KOSAKI Motohiro]

> This patch adds trace points to mmap, munmap, and brk that will report
> relevant addresses and sizes before each function exits successfully.
> 
> Signed-off-by: Eric B Munson <emunson@mgebm.net>

I don't think this is good idea. if you need syscall result, you should 
use syscall tracer. IOW, This tracepoint bring zero information.

Please see perf_event_mmap() usage. Our kernel manage adress space by
vm_area_struct. we need to trace it if we need to know what kernel does.

Thanks.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/2] Add trace points to mmap, munmap, and brk

[Eric B Munson]

[-- Attachment #1: Type: text/plain, Size: 1124 bytes --]

On Wed, 21 Jul 2010, KOSAKI Motohiro wrote:

> > This patch adds trace points to mmap, munmap, and brk that will report
> > relevant addresses and sizes before each function exits successfully.
> > 
> > Signed-off-by: Eric B Munson <emunson@mgebm.net>
> 
> I don't think this is good idea. if you need syscall result, you should 
> use syscall tracer. IOW, This tracepoint bring zero information.
> 
> Please see perf_event_mmap() usage. Our kernel manage adress space by
> vm_area_struct. we need to trace it if we need to know what kernel does.
> 
> Thanks.

The syscall tracer does not give you the address and size of the mmaped areas
so this does provide information above simply tracing the enter/exit points
for each call.

perf_event_mmap does provide the information for mmap calls.  Originally I sent
a patch to add a trace point to munmap and Peter Z asked for corresponding points
in the mmap family.  If the consensus is that the trace point in munmap is the
only one that should be added I can resend that patch.

-- 
Eric B Munson
IBM Linux Technology Center
ebmunson@us.ibm.com


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [PATCH 1/2] Add trace points to mmap, munmap, and brk

[KOSAKI Motohiro]

> On Wed, 21 Jul 2010, KOSAKI Motohiro wrote:
> 
> > > This patch adds trace points to mmap, munmap, and brk that will report
> > > relevant addresses and sizes before each function exits successfully.
> > > 
> > > Signed-off-by: Eric B Munson <emunson@mgebm.net>
> > 
> > I don't think this is good idea. if you need syscall result, you should 
> > use syscall tracer. IOW, This tracepoint bring zero information.
> > 
> > Please see perf_event_mmap() usage. Our kernel manage adress space by
> > vm_area_struct. we need to trace it if we need to know what kernel does.
> > 
> > Thanks.
> 
> The syscall tracer does not give you the address and size of the mmaped areas
> so this does provide information above simply tracing the enter/exit points
> for each call.

Why don't you fix this?



> perf_event_mmap does provide the information for mmap calls.  Originally I sent
> a patch to add a trace point to munmap and Peter Z asked for corresponding points
> in the mmap family.  If the consensus is that the trace point in munmap is the
> only one that should be added I can resend that patch.
> 
> -- 
> Eric B Munson
> IBM Linux Technology Center
> ebmunson@us.ibm.com
> 



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/2] Add trace points to mmap, munmap, and brk

[Eric B Munson]

[-- Attachment #1: Type: text/plain, Size: 1472 bytes --]

On Tue, 27 Jul 2010, KOSAKI Motohiro wrote:

> > On Wed, 21 Jul 2010, KOSAKI Motohiro wrote:
> > 
> > > > This patch adds trace points to mmap, munmap, and brk that will report
> > > > relevant addresses and sizes before each function exits successfully.
> > > > 
> > > > Signed-off-by: Eric B Munson <emunson@mgebm.net>
> > > 
> > > I don't think this is good idea. if you need syscall result, you should 
> > > use syscall tracer. IOW, This tracepoint bring zero information.
> > > 
> > > Please see perf_event_mmap() usage. Our kernel manage adress space by
> > > vm_area_struct. we need to trace it if we need to know what kernel does.
> > > 
> > > Thanks.
> > 
> > The syscall tracer does not give you the address and size of the mmaped areas
> > so this does provide information above simply tracing the enter/exit points
> > for each call.
> 
> Why don't you fix this?
> 
> 

Sorry for the long delay, the enter/exit routines are not compatible with the
information that these new trace points provides.  When tracing mmap, for
instance, the addr and len arguments can be altered by the function.  If you
use the enter/exit trace points you would not see this as the arguments are
sampled at function entrance and not given again on exit.  Also, the new
trace points are only hit on function success, the exit trace point happens
any time you leave the system call.

I will send out a new series after a rebase.

Thanks,
Eric


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

[PATCH 1/2] Add trace points to mmap, munmap, and brk

[Eric B Munson]

This patch adds trace points to mmap, munmap, and brk that will report
relevant addresses and sizes before each function exits successfully.

Signed-off-by: Eric B Munson <emunson@mgebm.net>
---
 include/trace/events/mm.h |   75 +++++++++++++++++++++++++++++++++++++++++++++
 mm/mmap.c                 |   15 ++++++++-
 2 files changed, 89 insertions(+), 1 deletions(-)
 create mode 100644 include/trace/events/mm.h

diff --git a/include/trace/events/mm.h b/include/trace/events/mm.h
new file mode 100644
index 0000000..892bbe3
--- /dev/null
+++ b/include/trace/events/mm.h
@@ -0,0 +1,75 @@
+/*
+ * Copyright (c) 2010, Eric Munson
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+ */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM mm
+
+#if !defined(_TRACE_MM_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_MM_H
+
+#include <linux/tracepoint.h>
+
+DECLARE_EVENT_CLASS(
+		mm_mmap_class,
+		TP_PROTO(unsigned long addr, unsigned long len),
+		TP_ARGS(addr, len),
+		TP_STRUCT__entry(
+			__field(unsigned long, addr)
+			__field(unsigned long, len)
+		),
+		TP_fast_assign(
+			__entry->addr = addr;
+			__entry->len = len;
+		),
+		TP_printk("%lu bytes at 0x%lx\n", __entry->len, __entry->addr)
+);
+
+DEFINE_EVENT(
+		mm_mmap_class,
+		mmap,
+		TP_PROTO(unsigned long addr, unsigned long len),
+		TP_ARGS(addr, len)
+);
+
+
+DEFINE_EVENT(
+		mm_mmap_class,
+		brk,
+		TP_PROTO(unsigned long addr, unsigned long len),
+		TP_ARGS(addr, len)
+);
+
+TRACE_EVENT(
+		munmap,
+		TP_PROTO(unsigned long start, size_t len),
+		TP_ARGS(start, len),
+		TP_STRUCT__entry(
+			__field(unsigned long, start)
+			__field(size_t, len)
+		),
+		TP_fast_assign(
+			__entry->start = start;
+			__entry->len = len;
+		),
+
+		TP_printk("%u bytes at 0x%lx\n", __entry->len, __entry->start)
+);
+
+#endif /* _TRACE_MM_H */
+
+#include <trace/define_trace.h>
+
diff --git a/mm/mmap.c b/mm/mmap.c
index 6128dc8..03f857b 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -29,6 +29,9 @@
 #include <linux/mmu_notifier.h>
 #include <linux/perf_event.h>
 
+#define CREATE_TRACE_POINTS
+#include <trace/events/mm.h>
+
 #include <asm/uaccess.h>
 #include <asm/cacheflush.h>
 #include <asm/tlb.h>
@@ -971,6 +974,7 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 	unsigned int vm_flags;
 	int error;
 	unsigned long reqprot = prot;
+	unsigned long ret;
 
 	/*
 	 * Does the application expect PROT_READ to imply PROT_EXEC?
@@ -1096,7 +1100,12 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 	if (error)
 		return error;
 
-	return mmap_region(file, addr, len, flags, vm_flags, pgoff);
+	ret = mmap_region(file, addr, len, flags, vm_flags, pgoff);
+
+	if(!(ret & ~PAGE_MASK))
+		trace_mmap(addr,len);
+
+	return ret;
 }
 EXPORT_SYMBOL(do_mmap_pgoff);
 
@@ -2104,6 +2113,8 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len)
 		}
 	}
 
+	trace_munmap(start, len);
+
 	/*
 	 * Remove the vma's, and unmap the actual pages
 	 */
@@ -2239,6 +2250,8 @@ out:
 		if (!mlock_vma_pages_range(vma, addr, addr + len))
 			mm->locked_vm += (len >> PAGE_SHIFT);
 	}
+
+	trace_brk(addr, len);
 	return addr;
 }
 
-- 
1.7.0.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/2] Add trace points to mmap, munmap, and brk

[KOSAKI Motohiro]

> On Tue, 27 Jul 2010, KOSAKI Motohiro wrote:
> 
> > > On Wed, 21 Jul 2010, KOSAKI Motohiro wrote:
> > > 
> > > > > This patch adds trace points to mmap, munmap, and brk that will report
> > > > > relevant addresses and sizes before each function exits successfully.
> > > > > 
> > > > > Signed-off-by: Eric B Munson <emunson@mgebm.net>
> > > > 
> > > > I don't think this is good idea. if you need syscall result, you should 
> > > > use syscall tracer. IOW, This tracepoint bring zero information.
> > > > 
> > > > Please see perf_event_mmap() usage. Our kernel manage adress space by
> > > > vm_area_struct. we need to trace it if we need to know what kernel does.
> > > > 
> > > > Thanks.
> > > 
> > > The syscall tracer does not give you the address and size of the mmaped areas
> > > so this does provide information above simply tracing the enter/exit points
> > > for each call.
> > 
> > Why don't you fix this?
> > 
> > 
> 
> Sorry for the long delay, 

no problem.

> the enter/exit routines are not compatible with the
> information that these new trace points provides.  When tracing mmap, for
> instance, the addr and len arguments can be altered by the function.  If you
> use the enter/exit trace points you would not see this as the arguments are
> sampled at function entrance and not given again on exit.  

Current output is here. It has rich output than yours. Also you can bind enter and exit output by pid.


            less-2130  [001]  3779.915324: sys_mmap(addr: 0, len: 1000, prot: 3, flags: 22, fd: ffffffff, off: 0)
            less-2130  [001]  3779.915331: sys_mmap -> 0x7fee22b17000
            less-2130  [001]  3779.915350: sys_mmap(addr: 38e8c00000, len: 3788a8, prot: 5, flags: 802, fd: 3, off: 0)
            less-2130  [001]  3779.915357: sys_mmap -> 0x38e8c00000
            less-2130  [001]  3779.915368: sys_mmap(addr: 38e8f6f000, len: 5000, prot: 3, flags: 812, fd: 3, off: 16f000)
            less-2130  [001]  3779.915380: sys_mmap -> 0x38e8f6f000
            less-2130  [001]  3779.915411: sys_mmap(addr: 38e8f74000, len: 48a8, prot: 3, flags: 32, fd: ffffffff, off: 0)
            less-2130  [001]  3779.915421: sys_mmap -> 0x38e8f74000
            less-2130  [001]  3779.915464: sys_mmap(addr: 0, len: 1000, prot: 3, flags: 22, fd: ffffffff, off: 0)
            less-2130  [001]  3779.915468: sys_mmap -> 0x7fee22b16000


> Also, the new
> trace points are only hit on function success, the exit trace point happens
> any time you leave the system call.

Special purpose filtering is no good design. That makes narrowing the feature usefulness.
It should be done on userland.



> I will send out a new series after a rebase.



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/2] Add trace points to mmap, munmap, and brk

[KOSAKI Motohiro]

> This patch adds trace points to mmap, munmap, and brk that will report
> relevant addresses and sizes before each function exits successfully.
> 
> Signed-off-by: Eric B Munson <emunson@mgebm.net>

These tracepoint are still poor than syscall trace. I don't think this is
good idea. Please avoid fixed specific tracepoint. Please consider make generic.




--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/2] Add trace points to mmap, munmap, and brk

[Ingo Molnar]

* Eric B Munson <emunson@mgebm.net> wrote:

> Sorry for the long delay, the enter/exit routines are not compatible 
> with the information that these new trace points provides.  When 
> tracing mmap, for instance, the addr and len arguments can be altered 
> by the function.  If you use the enter/exit trace points you would not 
> see this as the arguments are sampled at function entrance and not 
> given again on exit.  Also, the new trace points are only hit on 
> function success, the exit trace point happens any time you leave the 
> system call.

Would it be feasible to use enter/exit information as the main source of 
events - and only add new tracepoints for the _missing_ information? 
(such as when mmap arguments change)

Then user-space can combine the two. The new tracepoints would also 
carry useful information in themselves: they would show the cases where 
user-space did not get what it wished. (or so)

Thanks,

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>