[BUG] gcov causes vread_tsc to increment kernel memory

[BUG] gcov causes vread_tsc to increment kernel memory

[Steven Rostedt]

On bootup of the latest kernel my init segfaults. Debugging it, I found 
that vread_tsc (a vsyscall) increments some strange kernel memory:

0000000000000000 <vread_tsc>:
   0:   55                      push   %rbp
   1:   48 ff 05 00 00 00 00    incq   0(%rip)        # 8 <vread_tsc+0x8>
                        4: R_X86_64_PC32        .bss+0x3c
   8:   48 89 e5                mov    %rsp,%rbp
   b:   66 66 90                xchg   %ax,%ax
   e:   48 ff 05 00 00 00 00    incq   0(%rip)        # 15 <vread_tsc+0x15>
                        11: R_X86_64_PC32       .bss+0x44
  15:   66 66 90                xchg   %ax,%ax
  18:   48 ff 05 00 00 00 00    incq   0(%rip)        # 1f <vread_tsc+0x1f>
                        1b: R_X86_64_PC32       .bss+0x4c
  1f:   0f 31                   rdtsc  
	

Those "incq" is very bad to happen in vsyscall memory, since userspace can 
not modify it. You need to make something prevent profiling of vsyscall 
memory (like I do with ftrace).

-- Steve

Re: [BUG] gcov causes vread_tsc to increment kernel memory

[Peter Oberparleiter]

Steven Rostedt wrote:
> On bootup of the latest kernel my init segfaults. Debugging it, I found 
> that vread_tsc (a vsyscall) increments some strange kernel memory:
> 
> 0000000000000000 <vread_tsc>:
>    0:   55                      push   %rbp
>    1:   48 ff 05 00 00 00 00    incq   0(%rip)        # 8 <vread_tsc+0x8>
>                         4: R_X86_64_PC32        .bss+0x3c
>    8:   48 89 e5                mov    %rsp,%rbp
>    b:   66 66 90                xchg   %ax,%ax
>    e:   48 ff 05 00 00 00 00    incq   0(%rip)        # 15 <vread_tsc+0x15>
>                         11: R_X86_64_PC32       .bss+0x44
>   15:   66 66 90                xchg   %ax,%ax
>   18:   48 ff 05 00 00 00 00    incq   0(%rip)        # 1f <vread_tsc+0x1f>
>                         1b: R_X86_64_PC32       .bss+0x4c
>   1f:   0f 31                   rdtsc  
> 	
> 
> Those "incq" is very bad to happen in vsyscall memory, since userspace can 
> not modify it. You need to make something prevent profiling of vsyscall 
> memory (like I do with ftrace).
> 
> -- Steve

You're right, I missed that file. This should be fixed with the patch
below. As the problem didn't occur on my test machine, please retest
with the patch applied. Thanks!

Also seeing as function tracer and gcov work on a similar basis and
require similar files to be excluded from profiling, it would be nice
if we wouldn't need to mark those files separately. Instead it would
be great if the Makefile could be used to specify that a certain
object file has a certain property (e.g. PROPERTY_USERPACE_file.o := y)
and the mechanism (e.g. function tracer) would only need to specify
that the extra gcc options should not be applied when that property is
set. What do you think?

=================
Subject: [PATCH] gcov: exclude code operating in userspace from profiling

From: Peter Oberparleiter <oberpar@linux.vnet.ibm.com>

Fix for this issue on x86_64:

rostedt@goodmis.org wrote:
> On bootup of the latest kernel my init segfaults. Debugging it,
> I found  that vread_tsc (a vsyscall) increments some strange
> kernel memory:
> 
> 0000000000000000 <vread_tsc>:
>    0:   55                      push   %rbp
>    1:   48 ff 05 00 00 00 00    incq   0(%rip)
>                         # 8 <vread_tsc+0x8>
>                         4: R_X86_64_PC32        .bss+0x3c
>    8:   48 89 e5                mov    %rsp,%rbp
>    b:   66 66 90                xchg   %ax,%ax
>    e:   48 ff 05 00 00 00 00    incq   0(%rip)
>                         # 15 <vread_tsc+0x15>
>                         11: R_X86_64_PC32       .bss+0x44
>   15:   66 66 90                xchg   %ax,%ax
>   18:   48 ff 05 00 00 00 00    incq   0(%rip)
>                         # 1f <vread_tsc+0x1f>
>                         1b: R_X86_64_PC32       .bss+0x4c
>   1f:   0f 31                   rdtsc  
>
>
> Those "incq" is very bad to happen in vsyscall memory, since
> userspace can not modify it. You need to make something prevent
> profiling of vsyscall  memory (like I do with ftrace).

Signed-off-by: Peter Oberparleiter <oberpar@linux.vnet.ibm.com>
---
 arch/x86/kernel/Makefile |    2 ++
 1 file changed, 2 insertions(+)

Index: linux-2.6.31-rc1/arch/x86/kernel/Makefile
===================================================================
--- linux-2.6.31-rc1.orig/arch/x86/kernel/Makefile
+++ linux-2.6.31-rc1/arch/x86/kernel/Makefile
@@ -26,6 +26,8 @@ CFLAGS_tsc.o		:= $(nostackp)
 CFLAGS_paravirt.o	:= $(nostackp)
 GCOV_PROFILE_vsyscall_64.o	:= n
 GCOV_PROFILE_hpet.o		:= n
+GCOV_PROFILE_tsc.o		:= n
+GCOV_PROFILE_paravirt.o		:= n
 
 obj-y			:= process_$(BITS).o signal.o entry_$(BITS).o
 obj-y			+= traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o

Re: [BUG] gcov causes vread_tsc to increment kernel memory

[Steven Rostedt]

On Thu, 2 Jul 2009, Peter Oberparleiter wrote:
> You're right, I missed that file. This should be fixed with the patch
> below. As the problem didn't occur on my test machine, please retest
> with the patch applied. Thanks!
> 
> Also seeing as function tracer and gcov work on a similar basis and
> require similar files to be excluded from profiling, it would be nice
> if we wouldn't need to mark those files separately. Instead it would
> be great if the Makefile could be used to specify that a certain
> object file has a certain property (e.g. PROPERTY_USERPACE_file.o := y)
> and the mechanism (e.g. function tracer) would only need to specify
> that the extra gcc options should not be applied when that property is
> set. What do you think?

I think a property for vsyscall would be nice. Then you can add your 
restrictions there. But ftrace also has to deal with other issues as well, 
not just vsyscall.

> 
> =================
> Subject: [PATCH] gcov: exclude code operating in userspace from profiling
> 
> From: Peter Oberparleiter <oberpar@linux.vnet.ibm.com>
> 
> Fix for this issue on x86_64:
> 
> rostedt@goodmis.org wrote:
> > On bootup of the latest kernel my init segfaults. Debugging it,
> > I found  that vread_tsc (a vsyscall) increments some strange
> > kernel memory:
> > 
> > 0000000000000000 <vread_tsc>:
> >    0:   55                      push   %rbp
> >    1:   48 ff 05 00 00 00 00    incq   0(%rip)
> >                         # 8 <vread_tsc+0x8>
> >                         4: R_X86_64_PC32        .bss+0x3c
> >    8:   48 89 e5                mov    %rsp,%rbp
> >    b:   66 66 90                xchg   %ax,%ax
> >    e:   48 ff 05 00 00 00 00    incq   0(%rip)
> >                         # 15 <vread_tsc+0x15>
> >                         11: R_X86_64_PC32       .bss+0x44
> >   15:   66 66 90                xchg   %ax,%ax
> >   18:   48 ff 05 00 00 00 00    incq   0(%rip)
> >                         # 1f <vread_tsc+0x1f>
> >                         1b: R_X86_64_PC32       .bss+0x4c
> >   1f:   0f 31                   rdtsc  
> >
> >
> > Those "incq" is very bad to happen in vsyscall memory, since
> > userspace can not modify it. You need to make something prevent
> > profiling of vsyscall  memory (like I do with ftrace).
> 

Reported-and-tested-by: Steven Rostedt <rostedt@goodmis.org>

Thanks!

-- Steve

> Signed-off-by: Peter Oberparleiter <oberpar@linux.vnet.ibm.com>
> ---
>  arch/x86/kernel/Makefile |    2 ++
>  1 file changed, 2 insertions(+)
> 
> Index: linux-2.6.31-rc1/arch/x86/kernel/Makefile
> ===================================================================
> --- linux-2.6.31-rc1.orig/arch/x86/kernel/Makefile
> +++ linux-2.6.31-rc1/arch/x86/kernel/Makefile
> @@ -26,6 +26,8 @@ CFLAGS_tsc.o		:= $(nostackp)
>  CFLAGS_paravirt.o	:= $(nostackp)
>  GCOV_PROFILE_vsyscall_64.o	:= n
>  GCOV_PROFILE_hpet.o		:= n
> +GCOV_PROFILE_tsc.o		:= n
> +GCOV_PROFILE_paravirt.o		:= n
>  
>  obj-y			:= process_$(BITS).o signal.o entry_$(BITS).o
>  obj-y			+= traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o
> 
> 

Re: [BUG] gcov causes vread_tsc to increment kernel memory

[Ingo Molnar]

* Peter Oberparleiter <oberpar@linux.vnet.ibm.com> wrote:

> Also seeing as function tracer and gcov work on a similar basis 
> and require similar files to be excluded from profiling, it would 
> be nice if we wouldn't need to mark those files separately. [...]

One of the general things i pointed out during review is that 
there's a fair amount of overlap between tracing/instrumentation and 
the gcov support patches and now for a change Andrew has merged 
incomplete stuff upstream prematurely ;-) Andrew, could you please 
make sure this all gets fixed and the review suggestions get 
implemented?

	Ingo

Re: [BUG] gcov causes vread_tsc to increment kernel memory

[Peter Oberparleiter]

Ingo Molnar wrote:
> * Peter Oberparleiter <oberpar@linux.vnet.ibm.com> wrote:
> 
>> Also seeing as function tracer and gcov work on a similar basis 
>> and require similar files to be excluded from profiling, it would 
>> be nice if we wouldn't need to mark those files separately. [...]
> 
> One of the general things i pointed out during review is that 
> there's a fair amount of overlap between tracing/instrumentation and 
> the gcov support patches and now for a change Andrew has merged 
> incomplete stuff upstream prematurely ;-)

Yes there is overlap between ftrace and gcov and I will help in finding
and merging those bits, but as I also pointed out during review, there
are also fundamental differences.

By including the gcov infrastructure upstream now, we can incrementally
improve its integration with any similar mechanism _while_ people are
improving their test cases and identifying more bugs based on code
coverage measurements.