Re: perf TUI fails with "failed to process type: 64"

Re: perf TUI fails with "failed to process type: 64"

[Michael Ellerman]

Anton Blanchard <anton@samba.org> writes:

> Hi,
>
> Updating to mainline as of last night, I started seeing the following
> error when running the perf report TUI:
>
> 0x46068 [0x8]: failed to process type: 68
>
> This event is just PERF_RECORD_FINISHED_ROUND:
>
> 0x46068 [0x8]: event: 68
> .
> . ... raw event: size 8 bytes
> .  0000:  44 00 00 00 00 00 08 00                          D.......        
>
> 0x46068 [0x8]: PERF_RECORD_FINISHED_ROUND
>
> Which of course is not our error. It took me a while to find the real
> culprit:
>
>  14c00-14c00 g exc_virt_0x4c00_system_call
   ^
   What's this? The address? If so it's wrong?
   

> A zero length symbol, which __symbol__inc_addr_samples() barfs on:
>
>         if (addr < sym->start || addr >= sym->end) {
> ...
> 		return -ERANGE;
>
> Seems like we have 3 bugs here:
...
>
> 3. Why do we have zero length symbols in the first place? Does the recent
>    ppc64 exception clean up have something to do with it?

Seems likely. But I can't see why.

AFAICS we have never emitted a size for those symbols:

Old:
$ nm --print-size build/vmlinux | grep -w system_call_relon_pSeries
c000000000004c00 T system_call_relon_pSeries

New:
$ nm --print-size build/vmlinux | grep -w exc_virt_0x4c00_system_call
c000000000004c00 T exc_virt_0x4c00_system_call


It also doesn't look like we're emitting another symbol with the same
address, which has caused confusion in the past:

Old:
c000000000004c00 T exc_virt_0x4c00_system_call
c000000000004d00 T exc_virt_0x4d00_single_step

New:
c000000000004c00 T system_call_relon_pSeries
c000000000004d00 T single_step_relon_pSeries


So more digging required.

cheers

Re: perf TUI fails with "failed to process type: 64"

[Nicholas Piggin]

On Mon, 10 Oct 2016 16:02:07 +1100
Michael Ellerman <mpe@ellerman.id.au> wrote:

> Anton Blanchard <anton@samba.org> writes:
> 
> > Hi,
> >
> > Updating to mainline as of last night, I started seeing the following
> > error when running the perf report TUI:
> >
> > 0x46068 [0x8]: failed to process type: 68
> >
> > This event is just PERF_RECORD_FINISHED_ROUND:
> >
> > 0x46068 [0x8]: event: 68
> > .
> > . ... raw event: size 8 bytes
> > .  0000:  44 00 00 00 00 00 08 00                          D.......        
> >
> > 0x46068 [0x8]: PERF_RECORD_FINISHED_ROUND
> >
> > Which of course is not our error. It took me a while to find the real
> > culprit:
> >
> >  14c00-14c00 g exc_virt_0x4c00_system_call  
>    ^
>    What's this? The address? If so it's wrong?
>    
> 
> > A zero length symbol, which __symbol__inc_addr_samples() barfs on:
> >
> >         if (addr < sym->start || addr >= sym->end) {
> > ...
> > 		return -ERANGE;
> >
> > Seems like we have 3 bugs here:  
> ...
> >
> > 3. Why do we have zero length symbols in the first place? Does the recent
> >    ppc64 exception clean up have something to do with it?  
> 
> Seems likely. But I can't see why.
> 
> AFAICS we have never emitted a size for those symbols:
> 
> Old:
> $ nm --print-size build/vmlinux | grep -w system_call_relon_pSeries
> c000000000004c00 T system_call_relon_pSeries
> 
> New:
> $ nm --print-size build/vmlinux | grep -w exc_virt_0x4c00_system_call
> c000000000004c00 T exc_virt_0x4c00_system_call
> 
> 
> It also doesn't look like we're emitting another symbol with the same
> address, which has caused confusion in the past:
> 
> Old:
> c000000000004c00 T exc_virt_0x4c00_system_call
> c000000000004d00 T exc_virt_0x4d00_single_step
> 
> New:
> c000000000004c00 T system_call_relon_pSeries
> c000000000004d00 T single_step_relon_pSeries
> 
> 
> So more digging required.

Yeah, strange. Maybe perf changes?

We were talking about adding size and type to the exception symbols
though.

Re: perf TUI fails with "failed to process type: 64"

[Anton Blanchard]

Hi Michael,

> >  14c00-14c00 g exc_virt_0x4c00_system_call  
>    ^
>    What's this? The address? If so it's wrong?

Offset into the binary I think, there's one 64kB page of ELF gunk at
the start.

> Seems likely. But I can't see why.
> 
> AFAICS we have never emitted a size for those symbols:
> 
> Old:
> $ nm --print-size build/vmlinux | grep -w system_call_relon_pSeries
> c000000000004c00 T system_call_relon_pSeries
> 
> New:
> $ nm --print-size build/vmlinux | grep -w exc_virt_0x4c00_system_call
> c000000000004c00 T exc_virt_0x4c00_system_call
> 
> 
> It also doesn't look like we're emitting another symbol with the same
> address, which has caused confusion in the past:
> 
> Old:
> c000000000004c00 T exc_virt_0x4c00_system_call
> c000000000004d00 T exc_virt_0x4d00_single_step
> 
> New:
> c000000000004c00 T system_call_relon_pSeries
> c000000000004d00 T single_step_relon_pSeries
> 
> So more digging required.

Thanks for checking, it's starting to sound like a perf bug.

Anton

Re: perf TUI fails with "failed to process type: 64"

[Michael Ellerman]

Anton Blanchard <anton@samba.org> writes:
>> >  14c00-14c00 g exc_virt_0x4c00_system_call  
>>    ^
>>    What's this? The address? If so it's wrong?
>
> Offset into the binary I think, there's one 64kB page of ELF gunk at
> the start.

Ah OK makes sense.

>> So more digging required.
>
> Thanks for checking, it's starting to sound like a perf bug.

Yeah possibly. I'll build an updated perf tomorrow and see if I can
reproduce.

As an aside, it looks like it would be easy to set the size with the new
macro. Does the patch below help at all?

The other thing Nick mentioned is we could set the type of those symbols
to function, though that's not entirely correct. But it might help perf
and probably wouldn't break anything else (that I can think of right now
anyway).

cheers


diff --git a/arch/powerpc/include/asm/head-64.h b/arch/powerpc/include/asm/head-64.h
index ab90c2fa1ea6..0e250d4eff44 100644
--- a/arch/powerpc/include/asm/head-64.h
+++ b/arch/powerpc/include/asm/head-64.h
@@ -122,6 +122,7 @@ name:
 	.error "Fixed entry overflow";				\
 	.abort;							\
 	.endif;							\
+	.size name, (. - name);					\
 	. = ((end) - sname##_start);				\