adding trace-cmd's plugins to perf

adding trace-cmd's plugins to perf

[David Ahern]

Right now perf script cannot process kvm tracepoints:

perf record -e kvm:* -p 6446 -- sleep 5

perf script
  Warning: Error: expected type 4 but read 7
  Warning: Error: expected type 5 but read 0
  Warning: failed to read event print fmt for kvm_apic
  Warning: Error: expected type 4 but read 7
  Warning: Error: expected type 5 but read 0
  Warning: failed to read event print fmt for kvm_inj_exception
  Fatal: bad op token {

trace-cmd can parse the events through the kvm plugin.

As I understand it trace-cmd and perf have a lot of similar code, so I
would expect to be able to add the plugin capability to perf somewhat
easily. However, that does not seem to be the right thing to do (copying
yet more code between the two).

Before I invest a lot of time on this path I figured I should ask what
the intentions (roadmap seems to be too formal a word ;-)) are about
merging common code between the two commands. Also, trace-cmd and perf
are in separate repositories so a shared lib is going to inconvenience
one of the two.

Thanks,
David

Re: adding trace-cmd's plugins to perf

[Frederic Weisbecker]

On Sat, Apr 09, 2011 at 09:50:29PM -0600, David Ahern wrote:
> Right now perf script cannot process kvm tracepoints:
> 
> perf record -e kvm:* -p 6446 -- sleep 5
> 
> perf script
>   Warning: Error: expected type 4 but read 7
>   Warning: Error: expected type 5 but read 0
>   Warning: failed to read event print fmt for kvm_apic
>   Warning: Error: expected type 4 but read 7
>   Warning: Error: expected type 5 but read 0
>   Warning: failed to read event print fmt for kvm_inj_exception
>   Fatal: bad op token {
> 
> trace-cmd can parse the events through the kvm plugin.
> 
> As I understand it trace-cmd and perf have a lot of similar code, so I
> would expect to be able to add the plugin capability to perf somewhat
> easily. However, that does not seem to be the right thing to do (copying
> yet more code between the two).
> 
> Before I invest a lot of time on this path I figured I should ask what
> the intentions (roadmap seems to be too formal a word ;-))

Hehe :)

> are about merging common code between the two commands. Also, trace-cmd and perf
> are in separate repositories so a shared lib is going to inconvenience
> one of the two.

So, we copied the tools/perf/util/trace-event-* files from trace-cmd to perf
a while go. Then both files took their own path, both pulling fixes/enhancement
from each others (probably more in the trace-cmd -> perf direction).

And perf is indeed a bit backward wrt parsing, because it lacks those plugins
for example. So now it would be nice to unify that in a common lib so that it
works well in both.

Steve proposed a shared tools/trace.so, that perf and trace-cmd could plug
into, I really would like to see that happening too.

I think Ingo had some reserves about this, due to potential versioning
and compatibility that such a dynamic lib would involve.

However, this seems to me a very important and necessary step to unify our
tools.

Re: adding trace-cmd's plugins to perf

[Arnaldo Carvalho de Melo]

Em Tue, Apr 12, 2011 at 06:22:47PM +0200, Frederic Weisbecker escreveu:
> On Sat, Apr 09, 2011 at 09:50:29PM -0600, David Ahern wrote:
> > Right now perf script cannot process kvm tracepoints:
> > 
> > perf record -e kvm:* -p 6446 -- sleep 5
> > 
> > perf script
> >   Warning: Error: expected type 4 but read 7

> > are about merging common code between the two commands. Also, trace-cmd and perf
> > are in separate repositories so a shared lib is going to inconvenience
> > one of the two.
> 
> So, we copied the tools/perf/util/trace-event-* files from trace-cmd to perf
> a while go. Then both files took their own path, both pulling fixes/enhancement
> from each others (probably more in the trace-cmd -> perf direction).
> 
> And perf is indeed a bit backward wrt parsing, because it lacks those plugins
> for example. So now it would be nice to unify that in a common lib so that it
> works well in both.
> 
> Steve proposed a shared tools/trace.so, that perf and trace-cmd could plug
> into, I really would like to see that happening too.
> 
> I think Ingo had some reserves about this, due to potential versioning
> and compatibility that such a dynamic lib would involve.

So we don't commit to an ABI for a while, just linking with it, changing
users when changing the ABI.

I encourage David to go and add the bits of trace-cmd he needs for
support the plugins he wants used in perf on tools/lib/ and make perf
use it.

That way we again reduce the differences between the codebases, or at least
reduce the feature gap.

- Arnaldo

Re: adding trace-cmd's plugins to perf

[Borislav Petkov]

On Tue, Apr 12, 2011 at 03:35:36PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Tue, Apr 12, 2011 at 06:22:47PM +0200, Frederic Weisbecker escreveu:
> > On Sat, Apr 09, 2011 at 09:50:29PM -0600, David Ahern wrote:
> > > Right now perf script cannot process kvm tracepoints:
> > > 
> > > perf record -e kvm:* -p 6446 -- sleep 5
> > > 
> > > perf script
> > >   Warning: Error: expected type 4 but read 7
> 
> > > are about merging common code between the two commands. Also, trace-cmd and perf
> > > are in separate repositories so a shared lib is going to inconvenience
> > > one of the two.
> > 
> > So, we copied the tools/perf/util/trace-event-* files from trace-cmd to perf
> > a while go. Then both files took their own path, both pulling fixes/enhancement
> > from each others (probably more in the trace-cmd -> perf direction).
> > 
> > And perf is indeed a bit backward wrt parsing, because it lacks those plugins
> > for example. So now it would be nice to unify that in a common lib so that it
> > works well in both.
> > 
> > Steve proposed a shared tools/trace.so, that perf and trace-cmd could plug
> > into, I really would like to see that happening too.
> > 
> > I think Ingo had some reserves about this, due to potential versioning
> > and compatibility that such a dynamic lib would involve.
> 
> So we don't commit to an ABI for a while, just linking with it, changing
> users when changing the ABI.
> 
> I encourage David to go and add the bits of trace-cmd he needs for
> support the plugins he wants used in perf on tools/lib/ and make perf
> use it.
> 
> That way we again reduce the differences between the codebases, or at least
> reduce the feature gap.

Also, just FYI, I have this RAS daemon patchset which splits perf functionality
into different sub-libs, which are more or less topic based:

tools/lib/trace/ - all the trace-event* stuff from Steven
tools/lib/lk/	- generic enough stuff (git headers, etc) which several tools could use
tools/lib/perf/ - functionality related to perf events

Here's the last submission http://marc.info/?l=linux-edac&m=129562244211501&w=2
and I'm currently finishing a new rebase against tip/perf/core.

If you're fine with that split I'd suggest I load off my stuff first so
that David can continue from there and I can finally stop rebasing like
crazy each time perf moves forward.

Opinions? Suggestions?

Thanks.

-- 
Regards/Gruss,
    Boris.

Re: adding trace-cmd's plugins to perf

[Arnaldo Carvalho de Melo]

Em Tue, Apr 12, 2011 at 08:51:04PM +0200, Borislav Petkov escreveu:
> On Tue, Apr 12, 2011 at 03:35:36PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Tue, Apr 12, 2011 at 06:22:47PM +0200, Frederic Weisbecker escreveu:
> > > On Sat, Apr 09, 2011 at 09:50:29PM -0600, David Ahern wrote:
> > > > Right now perf script cannot process kvm tracepoints:
> > > > 
> > > > perf record -e kvm:* -p 6446 -- sleep 5
> > > > 
> > > > perf script
> > > >   Warning: Error: expected type 4 but read 7
> > 
> > > > are about merging common code between the two commands. Also, trace-cmd and perf
> > > > are in separate repositories so a shared lib is going to inconvenience
> > > > one of the two.
> > > 
> > > So, we copied the tools/perf/util/trace-event-* files from trace-cmd to perf
> > > a while go. Then both files took their own path, both pulling fixes/enhancement
> > > from each others (probably more in the trace-cmd -> perf direction).
> > > 
> > > And perf is indeed a bit backward wrt parsing, because it lacks those plugins
> > > for example. So now it would be nice to unify that in a common lib so that it
> > > works well in both.
> > > 
> > > Steve proposed a shared tools/trace.so, that perf and trace-cmd could plug
> > > into, I really would like to see that happening too.
> > > 
> > > I think Ingo had some reserves about this, due to potential versioning
> > > and compatibility that such a dynamic lib would involve.
> > 
> > So we don't commit to an ABI for a while, just linking with it, changing
> > users when changing the ABI.
> > 
> > I encourage David to go and add the bits of trace-cmd he needs for
> > support the plugins he wants used in perf on tools/lib/ and make perf
> > use it.
> > 
> > That way we again reduce the differences between the codebases, or at least
> > reduce the feature gap.
> 
> Also, just FYI, I have this RAS daemon patchset which splits perf functionality
> into different sub-libs, which are more or less topic based:
> 
> tools/lib/trace/ - all the trace-event* stuff from Steven
> tools/lib/lk/	- generic enough stuff (git headers, etc) which several tools could use
> tools/lib/perf/ - functionality related to perf events
> 
> Here's the last submission http://marc.info/?l=linux-edac&m=129562244211501&w=2
> and I'm currently finishing a new rebase against tip/perf/core.
> 
> If you're fine with that split I'd suggest I load off my stuff first so
> that David can continue from there and I can finally stop rebasing like
> crazy each time perf moves forward.
> 
> Opinions? Suggestions?

Yeah, David, can you try Borislav's patchset and tell us what you think
wrt servicing your needs?

- Arnaldo

Re: adding trace-cmd's plugins to perf

[Borislav Petkov]

On Tue, Apr 12, 2011 at 03:56:52PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Tue, Apr 12, 2011 at 08:51:04PM +0200, Borislav Petkov escreveu:
> > On Tue, Apr 12, 2011 at 03:35:36PM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Tue, Apr 12, 2011 at 06:22:47PM +0200, Frederic Weisbecker escreveu:
> > > > On Sat, Apr 09, 2011 at 09:50:29PM -0600, David Ahern wrote:
> > > > > Right now perf script cannot process kvm tracepoints:
> > > > > 
> > > > > perf record -e kvm:* -p 6446 -- sleep 5
> > > > > 
> > > > > perf script
> > > > >   Warning: Error: expected type 4 but read 7
> > > 
> > > > > are about merging common code between the two commands. Also, trace-cmd and perf
> > > > > are in separate repositories so a shared lib is going to inconvenience
> > > > > one of the two.
> > > > 
> > > > So, we copied the tools/perf/util/trace-event-* files from trace-cmd to perf
> > > > a while go. Then both files took their own path, both pulling fixes/enhancement
> > > > from each others (probably more in the trace-cmd -> perf direction).
> > > > 
> > > > And perf is indeed a bit backward wrt parsing, because it lacks those plugins
> > > > for example. So now it would be nice to unify that in a common lib so that it
> > > > works well in both.
> > > > 
> > > > Steve proposed a shared tools/trace.so, that perf and trace-cmd could plug
> > > > into, I really would like to see that happening too.
> > > > 
> > > > I think Ingo had some reserves about this, due to potential versioning
> > > > and compatibility that such a dynamic lib would involve.
> > > 
> > > So we don't commit to an ABI for a while, just linking with it, changing
> > > users when changing the ABI.
> > > 
> > > I encourage David to go and add the bits of trace-cmd he needs for
> > > support the plugins he wants used in perf on tools/lib/ and make perf
> > > use it.
> > > 
> > > That way we again reduce the differences between the codebases, or at least
> > > reduce the feature gap.
> > 
> > Also, just FYI, I have this RAS daemon patchset which splits perf functionality
> > into different sub-libs, which are more or less topic based:
> > 
> > tools/lib/trace/ - all the trace-event* stuff from Steven
> > tools/lib/lk/	- generic enough stuff (git headers, etc) which several tools could use
> > tools/lib/perf/ - functionality related to perf events
> > 
> > Here's the last submission http://marc.info/?l=linux-edac&m=129562244211501&w=2
> > and I'm currently finishing a new rebase against tip/perf/core.
> > 
> > If you're fine with that split I'd suggest I load off my stuff first so
> > that David can continue from there and I can finally stop rebasing like
> > crazy each time perf moves forward.
> > 
> > Opinions? Suggestions?
> 
> Yeah, David, can you try Borislav's patchset and tell us what you think
> wrt servicing your needs?

Yeah, but first let me post the latest version - I'll try to do so in
the next couple of days.

Thanks.

-- 
Regards/Gruss,
    Boris.

Re: adding trace-cmd's plugins to perf

[David Ahern]

On 04/12/11 13:01, Borislav Petkov wrote:
>>> Also, just FYI, I have this RAS daemon patchset which splits perf functionality
>>> into different sub-libs, which are more or less topic based:
>>>
>>> tools/lib/trace/ - all the trace-event* stuff from Steven
>>> tools/lib/lk/	- generic enough stuff (git headers, etc) which several tools could use

Is this going to contain the option parsing as well? e.g., something
tools/kvm is picking up from perf.


>>> tools/lib/perf/ - functionality related to perf events
>>>
>>> Here's the last submission http://marc.info/?l=linux-edac&m=129562244211501&w=2
>>> and I'm currently finishing a new rebase against tip/perf/core.
>>>
>>> If you're fine with that split I'd suggest I load off my stuff first so
>>> that David can continue from there and I can finally stop rebasing like
>>> crazy each time perf moves forward.
>>>
>>> Opinions? Suggestions?
>>
>> Yeah, David, can you try Borislav's patchset and tell us what you think
>> wrt servicing your needs?
> 
> Yeah, but first let me post the latest version - I'll try to do so in
> the next couple of days.
> 
> Thanks.
> 

Sure. This is a background task, and I still need to spend time on
trace-cmd to understand it. ie. it's not something I expect to do within
the next week -- more like within the next month if I get the time.

David

Re: adding trace-cmd's plugins to perf

[Borislav Petkov]

On Tue, Apr 12, 2011 at 01:39:30PM -0600, David Ahern wrote:
> 
> 
> On 04/12/11 13:01, Borislav Petkov wrote:
> >>> Also, just FYI, I have this RAS daemon patchset which splits perf functionality
> >>> into different sub-libs, which are more or less topic based:
> >>>
> >>> tools/lib/trace/ - all the trace-event* stuff from Steven
> >>> tools/lib/lk/	- generic enough stuff (git headers, etc) which several tools could use
> 
> Is this going to contain the option parsing as well? e.g., something
> tools/kvm is picking up from perf.

Nope, not yet. Acme suggested I export only the stuff I need. But you
can definitely add them there since I have the Makefiles adjusted and
you'd have to move the .[ch] files and adjust the includes.

Here's what's there so far:

lib/lk/
|-- cpumap.c
|-- cpumap.h
|-- ctype.c
|-- debugfs.c
|-- debugfs.h
|-- Makefile
|-- thread_map.c
|-- thread_map.h
|-- types.h
|-- usage.c
|-- util.c
|-- util.h
|-- xyarray.c
`-- xyarray.h

> >>> tools/lib/perf/ - functionality related to perf events
> >>>
> >>> Here's the last submission http://marc.info/?l=linux-edac&m=129562244211501&w=2
> >>> and I'm currently finishing a new rebase against tip/perf/core.
> >>>
> >>> If you're fine with that split I'd suggest I load off my stuff first so
> >>> that David can continue from there and I can finally stop rebasing like
> >>> crazy each time perf moves forward.
> >>>
> >>> Opinions? Suggestions?
> >>
> >> Yeah, David, can you try Borislav's patchset and tell us what you think
> >> wrt servicing your needs?
> > 
> > Yeah, but first let me post the latest version - I'll try to do so in
> > the next couple of days.
> > 
> > Thanks.
> > 
> 
> Sure. This is a background task, and I still need to spend time on
> trace-cmd to understand it. ie. it's not something I expect to do within
> the next week -- more like within the next month if I get the time.

Ok, I'll let you know when I have it done.

-- 
Regards/Gruss,
    Boris.