crazy idea (was: Re: [PATCH] perf tool: Carve out ctype.h et al)

crazy idea (was: Re: [PATCH] perf tool: Carve out ctype.h et al)

[Borislav Petkov]

On Sat, Jun 28, 2014 at 01:28:19AM +0200, Borislav Petkov wrote:
> Ok, cool. So guys, can we apply this one so that I can continue with the
> next round?

Ok, I had this crazy idea recently:

How about we copy the perf tool code we need for the RAS daemon and
start hacking on it in a separate repo, somewhere else?

This way we can start adding features and stuff and finally do some real
work on it. The code which deals with opening a tracepoint and reading
from it should be pretty stable by now so we won't have to sync back
with perf too often. And if we do, we'll copy the changed bits.

In the meantime, we can still work on the splitting of the kernel code
if it is desired.

In any case, I would definitely like to parallelize the work because
if we wait for the split to happen, we won't have a RAS daemon with
persistent events by Christmas 2050 and by then who knows whether we're
even alive.</sarcastic joke>

So what do you guys think, makes some sense at least?

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: crazy idea (was: Re: [PATCH] perf tool: Carve out ctype.h et al)

[Jean Pihet]

Hi Boris,

-- added Fu Wei in the loop --

On 30 June 2014 09:07, Borislav Petkov <bp@alien8.de> wrote:
> On Sat, Jun 28, 2014 at 01:28:19AM +0200, Borislav Petkov wrote:
>> Ok, cool. So guys, can we apply this one so that I can continue with the
>> next round?
>
> Ok, I had this crazy idea recently:
>
> How about we copy the perf tool code we need for the RAS daemon and
> start hacking on it in a separate repo, somewhere else?
>
> This way we can start adding features and stuff and finally do some real
> work on it. The code which deals with opening a tracepoint and reading
> from it should be pretty stable by now so we won't have to sync back
> with perf too often. And if we do, we'll copy the changed bits.
>
> In the meantime, we can still work on the splitting of the kernel code
> if it is desired.
>
> In any case, I would definitely like to parallelize the work because
> if we wait for the split to happen, we won't have a RAS daemon with
> persistent events by Christmas 2050 and by then who knows whether we're
> even alive.</sarcastic joke>
>
> So what do you guys think, makes some sense at least?

That makes perfect sense since:
- we need the RAS daemon implementation soon (not just before the
final blackdown ;-),
- for other tools to exist (like RAS daemon) we need to split out the
perf code in small units. This will take a significant amount of work
and time to happen,
- the RAS daemon basically is a shrink-down perf with persistent events support.

How do we start the RAS daemon? As of today we have a prototype
implementation in C.

Fu Wei, can you confirm about the initial RAS daemon prototype?

Thx, regards,
Jean

>
> --
> Regards/Gruss,
>     Boris.
>
> Sent from a fat crate under my desk. Formatting is fine.
> --

Re: crazy idea (was: Re: [PATCH] perf tool: Carve out ctype.h et al)

[Borislav Petkov]

On Mon, Jun 30, 2014 at 11:39:39AM +0200, Jean Pihet wrote:
> That makes perfect sense since:
> - we need the RAS daemon implementation soon (not just before the
> final blackdown ;-),

Yeah :-)

> - for other tools to exist (like RAS daemon) we need to split out the
> perf code in small units. This will take a significant amount of work
> and time to happen,

Yep, this is what the experience so far tells us. And I'm not proposing
any change to the final objective we're following - to get the perf tool
split. I'm just saying we should "execute another thread" while the perf
tool pipeline of patches stalls, figuratively speaking. :-)

> - the RAS daemon basically is a shrink-down perf with persistent
> events support.

Right.

> How do we start the RAS daemon? As of today we have a prototype
> implementation in C.
>
> Fu Wei, can you confirm about the initial RAS daemon prototype?

Well, do you have it as a standalong program? If so, you can put it in
a repo somewhere and we can start hacking away and playing with it. If
not, we will have to copy the perf code *into* the RAS daemon first so
that we can have a separate source on which we all can work on.

Someone should be a maintainer of some sorts who merges the patches.

And we will continue working on the split so that once the perf tool is
properly separated, we can switch to it in the RAS daemon and drop the
copied code.

This is, to me at least, the best possible thing we can do right now so
as not to slow us all down.

Any other opinions?

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: crazy idea (was: Re: [PATCH] perf tool: Carve out ctype.h et al)

[Robert Richter]

On 30.06.14 12:01:15, Borislav Petkov wrote:
> Well, do you have it as a standalong program? If so, you can put it in
> a repo somewhere and we can start hacking away and playing with it. If
> not, we will have to copy the perf code *into* the RAS daemon first so
> that we can have a separate source on which we all can work on.
> 
> Someone should be a maintainer of some sorts who merges the patches.
> 
> And we will continue working on the split so that once the perf tool is
> properly separated, we can switch to it in the RAS daemon and drop the
> copied code.
> 
> This is, to me at least, the best possible thing we can do right now so
> as not to slow us all down.
> 
> Any other opinions?

Perfectly fine with me.

-Robert

Re: crazy idea (was: Re: [PATCH] perf tool: Carve out ctype.h et al)

[Arnaldo Carvalho de Melo]

Em Mon, Jun 30, 2014 at 04:50:31PM +0200, Robert Richter escreveu:
> On 30.06.14 12:01:15, Borislav Petkov wrote:
> > Well, do you have it as a standalong program? If so, you can put it in
> > a repo somewhere and we can start hacking away and playing with it. If
> > not, we will have to copy the perf code *into* the RAS daemon first so
> > that we can have a separate source on which we all can work on.

> > Someone should be a maintainer of some sorts who merges the patches.

> > And we will continue working on the split so that once the perf tool is
> > properly separated, we can switch to it in the RAS daemon and drop the
> > copied code.

> > This is, to me at least, the best possible thing we can do right now so
> > as not to slow us all down.

> > Any other opinions?
 
> Perfectly fine with me.

Yeap, guess its the most sensible to do given how things went so far.

Please submit patches when you fix things on code copied from
tools/perf/ and generally when improving it.

- Arnaldo

Re: crazy idea (was: Re: [PATCH] perf tool: Carve out ctype.h et al)

[Borislav Petkov]

On Mon, Jun 30, 2014 at 11:58:31AM -0300, Arnaldo Carvalho de Melo wrote:
> Yeap, guess its the most sensible to do given how things went so far.
>
> Please submit patches when you fix things on code copied from
> tools/perf/ and generally when improving it.

Sure.

Btw, should I still be working on the split even if the RAS daemon is
out-of-tree?

If I do, the advantage is that there'll be no pressure on anyone and
we'll be applying patches as they come.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: crazy idea (was: Re: [PATCH] perf tool: Carve out ctype.h et al)

[Arnaldo Carvalho de Melo]

Em Mon, Jun 30, 2014 at 05:18:02PM +0200, Borislav Petkov escreveu:
> On Mon, Jun 30, 2014 at 11:58:31AM -0300, Arnaldo Carvalho de Melo wrote:
> > Yeap, guess its the most sensible to do given how things went so far.
> >
> > Please submit patches when you fix things on code copied from
> > tools/perf/ and generally when improving it.
> 
> Sure.
> 
> Btw, should I still be working on the split even if the RAS daemon is
> out-of-tree?
> 
> If I do, the advantage is that there'll be no pressure on anyone and
> we'll be applying patches as they come.

Yes, I think we should continue moving stuff out of tools/perf/ and into
a place that can be shared with other tools/ living codebases.

Doing so will of course help tools that live elsewhere, outside the
kernel sources as well.

I thought that one way to overcome the fact that the ras daemon doesn't
live in the kernel sources (now or ever) would be to pick whatever lower
hanging fruits there are in place, like:

[acme@zoo linux]$ find tools/ -name list.h
tools/firewire/list.h
tools/lib/lockdep/uinclude/linux/list.h
tools/perf/util/include/linux/list.h
[acme@zoo linux]$ 

Haven't found much more than that tho at this point.

- Arnaldo

Re: crazy idea (was: Re: [PATCH] perf tool: Carve out ctype.h et al)

[Borislav Petkov]

On Mon, Jun 30, 2014 at 04:28:14PM -0300, Arnaldo Carvalho de Melo wrote:
> Yes, I think we should continue moving stuff out of tools/perf/ and
> into a place that can be shared with other tools/ living codebases.
>
> Doing so will of course help tools that live elsewhere, outside the
> kernel sources as well.

Ok.

> I thought that one way to overcome the fact that the ras daemon doesn't
> live in the kernel sources (now or ever) would be to pick whatever lower
> hanging fruits there are in place, like:
> 
> [acme@zoo linux]$ find tools/ -name list.h
> tools/firewire/list.h
> tools/lib/lockdep/uinclude/linux/list.h
> tools/perf/util/include/linux/list.h
> [acme@zoo linux]$ 
> 
> Haven't found much more than that tho at this point.

Right, the focus is only on tools that will be needed by other tools
only and carve only those out.

However, list.h is pretty generic and should be only one header which
contains the whole functionality.

Oh, and there is also another reason why pretty generic functionality in
tools/ should get merged - not only whether it is used by something else
or not: you don't want the wild growth of almost the same headers copied
at different times from the kernel in tools/ but rather one concise tree
of utilities, clean and organized. But for that I'd guess someone has
to take over the whole tools/ dir and orchestrate everything that gets
merged in there.

Right now we have a wild bunch of things in there, some of them build
and maybe work, some of them who knows, and so on... And then there's
perf tool.

:-)

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--