Re: [ltt-dev] Re: [PATCH] LTT for 2.5.38 1/9: Core infrastructure

bob
2002-09-23T13:54:19Z

Ingo Molnar writes:
>
> On Sun, 22 Sep 2002, bob wrote:
>
> > Yes this is simple code - similar to the model we use in K42. Still,
> > couple of things about the below.
> >
> > 1) the !event_wanted can be done outside the function, in a macro so
> > that the only cost if tracing is disabled is a hot cache hit on a mask
> > (not function call) - that helps with your comment:
> > > The event_wanted() filter function should be made as fast as possible.
>
> yes. It's a cost to be considered, but the main issue these days is the
> icache cost of inlining. So generally we are leaning towards the
> least-impact inlining cost.

mmm - that seems a reasonable trade-off.

> > 2) If you use the lockless scheme you do not need to disable interrupts.
> > In K42 we manage to do the entire log operation in 21 instructions and
> > about as many cycles (couple more for getting time). We do this from
> > user space as well, disabling interrupts precludes this model (may of
> > may not be a problem). I was really leaning hard away from even the
> > cost of making a system call and disabling interrupts. Do people on the
> > kernel dev team feel this is an acceptable cost? Is migration prevented
> > when interrupts are disabled? This is something for us to consider.
>
> the trace() functions runs purely in kernel-space, so doing a cli/sti is
> not a performance problem - if it can be avoided it saves a few cycles,
> but it does not have any global costs. But i dont think reliable tracing
> can be done without disabling interrupts - how do you guarantee that there
> will be no trace 'holes' due to interruption at the wrong instruction?

We do have a way of guaranteeing no 'holes' get created unless the process
is interrupted for a *very* long time or killed (which could happen) during
the logging of an event. The code is a little more complicated and does
require an atomic operation that may be more or less equivalent to the cli
cost. In K42, and other OSes I worked on, we wanted very efficient logging
from user space as well. I think there might be a place for understanding
libc, database, jvm, performance, for examples, but if we really only do
log events in kernel space then the cli/sti approach is simpler and roughly
equivalent performance.

>
> > 3) All trace events should not have to have the same number of data
> > words logged - though I think that's just a packaging/interface issue
> > the code below would just be placed behind macros which correctly
> > package up the right number of arguments.
>
> yes, agreed, this can be solved by having some sort of RLA, tightly packed
> trace buffer. Trace buffer usage is definitely one of the more important
> points.

Yes! and we also have a scheme to allowed such a packed buffer stream to be
randomaly accessed on disk (useful if you have 100Ms or Gs of data).

-bob

Robert Wisniewski
The K42 MP OS Project
Advanced Operating Systems
Scalable Parallel Systems
IBM T.J. Watson Research Center
914-945-3181
http://www.research.ibm.com/K42/
bob@watson.ibm.com