Re: [PATCH 0/11] LTTng-core (basic tracing infrastructure) 0.5.108

[Karim Yaghmour <karim@opersys.com>]

Alan Cox wrote:
> A lot of us have plenty of experience helping customers and end users
> trace bugs. Thats a good part of why we get paid in the first place.

But of course, and I wouldn't dare compare my experience with yours.

FWIW, though, I submit to you that there is a difference in between
helping a customer trace something and actually attempting to create
a tool which standard users can use to trace their own stuff.

Then, again, my experience may just be lacking.

Here's an example just for the fun of it: I was giving a class at
a customer's site. It so happened they scheduled this class right
after product delivery (advice: this is a mistake.) And, predictably,
in came the technician asking for Joe, out went Joe, in came Joe,
repeat. They spent quite some time after hours trying to figure
this one out. Midweek, they asked if I could help, they were
having some odd behavior in user-space on a custom-developed board.
Try as I may, none of the standard user-space stuff was effective.
Ok, time to try ltt. Now this was a "vendor" kernel, with
preemption (ok, I'm not telling who, but this was definitely
before Ingo's work) -- the sort of which I hadn't dabbled in
before. I spent the evening trying to figure out how the heck the
thing worked to no avail -- the locking mechanisms were just
wrong for what ltt needed at the time. Last day I asked him if
they could get a *normal* kernel on there and someone somewhere
found an odd-port stable enough to run. So got an ltt patch,
customized it for said kernel (would have had to do something
similar if it were probe points instead of static traces), got a
trace, and within 5 minutes we had found a bug in their custom
hardware (and no, their drivers were just fine). This customer
would not have even needed me or needed to waste their time if he
had been able to get a trace for his bastardized kernel. But
the way the anti-static-instrumentation creed goes this
customer would still have needed me ... or someone else ...
<conspiracy> wait a minute, maybe that's not a coincidence ...
</conspiracy> ;)

Karim