Re: [RFC patch 00/41] LTTng 0.105 core for Linux 2.6.27-rc9

Frederic Weisbecker
2009-03-06T19:02:22Z

On Fri, Mar 06, 2009 at 01:34:43PM -0500, Steven Rostedt wrote:
>
> Hi Mathieu,
>
> Thanks for posting this. But it might be better to post in much smaller
> chunks. Lets work out the little things first. Posting 41 patches is a bit
> overwhelming. Took me a few hours to look at them all, and when I got to
> the end, I forgot what was at the beginning.
>
> There's also minor changes to core kernel infrastructure code. seq_file,
> exporting functions, and such. These really need to be packaged
> separately, and sent to the proper maintainers. Having them in a patch
> bomb does not get the proper focus that they need.
>

Yes, I must confess I tried to review some of them but I have been discouraged
by the high volume and the multiple subjects that come with.

Iterating with smaller topics at a time, more focused subjects would help us to bring
the attention it deserves.

Frederic.

> On Thu, 5 Mar 2009, Mathieu Desnoyers wrote:
>
> > Hi,
> >
> > I spent the last 4-5 months working with the Fujitsu team at implementing the
> > tracer elements identified as goals at Kernel Summit 2008 and at the following
> > Plumber Conference. My idea was to incremententally adapt the LTTng tracer,
> > currently used in the industry and well tested, to those requirements.
>
> We really need to work together on this too. The biggest requirement that
> came out of that conference was to have a "single unified buffering
> system". And this was discussed quite heavily on LKML afterwards. All
> development was done incrementally and publicly.
> >
> > I spent the last days rearranging/folding/inspecting the LTTng patchset
> > to prepare it for an LKML post. The version 0.105 in the LTTng git tree
> > corresponds to the patchset I am posting here. The said patchset will
> > only include the core features of LTTng, excluding the timestamping
> > infrastructure (trace clock) and excluding the instrumentation.
> >
> > The corresponding git tree contains also the trace clock patches and the lttng
> > instrumentation. The trace clock is required to use the tracer, but it can be
> > used without the instrumentation : there is already a kprobes and userspace
> > event support included in this patchset.
> >
> > This tracer exports binary data through buffers using splice(). The resulting
> > binary files can be parsed from userspace because the format string metadata is
> > exported in the files. The event set can be enhanced by adding tracepoints to
> > the kernel code and by creating probe modules, which connects callbacks to the
> > tracepoints and contain the format string metainformation. Those callbacks are
> > responsible for writing the data in the trace buffers. This separation between
> > the trace buffer format string and the tracepoints is done on purpose so the
> > core kernel instrumentation (tracepoints) is not exported to userspace, which
> > will make maintainance much easier.
>
> I've discussed this in my previous email. There is still a separation with
> the TRACE_EVENT_FORMAT and the maintainers code. The format sting is
> "hint" only and may change without notice. LTTng could use it or ignore
> it, it is up to the tracer to actually export that string. ftrace chose to
> export it because it was a simple way to extract that information. My
> utility will need to do a bit more work when the events get more complex,
> but the way it is set up, we can do that on a case by case basis.
>
>
> >
> > The tree including the trace clock patches is available at :
> >
> > git://git.kernel.org/pub/scm/linux/kernel/git/compudj/linux-2.6-lttng.git
> > branch : 2.6.29-rc7-lttng-0.105
> >
> > Project website : http://www.lttng.org/
> >
> > Information about how to install and use the tracer is available at :
> >
> > http://ltt.polymtl.ca/svn/trunk/lttv/LTTngManual.html
> >
> > The size of the LTTng core patchset is 41 patches. The diffstat details
> > as follow :
>
>
> Again, this is overwhelming. This needs to be broken up into a small
> subsets that can be examined piece by piece.
>
> >
> > include/linux/ltt-core.h | 35
> > include/linux/ltt-relay.h | 161 +
> > include/linux/ltt-tracer.h | 43
> > include/linux/marker.h | 121
> > kernel/marker.c | 353 ++
> > kernel/module.c | 31
> > linux-2.6-lttng/Documentation/markers.txt | 17
> > linux-2.6-lttng/MAINTAINERS | 7
> > linux-2.6-lttng/Makefile | 2
> > linux-2.6-lttng/arch/powerpc/kernel/traps.c | 5
> > linux-2.6-lttng/arch/powerpc/platforms/cell/spufs/spufs.h | 6
> > linux-2.6-lttng/arch/sparc/Makefile | 2
> > linux-2.6-lttng/arch/x86/kernel/dumpstack.c | 5
> > linux-2.6-lttng/arch/x86/mm/fault.c | 1
> > linux-2.6-lttng/fs/ext4/fsync.c | 8
> > linux-2.6-lttng/fs/ext4/ialloc.c | 17
> > linux-2.6-lttng/fs/ext4/inode.c | 79
> > linux-2.6-lttng/fs/ext4/mballoc.c | 71
> > linux-2.6-lttng/fs/ext4/mballoc.h | 2
> > linux-2.6-lttng/fs/ext4/super.c | 6
> > linux-2.6-lttng/fs/jbd2/checkpoint.c | 7
> > linux-2.6-lttng/fs/jbd2/commit.c | 12
> > linux-2.6-lttng/fs/pipe.c | 5
> > linux-2.6-lttng/fs/select.c | 41
> > linux-2.6-lttng/fs/seq_file.c | 45
> > linux-2.6-lttng/fs/splice.c | 1
>
> There is a lot of code above that needs to be in their own patch series.
> Maintainers do not have the time to pick through 41 patches to find out
> which patch might deal with their code.
>
> Thanks,
>
> -- Steve
>
>
> > linux-2.6-lttng/include/linux/immediate.h | 94
> > linux-2.6-lttng/include/linux/kvm_host.h | 12
> > linux-2.6-lttng/include/linux/ltt-channels.h | 94
> > linux-2.6-lttng/include/linux/ltt-core.h | 47
> > linux-2.6-lttng/include/linux/ltt-relay.h | 186 +
> > linux-2.6-lttng/include/linux/ltt-tracer.h | 731 ++++++
> > linux-2.6-lttng/include/linux/ltt-type-serializer.h | 107
> > linux-2.6-lttng/include/linux/marker.h | 16
> > linux-2.6-lttng/include/linux/module.h | 6
> > linux-2.6-lttng/include/linux/poll.h | 2
> > linux-2.6-lttng/include/linux/seq_file.h | 20
> > linux-2.6-lttng/include/trace/ext4.h | 129 +
> > linux-2.6-lttng/include/trace/jbd2.h | 19
> > linux-2.6-lttng/init/Kconfig | 2
> > linux-2.6-lttng/kernel/kallsyms.c | 1
> > linux-2.6-lttng/kernel/marker.c | 12
> > linux-2.6-lttng/kernel/module.c | 32
> > linux-2.6-lttng/ltt/Kconfig | 130 +
> > linux-2.6-lttng/ltt/Makefile | 15
> > linux-2.6-lttng/ltt/ltt-channels.c | 338 ++
> > linux-2.6-lttng/ltt/ltt-core.c | 101
> > linux-2.6-lttng/ltt/ltt-filter.c | 66
> > linux-2.6-lttng/ltt/ltt-kprobes.c | 479 +++
> > linux-2.6-lttng/ltt/ltt-marker-control.c | 265 ++
> > linux-2.6-lttng/ltt/ltt-relay-alloc.c | 715 +++++
> > linux-2.6-lttng/ltt/ltt-relay-locked.c | 1704 ++++++++++++++
> > linux-2.6-lttng/ltt/ltt-serialize.c | 685 +++++
> > linux-2.6-lttng/ltt/ltt-trace-control.c | 1061 ++++++++
> > linux-2.6-lttng/ltt/ltt-tracer.c | 1210 +++++++++
> > linux-2.6-lttng/ltt/ltt-type-serializer.c | 96
> > linux-2.6-lttng/ltt/ltt-userspace-event.c | 131 +
> > linux-2.6-lttng/samples/markers/Makefile | 2
> > linux-2.6-lttng/samples/markers/marker-example.c | 4
> > linux-2.6-lttng/samples/markers/probe-example.c | 10
> > linux-2.6-lttng/samples/markers/test-multi.c | 120
> > linux-2.6-lttng/virt/kvm/kvm_trace.c | 12
> > ltt/Kconfig | 24
> > ltt/Makefile | 2
> > ltt/ltt-relay-alloc.c | 80
> > 65 files changed, 9445 insertions(+), 398 deletions(-)
> >
> >
> > Comments are welcome.
> >
> > Mathieu
> >
> >
> > --
> > Mathieu Desnoyers
> > OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F BA06 3F25 A8FE 3BAE 9A68
> >