Tracing KVM with LTTng

[Julien Desfossez <ju@klipix.org>]

Hello,

I'm currently doing my master degree on tracing and measuring with LTTng
the impact of virtualization on a physical host.
One focus of this work is to be able to represent in a graphical way the
state of a machine where virtual machines are running.

I have started to connect the already defined TRACE_EVENTs with LTTng
probes and adapted LTTV to represent a new state : VIRT.

In the following screenshots, I captured a trace of 4 seconds on the
physical machine. My machine has 8 CPUs, a KVM VM was started with 4
VCPUs. At the beginning, the VM was idle, after 1 second I started an
infinite loop and an other 1 second later.

On [1] you can see the impact of the loops on the kvm threads during the
4 seconds.
On [2] you can see a closer look of the state of the kvm threads (blue
means syscall, red means running in VM mode (vm_entry), dark yellow
means waiting for CPU).
On [3] you can see the relationship between the kvm threads and other
processes on the system (here especially kondemand and ksmd), each white
dot meaning there are events not displayed because the zoom level is to high
On [4] you can see the resource view, which makes clear the relationship
between physical resources, the interrupts and the VM
On [5] you can see the statistics view on a specific physical CPU and on
the whole trace.
And finally on [6] you can see an other test with a ping flood.

In the next days, I will send my patches to the official LTTng git.
My next step is to synchronise the traces collected from the host with
the traces collected from the guest (by finding an efficient way to
share the TSC_OFFSET) to have some infos of what is happening during the
time the VM has the control.

The reason I post these screenshots now is that I will be at Linuxcon
next week, and I would really appreciate some feedbacks and ideas for
future improvements from the KVM community.
So if you are interested, contact me directly and if you are there we'll
try to meet.

Thanks,

Julien

[1] http://ju.klipix.org/lttng/lttng-kvm-busy.png
[2] http://ju.klipix.org/lttng/lttng-kvm.png
[3] http://ju.klipix.org/lttng/lttng-kvm-ksmd.png
[4] http://ju.klipix.org/lttng/lttng-kvm-ressource-view.png
[5] http://ju.klipix.org/lttng/lttng-kvm-stats.png
[6] http://ju.klipix.org/lttng/lttng-kvm-ping.png