Asynchronous interruption of a compute-intensive guest

Asynchronous interruption of a compute-intensive guest

[Tommaso Cucinotta]

Hi,

I'd like to "intercept" from the host the exact times at which an 
incoming network packet directed to a guest VM:
a) is delivered from the host OS to the KVM process;
b) is delivered to the "CPU thread" of the KVM process.

Specifically, I don't have a clean idea of how b) happens when the CPU 
thread is doing compute-intensive activities within the VM. How is the 
flow of control of such thread asynchronously interrupted so as to hand 
over control to the proper network driver in kvm ? Any pointer to the 
exact points to look at, in the KVM code, are also very well appreciated.

Just in case you're interested, we need this for playing with some 
custom CPU scheduling policy we have at our lab, for the purpose of 
facing with the problem of responsiveness of VMs while doing 
CPU-intensive activities (i.e., a mixed batch/interactive guest).

Thanks very much for your kind help.

Regards,

     T.

-- 
Tommaso Cucinotta, Computer Engineering PhD, Researcher
ReTiS Lab, Scuola Superiore Sant'Anna, Pisa, Italy
Tel +39 050 882 024, Fax +39 050 882 003
http://retis.sssup.it/people/tommaso

Re: Asynchronous interruption of a compute-intensive guest

[Stefan Hajnoczi]

On Wed, Apr 13, 2011 at 1:09 AM, Tommaso Cucinotta
<tommaso.cucinotta@sssup.it> wrote:
> I'd like to "intercept" from the host the exact times at which an incoming
> network packet directed to a guest VM:
> a) is delivered from the host OS to the KVM process;
> b) is delivered to the "CPU thread" of the KVM process.
>
> Specifically, I don't have a clean idea of how b) happens when the CPU
> thread is doing compute-intensive activities within the VM. How is the flow
> of control of such thread asynchronously interrupted so as to hand over
> control to the proper network driver in kvm ? Any pointer to the exact
> points to look at, in the KVM code, are also very well appreciated.

If you are using userspace virtio-net (not in-kernel vhost-net), then
an incoming ("rx") packet results in the qemu-kvm iothread's select(2)
system call returning with a readable tap file descriptor:
vl.c:main_loop_wait()

(During this time the vcpu thread may still be executing guest code.)

The iothread runs the tap receive function:
net/tap.c:tap_send()

The iothread places the received packet into the rx virtqueue and
interrupts the guest:
hw/virtio-net.c:virtio_net_receive()
hw/virtio-pci.c:virtio_pci_notify()

The interrupt is injected by the KVM kernel module:
arch/x86/kvm/x86.c:kvm_arch_vm_ioctl() KVM_IRQ_LINE

There is some guest mode exiting logic here to kick the vcpu:
arch/x86/kvm/lapic.c:__apic_accept_irq()

During this whole time the vcpu may be executing guest code.  Only at
the very end has the interrupt been inject and the vcpu notified.

Stefan