Re: vtime accounting

["Radim Krčmář" <rkrcmar@redhat.com>]

2017-03-08 02:57-0800, Christoffer Dall:
> Hi Paolo,
> 
> I'm looking at improving KVM/ARM a bit by calling guest_exit_irqoff
> before enabling interrupts when coming back from the guest.
> 
> Unfortunately, this appears to mess up my view of CPU usage using
> something like htop on the host, because it appears all time is spent
> inside the kernel.
> 
> From my analysis, I think this is because we never handle any interrupts
> before enabling interrupts, where the x86 code does its
> handle_external_intr, and the result on ARM is that we never increment
> jiffies before doing the vtime accounting.

(Hm, the counting might be broken on nohz_full then.)

> So my current idea is to increment jiffies according to the clocksource
> before calling guest_exit_irqoff, but this would require some main
> clocksource infrastructure changes.

This seems similar to calling the function from the timer interrupt.
The timer interrupt would be delivered after that and only wasted time,
so it might actually be slower than just delivering it before ...

How expensive is the interrupt enable/disable cycle that this
optimization saves?

> My question is: how important is the vtime accounting on the host from
> your point of view?

No idea.  I'd keep the same behavior on all architectures, though.

The precision of accounting is in jiffies (millions of cycles), so we
could maybe move it from the hot path to vcpu_load/put(?) without
affecting the count in usual cases ...

>                      Worth poking the timekeeping folks about or even
> trying to convince ourselves that the handle_external_intr thing is
> worth it?

handle_external_intr() needs some hardware support in order to be more
than worthless 'local_irq_enable(); local_irq_disable()' ...

e.g. VMX doesn't queue the interrupt that caused a VM exit in the
interrupt controller.  (VMX's handle_external_intr() doesn't deliver any
other interrupts than might have become pending after the exit, but this
race is not very important due to accounting granularity ...)
Alternatively, the interrupt controller would need to support dequeing
without delivery (but delivering in software might still be slower than
cyclying interrupts on and off).

Thanks.