Re: How to do fast accesses to LAPIC TPR under kvm?

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2012-10-17 21:24, Stefan Fritsch wrote:
> Hi,
> 
> OpenBSD/i386 seems to be one of the few operating systems that still 
> uses the LAPIC taks priority register for interrupt handling. On AMD 

Yeah, only very special OSes do this... ;)

> CPUs and on older Intel CPUs without the flexpriority feature, this 
> causes a huge performance impact on kvm. I have seen slowdown by a 
> factor of 10.
> 
> Is there a way to use the TPR under kvm without the slowdown? There 
> are some MSRs inherited from Hyper-V, but using these does not make 
> that much difference. I think this is because they still cause an 
> vmexit for every TPR access. I expect the the same is true for x2apic 
> emulation, isn't it?

Didn't study the HyperV interface yet, but the trick is indeed to avoid
as many vmexits as possible, specifically when lowering the TPR value
has no effect as no interrupts are pending.

> 
> There is also the kvmvapic, but kvm does not expose a sane interface 
> to it and only uses it for Windows XP specific binary patching.

The kvmvapic is not a classic paravirtual interface in that it does not
really require guest OS awareness. But it requires the guest to accept
being patched. That's the case for certain Windows versions. Also, the
option ROMs, including our kvmvapic "ROM", have to be mapped at fixed,
accessible addresses to allow jumping to it from a patched TPR instructions.

Therefore, we limited the patching to known OS versions, avoiding to
mess around with other, untested OSes. However, it may be possible to
accept OpenBSD as well by adjusting the tests in kvmvapic and possibly
adjusting some other details.

> 
> Another possibility is TPR access via CR8 on AMD, but the missing 
> cr8_legacy CPUID bit and this discussion [1] make me believe that this 
> is not supported under kvm, at least in 32bit mode. Could this be 
> easily fixed? If yes, would it solve the performance problems, i.e. 
> offer performance comparable to Intel's flexpriority feature?

Everything that unconditionally traps, and so do CR8 accesses, does not
help.

> 
> OpenBSD seems to be reluctant to stop using the TPR. In fact, in a 
> recent discussion, there has been a suggestion that OpenBSD should 
> switch to using TPR also on OpenBSD/amd64 to solve some problems with 
> boot interrupts. How do you expect this would affect performance under 
> kvm (if using CR8)?
> 
> Or do you have any other suggestions? One could also modify kvm to 
> expose a real interface to kvmvapic, e.g. allow the guest OS to 
> provide the virtual address of the option rom and the offset of the 
> CPU number in the %fs segment, instead of using hard coded values for 
> Windows XP.

Of course, though all we need is a stable address in fact. See
vapic_write() for the existing PV interface (between option ROM and
hypervisor so far). We can extend it as long as it is compatible with
the existing one.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux