Re: About the performance of hyper-v

[Vitaly Kuznetsov <vkuznets@redhat.com>]

Liang Li <liliang324@gmail.com> writes:

> [resend for missing cc]
>
> Hi Vitaly,
>
> I found a case that the virtualization overhead was almost doubled
> when turning on Hper-v related features compared to that without any
> no hyper-v feature.  It happens when running a 3D game in windows
> guest in qemu kvm environment.
>
> By investigation, I found there are a lot of IPIs triggered by guest,
> when turning on the hyer-v related features including stimer, for the
> apicv is turned off, at least two vm exits are needed for processing a
> single IPI.
>
>
> perf stat will show something like below [recorded for 5 seconds]
>
> ---------
>
> Analyze events for all VMs, all VCPUs:
>              VM-EXIT    Samples  Samples%     Time%    Min Time    Max
> Time         Avg time
>   EXTERNAL_INTERRUPT     471831    59.89%    68.58%      0.64us
> 65.42us      2.34us ( +-   0.11% )
>            MSR_WRITE     238932    30.33%    23.07%      0.48us
> 41.05us      1.56us ( +-   0.14% )
>
> Total Samples:787803, Total events handled time:1611193.84us.
>
> I tried turning off hyper-v for the same workload and repeat the test,
> the overall virtualization overhead reduced by about of 50%:
>
> -------
>
> Analyze events for all VMs, all VCPUs:
>
>              VM-EXIT    Samples  Samples%     Time%    Min Time    Max
> Time         Avg time
>           APIC_WRITE     255152    74.43%    50.72%      0.49us
> 50.01us      1.42us ( +-   0.14% )
>        EPT_MISCONFIG      39967    11.66%    40.58%      1.55us
> 686.05us      7.27us ( +-   0.43% )
>            DR_ACCESS      35003    10.21%     4.64%      0.32us
> 40.03us      0.95us ( +-   0.32% )
>   EXTERNAL_INTERRUPT       6622     1.93%     2.08%      0.70us
> 57.38us      2.25us ( +-   1.42% )
>
> Total Samples:342788, Total events handled time:715695.62us.
>
> For this scenario,  hyper-v works really bad.  stimer works better
> than hpet, but on the other hand, it relies on SynIC which has
> negative effects for IPI intensive workloads.
> Do you have any plans for improvement?
>

Hey,

the above can be caused by the fact that when 'hv-synic' is enabled, KVM
automatically disables APICv and this can explain the overhead and the
fact that you're seeing more vmexits. KVM disables APICv because SynIC's
'AutoEOI' feature is incompatible with it. We can, however, tell Windows
to not use AutoEOI ('Recommend deprecating AutoEOI' bit) and only
inhibit APICv if the recommendation was ignored. This is implemented in
the following KVM patch series:
https://lore.kernel.org/kvm/20210518144339.1987982-1-vkuznets@redhat.com/

It will, however, require a new 'hv-something' flag to QEMU. For now, it
can be tested with 'hv-passthrough'.

It would be great if you could give it a spin!

-- 
Vitaly