Re: [Qemu-devel] [PATCH][RFC] Split non-TCG bits out of exec.c

[Jamie Lokier <jamie@shareable.org>]

Anthony Liguori wrote:
> Jamie Lokier wrote:
> >Also, an earlier thread pointed out that loops doing a lot of MMIO are
> >_slower_ with KVM than without - this manifested as very slow VGA
> >output for some guests.  Having KVM pass control to TCG for short runs
> >of guest instructions which do MMIO, or other instructions which need
> >to be emulated, would accelerate KVM in this respect.
...
> An MMIO exit to userspace typically costs around 6k cycles.  On the 
> other hand, a TB translation tends to average closer to 300k often times 
> reaching much higher.  This with was with dyngen so TCG may be more or 
> less expensive.
> 
> An in-kernel MMIO exit on the other hand will cost around 3k cycles.
...
> To make up the cost of TCG translation for just one TB, you need to have 
> a tight loop of at least 50 iterations.

Firstly:

That doesn't make sense: why would you do an expensive TCG translation
every time you hit the same code?  After the first encounter, if the
code page hasn't been modified, it should be a TB cache lookup to
already translated code.

I'm guessing the cost of TB cache lookup is much closer to 3k than
300k cycles, maybe even lower...

Secondly:

In these cases, you can use a special fast translation (when it's not
cached) which just copies the instructions 1:1 from the guest, simply
converting the special instructions (MMIO, anything else needing it)
to helper calls.  That's possible because you know the host is ture
architeccompatible with the guest, as it's running KVM.

> If you also consider all the potential locking issues with SMP guests, I 
> think it's pretty likely that there are few cases where dropping to TCG 
> is going to be a net performance win.

VMware claimed otherwise when Intel first brought out CPU support for
virtualisation.

SMP works fine if you map guest instructions 1:1 to host instructions
with helper calls for special cases.  Even atomics, load-locked
sequences and complex weak memory ordering things would behave
correctly.

Oops, I believe I just argued for keeping the TB cache and code
translation but not using TCG :-)

-- Jamie