Re: [RFC PATCH 0/3] generic hypercall support

Avi Kivity
2009-05-08T19:03:44Z

Marcelo Tosatti wrote:
> On Fri, May 08, 2009 at 08:43:40AM -0400, Gregory Haskins wrote:
>
>> The problem is the exit time in of itself isnt all that interesting to
>> me. What I am interested in measuring is how long it takes KVM to
>> process the request and realize that I want to execute function "X".
>> Ultimately that is what matters in terms of execution latency and is
>> thus the more interesting data. I think the exit time is possibly an
>> interesting 5th data point, but its more of a side-bar IMO. In any
>> case, I suspect that both exits will be approximately the same at the
>> VT/SVM level.
>>
>> OTOH: If there is a patch out there to improve KVMs code (say
>> specifically the PIO handling logic), that is fair-game here and we
>> should benchmark it. For instance, if you have ideas on ways to improve
>> the find_pio_dev performance, etc....
>>
>
> <guess mode on>
>
> One easy thing to try is to cache the last successful lookup on a
> pointer, to improve patterns where there's "device locality" (like
> nullio test).
>

We should do that everywhere, memory slots, pio slots, etc. Or even
keep statistics on accesses and sort by that.

> <guess mode off>
>
>

I'd leave it on if I were you.

>> One item may be to replace the kvm->lock on the bus scan with an RCU
>> or something.... (though PIOs are very frequent and the constant
>> re-entry to an an RCU read-side CS may effectively cause a perpetual
>> grace-period and may be too prohibitive). CC'ing pmck.
>>
>
> Yes, locking improvements are needed there badly (think for eg the cache
> bouncing of kvm->lock _and_ bouncing of kvm->slots_lock on 4-way SMP
> guests).
>

There's no reason for kvm->lock on pio. We should push the locking to
devices.

I'm going to rename slots_lock as
slots_lock_please_reimplement_me_using_rcu, this keeps coming up.

>> FWIW: the PIOoHCs were about 140ns slower than pure HC, so some of that
>> 140 can possibly be recouped. I currently suspect the lock acquisition
>> in the iobus-scan is the bulk of that time, but that is admittedly a
>> guess. The remaining 200-250ns is elsewhere in the PIO decode.
>>
>
> vmcs_read is significantly expensive
> (http://www.mail-archive.com/kvm@vger.kernel.org/msg00840.html,
> likely that my measurements were foobar, Avi mentioned 50 cycles for
> vmcs_write).
>

IIRC vmcs reads are pretty fast, and are being improved.

> See for eg how vmx.c reads VM_EXIT_INTR_INFO twice on every exit.
>

Ugh.

> Also this one looks pretty bad for a 32-bit PAE guest (and you can
> get away with the unconditional GUEST_CR3 read too).
>
> /* Access CR3 don't cause VMExit in paging mode, so we need
> * to sync with guest real CR3. */
> if (enable_ept && is_paging(vcpu)) {
> vcpu->arch.cr3 = vmcs_readl(GUEST_CR3);
> ept_load_pdptrs(vcpu);
> }
>
>

We should use an accessor here just like with registers and segment
registers.

--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.