Fixing nested SVM event injection

Fixing nested SVM event injection

[Jan Kiszka]

Hi all,

once again, I'm debugging KVM with Jailhouse workload, but this time on
AMD. I'm starting to believe that there are two issues, and I could use
some hints on how things are supposed to work (forgot too much since I
last debugged in this area).

One issue I almost understood and currently hacked away: Spurious
singlestep injection into L2 although L1 will take an NMI vmexit anyway.
I've currently addressed it by testing for is_guest_mode &&
nested.exit_required in enable_nmi_window. Now the spurious #DB events
for L2 are gone. But I suspect there is more to fix (singlestep
shouldn't be enabled if GIF=0).

But now I'm loosing interrupts for L2 (keep in mind: under Jailhouse, L1
takes no interrupts, only NMIs, L2 will get them all). The pattern is
that an interrupt is received and accepted over L1, but I'm not seeing
it being injected into L2 after vmrun. What should ensure the injection
when switching to guest mode?

Right now, I'm not even finding this for VMX anymore, but I know it's
working reliably there. Is the interrupt window on VMX opened over L1
and simply kept during guest transition? Then we probably lack that with
SVM, and I would have a trace.

Thanks,
Jan

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

Re: Fixing nested SVM event injection

[Paolo Bonzini]

On 23/05/2016 09:38, Jan Kiszka wrote:
> But now I'm loosing interrupts for L2 (keep in mind: under Jailhouse, L1
> takes no interrupts, only NMIs, L2 will get them all). The pattern is
> that an interrupt is received and accepted over L1, but I'm not seeing
> it being injected into L2 after vmrun. What should ensure the injection
> when switching to guest mode?
> 
> Right now, I'm not even finding this for VMX anymore, but I know it's
> working reliably there. Is the interrupt window on VMX opened over L1
> and simply kept during guest transition?

Is this check_nested_events?  After vmlaunch/vmresume,
check_nested_events returns -EBUSY, an immediate exit is requested,
nested_run_pending causes vmx_vcpu_run to do

        if (vmx->nested.nested_run_pending)
                kvm_make_request(KVM_REQ_EVENT, vcpu);

and on the next entry inject_pending_event sets in the
CPU_BASED_INTR_PENDING in vmcs02?

SVM doesn't have the equivalent of nested_run_pending, nor does it
implement check_nested_events.

Paolo

> Then we probably lack that with
> SVM, and I would have a trace.

Re: Fixing nested SVM event injection

[Joerg Roedel]

On Mon, May 23, 2016 at 09:38:42AM +0200, Jan Kiszka wrote:
> But now I'm loosing interrupts for L2 (keep in mind: under Jailhouse, L1
> takes no interrupts, only NMIs, L2 will get them all). The pattern is
> that an interrupt is received and accepted over L1, but I'm not seeing
> it being injected into L2 after vmrun. What should ensure the injection
> when switching to guest mode?

Does Jailhouse forward the IRQs from L1 to L2 or will they just
pass-through? The nested-svm code checks whether the L2 is supposed to
get the IRQs directly, and injects them if so. At least that is how it
should work, there might be bugs somewhere...



	Joerg

Re: Fixing nested SVM event injection

[Jan Kiszka]

On 2016-06-13 11:20, Joerg Roedel wrote:
> On Mon, May 23, 2016 at 09:38:42AM +0200, Jan Kiszka wrote:
>> But now I'm loosing interrupts for L2 (keep in mind: under Jailhouse, L1
>> takes no interrupts, only NMIs, L2 will get them all). The pattern is
>> that an interrupt is received and accepted over L1, but I'm not seeing
>> it being injected into L2 after vmrun. What should ensure the injection
>> when switching to guest mode?
> 
> Does Jailhouse forward the IRQs from L1 to L2 or will they just
> pass-through? The nested-svm code checks whether the L2 is supposed to
> get the IRQs directly, and injects them if so. At least that is how it
> should work, there might be bugs somewhere...
> 

Jailhouse doesn't take any interrupts (just NMIs) and instead allows
them to be injected while the guest runs.

I've started to port the logic we developed and stabilized on Intel for
injecting and cancelling events around L1<->L2 transitions (e.g.
check_nested_events callback). The result was some progress but still
also some lost events. Need to debug this further.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux