Re: [PATCH 2/2] x86, apicv: Add Posted Interrupt supporting

[Gleb Natapov <gleb@redhat.com>]

On Thu, Jan 31, 2013 at 11:44:43AM -0200, Marcelo Tosatti wrote:
> On Thu, Jan 31, 2013 at 03:38:37PM +0200, Gleb Natapov wrote:
> > On Thu, Jan 31, 2013 at 11:32:45AM -0200, Marcelo Tosatti wrote:
> > > On Thu, Jan 31, 2013 at 11:43:48AM +0200, Gleb Natapov wrote:
> > > > On Wed, Jan 30, 2013 at 09:03:11PM -0200, Marcelo Tosatti wrote:
> > > > > Posted interrupt patch:
> > > > > 2) Must move IN_GUEST_MODE assignment after local_irq_disable, in
> > > > > vcpu_enter_guest function. Otherwise:
> > > > > 
> > > > > cpu0			vcpu1<->cpu1
> > > > > 
> > > > > 			vcpu->mode = IN_GUEST_MODE
> > > > > 
> > > > > if IN_GUEST_MODE == true
> > > > > 	send IPI
> > > > > 			local_irq_disable
> > > > > 
> > > > > PIR not transferred to VIRR, misses interrupt.
> > > > > 
> > > > cpu0 will set KVM_REQ_EVENT, so vmentry will be aborted after
> > > > local_irq_disable() by ->requests check.
> > > 
> > > Yes, but you don't want KVM_REQ_EVENT+kick. It defeats the purpose 
> > > of posted interrupts. You want
> > > 
> > > if vcpu in guest mode
> > > 	send posted interrupt IPI
> > > else
> > > 	KVM_REQ_EVENT+kick
> > > 
> > I am thinking:
> > 
> >  set KVM_REQ_EVENT
> >  if pi is enabled
> >      send posted interrupt IPI
> >  else
> >      kick
> 
> KVM_REQ_EVENT must be after sending posted interrupt IPI. Otherwise on
> the vcpu entry side
> 
> test_and_clear(KVM_REQ_EVENT) {
> 	No bits set in PIR
>    }
> 
It should be after updating PIR, but before sending posted interrupt
IPI. Otherwise:

 cpu0                             cpu1/vcpu

                                  KVM_REQ_EVENT is not set
set pir                         
send IPI
                                  irq_disable()
                                  ->request is empty.
set KVM_REQ_EVENT

That's the same sequence as with IRR update, KVM_REQ_EVENT and kick
today.

> What about item 4 below?
> 
That's for Yang to answer :)

> > > > > 3) Must check outstanding PIR notification bit unconditionally on
> > > > > every VM-entry, because:
> > > > > 
> > > > > 	1. local_irq_disable
> > > > > 	2. vcpu->mode = IN_GUEST_MODE
> > > > > 	3. vmenter
> > > > > 	4. vmexit
> > > > > 	5. vcpu->mode = OUTSIDE_GUEST_MODE
> > > > > 
> > > > > If PIR-IPI-interrupt is sent between an event which triggers VM-exit
> > > > > (for example, an external interrupt due to a device), and step 5
> > > > > (assignment of vcpu->mode), the PIR->VIRR transfer before vmentry must
> > > > > be made.
> > > > Not sure I understand, but I think KVM_REQ_EVENT will cover that too.
> > > 
> > > See above.
> > > 
> > > > > 
> > > > > 4) Today, an interrupt notification is cached on IRR until its delivered - further
> > > > > interrupt injection is not generating further interrupt notification
> > > > > bits. With PIR, behaviour changes: Its possible to have one bit in PIR and another 
> > > > > on IRR APIC page (if timing is right). Is this harmless? Why?
> > > > > 
> > > > > 
> > > > 
> > > > --
> > > > 			Gleb.
> > 
> > --
> > 			Gleb.

--
			Gleb.