From mboxrd@z Thu Jan  1 00:00:00 1970
From: Marcelo Tosatti <mtosatti@redhat.com>
Subject: Re: [PATCH 2/2] x86, apicv: Add Posted Interrupt supporting
Date: Mon, 4 Feb 2013 18:47:30 -0200
Message-ID: <20130204204729.GA16442@amt.cnet>
References: <20130131094348.GN15004@redhat.com>
 <20130131133245.GB3179@amt.cnet>
 <20130131133837.GB23213@redhat.com>
 <20130131134443.GA4419@amt.cnet>
 <20130131135556.GC23213@redhat.com>
 <20130204005700.GA2705@amt.cnet>
 <20130204095553.GK23213@redhat.com>
 <20130204144345.GA11328@amt.cnet>
 <20130204171301.GB10756@redhat.com>
 <20130204195952.GA15856@amt.cnet>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: "Zhang, Yang Z" <yang.z.zhang@intel.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"Shan, Haitao" <haitao.shan@intel.com>,
	"Zhang, Xiantao" <xiantao.zhang@intel.com>,
	"Nakajima, Jun" <jun.nakajima@intel.com>,
	"Anvin, H Peter" <h.peter.anvin@intel.com>
To: Gleb Natapov <gleb@redhat.com>
Return-path: <kvm-owner@vger.kernel.org>
Received: from mx1.redhat.com ([209.132.183.28]:37995 "EHLO mx1.redhat.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1755764Ab3BDWiy (ORCPT <rfc822;kvm@vger.kernel.org>);
	Mon, 4 Feb 2013 17:38:54 -0500
Content-Disposition: inline
In-Reply-To: <20130204195952.GA15856@amt.cnet>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Mon, Feb 04, 2013 at 05:59:52PM -0200, Marcelo Tosatti wrote:
> On Mon, Feb 04, 2013 at 07:13:01PM +0200, Gleb Natapov wrote:
> > On Mon, Feb 04, 2013 at 12:43:45PM -0200, Marcelo Tosatti wrote:
> > > > > Any example how software relies on such two-interrupts-queued-in-IRR/ISR behaviour?
> > > > Don't know about guests, but KVM relies on it to detect interrupt
> > > > coalescing. So if interrupt is set in IRR but not in PIR interrupt will
> > > > not be reported as coalesced, but it will be coalesced during PIR->IRR
> > > > merge.
> > > 
> > > Yes, so:
> > > 
> > > 1. IRR=1, ISR=0, PIR=0. Event: set_irq, coalesced=no.
> > > 2. IRR=0, ISR=1, PIR=0. Event: IRR->ISR transfer.
> > > 3. vcpu outside of guest mode.
> > > 4. IRR=1, ISR=1, PIR=0. Event: set_irq, coalesced=no.
> > > 5. vcpu enters guest mode.
> > > 6. IRR=1, ISR=1, PIR=1. Event: set_irq, coalesced=no.
> > > 7. HW transfers PIR into IRR.
> > > 
> > > set_irq return value at 7 is incorrect, interrupt event was _not_
> > > queued.
> > Not sure I understand the flow of events in your description correctly. As I
> > understand it at 4 set_irq() will return incorrect result. Basically
> > when PIR is set to 1 while IRR has 1 for the vector the value of
> > set_irq() will be incorrect.
> 
> At 4 it has not been coalesced: it has been queued to IRR.
> At 6 it has been coalesced: PIR bit merged into IRR bit.
> 
> > Frankly I do not see how it can be fixed
> > without any race with present HW PIR design.
> 
> At kvm_accept_apic_interrupt, check IRR before setting PIR bit, if IRR
> already set, don't set PIR.

Or:

apic_accept_interrupt() {

1. Read ORIG_PIR=PIR, ORIG_IRR=IRR.
Never set IRR when HWAPIC enabled, even if outside of guest mode.
2. Set PIR and let HW or SW VM-entry transfer it to IRR.
3. set_irq return value: (ORIG_PIR or ORIG_IRR set).
}

Two or more concurrent set_irq can race with each other, though. Can
either document the race or add a lock.