Re: [PATCH v2] interrupts: allow guest to set and clear MSI-X mask bit

[Joby Poriyath <joby.poriyath@citrix.com>]

On Tue, Aug 06, 2013 at 04:11:56PM +0300, Pasi Kärkkäinen wrote:
> On Tue, Aug 06, 2013 at 01:17:35PM +0300, Pasi Kärkkäinen wrote:
> > On Tue, Aug 06, 2013 at 10:52:58AM +0100, Andrew Cooper wrote:
> > > On 06/08/13 09:29, Jan Beulich wrote:
> > > >>>> On 05.08.13 at 18:03, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> > > >> On 05/08/13 12:49, Jan Beulich wrote:
> > > >>> But a proper solution would of course be preferred. That may involve
> > > >>> notifying Xen of the reset from the PF driver.
> > > >> That is making the assumption that the PF driver is even aware that the
> > > >> reset has occurred.  If the PF driver can get at the reset information,
> > > >> I still cant see Xen specific hooks being accepted upstream.
> > > > You sort of contradict your earlier statement here that "the VF
> > > > requests the PF to reset itself" (where you probably meant "it",
> > > > as the VF is hardly allowed to cause a reset of the PF): Such an
> > > > operation should imo be visible to the PF driver.
> > > >
> > > > And then, even if the reset is being done, shouldn't the interrupt
> > > > setup occur _after_ the reset? Which would make Xen clear the
> > > > flag...
> > > 
> > > The observed behaviour is this:  On startup, the VF driver issues this
> > > backchannel reset of itself (the VF) which, amongst other things, sets
> > > the mask bit.  It leaves the MSI/MSI-X addr and data fields alone, but
> > > on further consideration, relying on this behaviour might not be a good
> > > idea.  The VF is then expected to re-enable the interrupt at its
> > > convenience.
> > > 
> > > There are no obvious hooks in the PF driver to receive notification of
> > > these backchannel resets.
> > > 
> > 
> > Doesn't the PF driver log some entries to dom0 kernel dmesg when the VF in domU resets itself? 
> > 
> 
> At least I've seen such messages from the Intel igxbe PF driver.. or did I misunderstand something? 
> 

Looking at the ixgbevf code in kernel (tip), reset function is
ixgbevf_reset_hw_vf.

The reset happens with

      IXGBE_WRITE_REG(hw, IXGBE_VFCTRL, IXGBE_CTRL_RST);

This will in turn mask the MSI-X vector. The controller does this automagically.

<snip> from intel 82599 controller spec
MSI–X Vector Control — MSIXTVCTRL (BAR3: 0xC + 0x10*n, n=0...255; RW)

Bits Default Type
0    1b      RW 

Description    
-----------
Mask Bit. 

When this bit is set, the function is prohibited from sending a message using this
MSI-X table entry. However, any other MSI-X table entries programmed with the same 
vector are still capable of sending an equivalent message unless they are also 
masked. This bit’s state after reset is 1b (entry is masked).

This bit is read/write.

<snip>

After the reset, VF sends a message to PF informing PF about the reset. 

Guest was allowed to modify the MSI-X control bit to reduce the load on dom0.
Otherwise this would have to be emulated by qemu-dm. I found a paper 
which explains this in detail (http://zhenxiao.com/read/SR-IOV.pdf).


> -- Pasi
> 

Joby

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel