Re: Memory synchronization vs. interrupt handlers

[Benjamin Herrenschmidt <benh@kernel.crashing.org>]

On Thu, 2013-08-29 at 16:51 -0700, Paul E. McKenney wrote:
> On Wed, Aug 28, 2013 at 01:28:08PM -0700, H. Peter Anvin wrote:
> > On 08/28/2013 12:16 PM, Alan Stern wrote:
> > > Russell, Peter, and Ingo:
> > > 
> > > Can you folks enlighten us regarding this issue for some common 
> > > architectures?
> > 
> > On x86, IRET is a serializing instruction; it guarantees hard
> > serialization of absolutely everything.
> 
> So a second interrupt from this same device could not appear to happen
> before the IRET, no matter what device and/or I/O bus?  Or is IRET
> defined to synchronize all the way out to the whatever device is
> generating the next interrupt?
> 
> > I would expect architectures that have weak memory ordering to put
> > appropriate barriers in the IRQ entry/exit code.

Not sure why you would expect that, there is no reason to do such a
thing.

> Adding a few on CC.  Also restating the question as I understand it:
> 
> 	Suppose that a given device generates an interrupt on CPU 0,
> 	but that before CPU 0's interrupt handler completes, this device
> 	wants to generate a second interrupt on CPU 1.  This can happen
> 	as soon as CPU 0's handler does an EOI or equivalent.

By "interrupt handler" are you talking about the general handling of all
interrupts in the kernel or the device specific interrupt handler ?

Typically the EOI is done after the later has run.

> 	Can CPU 1's interrupt handler assume that all the in-memory effects
> 	of CPU 0's interrupt handler will be visible, even if neither
> 	interrupt handler uses locking or memory barriers?

We don't formally provide such a guarantee no. There is a spin_lock on
the way back from the device handler and before the EOI but not an
unlock so we don't have a full ordering here (see handle_irq_event).

At least on powerpc, the EOI will generally be an MMIO which will
however synchronize everything because we have a sync before the store
in our MMIO accessors, but that might not be true when running under the
pHyp hypervisor, as we do a hypercall there and I don't think that
includes a sync instruction inside the hypervisor.

So I'd say that as-is, no, we don't provide that guarantee.

Cheers,
Ben.

> 
> 							Thanx, Paul
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/