Re: Preventing IPI sending races in arch code

[Benjamin Herrenschmidt <benh@kernel.crashing.org>]

On Tue, 2013-11-26 at 10:17 +0530, Vineet Gupta wrote:
> On 11/26/2013 01:21 AM, Benjamin Herrenschmidt wrote:
> > On Mon, 2013-11-25 at 13:35 +0000, Vineet Gupta wrote:
> > 
> >> Before reading ur email I was coding something like below:
> >>
> >> void arch_send_ipi(int cpu, int type)
> >> {
> >>   u32 *pending_ptr = per_cpu_ptr(ipi_bits, cpu);
> >>
> >>   while (cmpxchg(pending_ptr, 0, 1 << type) != 0)
> >> 	cpu_relax();
> >>
> >>   raise_ipi(cpu);
> >> }
> > 
> > So you would have blocked the sender while there was already
> > a pending IPI on the target ? Why ?
> 
> A simplistic (but non optimal) way to cater to the race where 2 senders try to
> send the exact same msg to same receiver. Upon first IPI, receiver "consumes" the
> msg (using xchg with 0) so the 2nd IPI seems "empty" i.e. no msg.

But there is no race...

> > The optimization proposed by Peter is actually the only interesting
> > change here, without it the existing set_bit was perfectly fine.
> 
> I'm not sure, see below.
> 
> > Remember that set_bit is atomic.
> 
> Right, but the issue per-se is not clobbering of msg holder, but from POV of
> receiver, seeming coalescing of 2 set_bit writes to msg holder.

That's fine. There's no expectation that N ipi_send_msg turn into N
messages received... it turns into at least one.

Just like MSIs or other edge interrupts

Cheers,
Ben.

> core0		core1		core2
> 
> set_bit 1	
> kick IPI-2	set_bit 1	IPI-0 received
> 		kick IPI-2	read+clear bit
> 				IPI-1 received
> 				no msg
> 
> -Vineet
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/