Re: [PATCH] irq-gic: select all CPU's selected in interrupt affinity settings - Russell King

From: Russell King - ARM Linux admin <linux@armlinux.org.uk>
To: Leonid Movshovich <event.riga@gmail.com>
Cc: Marc Zyngier <maz@kernel.org>,
	Robin Murphy <robin.murphy@arm.com>,
	linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH] irq-gic: select all CPU's selected in interrupt affinity settings
Date: Wed, 20 Nov 2019 17:14:27 +0000	[thread overview]
Message-ID: <20191120171427.GT25745@shell.armlinux.org.uk> (raw)
In-Reply-To: <CAPaFbavWjCJKjUN6nA8Gc4urAMzLt-YVB4ED5DVarenrvMgnvQ@mail.gmail.com>

On Wed, Nov 20, 2019 at 03:28:31PM +0000, Leonid Movshovich wrote:
> On Wed, 20 Nov 2019 at 15:04, Marc Zyngier <maz@kernel.org> wrote:
> >
> > On 2019-11-20 01:15, Robin Murphy wrote:
> > > On 2019-11-20 12:24 am, Leonid Movshovich wrote:
> > >> On Tue, 19 Nov 2019 at 23:36, Russell King - ARM Linux admin
> > >> <linux@armlinux.org.uk> wrote:
> > >>>
> > >>> On Tue, Nov 19, 2019 at 11:12:26PM +0000, event wrote:
> > >>>> So far only a CPU selected with top affinity bit was selected.
> > >>>> This
> > >>>> resulted in all interrupts
> > >>>> being processed by CPU0 by default despite "FF" default affinity
> > >>>> setting for all interrupts
> > >>>
> > >>> Have you checked whether this causes _ALL_ CPUs in the mask to be
> > >>> delivered a single interrupt, thereby causing _ALL_ CPUs to be
> > >>> slowed down and hit the same locks at the same time.
> > >>>
> > >> Yes, I've checked this. No, interrupt is delivered to only one CPU.
> > >> Also ARM GIC architecture specification specifically states in
> > >> chapter
> > >> 3.1.1 that hardware interrupts are delivered to a single CPU in
> > >> multiprocessor system ("1-N model").
> > >
> > > But see also section 3.2.3 - just because only one CPU actually runs
> > > the given ISR doesn't necessarily guarantee that the others *weren't*
> > > interrupted. I'd also hesitate to make any assumptions that all GIC
> > > implementations behave exactly the same way.
> >
> > What happens is that *all* CPUs are being sent the interrupt, and there
> > is some logic in the GIC that ensures that only one sees it (the first
> > one to read the IAR register). All the other see a spurious (1023)
> > interrupt, and have wasted some precious cycles in doing so.
> 
> Cycles are only precious when system is under high load. Under high
> load, to achieve fair spread of interrupts between CPUs one would need
> a userspace app (irqbalance) to sit there and constantly rebalance
> smp_affinity based on /proc/interrupts. Hard to believe such an
> approach wastes less cycles.

So you have no idea how irqbalance works...

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel