Re: Control of IRQ Affinities from Userspace

["bigeasy@linutronix.de" <bigeasy@linutronix.de>]

On 2025-11-05 13:11:29 [+0000], Preclik, Tobias wrote:
> > > so if the IRQ is managed and you have IRQ isolation enabled then it
> > > should exclude the non-isolated CPUs. Would that work?
> 
> For that code path to be taken we would have to specify isolcpus and
> managed_irq on the kernel command-line. However, we already migrated
> away from the deprecated isolcpus parameter. Additionally, we need to
> dynamically change the interrupt affinities while in operation. For
> that matter we have to rely on setting the interrupt affinities in
> procfs.

I would be careful with the deprecated term here. The functionality is
not deprecated just the interface is. The CPU affinity has been migrated
a cgroup based interface. If the matching irq affinity is missing then
it should be added rather than avoiding the whole "affinity is managed"
interface since this looks as it has been meant for your use case.

…

> Thanks for sharing the details Florian. The discovery of newly
> registered/requested IRQs in userland is indeed an additional problem.
> Still I would say that we can work around it by polling procfs and
> setting the default interrupt affinity appropriately (given that it is
> not ignored by the drivers of course). RT applications might have to
> accept an initial delay until the interrupts on their I/O path are
> detected and properly tuned.

This depends how you sell it. The initial setup of the application/
system might take a small hit until everything is ready and setup for
your workload. I don't know how changing the XDP program fits into this.
Usually I would expect that you setup it once and use it. However if
switching the XDP program makes sense within your real time workload
then maybe the affinity should not be randomly assigned.

> > I tried to modify stmmac and let it evaluate the default affinity
> > while
> > doing the IRQ balancing dance. That turned out to be working at the
> > end,
> > but each line violated several coding/style/abstraction rules. There
> > is
> > no API at driver level to read the current default affinity - or I
> > missed it. I could sent that hack out as RFC if requested. Just let
> > me
> > know.
> 
> When drivers at least respect the default affinity when rebalancing
> interrupts we can avoid interfering with RT applications. However, we
> must also ensure that specific interrupts which are in use by RT
> applications on their I/O path must remain on the application core.
> Meaning the IRQ rebalancing should also respect the interrupt
> affinities set from userspace via /proc/irq/${IRQ_NO}/smp_affinity.

If free_irq() removes all information about the IRQ then it might also
lose the configured smp_affinity. 

> Here are steps to explore the behavior and reproduce the issue.
…
> online cpus ignoring the default interrupt affinity as well as ignoring
> and overwriting the explicitly set interrupt affinities from userspace.

I just compared with igb and here the affinity mask survives. So it is
just this driver that is doing things different. The igb also removes
all interrupts on down. The affinity remains after changing the number
of queues (which changes the number of used interrupts). 

> Tobias

Sebastian