genirq regression - CONFIG_AUTO_IRQ_AFFINITY

genirq regression - CONFIG_AUTO_IRQ_AFFINITY

[John Keller]

The addition of the generic affinity autoselector has introduced
a regression in SN2 Altix.

In the past, request_irq()'s call to select_smp_affinity() was
essentially a noop. 

#ifdef CONFIG_AUTO_IRQ_AFFINITY
extern int select_smp_affinity(unsigned int irq);
#else
static inline int select_smp_affinity(unsigned int irq)
{
        return 1;
}


Now changes have created do_irq_select_affinity(), which is
executed by a call to request_irq().

  request_irq()
   __setup_irq()
    do_irq_select_affinity()

do_irq_select_affinity() will call the set_affinity() routine,
with a default mask. On SN2 (and possibly all ia64 platforms),
this will retarget all device interrupts to the same CPU, most
likely CPU0.


#ifndef CONFIG_AUTO_IRQ_AFFINITY
/*
 * Generic version of the affinity autoselector.
 */
int do_irq_select_affinity(unsigned int irq, struct irq_desc *desc)
{
        cpumask_t mask;

        if (!irq_can_set_affinity(irq))
                return 0;

        cpus_and(mask, cpu_online_map, irq_default_affinity);

        /*
         * Preserve an userspace affinity setup, but make sure that
         * one of the targets is online.
         */
        if (desc->status & (IRQ_AFFINITY_SET | IRQ_NO_BALANCING)) {
                if (cpus_intersects(desc->affinity, cpu_online_map))
                        mask = desc->affinity;
                else
                        desc->status &= ~IRQ_AFFINITY_SET;
        }

        desc->affinity = mask;
        desc->chip->set_affinity(irq, mask);

        return 0;
}

This set_affinity call undoes all the performance related interrupt
targetting that the SN2 PROM did at boot.

I'm looking at a patch that makes use of the new IRQ_AFFINITY_SET
flag in the irq_desc to get around this.

Is this an issue for all ia64 platforms? Or does irq balancing
factor in here? IRQ balancing is disabled on SN2.

John Keller

Re: genirq regression - CONFIG_AUTO_IRQ_AFFINITY

[Yinghai Lu]

On Fri, Dec 5, 2008 at 8:40 AM, John Keller <jpk@sgi.com> wrote:
> The addition of the generic affinity autoselector has introduced
> a regression in SN2 Altix.
>
> In the past, request_irq()'s call to select_smp_affinity() was
> essentially a noop.
>
> #ifdef CONFIG_AUTO_IRQ_AFFINITY
> extern int select_smp_affinity(unsigned int irq);
> #else
> static inline int select_smp_affinity(unsigned int irq)
> {
>        return 1;
> }
>
>
> Now changes have created do_irq_select_affinity(), which is
> executed by a call to request_irq().
>
>  request_irq()
>   __setup_irq()
>    do_irq_select_affinity()
>
> do_irq_select_affinity() will call the set_affinity() routine,
> with a default mask. On SN2 (and possibly all ia64 platforms),
> this will retarget all device interrupts to the same CPU, most
> likely CPU0.

> I'm looking at a patch that makes use of the new IRQ_AFFINITY_SET
> flag in the irq_desc to get around this.
>

that bit only be set in irq_set_affinity(),

arch/mips/kernel/cevt-bcm1480.c:        irq_set_affinity(irq,
cpumask_of_cpu(cpu));
arch/mips/kernel/cevt-sb1250.c: irq_set_affinity(irq, cpumask_of_cpu(cpu));
arch/sparc64/kernel/of_device.c:                irq_set_affinity(irq,
numa_mask);
arch/sparc64/kernel/pci_msi.c:          irq_set_affinity(irq, numa_mask);
arch/x86/kernel/hpet.c:                 irq_set_affinity(hdev->irq,
cpumask_of_cpu(hdev->cpu));
arch/x86/kernel/hpet.c: irq_set_affinity(dev->irq, cpumask_of_cpu(dev->cpu));
drivers/xen/events.c:   irq_set_affinity(irq, cpumask_of_cpu(0));

kernel/irq/proc.c:      irq_set_affinity(irq, new_value);

kernel/time/tick-common.c:              irq_set_affinity(newdev->irq, *cpumask);

so you may need to call it in your arch code. like
setup_IO_APIC_irq(),  create_irq()...

YH

Re: genirq regression - CONFIG_AUTO_IRQ_AFFINITY

[John Keller]

> 
> On Fri, Dec 5, 2008 at 8:40 AM, John Keller <jpk@sgi.com> wrote:
> > The addition of the generic affinity autoselector has introduced
> > a regression in SN2 Altix.
> >
> > In the past, request_irq()'s call to select_smp_affinity() was
> > essentially a noop.
> >
> > #ifdef CONFIG_AUTO_IRQ_AFFINITY
> > extern int select_smp_affinity(unsigned int irq);
> > #else
> > static inline int select_smp_affinity(unsigned int irq)
> > {
> >        return 1;
> > }
> >
> >
> > Now changes have created do_irq_select_affinity(), which is
> > executed by a call to request_irq().
> >
> >  request_irq()
> >   __setup_irq()
> >    do_irq_select_affinity()
> >
> > do_irq_select_affinity() will call the set_affinity() routine,
> > with a default mask. On SN2 (and possibly all ia64 platforms),
> > this will retarget all device interrupts to the same CPU, most
> > likely CPU0.
> 
> > I'm looking at a patch that makes use of the new IRQ_AFFINITY_SET
> > flag in the irq_desc to get around this.
> >
> 
> that bit only be set in irq_set_affinity(),
> 
> arch/mips/kernel/cevt-bcm1480.c:        irq_set_affinity(irq,
> cpumask_of_cpu(cpu));
> arch/mips/kernel/cevt-sb1250.c: irq_set_affinity(irq, cpumask_of_cpu(cpu));
> arch/sparc64/kernel/of_device.c:                irq_set_affinity(irq,
> numa_mask);
> arch/sparc64/kernel/pci_msi.c:          irq_set_affinity(irq, numa_mask);
> arch/x86/kernel/hpet.c:                 irq_set_affinity(hdev->irq,
> cpumask_of_cpu(hdev->cpu));
> arch/x86/kernel/hpet.c: irq_set_affinity(dev->irq, cpumask_of_cpu(dev->cpu));
> drivers/xen/events.c:   irq_set_affinity(irq, cpumask_of_cpu(0));
> 
> kernel/irq/proc.c:      irq_set_affinity(irq, new_value);
> 
> kernel/time/tick-common.c:              irq_set_affinity(newdev->irq, *cpumask);
> 
> so you may need to call it in your arch code. like
> setup_IO_APIC_irq(),  create_irq()...
> 
> YH
> 

Yes, I am planning to set it in SN platform specific code,
sn_irq_fixup().

John