SMP CPU affinity questions

SMP CPU affinity questions

[listmonkey]

Hi-

I am trying to use a quad Opteron motherboard with SMP Kernel 2.6.5 for a quasi-real-time task.
I need to assign all processes to specific CPUs, including interrupt handlers.
I have had success using sched_setaffinity() to set the CPU for processes I create, but I am unable,
as root, to force system processes to move to another CPU.  Any ideas?

I can find no documentation about how to force an interrupt handler to a specific CPU - is this
possible without modifying the kernel?


--Pete

Re: SMP CPU affinity questions

[Jeffrey V. Merkey]

listmonkey@neo.relay-host.net wrote:

>Hi-
>
>I am trying to use a quad Opteron motherboard with SMP Kernel 2.6.5 for a quasi-real-time task.
>I need to assign all processes to specific CPUs, including interrupt handlers.
>I have had success using sched_setaffinity() to set the CPU for processes I create, but I am unable,
>as root, to force system processes to move to another CPU.  Any ideas?
>
>I can find no documentation about how to force an interrupt handler to a specific CPU - is this
>possible without modifying the kernel?
>
>  
>


IOApic's support binding of interrupt delivery in intel based platforms, 
but I am unaware of tools which force this
setting by default on Linux, but someone else may be able to point you 
in that direction.  Most folks code APIC ICC delivery to
AV_LOPRI (meaning lowest priority processor gets next interrupt).   This 
is advantageous for cache coherency since the IRQ code
is probaby still in that processors cache.  You may have to modify the 
kernel.  Linux doesn't allow processors to be shutdown and
reactiviated real time, it just starts them and lets them run, so you 
don;t have to worry about the case of migrating interrupts
off pinned APICs.  The APIC supports what you are asking for, but I am 
not certain anyone implemented anything other
than AV_LOPRI settings by default in the IO APIC code.  I would suggest 
you look over the IO APIC Code -- this is a lot
of work, BTW.

Jeff

>--Pete
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>  
>

Re: SMP CPU affinity questions

[Zwane Mwaikambo]

On Wed, 2 Nov 2005, Jeffrey V. Merkey wrote:

> IOApic's support binding of interrupt delivery in intel based platforms, but I
> am unaware of tools which force this
> setting by default on Linux, but someone else may be able to point you in that
> direction.  Most folks code APIC ICC delivery to
> AV_LOPRI (meaning lowest priority processor gets next interrupt).   This is
> advantageous for cache coherency since the IRQ code
> is probaby still in that processors cache.  You may have to modify the kernel.
> Linux doesn't allow processors to be shutdown and
> reactiviated real time, it just starts them and lets them run, so you don;t
> have to worry about the case of migrating interrupts
> off pinned APICs.  The APIC supports what you are asking for, but I am not
> certain anyone implemented anything other
> than AV_LOPRI settings by default in the IO APIC code.  I would suggest you
> look over the IO APIC Code -- this is a lot
> of work, BTW.

Or you could just put something which writes to 
/proc/irq/$IRQ/smp_affinity in your initscripts.

Cheers,
	Zwane

Re: SMP CPU affinity questions

[Arjan van de Ven]

On Wed, 2005-11-02 at 09:39 -0700, Jeffrey V. Merkey wrote:
> listmonkey@neo.relay-host.net wrote:
> 
> >Hi-
> >
> >I am trying to use a quad Opteron motherboard with SMP Kernel 2.6.5 for a quasi-real-time task.
> >I need to assign all processes to specific CPUs, including interrupt handlers.
> >I have had success using sched_setaffinity() to set the CPU for processes I create, but I am unable,
> >as root, to force system processes to move to another CPU.  Any ideas?
> >
> >I can find no documentation about how to force an interrupt handler to a specific CPU - is this
> >possible without modifying the kernel?
> >
> >  
> >
> 
> 
> IOApic's support binding of interrupt delivery in intel based platforms, 
> but I am unaware of tools which force this
> setting by default on Linux, but someone else may be able to point you 
> in that direction.  

http://people.redhat.com/arjanv/irqbalance

Re: SMP CPU affinity questions

[Lee Revell]

On Wed, 2005-11-02 at 17:50 +0000, listmonkey@neo.relay-host.net wrote:
> Hi-
> 
> I am trying to use a quad Opteron motherboard with SMP Kernel 2.6.5 for a quasi-real-time task.

2.6.5 is a terrible choice for soft realtime applications, the serious
work on making 2.6 a viable soft RT platform didn't even start until
around 2.6.8.  Try 2.6.12 or newer (ideally 2.6.14).

Lee