cpus_allowed

cpus_allowed

[Rolf Fokkens]

Hi!

I was curious about "CPU affinity" in linux. I found some patches which add 
affinity in task_struct but later I found out that "cpus_allowed" in 
task_struct almost does the same thing.

It resulted in some new curiosity: where's cpus_allowed initialized? I can 
only find an assignment to cpus_allowed for softirq's but no initialization 
for other processes. I assume the correct init value would be "0xffffffff" or 
-1. Can't find it though.

I'm sure I'm overlooking something, but that doesn't help me finding the 
answer. So would someone be so kind to enlighten me?

Thanks,

Rolf

Re: cpus_allowed

[Robert Love]

On Sat, 2001-10-13 at 19:59, Rolf Fokkens wrote:
> I was curious about "CPU affinity" in linux. I found some patches which add 
> affinity in task_struct but later I found out that "cpus_allowed" in 
> task_struct almost does the same thing.
> 
> It resulted in some new curiosity: where's cpus_allowed initialized? I can 
> only find an assignment to cpus_allowed for softirq's but no initialization 
> for other processes. I assume the correct init value would be "0xffffffff" or 
> -1. Can't find it though.
> 
> I'm sure I'm overlooking something, but that doesn't help me finding the 
> answer. So would someone be so kind to enlighten me?

It is initialized to -1 (0xffffffff) by struct definition at
linux/kernel/sched.h.  Since it is a mask, this means all CPUs
(obviously).

It isn't used much.  The softirq code uses it to try to keep some tasks
on one CPU, for performance reasons.  The Tux in-kernel httpd uses it
for performance reasons.  Some patches use it, too.

Most of the CPU affinity patches you see were written before
cpus_allowed.  They go through all sorts of trouble to do what the OS
now does on its own.  If you want to change CPU affinity then you just
need a patch that adds a syscall or proc interface for setting the
cpus_allowed mask.

I wrote a proc interface but for the life of me I can't find it.  If you
can I would be happy to update it to your kernel.  It isn't very old,
anyhow.

Andrew Morton wrote has a similar patch that is quite good.  Get it at
http://www.uow.edu.au/~andrewm/linux/#cpus_allowed

	Robert Love

Re: cpus_allowed

[Tim Hockin]

> Most of the CPU affinity patches you see were written before
> cpus_allowed.  They go through all sorts of trouble to do what the OS
> now does on its own.  If you want to change CPU affinity then you just
> need a patch that adds a syscall or proc interface for setting the
> cpus_allowed mask.

I'm still porting pset to 2.4.  It is more robust than cpus_allowed and
does more.  My plan is to do away with cpus_allowed altogether, and just
provide a compat with pset, but if enough stuff uses cpus_allowed, perhaps
I'll have to leave it..

http://www.hockin.org/~thockin/pset

2.4.x patch not up yet

Re: cpus_allowed

[Rolf Fokkens]

On Saturday 13 October 2001 09:22, Robert Love wrote:
> On Sat, 2001-10-13 at 19:59, Rolf Fokkens wrote:
> > I'm sure I'm overlooking something, but that doesn't help me finding the
> > answer. So would someone be so kind to enlighten me?
>
> It is initialized to -1 (0xffffffff) by struct definition at
> linux/kernel/sched.h.  Since it is a mask, this means all CPUs
> (obviously).

Silly me, I didn't check the header files.

> Most of the CPU affinity patches you see were written before
> cpus_allowed.  They go through all sorts of trouble to do what the OS
> now does on its own.  If you want to change CPU affinity then you just
> need a patch that adds a syscall or proc interface for setting the
> cpus_allowed mask.

Just read the "Linux Scalability Effort" (LSE) on Sourceforge. The concept of 
limitiming applications to certain resources is appealing to me. With the use 
of cpus_allowed it's not to hard to restrict CPU power for some applications. 
I'm thinking of the following: we're running about 12 (!) Oracle databases on 
1 Linux Server with 4 CPU's. One of the databases is a datawarehouse database 
which handles all kinds of heavy queries. Assuming that the machine has 
enough memory (which is the case) restricting this database to certain CPU's 
could be very useful.

I have no idea however what the right API would be. LSE suggests a ulimit 
like setting. In this Oracle example one listener handles connections to all 
databases on the machine, it does so by forking and executing the database 
binary with some specific environment settings per database. So ulimit won't 
handle it. The solution might be to run 2 listeners, one with 
restricted cpus_allowed and the other one with unrestricted cpus_allowed.

Re: cpus_allowed

[Dave Jones]

On Sat, 13 Oct 2001, Rolf Fokkens wrote:

> It resulted in some new curiosity: where's cpus_allowed initialized? I can
> only find an assignment to cpus_allowed for softirq's but no initialization
> for other processes. I assume the correct init value would be "0xffffffff" or
> -1. Can't find it though.

include/linux/sched.h:    cpus_allowed: -1,

regards,

Dave.

-- 
| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs