[PATCH] 2.5: task cpu affinity syscalls

[PATCH] 2.5: task cpu affinity syscalls

[Aneesh Kumar K.V]

Hi, 

 Now that we have API that allow a process to say I would like to go to
these  CPU, Are there any API's available that will allow a CPU to say I
will take only these process. ( Resource Affinity domains ? )

 -aneesh 

Re: [PATCH] 2.5: task cpu affinity syscalls

[Robert Love]

On Thu, 2002-04-11 at 10:23, Aneesh Kumar K.V wrote:

>  Now that we have API that allow a process to say I would like to go to
> these  CPU, Are there any API's available that will allow a CPU to say I
> will take only these process. ( Resource Affinity domains ? )

First, the Linux scheduler is not really designed to do this.  It's cpu
affinity works on a per-process basis and in fact needs to explicitly
move CPUs from each CPU.  Second, we could probably do this in userspace
using the exported sched_setaffinity syscall (just loop over all tasks,
setting the affinity as-needed).

There is a problem, though.  In 2.5 with the O(1) scheduler, if the
process is not currently running on an allowed CPU when it is affined,
it must be forced off to a legal CPU via the migration threads.  This is
expensive and complex and _not_ something we want to do to every process
on the system.

For this reason, and because I honestly favor the simple interfaces I
wrote, I think we should stick with just the exported interfaces we
currently have.

This isn't to say we could not do this in userspace - it would not be
hard (but still gross to move mass processes around).  We could also
have a version of init that affines itself on boot, thereby having every
other process likewise affined.  Then explicitly move away those
processes we want elsewhere.  This is cheap and easy.

	Robert Love

[PATCH] 2.5: task cpu affinity syscalls

[Robert Love]

Linus,

Attached is a resync to 2.5.8-pre2 of my CPU affinity syscalls for 2.5's
scheduler.

This patch implements two new syscalls:

	int sched_setaffinity(pid_t pid, unsigned int len,
				unsigned long *new_mask_ptr)

	int sched_getaffinity(pid_t pid, unsigned int *user_len_ptr,
				unsigned long *user_mask_ptr)

which set and get a task's CPU affinity, respectively.  I think we do
indeed need to export an interface for setting affinity for specific
needs, but a simple exported bitmask representation of cpus_allowed is
more than enough and these calls fit into our existing sched_* family.

These syscalls provide forward-compatibility with cpus_allowed changes,
implement security, and use Ingo's existing set_cpus_allowed method.

These syscalls are based on code of Ingo's for 2.4.

Changes since last post:

- drop tasklist_lock before calling set_cpus_allowed.  Holding it across
  the call may not be safe.  Instead, pin the specific task_struct with
  get_task_struct.
- remove ifdef in code
- minor cleanups, resync, et cetera

Well-tested on both UP and SMP.  A test program and a tool for setting
affinity of processes is available at
	ftp://ftp.kernel.org/pub/linux/kernel/people/rml/cpu-affinity
and your favorite mirror.  Please apply.

	Robert Love

diff -urN linux-2.5.8-pre2/arch/i386/kernel/entry.S linux/arch/i386/kernel/entry.S
--- linux-2.5.8-pre2/arch/i386/kernel/entry.S	Sat Apr  6 14:04:16 2002
+++ linux/arch/i386/kernel/entry.S	Sat Apr  6 19:44:29 2002
@@ -717,6 +717,8 @@
 	.long SYMBOL_NAME(sys_tkill)
 	.long SYMBOL_NAME(sys_sendfile64)
 	.long SYMBOL_NAME(sys_futex)		/* 240 */
+	.long SYMBOL_NAME(sys_sched_setaffinity)
+	.long SYMBOL_NAME(sys_sched_getaffinity)
 
 	.rept NR_syscalls-(.-sys_call_table)/4
 		.long SYMBOL_NAME(sys_ni_syscall)
diff -urN linux-2.5.8-pre2/include/asm-i386/unistd.h linux/include/asm-i386/unistd.h
--- linux-2.5.8-pre2/include/asm-i386/unistd.h	Mon Mar 18 15:37:16 2002
+++ linux/include/asm-i386/unistd.h	Sat Apr  6 19:44:29 2002
@@ -245,6 +245,8 @@
 #define __NR_tkill		238
 #define __NR_sendfile64		239
 #define __NR_futex		240
+#define __NR_sched_setaffinity	241
+#define __NR_sched_getaffinity	242
 
 /* user-visible error numbers are in the range -1 - -124: see <asm-i386/errno.h> */
 
diff -urN linux-2.5.8-pre2/kernel/sched.c linux/kernel/sched.c
--- linux-2.5.8-pre2/kernel/sched.c	Sat Apr  6 14:04:16 2002
+++ linux/kernel/sched.c	Sat Apr  6 19:44:29 2002
@@ -1234,6 +1234,103 @@
 	return retval;
 }
 
+/**
+ * sys_sched_setaffinity - set the cpu affinity of a process
+ * @pid: pid of the process
+ * @len: length of new_mask
+ * @new_mask: user-space pointer to the new cpu mask
+ */
+asmlinkage int sys_sched_setaffinity(pid_t pid, unsigned int len,
+				      unsigned long *new_mask_ptr)
+{
+	unsigned long new_mask;
+	task_t *p;
+	int retval;
+
+	if (len < sizeof(new_mask))
+		return -EINVAL;
+
+	if (copy_from_user(&new_mask, new_mask_ptr, sizeof(new_mask)))
+		return -EFAULT;
+
+	new_mask &= cpu_online_map;
+	if (!new_mask)
+		return -EINVAL;
+
+	read_lock(&tasklist_lock);
+
+	p = find_process_by_pid(pid);
+	if (!p) {
+		read_unlock(&tasklist_lock);
+		return -ESRCH;
+	}
+
+	/*
+	 * It is not safe to call set_cpus_allowed with the
+	 * tasklist_lock held.  We will bump the task_struct's
+	 * usage count and then drop tasklist_lock.
+	 */
+	get_task_struct(p);
+	read_unlock(&tasklist_lock);
+
+	retval = -EPERM;
+	if ((current->euid != p->euid) && (current->euid != p->uid) &&
+			!capable(CAP_SYS_NICE))
+		goto out_unlock;
+
+	retval = 0;
+	set_cpus_allowed(p, new_mask);
+
+out_unlock:
+	put_task_struct(p);
+	return retval;
+}
+
+/**
+ * sys_sched_getaffinity - get the cpu affinity of a process
+ * @pid: pid of the process
+ * @user_len_ptr: userspace pointer to the length of the mask
+ * @user_mask_ptr: userspace pointer to the mask
+ */
+asmlinkage int sys_sched_getaffinity(pid_t pid, unsigned int *user_len_ptr,
+				      unsigned long *user_mask_ptr)
+{
+	unsigned long mask;
+	unsigned int len, user_len;
+	task_t *p;
+	int retval;
+
+	len = sizeof(mask);
+
+	if (copy_from_user(&user_len, user_len_ptr, sizeof(user_len)))
+		return -EFAULT;
+
+	/* return to the user the actual size of the bitmask */
+	if (copy_to_user(user_len_ptr, &len, sizeof(len)))
+		return -EFAULT;
+
+	if (user_len < len)
+		return -EINVAL;
+
+	read_lock(&tasklist_lock);
+
+	retval = -ESRCH;
+	p = find_process_by_pid(pid);
+	if (!p)
+		goto out_unlock;
+
+	retval = 0;
+	mask = p->cpus_allowed & cpu_online_map;
+
+out_unlock:
+	read_unlock(&tasklist_lock);
+	if (retval)
+		return retval;
+	if (copy_to_user(user_mask_ptr, &mask, sizeof(mask)))
+		return -EFAULT;
+	return 0;
+}
+
 asmlinkage long sys_sched_yield(void)
 {
 	runqueue_t *rq;

Re: [PATCH] 2.5: task cpu affinity syscalls

[Anton Blanchard]

> Attached is a resync to 2.5.8-pre2 of my CPU affinity syscalls for 2.5's
> scheduler.
> 
> This patch implements two new syscalls:
> 
> 	int sched_setaffinity(pid_t pid, unsigned int len,
> 				unsigned long *new_mask_ptr)
> 
> 	int sched_getaffinity(pid_t pid, unsigned int *user_len_ptr,
> 				unsigned long *user_mask_ptr)

Since this isnt 32/64 bit compatible on big endian machines, can you
write some wrapper functions before all the archs implement it
themselves?

Think of 32 bit userspace passing a big endian bitmask into a 64 bit
kernel:

32 bit:
31-0 63-32 95-64 127-96

64 bit:
63-0 127-64

Anton