[PATCH] optional non-interactive mode for cpu scheduler

[PATCH] optional non-interactive mode for cpu scheduler

[Con Kolivas]

[-- Attachment #1.1: Type: text/plain, Size: 50 bytes --]

optional non-interactive mode for cpu scheduler



[-- Attachment #1.2: sched-optional_non-interactive.diff --]
[-- Type: text/x-patch, Size: 3292 bytes --]

Some environments desire strict control over cpu resources based on nice
values without interactive determination.

Add a sysctl (/proc/sys/kernel/interactive) to disable dynamic priority
behaviour and interactive reinsertion.

Signed-off-by: Con Kolivas <kernel@kolivas.org>


Index: linux-2.6.10-rc1-mm2/include/linux/sched.h
===================================================================
--- linux-2.6.10-rc1-mm2.orig/include/linux/sched.h	2004-11-02 13:20:03.000000000 +1100
+++ linux-2.6.10-rc1-mm2/include/linux/sched.h	2004-11-02 16:03:53.807209246 +1100
@@ -738,6 +738,7 @@ extern int task_prio(const task_t *p);
 extern int task_nice(const task_t *p);
 extern int task_curr(const task_t *p);
 extern int idle_cpu(int cpu);
+extern int sched_interactive;
 
 void yield(void);
 
Index: linux-2.6.10-rc1-mm2/include/linux/sysctl.h
===================================================================
--- linux-2.6.10-rc1-mm2.orig/include/linux/sysctl.h	2004-11-02 13:19:19.000000000 +1100
+++ linux-2.6.10-rc1-mm2/include/linux/sysctl.h	2004-11-02 16:07:46.071023189 +1100
@@ -134,6 +134,7 @@ enum
 	KERN_SPARC_SCONS_PWROFF=64, /* int: serial console power-off halt */
 	KERN_HZ_TIMER=65,	/* int: hz timer on or off */
 	KERN_UNKNOWN_NMI_PANIC=66, /* int: unknown nmi panic flag */
+	KERN_INTERACTIVE=67,	/* int: dynamic priorities */
 };
 
 
Index: linux-2.6.10-rc1-mm2/kernel/sched.c
===================================================================
--- linux-2.6.10-rc1-mm2.orig/kernel/sched.c	2004-11-02 14:52:51.000000000 +1100
+++ linux-2.6.10-rc1-mm2/kernel/sched.c	2004-11-02 16:13:48.419712248 +1100
@@ -150,8 +150,8 @@
 #define DELTA(p) \
 	(SCALE(TASK_NICE(p), 40, MAX_BONUS) + INTERACTIVE_DELTA)
 
-#define TASK_INTERACTIVE(p) \
-	((p)->prio <= (p)->static_prio - DELTA(p))
+#define TASK_INTERACTIVE(p) ((sched_interactive) && \
+	((p)->prio <= (p)->static_prio - DELTA(p)))
 
 #define INTERACTIVE_SLEEP(p) \
 	(JIFFIES_TO_NS(MAX_SLEEP_AVG * \
@@ -602,6 +602,13 @@ static inline void enqueue_task_head(str
 }
 
 /*
+ * This is a sysctl which allows dynamic priorities to be applied based on
+ * interactive behaviour, and selective interactive reinsertion into the
+ * active array despite timeslice expiration.
+ */
+int sched_interactive = 1;
+
+/*
  * effective_prio - return the priority that is based on the static
  * priority but is modified by bonuses/penalties.
  *
@@ -622,6 +629,10 @@ static int effective_prio(task_t *p)
 	if (rt_task(p))
 		return p->prio;
 
+	/* Dynamic priorities are not used with interactive mode off */
+	if (!sched_interactive)
+		return p->static_prio;
+
 	bonus = CURRENT_BONUS(p) - MAX_BONUS / 2;
 
 	prio = p->static_prio - bonus;
Index: linux-2.6.10-rc1-mm2/kernel/sysctl.c
===================================================================
--- linux-2.6.10-rc1-mm2.orig/kernel/sysctl.c	2004-11-02 13:19:19.000000000 +1100
+++ linux-2.6.10-rc1-mm2/kernel/sysctl.c	2004-11-02 16:06:27.621555227 +1100
@@ -624,6 +624,14 @@ static ctl_table kern_table[] = {
 		.proc_handler   = &proc_unknown_nmi_panic,
 	},
 #endif
+	{
+		.ctl_name	= KERN_INTERACTIVE,
+		.procname	= "interactive",
+		.data		= &sched_interactive,
+		.maxlen		= sizeof (int),
+		.mode		= 0644,
+		.proc_handler	= &proc_dointvec,
+	},
 	{ .ctl_name = 0 }
 };
 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: [PATCH] optional non-interactive mode for cpu scheduler

[Ingo Molnar]

* Con Kolivas <kernel@kolivas.org> wrote:

> optional non-interactive mode for cpu scheduler

i think the following scheme would work better:

 - introduce a new SCHED_CPUBOUND policy
 - return ->static_prio + 5 for such tasks
 - keep their timeslice based off ->static_prio

the point is this: such tasks would thus be automatically and
perpetually considered 'CPU hogs'. Applications cannot abuse this
mechanism because they get the maximum 'penalty'.

and as a bonus, no magic sysctl and inherently more flexibility.

(note that this scheme has advantages above nice +5 because nice +5
still has the interactivity stuff on which can create priority
fluctuations and may thus affect workloads.)

if you agree with this scheme, would you be interested in implementing
this?

	Ingo

Re: [PATCH] optional non-interactive mode for cpu scheduler

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1574 bytes --]

Ingo Molnar wrote:
> * Con Kolivas <kernel@kolivas.org> wrote:
> 
> 
>>optional non-interactive mode for cpu scheduler
> 
> 
> i think the following scheme would work better:
> 
>  - introduce a new SCHED_CPUBOUND policy
>  - return ->static_prio + 5 for such tasks
>  - keep their timeslice based off ->static_prio
> 
> the point is this: such tasks would thus be automatically and
> perpetually considered 'CPU hogs'. Applications cannot abuse this
> mechanism because they get the maximum 'penalty'.
> 
> and as a bonus, no magic sysctl and inherently more flexibility.
> 
> (note that this scheme has advantages above nice +5 because nice +5
> still has the interactivity stuff on which can create priority
> fluctuations and may thus affect workloads.)
> 
> if you agree with this scheme, would you be interested in implementing
> this?

The better cpu proportion guarantee without low latency of such a policy 
would be desirable to video encoding in the background while capturing 
in the foreground as one immediately recognisable purpose, and there are 
likely numerous others, so I agree it's a good idea.

However the non-interactive mode addresses a number of different needs 
that seem to have come up. Specifically:
I have had users report great success with such a mode on my own 
scheduler in multiple X session setups where very choppy behaviour 
occurs in mainline.
Many high performance computing people do not wish interactivity code 
modifying their choice of latency/distribution - admittedly this is a 
soft one.

What are your thoughts on this?

Con

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: [PATCH] optional non-interactive mode for cpu scheduler

[Ingo Molnar]

* Con Kolivas <kernel@kolivas.org> wrote:

> However the non-interactive mode addresses a number of different needs 
> that seem to have come up. Specifically:
> I have had users report great success with such a mode on my own 
> scheduler in multiple X session setups where very choppy behaviour 
> occurs in mainline.

since SCHED_CPUBOUND would be inherited across fork(), it should be
rather easy to start an X session with all tasks as SCHED_CPUBOUND.

but i think the above rather points in the direction of some genuine
weakness in the interactivity code (i know, for which the fix is
staircase ;) which would be nice to debug.

> Many high performance computing people do not wish interactivity code
> modifying their choice of latency/distribution - admittedly this is a
> soft one.

well, SCHED_CPUBOUND would solve their needs too, right?

	Ingo

Re: [PATCH] optional non-interactive mode for cpu scheduler

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1717 bytes --]

Ingo Molnar wrote:
> * Con Kolivas <kernel@kolivas.org> wrote:
> 
> 
>>However the non-interactive mode addresses a number of different needs 
>>that seem to have come up. Specifically:
>>I have had users report great success with such a mode on my own 
>>scheduler in multiple X session setups where very choppy behaviour 
>>occurs in mainline.
> 
> 
> since SCHED_CPUBOUND would be inherited across fork(), it should be
> rather easy to start an X session with all tasks as SCHED_CPUBOUND.
> 
> but i think the above rather points in the direction of some genuine
> weakness in the interactivity code (i know, for which the fix is
> staircase ;) which would be nice to debug.

Heh I wasnt implying we should move to staircase to fix our problems 
(then I'd have nothing special for -ck would I?). I was simply trying to 
reproduce that behaviour with a similar mode. As for the choppiness, the 
reports were that it would go away if X was run nice+19 which implies no 
dynamic priority adjustment was the answer.

>>Many high performance computing people do not wish interactivity code
>>modifying their choice of latency/distribution - admittedly this is a
>>soft one.
> 
> 
> well, SCHED_CPUBOUND would solve their needs too, right?

Assuming they wanted to run everything SCHED_CPUBOUND, yes.

Your solution has quite some merit to it :)

I'll look into coding it later this week (thanks for suggesting I do it 
btw). This ordeal has left me seriously sleep deprived :P

Since we're considering providing a special cpu policy for high latency 
high cpu usage, does that mean we can now talk about other policies like 
batch, isochronous etc? And in the medium to long term future, gang and 
group?

Regards,
Con

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: [PATCH] optional non-interactive mode for cpu scheduler

[Ingo Molnar]

* Con Kolivas <kernel@kolivas.org> wrote:

> I'll look into coding it later this week (thanks for suggesting I do
> it btw). This ordeal has left me seriously sleep deprived :P

:-|

> Since we're considering providing a special cpu policy for high
> latency high cpu usage, does that mean we can now talk about other
> policies like batch, isochronous etc? And in the medium to long term
> future, gang and group?

SCHED_ISO would be interesting, but all SCHED_BATCH patches that i've
seen so far were fundamentally broken. [ none protects against the
possibility of a simple CPU hog starving a SCHED_BATCH task in kernel
mode holding say /home's i_sem forever. None except the one i wrote a
couple of years ago that is ;-) ]

but obviously any new scheduling policy first needs considerable
testing, exposure and concensus. The main thing that makes
SCHED_CPUBOUND possibly objectionable is that it could easily be used as
a flag to 'turn off the interactivity code', which is wrong and just
prolongs the fixing of interactivity-estimator bugs. Scientific apps
burn CPU time exclusively and they have a stable priority at the low end
of the range.

One exception would be CPU-bound code with multiple threads which
interact with each other - one always runs but the others always sleep.
A possible solution would be to exclude all inter-task synchronization
methods from the 'interactivity boost' and only hard-device-waits would
be considered true 'waiting', such as keyboard, mouse, disk or network
IO.

	Ingo

Re: [PATCH] optional non-interactive mode for cpu scheduler

[Kyle Moffett]

On Nov 02, 2004, at 08:52, Ingo Molnar wrote:
> One exception would be CPU-bound code with multiple threads which
> interact with each other - one always runs but the others always sleep.
> A possible solution would be to exclude all inter-task synchronization
> methods from the 'interactivity boost' and only hard-device-waits would
> be considered true 'waiting', such as keyboard, mouse, disk or network
> IO.

Alternatively, you might try a system where each "hard-device-wait"
gets a specific interactivity rating, the default would be 1.0, but one
could specify that /dev/input/mice gets a rating of 10.0.  Then when
handling inter-process communication, a wait on IPC connected to
some local program would cause a change in interactivity rating
based on the current interactivity of the other process. EX:

gpm waits on /dev/input/mice, gets a high interactivity rating.
X waits on /dev/gpmdata, gets a lesser but still significant rating.

This clearly has some flaws, but it may be useful in some scenarios.

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a17 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$
L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+
PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r  
!y?(-)
------END GEEK CODE BLOCK------

Re: [PATCH] optional non-interactive mode for cpu scheduler

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1167 bytes --]

Ingo Molnar wrote:
> SCHED_ISO would be interesting, 

Cool! I've been toying with this too :)

> but all SCHED_BATCH patches that i've
> seen so far were fundamentally broken. [ none protects against the
> possibility of a simple CPU hog starving a SCHED_BATCH task in kernel
> mode holding say /home's i_sem forever. None except the one i wrote a
> couple of years ago that is ;-) ]

I guess the one I wrote for staircase is inadequate too. Although in the 
field the implementation has been safe as far as I can tell.

I'm thinking of holding off for a bit to allow those current changes to 
be tried in -mm for a bit.

I have two more questions - there are already userspace tools and older 
out-of-tree kernels (inluding my current one) that use SCHED_BATCH and 
SCHED_ISO.

Should we respect the values for these policies and use numbering 
consistent with them (meaning SCHED_BATCH at 3 would be reserved but not 
used) or should we dish out values according to when they're implemented 
and demand userspace be updated.

Should we move to a policy bitmask numbering system and/or make 
SCHED_CPUBOUND, SCHED_ISO etc subpolicies of SCHED_NORMAL?

Regards,
Con

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]