2.6.28 has unconfigurable group sched?

2.6.28 has unconfigurable group sched?

[Mark Glines]

Hi,

I've got an issue on x86-64 where I can't configure the system to allow
RT tasks for a non-root user.

In 2.6.26.5, I was able to do the following to set things up nicely:
echo 450000 >/sys/kernel/uids/0/cpu_rt_runtime
echo 450000 >/sys/kernel/uids/1000/cpu_rt_runtime

I've searched lkml archives and am unable to figure out what changed.
As far as I can tell, the documentation (sched-rt-group.txt) says this
should work, and indeed, it did work with 2.6.26.

Seems like every value I try to echo into the /sys files returns EINVAL.
 Even when I'm just writing the same value it already had:

chirp uids # pwd
/sys/kernel/uids
chirp uids # cat 0/cpu_rt_period
1000000
chirp uids # cat 0/cpu_rt_runtime
950000
chirp uids # cat 1000/cpu_rt_period
1000000
chirp uids # cat 1000/cpu_rt_runtime
0
chirp uids # echo 450000 >0/cpu_rt_runtime
echo: write error: invalid argument
chirp uids # echo 450000 >1000/cpu_rt_runtime
echo: write error: invalid argument
chirp uids # echo 950000 >0/cpu_rt_runtime
echo: write error: invalid argument
chirp uids # echo 0 >0/cpu_rt_runtime
echo: write error: invalid argument

So it seems rt_schedulable() is never succeeding.  Any ideas?


sched-related snippets from .config:

CONFIG_X86_SMP=y
CONFIG_USE_GENERIC_SMP_HELPERS=y
CONFIG_X86_64_SMP=y
CONFIG_X86_HT=y

# CONFIG_CGROUPS is not set
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
CONFIG_GROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_USER_SCHED=y
# CONFIG_CGROUP_SCHED is not set

Re: 2.6.28 has unconfigurable group sched?

[Peter Zijlstra]

On Tue, 2009-01-13 at 10:31 -0800, Mark Glines wrote:
> Hi,
> 
> I've got an issue on x86-64 where I can't configure the system to allow
> RT tasks for a non-root user.
> 
> In 2.6.26.5, I was able to do the following to set things up nicely:
> echo 450000 >/sys/kernel/uids/0/cpu_rt_runtime
> echo 450000 >/sys/kernel/uids/1000/cpu_rt_runtime
> 
> I've searched lkml archives and am unable to figure out what changed.
> As far as I can tell, the documentation (sched-rt-group.txt) says this
> should work, and indeed, it did work with 2.6.26.
> 
> Seems like every value I try to echo into the /sys files returns EINVAL.
>  Even when I'm just writing the same value it already had:
> 
> chirp uids # pwd
> /sys/kernel/uids
> chirp uids # cat 0/cpu_rt_period
> 1000000
> chirp uids # cat 0/cpu_rt_runtime
> 950000
> chirp uids # cat 1000/cpu_rt_period
> 1000000
> chirp uids # cat 1000/cpu_rt_runtime
> 0
> chirp uids # echo 450000 >0/cpu_rt_runtime
> echo: write error: invalid argument
> chirp uids # echo 450000 >1000/cpu_rt_runtime
> echo: write error: invalid argument
> chirp uids # echo 950000 >0/cpu_rt_runtime
> echo: write error: invalid argument
> chirp uids # echo 0 >0/cpu_rt_runtime
> echo: write error: invalid argument
> 
> So it seems rt_schedulable() is never succeeding.  Any ideas?

Yeah, I should test this user grouping stuff more often :/

The below fixes it for me.
---
Subject: sched: fix bandwidth validation for UID grouping 

For UID grouping we initialize the root group with infinite bandwidth
which by default is actually more than the global limit, therefore the
bandwidth check always fails.

Because the root group is a phantom group (for UID grouping) we cannot
runtime adjust it, therefore we let it reflect the global bandwidth
settings.

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
CC: stable@kernel.org
---
diff --git a/kernel/sched.c b/kernel/sched.c
index 02f35a1..a61fb0e 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -9195,6 +9195,13 @@ static int tg_schedulable(struct task_group *tg, void *data)
 		runtime = d->rt_runtime;
 	}
 
+#ifdef CONFIG_USER_SCHED
+	if (tg == &root_task_group) {
+		period = global_rt_period();
+		runtime = global_rt_runtime();
+	}
+#endif
+
 	/*
 	 * Cannot have more runtime than the period.
 	 */

Re: 2.6.28 has unconfigurable group sched?

[Mark Glines]

Peter Zijlstra wrote:
>> So it seems rt_schedulable() is never succeeding.  Any ideas?
> 
> Yeah, I should test this user grouping stuff more often :/
> 
> The below fixes it for me.
> ---
> Subject: sched: fix bandwidth validation for UID grouping 
> 
> For UID grouping we initialize the root group with infinite bandwidth
> which by default is actually more than the global limit, therefore the
> bandwidth check always fails.
> 
> Because the root group is a phantom group (for UID grouping) we cannot
> runtime adjust it, therefore we let it reflect the global bandwidth
> settings.


Thanks.  Sadly, this doesn't seem to change anything for me.  I applied
it to 2.6.28, rebuilt and reinstalled, and got the same result.  I did a
clean rebuild to be sure; and got the same behavior again.

Please let me know if there are any additional diagnostics I can provide.

Mark

Re: 2.6.28 has unconfigurable group sched?

[Peter Zijlstra]

On Wed, 2009-01-14 at 06:46 -0800, Mark Glines wrote:
> Peter Zijlstra wrote:
> >> So it seems rt_schedulable() is never succeeding.  Any ideas?
> > 
> > Yeah, I should test this user grouping stuff more often :/
> > 
> > The below fixes it for me.
> > ---
> > Subject: sched: fix bandwidth validation for UID grouping 
> > 
> > For UID grouping we initialize the root group with infinite bandwidth
> > which by default is actually more than the global limit, therefore the
> > bandwidth check always fails.
> > 
> > Because the root group is a phantom group (for UID grouping) we cannot
> > runtime adjust it, therefore we let it reflect the global bandwidth
> > settings.
> 
> 
> Thanks.  Sadly, this doesn't seem to change anything for me.  I applied
> it to 2.6.28, rebuilt and reinstalled, and got the same result.  I did a
> clean rebuild to be sure; and got the same behavior again.
> 
> Please let me know if there are any additional diagnostics I can provide.

Most strange, I did the patch on .28 just to be sure (my initial testing
was done on -linus since I couldn't remember it having changed in a
while), and it still works for me.

CONFIG_GROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_RT_GROUP_SCHED=y
CONFIG_USER_SCHED=y
# CONFIG_CGROUP_SCHED is not set

Linux opteron 2.6.28 #634 SMP PREEMPT Wed Jan 14 18:13:31 CET 2009
x86_64 x86_64 x86_64 GNU/Linux

# echo 450000 > /sys/kernel/uids/0/cpu_rt_runtime 
# cat /sys/kernel/uids/0/cpu_rt_runtime
450000

Re: 2.6.28 has unconfigurable group sched?

[Mark Glines]

Peter Zijlstra wrote:
> On Wed, 2009-01-14 at 06:46 -0800, Mark Glines wrote:
>> Thanks.  Sadly, this doesn't seem to change anything for me.  I applied
>> it to 2.6.28, rebuilt and reinstalled, and got the same result.  I did a
>> clean rebuild to be sure; and got the same behavior again.

> Most strange, I did the patch on .28 just to be sure (my initial testing
> was done on -linus since I couldn't remember it having changed in a
> while), and it still works for me.

No, I'm sorry, you're right.  It does work beautifully now with your
patch.  I just hadn't been testing the correct kernel image (oops).

Thanks!

Mark