[BUG?,FEATURE?] One core idles with 4 nice and 1 regular process

[BUG?,FEATURE?] One core idles with 4 nice and 1 regular process

[Jason Lynch]

First, a description of the problem, and a fairly simple way to test it:

All of this is running on my quad-core Q6600. First, I start four nice -n 
19 processes, each of which does nothing but busy-wait. (In my case, I 
used a simple python script, but any CPU-bound process will do.) At this 
point, top shows all four cores being utilized, at approximately 100% 
nice each.

At this point, I start another of these busy-waiting processes, except 
with no nice adjustment, so it runs at regular priority. After doing so, 
top now reports the following CPU usage: one core is 100% user, two cores 
are 100% nice, and finally, one last core is 100% idle, doing nothing, 
despite a total of five processes being available to run.

I thought I first noticed this running a 2.6.29 kernel, and thought about 
bisecting, but my test 2.6.28 kernel also exhibited the behavior. I 
couldn't test any previous kernels due to a lack of ext4 support. Both my 
current 2.6.30 kernel and the torvalds/linux-2.6.git HEAD also show the 
behavior.

Now, I'm not sure if this is a bug or a strange feature (though it seems 
very bug-like to me) or perhaps a configuration problem, but I'd like to 
find out nonetheless.

If I can provide any additional information to help, I'll be happy to 
oblige.

Re: [BUG?,FEATURE?] One core idles with 4 nice and 1 regular process

[Jason Lynch]

On Tue, 16 Jun 2009 19:09:59 +0000, Jason Lynch wrote:
> If I can provide any additional information to help, I'll be happy to
> oblige.

I discovered a post on lkml from last October, wherein a user reports 
what sounds like the same issue on 2.6.25.9.

http://article.gmane.org/gmane.linux.kernel/748316/

In addition, Ubuntu's current 2.6.28 kernel appears to be immune from the 
problem, suggesting a possible configuration issue, or at least an option 
that can trigger/avoid the problem. However, comparing the configs is 
difficult due to the number of added, changed, and removed options 
between kernel versions. The differences that stood out in my initial 
comparisons were CONFIG_RT_GROUP_SCHED and CONFIG_CPUSETS, but I haven't 
yet had an opportunity to test either of them, and neither sounds 
especially promising.

I won't personally be able to do any more extensive kernel testing for 
perhaps a couple of weeks. However, if this is merely a configuration 
issue, it seems rather opaque to me, as I can't imagine this behavior 
being desirable in any case.