[RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Adrian Bunk]

I know that this is controversial, for further background please see the
discussion in http://bugzilla.kernel.org/show_bug.cgi?id=10761

Facts:
- Bugzilla statistics
  (not counting duplicate bugs and bugs closed as INVALID):
  - 6 CPU scheduler regressions reported during 2.6.25-rc
    1 of them still open as of 2.6.26-rc4
  - 6 CPU scheduler regressions reported during 2.6.26-rc until now
    5 of them still open as of 2.6.26-rc4
  - at about half of the CPU scheduler regressions are related to
    group scheduling
- Peter Zijlstra said in Bugzilla bug #10761:
  > Adrian Bunk said:
  > The CPU scheduler is currently regressing horribly often, and half of the
  > regressions are in group scheduling.
  That is because group scheduling is horribly complex and was never
  feature complete - trying to solve that is high on my list of
  priorities.

My opinion:
- features that alter existing kernel functionality should (different
  from drivers for previously not supported hardware) be in a very good
  state before being offered in stable kernels
- Peter is wrong when he thinks that EXPERIMENTAL would somehow hide
  known-problematic features - in practice nearly everyone has to have 
  EXPERIMENTAL enabled (e.g. for some hardware driver)
- group scheduling already has a history of people accidentally enabling 
  it and then running into problems

I therefore suggest this patch to let GROUP_SCHED depend on BROKEN.

Once it is feature complete and then got the usual testing through
-next I'll have no objections against offering it again to users.

Signed-off-by: Adrian Bunk <bunk@kernel.org>

---
c67cfbbb40895b72760865527dd1949631b1d183 diff --git a/init/Kconfig b/init/Kconfig
index d9526b5..564deba 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -333,7 +333,7 @@ config HAVE_UNSTABLE_SCHED_CLOCK
 
 config GROUP_SCHED
 	bool "Group CPU scheduler"
-	depends on EXPERIMENTAL
+	depends on BROKEN
 	default n
 	help
 	  This feature lets CPU scheduler recognize task groups and control CPU

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Mike Galbraith]

On Tue, 2008-05-27 at 11:58 +0300, Adrian Bunk wrote:

> Once it is feature complete and then got the usual testing through
> -next I'll have no objections against offering it again to users.
> 
> Signed-off-by: Adrian Bunk <bunk@kernel.org>
> 
> ---
> c67cfbbb40895b72760865527dd1949631b1d183 diff --git a/init/Kconfig b/init/Kconfig
> index d9526b5..564deba 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -333,7 +333,7 @@ config HAVE_UNSTABLE_SCHED_CLOCK
>  
>  config GROUP_SCHED
>  	bool "Group CPU scheduler"
> -	depends on EXPERIMENTAL
> +	depends on BROKEN
>  	default n
>  	help
>  	  This feature lets CPU scheduler recognize task groups and control CPU

FWIW, here's my NAK, and the grounds therefore:  Group scheduling
clearly fits the current criteria for CONFIG_EXPERIMENTAL.  In-tree
development is a necessary fact of life, and has always been a fact of
life.  IMO, your attempt to change the rules is seriously misguided.

<quote>
config EXPERIMENTAL
	bool "Prompt for non-development and/or absolutely complete code/drivers"
	---help---
	  Some of the various things that Linux supports (such as network
	  drivers, file systems, network protocols, etc.) can be in a state
	  of development where the functionality, stability, or the level of
	  testing is not yet high enough for general use. This is usually
	  known as the "alpha-test" phase among developers. If a feature is
	  currently in alpha-test, then the developers usually discourage
	  uninformed widespread use of this feature by the general public to
	  avoid "Why doesn't this work?" type mail messages. However, active
	  testing and use of these systems is welcomed. Just be aware that it
	  may not meet the normal level of reliability or it may fail to work
	  in some special cases. Detailed bug reports from people familiar
	  with the kernel internals are usually welcomed by the developers
	  (before submitting bug reports, please read the documents
	  <file:README>, <file:MAINTAINERS>, <file:REPORTING-BUGS>,
	  <file:Documentation/BUG-HUNTING>, and
	  <file:Documentation/oops-tracing.txt> in the kernel source).

	  This option will also make obsoleted drivers available. These are
	  drivers that have been replaced by something else, and/or are
	  scheduled to be removed in a future kernel release.

	  Unless you intend to help test and develop a feature or driver that
	  falls into this category, or you have a situation that requires
	  using these features, you should probably say N here, which will
	  cause the configurator to present you with fewer choices. If
	  you say Y here, you will be offered the choice of using features or
	  drivers that are currently considered to be in the alpha-test phase.
</quote>

	-Mike

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Mike Galbraith]

On Tue, 2008-05-27 at 11:47 +0200, Mike Galbraith wrote:

> <quote>
> config EXPERIMENTAL
> 	bool "Prompt for non-development and/or absolutely complete code/drivers"

Heh, a joking reply was _going_ to be my response, but since I strongly
disagree, I decided to just quote the damn text.. but didn't remove the
joke first :)

Oh well, you mean what I know.

	-Mike

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Adrian Bunk]

On Tue, May 27, 2008 at 11:47:38AM +0200, Mike Galbraith wrote:
>...
> <quote>
> config EXPERIMENTAL
> 	bool "Prompt for non-development and/or absolutely complete code/drivers"
 	---help---
> 	  Some of the various things that Linux supports (such as network
> 	  drivers, file systems, network protocols, etc.) can be in a state
> 	  of development where the functionality, stability, or the level of
> 	  testing is not yet high enough for general use. This is usually
> 	  known as the "alpha-test" phase among developers. If a feature is
> 	  currently in alpha-test, then the developers usually discourage
> 	  uninformed widespread use of this feature by the general public to
> 	  avoid "Why doesn't this work?" type mail messages. However, active
> 	  testing and use of these systems is welcomed. Just be aware that it
>...

If you take this text seriously then companies like Redhat advertise 
their enterprise distributions with alpha-test quality software...

Facts are:
- as long as we allow hardware drivers to depend on EXPERIMENTAL we
  force many users to enable EXPERIMENTAL
- the EXPERIMENTAL dependencies in many areas of the kernel are so
  out of sync with the actual code quality that the presence or absence
  of such a dependency isn't a clear indication of the current status

> 	-Mike

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Alan Cox]

> - as long as we allow hardware drivers to depend on EXPERIMENTAL we
>   force many users to enable EXPERIMENTAL
> - the EXPERIMENTAL dependencies in many areas of the kernel are so
>   out of sync with the actual code quality that the presence or absence
>   of such a dependency isn't a clear indication of the current status

It would probably be very useful to actually do an assessment of what is
marked experimental especially a cross compare with enterprise distros as
what they ship they have reviewed and consider supportable.

Alan

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Mike Galbraith]

On Tue, 2008-05-27 at 13:26 +0300, Adrian Bunk wrote:

> - the EXPERIMENTAL dependencies in many areas of the kernel are so
>   out of sync with the actual code quality that the presence or absence
>   of such a dependency isn't a clear indication of the current status

Then the thing to do is fix _that_, not move group scheduling.

	-Mike

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Peter Zijlstra]

On Tue, 2008-05-27 at 11:58 +0300, Adrian Bunk wrote: 

> I therefore suggest this patch to let GROUP_SCHED depend on BROKEN.

NAK - Most if not all issues have fixes or specific reverts already.

Your suggestion is really quite ridiculous and offensive to me. BROKEN
is for stuff that crashes kernels or corrupts user data. Group
scheduling does neither.

As I've said - and which you so 'tactfully' quoted - group scheduling is
a mathmatically complex topic that we are improving actively. The
feature is not enabled by default and it depends on EXPERIMENTAL.

Adrian - if you care about the subject so much - dive into the code and
help us, instead of becoming an obstacle. These past years you've shown
you're capable of understanding trivial C snippets, here is an
opportunity to show you can do real stuff too.

FYI your attitude in the past few weeks is getting on my nerves - you
act all self important but have not made a single contribution to the
kernel above the level of a janitor-newbie.

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Adrian Bunk]

On Tue, May 27, 2008 at 01:47:29PM +0200, Peter Zijlstra wrote:
>...
> FYI your attitude in the past few weeks is getting on my nerves - you
> act all self important but have not made a single contribution to the
> kernel above the level of a janitor-newbie.

Not a single contribution to the kernel above the level of a 
janitor-newbie?

http://article.gmane.org/gmane.linux.kernel/676733
http://lkml.org/lkml/2008/5/14/498
http://lkml.org/lkml/2008/4/28/561
...

Awaiting your apology
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Peter Zijlstra]

On Tue, 2008-05-27 at 16:08 +0300, Adrian Bunk wrote:
> On Tue, May 27, 2008 at 01:47:29PM +0200, Peter Zijlstra wrote:
> >...
> > FYI your attitude in the past few weeks is getting on my nerves - you
> > act all self important but have not made a single contribution to the
> > kernel above the level of a janitor-newbie.
> 
> Not a single contribution to the kernel above the level of a 
> janitor-newbie?
> 
> http://article.gmane.org/gmane.linux.kernel/676733
> http://lkml.org/lkml/2008/5/14/498
> http://lkml.org/lkml/2008/4/28/561
> ....
> 
> Awaiting your apology

Oooh, you caught a few bugs in review... *applause*. Of course I
understand you just forgot to append the url or git commit of the patch
series that added a significant feature to the core kernel, or solved a
significant issue.

I mean, counting all the way up to 3, that must have been taxing you.

So of course I appologise for my ignorance of your intellectual feats
and invaluable contributions to the linux kernel. </sarcasm>

Seriously, you're a good janitor, and I appreciate that. Stick to it, or
help write code.

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Ingo Molnar]

* Adrian Bunk <bunk@kernel.org> wrote:

> I know that this is controversial, [...]

NAK.

The regressions you've listed should all be fixed by Peter and Mike in 
the current scheduler queue either via fixes or via reverts. Basically 
all the group scheduling ones revolve around a single technical issue: 
group-aware load-balancing which got touched in .26-rc1 and which is not 
fully cooked yet.

You are distorting reality: there's not a single kernel crash/hang or 
app crash open in -rc4 and the fixes (or reverts) are all in the v2.6.26 
scheduler pipeline and performance/scalability and interactivity is back 
to .25 levels or better.

But more generally, beyond this specific incident that you generated, i 
can see a big problem in another area: lately you have visibly 
intensified your attempts to boss people around on lkml.
 
Doing small janitorial fixes is fine (and useful), but you are now also 
actively wasting people's time, you are frequently showing an attitude, 
you are exaggerating bugs and you are interfering with the workflow of 
kernel developers.

Furthermore, you've now also upset a MAJOR contributor (Peter Zijstra), 
and upsetting Peter is quite a feat i have to say.

You do all that while the shocking reality is that you have no real 
experience and no real track record _at all_ in writing more complex 
kernel code yourself! How do you expect to be able to judge such issues 
without having the first-hand experience?

To make sure i'm accurate about this i just spent 15 minutes
reviewing all your past commits to the Linux kernel using
"git-shortlog --author=Bunk", and the rather sobering truth
appears to be that:

  - you've _never_ before fixed a complex Linux kernel bug/problem
  - you've never before written a _single_ Linux kernel feature
  - you've never before written a _single_ Linux driver
  - you've never before done any true self-driven functional 
    changes/improvements to Linux

What you did are 1500+ trivial commits that all revolve around the same
janitorial topics: symbol fixes, trivial fixes (often gleaned off
Coverity logs, sparse or namespacecheck output) and dead code removal.

The method i used to review your past contributions was to first exclude 
the main body of your commits via this shell pattern:

git-shortlog --author=Bunk | grep -viE "remov|static|build|license|\
extern|export|syntax|off by|overflow|cleanup|prototype|inline|select|\
bool|tristate|include|depend|init|section|config|scheduled|marked|#if|\
text|off-by-one|\.h|firmware|NULL|global|return|compare|leak|use-after|\
typo|dead|compil|delete|trivial|check after|spelling|small|update|\
check|kill|=|indent|tiny|deprecated|obsolete|endian|default|__dev|renam"

this eliminated 94.8% of your 1607 commits of the past 3 years. I 
reviewed the remaining 83 commits manually, and they all, without 
exception, appeared trivial too. I also have direct experience with you 
as the maintainer of various subsystems: you never before approched me 
about any non-trivial technical issue about _anything_ in the kernel.

So it appears to me - and all facts i have show it - that you appear to 
have no desire _at all_ to write true kernel code. You appear to have no 
ideas of your own (that you express publicly) and you appear to have no 
technical wishes or intellectual curiosity about Linux that rises beyond 
the level of trivialities.

... which might still all be fine and you could still be useful to Linux 
even if you are never able to rise above that trivial level, but the 
difference is that you also do appear to have a very strong desire to be 
seen as an "important person".

Worse than that, you now have also decided to become a major obstacle in 
other people's workflow. And _that_ is a VERY big problem. You are quite 
delusive if you think people will tolerate that kind of behavior.

I used to be a defender of your trivial commits in the past, because i'm 
strongly convinced that the path towards more complex code is through 
doing trivial changes. But you appear to be the exception that proves 
the rule: you stubbornly remained at the intellectual level of trivial 
commits.

Adrian, the thing is, if you have the desire to become a maintainer, the 
very first basic step towards maintaining any kernel code is to start 
_writing_ kernel code, then you can maintain the code you wrote. Linux 
is not a project where you can get away without actually being able to 
write code.

This total, 100% lack of substantial commits from you is OK as long as 
someone only has tangential (or initial) interest in Linux, but it is 
rather shocking from someone like you who has been around for years and 
who tries to be a force in kernel development. I dont really understand 
how you can think that this will fly with us.

You also seem to have a track record of ... similar incidents and a 
crusade in another space - here is this 2002 mail from you on one of the 
Debian lists:

  http://lists.debian.org/debian-devel/2002/01/msg02160.html

  " I do hereby completely retire from Debian. [...] "

You've recently asked on lkml whether you should quit kernel hacking too 
- and my answer is that you need to start kernel hacking before you can 
quit it. You are now on the path to become the equivalent of a 
self-important buerocrat who creates his own work and who plays petty 
office politics. Which is quite a feat i have to say, given the tons of 
real technical work that is still to be done in Linux.

	Ingo

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Thomas Gleixner]

On Tue, 27 May 2008, Adrian Bunk wrote:
> On Tue, May 27, 2008 at 01:47:29PM +0200, Peter Zijlstra wrote:
> >...
> > FYI your attitude in the past few weeks is getting on my nerves - you
> > act all self important but have not made a single contribution to the
> > kernel above the level of a janitor-newbie.
> 
> Not a single contribution to the kernel above the level of a 
> janitor-newbie?
> 
> http://article.gmane.org/gmane.linux.kernel/676733
> http://lkml.org/lkml/2008/5/14/498
> http://lkml.org/lkml/2008/4/28/561
> ...

Adrian, do you really believe that the above is substantial ? 

You spotted a bug, decoded a bug report and fixed a Kconfig. Nobody
says that this is a bad contribution, but it does not qualify you at
all to act as the uber controller of the kernel.

I always defended your contributions, which were questioned by a lot
of kernel developers, but your recent attitude of artificially
inflating bugs, which are either already resolved, have patches
pending or are actively worked on is annoying a lot of kernel folks,
not only Peter.

I'm sure Peter would have accepted such a patch, if there was a good
technical reason for it.

Your book-keeper list of bugreports lacks technical insight and is not
a good technical reason, it's just a strawman to gain control and/or
importance.

The bugzilla lists are good to remind us about outstanding problems,
but they are in no way the sole technical measure for marking a
functionality broken. Bugzilla is a tracking tool, nothing else. Using
it as a technical measure is just like the management illusion that
you can control a company with spread-sheets.

Following your book-keeper logic we have to mark HIBERNATE, SUSPEND,
NOHZ, HIGHRES, TSC and tons of other things BROKEN as well.

Marking it BROKEN is the last resort when we have something unfixable
which causes data corruption or crashes for which there is no active
maintainer who's willing to stand up for breakage.

Marking an incomplete default off feature, which does not result in
crashes or data corruption, BROKEN is just wrong. EXPERIMENTAL is the
correct category to get it exposed to users which helps to get
information about the side effects.

I enjoy working with Rafael on a technical level on those bugzilla
issues, but your vain attempt to influence the kernel development by
bureaucrazy will just lead to further ignorance vs. the kernel
bugzilla as the signal/noise ratio already reached an high annoyance
level since you started to abuse it for your renewed quality crusade.

Your previous attempt to control kernel development via the regression
list failed and you resigned. Now you come back and attack particular
subsystem developers/maintainers with the same method. Do you really
believe that abusing statistics for your exaggerated problem inflation
is solving any problems ?

No, it's causing more problems because you slander and frustrate
active developers who are aware of the issues and are working hard to
resolve them. The least thing they need is "motivation" in form of
such patches.

> Awaiting your apology

For what ? For telling the truth ?

Thanks,

	tglx

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Adrian Bunk]

On Tue, May 27, 2008 at 04:22:12PM +0200, Ingo Molnar wrote:
>...
> To make sure i'm accurate about this i just spent 15 minutes
> reviewing all your past commits to the Linux kernel using
> "git-shortlog --author=Bunk", and the rather sobering truth
> appears to be that:
> 
>   - you've _never_ before fixed a complex Linux kernel bug/problem

What is a "complex kernel problem"?

Just looking a bit through the last month:

E.g. http://lkml.org/lkml/2008/5/14/498 was not horribly complex, but 
for 3 days noone else had bothered to answer the request of the driver 
maintainer for help with this regression.

As you yourself said http://article.gmane.org/gmane.linux.kernel/676733
is something where I spotted a real bug just by reading through patches 
on linux-kernel. The bug might or might not have been complex, but since 
someone spotted it just when reading through patches (it's entirly 
possible someone else might have spotted it as well, but we will never 
know for sure) it never became a bug in any tree.

And for me stuff like http://lkml.org/lkml/2008/4/28/561 isn't complex, 
but quite frankly my impression is that I'm currently one out of only 
two people understanding our Kconfig files good enough for fixing such 
stuff.

>   - you've never before written a _single_ Linux kernel feature
>   - you've never before written a _single_ Linux driver

True.

I am not a coder.
I am just a cleanup and QA guy.

I know that this makes me a persona non grata for some people on this 
list, but I prefer doing what I'm good at (bugfixes and cleanup
patches, aiming at not introducing new bugs) over doing what I'm
bad at (writing code).

>   - you've never before done any true self-driven functional 
>     changes/improvements to Linux

I'd claim having started regression tracking (which was 100% self-driven)
was a functional improvement for Linux kernel development - and I don't 
care that you disagree on this.

>...
> 	Ingo

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Mike Galbraith]

Let's all take a deep breath and relax, ok?  A patch was aggressively
presented and aggressively NAKed.  Let's not get our bowels in an uproar
over it.

Nothing to see here folks, move along...

	-Mike

Re: [RFC: 2.6 patch] let GROUP_SCHED depend on BROKEN

[Adrian Bunk]

On Tue, May 27, 2008 at 07:30:42PM +0200, Thomas Gleixner wrote:
> On Tue, 27 May 2008, Adrian Bunk wrote:
> > On Tue, May 27, 2008 at 01:47:29PM +0200, Peter Zijlstra wrote:
> > >...
> > > FYI your attitude in the past few weeks is getting on my nerves - you
> > > act all self important but have not made a single contribution to the
> > > kernel above the level of a janitor-newbie.
> > 
> > Not a single contribution to the kernel above the level of a 
> > janitor-newbie?
> > 
> > http://article.gmane.org/gmane.linux.kernel/676733
> > http://lkml.org/lkml/2008/5/14/498
> > http://lkml.org/lkml/2008/4/28/561
> > ...
> 
> Adrian, do you really believe that the above is substantial ? 

Whether they are substantial one might dispute, but in my opinion these 
are "above the level of a janitor-newbie".

E.g. how many newbies on the janitors list know how to fix bugs 
involving complicated kconfig dependencies (like bools mixed with 
tristates and selects)?

>...
> Following your book-keeper logic we have to mark HIBERNATE, SUSPEND,
> NOHZ, HIGHRES, TSC and tons of other things BROKEN as well.

My personal opinion (I did put an RFC into the subject) is that 
something that gets offered to kconfig users must be worth it.

If it would bring real benefits in it's current state or if it was just
the usual shaking-out of bugs after merging (I remember NOHZ caused
quite a few issues, and also recent stuff like PAT or snd_pcsp caused
problems that had to be fixed) problems were kind of unavoidable.

And e.g. both HIBERNATE/SUSPEND and GROUP_SCHED have some history of 
causing regressions and resulting in people debugging and bisecting for 
finding the cause of some regression (even Rafael spent some time 
bisecting a 2.6.24-git group scheduling regression).

For HIBERNATE/SUSPEND the value for users is clear.
And no matter how many regressions it causes it is therefore worth it.

But how big is the value of GROUP_SCHED in it's current state for users?

And Peter said "group scheduling is horribly complex and was never 
feature complete", so regular breakages are kind of expected until
it is feature complete.

>...
> > Awaiting your apology
> 
> For what ? For telling the truth ?

If someone says most of my contributions are janitorial stuff that's 
the truth.

But Peter claiming not a single contribution I did was above the level 
of a janitor-newbie is not the truth.

> Thanks,
> 
> 	tglx

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed