Re: Linux v2.6.24-rc1, x86 arch code quality, unifications

Ingo Molnar
2007-10-24T10:17:43Z

* Christoph Hellwig <hch@infradead.org> wrote:

> On Tue, Oct 23, 2007 at 09:19:16PM -0700, Linus Torvalds wrote:
> > In short, we just had an unusually large amount of not just x86 merges,
>
> Btw, can we please finis up this merge a little more before we freeze
> 2.6.24? The way we currently have leftovers of arch/i386/ and
> arch/x86_64/ is quite a nightmare and not how the other architectures
> were merged.
>
> Thomas, what again prevents us from just killing these leftovers?

to answer that question one should first be aware of the fundamental
code quality problems that the unified x86 architecture has inherited
from the split i386 and x86_64 architectures.

To get objective and automated metrics about code quality, i've
constructed a table of "coding style errors per one thousand lines of
source code" numbers with the help of the latest checkpatch.pl. The
codebases i measured are the pre-merge i386 and x86_64 tree, the
post-merge arch/x86 unified architecture, and i've also added a handful
of other architectures and selected core subsystems, as comparison:

-------------------------------------------------------
| errors | lines of code | errors/KLOC
| | | (smaller is better)
--------------|------------|----------------|------------------------
arch/i386/ 5717 73865 77.3
arch/x86_64/ 2993 31155 96.0
arch/x86/ 8504 114654 74.1
..............|............|................|........................
arch/ia64/ 1779 64022 27.7
arch/mips/ 2110 94692 22.2
arch/sparc64/ 1387 49253 28.1
..............|............|................|........................
kernel/ 762 83540 9.1
kernel/time/ 15 4191 3.5
kernel/irq/ 1 2317 0.4
mm/ 464 46324 10.0
net/core 176 24413 7.2
..............|............|................|........................

a couple of observations. Firstly, it is plainly obvious that the x86_64
and i386 architectures were in a dreadful state of code quality before
the unification. Their code quality was almost an order of magnitude
worse than that of the core kernel (!) - and their code quality was
significantly worse than that of a couple of other, comparable
architectures. (we knew this when we started the x86 unification effort
- but i suspect it's even more apparent via the hard numbers in this
table.)

( Note: code metrics should be taken with a grain of salt, as they
often over-simplify the picture, but in this particular situation the
trends are clear and the numbers match my personal impressions of
code quality and robustness of these codebases. )

paradoxically the x86_64 architecture that had a _worse_ code quality
than the "legacy" 32-bit code - so much about the "newer code must be
better" misconception. The first, mechanic round of unifications thus
brought a net degradation in quality - but we've reversed that trend in
2.6.24-rc1 already, via unifications and cleanups, as it can be seen
from the table. (and we did that while adding new features like
high-resolution timers and dynticks to the x86-64bit architecture in
v2.6.24-rc1 - or the new IOMMU code. So the x86 architecture is not
standing still at all while the unification is going on.)

so to answer your question: full unification is no easy task and it is
not automatic at all. The x86_64 tree has diverged from the i386 tree in
the past 5 years due to their illogical, forced separation and a
resulting bitrot. The two architectures have grown different sets of
cleanliness problems and different sets of functions with arbitrary
differences that often cover the same functionality. It's all compounded
by the fact that the 64-bit code is in worse shape than the 32-bit - so
it's not like we could just pick the 64-bit code and use that as the
unified code. The 32-bit code is also used about 8-10 times more
frequently than the 64-bit code. So there is no easy "just unify it"
path.

The new maintainers of the x86 architecture (Thomas, Peter and me), and
many other x86 developers are highly motivated to improve the x86
architecture's code quality and unify the heck out of it, and there are
some real improvements in 2.6.24-rc1 already, but we _must_ be (and are)
working on this carefully. So we do unifications on a case by case
basis, with the highest priority being to not introduce "unification
regressions". The x86 architecture is the most common Linux architecture
after all - and users care much more about having a working kernel than
they care about cleanups and unifications. So yes, we agree with you,
but please be patient! :-) This cannot be realistically finished in
v2.6.24, without upsetting the codebase.

Ingo