The evolution of the Linux kernel build system

The evolution of the Linux kernel build system

[Bram Adams]

Hi,

My name is Bram Adams and I'm a PhD student at Ghent University in  
Belgium. Part of the research of our lab is about understanding build  
systems to investigate the close (co-)evolution of source code and the  
build system. For this, we have examined the evolution of the (major  
releases of the) Linux kernel build system from 1991 to 2007. A first  
report of this work has been published online at http://eceasst.cs.tu-berlin.de/index.php/eceasst/article/view/115 
, and Sam Ravnborg encouraged us to share this with the kbuild mailing  
list. A more detailed report is expected to be available in two or  
three months.

Our paper looks at the evolution of the Linux kernel build system  
(1991-2007) from various angles:
  * measurement of the number of lines of source code, build and  
configuration scripts
  * measurements of the complexity of the build dependency graph
  * visualisation of the build dependency graph
  * filtering of the build dependency graph to uncover build idioms
  * the kbuild documentation
  * mailing list discussions
  * notes on web sites
  * and of course the makefiles and configuration scripts themselves  
when in doubt about something

We have used this data to find measurable evidence of the evolution of  
the internal complexity of the build layer (kbuild). We did not  
investigate in detail the "user interface" of makefiles and  
configuration scripts, i.e. how developers add new modules or kernel  
source files. Further investigation is needed on this. We are planning  
to analyse other systems besides the Linux kernel to generalise our  
findings and to examine whether predictions could be made on the  
health or evolvability of a build system based on the above  
characteristics.

We would like to thank Sam Ravnborg and Kai Germaschewski for their  
constructive comments on an earlier version of the report. Any  
feedback on the published report or suggestions for further analysis  
are gladly appreciated.

Kind regards,

Bram Adams
GH-SEL, INTEC, Ghent University (Belgium)
http://users.ugent.be/~badams/

Re: The evolution of the Linux kernel build system

[Randy Dunlap]

On Thu, 20 Mar 2008 15:19:39 +0100 Bram Adams wrote:

> Hi,
> 
> My name is Bram Adams and I'm a PhD student at Ghent University in  
> Belgium. Part of the research of our lab is about understanding build  
> systems to investigate the close (co-)evolution of source code and the  
> build system. For this, we have examined the evolution of the (major  
> releases of the) Linux kernel build system from 1991 to 2007. A first  
> report of this work has been published online at http://eceasst.cs.tu-berlin.de/index.php/eceasst/article/view/115 
> , and Sam Ravnborg encouraged us to share this with the kbuild mailing  
> list. A more detailed report is expected to be available in two or  
> three months.

Hi,
Please let us know when the detailed report is available.

Thanks,
---
~Randy

Re: The evolution of the Linux kernel build system

[Bram Adams]

Hi,

Op 20-mrt-08, om 16:01 heeft Randy Dunlap het volgende geschreven:

> Please let us know when the detailed report is available.

OK, thanks!

Kind regards,

Bram Adams
GH-SEL, INTEC, Ghent University (Belgium)

Re: The evolution of the Linux kernel build system

[Giacomo A. Catenazzi]

Bram Adams wrote:
> We would like to thank Sam Ravnborg and Kai Germaschewski for their 
> constructive comments on an earlier version of the report. Any feedback 
> on the published report or suggestions for further analysis are gladly 
> appreciated.

Now I've done only a quick look.

I think you missed a point about dependencies.
Also in the old method, the .h dependencies were calculated
on build time (with the gcc -MM option IIRC).
But big problem is the dependencies of configuration.
Instead of having the dependency of config.h,
the kbuild checks about changes on every single
configuration item.

So "make dep" read the sources looking for CONFIG_ options.

IIRC it was a separate step, not to check file
for CONFIG_ string at every modification.
"make dep" was not so mandatory after every patch,
and anyway before big releases a "make mrproper"
was still recommended.

There was a lot of active development on this program,
to increment speed, and AFAIK no other project split
header dependencies in a more precise way.


This weekend I'll read carefully the paper and I'll give
you more comments.


ciao
	cate