2.6.13-rc3 test: finding compile errors with make randconfig

2.6.13-rc3 test: finding compile errors with make randconfig

[Grant Coady]

Greetings,

Few days ago I compiled 241 random configurations of 2.6.13-rc3, today 
I finally got around to parsing the results, top 40, sorted by name.  
Percentage is error_builds / total_builds.

build script similar to:
count=0
while [ $((++count)) -le $limit ]; do
        trial=$(printf %003d $count)
        make randconfig
        cp .config "$store/$trial-config"
        make clean
        make -j2 2> "$store/$trial-error"
done

Curious whether this is worth doing, I'm about to start a run for 2.6.12.3, 
any interesting errors I can find the particular config + error to recover 
context.  Deliberately simplistic for traceability at the moment, truncated 
error length for this post.

# Linux kernel version: 2.6.13-rc3
# Tue Jul 19 04:04:05 2005..Wed Jul 20 09:46:33 2005  241 runs in three sessions

compile host 2.4.31-hf2 on: http://scatter.mine.nu/test/boxen/sempro/
data filter: http://scatter.mine.nu/test/scripts/count_errors-2.6.13-rc3.gz

arch/i386/mach-es7000/es7000.h: error: field `Header' has incomplete type   4.9%
arch/i386/mach-es7000/es7000.h: error: field `id' has incomplete type       4.9%
arch/i386/mach-es7000/es7000plat.c: error: `es7000_rename_gsi' undeclared   4.9%
arch/i386/mach-es7000/es7000plat.c: error: dereferencing pointer to incomp 49.7%
arch/i386/mach-es7000/es7000plat.c: error: invalid application of `sizeof' 14.9%
drivers/char/drm/gamma_context.h: error: `DRM' declared as function return 24.8%
drivers/char/drm/gamma_context.h: error: `DRM' previously defined here     24.8%
drivers/char/drm/gamma_context.h: error: `arg' undeclared (first use in th 14.5%
drivers/char/drm/gamma_context.h: error: `dev' undeclared (first use in th 10.3%
drivers/char/drm/gamma_context.h: error: `filp' undeclared (first use in t 12.4%
drivers/char/drm/gamma_context.h: error: called object is not a function   26.9%
drivers/char/drm/gamma_context.h: error: redefinition of `DRM'             22.8%
drivers/char/drm/gamma_context.h: error: structure has no member named `de 16.5%
drivers/char/drm/gamma_drv.h: error: `DRM' declared as function returning  31.1%
drivers/char/drm/gamma_lists.h: error: `DRM' declared as function returnin 18.6%
drivers/char/drm/gamma_lists.h: error: `DRM' previously defined here       18.6%
drivers/char/drm/gamma_lists.h: error: `bl' undeclared (first use in this  18.6%
drivers/char/drm/gamma_lists.h: error: called object is not a function      8.2%
drivers/char/drm/gamma_lists.h: error: redefinition of `DRM'               18.6%
drivers/char/drm/gamma_lock.h: error: `DRM' declared as function returning 10.3%
drivers/char/drm/gamma_lock.h: error: `DRM' previously defined here         8.2%
drivers/char/drm/gamma_lock.h: error: `context' undeclared (first use in t  8.2%
drivers/char/drm/gamma_lock.h: error: `dev' undeclared (first use in this   8.2%
drivers/char/drm/gamma_lock.h: error: called object is not a function      16.5%
drivers/char/drm/gamma_lock.h: error: redefinition of `DRM'                10.3%
drivers/char/drm/gamma_old_dma.h: error: `DRM' declared as function return 10.3%
drivers/char/drm/gamma_old_dma.h: error: `DRM' previously defined here     10.3%
drivers/char/drm/gamma_old_dma.h: error: `filp' undeclared (first use in t  6.2%
drivers/char/drm/gamma_old_dma.h: error: called object is not a function   18.6%
drivers/char/drm/gamma_old_dma.h: error: redefinition of `DRM'             10.3%
drivers/char/drm/gamma_old_dma.h: error: structure has no member named `ne 18.6%
drivers/char/ipmi/ipmi_msghandler.c: error: (near initialization for `__ks 14.9%
drivers/char/ipmi/ipmi_msghandler.c: error: __ksymtab_proc_ipmi_root cause 14.9%
drivers/char/ipmi/ipmi_msghandler.c: error: `proc_ipmi_root' undeclared he 14.9%
drivers/char/ipmi/ipmi_msghandler.c: error: initializer element is not con 14.9%
drivers/mtd/chips/amd_flash.c: error: structure has no member named `buswi 14.9%
drivers/mtd/chips/jedec.c: error: structure has no member named `buswidth' 13.2%
include/asm-i386/mach-default/mach_apic.h: error: `phys_cpu_present_map' u  4.9%
include/asm-i386/mach-default/mach_apic.h: error: dereferencing pointer to  8.2%
include/asm-i386/mach-visws/do_timer.h: error: `i8259A_lock' undeclared (f 14.1%

Grant.

Re: 2.6.13-rc3 test: finding compile errors with make randconfig

[Adrian Bunk]

On Sun, Jul 24, 2005 at 04:28:54PM +1000, Grant Coady wrote:

> Greetings,

Hi Grant,

> Few days ago I compiled 241 random configurations of 2.6.13-rc3, today 
> I finally got around to parsing the results, top 40, sorted by name.  
> Percentage is error_builds / total_builds.
> 
> build script similar to:
> count=0
> while [ $((++count)) -le $limit ]; do
>         trial=$(printf %003d $count)
>         make randconfig
>         cp .config "$store/$trial-config"
>         make clean
>         make -j2 2> "$store/$trial-error"
> done
> 
> Curious whether this is worth doing, I'm about to start a run for 2.6.12.3, 
> any interesting errors I can find the particular config + error to recover 
> context.  Deliberately simplistic for traceability at the moment, truncated 
> error length for this post.
>...

it's generally useful, but the target kernel should be the latest -mm
kernel. 

And doing the compilations is really the trivial part of the work, the 
main work is to analyze what causes the build failures and sending 
patches.

> Grant.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: 2.6.13-rc3 test: finding compile errors with make randconfig

[Grant Coady]

On Sun, 24 Jul 2005 11:13:27 +0200, Adrian Bunk <bunk@stusta.de> wrote:
>
>it's generally useful, but the target kernel should be the latest -mm
>kernel. 
097-error:drivers/char/drm/drm_memory.h:163: error: redefinition of `drm_ioremap_nocache'
097-error:drivers/char/drm/drm_memory.h:163: error: `drm_ioremap_nocache' previously defined here
097-error:drivers/char/drm/drm_memory.h:174: error: redefinition of `drm_ioremapfree'
097-error:drivers/char/drm/drm_memory.h:174: error: `drm_ioremapfree' previously defined here
098-error:drivers/usb/gadget/ether.c:2510: error: `STATUS_BYTECOUNT' undeclared (first use in this function)
098-error:drivers/usb/gadget/ether.c:2510: error: (Each undeclared identifier is reported only once
098-error:drivers/usb/gadget/ether.c:2510: error: for each function it appears in.)
grant@sempro:/opt/linux/trial4$ grep error *-error |wc -l
2105

With > 2k (raw) errors in 97.something builds of 2.6.12.3, why go 
looking for trouble in -mm?  
>
>And doing the compilations is really the trivial part of the work, the 
Got to start somewhere :)  

Grant.

Re: 2.6.13-rc3 test: finding compile errors with make randconfig

[Jesper Juhl]

On 7/24/05, Grant Coady <lkml@dodo.com.au> wrote:
> On Sun, 24 Jul 2005 11:13:27 +0200, Adrian Bunk <bunk@stusta.de> wrote:
> 
> With > 2k (raw) errors in 97.something builds of 2.6.12.3, why go
> looking for trouble in -mm?

Because -mm is the development tree. The things in -mm are what's
eventually going to end up in mainline, so that's what you want to be
testing and fixing, and it's also further ahead than 2.6.12.3 (which
is esentially a dead branch except for critical fixes) so stuff may
already have been fixed there that was broken in 2.6.12.3

> >
> >And doing the compilations is really the trivial part of the work, the
> Got to start somewhere :)
> 
Right you are, and I for one am glad you do it. I build randconfig
kernels myself to look for trouble spots, but I can't get anywhere
near building 200+ configs. On a good day I may build 5 or 6
randconfigs of the latest kernel inbetween doing other things, so
getting hold of the results of several hundred randconfig builds gives
me a lot of material to work on that I would never have the time to
gather myself. Thanks.

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: 2.6.13-rc3 test: finding compile errors with make randconfig

[Adrian Bunk]

On Mon, Jul 25, 2005 at 05:42:58AM +1000, Grant Coady wrote:
> On Sun, 24 Jul 2005 11:13:27 +0200, Adrian Bunk <bunk@stusta.de> wrote:
> >
> >it's generally useful, but the target kernel should be the latest -mm
> >kernel. 
> 097-error:drivers/char/drm/drm_memory.h:163: error: redefinition of `drm_ioremap_nocache'
> 097-error:drivers/char/drm/drm_memory.h:163: error: `drm_ioremap_nocache' previously defined here
> 097-error:drivers/char/drm/drm_memory.h:174: error: redefinition of `drm_ioremapfree'
> 097-error:drivers/char/drm/drm_memory.h:174: error: `drm_ioremapfree' previously defined here

This requires the .config for debugging.

My first guess is that drm_memory.h requires a simple #ifdef to allow 
multiple inclusions.

> 098-error:drivers/usb/gadget/ether.c:2510: error: `STATUS_BYTECOUNT' undeclared (first use in this function)
> 098-error:drivers/usb/gadget/ether.c:2510: error: (Each undeclared identifier is reported only once
> 098-error:drivers/usb/gadget/ether.c:2510: error: for each function it appears in.)

Already fixed in 2.6.13-rc3.

> grant@sempro:/opt/linux/trial4$ grep error *-error |wc -l
> 2105
> 
> With > 2k (raw) errors in 97.something builds of 2.6.12.3, why go 
> looking for trouble in -mm?  
>...

You aren't running into problems that are already fixed (see your second 
example above).

> Grant.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: 2.6.13-rc3 test: finding compile errors with make randconfig

[Grant Coady]

On Sun, 24 Jul 2005 22:39:32 +0200, Adrian Bunk <bunk@stusta.de> wrote:

>On Mon, Jul 25, 2005 at 05:42:58AM +1000, Grant Coady wrote:
>> On Sun, 24 Jul 2005 11:13:27 +0200, Adrian Bunk <bunk@stusta.de> wrote:
>> >
>> >it's generally useful, but the target kernel should be the latest -mm
>> >kernel. 
>> 097-error:drivers/char/drm/drm_memory.h:163: error: redefinition of `drm_ioremap_nocache'
>> 097-error:drivers/char/drm/drm_memory.h:163: error: `drm_ioremap_nocache' previously defined here
>> 097-error:drivers/char/drm/drm_memory.h:174: error: redefinition of `drm_ioremapfree'
>> 097-error:drivers/char/drm/drm_memory.h:174: error: `drm_ioremapfree' previously defined here
>
>This requires the .config for debugging.
Here:
  ftp://ftp.scatter.mine.nu/develop/trial4-097-config.gz

>My first guess is that drm_memory.h requires a simple #ifdef to allow 
>multiple inclusions.

I can tell you:
--- linux-2.6.12.3b/drivers/char/drm/drm_memory.h.orig	2005-06-18 05:48:29.000000000 +1000
+++ linux-2.6.12.3b/drivers/char/drm/drm_memory.h	2005-07-25 06:57:41.000000000 +1000
@@ -33,6 +33,9 @@
  * OTHER DEALINGS IN THE SOFTWARE.
  */
 
+#ifndef DRM_MEMORY_H
+#define DRM_MEMORY_H
+
 #include <linux/config.h>
 #include <linux/highmem.h>
 #include <linux/vmalloc.h>
@@ -194,4 +197,5 @@
 	iounmap(pt);
 }
 
+#endif

does not fix it, though it's probably not what you had in mind, first try...
Simple fix didn't...  Now I got to read the code, takes a little more 
effort :)
...
>You aren't running into problems that are already fixed (see your second 
>example above).
I see your point, thanks for feedback.  

Grant.

Re: 2.6.13-rc3 test: finding compile errors with make randconfig

[Adrian Bunk]

On Mon, Jul 25, 2005 at 07:13:02AM +1000, Grant Coady wrote:
> On Sun, 24 Jul 2005 22:39:32 +0200, Adrian Bunk <bunk@stusta.de> wrote:
> 
> >On Mon, Jul 25, 2005 at 05:42:58AM +1000, Grant Coady wrote:
> >> On Sun, 24 Jul 2005 11:13:27 +0200, Adrian Bunk <bunk@stusta.de> wrote:
> >> >
> >> >it's generally useful, but the target kernel should be the latest -mm
> >> >kernel. 
> >> 097-error:drivers/char/drm/drm_memory.h:163: error: redefinition of `drm_ioremap_nocache'
> >> 097-error:drivers/char/drm/drm_memory.h:163: error: `drm_ioremap_nocache' previously defined here
> >> 097-error:drivers/char/drm/drm_memory.h:174: error: redefinition of `drm_ioremapfree'
> >> 097-error:drivers/char/drm/drm_memory.h:174: error: `drm_ioremapfree' previously defined here
> >
> >This requires the .config for debugging.
> Here:
>   ftp://ftp.scatter.mine.nu/develop/trial4-097-config.gz
> 
> >My first guess is that drm_memory.h requires a simple #ifdef to allow 
> >multiple inclusions.
> 
> I can tell you:
> --- linux-2.6.12.3b/drivers/char/drm/drm_memory.h.orig	2005-06-18 05:48:29.000000000 +1000
> +++ linux-2.6.12.3b/drivers/char/drm/drm_memory.h	2005-07-25 06:57:41.000000000 +1000
> @@ -33,6 +33,9 @@
>   * OTHER DEALINGS IN THE SOFTWARE.
>   */
>  
> +#ifndef DRM_MEMORY_H
> +#define DRM_MEMORY_H
> +
>  #include <linux/config.h>
>  #include <linux/highmem.h>
>  #include <linux/vmalloc.h>
> @@ -194,4 +197,5 @@
>  	iounmap(pt);
>  }
>  
> +#endif
> 
> does not fix it, though it's probably not what you had in mind, first try...

That's what I had in mind.

> Simple fix didn't...  Now I got to read the code, takes a little more 
> effort :)
>...

Looking at the .config, the problem is actually:
  CONFIG_BROKEN=y

You should edit init/Kconfig to disallow CONFIG_CLEAN_COMPILE=n, since 
any errors you see with CONFIG_BROKEN=y aren't interesting.

> Grant.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: 2.6.13-rc3 test: finding compile errors with make randconfig

[Grant Coady]

On Sun, 24 Jul 2005 23:27:22 +0200, Adrian Bunk <bunk@stusta.de> wrote:
>Looking at the .config, the problem is actually:
>  CONFIG_BROKEN=y
>
>You should edit init/Kconfig to disallow CONFIG_CLEAN_COMPILE=n, since 
>any errors you see with CONFIG_BROKEN=y aren't interesting.

Very good point.

$ grep CONFIG_BROKEN=y *config |wc -l
24

>From 106 .configs, trap for young players...  I'll skip compiling 
"CONFIG_BROKEN=y" .configs then.

Thanks,
Grant.

Re: 2.6.13-rc3 test: finding compile errors with make randconfig

[Grant Coady]

On Sun, 24 Jul 2005 23:27:22 +0200, Adrian Bunk <bunk@stusta.de> wrote:
>
>You should edit init/Kconfig to disallow CONFIG_CLEAN_COMPILE=n, since 
>any errors you see with CONFIG_BROKEN=y aren't interesting.

Straight over the top of my head yesterday :)  Is the following 
what you had in mind?  (current script does retry if BROKEN)

grant@sempro:/opt/linux$ diff -u linux-2.6.13-rc3-git6/init/Kconfig~ linux-2.6.13-rc3-git6/init/Kconfig
--- linux-2.6.13-rc3-git6/init/Kconfig~ 2005-07-25 08:23:04.000000000 +1000
+++ linux-2.6.13-rc3-git6/init/Kconfig  2005-07-26 10:25:59.000000000 +1000
@@ -42,7 +42,7 @@

 config BROKEN
        bool
-       depends on !CLEAN_COMPILE
+       depends on !CLEAN_COMPILE && 0
        default y

 config BROKEN_ON_SMP

- - - 
results first run:
------------------
Here http://scatter.mine.nu/test/scripts/counterror-2005-07-26.gz
is post-processing script (work in progress) extracts errors and 
non-deprecated warnings, linked to first triggering .config like so:

grant@sempro:/opt/linux$ cat error-error-list
trial1/run-004:arch/i386/mach-es7000/es7000.h
trial1/run-004:arch/i386/mach-es7000/es7000plat.c
trial1/run-009:drivers/char/ipmi/ipmi_msghandler.c
trial2/run-082:drivers/mtd/maps/nettel.c
trial3/run-018:drivers/pnp/pnpbios/rsparser.c
trial2/run-044:include/asm-i386/mach-default/mach_apic.h
trial1/run-008:include/asm-i386/mach-visws/do_timer.h
trial2/run-081:ipc/shm.c
trial1/run-015:net/rxrpc/main.c
trial2/run-027:sound/core/memalloc.c

then extracts compiler messages:

grant@sempro:/opt/linux$ head error-error-report |cut -c-78
trial1/run-004:arch/i386/mach-es7000/es7000.h:82: error: field `id' has incomp
trial1/run-004:arch/i386/mach-es7000/es7000.h:88: error: field `Header' has in
trial1/run-004:arch/i386/mach-es7000/es7000plat.c: In function `parse_unisys_o
trial1/run-004:arch/i386/mach-es7000/es7000plat.c:154: error: `es7000_rename_g
trial1/run-004:arch/i386/mach-es7000/es7000plat.c: In function `find_unisys_ac
trial1/run-004:arch/i386/mach-es7000/es7000plat.c:168: warning: implicit decla
trial1/run-004:arch/i386/mach-es7000/es7000plat.c:170: error: dereferencing po
trial1/run-004:arch/i386/mach-es7000/es7000plat.c:172: error: dereferencing po
trial1/run-004:arch/i386/mach-es7000/es7000plat.c:175: warning: implicit decla
trial1/run-004:arch/i386/mach-es7000/es7000plat.c:175: error: invalid applicat
^^^^^^^^^^^^^^\
                --> link to .config producing the error / warning:

grant@sempro:/opt/linux$ head trial1/run-004-config
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.13-rc3
# Tue Jul 19 04:31:27 2005
#
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y

Current compile run is with linux-2.6.13-rc3-git6

Datastore is denormalised flat text.  Query language: grep :)

Thanks,
Grant.

Re: 2.6.13-rc3 test: finding compile errors with make randconfig

[Adrian Bunk]

On Tue, Jul 26, 2005 at 11:26:33AM +1000, Grant Coady wrote:
> On Sun, 24 Jul 2005 23:27:22 +0200, Adrian Bunk <bunk@stusta.de> wrote:
> >
> >You should edit init/Kconfig to disallow CONFIG_CLEAN_COMPILE=n, since 
> >any errors you see with CONFIG_BROKEN=y aren't interesting.
> 
> Straight over the top of my head yesterday :)  Is the following 
> what you had in mind?  (current script does retry if BROKEN)
>... 
> -       depends on !CLEAN_COMPILE
> +       depends on !CLEAN_COMPILE && 0
>...

I don't know whether this will work, I was thinking about

--- linux-2.6.13-rc3-mm1/init/Kconfig.old	2005-07-26 11:47:49.000000000 +0200
+++ linux-2.6.13-rc3-mm1/init/Kconfig	2005-07-26 11:48:01.000000000 +0200
@@ -32,7 +32,7 @@
 	  drivers that are currently considered to be in the alpha-test phase.
 
 config CLEAN_COMPILE
-	bool "Select only drivers expected to compile cleanly" if EXPERIMENTAL
+	bool
 	default y
 	help
 	  Select this option if you don't even want to see the option


> Thanks,
> Grant.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: 2.6.13-rc3 test: finding compile errors with make randconfig

[Jesper Juhl]

On 7/24/05, Grant Coady <lkml@dodo.com.au> wrote:
> Greetings,
> 
> Few days ago I compiled 241 random configurations of 2.6.13-rc3, today
> I finally got around to parsing the results, top 40, sorted by name.
> Percentage is error_builds / total_builds.
> 
> build script similar to:
> count=0
> while [ $((++count)) -le $limit ]; do
>         trial=$(printf %003d $count)
>         make randconfig
>         cp .config "$store/$trial-config"
>         make clean
>         make -j2 2> "$store/$trial-error"
> done
> 
> Curious whether this is worth doing, I'm about to start a run for 2.6.12.3,
> any interesting errors I can find the particular config + error to recover
> context.  Deliberately simplistic for traceability at the moment, truncated
> error length for this post.
> 
If you could put the data online somewhere I'd be interrested in
taking a look at it.
An easy way to look at the build log and grab the matching .config for
any given run would be great.

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

Re: 2.6.13-rc3 test: finding compile errors with make randconfig

[Grant Coady]

On Sun, 24 Jul 2005 15:01:22 +0200, Jesper Juhl <jesper.juhl@gmail.com> wrote:
>> context.  Deliberately simplistic for traceability at the moment, truncated
>> error length for this post.
>> 
>If you could put the data online somewhere I'd be interrested in
>taking a look at it.
7.4MB raw data --> low info content.  Needs garbage removal.  Good 
test case for gzip vs bzip2 --> 1.4MB vs 481kB, 

  ftp://ftp.scatter.mine.nu/develop/first_run.tar.bz2 (481kB)

If you mean online info-sys, I don't have bandwidth for that :(

>An easy way to look at the build log and grab the matching .config for
>any given run would be great.

Revisit the data extraction and build an errorlog line_index...  
Will let you know.  

Grant.

Re: 2.6.13-rc3 test: finding compile errors with make randconfig

[Jesper Juhl]

On 7/24/05, Grant Coady <lkml@dodo.com.au> wrote:
> On Sun, 24 Jul 2005 15:01:22 +0200, Jesper Juhl <jesper.juhl@gmail.com> wrote:
> >> context.  Deliberately simplistic for traceability at the moment, truncated
> >> error length for this post.
> >>
> >If you could put the data online somewhere I'd be interrested in
> >taking a look at it.
> 7.4MB raw data --> low info content.  Needs garbage removal.  Good
> test case for gzip vs bzip2 --> 1.4MB vs 481kB,
> 
>   ftp://ftp.scatter.mine.nu/develop/first_run.tar.bz2 (481kB)
> 
> If you mean online info-sys, I don't have bandwidth for that :(
> 
Ok. Would you be able to bzip2 the raw data and email it to me off list ?


-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html