Re: test suite?

Re: test suite?

[Randy.Dunlap]

| From: Robert P. J. Day <rpjday@mindspring.com>
|
|   i've noticed references to "test suites" for kernels, but
| is there any one-step convenient way to select every possible
| option for test-compiling a new kernel, just to see if it builds?
| perhaps an "everything" option?
|
|   and, related to that, should such a kernel theoretically
| work?  as in, are there any options that would be mutually
| exclusive that would cause such a build to fail?
|
| still thinking about reorganizing the overall option structure,

Hi,

I notice that you've already had a reply on this (use "make help",
"make allmodconfig" or "make allyesconfig" etc.).

OSDL's PLM does this (make allmodconfig) for each new (2.5) kernel
release and the results are posted at
  http://www.osdl.org/archive/cherry/stability/
and also in the PLM web page interface.
The script that is used for this is at that same URL above.
PLM is at http://www.osdl.org/cgi-bin/plm/ .

Has anyone tried to boot an 'allyesconfig' kernel?
I'll give that a shot now.

HTH.
-- 
~Randy

Re: test suite?

[Randy.Dunlap]

On Fri, 24 Jan 2003, Randy.Dunlap wrote:

| | From: Robert P. J. Day <rpjday@mindspring.com>
| |
| |   i've noticed references to "test suites" for kernels, but
| | is there any one-step convenient way to select every possible
| | option for test-compiling a new kernel, just to see if it builds?
| | perhaps an "everything" option?
| |
| |   and, related to that, should such a kernel theoretically
| | work?  as in, are there any options that would be mutually
| | exclusive that would cause such a build to fail?

I tried this...see below.
Short answer is that the resulting kernel is considered too large,
but I don't know by what.
Anyone, where is this kernel size limit coming from?

| | still thinking about reorganizing the overall option structure,
|
| Hi,
|
| I notice that you've already had a reply on this (use "make help",
| "make allmodconfig" or "make allyesconfig" etc.).
|
| OSDL's PLM does this (make allmodconfig) for each new (2.5) kernel
| release and the results are posted at
|   http://www.osdl.org/archive/cherry/stability/
| and also in the PLM web page interface.
| The script that is used for this is at that same URL above.
| PLM is at http://www.osdl.org/cgi-bin/plm/ .
|
| Has anyone tried to boot an 'allyesconfig' kernel?
| I'll give that a shot now.

On a dual-proc P4 x86 machine, with 2.5.59, I did 'make allyesconfig'
and tried to build the kernel.  There were several classes of problems.

a.  many syntax errors (> 50 source files):  disabled these configs

b.  many linker errors (> 50 object files):  disabled these configs
    (mostly references to save_flags/restore_flags etc.)

c.  One particular error implies that 2 frame buffer options are
    mutually exclusive:  linking hgafb and fbmem give 6 duplicate
    symbols (all of the form 'linux_logo*').

d.  There were 4 cases of referenced symbols being discarded or being
    outside of the referencing code section.  I used Keith's
    'reference_discarded.pl' script and fixed these (or tried to).
    I will post a separate email for these.

e.  Now the only remaining error is this:
  arch/i386/boot/tools/build -b arch/i386/boot/bootsect
  arch/i386/boot/setup arch/i386/boot/vmlinux.bin CURRENT >
    arch/i386/boot/bzImage
  Root device is (3, 4)
  Boot sector 512 bytes.
  Setup is 4880 bytes.
  System is 8384 kB
  System is too big. Try using modules.
  make[1]: *** [arch/i386/boot/bzImage] Error 1
  make: *** [bzImage] Error 2

-- 
~Randy

Re: test suite?

[Hugh Dickins]

On Fri, 24 Jan 2003, Randy.Dunlap wrote:
> Anyone, where is this kernel size limit coming from?
>   System is 8384 kB
>   System is too big. Try using modules.

See pg0 and pg1 in arch/i386/kernel/head.S.  There's no technical
reason (but well-justified resistance to bloat) why pg2... cannot
be added, but you might find a few little adjustments needed to
match elsewhere (if you want your testbuild kernel to boot).

Hugh

Re: test suite?

[Randy.Dunlap]

On Sat, 25 Jan 2003, Hugh Dickins wrote:

| On Fri, 24 Jan 2003, Randy.Dunlap wrote:
| > Anyone, where is this kernel size limit coming from?
| >   System is 8384 kB
| >   System is too big. Try using modules.
|
| See pg0 and pg1 in arch/i386/kernel/head.S.  There's no technical
| reason (but well-justified resistance to bloat) why pg2... cannot
| be added, but you might find a few little adjustments needed to
| match elsewhere (if you want your testbuild kernel to boot).

Thanks, Hugh.

Just after I sent that email, I said to myself:
um, maybe something about GDT, related to (max) 1-1 memory mapping.

No, I don't need to boot that kernel.
It was enough of a problem just to build it.

-- 
~Randy

Re: test suite?

[William Lee Irwin III]

On Fri, 24 Jan 2003, Randy.Dunlap wrote:
>> Anyone, where is this kernel size limit coming from?
>>   System is 8384 kB
>>   System is too big. Try using modules.

On Sat, Jan 25, 2003 at 08:22:10PM +0000, Hugh Dickins wrote:
> See pg0 and pg1 in arch/i386/kernel/head.S.  There's no technical
> reason (but well-justified resistance to bloat) why pg2... cannot
> be added, but you might find a few little adjustments needed to
> match elsewhere (if you want your testbuild kernel to boot).

It's actually a relatively annoying limit, as various boxen's MP tables
ACPI tables etc. etc. are well above 8MB. At some point one of us should
quash that issue permanently and dynamically map the things. IIRC Linux
needs NUMA-Q boxen to get lynxers reflashed to move MP tables below 8MB
to boot atm. mbligh can more accurate describe the pain involved there.
As for the bzImage, don't bother supporting excess bloat.

All I can say is, "bring me the head of the first moron who shoves some
table needed at boot-time above 4GB".


-- wli

Re: test suite?

[Martin J. Bligh]

> It's actually a relatively annoying limit, as various boxen's MP tables
> ACPI tables etc. etc. are well above 8MB. At some point one of us should
> quash that issue permanently and dynamically map the things. IIRC Linux
> needs NUMA-Q boxen to get lynxers reflashed to move MP tables below 8MB
> to boot atm. mbligh can more accurate describe the pain involved there.
> As for the bzImage, don't bother supporting excess bloat.

One of these days, I just need to make it remap the MPS tables with
set_fixmap instead, would be a damned sight easier. Was quicker just
to hack the firmware at the time, but it's not the RightThingToDo.

M.

RE: Test suite?

[Villalovos, John L]

Thanks for the info.  I will try to play around with it and see if it
helps :)

John


> -----Original Message-----
> From: Nikita Danilov [mailto:Nikita@Namesys.COM] 
> Sent: Thursday, January 22, 2004 5:24 AM
> To: markw@osdl.org
> Cc: Villalovos, John L; reiserfs-list@Namesys.COM
> Subject: Re: Test suite?
> 
> 
> markw@osdl.org writes:
>  > On 21 Jan, Villalovos, John L wrote:
>  > > Is there a test suite for ReiserFS?
>  > > 
>  > > For regression testing and the like?
> 
> We use various stress tests and benchmarks for this:
> 
> bonnie   (http://www.textuality.com/bonnie/)
> bonnie++ (http://www.coker.com.au/bonnie++/)
> fsx      (http://www.zip.com.au/~akpm/linux/patches/stuff/fsx-linux.c)
> iozone   (http://www.iozone.org/)
> dbench   (http://freshmeat.net/projects/dbench/)
> mongo    (http://www.namesys.com/benchmarks/mongo_readme.html)
> cerberus (http://sourceforge.net/projects/va-ctcs/)
> fsstress 
> (http://cvs.sourceforge.net/viewcvs.py/ltp/ltp/testcases/kerne
l/fs/fsstress/)

And some scripts/programs that were created specifically on purpose. I
put few of them (without any support or documentation) into
ftp://ftp.namesys.com/pub/testtools

Plus scripts to start a bunch of concurrent kernel compilations,
bitkeeper pulls, tars, gzips, etc.

 > > 
 > > I am just wondering if there is something that I can run after
making
 > > modifications that would help give me a warm-fuzzy feeling :)
 > > 
 > > John
 > > 

Nikita.

RE: Test suite?

[Villalovos, John L]

Thanks for the info.

I will check it out.

John



> -----Original Message-----
> From: Mike Fedyk [mailto:mfedyk@srv-lnx2600.matchmail.com] On 
> Behalf Of Mike Fedyk
> Sent: Wednesday, January 21, 2004 1:06 PM
> To: Villalovos, John L
> Cc: reiserfs-list@namesys.com
> Subject: Re: Test suite?
> 
> 
> On Wed, Jan 21, 2004 at 11:06:15AM -0800, Villalovos, John L wrote:
> > Is there a test suite for ReiserFS?
> > 
> > For regression testing and the like?
> > 
> > I am just wondering if there is something that I can run 
> after making
> > modifications that would help give me a warm-fuzzy feeling :)
> 
> I know namesys at least mentioned having a test that shows if 
> they'll have
> problems serving files over nfs, but I don't remember if it's 
> posted for the
> public.
> 

Test suite?

[Villalovos, John L]

Is there a test suite for ReiserFS?

For regression testing and the like?

I am just wondering if there is something that I can run after making
modifications that would help give me a warm-fuzzy feeling :)

John

John Villalovos
Intel Corporation 
15400 NW GREENBRIER PKWY # CO5-162
BEAVERTON, OR  97006-5723
(503) 677-5777   Fax: (503) 677-6670

My opinions are my own and not necessarily the opinions of Intel
Corporation.

Re: Test suite?

[markw]

On 21 Jan, Villalovos, John L wrote:
> Is there a test suite for ReiserFS?
> 
> For regression testing and the like?
> 
> I am just wondering if there is something that I can run after making
> modifications that would help give me a warm-fuzzy feeling :)
> 
> John
> 
> John Villalovos
> Intel Corporation 
> 15400 NW GREENBRIER PKWY # CO5-162
> BEAVERTON, OR  97006-5723
> (503) 677-5777   Fax: (503) 677-6670
> 
> My opinions are my own and not necessarily the opinions of Intel
> Corporation.

Hi John,

We have some tests at the OSDL on our Scalable Test Platform you might
be able to use of.  Here's a link to the web page:
	http://www.osdl.org/lab_activities/kernel_testing/stp/

If that interests you, let me know if you have any additional questions.

Mark

Re: Test suite?

[Nikita Danilov]

markw@osdl.org writes:
 > On 21 Jan, Villalovos, John L wrote:
 > > Is there a test suite for ReiserFS?
 > > 
 > > For regression testing and the like?

We use various stress tests and benchmarks for this:

bonnie   (http://www.textuality.com/bonnie/)
bonnie++ (http://www.coker.com.au/bonnie++/)
fsx      (http://www.zip.com.au/~akpm/linux/patches/stuff/fsx-linux.c)
iozone   (http://www.iozone.org/)
dbench   (http://freshmeat.net/projects/dbench/)
mongo    (http://www.namesys.com/benchmarks/mongo_readme.html)
cerberus (http://sourceforge.net/projects/va-ctcs/)
fsstress (http://cvs.sourceforge.net/viewcvs.py/ltp/ltp/testcases/kernel/fs/fsstress/)

And some scripts/programs that were created specifically on purpose. I
put few of them (without any support or documentation) into
ftp://ftp.namesys.com/pub/testtools

Plus scripts to start a bunch of concurrent kernel compilations,
bitkeeper pulls, tars, gzips, etc.

 > > 
 > > I am just wondering if there is something that I can run after making
 > > modifications that would help give me a warm-fuzzy feeling :)
 > > 
 > > John
 > > 

Nikita.

Re: Test suite?

[Hans Reiser]

Nikita Danilov wrote:

>markw@osdl.org writes:
> > On 21 Jan, Villalovos, John L wrote:
> > > Is there a test suite for ReiserFS?
> > > 
> > > For regression testing and the like?
>
>We use various stress tests and benchmarks for this:
>
>bonnie   (http://www.textuality.com/bonnie/)
>bonnie++ (http://www.coker.com.au/bonnie++/)
>fsx      (http://www.zip.com.au/~akpm/linux/patches/stuff/fsx-linux.c)
>iozone   (http://www.iozone.org/)
>dbench   (http://freshmeat.net/projects/dbench/)
>mongo    (http://www.namesys.com/benchmarks/mongo_readme.html)
>cerberus (http://sourceforge.net/projects/va-ctcs/)
>fsstress (http://cvs.sourceforge.net/viewcvs.py/ltp/ltp/testcases/kernel/fs/fsstress/)
>
>And some scripts/programs that were created specifically on purpose. I
>put few of them (without any support or documentation) into
>ftp://ftp.namesys.com/pub/testtools
>
>Plus scripts to start a bunch of concurrent kernel compilations,
>bitkeeper pulls, tars, gzips, etc.
>
> > > 
> > > I am just wondering if there is something that I can run after making
> > > modifications that would help give me a warm-fuzzy feeling :)
> > > 
> > > John
> > > 
>
>Nikita.
>
>
>  
>
fsx is probably the most thorough single test

-- 
Hans

Re: Test suite?

[Yusuf Goolamabbas]

Could you also consider fstress which provides for a SPECsfs97 like
workload

http://www.cs.duke.edu/ari/fstress/

Regards, Yusuf
-- 
If you're not using Firebird, you're not surfing the web 
   you're suffering it
http://www.mozilla.org/products/firebird/why/

Re: Test suite?

[Nikita Danilov]

Yusuf Goolamabbas writes:
 > Could you also consider fstress which provides for a SPECsfs97 like
 > workload

I know about it, but it claims to be NFS-server oriented benchmark.

Probably we should try it anyway, thanks for the pointer.

 > 
 > http://www.cs.duke.edu/ari/fstress/
 > 
 > Regards, Yusuf

Nikita.

Re: Test suite?

[Mike Fedyk]

On Wed, Jan 21, 2004 at 11:06:15AM -0800, Villalovos, John L wrote:
> Is there a test suite for ReiserFS?
> 
> For regression testing and the like?
> 
> I am just wondering if there is something that I can run after making
> modifications that would help give me a warm-fuzzy feeling :)

I know namesys at least mentioned having a test that shows if they'll have
problems serving files over nfs, but I don't remember if it's posted for the
public.

test suite?

[Robert P. J. Day]

  i've noticed references to "test suites" for kernels, but
is there any one-step convenient way to select every possible
option for test-compiling a new kernel, just to see if it builds?
perhaps an "everything" option?

  and, related to that, should such a kernel theoretically
work?  as in, are there any options that would be mutually
exclusive that would cause such a build to fail?

still thinking about reorganizing the overall option structure,
rday

Re: test suite?

[Sam Ravnborg]

On Wed, Jan 22, 2003 at 08:44:05AM -0500, Robert P. J. Day wrote:
> 
>   i've noticed references to "test suites" for kernels, but
> is there any one-step convenient way to select every possible
> option for test-compiling a new kernel, just to see if it builds?
> perhaps an "everything" option?

Try "make help" some day..

make allyesconfig
make allmodconfig
make allnoconfig <- Opposite of what you ask for.

	Sam

Re: test suite?

[Robert P. J. Day]

On Wed, 22 Jan 2003, Sam Ravnborg wrote:

> On Wed, Jan 22, 2003 at 08:44:05AM -0500, Robert P. J. Day wrote:
> > 
> >   i've noticed references to "test suites" for kernels, but
> > is there any one-step convenient way to select every possible
> > option for test-compiling a new kernel, just to see if it builds?
> > perhaps an "everything" option?
> 
> Try "make help" some day..
> 
> make allyesconfig
> make allmodconfig
> make allnoconfig <- Opposite of what you ask for.
> 
> 	Sam

thanks for the info.  i assume the condescension was no extra charge.

rday

Re: test suite?

[Cliff White]

> 
>   i've noticed references to "test suites" for kernels, but
> is there any one-step convenient way to select every possible
> option for test-compiling a new kernel, just to see if it builds?
> perhaps an "everything" option?
> 
>   and, related to that, should such a kernel theoretically
> work?  as in, are there any options that would be mutually
> exclusive that would cause such a build to fail?
> 
> still thinking about reorganizing the overall option structure,
> rday

The OSDL's patch lifecycle manager (http://www.osdl.org/cgi-bin/plm ) 
does something like this. It does a series of builds include a few that 
select most every option. So you could toss the patch into PLM and look at
the results of those test compiles.  In particular, the 'Desktop' config
turns on a ton of stuff ( and almost always fails ;)
We also do a 'regress' build, where we do 'make defconfig' and 
'make allmodconfig' and count the errors.

I don't think we have any more detailed docs, but the test report does include
the .config files we run.
cliffw

> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: test suite?

[Adrian Bunk]

On Wed, Jan 22, 2003 at 08:44:05AM -0500, Robert P. J. Day wrote:
> 
>   i've noticed references to "test suites" for kernels, but
> is there any one-step convenient way to select every possible
> option for test-compiling a new kernel, just to see if it builds?
> perhaps an "everything" option?
> 
>   and, related to that, should such a kernel theoretically
> work?  as in, are there any options that would be mutually
> exclusive that would cause such a build to fail?

It shouldn't be possible to select mutually exclusive options.


I'm using three .config's:

1. CONFIG_MODULES not set, everything else turned on
2. CONFIG_MODULES and CONFIG_HOTPLUG not set, everything else turned on
3. as much as possible modular

With 2.4.20 there are no compile errors with the first two and only two
errors at "depmod" with the third one (one of these two compile errors
is clearly pathological).

The bigger problem are unusual .config's, e.g. if you turn off common
things like CONFIG_PCI or CONFIG_NET.

Another example is that in 2.4.20 there's a problem with the compilation
of one specific net driver (fixed in 2.4.21-pre) if you select this
specific net driver and not additionally one or more of several other
net drivers.

Unfortunately there doesn't seem to be a good solution for this problem
that doesn't need O(n^2) .config's for n kernel options.


> still thinking about reorganizing the overall option structure,
> rday

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed