Re: page fault scalability patch V12 [0/7]: Overview and performance tests

Martin J. Bligh
2004-12-02T07:26:59Z

--Andrew Morton <akpm@osdl.org> wrote (on Wednesday, December 01, 2004 23:02:17 -0800):

> Jeff Garzik <jgarzik@pobox.com> wrote:
>>
>> Andrew Morton wrote:
>> > We need to be be achieving higher-quality major releases than we did in
>> > 2.6.8 and 2.6.9. Really the only tool we have to ensure this is longer
>> > stabilisation periods.
>>
>>
>> I'm still hoping that distros (like my employer) and orgs like OSDL will
>> step up, and hook 2.6.x BK snapshots into daily test harnesses.
>
> I believe that both IBM and OSDL are doing this, or are getting geared up
> to do this. With both Linus bk and -mm.

I already run a bunch of tests on a variety of machines for every new
kernel ... but don't have an automated way to compare the results as yet,
so don't actually look at them much ;-(. Sometime soon (quite possibly over
Christmas) things will calm down enough I'll get a couple of days to write
the appropriate perl script, and start publishing stuff.

> However I have my doubts about how useful it will end up being. These test
> suites don't seem to pick up many regressions. I've challenged Gerrit to
> go back through a release cycle's bugfixes and work out how many of those
> bugs would have been detected by the test suite.
>
> My suspicion is that the answer will be "a very small proportion", and that
> really is the bottom line.

Yeah, probably. Though the stress tests catch a lot more than the
functionality ones. The big pain in the ass is drivers, because I don't
have a hope in hell of testing more than 1% of them.

M.