* [BENCHMARK] 2.5.73-bk3, -bk4, -mjb1 regression test results
@ 2003-06-27 18:34 Mark Peloquin
2003-06-27 18:42 ` Jeff Garzik
2003-06-27 18:43 ` Dave Hansen
0 siblings, 2 replies; 5+ messages in thread
From: Mark Peloquin @ 2003-06-27 18:34 UTC (permalink / raw)
To: linux-kernel, linstab, ltp-results
Nightly Regression Summary
for
2.5.73-bk2 vs 2.5.73-bk3
Benchmark Pass/Fail Improvements Regressions Results Results Summary
--------------- --------- ------------ ----------- ----------- ----------- -------
dbench.ext2 P N N 2.5.73-bk2 2.5.73-bk3 report
dbench.ext3 P Y N 2.5.73-bk2 2.5.73-bk3 report
dbench.jfs P N N 2.5.73-bk2 2.5.73-bk3 report
dbench.reiser P N N 2.5.73-bk2 2.5.73-bk3 report
dbench.xfs P N N 2.5.73-bk2 2.5.73-bk3 report
kernbench P N N 2.5.73-bk2 2.5.73-bk3 report
lmbench P Y Y 2.5.73-bk2 2.5.73-bk3 report
rawiobench P Y Y 2.5.73-bk2 2.5.73-bk3 report
specjbb P Y N 2.5.73-bk2 2.5.73-bk3 report
specsdet P N N 2.5.73-bk2 2.5.73-bk3 report
tbench P Y Y 2.5.73-bk2 2.5.73-bk3 report
tiobench.ext2 P N N 2.5.73-bk2 2.5.73-bk3 report
tiobench.ext3 P N Y 2.5.73-bk2 2.5.73-bk3 report
tiobench.jfs P N N 2.5.73-bk2 2.5.73-bk3 report
tiobench.reiser P N Y 2.5.73-bk2 2.5.73-bk3 report
tiobench.xfs P N N 2.5.73-bk2 2.5.73-bk3 report
volanomark P N N 2.5.73-bk2 2.5.73-bk3 report
http://ltcperf.ncsa.uiuc.edu/data/2.5.73-bk3/2.5.73-bk2-vs-2.5.73-bk3/
Nightly Regression Summary
for
2.5.73 vs 2.5.73-bk3
Benchmark Pass/Fail Improvements Regressions Results Results Summary
--------------- --------- ------------ ----------- ----------- ----------- -------
dbench.ext2 P N N 2.5.73 2.5.73-bk3 report
dbench.ext3 P N N 2.5.73 2.5.73-bk3 report
dbench.jfs P N N 2.5.73 2.5.73-bk3 report
dbench.reiser P N N 2.5.73 2.5.73-bk3 report
dbench.xfs P N N 2.5.73 2.5.73-bk3 report
kernbench P N N 2.5.73 2.5.73-bk3 report
lmbench P Y Y 2.5.73 2.5.73-bk3 report
rawiobench P Y Y 2.5.73 2.5.73-bk3 report
specjbb P Y Y 2.5.73 2.5.73-bk3 report
specsdet P Y N 2.5.73 2.5.73-bk3 report
tbench P N Y 2.5.73 2.5.73-bk3 report
tiobench.ext2 P N N 2.5.73 2.5.73-bk3 report
tiobench.ext3 P Y N 2.5.73 2.5.73-bk3 report
tiobench.jfs P N N 2.5.73 2.5.73-bk3 report
tiobench.reiser P N N 2.5.73 2.5.73-bk3 report
tiobench.xfs P N N 2.5.73 2.5.73-bk3 report
volanomark P N N 2.5.73 2.5.73-bk3 report
http://ltcperf.ncsa.uiuc.edu/data/2.5.73-bk3/2.5.73-vs-2.5.73-bk3/
Nightly Regression Summary
for
2.5.73-bk3 vs 2.5.73-bk4
Benchmark Pass/Fail Improvements Regressions Results Results Summary
--------------- --------- ------------ ----------- ----------- ----------- -------
dbench.ext2 P N N 2.5.73-bk3 2.5.73-bk4 report
dbench.ext3 P N Y 2.5.73-bk3 2.5.73-bk4 report
dbench.jfs P N N 2.5.73-bk3 2.5.73-bk4 report
dbench.reiser P N N 2.5.73-bk3 2.5.73-bk4 report
dbench.xfs P N N 2.5.73-bk3 2.5.73-bk4 report
kernbench P N N 2.5.73-bk3 2.5.73-bk4 report
lmbench P Y Y 2.5.73-bk3 2.5.73-bk4 report
rawiobench P Y Y 2.5.73-bk3 2.5.73-bk4 report
specjbb P N Y 2.5.73-bk3 2.5.73-bk4 report
specsdet P N N 2.5.73-bk3 2.5.73-bk4 report
tbench P Y Y 2.5.73-bk3 2.5.73-bk4 report
tiobench.ext2 P N N 2.5.73-bk3 2.5.73-bk4 report
tiobench.ext3 P N Y 2.5.73-bk3 2.5.73-bk4 report
tiobench.jfs P N Y 2.5.73-bk3 2.5.73-bk4 report
tiobench.reiser P N Y 2.5.73-bk3 2.5.73-bk4 report
tiobench.xfs P N Y 2.5.73-bk3 2.5.73-bk4 report
volanomark P N N 2.5.73-bk3 2.5.73-bk4 report
http://ltcperf.ncsa.uiuc.edu/data/2.5.73-bk4/2.5.73-bk3-vs-2.5.73-bk4/
Nightly Regression Summary
for
2.5.73 vs 2.5.73-bk4
Benchmark Pass/Fail Improvements Regressions Results Results Summary
--------------- --------- ------------ ----------- ----------- ----------- -------
dbench.ext2 P N N 2.5.73 2.5.73-bk4 report
dbench.ext3 P N N 2.5.73 2.5.73-bk4 report
dbench.jfs P N N 2.5.73 2.5.73-bk4 report
dbench.reiser P N N 2.5.73 2.5.73-bk4 report
dbench.xfs P N N 2.5.73 2.5.73-bk4 report
kernbench P N N 2.5.73 2.5.73-bk4 report
lmbench P Y Y 2.5.73 2.5.73-bk4 report
rawiobench P Y Y 2.5.73 2.5.73-bk4 report
specjbb P Y Y 2.5.73 2.5.73-bk4 report
specsdet P Y N 2.5.73 2.5.73-bk4 report
tbench P Y Y 2.5.73 2.5.73-bk4 report
tiobench.ext2 P N Y 2.5.73 2.5.73-bk4 report
tiobench.ext3 P N Y 2.5.73 2.5.73-bk4 report
tiobench.jfs P N Y 2.5.73 2.5.73-bk4 report
tiobench.reiser P N Y 2.5.73 2.5.73-bk4 report
tiobench.xfs P N Y 2.5.73 2.5.73-bk4 report
volanomark P N N 2.5.73 2.5.73-bk4 report
http://ltcperf.ncsa.uiuc.edu/data/2.5.73-bk4/2.5.73-vs-2.5.73-bk4/
Nightly Regression Summary
for
2.5.73 vs 2.5.73-mjb1
Benchmark Pass/Fail Improvements Regressions Results Results Summary
--------------- --------- ------------ ----------- ----------- ----------- -------
dbench.ext2 P N N 2.5.73 2.5.73-mjb1 report
dbench.ext3 P N Y 2.5.73 2.5.73-mjb1 report
dbench.jfs P N N 2.5.73 2.5.73-mjb1 report
dbench.reiser P Y N 2.5.73 2.5.73-mjb1 report
dbench.xfs P N N 2.5.73 2.5.73-mjb1 report
kernbench P N N 2.5.73 2.5.73-mjb1 report
lmbench P Y Y 2.5.73 2.5.73-mjb1 report
rawiobench P Y Y 2.5.73 2.5.73-mjb1 report
specjbb P N Y 2.5.73 2.5.73-mjb1 report
specsdet P Y N 2.5.73 2.5.73-mjb1 report
tbench P N Y 2.5.73 2.5.73-mjb1 report
tiobench.ext2 P Y N 2.5.73 2.5.73-mjb1 report
tiobench.ext3 P Y Y 2.5.73 2.5.73-mjb1 report
tiobench.jfs P N Y 2.5.73 2.5.73-mjb1 report
tiobench.reiser P N N 2.5.73 2.5.73-mjb1 report
tiobench.xfs P Y Y 2.5.73 2.5.73-mjb1 report
volanomark P N N 2.5.73 2.5.73-mjb1 report
http://ltcperf.ncsa.uiuc.edu/data/2.5.73-mjb1/2.5.73-vs-2.5.73-mjb1/
Mark
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [BENCHMARK] 2.5.73-bk3, -bk4, -mjb1 regression test results
2003-06-27 18:34 [BENCHMARK] 2.5.73-bk3, -bk4, -mjb1 regression test results Mark Peloquin
@ 2003-06-27 18:42 ` Jeff Garzik
2003-06-27 18:50 ` Mark Peloquin
2003-06-27 18:43 ` Dave Hansen
1 sibling, 1 reply; 5+ messages in thread
From: Jeff Garzik @ 2003-06-27 18:42 UTC (permalink / raw)
To: Mark Peloquin; +Cc: linux-kernel, linstab, ltp-results
On Fri, Jun 27, 2003 at 01:34:37PM -0500, Mark Peloquin wrote:
> Nightly Regression Summary
> for
> 2.5.73-bk2 vs 2.5.73-bk3
This is very cool stuff.
Thanks!
Jeff
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [BENCHMARK] 2.5.73-bk3, -bk4, -mjb1 regression test results
2003-06-27 18:34 [BENCHMARK] 2.5.73-bk3, -bk4, -mjb1 regression test results Mark Peloquin
2003-06-27 18:42 ` Jeff Garzik
@ 2003-06-27 18:43 ` Dave Hansen
2003-06-27 18:54 ` Mark Peloquin
1 sibling, 1 reply; 5+ messages in thread
From: Dave Hansen @ 2003-06-27 18:43 UTC (permalink / raw)
To: Mark Peloquin; +Cc: linux-kernel, linstab, ltp-results
On Fri, 2003-06-27 at 11:34, Mark Peloquin wrote:
> Nightly Regression Summary
> for
...
Have you ever had these regression tests run against the same kernel?
say, 2.5.73 vs 2.5.73
How often do you see improvements or regressions reported then?
--
Dave Hansen
haveblue@us.ibm.com
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [BENCHMARK] 2.5.73-bk3, -bk4, -mjb1 regression test results
2003-06-27 18:43 ` Dave Hansen
@ 2003-06-27 18:54 ` Mark Peloquin
0 siblings, 0 replies; 5+ messages in thread
From: Mark Peloquin @ 2003-06-27 18:54 UTC (permalink / raw)
To: Dave Hansen; +Cc: linux-kernel, linstab, ltp-results
Dave Hansen wrote:
>On Fri, 2003-06-27 at 11:34, Mark Peloquin wrote:
>
>
>>Nightly Regression Summary
>>for
>>
>>
>...
>
>Have you ever had these regression tests run against the same kernel?
>say, 2.5.73 vs 2.5.73
>
>How often do you see improvements or regressions reported then?
>
>
Its benchmark specific. Currently dbench, specjbb, and lmbench have a
few problematic data points that aren't terribly repeatable. So on the
same kernel, one can see improvements and regressions. That said, the
majority of data points are very repeatable and give good indications of
real change. We will continue to tweak things here and there to reduce
the number of false positives.
Mark
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2003-06-27 18:41 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2003-06-27 18:34 [BENCHMARK] 2.5.73-bk3, -bk4, -mjb1 regression test results Mark Peloquin
2003-06-27 18:42 ` Jeff Garzik
2003-06-27 18:50 ` Mark Peloquin
2003-06-27 18:43 ` Dave Hansen
2003-06-27 18:54 ` Mark Peloquin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox