WimMark I report for 2.5.65-mm2 (now with deadline)

WimMark I report for 2.5.65-mm2 (now with deadline)

[Joel Becker]

WimMark I report for 2.5.65-mm2

Runs (antic):  1374.22 1487.19 1437.26
Runs (deadline):  1238.58 1537.36 1513.04

	WimMark I is a rough benchmark we have been running
here at Oracle against various kernels.  Each run tests an OLTP
workload on the Oracle database with somewhat restrictive memory
conditions.  This reduces in-memory buffering of data, allowing for
more I/O.  The I/O is read and sync write, random and seek-laden.
	The benchmark is called "WimMark I" because it has no
official standing and is only a relative benchmark useful for comparing
kernel changes.  The benchmark is normalized an arbitrary kernel, which
scores 1000.0.  All other numbers are relative to this.
	The machine in question is a 4 way 700 MHz Xeon machine with 2GB
of RAM.  CONFIG_HIGHMEM4GB is selected.  The disk accessed for data is a
10K RPM U2W SCSI of similar vintage.  The data files are living on an
ext3 filesystem.  Unless mentioned, all runs are
on this machine (variation in hardware would indeed change the
benchmark).


-- 

"Always give your best, never get discouraged, never be petty; always
 remember, others may hate you.  Those who hate you don't win unless
 you hate them.  And then you destroy yourself."
	- Richard M. Nixon

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: WimMark I report for 2.5.65-mm2 (now with deadline)

[Andrew Morton]

Joel Becker <Joel.Becker@oracle.com> wrote:
>
> 
> WimMark I report for 2.5.65-mm2
> 
> Runs (antic):  1374.22 1487.19 1437.26
> Runs (deadline):  1238.58 1537.36 1513.04

The averages of these are equal.  Can we safely conclude that this is fixed
up now?

Re: WimMark I report for 2.5.65-mm2 (now with deadline)

[Joel Becker]

On Thu, Mar 20, 2003 at 05:58:05PM -0800, Andrew Morton wrote:
> > WimMark I report for 2.5.65-mm2
> > 
> > Runs (antic):  1374.22 1487.19 1437.26
> > Runs (deadline):  1238.58 1537.36 1513.04
> 
> The averages of these are equal.  Can we safely conclude that this is fixed
> up now?

	Not really, I think, because the 1238 seems a slightly fluke
run.  I see these from time to time, but almost all of the deadline runs
are in the 1500-1600 range.  In fact, -mm2 is lower than some other
kernels by about 50 for deadline.
	I wouldn't disagree with you if I didn't see that consistency.
antic has never really jumped above 1500, and deadline almost never goes
below.
	Certainly they are much closer than they were.

Joel

-- 

Life's Little Instruction Book #109

	"Know how to drive a stick shift."

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: WimMark I report for 2.5.65-mm2 (now with deadline)

[Nick Piggin]

Joel Becker wrote:

>On Thu, Mar 20, 2003 at 05:58:05PM -0800, Andrew Morton wrote:
>
>>>WimMark I report for 2.5.65-mm2
>>>
>>>Runs (antic):  1374.22 1487.19 1437.26
>>>Runs (deadline):  1238.58 1537.36 1513.04
>>>
>>The averages of these are equal.  Can we safely conclude that this is fixed
>>up now?
>>
>
>	Not really, I think, because the 1238 seems a slightly fluke
>run.  I see these from time to time, but almost all of the deadline runs
>are in the 1500-1600 range.  In fact, -mm2 is lower than some other
>kernels by about 50 for deadline.
>	I wouldn't disagree with you if I didn't see that consistency.
>antic has never really jumped above 1500, and deadline almost never goes
>below.
>	Certainly they are much closer than they were.
>
The smaller first runs are not due to the benchmark running for the first
time, are they? In your mm1 tests you wrote:

>WimMark I report for 2.5.65-mm1
>
>Runs (antic):  1559.32 1025.38 1579.98
>Runs (deadline):  1554.48 1589.89 1350.37
>
So it does seem to be quite varied, but yes I'll keep working on it.
BTW. how do these results compare with 2.4 and other operating
systems on the same hardware, out of interest?

Re: WimMark I report for 2.5.65-mm2 (now with deadline)

[Joel Becker]

On Fri, Mar 21, 2003 at 06:19:49PM +1100, Nick Piggin wrote:
> The smaller first runs are not due to the benchmark running for the first
> time, are they? In your mm1 tests you wrote:

	The benchmark has a ramp-up period built-in to populate the
database caches.  The first run is not always the slowest.
	The thing is, the runs can be sensitive to some things (someone
logs in and runs something, Linux decides to flush some cache, etc).
The runs take long enough without extending them to flatten this some.

> >Runs (antic):  1559.32 1025.38 1579.98
> >Runs (deadline):  1554.48 1589.89 1350.37

	If you notice, the variance always is of a fluke sort.  This is
why I do multiple runs.  The non-fluke runs are very consistent.  See
runs 1 and 3 for antic and 1 and 2 for deadline in the quote.

> So it does seem to be quite varied, but yes I'll keep working on it.
> BTW. how do these results compare with 2.4 and other operating
> systems on the same hardware, out of interest?

	I've not run other operating systems, as I don't have the
software or OSes installed.  I've not run a vanilla 2.4 recently, and
probably should do that.

Joel

-- 

 "I'm living so far beyond my income that we may almost be said
 to be living apart."
         - e e cummings

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127