[OSDL][BENCHMARK] DBT-2 2.5.65/mjb/osdl comparison data

[OSDL][BENCHMARK] DBT-2 2.5.65/mjb/osdl comparison data

[Mary Edie Meredith]

We have now comparison data for DBT-2 (readprofile included) from
multiple kernels. To provide a quick comparison for those not
familiar with DBT-2, we've compared the results, using 2.5.65 stock
as the baseline (bigger is better).

                        Score           Score
Kernel                  Cached          Non-Cached
2.5.65 base             100 (baseline)  100
2.5.65-mjb2 HZ=100      90.95           99.26
2.5.65-mjb2 HZ=1000     102.38          99.92
2.5.65-osdl1            101.69          99.89
2.5.64-osdl1            104.16          99.67

HZ is defined as 1000 in the base and osdl1 kernels. mjb2 kernel uses
Andrew Morton / Dave Hansen patch making HZ a config option of
100 Hz or 1000 Hz).  Also we reversed out the 400-shpte patch.

Link to .config, readprofiles, metric info, raw data:

http://www.osdl.org/projects/dbt2dev/results/8way/MJB65/8way_2_5_65.html


Guided tour:

At the top of the screen, you will see a row that includes the .config
and the readprofile data for each kernel tested.

Next is the list of runs of each type, the average metric (Green
line) bigger numbers are better. The first set of these is the
cached workload case, second is non-cached.

Click on "Raw data" for the vmstat, iostat raw info from each run of
that kernel and workload type.


Just some things noticed looking at the vmstat plotted data:

Notible difference in processes waiting for run time, all 2.5.65
are high relative to 2.5.64(the last frame) for the cached case
(first row of frames) in these plots:
http://www.osdl.org/projects/dbt2dev/results/8way/MJB65/r.html


Of course, interrupts are down for the HZ=100 case (second frame,
both rows):
http://www.osdl.org/projects/dbt2dev/results/8way/MJB65/In.html

Context switches per second slightly down to for HZ=100:
http://www.osdl.org/projects/dbt2dev/results/8way/MJB65/cs.html

Moving on to 2.5.66 to escape problems with "sleep".

Mary Meredith
Mark Wong
Cliff White

Open Source Development Lab
www.osdl.org
~                                             
-- 

Re: [OSDL][BENCHMARK] DBT-2 2.5.65/mjb/osdl comparison data

[Patrick Mansfield]

On Fri, Mar 28, 2003 at 02:15:24PM -0800, Mary Edie Meredith wrote:
>                         Score           Score
> Kernel                  Cached          Non-Cached
> 2.5.65 base             100 (baseline)  100
> 2.5.65-mjb2 HZ=100      90.95           99.26
> 2.5.65-mjb2 HZ=1000     102.38          99.92
> 2.5.65-osdl1            101.69          99.89
> 2.5.64-osdl1            104.16          99.67
> 
> HZ is defined as 1000 in the base and osdl1 kernels. mjb2 kernel uses
> Andrew Morton / Dave Hansen patch making HZ a config option of
> 100 Hz or 1000 Hz).  Also we reversed out the 400-shpte patch.
> 
> Link to .config, readprofiles, metric info, raw data:
> 
> http://www.osdl.org/projects/dbt2dev/results/8way/MJB65/8way_2_5_65.html

Do you have readprofiles of the non-cached runs?

-- Patrick Mansfield

Re: [Lse-tech] Re: [OSDL][BENCHMARK] DBT-2 2.5.65/mjb/osdl comparison data

[Mary Edie Meredith]

No, I didn't capture read profiles on the non-cached cases.  I will be
collecting readprofile on all cases when I go to 2.5.66, but I didn't on
this round.

If you have a special interest, I can go back and get the data.  It
takes 2hrs per kernel.  If you could identify a subset most interesting
to you, I can do those first.

On Fri, 2003-03-28 at 15:25, Patrick Mansfield wrote:
> On Fri, Mar 28, 2003 at 02:15:24PM -0800, Mary Edie Meredith wrote:
> >                         Score           Score
> > Kernel                  Cached          Non-Cached
> > 2.5.65 base             100 (baseline)  100
> > 2.5.65-mjb2 HZ=100      90.95           99.26
> > 2.5.65-mjb2 HZ=1000     102.38          99.92
> > 2.5.65-osdl1            101.69          99.89
> > 2.5.64-osdl1            104.16          99.67
> > 
> > HZ is defined as 1000 in the base and osdl1 kernels. mjb2 kernel uses
> > Andrew Morton / Dave Hansen patch making HZ a config option of
> > 100 Hz or 1000 Hz).  Also we reversed out the 400-shpte patch.
> > 
> > Link to .config, readprofiles, metric info, raw data:
> > 
> > http://www.osdl.org/projects/dbt2dev/results/8way/MJB65/8way_2_5_65.html
> 
> Do you have readprofiles of the non-cached runs?
> 
> -- Patrick Mansfield
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by:
> The Definitive IT and Networking Event. Be There!
> NetWorld+Interop Las Vegas 2003 -- Register today!
> http://ads.sourceforge.net/cgi-bin/redirect.pl?keyn0001en
> _______________________________________________
> Lse-tech mailing list
> Lse-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/lse-tech
-- 
Mary Edie Meredith <maryedie@osdl.org>
Open Source Development Lab

Re: [Lse-tech] Re: [OSDL][BENCHMARK] DBT-2 2.5.65/mjb/osdl comparison data

[Patrick Mansfield]

On Fri, Mar 28, 2003 at 03:52:24PM -0800, Mary Edie Meredith wrote:
> No, I didn't capture read profiles on the non-cached cases.  I will be
> collecting readprofile on all cases when I go to 2.5.66, but I didn't on
> this round.
> 
> If you have a special interest, I can go back and get the data.  It
> takes 2hrs per kernel.  If you could identify a subset most interesting
> to you, I can do those first.

I am wondering if you are seeing any lock contention for the
host_lock/queue_lock in scsi.

There are some patches not in mainline that I would like to see run
against a multi-disk IO intensive benchmark.

They are currently applied in this bk tree:

bk://linux-scsi.bkbits.net/scsi-locking-2.5

Or, I could send you a patch against 2.5.66.

If you can run with the above and compare to 2.5.66 I would appreciate it
(readprofiles for both cases would still be useful). I would hope to see
at least lower system times.

Thanks.

-- Patrick Mansfield

Re: [Lse-tech] [OSDL][BENCHMARK] DBT-2 2.5.65/mjb/osdl comparison data

[Martin J. Bligh]

Looks like you're going to get more bang for the buck for this
benchmark by playing around with some of the IO stuff like readahead
or whatever ...

00000000 total                                    6309908  36.9433
c01089b0 default_idle                             6187707 96682.9219

Not all that much room for improvment with CPU efficiency ;-)

M.

--On Friday, March 28, 2003 14:15:24 -0800 Mary Edie Meredith <maryedie@osdl.org> wrote:

> We have now comparison data for DBT-2 (readprofile included) from
> multiple kernels. To provide a quick comparison for those not
> familiar with DBT-2, we've compared the results, using 2.5.65 stock
> as the baseline (bigger is better).
> 
>                         Score           Score
> Kernel                  Cached          Non-Cached
> 2.5.65 base             100 (baseline)  100
> 2.5.65-mjb2 HZ=100      90.95           99.26
> 2.5.65-mjb2 HZ=1000     102.38          99.92
> 2.5.65-osdl1            101.69          99.89
> 2.5.64-osdl1            104.16          99.67
> 
> HZ is defined as 1000 in the base and osdl1 kernels. mjb2 kernel uses
> Andrew Morton / Dave Hansen patch making HZ a config option of
> 100 Hz or 1000 Hz).  Also we reversed out the 400-shpte patch.
> 
> Link to .config, readprofiles, metric info, raw data:
> 
> http://www.osdl.org/projects/dbt2dev/results/8way/MJB65/8way_2_5_65.html
> 
> 
> Guided tour:
> 
> At the top of the screen, you will see a row that includes the .config
> and the readprofile data for each kernel tested.
> 
> Next is the list of runs of each type, the average metric (Green
> line) bigger numbers are better. The first set of these is the
> cached workload case, second is non-cached.
> 
> Click on "Raw data" for the vmstat, iostat raw info from each run of
> that kernel and workload type.
> 
> 
> Just some things noticed looking at the vmstat plotted data:
> 
> Notible difference in processes waiting for run time, all 2.5.65
> are high relative to 2.5.64(the last frame) for the cached case
> (first row of frames) in these plots:
> http://www.osdl.org/projects/dbt2dev/results/8way/MJB65/r.html
> 
> 
> Of course, interrupts are down for the HZ=100 case (second frame,
> both rows):
> http://www.osdl.org/projects/dbt2dev/results/8way/MJB65/In.html
> 
> Context switches per second slightly down to for HZ=100:
> http://www.osdl.org/projects/dbt2dev/results/8way/MJB65/cs.html
> 
> Moving on to 2.5.66 to escape problems with "sleep".
> 
> Mary Meredith
> Mark Wong
> Cliff White
> 
> Open Source Development Lab
> www.osdl.org
> ~                                             
> -- 
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by:
> The Definitive IT and Networking Event. Be There!
> NetWorld+Interop Las Vegas 2003 -- Register today!
> http://ads.sourceforge.net/cgi-bin/redirect.pl?keyn0001en
> _______________________________________________
> Lse-tech mailing list
> Lse-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/lse-tech
> 
>