[BENCHMARK] O1int with contest

[BENCHMARK] O1int with contest

[Roberto Orenstein]

Hi guys,

Here are some numbers from three kernels tested on my home machine.
All threes are 2.5.73 based.
Vanilla is a plain one, O1int-0307020011 is the latest (as I last
checked) O1int patch w/o the granularity patch, and
O1int-granu-0307020011 is the former with granularity.
One can see that with granularity, the kernel compile suffers a bit, but
the response is usually high. In my machine, this was the kernel with
the best responsiveness.

Each kernel was run once, except O1int-0307020011 with three iterations.
This was the first I tested, and as soon I noticed the time it took, I
decided to run once the others 8). Maybe this has some bad influence on
the results. I appreciate any comments...

regards,
Roberto

no_load:
Kernel                 [runs]	Time	CPU%	Loads	LCPU%	Ratio
O1int-0307020011            3	122	93.4	0.0	0.0	1.00
O1int-granu-0307020011      1	124	93.5	0.0	0.0	1.00
vanilla                     1	123	92.7	0.0	0.0	1.00
cacherun:
Kernel                 [runs]	Time	CPU%	Loads	LCPU%	Ratio
O1int-0307020011            3	115	99.1	0.0	0.0	0.94
O1int-granu-0307020011      1	119	97.5	0.0	0.0	0.96
vanilla                     1	116	98.3	0.0	0.0	0.94
process_load:
Kernel                 [runs]	Time	CPU%	Loads	LCPU%	Ratio
O1int-0307020011            3	161	70.8	72.5	27.3	1.32
O1int-granu-0307020011      1	166	69.9	79.0	28.7	1.34
vanilla                     1	160	71.2	79.0	27.5	1.30
ctar_load:
Kernel                 [runs]	Time	CPU%	Loads	LCPU%	Ratio
O1int-0307020011            3	154	77.3	1.0	3.2	1.26
O1int-granu-0307020011      1	163	74.8	2.0	7.4	1.31
vanilla                     1	161	74.5	2.0	7.5	1.31
xtar_load:
Kernel                 [runs]	Time	CPU%	Loads	LCPU%	Ratio
O1int-0307020011            3	160	72.5	1.0	6.2	1.31
O1int-granu-0307020011      1	166	71.1	1.0	7.2	1.34
vanilla                     1	160	72.5	1.0	7.5	1.30
io_load:
Kernel                 [runs]	Time	CPU%	Loads	LCPU%	Ratio
O1int-0307020011            3	310	39.0	66.3	15.4	2.54
O1int-granu-0307020011      1	332	36.7	75.8	16.3	2.68
vanilla                     1	306	39.5	71.7	16.7	2.49
io_other:
Kernel                 [runs]	Time	CPU%	Loads	LCPU%	Ratio
O1int-0307020011            3	286	42.3	62.3	16.0	2.34
O1int-granu-0307020011      1	296	41.2	65.4	15.9	2.39
vanilla                     1	364	33.2	93.0	18.7	2.96
read_load:
Kernel                 [runs]	Time	CPU%	Loads	LCPU%	Ratio
O1int-0307020011            3	156	75.0	5.4	3.2	1.28
O1int-granu-0307020011      1	159	75.5	5.7	3.8	1.28
vanilla                     1	155	76.1	5.7	3.9	1.26
list_load:
Kernel                 [runs]	Time	CPU%	Loads	LCPU%	Ratio
O1int-0307020011            3	147	79.6	0.0	7.5	1.20
O1int-granu-0307020011      1	149	79.2	0.0	7.4	1.20
vanilla                     1	147	79.6	0.0	7.5	1.20
mem_load:
Kernel                 [runs]	Time	CPU%	Loads	LCPU%	Ratio
O1int-0307020011            3	177	65.5	42.3	1.1	1.45
O1int-granu-0307020011      1	188	62.8	43.0	1.1	1.52
vanilla                     1	189	61.9	44.0	1.1	1.54
dbench_load:
Kernel                 [runs]	Time	CPU%	Loads	LCPU%	Ratio
O1int-0307020011            3	152	75.0	23535.3	19.1	1.25
O1int-granu-0307020011      1	142	82.4	20464.0	16.2	1.15
vanilla                     1	149	76.5	27791.0	20.8	1.21

Re: [BENCHMARK] O1int with contest

[Con Kolivas]

On Wed, 2 Jul 2003 14:26, Roberto Orenstein wrote:
> Hi guys,
>
> Here are some numbers from three kernels tested on my home machine.
> All threes are 2.5.73 based.
> Vanilla is a plain one, O1int-0307020011 is the latest (as I last
> checked) O1int patch w/o the granularity patch, and
> O1int-granu-0307020011 is the former with granularity.

Thanks for doing these.

> One can see that with granularity, the kernel compile suffers a bit, but
> the response is usually high. In my machine, this was the kernel with
> the best responsiveness.

Can you please describe your experiences? The more feedback I get the more I 
can get it working well.

> Each kernel was run once, except O1int-0307020011 with three iterations.
> This was the first I tested, and as soon I noticed the time it took, I
> decided to run once the others 8). Maybe this has some bad influence on
> the results. I appreciate any comments...

Ok well here is my summary of the situation. My patch has virtually no effect 
on contest results (except perhaps io_other). This is good because my earlier 
attempts did affect it, and possibly starved some of the loads. Dare I say 
it, contest is not very good at picking up _these_ sort of scheduler tweaks 
unless they do something wrong. Sorry if my system responsiveness benchmark 
doesn't show this effect; I think they're different. This is more about 
picking the right thing to give preference to. There's a long discussion in 
that, but I'll try not to get into it.

By the way it doesn't look like your dbench in dbench load actually worked.

O1int still remains a work in progress.

Con

Re: [BENCHMARK] O1int with contest

[Roberto Orenstein]

On Wed, 2003-07-02 at 03:24, Con Kolivas wrote:
> On Wed, 2 Jul 2003 14:26, Roberto Orenstein wrote:
> > Hi guys,
> >
> > Here are some numbers from three kernels tested on my home machine.
> > All threes are 2.5.73 based.
> > Vanilla is a plain one, O1int-0307020011 is the latest (as I last
> > checked) O1int patch w/o the granularity patch, and
> > O1int-granu-0307020011 is the former with granularity.
> 
> Thanks for doing these.
> 
> > One can see that with granularity, the kernel compile suffers a bit, but
> > the response is usually high. In my machine, this was the kernel with
> > the best responsiveness.
> 
> Can you please describe your experiences? The more feedback I get the more I 
> can get it working well.
> 
Well, basically I load the machine with some make -j nuts_number_here,
and I do some other things. The vanilla kernel always behaves badly,
with things like switching windows, loading web pages taking a visible
amount of time to get some cpu attention. The others always win. I mean,
for every load, ranging from a make -j5 to a make -j25 the system
_feels_ better than vanilla. With granularity, it was quite noticeable
(but the kernel compiles were noticeable longer too - some you win and
some you loose :-).
As I said to you on IRC, I wasn't able to make xmms skip with any
kernels (does anyone have a recipe for this?), which I guess is the
issue that started this thread.
The only issue I had was with app startup time (although starting a new
X session was indeed faster - strange).

> > Each kernel was run once, except O1int-0307020011 with three iterations.
> > This was the first I tested, and as soon I noticed the time it took, I
> > decided to run once the others 8). Maybe this has some bad influence on
> > the results. I appreciate any comments...
> 
> Ok well here is my summary of the situation. My patch has virtually no effect 
> on contest results (except perhaps io_other). This is good because my earlier 
> attempts did affect it, and possibly starved some of the loads. Dare I say 
> it, contest is not very good at picking up _these_ sort of scheduler tweaks 
> unless they do something wrong. Sorry if my system responsiveness benchmark 
> doesn't show this effect; I think they're different. This is more about 
> picking the right thing to give preference to. There's a long discussion in 
> that, but I'll try not to get into it.

Well, if things doesn't get worse, it's already a start, right? :-)
In fact I believe it's moving forward. It feels clearly better than
plain 2.5.73, but I don't know if others share the same view. My tests
weren't very scientific nor realistic, it's just a matter of 'looks
good'. Although I think 'feels good' is sometimes better than a lot of
numbers. If there's a way to get some precise numbers, let me know of
it. I thought that contest was a good choice. 

> 
> By the way it doesn't look like your dbench in dbench load actually worked.
Dunno what happened, I've Followed All The Instructions(TM). I'll look
at this later.

> 
> O1int still remains a work in progress.
I'll test your latest patch later to see the improvements. Should I run
another contest on it or doesn't make any difference?

regards,
Roberto