Sched - graphic smoothness under load - cfs-v13 sd-0.48

Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Bill Davidsen]

I generated a table of results from the latest glitch1 script, using an 
HTML postprocessor I not *quite* ready to foist on the word. In any case 
it has some numbers for frames per second, fairness of the processor 
time allocated to the compute bound processes which generate a lot of 
other screen activity for X, and my subjective comments on how smooth it 
looked and felt.

The chart is at http://www.tmr.com/~davidsen/sched_smooth_01.html for 
your viewing pleasure. The only "tuned" result was with sd, since what I 
observed was so bad using the default settings. If any scheduler 
developers would like me to try other tunings or new versions let me know.

-- 
Bill Davidsen <davidsen@tmr.com>
  "We have more to fear from the bungling of the incompetent than from
the machinations of the wicked."  - from Slashdot

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Ray Lee]

On 5/19/07, Bill Davidsen <davidsen@tmr.com> wrote:
> I generated a table of results from the latest glitch1 script, using an
> HTML postprocessor I not *quite* ready to foist on the word. In any case
> it has some numbers for frames per second, fairness of the processor
> time allocated to the compute bound processes which generate a lot of
> other screen activity for X, and my subjective comments on how smooth it
> looked and felt.
>
> The chart is at http://www.tmr.com/~davidsen/sched_smooth_01.html for
> your viewing pleasure.

Is the S.D. columns (immediately after the average) standard
deviation? If so, you may want to rename those 'stdev', as it's a
little confusing to have S.D. stand for that and Staircase Deadline.
Further, which standard deviation is it? (The standard deviation of
the values (stdev), or the standard deviation of the mean (sdom)?)

Finally, if it is the standard deviation (of either), then I don't
really believe those numbers for the glxgears case. The deviation is
huge for all but one of those results.

Regardless, it's good that you're doing measurements, and keep it up :-).

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Bill Davidsen]

Ray Lee wrote:
> On 5/19/07, Bill Davidsen <davidsen@tmr.com> wrote:
>> I generated a table of results from the latest glitch1 script, using an
>> HTML postprocessor I not *quite* ready to foist on the word. In any case
>> it has some numbers for frames per second, fairness of the processor
>> time allocated to the compute bound processes which generate a lot of
>> other screen activity for X, and my subjective comments on how smooth it
>> looked and felt.
>>
>> The chart is at http://www.tmr.com/~davidsen/sched_smooth_01.html for
>> your viewing pleasure.
> 
> Is the S.D. columns (immediately after the average) standard
> deviation? If so, you may want to rename those 'stdev', as it's a
> little confusing to have S.D. stand for that and Staircase Deadline.
> Further, which standard deviation is it? (The standard deviation of
> the values (stdev), or the standard deviation of the mean (sdom)?)
> 
What's intended is the stddev from the average, and perl bit me on that 
one. If you spell a variable wrong the same way more than once it 
doesn't flag it as a possible spelling error.

Note on the math, even when coded as intended, the divide of the squares 
of the errors is by N-1 not N. I found it both ways in online doc, but I 
learned it decades ago as "N-1" so I used that.
> Finally, if it is the standard deviation (of either), then I don't
> really believe those numbers for the glxgears case. The deviation is
> huge for all but one of those results.
> 
I had the same feeling, but because of the code error above, what failed 
was zeroing the sum of the errors, so (a) values after the first kept 
getting larger, and when I debugged it against the calculation by hand, 
the first one matched so I thought I had it right.

> Regardless, it's good that you're doing measurements, and keep it up :-).

Okay, here's a bonus, http://www.tmr.com/~davidsen/sched_smooth_02.html 
not only has the right values, the labels are changed, and I included 
more data points from the fc6 recent kernel and the 2.6.21.1 kernel with 
the mainline scheduler.

The nice thing about this test and the IPC test I posted recently is 
that they are reasonable stable on the same hardware, so even if someone 
argues about what they show, they show the same thing each time and can 
therefore be used to compare changes.

As I told a manger at the old Prodigy after coding up some log analysis 
with pretty graphs, "getting the data was the easy part, the hard part 
is figuring out what it means." If this data is useful in suggesting 
changes, then it has value. Otherwise it was a fun way to spend some time.

-- 
Bill Davidsen <davidsen@tmr.com>
   "We have more to fear from the bungling of the incompetent than from
the machinations of the wicked."  - from Slashdot

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Michael Gerdau]

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]

> Okay, here's a bonus, http://www.tmr.com/~davidsen/sched_smooth_02.html 
> not only has the right values, the labels are changed, and I included 
> more data points from the fc6 recent kernel and the 2.6.21.1 kernel with 
> the mainline scheduler.
> 
> The nice thing about this test and the IPC test I posted recently is 
> that they are reasonable stable on the same hardware, so even if someone 
> argues about what they show, they show the same thing each time and can 
> therefore be used to compare changes.

I'm not sure I follow you here. The difference between 2.6.21 and 2.6.21.1
are two simple (as in involving little code) changes to ip4 and ip6 net
and I'm not even sure that code is used at all in your tests.
[Read: IMO the 2.6.21 and 2.6.21.1 figures are for identical cases].

Assuming the above is correct then IMO the variance between the two
"dublicated" lines (cfs-v13 and sd048) is such that I would not have
written "that they are reasonable stable on the same hardware".

I don't want to say the values aren't useful. I simply think there is
a high noiselevel.

Best,
Michael
-- 
 Technosis GmbH, Geschäftsführer: Michael Gerdau, Tobias Dittmar
 Sitz Hamburg; HRB 89145 Amtsgericht Hamburg
 Vote against SPAM - see http://www.politik-digital.de/spam/
 Michael Gerdau       email: mgd@technosis.de
 GPG-keys available on request or at public keyserver

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Ray Lee]

On 5/19/07, Michael Gerdau <mgd@technosis.de> wrote:
> > Okay, here's a bonus, http://www.tmr.com/~davidsen/sched_smooth_02.html
> > not only has the right values, the labels are changed, and I included
> > more data points from the fc6 recent kernel and the 2.6.21.1 kernel with
> > the mainline scheduler.
> >
> > The nice thing about this test and the IPC test I posted recently is
> > that they are reasonable stable on the same hardware, so even if someone
> > argues about what they show, they show the same thing each time and can
> > therefore be used to compare changes.
>
> I'm not sure I follow you here. The difference between 2.6.21 and 2.6.21.1
> are two simple (as in involving little code) changes to ip4 and ip6 net
> and I'm not even sure that code is used at all in your tests.
> [Read: IMO the 2.6.21 and 2.6.21.1 figures are for identical cases].
>
> Assuming the above is correct then IMO the variance between the two
> "dublicated" lines (cfs-v13 and sd048) is such that I would not have
> written "that they are reasonable stable on the same hardware".
>
> I don't want to say the values aren't useful. I simply think there is
> a high noiselevel.

The noise is reflected in the standard deviation he has on those rows.
The average +- stdev  of one overlaps the average +- stdev of the
other, which shows that the test *is* stable, where stable (as always)
is defined to some accuracy.

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Michael Gerdau]

[-- Attachment #1: Type: text/plain, Size: 743 bytes --]

> > I don't want to say the values aren't useful. I simply think there is
> > a high noiselevel.
> 
> The noise is reflected in the standard deviation he has on those rows.
> The average +- stdev  of one overlaps the average +- stdev of the
> other,

For the fairness test on cfs13 this simply is wrong. And for the
test to be stable would require that avg of one is within avg+-stddev
of the other (simple overlap does not suffice).

Best,
Michael
-- 
 Technosis GmbH, Geschäftsführer: Michael Gerdau, Tobias Dittmar
 Sitz Hamburg; HRB 89145 Amtsgericht Hamburg
 Vote against SPAM - see http://www.politik-digital.de/spam/
 Michael Gerdau       email: mgd@technosis.de
 GPG-keys available on request or at public keyserver

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Ray Lee]

On 5/19/07, Michael Gerdau <mgd@technosis.de> wrote:
> > > I don't want to say the values aren't useful. I simply think there is
> > > a high noiselevel.
> >
> > The noise is reflected in the standard deviation he has on those rows.
> > The average +- stdev  of one overlaps the average +- stdev of the
> > other,
>
> For the fairness test on cfs13 this simply is wrong.

Ah, you are correct. I was looking at the glxgears test and mostly
ignoring the fairness one as I'm not sure what the test is doing. As
the deviations seem erratic, the fairness test looks like it needs
more passes to converge, but that's just a guess. I honestly don't
know what's going on with that one.

> And for the
> test to be stable would require that avg of one is within avg+-stddev
> of the other (simple overlap does not suffice).

Er, I think you took a different statistics class than I did :-). In
general, the 'true' value we're trying to find/measure has a certain
percentage chance of lying within the average +- n*deviation, with the
percentage going up as n increases, right? (68% for n=1, 95% for n=2,
etc.)

Both distributions have that property, and so to consider one of the
averages as the 'true' value and the other a mere distribution that
has to overlap it is... somewhat inaccurate.

In general, if they both overlap within one standard deviation of each
other, you can say that the data has pretty good agreement with
itself. Which for the glxgears case, it does. The fairness test isn't
so cut and dried, but it's not horrifically out of whack, either. So
I'll reserve judgment on that one :-).

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Ray Lee]

On 5/19/07, Bill Davidsen <davidsen@tmr.com> wrote:
> Ray Lee wrote:
> > Is the S.D. columns (immediately after the average) standard
> > deviation? If so, you may want to rename those 'stdev', as it's a
> > little confusing to have S.D. stand for that and Staircase Deadline.
> > Further, which standard deviation is it? (The standard deviation of
> > the values (stdev), or the standard deviation of the mean (sdom)?)
> >
> What's intended is the stddev from the average, and perl bit me on that
> one. If you spell a variable wrong the same way more than once it
> doesn't flag it as a possible spelling error.
>
> Note on the math, even when coded as intended, the divide of the squares
> of the errors is by N-1 not N. I found it both ways in online doc, but I
> learned it decades ago as "N-1" so I used that.

<nod> sqrt(N-1) is almost always the correct form, and gives the
"sample standard deviation". sqrt(N) (or the "population standard
deviation") should only be used when you can prove that you're not
doing any sub-sampling of the population at all, which is pretty much
never the case.

In practice, for any decent values of N, the difference between the
two is usually insignificant compared to everything else going on.

> Okay, here's a bonus, http://www.tmr.com/~davidsen/sched_smooth_02.html
> not only has the right values, the labels are changed, and I included
> more data points from the fc6 recent kernel and the 2.6.21.1 kernel with
> the mainline scheduler.

You rock.

> The nice thing about this test and the IPC test I posted recently is
> that they are reasonable stable on the same hardware, so even if someone
> argues about what they show, they show the same thing each time and can
> therefore be used to compare changes.

Yup, these look good to me.

> As I told a manger at the old Prodigy after coding up some log analysis
> with pretty graphs, "getting the data was the easy part, the hard part
> is figuring out what it means." If this data is useful in suggesting
> changes, then it has value. Otherwise it was a fun way to spend some time.

Right. At minimum, the average +- standard deviations of the glxgears
case tells a pretty consistent story, especially if one looks at the
deviation as a percentage of the average. (Mainline is far more
erratic than either SD or CFS, and CFS seems to be able to maintain
high throughput and low jitter.)

I'm not quite sure what to read into the fairness test, other than
mainline is obviously different for the other two in terms of
throughput. Deviations are across the board, but perhaps that's more a
reflection of how much (little) data was collected for this one?
Dunno.

Anyway, as you point out, the usefulness of the tests chosen could
still be up for debate, but I think you've shown that the results are
pretty stable, and at least can serve as a good measure between
versions of schedulers.

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Matt Keenan]

Bill Davidsen wrote:
> Ray Lee wrote:
>> On 5/19/07, Bill Davidsen <davidsen@tmr.com> wrote:
>>> I generated a table of results from the latest glitch1 script, using an
>>> HTML postprocessor I not *quite* ready to foist on the word. In any 
>>> case
>>> it has some numbers for frames per second, fairness of the processor
>>> time allocated to the compute bound processes which generate a lot of
>>> other screen activity for X, and my subjective comments on how 
>>> smooth it
>>> looked and felt.
>>>
>>> The chart is at http://www.tmr.com/~davidsen/sched_smooth_01.html for
>>> your viewing pleasure.
>>
>> Is the S.D. columns (immediately after the average) standard
>> deviation? If so, you may want to rename those 'stdev', as it's a
>> little confusing to have S.D. stand for that and Staircase Deadline.
>> Further, which standard deviation is it? (The standard deviation of
>> the values (stdev), or the standard deviation of the mean (sdom)?)
>>
> What's intended is the stddev from the average, and perl bit me on 
> that one. If you spell a variable wrong the same way more than once it 
> doesn't flag it as a possible spelling error.
>
> Note on the math, even when coded as intended, the divide of the 
> squares of the errors is by N-1 not N. I found it both ways in online 
> doc, but I learned it decades ago as "N-1" so I used that.
N-1 is for estimating the standard deviation from a random sample of a 
population (s), N is for calculating the standard deviation of a whole 
population (σ).

xref http://en.wikipedia.org/wiki/Standard_deviation

Yes, I am aware of the potential for Wikipedia articles to be wrong, but 
this article seems correct at first blush :)

Matt

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Diego Calleja]

El Sat, 19 May 2007 16:02:37 -0400, Bill Davidsen <davidsen@tmr.com> escribió:

> The chart is at http://www.tmr.com/~davidsen/sched_smooth_01.html for 
> your viewing pleasure. The only "tuned" result was with sd, since what I 
> observed was so bad using the default settings. If any scheduler 
> developers would like me to try other tunings or new versions let me know.

How useful is glxgears as benchmark here? The X.org people has been saying
for ages that "glxgears is not a benchmark". Some people (at least on ubuntu)
even patched it so that if you want to get FPS numbers, you need to pass the
"-iacknowledgethatthistoolisnotabenchmark" option. There's a good page
explaining why at http://wiki.cchtml.com/index.php/Glxgears_is_not_a_Benchmark

I realize that it may be OK for scheduler testing, but maybe it'd be more
interesting to test other kind of 3d benchmark tools.

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Ray Lee]

On 5/19/07, Diego Calleja <diegocg@gmail.com> wrote:
> El Sat, 19 May 2007 16:02:37 -0400, Bill Davidsen <davidsen@tmr.com> escribió:
>
> > The chart is at http://www.tmr.com/~davidsen/sched_smooth_01.html for
> > your viewing pleasure. The only "tuned" result was with sd, since what I
> > observed was so bad using the default settings. If any scheduler
> > developers would like me to try other tunings or new versions let me know.
>
> How useful is glxgears as benchmark here? The X.org people has been saying
> for ages that "glxgears is not a benchmark".

If you hold everything in the system constant other than the
scheduler, then it's a useful measure of the effects the scheduler has
upon glxgears, regardless of how well or poorly written it might be.

I suspect the X.org people are correct in regards how most people
would try to use it, which would be measurements against different
video cards, CPUs, etc.

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Mike Galbraith]

On Sat, 2007-05-19 at 22:36 +0200, Diego Calleja wrote:
> El Sat, 19 May 2007 16:02:37 -0400, Bill Davidsen <davidsen@tmr.com> escribió:
> 
> > The chart is at http://www.tmr.com/~davidsen/sched_smooth_01.html for 
> > your viewing pleasure. The only "tuned" result was with sd, since what I 
> > observed was so bad using the default settings. If any scheduler 
> > developers would like me to try other tunings or new versions let me know.
> 
> How useful is glxgears as benchmark here? The X.org people has been saying
> for ages that "glxgears is not a benchmark". Some people (at least on ubuntu)
> even patched it so that if you want to get FPS numbers, you need to pass the
> "-iacknowledgethatthistoolisnotabenchmark" option. There's a good page
> explaining why at http://wiki.cchtml.com/index.php/Glxgears_is_not_a_Benchmark
> 
> I realize that it may be OK for scheduler testing, but maybe it'd be more
> interesting to test other kind of 3d benchmark tools.

The most interesting thing about graphics to me is the effect that
massive bus blasting could have on time measurement.  Generally, cpu
schedulers really want to keep track of time,.. but there's no telling
when some IO controller may blast you right off the bus?

	-Mike

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Miguel Figueiredo]

Bill Davidsen wrote:
> I generated a table of results from the latest glitch1 script, using an 
> HTML postprocessor I not *quite* ready to foist on the word. In any case 
> it has some numbers for frames per second, fairness of the processor 
> time allocated to the compute bound processes which generate a lot of 
> other screen activity for X, and my subjective comments on how smooth it 
> looked and felt.
> 
> The chart is at http://www.tmr.com/~davidsen/sched_smooth_01.html for 
> your viewing pleasure. The only "tuned" result was with sd, since what I 
> observed was so bad using the default settings. If any scheduler 
> developers would like me to try other tunings or new versions let me know.
> 

As I tryied myself kernels 2.6.21, 2.6.21-cfs-v13, and 2.6.21-ck2 on the 
same machine i found *very* odd those numbers you posted, so i tested 
myself those kernels to see the numbers I get instead of talking about 
the usage of kernel xpto feels like.

I did run glxgears with kernels 2.6.21, 2.6.21-cfs-v13 and 2.6.21-ck2 
inside Debian's GNOME environment. The hardware is an AMD Sempron64 3.0 
GHz, 1 GB RAM, Nvidia 6800XT.
Average and standard deviation from the gathered data:

* 2.6.21: 		average = 11251.1; stdev = 0.172
* 2.6.21-cfs-v13:	average = 11242.8; stdev = 0.033
* 2.6.21-ck2:		average = 11257.8; stdev = 0.067

Keep in mind those numbers don't mean anything we all know glxgears is 
not a benchmark, their purpose is only to be used as comparison under 
the same conditions.

One odd thing i noticed, with 2.6.21-cfs-v13 the gnome's time applet in 
the bar skipped some minutes (e.g. 16:23 -> 16:25) several times.

The data is available on: 
http://www.debianPT.org/~elmig/pool/kernel/20070520/


How did you get your data? I am affraid your data it's wrong, there's no 
  such big difference between the schedulers...

-- 

Com os melhores cumprimentos/Best regards,

Miguel Figueiredo
http://www.DebianPT.org

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Ray Lee]

On 5/20/07, Miguel Figueiredo <elmig@debianpt.org> wrote:
> As I tryied myself kernels 2.6.21, 2.6.21-cfs-v13, and 2.6.21-ck2 on the
> same machine i found *very* odd those numbers you posted, so i tested
> myself those kernels to see the numbers I get instead of talking about
> the usage of kernel xpto feels like.
>
> I did run glxgears with kernels 2.6.21, 2.6.21-cfs-v13 and 2.6.21-ck2
> inside Debian's GNOME environment. The hardware is an AMD Sempron64 3.0
> GHz, 1 GB RAM, Nvidia 6800XT.
> Average and standard deviation from the gathered data:
>
> * 2.6.21:               average = 11251.1; stdev = 0.172
> * 2.6.21-cfs-v13:       average = 11242.8; stdev = 0.033
> * 2.6.21-ck2:           average = 11257.8; stdev = 0.067
>
> Keep in mind those numbers don't mean anything we all know glxgears is
> not a benchmark, their purpose is only to be used as comparison under
> the same conditions.

Uhm, then why are you trying to use them to compare against Bill's
numbers? You two have completely different hardware setups, and this
is a test that is dependent upon hardware. Stated differently, this is
a worthless comparison between your results and his as you are
changing multiple variables at the same time. (At minimum: the
scheduler, cpu, and video card.)

> One odd thing i noticed, with 2.6.21-cfs-v13 the gnome's time applet in
> the bar skipped some minutes (e.g. 16:23 -> 16:25) several times.
>
> The data is available on:
> http://www.debianPT.org/~elmig/pool/kernel/20070520/
>
>
> How did you get your data? I am affraid your data it's wrong, there's no
>   such big difference between the schedulers...

It doesn't look like you were running his glitch1 script which starts
several in glxgears parallel. Were you, or were you just running one?

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Miguel Figueiredo]

Ray Lee wrote:
> On 5/20/07, Miguel Figueiredo <elmig@debianpt.org> wrote:
>> As I tryied myself kernels 2.6.21, 2.6.21-cfs-v13, and 2.6.21-ck2 on the
>> same machine i found *very* odd those numbers you posted, so i tested
>> myself those kernels to see the numbers I get instead of talking about
>> the usage of kernel xpto feels like.
>>
>> I did run glxgears with kernels 2.6.21, 2.6.21-cfs-v13 and 2.6.21-ck2
>> inside Debian's GNOME environment. The hardware is an AMD Sempron64 3.0
>> GHz, 1 GB RAM, Nvidia 6800XT.
>> Average and standard deviation from the gathered data:
>>
>> * 2.6.21:               average = 11251.1; stdev = 0.172
>> * 2.6.21-cfs-v13:       average = 11242.8; stdev = 0.033
>> * 2.6.21-ck2:           average = 11257.8; stdev = 0.067
>>
>> Keep in mind those numbers don't mean anything we all know glxgears is
>> not a benchmark, their purpose is only to be used as comparison under
>> the same conditions.
> 
> Uhm, then why are you trying to use them to compare against Bill's
> numbers? You two have completely different hardware setups, and this
> is a test that is dependent upon hardware. Stated differently, this is
> a worthless comparison between your results and his as you are
> changing multiple variables at the same time. (At minimum: the
> scheduler, cpu, and video card.)

The only thing i want to see it's the difference between the behaviour 
of the different schedulers on the same test setup. In my test -ck2 was 
a bit better, not 200% worse as in Bill's measurements. I don't compare 
absolute values on different test setups.

> 
>> One odd thing i noticed, with 2.6.21-cfs-v13 the gnome's time applet in
>> the bar skipped some minutes (e.g. 16:23 -> 16:25) several times.
>>
>> The data is available on:
>> http://www.debianPT.org/~elmig/pool/kernel/20070520/
>>
>>
>> How did you get your data? I am affraid your data it's wrong, there's no
>>   such big difference between the schedulers...
> 
> It doesn't look like you were running his glitch1 script which starts
> several in glxgears parallel. Were you, or were you just running one?

No i'm not, i'm running only one instance of glxgears inside the GNOME's 
environment.


-- 

Com os melhores cumprimentos/Best regards,

Miguel Figueiredo
http://www.DebianPT.org

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Ray Lee]

On 5/20/07, Miguel Figueiredo <elmig@debianpt.org> wrote:
> > It doesn't look like you were running his glitch1 script which starts
> > several in glxgears parallel. Were you, or were you just running one?
>
> No i'm not, i'm running only one instance of glxgears inside the GNOME's
> environment.

Then not only were you on different hardware, but you weren't even
running the same test he was! He wasn't reporting raw numbers for one
glxgears, he was running several in parallel.

Whether your test is valid is a different question. Regardless, it's
not the same test he was doing, and so your results have no bearing on
his. You may wish to reread his message from the start of the thread,
and find the glitch1 script he was using to perform the test. He's
sent it to the list before as an attachment.

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Bill Davidsen]

Miguel Figueiredo wrote:
> Ray Lee wrote:
>> On 5/20/07, Miguel Figueiredo <elmig@debianpt.org> wrote:
>>> As I tryied myself kernels 2.6.21, 2.6.21-cfs-v13, and 2.6.21-ck2 on the
>>> same machine i found *very* odd those numbers you posted, so i tested
>>> myself those kernels to see the numbers I get instead of talking about
>>> the usage of kernel xpto feels like.
>>>
>>> I did run glxgears with kernels 2.6.21, 2.6.21-cfs-v13 and 2.6.21-ck2
>>> inside Debian's GNOME environment. The hardware is an AMD Sempron64 3.0
>>> GHz, 1 GB RAM, Nvidia 6800XT.
>>> Average and standard deviation from the gathered data:
>>>
>>> * 2.6.21:               average = 11251.1; stdev = 0.172
>>> * 2.6.21-cfs-v13:       average = 11242.8; stdev = 0.033
>>> * 2.6.21-ck2:           average = 11257.8; stdev = 0.067
>>>
>>> Keep in mind those numbers don't mean anything we all know glxgears is
>>> not a benchmark, their purpose is only to be used as comparison under
>>> the same conditions.
>>
>> Uhm, then why are you trying to use them to compare against Bill's
>> numbers? You two have completely different hardware setups, and this
>> is a test that is dependent upon hardware. Stated differently, this is
>> a worthless comparison between your results and his as you are
>> changing multiple variables at the same time. (At minimum: the
>> scheduler, cpu, and video card.)
> 
> The only thing i want to see it's the difference between the behaviour 
> of the different schedulers on the same test setup. In my test -ck2 was 
> a bit better, not 200% worse as in Bill's measurements. I don't compare 
> absolute values on different test setups.
> 
Since I didn't test ck2 I'm sure your numbers are unique, I only tested 
the sd-0.48 patch set. I have the ck2 patch, just haven't tried it 
yet... But since there are a lot of other things in it, I'm unsure how 
it relates to what I was testing.
>>
>>> One odd thing i noticed, with 2.6.21-cfs-v13 the gnome's time applet in
>>> the bar skipped some minutes (e.g. 16:23 -> 16:25) several times.
>>>
>>> The data is available on:
>>> http://www.debianPT.org/~elmig/pool/kernel/20070520/
>>>
>>>
>>> How did you get your data? I am affraid your data it's wrong, there's no
>>>   such big difference between the schedulers...
>>
>> It doesn't look like you were running his glitch1 script which starts
>> several in glxgears parallel. Were you, or were you just running one?
> 
> No i'm not, i'm running only one instance of glxgears inside the GNOME's 
> environment.
> 
If you test the same conditions as I did let me know your results.

-- 
Bill Davidsen <davidsen@tmr.com>
   "We have more to fear from the bungling of the incompetent than from
the machinations of the wicked."  - from Slashdot

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Miguel Figueiredo]

Bill Davidsen wrote:
> Miguel Figueiredo wrote:
>> Ray Lee wrote:
>>> On 5/20/07, Miguel Figueiredo <elmig@debianpt.org> wrote:
>>>> As I tryied myself kernels 2.6.21, 2.6.21-cfs-v13, and 2.6.21-ck2 on 
>>>> the
>>>> same machine i found *very* odd those numbers you posted, so i tested
>>>> myself those kernels to see the numbers I get instead of talking about
>>>> the usage of kernel xpto feels like.
>>>>
>>>> I did run glxgears with kernels 2.6.21, 2.6.21-cfs-v13 and 2.6.21-ck2
>>>> inside Debian's GNOME environment. The hardware is an AMD Sempron64 3.0
>>>> GHz, 1 GB RAM, Nvidia 6800XT.
>>>> Average and standard deviation from the gathered data:
>>>>
>>>> * 2.6.21:               average = 11251.1; stdev = 0.172
>>>> * 2.6.21-cfs-v13:       average = 11242.8; stdev = 0.033
>>>> * 2.6.21-ck2:           average = 11257.8; stdev = 0.067
>>>>
>>>> Keep in mind those numbers don't mean anything we all know glxgears is
>>>> not a benchmark, their purpose is only to be used as comparison under
>>>> the same conditions.
>>>
>>> Uhm, then why are you trying to use them to compare against Bill's
>>> numbers? You two have completely different hardware setups, and this
>>> is a test that is dependent upon hardware. Stated differently, this is
>>> a worthless comparison between your results and his as you are
>>> changing multiple variables at the same time. (At minimum: the
>>> scheduler, cpu, and video card.)
>>
>> The only thing i want to see it's the difference between the behaviour 
>> of the different schedulers on the same test setup. In my test -ck2 
>> was a bit better, not 200% worse as in Bill's measurements. I don't 
>> compare absolute values on different test setups.
>>
> Since I didn't test ck2 I'm sure your numbers are unique, I only tested 
> the sd-0.48 patch set. I have the ck2 patch, just haven't tried it 
> yet... But since there are a lot of other things in it, I'm unsure how 
> it relates to what I was testing.
>>>
>>>> One odd thing i noticed, with 2.6.21-cfs-v13 the gnome's time applet in
>>>> the bar skipped some minutes (e.g. 16:23 -> 16:25) several times.
>>>>
>>>> The data is available on:
>>>> http://www.debianPT.org/~elmig/pool/kernel/20070520/
>>>>
>>>>
>>>> How did you get your data? I am affraid your data it's wrong, 
>>>> there's no
>>>>   such big difference between the schedulers...
>>>
>>> It doesn't look like you were running his glitch1 script which starts
>>> several in glxgears parallel. Were you, or were you just running one?
>>
>> No i'm not, i'm running only one instance of glxgears inside the 
>> GNOME's environment.
>>
> If you test the same conditions as I did let me know your results.
> 

Hi Bill,

if i've understood correctly the script runs glxgears for 43 seconds and 
in that time generates random numbers in a random number of times 
(processes, fork and forget), is that it?

You find the data, for 2.6.21-{cfs-v13, ck2} in 
http://www.debianpt.org/~elmig/pool/kernel/20070522/

Here's the funny part...

Lets call:

a) to "random number of processes run while glxgears is running", 
gl_fairloops file

b) to "generated frames while running a burst of processes" aka "massive 
and uknown amount of operations in one process", gl_gears file

kernel	2.6.21-cfs-v13	2.6.21-ck2
a)	194464		254669		
b)	54159		124


-- 

Com os melhores cumprimentos/Best regards,

Miguel Figueiredo
http://www.DebianPT.org

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Bill Davidsen]

Miguel Figueiredo wrote:
> Bill Davidsen wrote:
>> Miguel Figueiredo wrote:
>>> Ray Lee wrote:
>>>> On 5/20/07, Miguel Figueiredo <elmig@debianpt.org> wrote:
>>>>> As I tryied myself kernels 2.6.21, 2.6.21-cfs-v13, and 2.6.21-ck2 
>>>>> on the
>>>>> same machine i found *very* odd those numbers you posted, so i tested
>>>>> myself those kernels to see the numbers I get instead of talking 
>>>>> about
>>>>> the usage of kernel xpto feels like.
>>>>>
>>>>> I did run glxgears with kernels 2.6.21, 2.6.21-cfs-v13 and 2.6.21-ck2
>>>>> inside Debian's GNOME environment. The hardware is an AMD 
>>>>> Sempron64 3.0
>>>>> GHz, 1 GB RAM, Nvidia 6800XT.
>>>>> Average and standard deviation from the gathered data:
>>>>>
>>>>> * 2.6.21:               average = 11251.1; stdev = 0.172
>>>>> * 2.6.21-cfs-v13:       average = 11242.8; stdev = 0.033
>>>>> * 2.6.21-ck2:           average = 11257.8; stdev = 0.067
>>>>>
>>>>> Keep in mind those numbers don't mean anything we all know 
>>>>> glxgears is
>>>>> not a benchmark, their purpose is only to be used as comparison under
>>>>> the same conditions.
>>>>
>>>> Uhm, then why are you trying to use them to compare against Bill's
>>>> numbers? You two have completely different hardware setups, and this
>>>> is a test that is dependent upon hardware. Stated differently, this is
>>>> a worthless comparison between your results and his as you are
>>>> changing multiple variables at the same time. (At minimum: the
>>>> scheduler, cpu, and video card.)
>>>
>>> The only thing i want to see it's the difference between the 
>>> behaviour of the different schedulers on the same test setup. In my 
>>> test -ck2 was a bit better, not 200% worse as in Bill's 
>>> measurements. I don't compare absolute values on different test setups.
>>>
>> Since I didn't test ck2 I'm sure your numbers are unique, I only 
>> tested the sd-0.48 patch set. I have the ck2 patch, just haven't 
>> tried it yet... But since there are a lot of other things in it, I'm 
>> unsure how it relates to what I was testing.
>>>>
>>>>> One odd thing i noticed, with 2.6.21-cfs-v13 the gnome's time 
>>>>> applet in
>>>>> the bar skipped some minutes (e.g. 16:23 -> 16:25) several times.
>>>>>
>>>>> The data is available on:
>>>>> http://www.debianPT.org/~elmig/pool/kernel/20070520/
>>>>>
>>>>>
>>>>> How did you get your data? I am affraid your data it's wrong, 
>>>>> there's no
>>>>>   such big difference between the schedulers...
>>>>
>>>> It doesn't look like you were running his glitch1 script which starts
>>>> several in glxgears parallel. Were you, or were you just running one?
>>>
>>> No i'm not, i'm running only one instance of glxgears inside the 
>>> GNOME's environment.
>>>
>> If you test the same conditions as I did let me know your results.
>>
>
> Hi Bill,
>
> if i've understood correctly the script runs glxgears for 43 seconds 
> and in that time generates random numbers in a random number of times 
> (processes, fork and forget), is that it?
>
No, I haven't made it clear. A known number (default four) of xterms are 
started, each of which calculates random numbers and prints them, using 
much CPU time and causing a lot of scrolling. At the same time glxgears 
is running, and the smoothness (or not) is observed manually. The script 
records raw data on the number of frames per second and the number of 
random numbers calculated by each shell. Since these are FAIR 
schedulers, the variance between the scripts, and between multiple 
samples from glxgears is of interest. To avoid startup effects the 
glxgears value from the first sample is reported separately and not 
included in the statistics.

I looked at your results, and they are disturbing to say the least, it 
appears that using the ck2 scheduler glxgears stopped for all practical 
purposes. You don't have quite the latest glitch1, the new one runs 
longer and allows reruns to get several datasets, but the results still 
show very slow gears and a large difference between the work done by the 
four shells. That's not a good result, how did the system feel?
> You find the data, for 2.6.21-{cfs-v13, ck2} in 
> http://www.debianpt.org/~elmig/pool/kernel/20070522/
>
Thank you, these results are very surprising, and I would not expect the 
system to be pleasing the use under load, based on this.
> Here's the funny part...
>
> Lets call:
>
> a) to "random number of processes run while glxgears is running", 
> gl_fairloops file
>
It's really the relative work done by identical processes, hopefully 
they are all nearly the same, magnitude is interesting but related to 
responsiveness rather than fairness.
> b) to "generated frames while running a burst of processes" aka 
> "massive and uknown amount of operations in one process", gl_gears file
>
Well, top or ps will give you a good idea of processing, but it tried to 
use all of one CPU if allowed. Again, similarity of samples reflects 
fairness and magnitude reflects work done.
> kernel    2.6.21-cfs-v13    2.6.21-ck2
> a)    194464        254669       
> b)    54159        124
>
>
Everyone seems to like ck2, this makes it look as if the video display 
would be really pretty unusable. While sd-0.48 does show an occasional 
video glitch when watching video under heavy load, it's annoying rather 
than unusable.

Your subjective impressions would be helpful, and you may find that the 
package in the www.tmr.com/~public/source is slightly easier to use and 
gives more stable results. The documentation suggests the way to take 
samples (the way I did it) but if you feel more or longer samples would 
help it is tunable.

I added Con to the cc list, he may have comments or suggestions (against 
the current versions, please). Or he may feel that video combined with 
other heavy screen updating is unrealistic or not his chosen load. I'm 
told the load is similar to games which use threads and do lots of 
independent action, if that's a reference.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Bill Davidsen]

I was unable to reproduce the numbers Miguel generated, comments below. 
The -ck2 patch seems to run nicely, although the memory repopulation 
from swap would be most useful on system which have a lot of memory 
pressure.

Bill Davidsen wrote:
> Miguel Figueiredo wrote:
>>
>> Hi Bill,
>>
>> if i've understood correctly the script runs glxgears for 43 seconds 
>> and in that time generates random numbers in a random number of times 
>> (processes, fork and forget), is that it?
>>
> No, I haven't made it clear. A known number (default four) of xterms 
> are started, each of which calculates random numbers and prints them, 
> using much CPU time and causing a lot of scrolling. At the same time 
> glxgears is running, and the smoothness (or not) is observed manually. 
> The script records raw data on the number of frames per second and the 
> number of random numbers calculated by each shell. Since these are 
> FAIR schedulers, the variance between the scripts, and between 
> multiple samples from glxgears is of interest. To avoid startup 
> effects the glxgears value from the first sample is reported 
> separately and not included in the statistics.
>
> I looked at your results, and they are disturbing to say the least, it 
> appears that using the ck2 scheduler glxgears stopped for all 
> practical purposes. You don't have quite the latest glitch1, the new 
> one runs longer and allows reruns to get several datasets, but the 
> results still show very slow gears and a large difference between the 
> work done by the four shells. That's not a good result, how did the 
> system feel?
>> You find the data, for 2.6.21-{cfs-v13, ck2} in 
>> http://www.debianpt.org/~elmig/pool/kernel/20070522/
>>
> Thank you, these results are very surprising, and I would not expect 
> the system to be pleasing the use under load, based on this.
>> Here's the funny part...
>>
>> Lets call:
>>
>> a) to "random number of processes run while glxgears is running", 
>> gl_fairloops file
>>
> It's really the relative work done by identical processes, hopefully 
> they are all nearly the same, magnitude is interesting but related to 
> responsiveness rather than fairness.
>> b) to "generated frames while running a burst of processes" aka 
>> "massive and uknown amount of operations in one process", gl_gears file
>>
> Well, top or ps will give you a good idea of processing, but it tried 
> to use all of one CPU if allowed. Again, similarity of samples 
> reflects fairness and magnitude reflects work done.
>> kernel    2.6.21-cfs-v13    2.6.21-ck2
>> a)    194464        254669       b)    54159        124
>>
>>
> Everyone seems to like ck2, this makes it look as if the video display 
> would be really pretty unusable. While sd-0.48 does show an occasional 
> video glitch when watching video under heavy load, it's annoying 
> rather than unusable.
>
I spent a few hours running the -ck2 patch, and I didn't see any numbers 
like yours. What I did see is going up with my previous results as 
http://www.tmr.com/~davidsen/sched_smooth_04.html. While there were 
still some minor pauses in glxgears with my test, performance was very 
similar to the sd-0.48 results. And I did try watching video with high 
load, without problems. Only when I run a lot of other screen-changing 
processes can I see pauses in the display.
> Your subjective impressions would be helpful, and you may find that 
> the package in the www.tmr.com/~public/source is slightly easier to 
> use and gives more stable results. The documentation suggests the way 
> to take samples (the way I did it) but if you feel more or longer 
> samples would help it is tunable.
>
> I added Con to the cc list, he may have comments or suggestions 
> (against the current versions, please). Or he may feel that video 
> combined with other heavy screen updating is unrealistic or not his 
> chosen load. I'm told the load is similar to games which use threads 
> and do lots of independent action, if that's a reference.
>
I'll include the -ck2 patch in my testing on other hardware.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Miguel Figueiredo]

Bill Davidsen wrote:
> I was unable to reproduce the numbers Miguel generated, comments below. 
> The -ck2 patch seems to run nicely, although the memory repopulation 
> from swap would be most useful on system which have a lot of memory 
> pressure.
> 
> Bill Davidsen wrote:
>> Miguel Figueiredo wrote:
>>>
>>> Hi Bill,
>>>
>>> if i've understood correctly the script runs glxgears for 43 seconds 
>>> and in that time generates random numbers in a random number of times 
>>> (processes, fork and forget), is that it?
>>>
>> No, I haven't made it clear. A known number (default four) of xterms 
>> are started, each of which calculates random numbers and prints them, 
>> using much CPU time and causing a lot of scrolling. At the same time 
>> glxgears is running, and the smoothness (or not) is observed manually. 
>> The script records raw data on the number of frames per second and the 
>> number of random numbers calculated by each shell. Since these are 
>> FAIR schedulers, the variance between the scripts, and between 
>> multiple samples from glxgears is of interest. To avoid startup 
>> effects the glxgears value from the first sample is reported 
>> separately and not included in the statistics.
>>
>> I looked at your results, and they are disturbing to say the least, it 
>> appears that using the ck2 scheduler glxgears stopped for all 
>> practical purposes. You don't have quite the latest glitch1, the new 
>> one runs longer and allows reruns to get several datasets, but the 
>> results still show very slow gears and a large difference between the 
>> work done by the four shells. That's not a good result, how did the 
>> system feel?
>>> You find the data, for 2.6.21-{cfs-v13, ck2} in 
>>> http://www.debianpt.org/~elmig/pool/kernel/20070522/
>>>
>> Thank you, these results are very surprising, and I would not expect 
>> the system to be pleasing the use under load, based on this.
>>> Here's the funny part...
>>>
>>> Lets call:
>>>
>>> a) to "random number of processes run while glxgears is running", 
>>> gl_fairloops file
>>>
>> It's really the relative work done by identical processes, hopefully 
>> they are all nearly the same, magnitude is interesting but related to 
>> responsiveness rather than fairness.
>>> b) to "generated frames while running a burst of processes" aka 
>>> "massive and uknown amount of operations in one process", gl_gears file
>>>
>> Well, top or ps will give you a good idea of processing, but it tried 
>> to use all of one CPU if allowed. Again, similarity of samples 
>> reflects fairness and magnitude reflects work done.
>>> kernel    2.6.21-cfs-v13    2.6.21-ck2
>>> a)    194464        254669       b)    54159        124
>>>
>>>
>> Everyone seems to like ck2, this makes it look as if the video display 
>> would be really pretty unusable. While sd-0.48 does show an occasional 
>> video glitch when watching video under heavy load, it's annoying 
>> rather than unusable.
>>
> I spent a few hours running the -ck2 patch, and I didn't see any numbers 
> like yours. What I did see is going up with my previous results as 
> http://www.tmr.com/~davidsen/sched_smooth_04.html. While there were 
> still some minor pauses in glxgears with my test, performance was very 
> similar to the sd-0.48 results. And I did try watching video with high 
> load, without problems. Only when I run a lot of other screen-changing 
> processes can I see pauses in the display.
>> Your subjective impressions would be helpful, and you may find that 
>> the package in the www.tmr.com/~public/source is slightly easier to 
>> use and gives more stable results. The documentation suggests the way 
>> to take samples (the way I did it) but if you feel more or longer 
>> samples would help it is tunable.
>>
>> I added Con to the cc list, he may have comments or suggestions 
>> (against the current versions, please). Or he may feel that video 
>> combined with other heavy screen updating is unrealistic or not his 
>> chosen load. I'm told the load is similar to games which use threads 
>> and do lots of independent action, if that's a reference.
>>
> I'll include the -ck2 patch in my testing on other hardware.
> 

Hi Bill,

  the numbers i posted before are repeatable on that machine.


I did run, again, glitch1 on my laptop (T2500 CoreDuo, also Nvidia) 
please check: http://www.debianpt.org/~elmig/pool/kernel/20070523/



-- 

Com os melhores cumprimentos/Best regards,

Miguel Figueiredo
http://www.DebianPT.org

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Bill Davidsen]

Miguel Figueiredo wrote:
> Bill Davidsen wrote:
>> I was unable to reproduce the numbers Miguel generated, comments 
>> below. The -ck2 patch seems to run nicely, although the memory 
>> repopulation from swap would be most useful on system which have a 
>> lot of memory pressure.
>>
>> I spent a few hours running the -ck2 patch, and I didn't see any 
>> numbers like yours. What I did see is going up with my previous 
>> results as http://www.tmr.com/~davidsen/sched_smooth_04.html. While 
>> there were still some minor pauses in glxgears with my test, 
>> performance was very similar to the sd-0.48 results. And I did try 
>> watching video with high load, without problems. Only when I run a 
>> lot of other screen-changing processes can I see pauses in the display.
>>> Your subjective impressions would be helpful, and you may find that 
>>> the package in the www.tmr.com/~public/source is slightly easier to 
>>> use and gives more stable results. The documentation suggests the 
>>> way to take samples (the way I did it) but if you feel more or 
>>> longer samples would help it is tunable.
>>>
>>> I added Con to the cc list, he may have comments or suggestions 
>>> (against the current versions, please). Or he may feel that video 
>>> combined with other heavy screen updating is unrealistic or not his 
>>> chosen load. I'm told the load is similar to games which use threads 
>>> and do lots of independent action, if that's a reference.
>>>
>> I'll include the -ck2 patch in my testing on other hardware.
>>
>
> Hi Bill,
>
>  the numbers i posted before are repeatable on that machine.
>
The numbers you posted in <46534C31.1060306@debianpt.org> are not the 
same... From my inbox I grab some very non-matching values:
=====
Here's the funny part...

Lets call:

a) to "random number of processes run while glxgears is running", 
gl_fairloops file

b) to "generated frames while running a burst of processes" aka "massive 
and uknown amount of operations in one process", gl_gears file

kernel    2.6.21-cfs-v13    2.6.21-ck2
a)    194464        254669       
b)    54159        124
=====

The numbers in your glitch1.html file show a close correlation for cfs 
and -ck2, well within what I would expect. The stddev for the loops is 
larger for -cf2, but not out of line with what I see, and nothing like 
the numbers you originally sent me (which may have been testing 
something else, or from an old version before I made improvements, or 
???). In any case thanks for testing.

>
> I did run, again, glitch1 on my laptop (T2500 CoreDuo, also Nvidia) 
> please check: http://www.debianpt.org/~elmig/pool/kernel/20070523/
>
>
Thanks, those data seem as expected.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Miguel Figueiredo]

Bill Davidsen wrote:
> Miguel Figueiredo wrote:
>> Bill Davidsen wrote:
>>> I was unable to reproduce the numbers Miguel generated, comments 
>>> below. The -ck2 patch seems to run nicely, although the memory 
>>> repopulation from swap would be most useful on system which have a 
>>> lot of memory pressure.
>>>
>>> I spent a few hours running the -ck2 patch, and I didn't see any 
>>> numbers like yours. What I did see is going up with my previous 
>>> results as http://www.tmr.com/~davidsen/sched_smooth_04.html. While 
>>> there were still some minor pauses in glxgears with my test, 
>>> performance was very similar to the sd-0.48 results. And I did try 
>>> watching video with high load, without problems. Only when I run a 
>>> lot of other screen-changing processes can I see pauses in the display.
>>>> Your subjective impressions would be helpful, and you may find that 
>>>> the package in the www.tmr.com/~public/source is slightly easier to 
>>>> use and gives more stable results. The documentation suggests the 
>>>> way to take samples (the way I did it) but if you feel more or 
>>>> longer samples would help it is tunable.
>>>>
>>>> I added Con to the cc list, he may have comments or suggestions 
>>>> (against the current versions, please). Or he may feel that video 
>>>> combined with other heavy screen updating is unrealistic or not his 
>>>> chosen load. I'm told the load is similar to games which use threads 
>>>> and do lots of independent action, if that's a reference.
>>>>
>>> I'll include the -ck2 patch in my testing on other hardware.
>>>
>>
>> Hi Bill,
>>
>>  the numbers i posted before are repeatable on that machine.
>>
> The numbers you posted in <46534C31.1060306@debianpt.org> are not the 
> same... From my inbox I grab some very non-matching values:
> =====
> Here's the funny part...
> 
> Lets call:
> 
> a) to "random number of processes run while glxgears is running", 
> gl_fairloops file
> 
> b) to "generated frames while running a burst of processes" aka "massive 
> and uknown amount of operations in one process", gl_gears file
> 
> kernel    2.6.21-cfs-v13    2.6.21-ck2
> a)    194464        254669       b)    54159        124
> =====
> 
> The numbers in your glitch1.html file show a close correlation for cfs 
> and -ck2, well within what I would expect. The stddev for the loops is 
> larger for -cf2, but not out of line with what I see, and nothing like 
> the numbers you originally sent me (which may have been testing 
> something else, or from an old version before I made improvements, or 
> ???). In any case thanks for testing.
> 
>>
>> I did run, again, glitch1 on my laptop (T2500 CoreDuo, also Nvidia) 
>> please check: http://www.debianpt.org/~elmig/pool/kernel/20070523/
>>
>>
> Thanks, those data seem as expected.
> 

These numbers are from a different machine (this has 2 cores).
The other machine it's Debian Unstable and this Debian stable.


-- 

Com os melhores cumprimentos/Best regards,

Miguel Figueiredo
http://www.DebianPT.org

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Con Kolivas]

On Wednesday 23 May 2007 10:28, Bill Davidsen wrote:
> > kernel    2.6.21-cfs-v13    2.6.21-ck2
> > a)    194464        254669
> > b)    54159        124
>
> Everyone seems to like ck2, this makes it look as if the video display
> would be really pretty unusable. While sd-0.48 does show an occasional
> video glitch when watching video under heavy load, it's annoying rather
> than unusable.

That's because the whole premise of your benchmark relies on a workload that 
yield()s itself to the eyeballs on most graphic card combinations when using 
glxgears. Your test remains a test of sched_yield in the presence of your 
workloads rather than anything else. If people like ck2 it's because in the 
real world with real workloads it is better, rather than on a yield() based 
benchmark. Repeatedly the reports are that 3d apps and games in normal usage 
under -ck are better than mainline and cfs.

--
-ck

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[William Lee Irwin III]

On Wednesday 23 May 2007 10:28, Bill Davidsen wrote:
>>> kernel    2.6.21-cfs-v13    2.6.21-ck2
>>> a)    194464        254669
>>> b)    54159        124
>> Everyone seems to like ck2, this makes it look as if the video display
>> would be really pretty unusable. While sd-0.48 does show an occasional
>> video glitch when watching video under heavy load, it's annoying rather
>> than unusable.

On Thu, May 24, 2007 at 10:36:44AM +1000, Con Kolivas wrote:
> That's because the whole premise of your benchmark relies on a workload that 
> yield()s itself to the eyeballs on most graphic card combinations when using 
> glxgears. Your test remains a test of sched_yield in the presence of your 
> workloads rather than anything else. If people like ck2 it's because in the 
> real world with real workloads it is better, rather than on a yield() based 
> benchmark. Repeatedly the reports are that 3d apps and games in normal usage 
> under -ck are better than mainline and cfs.

Maybe people should explicitly hack on sched_yield() for these things
and do comparative benchmarking of sched_yield() implementations.


-- wli

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Michael Gerdau]

[-- Attachment #1: Type: text/plain, Size: 1662 bytes --]

> That's because the whole premise of your benchmark relies on a workload that 
> yield()s itself to the eyeballs on most graphic card combinations when using 
> glxgears. Your test remains a test of sched_yield in the presence of your 
> workloads rather than anything else. If people like ck2 it's because in the 
> real world with real workloads it is better, rather than on a yield() based 
> benchmark. Repeatedly the reports are that 3d apps and games in normal usage 
> under -ck are better than mainline and cfs.

While I can't comment on the technical/implementational details of
Con's claim I definitely have to agree from a users POV.

All my recent CPU intensive benchmarks show that both ck/sd and cfs
are very decent scheduler and IMO superior to mainline for all _my_
usecases. In particular playing supertux while otherwise fully utilizing
both CPUs on a dualcore works without any glitch and better than
on mainline for both sd and cfs.

For me the huge difference you have for sd to the others increases the
likelyhood the glxgears benchmark does not measure scheduling of graphic
but something else.

Anyway, I'm still in the process of collecting data or more precisely
until recently constantly refined what data to collect and how. I plan
to provide new benchmark results on CPU intensive tasks in a couple of
days.

Best,
Michael
-- 
 Technosis GmbH, Geschäftsführer: Michael Gerdau, Tobias Dittmar
 Sitz Hamburg; HRB 89145 Amtsgericht Hamburg
 Vote against SPAM - see http://www.politik-digital.de/spam/
 Michael Gerdau       email: mgd@technosis.de
 GPG-keys available on request or at public keyserver

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Xavier Bestel]

On Wed, 2007-05-23 at 07:23 +0200, Michael Gerdau wrote:
> For me the huge difference you have for sd to the others increases the
> likelyhood the glxgears benchmark does not measure scheduling of graphic
> but something else.

I think some people forget that X11 has its own scheduler for graphics
operations.

	Xav

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Bill Huey]

On Wed, May 23, 2007 at 09:58:35AM +0200, Xavier Bestel wrote:
> On Wed, 2007-05-23 at 07:23 +0200, Michael Gerdau wrote:
> > For me the huge difference you have for sd to the others increases the
> > likelyhood the glxgears benchmark does not measure scheduling of graphic
> > but something else.
> 
> I think some people forget that X11 has its own scheduler for graphics
> operations.

OpenGL is generally orthogonal to X11 or at least should be. But this could
vary with the implementation depending on how brain damaged the system is.

I'd expect the performance charateristics to be different depending on what
subsystem is being used.

bill

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Ian Romanick]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Xavier Bestel wrote:
> On Wed, 2007-05-23 at 07:23 +0200, Michael Gerdau wrote:
>> For me the huge difference you have for sd to the others increases the
>> likelyhood the glxgears benchmark does not measure scheduling of graphic
>> but something else.
> 
> I think some people forget that X11 has its own scheduler for graphics
> operations.

And in the direct-rendering case, this scheduler is not used for OpenGL.
 The client-side driver submits rendering commands directly to its
kernel module.  It is possible that some kernel modules perform some
sort of scheduling, but none of the open-source drivers implement such a
thing.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQFGVIZ7X1gOwKyEAw8RAottAJ9oQtnKVZ+xwrZXxyndanwkswp3IACeNY2v
JJeDU3cgBtn1dBOr3erl3Ww=
=YnG7
-----END PGP SIGNATURE-----

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Xavier Bestel]

Le mercredi 23 mai 2007 à 11:22 -0700, Ian Romanick a écrit :
> > I think some people forget that X11 has its own scheduler for graphics
> > operations.
> 
> And in the direct-rendering case, this scheduler is not used for OpenGL.
>  The client-side driver submits rendering commands directly to its
> kernel module.  It is possible that some kernel modules perform some
> sort of scheduling, but none of the open-source drivers implement such a
> thing.

True, but glxgears isn't a pure DRI app: it calls XNextEvent() each time
it refreshes its window (which is quite often).

	Xav

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Bill Davidsen]

Michael Gerdau wrote:
>> That's because the whole premise of your benchmark relies on a workload that 
>> yield()s itself to the eyeballs on most graphic card combinations when using 
>> glxgears. Your test remains a test of sched_yield in the presence of your 
>> workloads rather than anything else. If people like ck2 it's because in the 
>> real world with real workloads it is better, rather than on a yield() based 
>> benchmark. Repeatedly the reports are that 3d apps and games in normal usage 
>> under -ck are better than mainline and cfs.
>>     
>
> While I can't comment on the technical/implementational details of
> Con's claim I definitely have to agree from a users POV.
>
>   
Any of the sd/ck/cfs schedulers are an improvement on the current 
mainline, and hopefully they will continue to cross pollinate and 
evolve. Perhaps by 2.6.23 a clear "best" will emerge, or Linus will 
change his mind and make sd and cfs be compile options at build time.
> All my recent CPU intensive benchmarks show that both ck/sd and cfs
> are very decent scheduler and IMO superior to mainline for all _my_
> usecases. In particular playing supertux while otherwise fully utilizing
> both CPUs on a dualcore works without any glitch and better than
> on mainline for both sd and cfs.
>
>   
I did some kernel compile timing numbers as part of my work with 
ctxbench, and there is little to choose between the schedulers under 
load, although the special case for sched_yield makes some loads perform 
better with cfs. With large memory and fast disk, a kernel make becomes 
a CPU benchmark, there's virtually no iowait not filled with another 
process.
> For me the huge difference you have for sd to the others increases the
> likelyhood the glxgears benchmark does not measure scheduling of graphic
> but something else.
>
>   
The glitch1 script generates a number of CPU bound processes updating 
the screen independently, which stresses both graphics performance and 
scheduler fairness. And once again I note that it's a *characterization* 
rather than a benchmark. The ability of the scheduler to deliver the 
same resources to multiple identical processes, and to keep another CPU 
bound process (glxgears) getting the processor at regular intervals is 
more revealing than the frames per second or loops run.

I would expect sd to be better at this, since it uses a deadline 
concept, but in practice the gears pause, and then move rapidly or 
appear to jump. My reading on this is that the process starves for some 
ms, then gets a lot of CPU because it is "owed" more. I think I see this 
in games, but not being a game player I can't tell from experience if 
it's artifact or the games suck. That's what my test rig, based on a 15 
year old boy and several cans of high caffeine soda, is used for. ;-)
> Anyway, I'm still in the process of collecting data or more precisely
> until recently constantly refined what data to collect and how. I plan
> to provide new benchmark results on CPU intensive tasks in a couple of
> days.
>   
-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Bill Davidsen]

Con Kolivas wrote:
> On Wednesday 23 May 2007 10:28, Bill Davidsen wrote:
>   
>>> kernel    2.6.21-cfs-v13    2.6.21-ck2
>>> a)    194464        254669
>>> b)    54159        124
>>>       
>> Everyone seems to like ck2, this makes it look as if the video display
>> would be really pretty unusable. While sd-0.48 does show an occasional
>> video glitch when watching video under heavy load, it's annoying rather
>> than unusable.
>>     
>
> That's because the whole premise of your benchmark relies on a workload that 
> yield()s itself to the eyeballs on most graphic card combinations when using 
> glxgears. Your test remains a test of sched_yield in the presence of your 
> workloads rather than anything else. If people like ck2 it's because in the 
> real world with real workloads it is better, rather than on a yield() based 
> benchmark. Repeatedly the reports are that 3d apps and games in normal usage 
> under -ck are better than mainline and cfs.
>   
I have to admit that I call in the teen reserves to actually get good 
feedback on games, but I do watch a fair number of videos and under high 
load I find sd acceptable and cfs totally smooth. The next time my game 
expert comes to visit I'll get some subjective feedback. My use of 
glxgears was mainly intended to use something readily available, and 
which gave me the ability to make both subjective and objective evaluations.

My -ck2 results certainly show no significant difference from sd-0.48, I 
suspect that on a machine with less memory the swap reload would be more 
beneficial.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Sched - graphic smoothness under load - cfs-v13 sd-0.48

[Bill Davidsen]

Miguel Figueiredo wrote:
> Bill Davidsen wrote:
>> I generated a table of results from the latest glitch1 script, using 
>> an HTML postprocessor I not *quite* ready to foist on the word. In 
>> any case it has some numbers for frames per second, fairness of the 
>> processor time allocated to the compute bound processes which 
>> generate a lot of other screen activity for X, and my subjective 
>> comments on how smooth it looked and felt.
>>
>> The chart is at http://www.tmr.com/~davidsen/sched_smooth_01.html for 
>> your viewing pleasure. The only "tuned" result was with sd, since 
>> what I observed was so bad using the default settings. If any 
>> scheduler developers would like me to try other tunings or new 
>> versions let me know.
>>
>
> As I tryied myself kernels 2.6.21, 2.6.21-cfs-v13, and 2.6.21-ck2 on 
> the same machine i found *very* odd those numbers you posted, so i 
> tested myself those kernels to see the numbers I get instead of 
> talking about the usage of kernel xpto feels like.
>
> I did run glxgears with kernels 2.6.21, 2.6.21-cfs-v13 and 2.6.21-ck2 
> inside Debian's GNOME environment. The hardware is an AMD Sempron64 
> 3.0 GHz, 1 GB RAM, Nvidia 6800XT.
> Average and standard deviation from the gathered data:
>
> * 2.6.21:         average = 11251.1; stdev = 0.172
> * 2.6.21-cfs-v13:    average = 11242.8; stdev = 0.033
> * 2.6.21-ck2:        average = 11257.8; stdev = 0.067
>
> Keep in mind those numbers don't mean anything we all know glxgears is 
> not a benchmark, their purpose is only to be used as comparison under 
> the same conditions.
>
> One odd thing i noticed, with 2.6.21-cfs-v13 the gnome's time applet 
> in the bar skipped some minutes (e.g. 16:23 -> 16:25) several times.
>
> The data is available on: 
> http://www.debianPT.org/~elmig/pool/kernel/20070520/
>
>
> How did you get your data? I am affraid your data it's wrong, there's 
> no  such big difference between the schedulers...
>
The glitch1 script starts multiple scrolling xterms at the same time as 
the glxgears, and allows observation of smoothness of the gears. It's 
not a benchmark, although the fps is reported since fast or slow and 
scheduler with "fair" aspirations should have similar results in 5 sec 
time slices, and between multiple CPU-bound xterms scrolling  with the 
same code. The comments column can be used to report the user 
impressions, since that's the important thing if you want to listen to 
music or watch video.

Perhaps my data appear "wrong" because you have failed to measure the 
same thing?

You can get the most recent info at http://www.tmr.com/~public/source/ 
if you want to duplicate the test on your hardware, or view the most 
recent tests at http://www.tmr.com/~davidsen/sched_smooth_03.html to see 
what the data look like when you run the same test.

Note: there have been some minor changes in the test and analysis 
resulting from suggestions, only the recent results are worth investigating.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979