Incorrect CPU process accounting using CONFIG_HZ=100

Incorrect CPU process accounting using CONFIG_HZ=100

[Al Boldi]

Setting CONFIG_HZ=100 results in incorrect CPU process accounting.

This can be seen running top d.1, that shows top, itself, consuming 0ms 
CPUtime.

Will this bug have consequences for sched.c?

Thanks!

--
Al

Re: Incorrect CPU process accounting using CONFIG_HZ=100

[Jan Engelhardt]

>
>
>Setting CONFIG_HZ=100 results in incorrect CPU process accounting.
>
>This can be seen running top d.1, that shows top, itself, consuming 0ms 
>CPUtime.
>
>Will this bug have consequences for sched.c?

Works for me, somewhat.
TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this CPU.)

>Thanks!

Jan Engelhardt
-- 

Re: Incorrect CPU process accounting using CONFIG_HZ=100

[Al Boldi]

Jan Engelhardt wrote:
> >Setting CONFIG_HZ=100 results in incorrect CPU process accounting.
> >
> >This can be seen running top d.1, that shows top, itself, consuming 0ms
> >CPUtime.
> >
> >Will this bug have consequences for sched.c?
>
> Works for me, somewhat.
> TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this
> CPU.)

That's what I thought for a long time.  But at closer inspection, top d.1 
slows down other apps by about the same amount of time at 1000Hz and 100Hz, 
only at 1000Hz it is accounted for whereas at 100Hz it is not.

Thanks!

--
Al

Re: Incorrect CPU process accounting using CONFIG_HZ=100

[Pavel Machek]

On Thu 2006-06-22 20:36:39, Al Boldi wrote:
> Jan Engelhardt wrote:
> > >Setting CONFIG_HZ=100 results in incorrect CPU process accounting.
> > >
> > >This can be seen running top d.1, that shows top, itself, consuming 0ms
> > >CPUtime.
> > >
> > >Will this bug have consequences for sched.c?
> >
> > Works for me, somewhat.
> > TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this
> > CPU.)
> 
> That's what I thought for a long time.  But at closer inspection, top d.1 
> slows down other apps by about the same amount of time at 1000Hz and 100Hz, 
> only at 1000Hz it is accounted for whereas at 100Hz it is not.

It is not a bug... it is design decision. If you eat "too little" cpu
time, you'll be accouted 0 msec. That's what happens at 100Hz...
										Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: Incorrect CPU process accounting using CONFIG_HZ=100

[Al Boldi]

Pavel Machek wrote:
> On Thu 2006-06-22 20:36:39, Al Boldi wrote:
> > Jan Engelhardt wrote:
> > > >Setting CONFIG_HZ=100 results in incorrect CPU process accounting.
> > > >
> > > >This can be seen running top d.1, that shows top, itself, consuming
> > > > 0ms CPUtime.
> > > >
> > > >Will this bug have consequences for sched.c?
> > >
> > > Works for me, somewhat.
> > > TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this
> > > CPU.)
> >
> > That's what I thought for a long time.  But at closer inspection, top
> > d.1 slows down other apps by about the same amount of time at 1000Hz and
> > 100Hz, only at 1000Hz it is accounted for whereas at 100Hz it is not.
>
> It is not a bug... it is design decision. If you eat "too little" cpu
> time, you'll be accouted 0 msec. That's what happens at 100Hz...

Bummer!

Meanwhile, can't "too little" cpu time be made relative to CONFIG_HZ?

Thanks!

--
Al

Re: Incorrect CPU process accounting using CONFIG_HZ=100

[Con Kolivas]

On Tuesday 27 June 2006 22:32, Al Boldi wrote:
> Pavel Machek wrote:
> > On Thu 2006-06-22 20:36:39, Al Boldi wrote:
> > > Jan Engelhardt wrote:
> > > > >Setting CONFIG_HZ=100 results in incorrect CPU process accounting.
> > > > >
> > > > >This can be seen running top d.1, that shows top, itself, consuming
> > > > > 0ms CPUtime.
> > > > >
> > > > >Will this bug have consequences for sched.c?
> > > >
> > > > Works for me, somewhat.
> > > > TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this
> > > > CPU.)
> > >
> > > That's what I thought for a long time.  But at closer inspection, top
> > > d.1 slows down other apps by about the same amount of time at 1000Hz
> > > and 100Hz, only at 1000Hz it is accounted for whereas at 100Hz it is
> > > not.
> >
> > It is not a bug... it is design decision. If you eat "too little" cpu
> > time, you'll be accouted 0 msec. That's what happens at 100Hz...
>
> Bummer!
>
> Meanwhile, can't "too little" cpu time be made relative to CONFIG_HZ?

It is and that's what you're perceiving as the problem. We only charge tasks 
in ticks and it's the tick size they get charged with. So at 100HZ if a task 
is running when a tick fires it gets charged 1% cpu. If it runs for 100 ticks 
it gets charged with 100% cpu. At 1000HZ it gets charged .1% cpu per tick and 
so on. The actual problem is that tasks only get charged if they happen to be 
running at the precise moment the tick fires. Now you could increase the 
accuracy of this timekeeping but it is expensive and this is exactly the sort 
of thing that we're saving cpu resources on by running at 100HZ (one of 
many).

-- 
-ck

Re: Incorrect CPU process accounting using CONFIG_HZ=100

[Peter Williams]

Con Kolivas wrote:
> On Tuesday 27 June 2006 22:32, Al Boldi wrote:
>> Pavel Machek wrote:
>>> On Thu 2006-06-22 20:36:39, Al Boldi wrote:
>>>> Jan Engelhardt wrote:
>>>>>> Setting CONFIG_HZ=100 results in incorrect CPU process accounting.
>>>>>>
>>>>>> This can be seen running top d.1, that shows top, itself, consuming
>>>>>> 0ms CPUtime.
>>>>>>
>>>>>> Will this bug have consequences for sched.c?
>>>>> Works for me, somewhat.
>>>>> TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on this
>>>>> CPU.)
>>>> That's what I thought for a long time.  But at closer inspection, top
>>>> d.1 slows down other apps by about the same amount of time at 1000Hz
>>>> and 100Hz, only at 1000Hz it is accounted for whereas at 100Hz it is
>>>> not.
>>> It is not a bug... it is design decision. If you eat "too little" cpu
>>> time, you'll be accouted 0 msec. That's what happens at 100Hz...
>> Bummer!
>>
>> Meanwhile, can't "too little" cpu time be made relative to CONFIG_HZ?
> 
> It is and that's what you're perceiving as the problem. We only charge tasks 
> in ticks and it's the tick size they get charged with. So at 100HZ if a task 
> is running when a tick fires it gets charged 1% cpu. If it runs for 100 ticks 
> it gets charged with 100% cpu. At 1000HZ it gets charged .1% cpu per tick and 
> so on. The actual problem is that tasks only get charged if they happen to be 
> running at the precise moment the tick fires. Now you could increase the 
> accuracy of this timekeeping but it is expensive and this is exactly the sort 
> of thing that we're saving cpu resources on by running at 100HZ (one of 
> many).
> 

It could be (partly) done fairly cheaply in nanoseconds if sched_clock() 
was reliable enough (but comments on this mail list indicate that it 
currently isn't) as it is already called in all the right places for 
getting the total cpu time used (so just a subtraction, addition and 
assignment).  The reason that I say partly is that splitting the time 
into "system" and "user" would be a more complex problem.

Peter
PS It's also called already at appropriate places for measuring total 
sleep and total cpu delay time (i.e. time spent on a run queue waiting 
to get on to a CPU).
PPS The task time stamp is also already updated at all the appropriate 
places for use in this mechanism.
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce

Re: Incorrect CPU process accounting using CONFIG_HZ=100

[Al Boldi]

Peter Williams wrote:
> Con Kolivas wrote:
> > On Tuesday 27 June 2006 22:32, Al Boldi wrote:
> >> Pavel Machek wrote:
> >>> On Thu 2006-06-22 20:36:39, Al Boldi wrote:
> >>>> Jan Engelhardt wrote:
> >>>>>> Setting CONFIG_HZ=100 results in incorrect CPU process accounting.
> >>>>>>
> >>>>>> This can be seen running top d.1, that shows top, itself, consuming
> >>>>>> 0ms CPUtime.
> >>>>>>
> >>>>>> Will this bug have consequences for sched.c?
> >>>>>
> >>>>> Works for me, somewhat.
> >>>>> TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on
> >>>>> this CPU.)
> >>>>
> >>>> That's what I thought for a long time.  But at closer inspection, top
> >>>> d.1 slows down other apps by about the same amount of time at 1000Hz
> >>>> and 100Hz, only at 1000Hz it is accounted for whereas at 100Hz it is
> >>>> not.
> >>>
> >>> It is not a bug... it is design decision. If you eat "too little" cpu
> >>> time, you'll be accouted 0 msec. That's what happens at 100Hz...
> >>
> >> Bummer!
> >>
> > The actual problem is that tasks
> > only get charged if they happen to be running at the precise moment the
> > tick fires. Now you could increase the accuracy of this timekeeping but
> > it is expensive and this is exactly the sort of thing that we're saving
> > cpu resources on by running at 100HZ (one of many).
>
> It could be (partly) done fairly cheaply in nanoseconds if sched_clock()
> was reliable enough (but comments on this mail list indicate that it
> currently isn't) as it is already called in all the right places for
> getting the total cpu time used (so just a subtraction, addition and
> assignment).  The reason that I say partly is that splitting the time
> into "system" and "user" would be a more complex problem.

If I am reading this correctly, then the kernel is accounting process times 
twice:
	1. for external proc monitoring, using a probed approach
	2. for scheduling, using an inlined approach

Wouldn't merging the two approaches be in the interest of conserving cpu 
resources, while at the same time reflecting an accurate view of cpu 
utilization?

Thanks!

--
Al

Re: Incorrect CPU process accounting using CONFIG_HZ=100

[Peter Williams]

Al Boldi wrote:
> Peter Williams wrote:
>> Con Kolivas wrote:
>>> On Tuesday 27 June 2006 22:32, Al Boldi wrote:
>>>> Pavel Machek wrote:
>>>>> On Thu 2006-06-22 20:36:39, Al Boldi wrote:
>>>>>> Jan Engelhardt wrote:
>>>>>>>> Setting CONFIG_HZ=100 results in incorrect CPU process accounting.
>>>>>>>>
>>>>>>>> This can be seen running top d.1, that shows top, itself, consuming
>>>>>>>> 0ms CPUtime.
>>>>>>>>
>>>>>>>> Will this bug have consequences for sched.c?
>>>>>>> Works for me, somewhat.
>>>>>>> TIME+ says 0:00.02 after 70 secs. (Ergo: top is not expensive on
>>>>>>> this CPU.)
>>>>>> That's what I thought for a long time.  But at closer inspection, top
>>>>>> d.1 slows down other apps by about the same amount of time at 1000Hz
>>>>>> and 100Hz, only at 1000Hz it is accounted for whereas at 100Hz it is
>>>>>> not.
>>>>> It is not a bug... it is design decision. If you eat "too little" cpu
>>>>> time, you'll be accouted 0 msec. That's what happens at 100Hz...
>>>> Bummer!
>>>>
>>> The actual problem is that tasks
>>> only get charged if they happen to be running at the precise moment the
>>> tick fires. Now you could increase the accuracy of this timekeeping but
>>> it is expensive and this is exactly the sort of thing that we're saving
>>> cpu resources on by running at 100HZ (one of many).
>> It could be (partly) done fairly cheaply in nanoseconds if sched_clock()
>> was reliable enough (but comments on this mail list indicate that it
>> currently isn't) as it is already called in all the right places for
>> getting the total cpu time used (so just a subtraction, addition and
>> assignment).  The reason that I say partly is that splitting the time
>> into "system" and "user" would be a more complex problem.
> 
> If I am reading this correctly, then the kernel is accounting process times 
> twice:
> 	1. for external proc monitoring, using a probed approach
> 	2. for scheduling, using an inlined approach

Not exactly (e.g. there's no separation between user and sys time 
available in line) but the possibilities are there.

> 
> Wouldn't merging the two approaches be in the interest of conserving cpu 
> resources, while at the same time reflecting an accurate view of cpu 
> utilization?

I think that this would be a worthwhile endeavour once/if sched_clock() 
is fixed.  This is especially the case as CPUs get faster as many tasks 
may run to completion in less than a tick.

Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce

Re: Incorrect CPU process accounting using CONFIG_HZ=100

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1004 bytes --]

Peter Williams writes:

> Al Boldi wrote:
>> Peter Williams wrote:

>> twice:
>> 	1. for external proc monitoring, using a probed approach
>> 	2. for scheduling, using an inlined approach
> 
> Not exactly (e.g. there's no separation between user and sys time 
> available in line) but the possibilities are there.
> 
>> 
>> Wouldn't merging the two approaches be in the interest of conserving cpu 
>> resources, while at the same time reflecting an accurate view of cpu 
>> utilization?
> 
> I think that this would be a worthwhile endeavour once/if sched_clock() 
> is fixed.  This is especially the case as CPUs get faster as many tasks 
> may run to completion in less than a tick.

That may not be as simple as it seems. To properly account system v user 
time using the sched_clock we'd have to hook into arch dependant asm code to 
know when entering and exiting kernel context. That is far more invasive 
than the simple on/off runqueue timing we currently do for scheduling 
accounting.

--
-ck


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Incorrect CPU process accounting using CONFIG_HZ=100

[Peter Williams]

Con Kolivas wrote:
> Peter Williams writes:
> 
>> Al Boldi wrote:
>>> Peter Williams wrote:
> 
>>> twice:
>>>     1. for external proc monitoring, using a probed approach
>>>     2. for scheduling, using an inlined approach
>>
>> Not exactly (e.g. there's no separation between user and sys time 
>> available in line) but the possibilities are there.
>>
>>>
>>> Wouldn't merging the two approaches be in the interest of conserving 
>>> cpu resources, while at the same time reflecting an accurate view of 
>>> cpu utilization?
>>
>> I think that this would be a worthwhile endeavour once/if 
>> sched_clock() is fixed.  This is especially the case as CPUs get 
>> faster as many tasks may run to completion in less than a tick.
> 
> That may not be as simple as it seems. To properly account system v user 
> time using the sched_clock we'd have to hook into arch dependant asm 
> code to know when entering and exiting kernel context. That is far more 
> invasive than the simple on/off runqueue timing we currently do for 
> scheduling accounting.

Yes, it is a problem and we may have to do something approximate like 
counting ticks for sys time and subtracting that from the total to get 
user time when reporting the times to user space (only a bit more 
complex to make sure we don't end up with negative times).

How is it intended to handle this problem in the tickless kernel?

Peter
PS It's all moot until sched_clock() is fixed anyway.
PPS Recent kernels (-mm ones at least) keep a sched_time in nsecs for 
each task but I've no idea what it's used for.
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce

Re: Incorrect CPU process accounting using CONFIG_HZ=100

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1715 bytes --]

Peter Williams writes:

> Con Kolivas wrote:
>> Peter Williams writes:
>> 
>>> Al Boldi wrote:
>>>> Peter Williams wrote:
>> 
>>>> twice:
>>>>     1. for external proc monitoring, using a probed approach
>>>>     2. for scheduling, using an inlined approach
>>>
>>> Not exactly (e.g. there's no separation between user and sys time 
>>> available in line) but the possibilities are there.
>>>
>>>>
>>>> Wouldn't merging the two approaches be in the interest of conserving 
>>>> cpu resources, while at the same time reflecting an accurate view of 
>>>> cpu utilization?
>>>
>>> I think that this would be a worthwhile endeavour once/if 
>>> sched_clock() is fixed.  This is especially the case as CPUs get 
>>> faster as many tasks may run to completion in less than a tick.
>> 
>> That may not be as simple as it seems. To properly account system v user 
>> time using the sched_clock we'd have to hook into arch dependant asm 
>> code to know when entering and exiting kernel context. That is far more 
>> invasive than the simple on/off runqueue timing we currently do for 
>> scheduling accounting.
> 
> Yes, it is a problem and we may have to do something approximate like 
> counting ticks for sys time and subtracting that from the total to get 
> user time when reporting the times to user space (only a bit more 
> complex to make sure we don't end up with negative times).
> 
> How is it intended to handle this problem in the tickless kernel?

The tickless name is a misnomer. The implemenation I had and the one that 
tglx/mingo are maintaining still tick but they skip idle ticks. This means 
that all we need to do is add the number of skipped idle ticks whenever we 
tick to the idle count.

--
-ck


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Incorrect CPU process accounting using CONFIG_HZ=100

[Pavel Machek]

Hi!

> >>>Wouldn't merging the two approaches be in the interest of conserving 
> >>>cpu resources, while at the same time reflecting an accurate view of 
> >>>cpu utilization?
> >>
> >>I think that this would be a worthwhile endeavour once/if 
> >>sched_clock() is fixed.  This is especially the case as CPUs get 
> >>faster as many tasks may run to completion in less than a tick.
> >
> >That may not be as simple as it seems. To properly account system v user 
> >time using the sched_clock we'd have to hook into arch dependant asm 
> >code to know when entering and exiting kernel context. That is far more 
> >invasive than the simple on/off runqueue timing we currently do for 
> >scheduling accounting.
> 
> Yes, it is a problem and we may have to do something approximate like 
> counting ticks for sys time and subtracting that from the total to get 
> user time when reporting the times to user space (only a bit more 
> complex to make sure we don't end up with negative times).

Yes, please.

> How is it intended to handle this problem in the tickless kernel?

Even our "tickless" kernels do tick while CPU is busy.
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html