Re: Merge of per task delay accounting (was Re: 2.6.18 -mm merge plans)

[Jay Lan <jlan@sgi.com>]

Shailabh Nagar wrote:
> Balbir Singh wrote:
> 
>>Andrew Morton wrote:
>>
>>
>>>per-task-delay-accounting-setup.patch
>>>per-task-delay-accounting-setup-fix-1.patch
>>>per-task-delay-accounting-setup-fix-2.patch
>>>per-task-delay-accounting-sync-block-i-o-and-swapin-delay-collection.patch
>>>
>>>per-task-delay-accounting-sync-block-i-o-and-swapin-delay-collection-fix-1.patch
>>>
>>>per-task-delay-accounting-cpu-delay-collection-via-schedstats.patch
>>>per-task-delay-accounting-cpu-delay-collection-via-schedstats-fix-1.patch
>>>per-task-delay-accounting-utilities-for-genetlink-usage.patch
>>>per-task-delay-accounting-taskstats-interface.patch
>>>per-task-delay-accounting-taskstats-interface-fix-1.patch
>>>per-task-delay-accounting-taskstats-interface-fix-2.patch
>>>per-task-delay-accounting-delay-accounting-usage-of-taskstats-interface.patch
>>>
>>>per-task-delay-accounting-delay-accounting-usage-of-taskstats-interface-use-portable-cputime-api-in-__delayacct_add_tsk.patch
>>>
>>>per-task-delay-accounting-documentation.patch
>>>per-task-delay-accounting-proc-export-of-aggregated-block-i-o-delays.patch
>>>
>>>per-task-delay-accounting-proc-export-of-aggregated-block-i-o-delays-warning-fix.patch
>>>
>>>
>>> I just don't know.  There are a number of groups who pop up with various
>>> enhanced accounting requirements and patches (all quite different) but I
>>> haven't heard a lot of enthusiasm from any of them over this work, which
>>> attempts to provide an extensible framework for accumulation and
>>>querying
>>> of per-task metrics.
>>>
>>> But then again, we cannot just sit there and wait for everyone to be
>>>100%
>>> happy.  So I'm 51% inclined to push this along.
>>>
>>> Anyone else who has an interest in this sort of thing needs to be aware
>>> that there will be an expectation that any future statistics submissions
>>> should use these interfaces.  So the time to pay attention is right now.
>>>
>>
>>Hi, Andrew,
>>
>>Here is a brief summary of the status of the response we have received from
>>the stakeholders (some of it has been duplicated in previous postings)
>>
>>Project                                         Response
>>
>>1. CSA accounting/PAGG/JOB:                    Has agreed to use taskstats
>> Jay Lan <jlan@engr.sgi.com>                  interface
>>
>>2. per-process IO statistics:                  None
>> Levent Serinol <lserinol@gmail.com>          Needs are subset of CSA
>>
>>3. per-cpu time statistics:                    None (email bounced)
>> Erich Focht <efocht@ess.nec.de>              Needs can be met by taskstats
>>                                              Statistics not yet submitted
>>
>>4. Microstate accounting:                      None
>> Peter Chubb <peterc@gelato.unsw.edu.au>      overlap with delay accounting
>>                                              prefers /proc due to
>>convenience
>>                                              taskstats can meet the needs
>>
>>
>>5. ELSA: Guillaume Thouvenin                   None
>> <guillaume.thouvenin@bull.net>               ELSA is not a direct user
>>                                              of new kernel statistics
>>                                              Consumer of CSA/BSD
>>accounting
>>                                              statistics
>>
>>6. pnotify: Jes Sorensen <jes@sgi.com>         None
>>(taken over pnotify from Erik Jacobson)        Informed over private email
>>                                              that pnotify replacement is
>>                                              being worked on. pnotify
>>                                              or its replacement will
>>                                              not be concerned with
>>                                              exporting data to user space
>>                                              or collecting any statistics.
>>
>>
>>7. Scalable statistics counters with /proc      Not working on it
>> reporting:                                   anymore
>> Ravikiran G Thirumalai,
>> Dipankar Sarma <dipankar@in.ibm.com>
>>
>>Studying the responses from all stake holders, Jay Lan's was the most
>>encouraging. Peter Chubb prefers the /proc interface due to the text
>>interface
>>and ease of parsing. (in our opinion, taskstats can meet the needs easily
>>and the getdelays utility can provide the same ease for parsing).
>>The others did not respond.
>>Some performance numbers of taskstats were posted at
>>http://lkml.org/lkml/2006/3/23/141. The result highlights are included
>>below
>>
>>   Results highlights
>>
>>   - Configuring delay accounting adds < 0.5%
>>     overhead in most cases and even reduces overhead
>>     in some cases
>>
>>   - Enabling delay accounting has similar results
>>     with a maximum overhead of 1.2% for hackbench,
>>     most other overheads < 1% and reduction in
>>     overhead in some cases
>>
>>These statistics are _per task_ and can be extended easily by anyone
>>who wishes to obtain per task data. An example of per task improved
>>scheduler statistics was mentioned in http://lkml.org/lkml/2006/6/1/381
>>(I am not sure if the email refers to our per-task statistics). If not,
>>the new statistics could easily use the taskstats interface.
>>
>>These statistics can be used by software product stacks to monitor
>>usage information about the various tasks they create and control.
>>I also informally spoke to a group of students (verbally), who were
>>excited at the possibility of using the per-task statistics to do
>>dynamic deadline based power management. They want to use the delay data
>>(CPU and IO) to predict deadlines for a task and then use these results
>>for dynamically scaling CPU frequency.
>>
>>
>>The ability to monitor the CPU run and delay data and IO delay data is
>>useful.
>>
>>I would request you to consider the inclusion per-task delay accounting
>>into
>>2.6.18.
>>
> 
> 
> 
> Andrew,
> 
> The only other new set of patches to be discussed in this context are the
> statistics-infrastructure patches from Martin Peschke.
> 
> That infrastructure cannot meet the needs of delay accounting, CSA etc. because
> - it only provides "user pull" model of getting stats whereas "kernel push" is
> needed for delay accounting

Doesn't taskstats interface provide "user pull" request-reply model
also? Serious accounting needs to push accounting data as soon as
possible.

> - it uses a relatively slow interface unsuitable for high volumes of data. Each
> statistic has its own definition, needs to be read separately using ASCII,
> reading data continuously means open/read/close each time.....all of
> which is not very conducive to large structures being sent to userspace.

Yes, i second the point. It won't be able to catch up the traffic.

> - its oriented towards sampled data whereas taskstats isn't.
> 
> So, we have a good consensus from existing/potential users of taskstats and would
> very much appreciate it being included in 2.6.18.

Andrew, it has become clear that the community wants to see accounting
data processing being moved to userspace. Thus there is a need for a
common accounting interface to provide minimal works at kernel (via
hooks at fork and exit) and deliver data to userspace.

The delayacct patchset provides a good framework and example that
i believe CSA/Job can follow and build upon to move most of our work
to userspace and thus cut off dependency of PAGG. We will submit CSA
patch soon based on the taskstats interface.

Thanks,
  - jay

P.S. Balbir and Shailabh, Chris Sturtivant will continue the CSA work
      at SGI. Please also cc Chris <csturtiv@sgi.com> in the future.
      Thanks!


> 
> --Shailabh
> 
> 
> 
>