Re: Merge of per task delay accounting (was Re: 2.6.18 -mm merge plans)

[Martin Peschke <mp3@de.ibm.com>]

Jay Lan wrote:
> Shailabh Nagar wrote:
>> Balbir Singh wrote:

>> Andrew,
>>
>> The only other new set of patches to be discussed in this context are the
>> statistics-infrastructure patches from Martin Peschke.
>>
>> That infrastructure cannot meet the needs of delay accounting, CSA 
>> etc. because
>> - it only provides "user pull" model of getting stats whereas "kernel 
>> push" is
>> needed for delay accounting
> 
> Doesn't taskstats interface provide "user pull" request-reply model
> also? Serious accounting needs to push accounting data as soon as
> possible.
> 
>> - it uses a relatively slow interface unsuitable for high volumes of 
>> data. 

By design.

I think it would be fatal to report every event relevant
to statistical data gathering up to user space. It's fine to have
the kernel maintain counters and to provide preprocessed data.

Given that, is there a need for a high-speed interface for a
huge amount of unprocessed statistical data?

However, the user interface is a just one building brick,
which can be enhanced or replaced with moderate effort, if
there is a need.

 >> Each statistic has its own definition,

Allowing users to restrict accounting to what they need in their
particular case. Sensible defaults are usually available.

>> needs to be read separately using ASCII,
>> reading data continuously means open/read/close each time.....all of
>> which is not very conducive to large structures being sent to userspace.

Debugfs file are fine for larger structures.

Unless one keeps reading statistics dozens of times per second,
I don't see an issue with that.

The question is: what are the requirements to be covered?

> Yes, i second the point. It won't be able to catch up the traffic.
> 
>> - its oriented towards sampled data whereas taskstats isn't.
>>
>> So, we have a good consensus from existing/potential users of 
>> taskstats and would
>> very much appreciate it being included in 2.6.18.
> 
> Andrew, it has become clear that the community wants to see accounting
> data processing being moved to userspace. Thus there is a need for a
> common accounting interface to provide minimal works at kernel (via
> hooks at fork and exit) and deliver data to userspace.

Both, the statistics infrastructure on behalf of its exploiters as well
as the exploiters of the taskstats interface do data preprocessing,
that is, maintain counters in the kernel.
User space counters won't perform, of course.

AFAICS, actual differences are:
- triggers for data delivery to user space
   (statistics infrastructure: when user reads statistics through file,
    taskstats: on certain task related events, right?)
- and, therewith, frequency of data delivery to user space


Martin