taskstats interface for accounting

taskstats interface for accounting

[Jay Lan]

Hi Balbir and Shailabh,

I finally have time to think about implementation details of CSA over
taskstats interface. I took another look at the taskstats interface
proposal and was a little bit nervous.

Do you remember i suggested to use #ifdef to cut down traffic and i
was told a generic netlink header would serve the purpose?
What i see now at Documentation/accounting/taskstats.txt saying
NETLINK_GENERIC family is used for unicast query/reply mode. The
NETLINK_GENERIC family provides great flexsibility on what to receive. 
However, CSA only uses the multicast mode to receive from kernel
whenever tasks are existing. I guess i would need to read the netlink
documentation more carefully to see whether my understanding is
correct.

Another thing i overlooked when i did the review was that taskstats
interface is designed to provide _BOTH_ per task _AND_ per thread
accounting data EVERY TIME a task exists. A thread is an aggregate
of (per-pid) tasks. Since this type of aggregation is not used in
CSA, half of data traffic would be useless. Can we add a way to
configure to not send per-thread data to the socket?

Regards,
jay

Re: taskstats interface for accounting

[Shailabh Nagar]

Jay Lan wrote:

> Hi Balbir and Shailabh,
>
> I finally have time to think about implementation details of CSA over
> taskstats interface. I took another look at the taskstats interface
> proposal and was a little bit nervous.
>
> Do you remember i suggested to use #ifdef to cut down traffic and i
> was told a generic netlink header would serve the purpose?
> What i see now at Documentation/accounting/taskstats.txt saying
> NETLINK_GENERIC family is used for unicast query/reply mode. The
> NETLINK_GENERIC family provides great flexsibility on what to receive. 
> However, CSA only uses the multicast mode to receive from kernel
> whenever tasks are existing. I guess i would need to read the netlink
> documentation more carefully to see whether my understanding is
> correct.

I don't think there's a problem here. The netlink socket opened for 
listening to
taskstat data sent on task exit can be done in multicast mode....the 
example code in the
Documentation does that.

>
> Another thing i overlooked when i did the review was that taskstats
> interface is designed to provide _BOTH_ per task _AND_ per thread
> accounting data EVERY TIME a task exists. A thread is an aggregate
> of (per-pid) tasks. Since this type of aggregation is not used in
> CSA, half of data traffic would be useless. Can we add a way to
> configure to not send per-thread data to the socket?

I don't see why not. We could extend the command set to set tgid sending 
on/off.

Regards,
Shailabh

> Regards,
> jay

Re: taskstats interface for accounting

[Jay Lan]

Shailabh Nagar wrote:
> Jay Lan wrote:
>
>> Hi Balbir and Shailabh,
>>
>> I finally have time to think about implementation details of CSA over
>> taskstats interface. I took another look at the taskstats interface
>> proposal and was a little bit nervous.
>>
>> Do you remember i suggested to use #ifdef to cut down traffic and i
>> was told a generic netlink header would serve the purpose?
>> What i see now at Documentation/accounting/taskstats.txt saying
>> NETLINK_GENERIC family is used for unicast query/reply mode. The
>> NETLINK_GENERIC family provides great flexsibility on what to
>> receive. However, CSA only uses the multicast mode to receive from
>> kernel
>> whenever tasks are existing. I guess i would need to read the netlink
>> documentation more carefully to see whether my understanding is
>> correct.
>
> I don't think there's a problem here. The netlink socket opened for
> listening to
> taskstat data sent on task exit can be done in multicast mode....the
> example code in the
> Documentation does that.
>
>>
>> Another thing i overlooked when i did the review was that taskstats
>> interface is designed to provide _BOTH_ per task _AND_ per thread
>> accounting data EVERY TIME a task exists. A thread is an aggregate
>> of (per-pid) tasks. Since this type of aggregation is not used in
>> CSA, half of data traffic would be useless. Can we add a way to
>> configure to not send per-thread data to the socket?
>
> I don't see why not. We could extend the command set to set tgid
> sending on/off.

This would be great!

But, we can have a number of applications listening on the socket. We
surely do not want applications to send conflicting commands to the kernel.
Maybe we can make it a /etc/sysconfig option.

Regards,
 - jay

>
> Regards,
> Shailabh
>
>> Regards,
>> jay
>
>

Re: taskstats interface for accounting

[Shailabh Nagar]

Jay Lan wrote:

>Shailabh Nagar wrote:
>  
>
>>Jay Lan wrote:
>>
>>    
>>
>>>Hi Balbir and Shailabh,
>>>
>>>I finally have time to think about implementation details of CSA over
>>>taskstats interface. I took another look at the taskstats interface
>>>proposal and was a little bit nervous
>>>
<snip>

>>>Another thing i overlooked when i did the review was that taskstats
>>>interface is designed to provide _BOTH_ per task _AND_ per thread
>>>accounting data EVERY TIME a task exists. A thread is an aggregate
>>>of (per-pid) tasks. Since this type of aggregation is not used in
>>>CSA, half of data traffic would be useless. Can we add a way to
>>>configure to not send per-thread data to the socket?
>>>      
>>>
>>I don't see why not. We could extend the command set to set tgid
>>sending on/off.
>>    
>>
>
>This would be great!
>
>But, we can have a number of applications listening on the socket. We
>surely do not want applications to send conflicting commands to the kernel.
>Maybe we can make it a /etc/sysconfig option.
>  
>
Yup...allowing non-privileged apps to set the param is not desirable.
I think by setting the GENL_ADMIN_PERM flag on the configure operation 
being added, we should
be able to limit this command to privileged users.
If thats not the case, we can go with some sysfs option.

Doing the config through the command interface, using a new CONFIG or 
SET type command (just like the GET
command would keep it kind of unified and also be along the lines
of how Jamal had described the interface be used.

--Shailabh