Re: [Lse-tech] Re: [PATCH] new CSA patchset for 2.6.8

[Jay Lan <jlan@engr.sgi.com>]

The block IO data was not used in billing for CSA customers.

Nobody has ever charged for this data. It is more for accountability
and resource consumption tracking. Accuracy changes as the program
runs for hours or days in our market. Repeatable billing is critical
and block IO is one that is not repeatable.

The write blocks was useful and with the type of programs running for
hours and days (which are the ones of interest in block IO data). The
delayed write after the process terminates is ignored because that is
insignificant for a process doing io for hours or days. We use the
bytes transferred vs blocks transferred to see programs dominating
and polluting the cache. Processes big in one and not the other are of
interest to profile more closely.

However, if nobody else wants to have this feature, we can pull it out
until we can find a way of doing this that makes people happy.

I will submit a new set of patch based on 2.6.8.1 later.

Thanks!
  - jay


Jay Lan wrote:
> Adding csa@oss.sgi.com, the CSA user group mailing list, to Cc.
> 
> Tim Schmielau wrote:
> 
>> On Mon, 30 Aug 2004, Guillaume Thouvenin wrote:
>>
>>
>>>  Thus, to be clear, the enhanced accounting can be divided into
>>> three parts:
>>>
>>>    1) A common data collection method in the kernel.
>>>       We could start from BSD-accounting and add CSA information. Could
>>>       it be something like BSD version4?
>>
>>
>>
>> I've had a quick look at the CSA data collection patches. To get the 
>> discussion started, here are my comments:
>>
>>
>>> --- linux.orig/drivers/block/ll_rw_blk.c        2004-08-13 
>>> 22:36:16.000000000 -0700
>>> +++ linux/drivers/block/ll_rw_blk.c     2004-08-18 12:07:10.000000000 
>>> -0700
>>> @@ -1948,10 +1950,12 @@
>>>
>>>        if (rw == READ) {
>>>                disk_stat_add(rq->rq_disk, read_sectors, nr_sectors);
>>> +               current->rblk += nr_sectors;
>>>                if (!new_io)
>>>                        disk_stat_inc(rq->rq_disk, read_merges);
>>>        } else if (rw == WRITE) {
>>>                disk_stat_add(rq->rq_disk, write_sectors, nr_sectors);
>>> +               current->wblk += nr_sectors;
>>>                if (!new_io)
>>>                        disk_stat_inc(rq->rq_disk, write_merges);
>>>        }
>>
>>
>>
>> Andi Kleen's comment on the ELSA patch also applies here - most writes
>> will get accounted to pdflushd. See
>>
>> http://www.lib.uaa.alaska.edu/linux-kernel/archive/2004-Week-31/0047.html
>>
>> for his comment.
> 
> 
> I need more time on this. :)
> 
>>
>>
>>> --- /dev/null   1970-01-01 00:00:00.000000000 +0000
>>> +++ linux/include/linux/csa_internal.h  2004-08-19 15:19:05.000000000 
>>> -0700
>>
>>
>> [...]
>>
>>> +#else  /* CONFIG_CSA || CONFIG_CSA_MODULE */
>>> +
>>> +#define csa_update_integrals()         do { } while (0);
>>> +#define csa_clear_integrals(task)      do { } while (0);
>>> +#endif /* CONFIG_CSA || CONFIG_CSA_MODULE */
>>
>>
>>
>> I suppose the semicolons are unintentional.
> 
> 
> Good catch! I fixed this in our internal tree.
> 
>>
>>
>>> --- linux.orig/include/linux/sched.h    2004-08-19 15:17:52.000000000 
>>> -0700
>>> +++ linux/include/linux/sched.h 2004-08-19 15:19:05.000000000 -0700
>>
>>
>> [...]
>>
>>> @@ -525,6 +527,10 @@
>>>
>>> /* i/o counters(bytes read/written, blocks read/written, #syscalls, 
>>> waittime */
>>>         unsigned long rchar, wchar, rblk, wblk, syscr, syscw, bwtime;
>>> +#if defined(CONFIG_CSA) || defined(CONFIG_CSA_MODULE)
>>> +       unsigned long csa_rss_mem1, csa_vm_mem1;
>>> +       clock_t csa_stimexpd;
>>> +#endif
>>
>>
>>
>> These probably need to be u64, otherwise they might easily overflow 
>> within
>> a view seconds on 32 bit platforms.
> 
> 
> Will fix it.
> 
>>
>>
>>> --- /dev/null   1970-01-01 00:00:00.000000000 +0000
>>> +++ linux/include/linux/acct_eop.h      2004-08-19 18:48:44.000000000 
>>> -0700
>>
>>
>>
>> This should probably be unified with BSD accounting to a general 
>> accounting
>> hook.
> 
> 
> Do you suggest to merge acct_eop.h into acct.h? It sounds good to me!
> 
> Thanks!
>  - jay
> 
>>
>>
>> Tim
>>
>>
>> -------------------------------------------------------
>> This SF.Net email is sponsored by BEA Weblogic Workshop
>> FREE Java Enterprise J2EE developer tools!
>> Get your free copy of BEA WebLogic Workshop 8.1 today.
>> http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
>> _______________________________________________
>> Lse-tech mailing list
>> Lse-tech@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/lse-tech
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by BEA Weblogic Workshop
> FREE Java Enterprise J2EE developer tools!
> Get your free copy of BEA WebLogic Workshop 8.1 today.
> http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
> _______________________________________________
> Lse-tech mailing list
> Lse-tech@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/lse-tech