Re: [Patch 0/5] I/O statistics through request queues

[Martin Peschke <mp3@de.ibm.com>]

Frank Ch. Eigler wrote:
> Hi -
> 
> On Thu, Oct 26, 2006 at 01:07:53PM +0200, Martin Peschke wrote:
>> [...]
>> I suppose the marker approach will be adopted if jumping from a
>> marker to code hooked up there can be made fast and secure enough
>> for prominent architectures.
> 
> Agree, and I think we're not far.  By "secure" you mean "robust"
> right?

yes

>> [...]
>> Dynamic instrumentation based on markers allows to grow code,
>> but it doesn't allow to grow data structure, AFAICS.
>>
>> Statistics might require temporary results to be stored per
>> entity.
> 
> The data can be kept in data structures private to the instrumentation
> module.  Instead of growing the base structure, you have a lookup
> table indexed by a key of the base structure.  In the lookup table,
> you store whatever you would need: timestamps, whatnot.

lookup_table[key] = value	, or
lookup_table[key]++

How does this scale?

It must be someting else than an array, because key boundaries
aren't known when the lookup table is created, right?
And actual keys might be few and far between.
So you have got some sort of list or tree and do some searching,
don't you?
What if the heap of intermediate results grows into thousands or more?

>> The workaround would be to pass any intermediate result in the form
>> of a trace event up to user space and try to sort it out later -
>> which takes us back to the blktrace approach.
> 
> In systemtap, it is routine to store such intermediate data in kernel
> space, and process it into aggregate statistics on demand, still in
> kernel space.  User space need only see finished results.  This part
> is not complicated.

Yes. I tried out earlier this year.