Re: [Patch 0/5] I/O statistics through request queues

[fche@redhat.com (Frank Ch. Eigler)]

Martin Peschke <mp3@de.ibm.com> writes:

> [...]  The tricky question is: is event processing, that is,
> statistics data aggregation, better done later (in user space), or
> immediately (in the kernel). Both approaches exist: blktrace/btt vs.
> gendisk statistics used by iostat, for example. [...]

I would put it one step farther: the tricky question is whether it's
worth separating marking the state change events ("request X
enqueued") from the action to be taken ("track statistics", "collect
trace records").

The reason I brought up the lttng/marker thread here was because that
suggests a way of addressing several of the problems at the same time.
This could work thusly: (This will sound familiar to OLS attendees.)

- The blktrace code would adopt a generic marker mechanism such as
  that (still) evolving within the lttng/systemtap effort.  These
  markers would replace calls to inline functions such as
      blk_add_trace_bio(q,bio,BLK_TA_QUEUE);
  with something like
      MARK(blk_bio_queue,q,bio);

- The blktrace code that formats and manages trace data would become a
  consumer of the marker API.  It would be hooked up at runtime to
  these markers.  When the events fire, the tracing backend receiving
  the callbacks could do the same thing it does now.  (With the
  markers dormant, the cost should not be much higher than the current
  (likely (!q->blk_trace)) conditional.)

- The mp3 statistics code would be an alternate backend to these same
  markers.  It could be activated or deactivated on the fly (to let
  another subsystem use the markers).  The code would maintain statistics
  in its own memory and could present the data on /proc or whatnot, the
  same way as today.

- Additional backends would be immediately possible: lttng style
  tracing or even fully programmable systemtap probing / analysis
  could all be dynamically activated without further kernel patches or
  rebooting.

>From a user's point of view, it could be the best of all worlds: easy
to get a complete trace for detailed analysis, easy to retain plain
statistics for simple monitoring, easy to do something more elaborate
if necessary.

- FChE