Re: [patch 6/6] statistics infrastructure - exploitation: zfcp

[Marcelo Tosatti <marcelo.tosatti@cyclades.com>]

Guys,

Might be a dumb question, but have you considered doing all of this 
statistical gathering with SystemTap script-fu? 

On Thu, Dec 15, 2005 at 06:47:47PM +0100, Martin Peschke wrote:
> James.Smart@Emulex.Com wrote:
> 
> >This patch is right along the lines of what we did with this earlier patch
> >http://marc.theaimsgroup.com/?l=linux-scsi&m=110700743207792&w=2
> >which adds, at the sdev level, counters for # of requests performed,
> ># of requests completed, # of requests completed in error.
> >
> >of all the stats you propose:
> > 
> >
> >>>+	struct statistic		*stat_sizes_scsi_write;
> >>>+	struct statistic		*stat_sizes_scsi_read;
> >>>+	struct statistic		*stat_sizes_scsi_nodata;
> >>>+	struct statistic		*stat_sizes_scsi_nofit;
> >>>+	struct statistic		*stat_sizes_scsi_nomem;
> >>>+	struct statistic		*stat_sizes_timedout_write;
> >>>+	struct statistic		*stat_sizes_timedout_read;
> >>>+	struct statistic		*stat_sizes_timedout_nodata;
> >>>+	struct statistic		*stat_latencies_scsi_write;
> >>>+	struct statistic		*stat_latencies_scsi_read;
> >>>+	struct statistic		*stat_latencies_scsi_nodata;
> >>>+	struct statistic		*stat_pending_scsi_write;
> >>>+	struct statistic		*stat_pending_scsi_read;
> >>>+	struct statistic		*stat_erp;
> >>>+	struct statistic		*stat_eh_reset;
> >>>     
> >>>
> >
> >All of these are best served to be managed by the midlayer at the
> >sdev.
> >
> Yes.
> 
> >Please don't make the LLDDs do the incrementing, etc
> >
> Though this has been my initial (cautious) attempt, I won't object to 
> moving things up one layer or two.
> 
> >- or something
> >that is owned by one layer, modified by another, and bounces back and 
> >forth.
> > 
> >
> I try to avoid that.
> 
> >The only question is the latency fields, as your statement on latency is a
> >good one. IMHO, the latency that matters is the latency the midlayer sees
> >when it calls queuecommand.  The LLDD interface is designed to not queue in
> >the LLDD's, and you really can't peek lower than the LLDD to know when the
> >i/o actually hits the wire. So - just measure at the queuecommand/scsidone 
> >level
> >(or up in the block request queue level).
> > 
> >
> I agree. For now, I tend to stick to queuecommand/scsi_done.
> 
> >And, if we are expanding stats from the earlier 3 things, we ought to 
> >follow
> >the framework the fc transport used for stats (stolen from the network 
> >world)
> >and create a subdirectory under the /sys/block/<>/ object that reports
> >the stats.
> > 
> >
> Well, I am not yet convinced that sysfs is necessarily the answer for 
> this, because of the one value per file policy,
> which is fine for some purposes, but not for potentially larger amounts 
> of data, or data where the number of values comprising the statistic is 
> variable.
> 
> In general, I see a statistic as something that can be more than a 
> single counter. It can be a histogram, or a counter over time in the 
> form of a history buffer.
> Certainly we could split up every histogram, fill level indicator, 
> megabytes-per-seconds-like statistic and so on. But I think that would 
> result in nasty directory structures which would impose effort to 
> collect the data available for one entitity like.
> 
> My output in debugfs currently looks like this:
> 
> latencies_scsi_write <=0 0
> latencies_scsi_write <=1 0
> latencies_scsi_write <=2 0
> latencies_scsi_write <=4 174
> latencies_scsi_write <=8 872
> latencies_scsi_write <=16 2555
> latencies_scsi_write <=32 2483
> ...
> latencies_scsi_write <=1024 1872
> latencies_scsi_write >1024 1637
> 
> latencies_scsi_read <=0 0
> latencies_scsi_read <=1 0
> latencies_scsi_read <=2 0
> latencies_scsi_read <=4 57265
> latencies_scsi_read <=8 13610
> latencies_scsi_read <=16 1082
> latencies_scsi_read <=32 319
> latencies_scsi_read <=64 63
> ...
> latencies_scsi_read >1024 0
> 
> ...
> util_qdio_outb [3097394.211992] 865 1 1.052 5
> util_qdio_outb [3097395.211992] 737 1 4.558 125
> util_qdio_outb [3097396.211992] 396 1 11.765 77
> util_qdio_outb [3097397.211992] 270 1 12.863 128
> util_qdio_outb [3097398.211992] 765 1 7.271 26
> util_qdio_outb [3097399.211992] 577 1 4.036 27
> ...
> 
> I like it, because it can be read easily; it can be grep-ed; a simple 
> (generic) script could generate whatever output format is required from 
> whatever statistic is given.
> 
> The meaning of a value is not so much determined by its position in the 
> file but by preceding tags. There are a few generic rules on how a 
> histogram or a fill level indicator looks like. That's basically all a 
> user (or a script) needs to know in order to be able to read a 
> networking related histogram, or a disk I/O related histogram, or any 
> other histogram.
> 
> Besides, I'd like to allow users to customize statistics to some degree. 
> If a user needs to tune, say, a histogram to provide a better 
> resolution, then the number of buckets might be changed form 5 to 20, 
> resulting in longer output. Then the position of a counter is not 
> sufficient to determine which range of values it stands for. Counter #4 
> out of 5 probably has a different meaning than counter #4 out of 20. 
> That is why I propose to label buckets or counters appropriately (see 
> above). I doubt this can be reflected in sysfs without too much pain.
> 
> An then there is relayfs, mentioned by Andy Kleen. I guess, what it is 
> meant to be good at is quite opposite to what sysfs does for us. Still 
> it looks appropriate for some statistics, not necessarily for all forms 
> of output - similar issue as with sysfs.
> So far, neither of them seems to be a perfect fit, and, IMO, debugfs 
> looks like a feasible alternative. But the question of which fs to chose 
> probably depends on how we define a statistic and on which ways of 
> in-kernel data processing and corresponding output formats we agree on.
> 
> Martin
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/