[PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[david m. richter]

	Hello, Neil,

	This set of patches adds -D/--diff-stat to nfsstat.  When this 
flag is used, instead of printing out the current total statistics and 
exiting, nfsstat takes a snapshot of the current statistics and pauses 
until the user types ^C, at which point a second snapshot is taken and 
only the difference between the two is printed; i.e., only the statistics 
gathered during the pause.

- patches 1 and 2 just pull out existing shared code into separate 
functions.

- patches 3 and 4 standardize existing variable names to make it easier to 
declare and use --diff-stat's set of temporary variables.

- patch 5 changes the existing declarations of the struct statinfos by 
using macros (again, to make adding --diff-stat's vars easier).

- patch 6 just declares the extra --diff-stat variables.

- patch 7 adds the rest of --diff-stat.


	The patches have been individually compile- and run-tested on 
2.6.19-rc5 and 2.6.22-rc5, and are based off of your current nfs-utils git 
tree.  The patches contain code written by people at CITI:

	Andrew Richardson <chardson@umich.edu>
	David M. Richter <richterd@citi.umich.edu>
	Alex Soule <soule@umich.edu>


	thanks kindly,

	d
	.
_____
 david m. richter
 CITI -- Center for Information Technology Integration
 http://www.citi.umich.edu

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Neil Brown]

On Tuesday July 31, richterd@citi.umich.edu wrote:
> 	Hello, Neil,
> 
> 	This set of patches adds -D/--diff-stat to nfsstat.  When this 
> flag is used, instead of printing out the current total statistics and 
> exiting, nfsstat takes a snapshot of the current statistics and pauses 
> until the user types ^C, at which point a second snapshot is taken and 
> only the difference between the two is printed; i.e., only the statistics 
> gathered during the pause.

Sounds useful, though I think it would be even more useful if you
could specify a number of seconds rather than needing to interrupt.
That way you could use it in a script.
Maybe even get it to repeat every n seconds?

Thanks. I'll look through the patches.

NeilBrown

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[david m. richter]

> Sounds useful, though I think it would be even more useful if you
> could specify a number of seconds rather than needing to interrupt.
> That way you could use it in a script.
> Maybe even get it to repeat every n seconds?
> 
> Thanks. I'll look through the patches.
> 
> NeilBrown

	roger -- i'll look at adding an optional timeout w/ optional 
repeat.  good idea. 

	thanks,

	d
	.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[J. Bruce Fields]

On Tue, Jul 31, 2007 at 07:29:28PM -0400, david m. richter wrote:
> > Sounds useful, though I think it would be even more useful if you
> > could specify a number of seconds rather than needing to interrupt.
> > That way you could use it in a script.
> > Maybe even get it to repeat every n seconds?
> > 
> > Thanks. I'll look through the patches.
> > 
> > NeilBrown
> 
> 	roger -- i'll look at adding an optional timeout w/ optional 
> repeat.  good idea. 

Sure, I agree that it'd be more convenient for scripting.  Alternatively
(or also) we could allow the user to pass in a file with previous
nfsstat output, so you could do something like

	nfsstat >tmp
	sleep 60
	nfsstat --since=tmp

But personally the ctrl-C thing is what I expect to use 99% of the
time, so I'm happy to see that done first.

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Neil Brown]

On Tuesday July 31, richterd@citi.umich.edu wrote:
> > Sounds useful, though I think it would be even more useful if you
> > could specify a number of seconds rather than needing to interrupt.
> > That way you could use it in a script.
> > Maybe even get it to repeat every n seconds?
> > 
> > Thanks. I'll look through the patches.
> > 
> > NeilBrown
> 
> 	roger -- i'll look at adding an optional timeout w/ optional 
> repeat.  good idea. 
> 

Cool.  I have added everything to the git tree.
(Nicely broken up patch set by the way - very easy to review).
I made a couple of little cosmetic changes
 - removed all spaces before tabs or newlines
 - removed tabs before newlines
 - made these:
+#define DECLARE_SRV(n, s...)	static statinfo n##s[] = { \
+					SRV(net,s), \
+					SRV(rpc,s), \
+					SRV(rc,s), \
+					SRV(fh,s), \
+					SRV(proc2,s), \
+					SRV(proc3,s),\
+					SRV(proc4,s), \
+					SRV(proc4ops,s),\
+					{ NULL, 0, NULL }\

   one per line.

As well as the other bits I asked for, can I get a patch to the man
page for nfsstat too please?

Thanks,
NeilBrown

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Peter Staubach]

david m. richter wrote:
> 	Hello, Neil,
>
> 	This set of patches adds -D/--diff-stat to nfsstat.  When this 
> flag is used, instead of printing out the current total statistics and 
> exiting, nfsstat takes a snapshot of the current statistics and pauses 
> until the user types ^C, at which point a second snapshot is taken and 
> only the difference between the two is printed; i.e., only the statistics 
> gathered during the pause.

Hi.

Perhaps I am missing something, but is something that we really need
to complicate the nfsstat support with?  Is this not easy enough to
do with already existing tools or some easily written support in sh
or perl or awk or name your favorite administrative scripting language?

    Thanx...

       ps

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[david m. richter]

> david m. richter wrote:
> > 	Hello, Neil,
> > 
> > 	This set of patches adds -D/--diff-stat to nfsstat.  When this flag is
> > used, instead of printing out the current total statistics and exiting,
> > nfsstat takes a snapshot of the current statistics and pauses until the user
> > types ^C, at which point a second snapshot is taken and only the difference
> > between the two is printed; i.e., only the statistics gathered during the
> > pause.
> 
> Hi.
> 
> Perhaps I am missing something, but is something that we really need
> to complicate the nfsstat support with?  Is this not easy enough to
> do with already existing tools or some easily written support in sh
> or perl or awk or name your favorite administrative scripting language?
> 
>    Thanx...
> 
>       ps

	
	hello, peter,

	yes.  :)  right now, there's no easy way to run a test and get 
stats for just that test, since the counters persist -- so you have to do 
things like run a test, unmount, remount, etc.  this is a quick way to get 
stats collected over an arbitrary period.  and it's not very complicated, 
really.

	i don't think a shell/perl/python/etc script would be anywhere as 
useful a substitute.  and with neil's suggestion about adding an optional 
timeout to make it easier to use -in- scripts, i think we're on the right 
track with this approach.

	.. thoughts/ideas?

	thanks,

	d
	.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Trond Myklebust]

On Wed, 2007-08-01 at 11:59 -0400, david m. richter wrote:
> > david m. richter wrote:
> > > 	Hello, Neil,
> > > 
> > > 	This set of patches adds -D/--diff-stat to nfsstat.  When this flag is
> > > used, instead of printing out the current total statistics and exiting,
> > > nfsstat takes a snapshot of the current statistics and pauses until the user
> > > types ^C, at which point a second snapshot is taken and only the difference
> > > between the two is printed; i.e., only the statistics gathered during the
> > > pause.
> > 
> > Hi.
> > 
> > Perhaps I am missing something, but is something that we really need
> > to complicate the nfsstat support with?  Is this not easy enough to
> > do with already existing tools or some easily written support in sh
> > or perl or awk or name your favorite administrative scripting language?
> > 
> >    Thanx...
> > 
> >       ps
> 
> 	
> 	hello, peter,
> 
> 	yes.  :)  right now, there's no easy way to run a test and get 
> stats for just that test, since the counters persist -- so you have to do 
> things like run a test, unmount, remount, etc.  this is a quick way to get 
> stats collected over an arbitrary period.  and it's not very complicated, 
> really.
> 
> 	i don't think a shell/perl/python/etc script would be anywhere as 
> useful a substitute.  and with neil's suggestion about adding an optional 
> timeout to make it easier to use -in- scripts, i think we're on the right 
> track with this approach.
> 
> 	.. thoughts/ideas?

How about instead just adding support for the 'nfsstat -z' option that
Solaris offers? That would make it easy to set up the type of script
that Peter is advocating.

All you need to do is to save a snapshot of the current /proc counters
in a file, then use that as the baseline for the output of future
nfsstat commands.
It might be an idea to also save /proc/sys/kernel/random/boot_id
somewhere in this 'baseline file' in order to be able to avoid files
that were created in a past boot instance.

Cheers
  Trond


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[J. Bruce Fields]

On Wed, Aug 01, 2007 at 12:49:14PM -0400, Trond Myklebust wrote:
> How about instead just adding support for the 'nfsstat -z' option that
> Solaris offers? That would make it easy to set up the type of script
> that Peter is advocating.
> 
> All you need to do is to save a snapshot of the current /proc counters
> in a file, then use that as the baseline for the output of future
> nfsstat commands.

I kinda prefer an explicit path, so

	nfsstat --since=baseline

as opposed to an interface that hides the fact that we're storing the
baseline in a file, and possibly confuses Solaris users if e.g.

	nfsstat -z
	su
	nfsstat

doesn't do what's expected.  I dunno.

My one (very minor) complaint about the current implementation is the
name; -D and --diff-stat don't do it for me.

My honest preference would be for --sleep, with -z for short.  Partly
because it seems more descriptive.  Partly because it would trip up
Solaris users, but in a way that would make it obvious what's going on.
("I run nfsstat -z and it hangs?  Interrupt it, then----ohhh, I see.")
But mainly because I think it's funnier.

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[david m. richter]

> I kinda prefer an explicit path, so
> 
> 	nfsstat --since=baseline

	this explicit path approach would also make troubleshooting over 
email easier.


> My one (very minor) complaint about the current implementation is the
> name; -D and --diff-stat don't do it for me.

	and they never did it for me, either.  we were having trouble 
coming up with something that described how the flag affected the 
program's behavior ..


> My honest preference would be for --sleep, with -z for short.  Partly
> because it seems more descriptive.  Partly because it would trip up
> Solaris users, but in a way that would make it obvious what's going on.
> ("I run nfsstat -z and it hangs?  Interrupt it, then----ohhh, I see.")
> But mainly because I think it's funnier.

.. and --sleep does a much better job of describing how the flag affects 
the program from the _user's_ point of view.  i like it much better.

	is -Z any less ambiguous than -z?  (--zzz ;)  or does it matter?


	d
	.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[J. Bruce Fields]

On Wed, Aug 01, 2007 at 11:59:45AM -0400, david m. richter wrote:
> 	yes.  :)  right now, there's no easy way to run a test and get 
> stats for just that test, since the counters persist -- so you have to do 
> things like run a test, unmount, remount, etc.  this is a quick way to get 
> stats collected over an arbitrary period.  and it's not very complicated, 
> really.

Yah.  Especially for debugging other peoples' problems over email, it'd
be helpful to have simple one-line commands that use programs that are
installed everywhere by default.  In this case adding on to nfsstat
looks like the simplest way to achieve that.  But a simpler
implementation would be welcome, of course, if you want to have a go at
it.

> 	i don't think a shell/perl/python/etc script would be anywhere as 
> useful a substitute.  and with neil's suggestion about adding an optional 
> timeout to make it easier to use -in- scripts, i think we're on the right 
> track with this approach.
> 
> 	.. thoughts/ideas?

Can I make one more attempt to interest someone in implementing the
--since=baseline-file approach?  It'd be fun, and would make some more
flexible use cases very simple.  You could still get a time interval:

	nfsstat >tmp
	sleep 30
	nfsstat --since=tmp

or you could do other stuff:

	nfsstat >tmp
	run-my-weird-test-case-here
	nfsstat --since=tmp

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[david m. richter]

> Can I make one more attempt to interest someone in implementing the
> --since=baseline-file approach?  It'd be fun, and would make some more
> flexible use cases very simple.  You could still get a time interval:
> 
> 	nfsstat >tmp
> 	sleep 30
> 	nfsstat --since=tmp
> 
> or you could do other stuff:
> 
> 	nfsstat >tmp
> 	run-my-weird-test-case-here
> 	nfsstat --since=tmp

	
	just curious: would you be satisfied if the --since file's ctime 
were used as the beginning of the time interval?

	d
	.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[J. Bruce Fields]

On Wed, Aug 01, 2007 at 01:35:16PM -0400, david m. richter wrote:
> > Can I make one more attempt to interest someone in implementing the
> > --since=baseline-file approach?  It'd be fun, and would make some more
> > flexible use cases very simple.  You could still get a time interval:
> > 
> > 	nfsstat >tmp
> > 	sleep 30
> > 	nfsstat --since=tmp
> > 
> > or you could do other stuff:
> > 
> > 	nfsstat >tmp
> > 	run-my-weird-test-case-here
> > 	nfsstat --since=tmp
> 
> 	
> 	just curious: would you be satisfied if the --since file's ctime 
> were used as the beginning of the time interval?

Fine by me, but what were you going to use that for?  If you're just
producing a diff, then you don't need to know the length of the time
interval.  Did you just want to add a header saying "stats since
hh:mm:ss", or did you want to calculate per-second rates?

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[david m. richter]

On Wed, 1 Aug 2007, J. Bruce Fields wrote:

> On Wed, Aug 01, 2007 at 01:35:16PM -0400, david m. richter wrote:
> > > Can I make one more attempt to interest someone in implementing the
> > > --since=baseline-file approach?  It'd be fun, and would make some more
> > > flexible use cases very simple.  You could still get a time interval:
> > > 
> > > 	nfsstat >tmp
> > > 	sleep 30
> > > 	nfsstat --since=tmp
> > > 
> > > or you could do other stuff:
> > > 
> > > 	nfsstat >tmp
> > > 	run-my-weird-test-case-here
> > > 	nfsstat --since=tmp
> > 
> > 	
> > 	just curious: would you be satisfied if the --since file's ctime 
> > were used as the beginning of the time interval?
> 
> Fine by me, but what were you going to use that for?  If you're just
> producing a diff, then you don't need to know the length of the time
> interval.  Did you just want to add a header saying "stats since
> hh:mm:ss", or did you want to calculate per-second rates?

	oh, maybe i just misread your "You could still get a time 
interval" thing as meaning that you'd want it to display the length of 
time since --since ran.  but now that i think about it, that's kind of a 
silly thing -- maybe your --since file isn't your own, etc.

	--since (or something like it) should probably also have a "diff 
these two statfiles" mode, like you said bruce.


	d
	.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Trond Myklebust]

On Wed, 2007-08-01 at 13:35 -0400, david m. richter wrote:
> > Can I make one more attempt to interest someone in implementing the
> > --since=baseline-file approach?  It'd be fun, and would make some more
> > flexible use cases very simple.  You could still get a time interval:
> > 
> > 	nfsstat >tmp
> > 	sleep 30
> > 	nfsstat --since=tmp
> > 
> > or you could do other stuff:
> > 
> > 	nfsstat >tmp
> > 	run-my-weird-test-case-here
> > 	nfsstat --since=tmp

Exactly. Being able to zero out the stats is way more flexible than the
-D stuff...

> 	
> 	just curious: would you be satisfied if the --since file's ctime 
> were used as the beginning of the time interval?

What time interval? nfsstat doesn't output any time intervals, and I
can't see how these proposals introduce one.

Is it perhaps Bruce's choice of the word '--since' that is confusing?

Trond


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[david m. richter]

On Wed, 1 Aug 2007, Trond Myklebust wrote:

> On Wed, 2007-08-01 at 13:35 -0400, david m. richter wrote:
> > > Can I make one more attempt to interest someone in implementing the
> > > --since=baseline-file approach?  It'd be fun, and would make some more
> > > flexible use cases very simple.  You could still get a time interval:
> > > 
> > > 	nfsstat >tmp
> > > 	sleep 30
> > > 	nfsstat --since=tmp
> > > 
> > > or you could do other stuff:
> > > 
> > > 	nfsstat >tmp
> > > 	run-my-weird-test-case-here
> > > 	nfsstat --since=tmp
> 
> Exactly. Being able to zero out the stats is way more flexible than the
> -D stuff...

	perhaps i don't follow: how is zeroing the stats address the same 
use-case as --diff-stat/--sleep?  one can already zero the stats by 
unmounting, e.g., but that's a pain.  ?  i'm missing something.


> > 	just curious: would you be satisfied if the --since file's ctime 
> > were used as the beginning of the time interval?
> 
> What time interval? nfsstat doesn't output any time intervals, and I
> can't see how these proposals introduce one.
> 
> Is it perhaps Bruce's choice of the word '--since' that is confusing?

	right -- like i said in another letter, i was just being dumb.  :)


	thanks,
	
	d
	.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Trond Myklebust]

On Wed, 2007-08-01 at 13:48 -0400, david m. richter wrote:
> On Wed, 1 Aug 2007, Trond Myklebust wrote:
> 
> > On Wed, 2007-08-01 at 13:35 -0400, david m. richter wrote:
> > > > Can I make one more attempt to interest someone in implementing the
> > > > --since=baseline-file approach?  It'd be fun, and would make some more
> > > > flexible use cases very simple.  You could still get a time interval:
> > > > 
> > > > 	nfsstat >tmp
> > > > 	sleep 30
> > > > 	nfsstat --since=tmp
> > > > 
> > > > or you could do other stuff:
> > > > 
> > > > 	nfsstat >tmp
> > > > 	run-my-weird-test-case-here
> > > > 	nfsstat --since=tmp
> > 
> > Exactly. Being able to zero out the stats is way more flexible than the
> > -D stuff...
> 
> 	perhaps i don't follow: how is zeroing the stats address the same 
> use-case as --diff-stat/--sleep?  one can already zero the stats by 
> unmounting, e.g., but that's a pain.  ?  i'm missing something.

You can emulate the --sleep stuff using a simple script:

while true
do
	nfsstat >tmp
	sleep 30
	nfsstat --since=tmp
done

Actually, you can do better if you also introduce an '--input' command
which causes nfsstat to read from a file instead of reading
from /proc/net/rpc/nfs and /proc/net/rpc/nfsd. That could be used to
eliminate a race in the above using something like the following script
to return client diff output.

cat /proc/sys/rpc/nfs >prev
while true
	sleep 30
	cat /proc/sys/rpc/nfs >new
	nfsstat -c --since=prev --input=new
	mv new prev
done

Unfortunately the client and server raw formats differ, so you probably
need to require the user to specify '-c' and '-s' in order to tell
nfsstat what kind of file you are giving it.

Trond


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[david m. richter]

On Wed, 1 Aug 2007, david m. richter wrote:

> On Wed, 1 Aug 2007, Trond Myklebust wrote:
> 
> > On Wed, 2007-08-01 at 13:35 -0400, david m. richter wrote:
> > > > Can I make one more attempt to interest someone in implementing the
> > > > --since=baseline-file approach?  It'd be fun, and would make some more
> > > > flexible use cases very simple.  You could still get a time interval:
> > > > 
> > > > 	nfsstat >tmp
> > > > 	sleep 30
> > > > 	nfsstat --since=tmp
> > > > 
> > > > or you could do other stuff:
> > > > 
> > > > 	nfsstat >tmp
> > > > 	run-my-weird-test-case-here
> > > > 	nfsstat --since=tmp
> > 
> > Exactly. Being able to zero out the stats is way more flexible than the
> > -D stuff...
> 
> 	perhaps i don't follow: how is zeroing the stats address the same 
> use-case as --diff-stat/--sleep?  one can already zero the stats by 
> unmounting, e.g., but that's a pain.  ?  i'm missing something.

	oops -- my mistake.  i have a shell alias that i use to unmount, 
unload the modules, then reload and remount; that's what i've been doing 
to zero the stats.

	d
	.



> > > 	just curious: would you be satisfied if the --since file's ctime 
> > > were used as the beginning of the time interval?
> > 
> > What time interval? nfsstat doesn't output any time intervals, and I
> > can't see how these proposals introduce one.
> > 
> > Is it perhaps Bruce's choice of the word '--since' that is confusing?
> 
> 	right -- like i said in another letter, i was just being dumb.  :)
> 
> 
> 	thanks,
> 	
> 	d
> 	.
> 

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Muntz, Daniel]

>perhaps i don't follow: how is zeroing the stats address the same
>use-case as --diff-stat/--sleep?  one can already zero the stats
>by unmounting, e.g., but that's a pain.  ?  i'm missing something.

i think you're answering your own question.  being able to zero the
stats with 'nfsstat -z' allows you to avoid unmounting, while giving you
the ability to, um, zero the stats...  instead of being a pain, it's
trivial.

if you don't want to zero your stats, i agree with others here, that a
simple script accomplishes the same thing without adding a lot of extra
baggage to the command.  it's the unix way.

-----Original Message-----
From: david m. richter [mailto:richterd@citi.umich.edu] 
Sent: Wednesday, August 01, 2007 10:49 AM
To: Trond Myklebust
Cc: Peter Staubach; Neil Brown; Alex Soule; J. Bruce Fields; Andrew
Richardson; nfs@lists.sourceforge.net
Subject: Re: [NFS] [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

On Wed, 1 Aug 2007, Trond Myklebust wrote:

> On Wed, 2007-08-01 at 13:35 -0400, david m. richter wrote:
> > > Can I make one more attempt to interest someone in implementing 
> > > the --since=baseline-file approach?  It'd be fun, and would make 
> > > some more flexible use cases very simple.  You could still get a
time interval:
> > > 
> > > 	nfsstat >tmp
> > > 	sleep 30
> > > 	nfsstat --since=tmp
> > > 
> > > or you could do other stuff:
> > > 
> > > 	nfsstat >tmp
> > > 	run-my-weird-test-case-here
> > > 	nfsstat --since=tmp
> 
> Exactly. Being able to zero out the stats is way more flexible than 
> the -D stuff...

	perhaps i don't follow: how is zeroing the stats address the
same use-case as --diff-stat/--sleep?  one can already zero the stats by
unmounting, e.g., but that's a pain.  ?  i'm missing something.


> > 	just curious: would you be satisfied if the --since file's ctime

> > were used as the beginning of the time interval?
> 
> What time interval? nfsstat doesn't output any time intervals, and I 
> can't see how these proposals introduce one.
> 
> Is it perhaps Bruce's choice of the word '--since' that is confusing?

	right -- like i said in another letter, i was just being dumb.
:)


	thanks,
	
	d
	.

------------------------------------------------------------------------
-
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[david m. richter]

On Wed, 1 Aug 2007, Muntz, Daniel wrote:

> >perhaps i don't follow: how is zeroing the stats address the same
> >use-case as --diff-stat/--sleep?  one can already zero the stats
> >by unmounting, e.g., but that's a pain.  ?  i'm missing something.
> 
> i think you're answering your own question.  being able to zero the
> stats with 'nfsstat -z' allows you to avoid unmounting, while giving you
> the ability to, um, zero the stats...  instead of being a pain, it's
> trivial.

	did i miss an existing kernel interface to do this?  still, it's 
not quite what i wanted, which is something that i can quickly, easily use 
to get stats over a quick interval, repeatedly during lots of little 
tests.  our emphasis was on making that one thing easy.


> if you don't want to zero your stats, i agree with others here, that a
> simple script accomplishes the same thing without adding a lot of extra
> baggage to the command.  it's the unix way.

	yeah, i hear what you're saying, but i don't think it's "the unix 
way" to roll your own every time (scripts and whatnot, or having to say 
stuff in a tempfile and use --since over and over).  it's a simple 
feature, it's not a lot of baggage.  this discussion has gotten some 
feature-creep in it that maybe is obscuring the original intent.

	i think that zeroing all the counters is too much of a "big 
hammer" approach to what was intended to be a quick and simple use-case.  
if there's a kernel interface for zeroing stuff, then maybe that could be 
added, too; someone should have-at.

	thoughts, neil?

	for now, i'm just going to make patches to fix the 
false-positives-in-has_stats() bug, rename --diff-stat to --sleep, and 
update the manpage.  if this feature still has any future at that point, 
i'll add neil's timeout and repeat flags.

	if someone else wants to have a different go at this, please, by 
all means, do.


	d
	.

 
> -----Original Message-----
> From: david m. richter [mailto:richterd@citi.umich.edu] 
> Sent: Wednesday, August 01, 2007 10:49 AM
> To: Trond Myklebust
> Cc: Peter Staubach; Neil Brown; Alex Soule; J. Bruce Fields; Andrew
> Richardson; nfs@lists.sourceforge.net
> Subject: Re: [NFS] [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat
> 
> On Wed, 1 Aug 2007, Trond Myklebust wrote:
> 
> > On Wed, 2007-08-01 at 13:35 -0400, david m. richter wrote:
> > > > Can I make one more attempt to interest someone in implementing 
> > > > the --since=baseline-file approach?  It'd be fun, and would make 
> > > > some more flexible use cases very simple.  You could still get a
> time interval:
> > > > 
> > > > 	nfsstat >tmp
> > > > 	sleep 30
> > > > 	nfsstat --since=tmp
> > > > 
> > > > or you could do other stuff:
> > > > 
> > > > 	nfsstat >tmp
> > > > 	run-my-weird-test-case-here
> > > > 	nfsstat --since=tmp
> > 
> > Exactly. Being able to zero out the stats is way more flexible than 
> > the -D stuff...
> 
> 	perhaps i don't follow: how is zeroing the stats address the
> same use-case as --diff-stat/--sleep?  one can already zero the stats by
> unmounting, e.g., but that's a pain.  ?  i'm missing something.
> 
> 
> > > 	just curious: would you be satisfied if the --since file's ctime
> 
> > > were used as the beginning of the time interval?
> > 
> > What time interval? nfsstat doesn't output any time intervals, and I 
> > can't see how these proposals introduce one.
> > 
> > Is it perhaps Bruce's choice of the word '--since' that is confusing?
> 
> 	right -- like i said in another letter, i was just being dumb.
> :)
> 
> 
> 	thanks,
> 	
> 	d
> 	.
> 
> ------------------------------------------------------------------------
> -
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >>  http://get.splunk.com/
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs
> 

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Trond Myklebust]

On Wed, 2007-08-01 at 14:37 -0400, david m. richter wrote:

> 	i think that zeroing all the counters is too much of a "big 
> hammer" approach to what was intended to be a quick and simple use-case.  
> if there's a kernel interface for zeroing stuff, then maybe that could be 
> added, too; someone should have-at.

No! I see no reason to add such an interface when it can be emulated in
userland.

Trond


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[david m. richter]

On Wed, 1 Aug 2007, Trond Myklebust wrote:

> On Wed, 2007-08-01 at 14:37 -0400, david m. richter wrote:
> 
> > 	i think that zeroing all the counters is too much of a "big 
> > hammer" approach to what was intended to be a quick and simple use-case.  
> > if there's a kernel interface for zeroing stuff, then maybe that could be 
> > added, too; someone should have-at.
> 
> No! I see no reason to add such an interface when it can be emulated in
> userland.

	i completely agree; have no desire to add new kernel interfaces.

	d
	.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[J. Bruce Fields]

On Wed, Aug 01, 2007 at 02:37:09PM -0400, david m. richter wrote:
> 	i think that zeroing all the counters is too much of a "big 
> hammer" approach to what was intended to be a quick and simple use-case.  
> if there's a kernel interface for zeroing stuff, then maybe that could be 
> added, too; someone should have-at.
> 
> 	thoughts, neil?

I think it was Steve that tried that a couple years ago.  I objected,
and feel like a bit of a jerk for it, because the fact is he did a fine
job of it as far as I could tell.

But I'd still object, for the same reasons; global zeroing of the in-kernel
stats is an operation that:

	- isn't friendly to concurrent processes gathering stats:
	  someone might want to run a cron job that summarizes the day's
	  nfs stats, but still be able to log in and get a quick
	  snapshot of current activity.
	- requires special privileges: you may be comfortable allowing
	  anyone to read the stats, but you probably won't allow anyone
	  to zero them.
	- requires a new kernel interface, when you could just do
	  everything in userspace and get something that works for free
	  on older kernels too....

But I think more snapshot-and-diff operations would be a fine idea.
And probably easy and fun to implement.

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Steve Dickson]

J. Bruce Fields wrote:
> On Wed, Aug 01, 2007 at 02:37:09PM -0400, david m. richter wrote:
>> 	i think that zeroing all the counters is too much of a "big 
>> hammer" approach to what was intended to be a quick and simple use-case.  
>> if there's a kernel interface for zeroing stuff, then maybe that could be 
>> added, too; someone should have-at.
>>
>> 	thoughts, neil?
> 
> I think it was Steve that tried that a couple years ago.  I objected,
> and feel like a bit of a jerk for it, because the fact is he did a fine
> job of it as far as I could tell.
Yeah it was me.... and I should have fought harder because since then
I've had numerous requests and there has been a number of time I could
have used that functionality in my own debugging... Heck I think the
patch is still in the rhel3 nfs-utils...

> 
> But I'd still object, for the same reasons; global zeroing of the in-kernel
> stats is an operation that:
> 
> 	- isn't friendly to concurrent processes gathering stats:
> 	  someone might want to run a cron job that summarizes the day's
> 	  nfs stats, but still be able to log in and get a quick
> 	  snapshot of current activity.
So then don't zero them out... and if some one comes a long and
does zero them out, thats a communication problem.. not a
technical one... ;-)


> 	- requires special privileges: you may be comfortable allowing
> 	  anyone to read the stats, but you probably won't allow anyone
> 	  to zero them.
This is a nit, imho... you could use groups to allow access to clearing
the stats...

> 	- requires a new kernel interface, when you could just do
> 	  everything in userspace and get something that works for free
> 	  on older kernels too....
While its true there there was kernel changes needed... but those
changes very small and self-contained and they made the entire
process much much simpler....

> 
> But I think more snapshot-and-diff operations would be a fine idea.
> And probably easy and fun to implement.
Why not point that snapshot at /proc/self/mountstats? Those stats
will never be zero and the wealth of information in there truly
an untaped gold mine....

steved.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[J. Bruce Fields]

On Wed, Aug 01, 2007 at 04:03:13PM -0400, Steve Dickson wrote:
> Yeah it was me.... and I should have fought harder because since then
> I've had numerous requests and there has been a number of time I could
> have used that functionality in my own debugging...

I do feel bad about that.  So poking David and the others to do this bit
of nfsstat hacking was partly my attempt to make up for it (and to
provide something I'd personally like).

Does it look like the nfsstat --sleep/--diff-stats/whatever would have
done the job in the cases you're thinking of?

> > But I'd still object, for the same reasons; global zeroing of the in-kernel
> > stats is an operation that:
> > 
> > 	- isn't friendly to concurrent processes gathering stats:
> > 	  someone might want to run a cron job that summarizes the day's
> > 	  nfs stats, but still be able to log in and get a quick
> > 	  snapshot of current activity.
> So then don't zero them out... and if some one comes a long and
> does zero them out, thats a communication problem.. not a
> technical one... ;-)

What would you tell them to do instead?  Parsing the nfsstat output and
doing the subtraction yourself is a little tedious, so it's not
suprising if they both try to use -z, if that's all we provide.

Which is why I'd really rather not even emulate zeroing in userspace,
and instead have two alternatives:

	- --sleep: quick and convenient, no concurrency problems.

	- --since: or other operations that do the subtraction for you
	  and take explicit paths for saved snapshots.  That allows you
	  to do everything you could do with -z and more, and makes any
	  problems with concurrency or privileges solvable by choice of
	  appropriate paths to store the snapshots in.

Actually the only operation you *really* need is one that parses two
nfsstat outputs and subtracts:

	nfsstat >tmp1
	...
	nfsstat >tmp2
	nfsstat --diff tmp1 tmp2

and then anything else you can easily script.

> > But I think more snapshot-and-diff operations would be a fine idea.
> > And probably easy and fun to implement.
> Why not point that snapshot at /proc/self/mountstats? Those stats
> will never be zero and the wealth of information in there truly
> an untaped gold mine....

Yeah, that'd be great.  We'll still need nfsstat for the server at
least, I guess?  Does it still provide anything useful on the client
side, or does mountstats supercede it completely?

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Steve Dickson]

J. Bruce Fields wrote:
> 
> Does it look like the nfsstat --sleep/--diff-stats/whatever would have
> done the job in the cases you're thinking of?
It would a bit more awkward... but I'm sure I could make it work...

> 
>>> But I'd still object, for the same reasons; global zeroing of the in-kernel
>>> stats is an operation that:
>>>
>>> 	- isn't friendly to concurrent processes gathering stats:
>>> 	  someone might want to run a cron job that summarizes the day's
>>> 	  nfs stats, but still be able to log in and get a quick
>>> 	  snapshot of current activity.
>> So then don't zero them out... and if some one comes a long and
>> does zero them out, thats a communication problem.. not a
>> technical one... ;-)
> 
> What would you tell them to do instead?  Parsing the nfsstat output and
> doing the subtraction yourself is a little tedious, so it's not
> suprising if they both try to use -z, if that's all we provide.
A *little* tedious??? :-) I don't know how I would tell them
to solve that problem esp with the current tools... I would
probably tell to see if they could get the mountstats numbers
to do what they wanted... and then tell them to send me the patch! 8-)

> 
> Which is why I'd really rather not even emulate zeroing in userspace,
> and instead have two alternatives:
> 
> 	- --sleep: quick and convenient, no concurrency problems.
> 
> 	- --since: or other operations that do the subtraction for you
> 	  and take explicit paths for saved snapshots.  That allows you
> 	  to do everything you could do with -z and more, and makes any
> 	  problems with concurrency or privileges solvable by choice of
> 	  appropriate paths to store the snapshots in.
Understood and I can see how these would be useful in
long term system analyzation but they would be awkward for
short term system debugging... which I think is the real difference
in these approaches...

> 
> Actually the only operation you *really* need is one that parses two
> nfsstat outputs and subtracts:
> 
> 	nfsstat >tmp1
> 	...
> 	nfsstat >tmp2
> 	nfsstat --diff tmp1 tmp2
> 
> and then anything else you can easily script.
Personally I would make the output binary (with a -b flag) since
it seem like it would be easier to do binary compares than
string... But of course that would be giving you more ammo
and I would not want to do that... ;-)


> 
>>> But I think more snapshot-and-diff operations would be a fine idea.
>>> And probably easy and fun to implement.
>> Why not point that snapshot at /proc/self/mountstats? Those stats
>> will never be zero and the wealth of information in there truly
>> an untaped gold mine....
> 
> Yeah, that'd be great.  We'll still need nfsstat for the server at
> least, I guess?  
Good point...

> Does it still provide anything useful on the client
> side, or does mountstats supercede it completely?
I think the both are needed since they provide  different
types of information... The nfsstat stats are basically
"hello world" counters that show there is NFS activity....
The mountstats are much more granular and are per mount
which could be used  to for many different things... like
server responsive for an example...


steved.




-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Trond Myklebust]

On Wed, 2007-08-01 at 16:48 -0400, J. Bruce Fields wrote:

> What would you tell them to do instead?  Parsing the nfsstat output and
> doing the subtraction yourself is a little tedious, so it's not
> suprising if they both try to use -z, if that's all we provide.

The entire point of 'nfsstat' is to act as a parser. Otherwise we'd just
be all be reading the raw output from /proc/net/rpc/*... It makes sense
to add other parsing options to it. I'm a lot more sceptical about
attempts to add non-parsing options.

> Which is why I'd really rather not even emulate zeroing in userspace,
> and instead have two alternatives:
> 
> 	- --sleep: quick and convenient, no concurrency problems.

Has nothing to do with parsing.

> 	- --since: or other operations that do the subtraction for you
> 	  and take explicit paths for saved snapshots.  That allows you
> 	  to do everything you could do with -z and more, and makes any
> 	  problems with concurrency or privileges solvable by choice of
> 	  appropriate paths to store the snapshots in.
> 
> Actually the only operation you *really* need is one that parses two
> nfsstat outputs and subtracts:
> 
> 	nfsstat >tmp1
> 	...
> 	nfsstat >tmp2
> 	nfsstat --diff tmp1 tmp2
> 
> and then anything else you can easily script.

Parsing already-parsed output is IMO less useful. Just have it be able
to take input from somewhere other than just /proc/net/rpc/*

> > > But I think more snapshot-and-diff operations would be a fine idea.
> > > And probably easy and fun to implement.
> > Why not point that snapshot at /proc/self/mountstats? Those stats
> > will never be zero and the wealth of information in there truly
> > an untaped gold mine....
> 
> Yeah, that'd be great.  We'll still need nfsstat for the server at
> least, I guess?  Does it still provide anything useful on the client
> side, or does mountstats supercede it completely?




-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[J. Bruce Fields]

On Wed, Aug 01, 2007 at 11:15:23AM -0700, Muntz, Daniel wrote:
> i think you're answering your own question.  being able to zero the
> stats with 'nfsstat -z' allows you to avoid unmounting, while giving you
> the ability to, um, zero the stats...  instead of being a pain, it's
> trivial.
> 
> if you don't want to zero your stats, i agree with others here, that a
> simple script

I'm sure contributions of simple scripts would be welcome.

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[J. Bruce Fields]

On Wed, Aug 01, 2007 at 01:46:24PM -0400, Trond Myklebust wrote:
> Exactly. Being able to zero out the stats is way more flexible than the
> -D stuff...

It's definitely a better primitive for building scripts.  I think
-D/--sleep/whatever is what I'd personally end up using most of the time
for quick troubleshooting, if only because it's less typing.

> Is it perhaps Bruce's choice of the word '--since' that is confusing?

I'm not sure what'd be better: --previous? --baseline?

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Peter Staubach]

david m. richter wrote:
>> david m. richter wrote:
>>     
>>> 	Hello, Neil,
>>>
>>> 	This set of patches adds -D/--diff-stat to nfsstat.  When this flag is
>>> used, instead of printing out the current total statistics and exiting,
>>> nfsstat takes a snapshot of the current statistics and pauses until the user
>>> types ^C, at which point a second snapshot is taken and only the difference
>>> between the two is printed; i.e., only the statistics gathered during the
>>> pause.
>>>       
>> Hi.
>>
>> Perhaps I am missing something, but is something that we really need
>> to complicate the nfsstat support with?  Is this not easy enough to
>> do with already existing tools or some easily written support in sh
>> or perl or awk or name your favorite administrative scripting language?
>>
>>    Thanx...
>>
>>       ps
>>     
>
> 	
> 	hello, peter,
>
> 	yes.  :)  right now, there's no easy way to run a test and get 
> stats for just that test, since the counters persist -- so you have to do 
> things like run a test, unmount, remount, etc.  this is a quick way to get 
> stats collected over an arbitrary period.  and it's not very complicated, 
> really.
>
> 	i don't think a shell/perl/python/etc script would be anywhere as 
> useful a substitute.  and with neil's suggestion about adding an optional 
> timeout to make it easier to use -in- scripts, i think we're on the right 
> track with this approach.
>
> 	.. thoughts/ideas?

It doesn't seem very complicated to me to store the output from two
nfsstat commands and then write a small script or whatever to parse
them and create output with the differences.

This would address the issues with wanting to be able to measure
the statistics for a particular duration or for the duration that
a particular problem runs.  Either would be easy to do.

Now, if someone was proposing a way to do "nfsstat 5" and make the
output look useful, then I would be behind that 100%.

    Thanx...

       ps

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[david m. richter]

> Now, if someone was proposing a way to do "nfsstat 5" and make the
> output look useful, then I would be behind that 100%.
> 
>    Thanx...
> 
>       ps


	do you mean "gather stats for the next 5 seconds"?  i don't 
understand the arg.

	d
	.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Peter Staubach]

david m. richter wrote:
>> Now, if someone was proposing a way to do "nfsstat 5" and make the
>> output look useful, then I would be behind that 100%.
>>
>>    Thanx...
>>
>>       ps
>>     
>
>
> 	do you mean "gather stats for the next 5 seconds"?  i don't 
> understand the arg.

Sorry, I should have been more explicit.  I have wished for
something that would behave like "vmstat 5", ie. print nfsstat
statistics in some sort short hand fashion once every few
seconds, 5 in this example.

The problem has always been that there are too many operations
for summarize them on one line in any sort of interesting way.
This becomes particularly true for NFSv4 which has its multitude
of operations...

Perhaps we should just bite the bullet and produce the large
output, much like "iostat 5" seems to.

    Thanx...

       ps

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[david m. richter]

On Wed, 1 Aug 2007, Peter Staubach wrote:

> david m. richter wrote:
> > > Now, if someone was proposing a way to do "nfsstat 5" and make the
> > > output look useful, then I would be behind that 100%.
> > > 
> > >    Thanx...
> > > 
> > >       ps
> > >     
> > 
> > 
> > 	do you mean "gather stats for the next 5 seconds"?  i don't understand
> > the arg.
> 
> Sorry, I should have been more explicit.  I have wished for
> something that would behave like "vmstat 5", ie. print nfsstat
> statistics in some sort short hand fashion once every few
> seconds, 5 in this example.
> 
> The problem has always been that there are too many operations
> for summarize them on one line in any sort of interesting way.
> This becomes particularly true for NFSv4 which has its multitude
> of operations...
> 
> Perhaps we should just bite the bullet and produce the large
> output, much like "iostat 5" seems to.
> 
>    Thanx...
> 
>       ps


	oh, i see -- yeah, neil'd suggested adding flags for both "display 
stats over a time interval" and "repeat gathering stats instead of 
exiting after one go".  cool.  maybe the args could be set up so that 
"nfsstat 5"  was shorthand for whatever the extra flags would be.


	d
	. 

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[J. Bruce Fields]

On Wed, Aug 01, 2007 at 02:22:19PM -0400, Peter Staubach wrote:
> Sorry, I should have been more explicit.  I have wished for
> something that would behave like "vmstat 5", ie. print nfsstat
> statistics in some sort short hand fashion once every few
> seconds, 5 in this example.
> 
> The problem has always been that there are too many operations
> for summarize them on one line in any sort of interesting way.
> This becomes particularly true for NFSv4 which has its multitude
> of operations...

Yeah, I agree, it'd be great to have somebody look into this.

Also, do we need code (either in nfsstat or elsewhere) to help people
use the new per-mount client statistics?

I'm guessing among Trond and Neil and people at citi there may not be
anyone with the cycles to implement any of that right now, so these are
projects looking for volunteers.

--b.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Chuck Lever]

[-- Attachment #1: Type: text/plain, Size: 2461 bytes --]



Peter Staubach wrote:
> david m. richter wrote:
>>> Now, if someone was proposing a way to do "nfsstat 5" and make the
>>> output look useful, then I would be behind that 100%.
>>>
>>>    Thanx...
>>>
>>>       ps
>>>     
>>
>> 	do you mean "gather stats for the next 5 seconds"?  i don't 
>> understand the arg.
> 
> Sorry, I should have been more explicit.  I have wished for
> something that would behave like "vmstat 5", ie. print nfsstat
> statistics in some sort short hand fashion once every few
> seconds, 5 in this example.
> 
> The problem has always been that there are too many operations
> for summarize them on one line in any sort of interesting way.
> This becomes particularly true for NFSv4 which has its multitude
> of operations...
> 
> Perhaps we should just bite the bullet and produce the large
> output, much like "iostat 5" seems to.

First, thanks to David and the citi guys for the patches.

The kernel stats themselves should never be zeroed except by umount or 
reboot.  Otherwise, tools like "sar" and "iostat" that are looking 
directly at the same set of kernel stats, and producing a "one every 5 
seconds" type of output, would be totally confused if "nfsstat -z" 
actually cleared the kernel counters.

There's a bigger picture here.  nfsstat does not live in a vacuum, it 
lives in the presence of all the other I/O tools like sar, vmstat, and 
iostat.  For that matter, we already have "watch nfsstat" and 
"nfswatch".  Those tools already provide a "once every 5 seconds" type 
of interface.  I might even be inclined to argue that adding "once every 
5 seconds" type functionality is really outside of nfsstat's job 
description.

So if we wanted an "nfsstat --since" or "nfsstat 5 5" kind of thing, 
maybe we should think about the other tools, how they fit in, and how 
they work, and see if we can use one of them for that.  Even better, a 
GUI like gnome-system-monitor would be very nice for watching NFS client 
and server performance in real time.

I'm kind of tired of NFS living in its own little world with regard to 
the other file systems.  The NFS performance metrics I built were 
precisely for the purpose of making NFS a "first class" file system with 
regard to reporting errors and performance, and for the purpose of 
including NFS in the tools sysadmins normally use to watch I/O subsystem 
  performance data on local disks.

Can we come up with a plan that moves NFS closer to other file systems?

[-- Attachment #2: chuck.lever.vcf --]
[-- Type: text/x-vcard, Size: 290 bytes --]

begin:vcard
fn:Chuck Lever
n:Lever;Chuck
org:Oracle Corporation;Corporate Architecture: Linux Projects Group
adr:;;1015 Granger Avenue;Ann Arbor;MI;48104;USA
title:Principal Member of Staff
tel;work:+1 248 614 5091
x-mozilla-html:FALSE
url:http://oss.oracle.com/~cel
version:2.1
end:vcard


[-- Attachment #3: Type: text/plain, Size: 315 bytes --]

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

[-- Attachment #4: Type: text/plain, Size: 140 bytes --]

_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Steve Dickson]

Chuck Lever wrote:

> The kernel stats themselves should never be zeroed except by umount or 
> reboot.  Otherwise, tools like "sar" and "iostat" that are looking 
> directly at the same set of kernel stats, and producing a "one every 5 
> seconds" type of output, would be totally confused if "nfsstat -z" 
> actually cleared the kernel counters.
In general, I agree. Stats that "sar" and "iostat" see should
not be zeroed.. but the nfsstat's stats do not fall in this
category. They are more like counters than true statistics.
I think if they were call nfscounts instead of nfsstats
there would be less of an uproar since reseting counter
is no big deal...

So in the end.. these counter are used to show nfs activity
for the entire box... so being able to zero them would
incredibly handy for developers and customers alike...


> 
> So if we wanted an "nfsstat --since" or "nfsstat 5 5" kind of thing, 
> maybe we should think about the other tools, how they fit in, and how 
> they work, and see if we can use one of them for that.  Even better, a 
> GUI like gnome-system-monitor would be very nice for watching NFS client 
> and server performance in real time.
> 
> I'm kind of tired of NFS living in its own little world with regard to 
> the other file systems.  The NFS performance metrics I built were 
> precisely for the purpose of making NFS a "first class" file system with 
> regard to reporting errors and performance, and for the purpose of 
> including NFS in the tools sysadmins normally use to watch I/O subsystem 
>  performance data on local disks.
> 
> Can we come up with a plan that moves NFS closer to other file systems?
Yeah... lets build a tool that understand the stats in
/proc/self/mountstats and continue build there... because imho,
thats where the future lies with regard to understanding and
analyzing NFS traffic patterns...

steved.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Chuck Lever]

[-- Attachment #1: Type: text/plain, Size: 2276 bytes --]

Steve Dickson wrote:
> Chuck Lever wrote:
>> The kernel stats themselves should never be zeroed except by umount or 
>> reboot.  Otherwise, tools like "sar" and "iostat" that are looking 
>> directly at the same set of kernel stats, and producing a "one every 5 
>> seconds" type of output, would be totally confused if "nfsstat -z" 
>> actually cleared the kernel counters.
> In general, I agree. Stats that "sar" and "iostat" see should
> not be zeroed.. but the nfsstat's stats do not fall in this
> category. They are more like counters than true statistics.
> I think if they were call nfscounts instead of nfsstats
> there would be less of an uproar since reseting counter
> is no big deal...
> 
> So in the end.. these counter are used to show nfs activity
> for the entire box... so being able to zero them would
> incredibly handy for developers and customers alike...
> 
> 
>>
>> So if we wanted an "nfsstat --since" or "nfsstat 5 5" kind of thing, 
>> maybe we should think about the other tools, how they fit in, and how 
>> they work, and see if we can use one of them for that.  Even better, a 
>> GUI like gnome-system-monitor would be very nice for watching NFS 
>> client and server performance in real time.
>>
>> I'm kind of tired of NFS living in its own little world with regard to 
>> the other file systems.  The NFS performance metrics I built were 
>> precisely for the purpose of making NFS a "first class" file system 
>> with regard to reporting errors and performance, and for the purpose 
>> of including NFS in the tools sysadmins normally use to watch I/O 
>> subsystem  performance data on local disks.
>>
>> Can we come up with a plan that moves NFS closer to other file systems?
> Yeah... lets build a tool that understand the stats in
> /proc/self/mountstats and continue build there... because imho,
> thats where the future lies with regard to understanding and
> analyzing NFS traffic patterns...

I think I agree with this overall approach.

Leave nfsstat alone (or maybe add "-z" to zero the traditional 
/proc/net/rpc/nfs), and add support for /proc/self/mountstats in sar, 
iostat, and vmstat.

That way you can have your zeroed counters a la Solaris, and also have 
counters that can't be zeroed for the tools that look for trends.

[-- Attachment #2: chuck.lever.vcf --]
[-- Type: text/x-vcard, Size: 290 bytes --]

begin:vcard
fn:Chuck Lever
n:Lever;Chuck
org:Oracle Corporation;Corporate Architecture: Linux Projects Group
adr:;;1015 Granger Avenue;Ann Arbor;MI;48104;USA
title:Principal Member of Staff
tel;work:+1 248 614 5091
x-mozilla-html:FALSE
url:http://oss.oracle.com/~cel
version:2.1
end:vcard


[-- Attachment #3: Type: text/plain, Size: 315 bytes --]

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

[-- Attachment #4: Type: text/plain, Size: 140 bytes --]

_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [PATCH 0/7] nfsstat: adding -D/--diff-stat to nfsstat

[Steve Dickson]

Peter Staubach wrote:
> david m. richter wrote:
>>> david m. richter wrote:
>>>     
>>>> 	Hello, Neil,
>>>>
>>>> 	This set of patches adds -D/--diff-stat to nfsstat.  When this flag is
>>>> used, instead of printing out the current total statistics and exiting,
>>>> nfsstat takes a snapshot of the current statistics and pauses until the user
>>>> types ^C, at which point a second snapshot is taken and only the difference
>>>> between the two is printed; i.e., only the statistics gathered during the
>>>> pause.
>>>>       
>>> Hi.
>>>
>>> Perhaps I am missing something, but is something that we really need
>>> to complicate the nfsstat support with?  Is this not easy enough to
>>> do with already existing tools or some easily written support in sh
>>> or perl or awk or name your favorite administrative scripting language?
>>>
>>>    Thanx...
>>>
>>>       ps
>>>     
>> 	
>> 	hello, peter,
>>
>> 	yes.  :)  right now, there's no easy way to run a test and get 
>> stats for just that test, since the counters persist -- so you have to do 
>> things like run a test, unmount, remount, etc.  this is a quick way to get 
>> stats collected over an arbitrary period.  and it's not very complicated, 
>> really.
>>
>> 	i don't think a shell/perl/python/etc script would be anywhere as 
>> useful a substitute.  and with neil's suggestion about adding an optional 
>> timeout to make it easier to use -in- scripts, i think we're on the right 
>> track with this approach.
>>
>> 	.. thoughts/ideas?
> 
> It doesn't seem very complicated to me to store the output from two
> nfsstat commands and then write a small script or whatever to parse
> them and create output with the differences.
> 
> This would address the issues with wanting to be able to measure
> the statistics for a particular duration or for the duration that
> a particular problem runs.  Either would be easy to do.
> 
> Now, if someone was proposing a way to do "nfsstat 5" and make the
> output look useful, then I would be behind that 100%.
In a way we already have that... do 'watch nfsstat -c' I think
you get what your looking for...


steved.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs