per-PID network stats files in /proc

per-PID network stats files in /proc

[Matthew Hall]

Hello,

I have an application that I'd like to make self-tuning, based upon the values 
of some of the network stats counters. Thus I went reading through a copy of 
linux-3.11.1 to look for some more information, and began exploring procfs as 
well.

I discovered some system-wide counters in /proc/net/snmp which are pretty 
interesting so I was trying to use the per-PID counters in 
/proc/net/PID/net/snmp as well.

Unfortunately I found that all of these files seem to be identical on my own 
system running Linux 3.2.0:

$ md5sum /proc/net/snmp /proc/1/net/snmp /proc/2/net/snmp
8b92b426f860d0667a780cd8baabb7bf  /proc/net/snmp
8b92b426f860d0667a780cd8baabb7bf  /proc/1/net/snmp
8b92b426f860d0667a780cd8baabb7bf  /proc/2/net/snmp

I found the snmp_seq_show function in net/ipv4/proc.c which prints these 
statistics to see how it really worked, but then I was having a hard time 
finding out what's really calling this code since it hooks up to the /proc 
framework.

Is there some option one can change which enables gathering the network 
statistics per-PID, or per-socket? This would be a tremendous help for my 
application.

Thanks,
Matthew.

Re: per-PID network stats files in /proc

[Stephen Hemminger]

On Tue, 24 Sep 2013 13:15:37 -0700
Matthew Hall <mh@mhcomputing.net> wrote:

> Hello,
> 
> I have an application that I'd like to make self-tuning, based upon the values 
> of some of the network stats counters. Thus I went reading through a copy of 
> linux-3.11.1 to look for some more information, and began exploring procfs as 
> well.
> 
> I discovered some system-wide counters in /proc/net/snmp which are pretty 
> interesting so I was trying to use the per-PID counters in 
> /proc/net/PID/net/snmp as well.
> 
> Unfortunately I found that all of these files seem to be identical on my own 
> system running Linux 3.2.0:
> 
> $ md5sum /proc/net/snmp /proc/1/net/snmp /proc/2/net/snmp
> 8b92b426f860d0667a780cd8baabb7bf  /proc/net/snmp
> 8b92b426f860d0667a780cd8baabb7bf  /proc/1/net/snmp
> 8b92b426f860d0667a780cd8baabb7bf  /proc/2/net/snmp
> 
> I found the snmp_seq_show function in net/ipv4/proc.c which prints these 
> statistics to see how it really worked, but then I was having a hard time 
> finding out what's really calling this code since it hooks up to the /proc 
> framework.
> 
> Is there some option one can change which enables gathering the network 
> statistics per-PID, or per-socket? This would be a tremendous help for my 
> application.
> 
> Thanks,
> Matthew.

No. because most of these would be associated with global state.
Even sockets can be shared between PID's.

Re: per-PID network stats files in /proc

[Matthew Hall]

On Tue, Sep 24, 2013 at 01:41:57PM -0700, Stephen Hemminger wrote:
> No. because most of these would be associated with global state.
> Even sockets can be shared between PID's.

OK. So if this is true, then I feel compelled to ask, why does 
/proc/PID/net/snmp exist in the first place, if it would never really work?

Thanks,
Matthew.

Re: per-PID network stats files in /proc

[Eric Dumazet]

On Tue, 2013-09-24 at 13:44 -0700, Matthew Hall wrote:
> On Tue, Sep 24, 2013 at 01:41:57PM -0700, Stephen Hemminger wrote:
> > No. because most of these would be associated with global state.
> > Even sockets can be shared between PID's.
> 
> OK. So if this is true, then I feel compelled to ask, why does 
> /proc/PID/net/snmp exist in the first place, if it would never really work?

That because of network containers

You can perfectly use a network container to get your own copy of
counters

ip netns help

Re: per-PID network stats files in /proc

[Stephen Hemminger]

On Tue, 24 Sep 2013 13:44:42 -0700
Matthew Hall <mh@mhcomputing.net> wrote:

> On Tue, Sep 24, 2013 at 01:41:57PM -0700, Stephen Hemminger wrote:
> > No. because most of these would be associated with global state.
> > Even sockets can be shared between PID's.
> 
> OK. So if this is true, then I feel compelled to ask, why does 
> /proc/PID/net/snmp exist in the first place, if it would never really work?
> 
> Thanks,
> Matthew.

/proc/PID/net is symlink to the processes network namespace.

You could do what you want by putting each process in own network
namespace, but that might be more work than you want to bother with.

Re: per-PID network stats files in /proc

[Matthew Hall]

On Tue, Sep 24, 2013 at 02:04:35PM -0700, Stephen Hemminger wrote:
> You could do what you want by putting each process in own network
> namespace, but that might be more work than you want to bother with.

Putting my one process or group of processes into one namespace, separate from 
the one the rest of the system uses, could be perfectly OK actually.

So I'm going to look into this and see if it's possible. At least it would 
give me a better route than where I am right now.

Thanks!
Matthew.

Re: per-PID network stats files in /proc

[Matthew Hall]

On Tue, Sep 24, 2013 at 02:22:39PM -0700, Matthew Hall wrote:
> On Tue, Sep 24, 2013 at 02:04:35PM -0700, Stephen Hemminger wrote:
> > You could do what you want by putting each process in own network
> > namespace, but that might be more work than you want to bother with.
> 
> Putting my one process or group of processes into one namespace, separate from 
> the one the rest of the system uses, could be perfectly OK actually.
> 
> So I'm going to look into this and see if it's possible. At least it would 
> give me a better route than where I am right now.
> 
> Thanks!
> Matthew.

Hi Stephen,

I took a look at this possibility and it seems very powerful but not so easy 
to use. Is there a way to make a netns that's like a "child node" of another 
netns, and inherits all the settings of the default one, unless I configured 
separate settings for this "child netns"? Otherwise the netns has to duplicate 
everything already configured on the host system even if it's unchanged from 
the standard settings.

Thanks,
Matthew.