[PATCH net-next v3 0/2] socket statistics for ss

[PATCH net-next v3 0/2] socket statistics for ss

[Stephen Hemminger]

The eventual goal is to remove all accesses to slab statistics
by iproute2 ss command. This are preliminary steps to add two
statistics which are not working now, because of slab merging
and changes that were done years ago to allow for common inet_hash
tables.

Stephen Hemminger (2):
  inet: add bound ports statistic
  socket: keep track of the number of sockets allocated

v3
  - fix build :-(
  - add allocated sockets statistic

 include/net/inet_hashtables.h    |  3 +++
 include/net/inet_timewait_sock.h |  2 ++
 net/dccp/proto.c                 |  1 +
 net/ipv4/inet_hashtables.c       | 22 +++++++++++++++++++---
 net/ipv4/inet_timewait_sock.c    |  8 +++++---
 net/ipv4/proc.c                  |  5 +++--
 net/ipv4/tcp.c                   |  1 +
 net/socket.c                     | 21 +++++++++++++++++++--
 8 files changed, 53 insertions(+), 10 deletions(-)

-- 
2.17.0

Re: [PATCH net-next v3 0/2] socket statistics for ss

[Stephen Hemminger]

On Mon,  7 May 2018 11:43:31 -0700
Stephen Hemminger <stephen@networkplumber.org> wrote:

> The eventual goal is to remove all accesses to slab statistics
> by iproute2 ss command. This are preliminary steps to add two
> statistics which are not working now, because of slab merging
> and changes that were done years ago to allow for common inet_hash
> tables.
> 
> Stephen Hemminger (2):
>   inet: add bound ports statistic
>   socket: keep track of the number of sockets allocated
> 
> v3
>   - fix build :-(
>   - add allocated sockets statistic
> 
>  include/net/inet_hashtables.h    |  3 +++
>  include/net/inet_timewait_sock.h |  2 ++
>  net/dccp/proto.c                 |  1 +
>  net/ipv4/inet_hashtables.c       | 22 +++++++++++++++++++---
>  net/ipv4/inet_timewait_sock.c    |  8 +++++---
>  net/ipv4/proc.c                  |  5 +++--
>  net/ipv4/tcp.c                   |  1 +
>  net/socket.c                     | 21 +++++++++++++++++++--
>  8 files changed, 53 insertions(+), 10 deletions(-)
> 

I am not sure if these patches are worth applying.
The 'ss -s' command has had missing values since 2.4 kernel.
And the first complaints came in only this year.

Another alternative would be just to remove these fields from ss -s
output and move on.

Re: [PATCH net-next v3 0/2] socket statistics for ss

[Eric Dumazet]

On 05/09/2018 08:22 AM, Stephen Hemminger wrote:

> I am not sure if these patches are worth applying.
> The 'ss -s' command has had missing values since 2.4 kernel.
> And the first complaints came in only this year.
> 
> Another alternative would be just to remove these fields from ss -s
> output and move on.
> 

Anyway your patches are not netns ready, so lets remove these fields from ss.

Or you have to spend _much_ more time on writing and testing the kernel part.

Thanks.

Re: [PATCH net-next v3 0/2] socket statistics for ss

[Stephen Hemminger]

On Wed, 9 May 2018 10:18:23 -0700
Eric Dumazet <eric.dumazet@gmail.com> wrote:

> On 05/09/2018 08:22 AM, Stephen Hemminger wrote:
> 
> > I am not sure if these patches are worth applying.
> > The 'ss -s' command has had missing values since 2.4 kernel.
> > And the first complaints came in only this year.
> > 
> > Another alternative would be just to remove these fields from ss -s
> > output and move on.
> >   
> 
> Anyway your patches are not netns ready, so lets remove these fields from ss.
> 
> Or you have to spend _much_ more time on writing and testing the kernel part.
> 
> Thanks.

The patches only expose the existing TCP socket accounting infrastructure.
Several other pieces that sockstat has are not netns aware.
That is a completely different problem.

Re: [PATCH net-next v3 0/2] socket statistics for ss

[Eric Dumazet]

On 05/09/2018 10:31 AM, Stephen Hemminger wrote:
> On Wed, 9 May 2018 10:18:23 -0700
> Eric Dumazet <eric.dumazet@gmail.com> wrote:
> 
>> On 05/09/2018 08:22 AM, Stephen Hemminger wrote:
>>
>>> I am not sure if these patches are worth applying.
>>> The 'ss -s' command has had missing values since 2.4 kernel.
>>> And the first complaints came in only this year.
>>>
>>> Another alternative would be just to remove these fields from ss -s
>>> output and move on.
>>>   
>>
>> Anyway your patches are not netns ready, so lets remove these fields from ss.
>>
>> Or you have to spend _much_ more time on writing and testing the kernel part.
>>
>> Thanks.
> 
> The patches only expose the existing TCP socket accounting infrastructure.
> Several other pieces that sockstat has are not netns aware.
> That is a completely different problem.


Adding a new field counting 'bounds ports' without being netns ready is a total mistake,
as it is useless by current standards.

The first thing that users will do is add proper netns support, with extra complexity in the kernel.

So, instead of pushing some incomplete feature, trying to fool ourselves with a sentiment of 'small cost'
that will later need another 100 lines of code in the kernel, please give us the complete picture.

I am just saying, you can of course ignore my feedback.

Re: [PATCH net-next v3 0/2] socket statistics for ss

[Stephen Hemminger]

On Wed, 9 May 2018 10:53:58 -0700
Eric Dumazet <eric.dumazet@gmail.com> wrote:

> On 05/09/2018 10:31 AM, Stephen Hemminger wrote:
> > On Wed, 9 May 2018 10:18:23 -0700
> > Eric Dumazet <eric.dumazet@gmail.com> wrote:
> >   
> >> On 05/09/2018 08:22 AM, Stephen Hemminger wrote:
> >>  
> >>> I am not sure if these patches are worth applying.
> >>> The 'ss -s' command has had missing values since 2.4 kernel.
> >>> And the first complaints came in only this year.
> >>>
> >>> Another alternative would be just to remove these fields from ss -s
> >>> output and move on.
> >>>     
> >>
> >> Anyway your patches are not netns ready, so lets remove these fields from ss.
> >>
> >> Or you have to spend _much_ more time on writing and testing the kernel part.
> >>
> >> Thanks.  
> > 
> > The patches only expose the existing TCP socket accounting infrastructure.
> > Several other pieces that sockstat has are not netns aware.
> > That is a completely different problem.  
> 
> 
> Adding a new field counting 'bounds ports' without being netns ready is a total mistake,
> as it is useless by current standards.
> 
> The first thing that users will do is add proper netns support, with extra complexity in the kernel.
> 
> So, instead of pushing some incomplete feature, trying to fool ourselves with a sentiment of 'small cost'
> that will later need another 100 lines of code in the kernel, please give us the complete picture.
> 
> I am just saying, you can of course ignore my feedback.

The current TCP hashinfo should be moved into netns. The current method of scanning and matching
by net namespace is a scalability issue now.

Re: [PATCH net-next v3 0/2] socket statistics for ss

[Eric Dumazet]

On 05/09/2018 11:44 AM, Stephen Hemminger wrote:
> On Wed, 9 May 2018 10:53:58 -0700
> Eric Dumazet <eric.dumazet@gmail.com> wrote:
> 
>> On 05/09/2018 10:31 AM, Stephen Hemminger wrote:
>>> On Wed, 9 May 2018 10:18:23 -0700
>>> Eric Dumazet <eric.dumazet@gmail.com> wrote:
>>>   
>>>> On 05/09/2018 08:22 AM, Stephen Hemminger wrote:
>>>>  
>>>>> I am not sure if these patches are worth applying.
>>>>> The 'ss -s' command has had missing values since 2.4 kernel.
>>>>> And the first complaints came in only this year.
>>>>>
>>>>> Another alternative would be just to remove these fields from ss -s
>>>>> output and move on.
>>>>>     
>>>>
>>>> Anyway your patches are not netns ready, so lets remove these fields from ss.
>>>>
>>>> Or you have to spend _much_ more time on writing and testing the kernel part.
>>>>
>>>> Thanks.  
>>>
>>> The patches only expose the existing TCP socket accounting infrastructure.
>>> Several other pieces that sockstat has are not netns aware.
>>> That is a completely different problem.  
>>
>>
>> Adding a new field counting 'bounds ports' without being netns ready is a total mistake,
>> as it is useless by current standards.
>>
>> The first thing that users will do is add proper netns support, with extra complexity in the kernel.
>>
>> So, instead of pushing some incomplete feature, trying to fool ourselves with a sentiment of 'small cost'
>> that will later need another 100 lines of code in the kernel, please give us the complete picture.
>>
>> I am just saying, you can of course ignore my feedback.
> 
> The current TCP hashinfo should be moved into netns. The current method of scanning and matching
> by net namespace is a scalability issue now.

It is not the plan yet, and we have no scalability issue.

Before switching to netns hash table, this would need rhashtable conversions
but so far this has not been done.

- Time to create/delete netns is critical.
- Adding few Mbytes of overhead per netns is a nogo,

Please do not change subject, this is adding noise to this particular thread.

Re: [PATCH net-next v3 0/2] socket statistics for ss

[Stephen Hemminger]

On Wed, 9 May 2018 11:53:50 -0700
Eric Dumazet <eric.dumazet@gmail.com> wrote:

> On 05/09/2018 11:44 AM, Stephen Hemminger wrote:
> > On Wed, 9 May 2018 10:53:58 -0700
> > Eric Dumazet <eric.dumazet@gmail.com> wrote:
> >   
> >> On 05/09/2018 10:31 AM, Stephen Hemminger wrote:  
> >>> On Wed, 9 May 2018 10:18:23 -0700
> >>> Eric Dumazet <eric.dumazet@gmail.com> wrote:
> >>>     
> >>>> On 05/09/2018 08:22 AM, Stephen Hemminger wrote:
> >>>>    
> >>>>> I am not sure if these patches are worth applying.
> >>>>> The 'ss -s' command has had missing values since 2.4 kernel.
> >>>>> And the first complaints came in only this year.
> >>>>>
> >>>>> Another alternative would be just to remove these fields from ss -s
> >>>>> output and move on.
> >>>>>       
> >>>>
> >>>> Anyway your patches are not netns ready, so lets remove these fields from ss.
> >>>>
> >>>> Or you have to spend _much_ more time on writing and testing the kernel part.
> >>>>
> >>>> Thanks.    
> >>>
> >>> The patches only expose the existing TCP socket accounting infrastructure.
> >>> Several other pieces that sockstat has are not netns aware.
> >>> That is a completely different problem.    
> >>
> >>
> >> Adding a new field counting 'bounds ports' without being netns ready is a total mistake,
> >> as it is useless by current standards.
> >>
> >> The first thing that users will do is add proper netns support, with extra complexity in the kernel.
> >>
> >> So, instead of pushing some incomplete feature, trying to fool ourselves with a sentiment of 'small cost'
> >> that will later need another 100 lines of code in the kernel, please give us the complete picture.
> >>
> >> I am just saying, you can of course ignore my feedback.  
> > 
> > The current TCP hashinfo should be moved into netns. The current method of scanning and matching
> > by net namespace is a scalability issue now.  
> 
> It is not the plan yet, and we have no scalability issue.
> 
> Before switching to netns hash table, this would need rhashtable conversions
> but so far this has not been done.
> 
> - Time to create/delete netns is critical.
> - Adding few Mbytes of overhead per netns is a nogo,
> 
> Please do not change subject, this is adding noise to this particular thread.

Back to original subject. My current intention is to just pull all these statistics
from ss command. They are always zero now, and very few people noticed and  no one
really needs them.