netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH v3 0/7] Reduce cache miss for snmp_fold_field
@ 2016-09-09  6:33 Jia He
  2016-09-09  6:33 ` [RFC PATCH v3 1/7] net:snmp: Introduce generic batch interfaces for snmp_get_cpu_field{,64} Jia He
                   ` (6 more replies)
  0 siblings, 7 replies; 17+ messages in thread
From: Jia He @ 2016-09-09  6:33 UTC (permalink / raw)
  To: netdev
  Cc: linux-sctp, linux-kernel, davem, Alexey Kuznetsov, James Morris,
	Hideaki YOSHIFUJI, Patrick McHardy, Vlad Yasevich, Neil Horman,
	Steffen Klassert, Herbert Xu, marcelo.leitner, Jia He

In a PowerPc server with large cpu number(160), besides commit
a3a773726c9f ("net: Optimize snmp stat aggregation by walking all
the percpu data at once"), I watched several other snmp_fold_field
callsites which will cause high cache miss rate.

My simple test case, which read from the procfs items endlessly:
/***********************************************************/
#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#define LINELEN  2560
int main(int argc, char **argv)
{
        int i;
        int fd = -1 ;
        int rdsize = 0;
        char buf[LINELEN+1];

        buf[LINELEN] = 0;
        memset(buf,0,LINELEN);

        if(1 >= argc) {
                printf("file name empty\n");
                return -1;
        }

        fd = open(argv[1], O_RDWR, 0644);
        if(0 > fd){
                printf("open error\n");
                return -2;
        }

        for(i=0;i<0xffffffff;i++) {
                while(0 < (rdsize = read(fd,buf,LINELEN))){
                        //nothing here
                }

                lseek(fd, 0, SEEK_SET);
        }

        close(fd);
        return 0;
}
/**********************************************************/

compile and run:
gcc test.c -o test

perf stat -d -e cache-misses ./test /proc/net/snmp
perf stat -d -e cache-misses ./test /proc/net/snmp6
perf stat -d -e cache-misses ./test /proc/net/netstat
perf stat -d -e cache-misses ./test /proc/net/sctp/snmp
perf stat -d -e cache-misses ./test /proc/net/xfrm_stat

before the patch set:
====================
 Performance counter stats for 'system wide':

         355911097      cache-misses                                                 [40.08%]
        2356829300      L1-dcache-loads                                              [60.04%]
         355642645      L1-dcache-load-misses     #   15.09% of all L1-dcache hits   [60.02%]
         346544541      LLC-loads                                                    [59.97%]
            389763      LLC-load-misses           #    0.11% of all LL-cache hits    [40.02%]

       6.245162638 seconds time elapsed

After the patch set:
===================
 Performance counter stats for 'system wide':

         194992476      cache-misses                                                 [40.03%]
        6718051877      L1-dcache-loads                                              [60.07%]
         194871921      L1-dcache-load-misses     #    2.90% of all L1-dcache hits   [60.11%]
         187632232      LLC-loads                                                    [60.04%]
            464466      LLC-load-misses           #    0.25% of all LL-cache hits    [39.89%]

       6.868422769 seconds time elapsed
The cache-miss rate can be reduced from 15% to 2.9%

v3:
- introduce generic interface (suggested by Marcelo Ricardo Leitner)
- use max_t instead of self defined macro (suggested by David Miller)
v2:
- fix bug in udplite statistics.
- snmp_seq_show is split into 2 parts

Jia He (7):
  net:snmp: Introduce generic interfaces for snmp_get_cpu_field{,64}
  proc: Reduce cache miss in {snmp,netstat}_seq_show
  proc: Reduce cache miss in snmp6_seq_show
  proc: Reduce cache miss in sctp_snmp_seq_show
  proc: Reduce cache miss in xfrm_statistics_seq_show
  ipv6: Remove useless parameter in __snmp6_fill_statsdev
  net: Suppress the "Comparison to NULL could be written" warning

 include/net/ip.h     |  23 +++++++++
 net/ipv4/proc.c      | 138 +++++++++++++++++++++++++++++++++------------------
 net/ipv6/addrconf.c  |  12 ++---
 net/ipv6/proc.c      |  32 ++++++++----
 net/sctp/proc.c      |  10 ++--
 net/xfrm/xfrm_proc.c |  10 +++-
 6 files changed, 157 insertions(+), 68 deletions(-)

-- 
1.8.3.1

^ permalink raw reply	[flat|nested] 17+ messages in thread
end of thread, other threads:[~2016-09-22  5:38 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-09-09  6:33 [RFC PATCH v3 0/7] Reduce cache miss for snmp_fold_field Jia He
2016-09-09  6:33 ` [RFC PATCH v3 1/7] net:snmp: Introduce generic batch interfaces for snmp_get_cpu_field{,64} Jia He
2016-09-09  6:33 ` [RFC PATCH v3 2/7] proc: Reduce cache miss in {snmp,netstat}_seq_show Jia He
2016-09-12 18:57   ` Marcelo
2016-09-14  3:10     ` hejianet
2016-09-14  5:58     ` hejianet
2016-09-14 11:55       ` Marcelo
2016-09-21 16:18         ` hejianet
2016-09-21 18:24           ` Marcelo
2016-09-22  5:38             ` hejianet
2016-09-09  6:33 ` [RFC PATCH v3 3/7] proc: Reduce cache miss in snmp6_seq_show Jia He
2016-09-12 19:05   ` Marcelo
2016-09-14  3:11     ` hejianet
2016-09-09  6:33 ` [RFC PATCH v3 4/7] proc: Reduce cache miss in sctp_snmp_seq_show Jia He
2016-09-09  6:34 ` [RFC PATCH v3 5/7] proc: Reduce cache miss in xfrm_statistics_seq_show Jia He
2016-09-09  6:34 ` [RFC PATCH v3 6/7] ipv6: Remove useless parameter in __snmp6_fill_statsdev Jia He
2016-09-09  6:34 ` [RFC PATCH v3 7/7] net: Suppress the "Comparison to NULL could be written" warning Jia He

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).