Re: 3.4.1 and 3.5-rc1 Packet lost at 250Mb/s

[Eric Dumazet <eric.dumazet@gmail.com>]

On Fri, 2012-07-06 at 11:47 +0200, Nieścierowicz Adam wrote:
> Hello,
> Can I send something that will help determine the cause of the problem?
> 
> 
> W dniu 08.06.2012 11:41, Eric Dumazet napisał(a):
> 
> > On Fri, 2012-06-08 at 10:58 +0200, Nieścierowicz Adam wrote:
> >
> >> Hello, recently we changed on the router kernel from 2.6.38.1 to 
> >> 3.4.1
> >> and noticed 30% packet loss when traffic increases up to 250MB / s.
> >> Similar is for kernel 3.5-rc1 Here a link to ifstat
> >> http://wklej.org/id/767577/ [2]
> >
> > You should give as much as possible delails on your setup (hardware,
> > software)
> >
> > lspci
> > cat /proc/cpuinfo
> > cat /proc/interrupts
> > ifconfig -a
> > tc -s -d qdisc
> > dmesg
> > netstat -s
> 
> currently running on 2.6.38.1 and traffic is 100Mb / s
> 
> lspci: http://wklej.org/id/769102/
> /proc/cpuinfo: http://wklej.org/id/769104/
> /proc/interrupts: http://wklej.org/id/769106/
> ifconfig -a: http://wklej.org/id/769108/
> tc -s -d qdisc: http://wklej.org/id/769109/
> dmesg: here are some logs from iptables
> netstat -s: http://wklej.org/id/769110/
> lsmod: http://wklej.org/id/769117/
> /proc/net/softnet_stat: http://wklej.org/id/769116/


Same infos of 3.5-rcX kernel would be nice.

What NIC is eth0 ?  (dmesg please)

It seems all network traffic on 2.6.38 is handled by a single cpu (cpu0)

(seen in /proc/interrupts)

I suspect that with 3.4 or 3.5 kernels, traffic is handled by many cpus
and they hit false sharing and contention.

You probably get better performance doing some affinity tuning :

For example, 
  eth0 serviced by cpu0
  eth2 serviced by cpu1
  eth3 serviced by cpu2
  eth5 serviced by cpu3

and so on...

check and/or set /proc/irq/${NUM}/smp_affinity