From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Ahern Subject: Re: Kernel 4.19 network performance - forwarding/routing normal users traffic Date: Fri, 9 Nov 2018 09:21:10 -0700 Message-ID: <1017e0cd-d8a1-ca2f-7d1d-8edad5a37eeb@gmail.com> References: <61697e49-e839-befc-8330-fc00187c48ee@itcare.pl> <3a88bb53-9d17-3e85-638e-a605f5bfe0fb@gmail.com> <20181101115522.10b0dd0a@redhat.com> <63198d68-6752-3695-f406-d86fb395c12b@itcare.pl> <7141e1e0-93e4-ab20-bce6-17f1e14682f1@gmail.com> <394a0bf2-fa97-1085-2eda-98ddf476895c@itcare.pl> <6ed1666d-47bc-24e7-d432-a0c0027452ed@gmail.com> <8dde3b32-59ce-38f3-5913-2ce08264e9dc@itcare.pl> <6165513d-1e27-31dc-8f94-9de029a73f93@gmail.com> <77027868-2b96-8c1d-f485-c7b36c6d9fa9@itcare.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Cc: netdev , Yoel Caspersen To: =?UTF-8?Q?Pawe=c5=82_Staszewski?= , Jesper Dangaard Brouer Return-path: Received: from mail-pl1-f172.google.com ([209.85.214.172]:40668 "EHLO mail-pl1-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728382AbeKJCC3 (ORCPT ); Fri, 9 Nov 2018 21:02:29 -0500 Received: by mail-pl1-f172.google.com with SMTP id q19-v6so1151632pll.7 for ; Fri, 09 Nov 2018 08:21:14 -0800 (PST) In-Reply-To: <77027868-2b96-8c1d-f485-c7b36c6d9fa9@itcare.pl> Content-Language: en-US Sender: netdev-owner@vger.kernel.org List-ID: On 11/9/18 3:20 AM, Paweł Staszewski wrote: > > I just catch some weird behavior :) > All was working fine for about 20k packets > > Then after xdp start to forward every 10 packets Interesting. Any counter showing drops? > ping 172.16.0.2 -i 0.1 > PING 172.16.0.2 (172.16.0.2) 56(84) bytes of data. > 64 bytes from 172.16.0.2: icmp_seq=1 ttl=64 time=5.12 ms > 64 bytes from 172.16.0.2: icmp_seq=9 ttl=64 time=5.20 ms > 64 bytes from 172.16.0.2: icmp_seq=19 ttl=64 time=4.85 ms > 64 bytes from 172.16.0.2: icmp_seq=29 ttl=64 time=4.91 ms > 64 bytes from 172.16.0.2: icmp_seq=38 ttl=64 time=4.85 ms > 64 bytes from 172.16.0.2: icmp_seq=48 ttl=64 time=5.00 ms > ^C > --- 172.16.0.2 ping statistics --- > 55 packets transmitted, 6 received, 89% packet loss, time 5655ms > rtt min/avg/max/mdev = 4.850/4.992/5.203/0.145 ms > > > And again after some time back to normal > >  ping 172.16.0.2 -i 0.1 > PING 172.16.0.2 (172.16.0.2) 56(84) bytes of data. > 64 bytes from 172.16.0.2: icmp_seq=1 ttl=64 time=5.02 ms > 64 bytes from 172.16.0.2: icmp_seq=2 ttl=64 time=5.06 ms > 64 bytes from 172.16.0.2: icmp_seq=3 ttl=64 time=5.19 ms > 64 bytes from 172.16.0.2: icmp_seq=4 ttl=64 time=5.07 ms > 64 bytes from 172.16.0.2: icmp_seq=5 ttl=64 time=5.08 ms > 64 bytes from 172.16.0.2: icmp_seq=6 ttl=64 time=5.14 ms > 64 bytes from 172.16.0.2: icmp_seq=7 ttl=64 time=5.08 ms > 64 bytes from 172.16.0.2: icmp_seq=8 ttl=64 time=5.17 ms > 64 bytes from 172.16.0.2: icmp_seq=9 ttl=64 time=5.04 ms > 64 bytes from 172.16.0.2: icmp_seq=10 ttl=64 time=5.10 ms > 64 bytes from 172.16.0.2: icmp_seq=11 ttl=64 time=5.11 ms > 64 bytes from 172.16.0.2: icmp_seq=12 ttl=64 time=5.13 ms > 64 bytes from 172.16.0.2: icmp_seq=13 ttl=64 time=5.12 ms > 64 bytes from 172.16.0.2: icmp_seq=14 ttl=64 time=5.15 ms > 64 bytes from 172.16.0.2: icmp_seq=15 ttl=64 time=5.13 ms > 64 bytes from 172.16.0.2: icmp_seq=16 ttl=64 time=5.04 ms > 64 bytes from 172.16.0.2: icmp_seq=17 ttl=64 time=5.12 ms > 64 bytes from 172.16.0.2: icmp_seq=18 ttl=64 time=5.07 ms > 64 bytes from 172.16.0.2: icmp_seq=19 ttl=64 time=5.06 ms > 64 bytes from 172.16.0.2: icmp_seq=20 ttl=64 time=5.12 ms > 64 bytes from 172.16.0.2: icmp_seq=21 ttl=64 time=5.21 ms > 64 bytes from 172.16.0.2: icmp_seq=22 ttl=64 time=4.98 ms > ^C > --- 172.16.0.2 ping statistics --- > 22 packets transmitted, 22 received, 0% packet loss, time 2105ms > rtt min/avg/max/mdev = 4.988/5.104/5.210/0.089 ms > > > I will try to catch this with debug enabled > > > > > > Wondering also - cause xdp will bypass now vlan counters and other stuff > like tcpdump yes, xdp is before tcpdump based sockets. And the counters (vlan just being the current example) is another problem to be solved. The vlan net_device never sees the packet and you can not arbitrarily bump the counters just because the device lookups reference them. > > Is there possible to add only counters from xdp for vlans ? > This will help me in testing. I will take a look today at adding counters that you can dump using bpftool. It will be a temporary solution for this xdp program only. > > > And also - for non lab scenario there should be possible to sniff > sometimes on interface :) Yes, sampling is another problem. > Soo wondering if need to attack another xdp program to interface or all > this can be done by one > > I think this is time where i will need to learn more about xdp :) > >