From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Ahern Subject: Re: Kernel 4.19 network performance - forwarding/routing normal users traffic Date: Thu, 8 Nov 2018 09:06:51 -0700 Message-ID: <6165513d-1e27-31dc-8f94-9de029a73f93@gmail.com> References: <61697e49-e839-befc-8330-fc00187c48ee@itcare.pl> <3a88bb53-9d17-3e85-638e-a605f5bfe0fb@gmail.com> <20181101115522.10b0dd0a@redhat.com> <63198d68-6752-3695-f406-d86fb395c12b@itcare.pl> <7141e1e0-93e4-ab20-bce6-17f1e14682f1@gmail.com> <394a0bf2-fa97-1085-2eda-98ddf476895c@itcare.pl> <6ed1666d-47bc-24e7-d432-a0c0027452ed@gmail.com> <8dde3b32-59ce-38f3-5913-2ce08264e9dc@itcare.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Cc: netdev , Yoel Caspersen To: =?UTF-8?Q?Pawe=c5=82_Staszewski?= , Jesper Dangaard Brouer Return-path: Received: from mail-pf1-f196.google.com ([209.85.210.196]:39502 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726700AbeKIBnF (ORCPT ); Thu, 8 Nov 2018 20:43:05 -0500 Received: by mail-pf1-f196.google.com with SMTP id n11-v6so9498528pfb.6 for ; Thu, 08 Nov 2018 08:06:55 -0800 (PST) In-Reply-To: <8dde3b32-59ce-38f3-5913-2ce08264e9dc@itcare.pl> Content-Language: en-US Sender: netdev-owner@vger.kernel.org List-ID: On 11/8/18 6:33 AM, Paweł Staszewski wrote: > > > W dniu 07.11.2018 o 22:06, David Ahern pisze: >> On 11/3/18 6:24 PM, Paweł Staszewski wrote: >>>> Does your setup have any other device types besides physical ports with >>>> VLANs (e.g., any macvlans or bonds)? >>>> >>>> >>> no. >>> just >>> phy(mlnx)->vlans only config >> VLAN and non-VLAN (and a mix) seem to work ok. Patches are here: >>     https://github.com/dsahern/linux.git bpf/kernel-tables-wip >> >> I got lazy with the vlan exports; right now it requires 8021q to be >> builtin (CONFIG_VLAN_8021Q=y) >> >> You can use the xdp_fwd sample: >>    make O=kbuild -C samples/bpf -j 8 >> >> Copy samples/bpf/xdp_fwd_kern.o and samples/bpf/xdp_fwd to the server >> and run: >>     ./xdp_fwd >> >> e.g., in my testing I run: >>     xdp_fwd eth1 eth2 eth3 eth4 >> >> All of the relevant forwarding ports need to be on the same command >> line. This version populates a second map to verify the egress port has >> XDP enabled. > Installed today on some lab server with mellanox connectx4 > > And trying some simple static routing first - but after enabling xdp > program - receiver is not receiving frames > > Route table is simple as possible for tests :) > > icmp ping test send from 192.168.22.237 to 172.16.0.2 - incomming > packets on vlan 4081 > > ip r > default via 192.168.22.236 dev vlan4081 > 172.16.0.0/30 dev vlan1740 proto kernel scope link src 172.16.0.1 > 192.168.22.0/24 dev vlan4081 proto kernel scope link src 192.168.22.205 > > neigh table: > ip neigh ls > > 192.168.22.237 dev vlan4081 lladdr 00:25:90:fb:a6:8d REACHABLE > 172.16.0.2 dev vlan1740 lladdr ac:1f:6b:2c:2e:5a REACHABLE > > and interfaces: > 4: enp175s0f0: mtu 1500 qdisc mq state > UP mode DEFAULT group default qlen 1000 >     link/ether ac:1f:6b:07:c8:90 brd ff:ff:ff:ff:ff:ff > 5: enp175s0f1: mtu 1500 qdisc mq state > UP mode DEFAULT group default qlen 1000 >     link/ether ac:1f:6b:07:c8:91 brd ff:ff:ff:ff:ff:ff > 6: vlan4081@enp175s0f0: mtu 1500 qdisc > noqueue state UP mode DEFAULT group default qlen 1000 >     link/ether ac:1f:6b:07:c8:90 brd ff:ff:ff:ff:ff:ff > 7: vlan1740@enp175s0f1: mtu 1500 qdisc > noqueue state UP mode DEFAULT group default qlen 1000 >     link/ether ac:1f:6b:07:c8:91 brd ff:ff:ff:ff:ff:ff > > 5: enp175s0f1: mtu 1500 xdp/id:5 qdisc > mq state UP group default qlen 1000 >     link/ether ac:1f:6b:07:c8:91 brd ff:ff:ff:ff:ff:ff >     inet6 fe80::ae1f:6bff:fe07:c891/64 scope link >        valid_lft forever preferred_lft forever > 6: vlan4081@enp175s0f0: mtu 1500 qdisc > noqueue state UP group default qlen 1000 >     link/ether ac:1f:6b:07:c8:90 brd ff:ff:ff:ff:ff:ff >     inet 192.168.22.205/24 scope global vlan4081 >        valid_lft forever preferred_lft forever >     inet6 fe80::ae1f:6bff:fe07:c890/64 scope link >        valid_lft forever preferred_lft forever > 7: vlan1740@enp175s0f1: mtu 1500 qdisc > noqueue state UP group default qlen 1000 >     link/ether ac:1f:6b:07:c8:91 brd ff:ff:ff:ff:ff:ff >     inet 172.16.0.1/30 scope global vlan1740 >        valid_lft forever preferred_lft forever >     inet6 fe80::ae1f:6bff:fe07:c891/64 scope link >        valid_lft forever preferred_lft forever > > > xdp program detached: > Receiving side tcpdump: > 14:28:09.141233 IP 192.168.22.237 > 172.16.0.2: ICMP echo request, id > 30227, seq 487, length 64 > > I can see icmp requests > > > enabling xdp > ./xdp_fwd enp175s0f1 enp175s0f0 > > 4: enp175s0f0: mtu 1500 xdp qdisc mq > state UP mode DEFAULT group default qlen 1000 >     link/ether ac:1f:6b:07:c8:90 brd ff:ff:ff:ff:ff:ff >     prog/xdp id 5 tag 3c231ff1e5e77f3f > 5: enp175s0f1: mtu 1500 xdp qdisc mq > state UP mode DEFAULT group default qlen 1000 >     link/ether ac:1f:6b:07:c8:91 brd ff:ff:ff:ff:ff:ff >     prog/xdp id 5 tag 3c231ff1e5e77f3f > 6: vlan4081@enp175s0f0: mtu 1500 qdisc > noqueue state UP mode DEFAULT group default qlen 1000 >     link/ether ac:1f:6b:07:c8:90 brd ff:ff:ff:ff:ff:ff > 7: vlan1740@enp175s0f1: mtu 1500 qdisc > noqueue state UP mode DEFAULT group default qlen 1000 >     link/ether ac:1f:6b:07:c8:91 brd ff:ff:ff:ff:ff:ff > What hardware is this? Start with: echo 1 > /sys/kernel/debug/tracing/events/xdp/enable cat /sys/kernel/debug/tracing/trace_pipe >>From there, you can check the FIB lookups: sysctl -w kernel.perf_event_max_stack=16 perf record -e fib:* -a -g -- sleep 5 perf script