* Re: [PATCH net-next v2 1/2] dpaa2-eth: defer probe on object allocate
From: Andrew Lunn @ 2018-11-09 20:39 UTC (permalink / raw)
To: Ioana Ciornei
Cc: netdev@vger.kernel.org, davem@davemloft.net,
Ioana Ciocoi Radulescu
In-Reply-To: <1541777182-9135-2-git-send-email-ioana.ciornei@nxp.com>
On Fri, Nov 09, 2018 at 03:26:45PM +0000, Ioana Ciornei wrote:
> The fsl_mc_object_allocate function can fail because not all allocatable
> objects are probed by the fsl_mc_allocator at the call time. Defer the
> dpaa2-eth probe when this happens.
>
> Signed-off-by: Ioana Ciornei <ioana.ciornei@nxp.com>
> ---
> Changes in v2:
> - proper handling of IS_ERR_OR_NULL
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Andrew
^ permalink raw reply
* Re: [PATCH v2 net-next] net: phy: improve struct phy_device member interrupts handling
From: Florian Fainelli @ 2018-11-09 20:19 UTC (permalink / raw)
To: Heiner Kallweit, Andrew Lunn, David Miller; +Cc: netdev@vger.kernel.org
In-Reply-To: <63a44097-c37d-eb0f-98c8-2c1fefd1f246@gmail.com>
On 11/9/18 9:35 AM, Heiner Kallweit wrote:
> As a heritage from the very early days of phylib member interrupts is
> defined as u32 even though it's just a flag whether interrupts are
> enabled. So we can change it to a bitfield member. In addition change
> the code dealing with this member in a way that it's clear we're
> dealing with a bool value.
>
> Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
--
Florian
^ permalink raw reply
* Re: [PATCH v5 bpf-next 0/7] bpftool: support loading flow dissector
From: Jakub Kicinski @ 2018-11-09 20:15 UTC (permalink / raw)
To: Stanislav Fomichev
Cc: netdev, linux-kselftest, ast, daniel, shuah, quentin.monnet, guro,
jiong.wang, bhole_prashant_q7, john.fastabend, jbenc,
treeze.taeung, yhs, osk, sandipan
In-Reply-To: <20181109162146.78019-1-sdf@google.com>
On Fri, 9 Nov 2018 08:21:39 -0800, Stanislav Fomichev wrote:
> v5 changes:
> * FILE -> PATH for load/loadall (can be either file or directory now)
> * simpler implementation for __bpf_program__pin_name
> * removed p_err for REQ_ARGS checks
> * parse_atach_detach_args -> parse_attach_detach_args
> * for -> while in bpf_object__pin_{programs,maps} recovery
Thanks! Patch 3 needs attention from maintainers but the rest LGTM!
^ permalink raw reply
* Re: [PATCH v2 net-next] net: phy: improve struct phy_device member interrupts handling
From: Andrew Lunn @ 2018-11-09 20:14 UTC (permalink / raw)
To: Heiner Kallweit; +Cc: Florian Fainelli, David Miller, netdev@vger.kernel.org
In-Reply-To: <63a44097-c37d-eb0f-98c8-2c1fefd1f246@gmail.com>
On Fri, Nov 09, 2018 at 06:35:52PM +0100, Heiner Kallweit wrote:
> As a heritage from the very early days of phylib member interrupts is
> defined as u32 even though it's just a flag whether interrupts are
> enabled. So we can change it to a bitfield member. In addition change
> the code dealing with this member in a way that it's clear we're
> dealing with a bool value.
>
> Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Andrew
^ permalink raw reply
* Re: [PATCH bpf-next v2 02/13] bpf: btf: Add BTF_KIND_FUNC and BTF_KIND_FUNC_PROTO
From: Edward Cree @ 2018-11-09 20:00 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Martin KaFai Lau, Yonghong Song, Alexei Starovoitov,
Daniel Borkmann, Network Development, Kernel Team
In-Reply-To: <20181109043507.dyemsy6va4wto67l@ast-mbp>
On 09/11/18 04:35, Alexei Starovoitov wrote:
> On Thu, Nov 08, 2018 at 10:56:55PM +0000, Edward Cree wrote:
>> think this question of maps should be discussed in tomorrow's
>> call, since it is when we start having other kinds of instances
> turned out most of us have a conflict, so the earliest is 1:30pm on Friday.
> still works for you?
Yep (that's 9.30pm GMT right?)
I'm assuming same bluejeans link again.
-Ed
^ permalink raw reply
* Re: Kernel 4.19 network performance - forwarding/routing normal users traffic
From: Paweł Staszewski @ 2018-11-09 19:59 UTC (permalink / raw)
To: David Ahern, Jesper Dangaard Brouer; +Cc: netdev, Yoel Caspersen
In-Reply-To: <1017e0cd-d8a1-ca2f-7d1d-8edad5a37eeb@gmail.com>
W dniu 09.11.2018 o 17:21, David Ahern pisze:
> On 11/9/18 3:20 AM, Paweł Staszewski wrote:
>> I just catch some weird behavior :)
>> All was working fine for about 20k packets
>>
>> Then after xdp start to forward every 10 packets
> Interesting. Any counter showing drops?
nothing that will fit
NIC statistics:
rx_packets: 187041
rx_bytes: 10600954
tx_packets: 40316
tx_bytes: 16526844
tx_tso_packets: 797
tx_tso_bytes: 3876084
tx_tso_inner_packets: 0
tx_tso_inner_bytes: 0
tx_added_vlan_packets: 38391
tx_nop: 2
rx_lro_packets: 0
rx_lro_bytes: 0
rx_ecn_mark: 0
rx_removed_vlan_packets: 187041
rx_csum_unnecessary: 0
rx_csum_none: 150011
rx_csum_complete: 37030
rx_csum_unnecessary_inner: 0
rx_xdp_drop: 0
rx_xdp_redirect: 64893
rx_xdp_tx_xmit: 0
rx_xdp_tx_full: 0
rx_xdp_tx_err: 0
rx_xdp_tx_cqe: 0
tx_csum_none: 2468
tx_csum_partial: 35955
tx_csum_partial_inner: 0
tx_queue_stopped: 0
tx_queue_dropped: 0
tx_xmit_more: 0
tx_recover: 0
tx_cqes: 38423
tx_queue_wake: 0
tx_udp_seg_rem: 0
tx_cqe_err: 0
tx_xdp_xmit: 0
tx_xdp_full: 0
tx_xdp_err: 0
tx_xdp_cqes: 0
rx_wqe_err: 0
rx_mpwqe_filler_cqes: 0
rx_mpwqe_filler_strides: 0
rx_buff_alloc_err: 0
rx_cqe_compress_blks: 0
rx_cqe_compress_pkts: 0
rx_page_reuse: 0
rx_cache_reuse: 186302
rx_cache_full: 0
rx_cache_empty: 666768
rx_cache_busy: 174
rx_cache_waive: 0
rx_congst_umr: 0
rx_arfs_err: 0
ch_events: 249320
ch_poll: 249321
ch_arm: 249001
ch_aff_change: 0
ch_eq_rearm: 0
rx_out_of_buffer: 0
rx_if_down_packets: 57
rx_vport_unicast_packets: 142659
rx_vport_unicast_bytes: 42706914
tx_vport_unicast_packets: 40167
tx_vport_unicast_bytes: 16668096
rx_vport_multicast_packets: 39188170
rx_vport_multicast_bytes: 3466527450
tx_vport_multicast_packets: 58
tx_vport_multicast_bytes: 4556
rx_vport_broadcast_packets: 16343520
rx_vport_broadcast_bytes: 1031334602
tx_vport_broadcast_packets: 91
tx_vport_broadcast_bytes: 5460
rx_vport_rdma_unicast_packets: 0
rx_vport_rdma_unicast_bytes: 0
tx_vport_rdma_unicast_packets: 0
tx_vport_rdma_unicast_bytes: 0
rx_vport_rdma_multicast_packets: 0
rx_vport_rdma_multicast_bytes: 0
tx_vport_rdma_multicast_packets: 0
tx_vport_rdma_multicast_bytes: 0
tx_packets_phy: 40316
rx_packets_phy: 55674361
rx_crc_errors_phy: 0
tx_bytes_phy: 16839376
rx_bytes_phy: 4763267396
tx_multicast_phy: 58
tx_broadcast_phy: 91
rx_multicast_phy: 39188180
rx_broadcast_phy: 16343521
rx_in_range_len_errors_phy: 0
rx_out_of_range_len_phy: 0
rx_oversize_pkts_phy: 0
rx_symbol_err_phy: 0
tx_mac_control_phy: 0
rx_mac_control_phy: 0
rx_unsupported_op_phy: 0
rx_pause_ctrl_phy: 0
tx_pause_ctrl_phy: 0
rx_discards_phy: 1
tx_discards_phy: 0
tx_errors_phy: 0
rx_undersize_pkts_phy: 0
rx_fragments_phy: 0
rx_jabbers_phy: 0
rx_64_bytes_phy: 3792455
rx_65_to_127_bytes_phy: 51821620
rx_128_to_255_bytes_phy: 37669
rx_256_to_511_bytes_phy: 1481
rx_512_to_1023_bytes_phy: 434
rx_1024_to_1518_bytes_phy: 694
rx_1519_to_2047_bytes_phy: 20008
rx_2048_to_4095_bytes_phy: 0
rx_4096_to_8191_bytes_phy: 0
rx_8192_to_10239_bytes_phy: 0
link_down_events_phy: 0
rx_pcs_symbol_err_phy: 0
rx_corrected_bits_phy: 6
rx_err_lane_0_phy: 0
rx_err_lane_1_phy: 0
rx_err_lane_2_phy: 0
rx_err_lane_3_phy: 6
rx_buffer_passed_thres_phy: 0
rx_pci_signal_integrity: 0
tx_pci_signal_integrity: 82
outbound_pci_stalled_rd: 0
outbound_pci_stalled_wr: 0
outbound_pci_stalled_rd_events: 0
outbound_pci_stalled_wr_events: 0
rx_prio0_bytes: 4144920388
rx_prio0_packets: 48310037
tx_prio0_bytes: 16839376
tx_prio0_packets: 40316
rx_prio1_bytes: 481032
rx_prio1_packets: 7074
tx_prio1_bytes: 0
tx_prio1_packets: 0
rx_prio2_bytes: 9074194
rx_prio2_packets: 106207
tx_prio2_bytes: 0
tx_prio2_packets: 0
rx_prio3_bytes: 0
rx_prio3_packets: 0
tx_prio3_bytes: 0
tx_prio3_packets: 0
rx_prio4_bytes: 0
rx_prio4_packets: 0
tx_prio4_bytes: 0
tx_prio4_packets: 0
rx_prio5_bytes: 0
rx_prio5_packets: 0
tx_prio5_bytes: 0
tx_prio5_packets: 0
rx_prio6_bytes: 371961810
rx_prio6_packets: 4006281
tx_prio6_bytes: 0
tx_prio6_packets: 0
rx_prio7_bytes: 236830040
rx_prio7_packets: 3244761
tx_prio7_bytes: 0
tx_prio7_packets: 0
tx_pause_storm_warning_events : 0
tx_pause_storm_error_events: 0
module_unplug: 0
module_bus_stuck: 0
module_high_temp: 0
module_bad_shorted: 0
NIC statistics:
rx_packets: 843
rx_bytes: 58889
tx_packets: 324
tx_bytes: 23324
tx_tso_packets: 0
tx_tso_bytes: 0
tx_tso_inner_packets: 0
tx_tso_inner_bytes: 0
tx_added_vlan_packets: 293
tx_nop: 0
rx_lro_packets: 0
rx_lro_bytes: 0
rx_ecn_mark: 0
rx_removed_vlan_packets: 843
rx_csum_unnecessary: 0
rx_csum_none: 190
rx_csum_complete: 653
rx_csum_unnecessary_inner: 0
rx_xdp_drop: 0
rx_xdp_redirect: 0
rx_xdp_tx_xmit: 0
rx_xdp_tx_full: 0
rx_xdp_tx_err: 0
rx_xdp_tx_cqe: 0
tx_csum_none: 324
tx_csum_partial: 0
tx_csum_partial_inner: 0
tx_queue_stopped: 0
tx_queue_dropped: 0
tx_xmit_more: 1
tx_recover: 0
tx_cqes: 323
tx_queue_wake: 0
tx_udp_seg_rem: 0
tx_cqe_err: 0
tx_xdp_xmit: 64926
tx_xdp_full: 0
tx_xdp_err: 0
tx_xdp_cqes: 47958
rx_wqe_err: 0
rx_mpwqe_filler_cqes: 0
rx_mpwqe_filler_strides: 0
rx_buff_alloc_err: 0
rx_cqe_compress_blks: 0
rx_cqe_compress_pkts: 0
rx_page_reuse: 0
rx_cache_reuse: 648
rx_cache_full: 0
rx_cache_empty: 602112
rx_cache_busy: 0
rx_cache_waive: 0
rx_congst_umr: 0
rx_arfs_err: 0
ch_events: 49628
ch_poll: 49628
ch_arm: 49626
ch_aff_change: 0
ch_eq_rearm: 0
rx_out_of_buffer: 0
rx_if_down_packets: 46
rx_vport_unicast_packets: 5953
rx_vport_unicast_bytes: 4927049
tx_vport_unicast_packets: 65194
tx_vport_unicast_bytes: 31820150
rx_vport_multicast_packets: 37085249
rx_vport_multicast_bytes: 2449620421
tx_vport_multicast_packets: 55
tx_vport_multicast_bytes: 4278
rx_vport_broadcast_packets: 434654
rx_vport_broadcast_bytes: 31881063
tx_vport_broadcast_packets: 1
tx_vport_broadcast_bytes: 60
rx_vport_rdma_unicast_packets: 0
rx_vport_rdma_unicast_bytes: 0
tx_vport_rdma_unicast_packets: 0
tx_vport_rdma_unicast_bytes: 0
rx_vport_rdma_multicast_packets: 0
rx_vport_rdma_multicast_bytes: 0
tx_vport_rdma_multicast_packets: 0
tx_vport_rdma_multicast_bytes: 0
tx_packets_phy: 65250
rx_packets_phy: 37525857
rx_crc_errors_phy: 0
tx_bytes_phy: 32085488
rx_bytes_phy: 2636532027
tx_multicast_phy: 55
tx_broadcast_phy: 1
rx_multicast_phy: 37085250
rx_broadcast_phy: 434654
rx_in_range_len_errors_phy: 0
rx_out_of_range_len_phy: 0
rx_oversize_pkts_phy: 0
rx_symbol_err_phy: 0
tx_mac_control_phy: 0
rx_mac_control_phy: 0
rx_unsupported_op_phy: 0
rx_pause_ctrl_phy: 0
tx_pause_ctrl_phy: 0
rx_discards_phy: 0
tx_discards_phy: 0
tx_errors_phy: 0
rx_undersize_pkts_phy: 0
rx_fragments_phy: 0
rx_jabbers_phy: 0
rx_64_bytes_phy: 63346
rx_65_to_127_bytes_phy: 37434768
rx_128_to_255_bytes_phy: 14088
rx_256_to_511_bytes_phy: 10461
rx_512_to_1023_bytes_phy: 96
rx_1024_to_1518_bytes_phy: 1933
rx_1519_to_2047_bytes_phy: 1165
rx_2048_to_4095_bytes_phy: 0
rx_4096_to_8191_bytes_phy: 0
rx_8192_to_10239_bytes_phy: 0
link_down_events_phy: 0
rx_pcs_symbol_err_phy: 0
rx_corrected_bits_phy: 5
rx_err_lane_0_phy: 1
rx_err_lane_1_phy: 0
rx_err_lane_2_phy: 0
rx_err_lane_3_phy: 4
rx_buffer_passed_thres_phy: 0
rx_pci_signal_integrity: 0
tx_pci_signal_integrity: 82
outbound_pci_stalled_rd: 0
outbound_pci_stalled_wr: 0
outbound_pci_stalled_rd_events: 0
outbound_pci_stalled_wr_events: 0
rx_prio0_bytes: 23157221
rx_prio0_packets: 195789
tx_prio0_bytes: 32085488
tx_prio0_packets: 65250
rx_prio1_bytes: 0
rx_prio1_packets: 0
tx_prio1_bytes: 0
tx_prio1_packets: 0
rx_prio2_bytes: 0
rx_prio2_packets: 0
tx_prio2_bytes: 0
tx_prio2_packets: 0
rx_prio3_bytes: 23397578
rx_prio3_packets: 343182
tx_prio3_bytes: 0
tx_prio3_packets: 0
rx_prio4_bytes: 0
rx_prio4_packets: 0
tx_prio4_bytes: 0
tx_prio4_packets: 0
rx_prio5_bytes: 0
rx_prio5_packets: 0
tx_prio5_bytes: 0
tx_prio5_packets: 0
rx_prio6_bytes: 14643472
rx_prio6_packets: 203589
tx_prio6_bytes: 0
tx_prio6_packets: 0
rx_prio7_bytes: 2575333474
rx_prio7_packets: 36783293
tx_prio7_bytes: 0
tx_prio7_packets: 0
tx_pause_storm_warning_events : 0
tx_pause_storm_error_events: 0
module_unplug: 0
module_bus_stuck: 0
module_high_temp: 0
module_bad_shorted: 0
But wondering if any offloading now can do some things that we dont want
for xdp
currently all offloads are enabled.
ethtool -k enp175s0f0
Features for enp175s0f0:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: on
tx-gre-csum-segmentation: on
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: on
tx-udp_tnl-csum-segmentation: on
tx-gso-partial: on
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: on
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: on
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: on [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: on
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
Also at the time when xdp is forwarding 1/10 frame - same problem is
with local input/output traffic - testing server is also responding to
1/10 icmp request
>
>
>> ping 172.16.0.2 -i 0.1
>> PING 172.16.0.2 (172.16.0.2) 56(84) bytes of data.
>> 64 bytes from 172.16.0.2: icmp_seq=1 ttl=64 time=5.12 ms
>> 64 bytes from 172.16.0.2: icmp_seq=9 ttl=64 time=5.20 ms
>> 64 bytes from 172.16.0.2: icmp_seq=19 ttl=64 time=4.85 ms
>> 64 bytes from 172.16.0.2: icmp_seq=29 ttl=64 time=4.91 ms
>> 64 bytes from 172.16.0.2: icmp_seq=38 ttl=64 time=4.85 ms
>> 64 bytes from 172.16.0.2: icmp_seq=48 ttl=64 time=5.00 ms
>> ^C
>> --- 172.16.0.2 ping statistics ---
>> 55 packets transmitted, 6 received, 89% packet loss, time 5655ms
>> rtt min/avg/max/mdev = 4.850/4.992/5.203/0.145 ms
>>
>>
>> And again after some time back to normal
>>
>> ping 172.16.0.2 -i 0.1
>> PING 172.16.0.2 (172.16.0.2) 56(84) bytes of data.
>> 64 bytes from 172.16.0.2: icmp_seq=1 ttl=64 time=5.02 ms
>> 64 bytes from 172.16.0.2: icmp_seq=2 ttl=64 time=5.06 ms
>> 64 bytes from 172.16.0.2: icmp_seq=3 ttl=64 time=5.19 ms
>> 64 bytes from 172.16.0.2: icmp_seq=4 ttl=64 time=5.07 ms
>> 64 bytes from 172.16.0.2: icmp_seq=5 ttl=64 time=5.08 ms
>> 64 bytes from 172.16.0.2: icmp_seq=6 ttl=64 time=5.14 ms
>> 64 bytes from 172.16.0.2: icmp_seq=7 ttl=64 time=5.08 ms
>> 64 bytes from 172.16.0.2: icmp_seq=8 ttl=64 time=5.17 ms
>> 64 bytes from 172.16.0.2: icmp_seq=9 ttl=64 time=5.04 ms
>> 64 bytes from 172.16.0.2: icmp_seq=10 ttl=64 time=5.10 ms
>> 64 bytes from 172.16.0.2: icmp_seq=11 ttl=64 time=5.11 ms
>> 64 bytes from 172.16.0.2: icmp_seq=12 ttl=64 time=5.13 ms
>> 64 bytes from 172.16.0.2: icmp_seq=13 ttl=64 time=5.12 ms
>> 64 bytes from 172.16.0.2: icmp_seq=14 ttl=64 time=5.15 ms
>> 64 bytes from 172.16.0.2: icmp_seq=15 ttl=64 time=5.13 ms
>> 64 bytes from 172.16.0.2: icmp_seq=16 ttl=64 time=5.04 ms
>> 64 bytes from 172.16.0.2: icmp_seq=17 ttl=64 time=5.12 ms
>> 64 bytes from 172.16.0.2: icmp_seq=18 ttl=64 time=5.07 ms
>> 64 bytes from 172.16.0.2: icmp_seq=19 ttl=64 time=5.06 ms
>> 64 bytes from 172.16.0.2: icmp_seq=20 ttl=64 time=5.12 ms
>> 64 bytes from 172.16.0.2: icmp_seq=21 ttl=64 time=5.21 ms
>> 64 bytes from 172.16.0.2: icmp_seq=22 ttl=64 time=4.98 ms
>> ^C
>> --- 172.16.0.2 ping statistics ---
>> 22 packets transmitted, 22 received, 0% packet loss, time 2105ms
>> rtt min/avg/max/mdev = 4.988/5.104/5.210/0.089 ms
>>
>>
>> I will try to catch this with debug enabled
>>
>>
>>
>>
>>
>> Wondering also - cause xdp will bypass now vlan counters and other stuff
>> like tcpdump
> yes, xdp is before tcpdump based sockets.
>
> And the counters (vlan just being the current example) is another
> problem to be solved. The vlan net_device never sees the packet and you
> can not arbitrarily bump the counters just because the device lookups
> reference them.
Ok.
>> Is there possible to add only counters from xdp for vlans ?
>> This will help me in testing.
> I will take a look today at adding counters that you can dump using
> bpftool. It will be a temporary solution for this xdp program only.
Yes anything that can give me counters to check traffic lvls
>>
>> And also - for non lab scenario there should be possible to sniff
>> sometimes on interface :)
> Yes, sampling is another problem.
>
>
>> Soo wondering if need to attack another xdp program to interface or all
>> this can be done by one
>>
>> I think this is time where i will need to learn more about xdp :)
>>
>>
>
^ permalink raw reply
* Re: [PATCH iproute] ss: Actually print left delimiter for columns
From: Stefano Brivio @ 2018-11-09 19:47 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: Yoann P., netdev, Phil Sutter
In-Reply-To: <20181109090546.50c15c61@xeon-e3>
On Fri, 9 Nov 2018 09:05:46 -0800
Stephen Hemminger <stephen@networkplumber.org> wrote:
> On Mon, 29 Oct 2018 23:04:25 +0100
> Stefano Brivio <sbrivio@redhat.com> wrote:
>
> > While rendering columns, we use a local variable to keep track of the
> > field currently being printed, without touching current_field, which is
> > used for buffering.
> >
> > Use the right pointer to access the left delimiter for the current column,
> > instead of always printing the left delimiter for the last buffered field,
> > which is usually an empty string.
> >
> > This fixes an issue especially visible on narrow terminals, where some
> > columns might be displayed without separation.
> >
> > Reported-by: YoyPa <yoann.p.public@gmail.com>
> > Fixes: 691bd854bf4a ("ss: Buffer raw fields first, then render them as a table")
> > Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
> > Tested-by: YoyPa <yoann.p.public@gmail.com>
>
> This test broke the testsuite/ss/ssfilter.t test.
> Please fix the test to match your new output format, or I will have to revert it.
Ouch, sorry, I didn't notice that "new" test. I'll fix that by tomorrow.
--
Stefano
^ permalink raw reply
* Re: [PATCH net] ip: hash fragments consistently
From: Eric Dumazet @ 2018-11-09 19:44 UTC (permalink / raw)
To: Paolo Abeni, netdev; +Cc: David S. Miller, soukjin bae
In-Reply-To: <211e391b-3537-560b-0522-9ea595848477@gmail.com>
On 07/23/2018 09:26 AM, Eric Dumazet wrote:
>
>
> On 07/23/2018 07:50 AM, Paolo Abeni wrote:
>> The skb hash for locally generated ip[v6] fragments belonging
>> to the same datagram can vary in several circumstances:
>> * for connected UDP[v6] sockets, the first fragment get its hash
>> via set_owner_w()/skb_set_hash_from_sk()
>> * for unconnected IPv6 UDPv6 sockets, the first fragment can get
>> its hash via ip6_make_flowlabel()/skb_get_hash_flowi6(), if
>> auto_flowlabel is enabled
>>
>> For the following frags the hash is usually computed via
>> skb_get_hash().
>> The above can cause OoO for unconnected IPv6 UDPv6 socket: in that
>> scenario the egress tx queue can be selected on a per packet basis
>> via the skb hash.
>> It may also fool flow-oriented schedulers to place fragments belonging
>> to the same datagram in different flows.
>>
>
> It also fools bond_xmit_hash(), packets of the same datagram can be sent on
> two bonding slaves instead of one, meaning adding pressure on the defrag unit
> in receiver.
>
> Reviewed-by: Eric Dumazet <edumazet@google.com>
>
Also we might note that flow dissector itself is buggy as
found by Soukjin Bae ( https://patchwork.ozlabs.org/patch/994601/ )
I will send a v2 of his patch with a different changelog.
Defrag is fixed [1] but the bug in flow dissector is adding
extra work and hash inconsistencies.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git/commit/?id=0d5b9311baf27bb545f187f12ecfd558220c607d
^ permalink raw reply
* [Patch net-next] net: dump more useful information in netdev_rx_csum_fault()
From: Cong Wang @ 2018-11-09 19:43 UTC (permalink / raw)
To: netdev; +Cc: Cong Wang
Currently netdev_rx_csum_fault() only shows a device name,
we need more information about the skb for debugging.
Sample output:
ens3: hw csum failure
dev features: 0x0000000000014b89
skb len=84 data_len=0 gso_size=0 gso_type=0 ip_summed=0 csum=0, csum_complete_sw=0, csum_valid=0
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
---
include/linux/netdevice.h | 5 +++--
net/core/datagram.c | 6 +++---
net/core/dev.c | 10 ++++++++--
net/sunrpc/socklib.c | 2 +-
4 files changed, 15 insertions(+), 8 deletions(-)
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 857f8abf7b91..fabcd9fa6cf7 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -4332,9 +4332,10 @@ static inline bool can_checksum_protocol(netdev_features_t features,
}
#ifdef CONFIG_BUG
-void netdev_rx_csum_fault(struct net_device *dev);
+void netdev_rx_csum_fault(struct net_device *dev, struct sk_buff *skb);
#else
-static inline void netdev_rx_csum_fault(struct net_device *dev)
+static inline void netdev_rx_csum_fault(struct net_device *dev,
+ struct sk_buff *skb)
{
}
#endif
diff --git a/net/core/datagram.c b/net/core/datagram.c
index 57f3a6fcfc1e..d8f4d55cd6c5 100644
--- a/net/core/datagram.c
+++ b/net/core/datagram.c
@@ -736,7 +736,7 @@ __sum16 __skb_checksum_complete_head(struct sk_buff *skb, int len)
if (likely(!sum)) {
if (unlikely(skb->ip_summed == CHECKSUM_COMPLETE) &&
!skb->csum_complete_sw)
- netdev_rx_csum_fault(skb->dev);
+ netdev_rx_csum_fault(skb->dev, skb);
}
if (!skb_shared(skb))
skb->csum_valid = !sum;
@@ -756,7 +756,7 @@ __sum16 __skb_checksum_complete(struct sk_buff *skb)
if (likely(!sum)) {
if (unlikely(skb->ip_summed == CHECKSUM_COMPLETE) &&
!skb->csum_complete_sw)
- netdev_rx_csum_fault(skb->dev);
+ netdev_rx_csum_fault(skb->dev, skb);
}
if (!skb_shared(skb)) {
@@ -810,7 +810,7 @@ int skb_copy_and_csum_datagram_msg(struct sk_buff *skb,
if (unlikely(skb->ip_summed == CHECKSUM_COMPLETE) &&
!skb->csum_complete_sw)
- netdev_rx_csum_fault(NULL);
+ netdev_rx_csum_fault(NULL, skb);
}
return 0;
fault:
diff --git a/net/core/dev.c b/net/core/dev.c
index 0ffcbdd55fa9..2b337df26117 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -3091,10 +3091,16 @@ EXPORT_SYMBOL(__skb_gso_segment);
/* Take action when hardware reception checksum errors are detected. */
#ifdef CONFIG_BUG
-void netdev_rx_csum_fault(struct net_device *dev)
+void netdev_rx_csum_fault(struct net_device *dev, struct sk_buff *skb)
{
if (net_ratelimit()) {
pr_err("%s: hw csum failure\n", dev ? dev->name : "<unknown>");
+ if (dev)
+ pr_err("dev features: %pNF\n", &dev->features);
+ pr_err("skb len=%d data_len=%d gso_size=%d gso_type=%d ip_summed=%d csum=%x, csum_complete_sw=%d, csum_valid=%d\n",
+ skb->len, skb->data_len, skb_shinfo(skb)->gso_size,
+ skb_shinfo(skb)->gso_type, skb->ip_summed, skb->csum,
+ skb->csum_complete_sw, skb->csum_valid);
dump_stack();
}
}
@@ -5779,7 +5785,7 @@ __sum16 __skb_gro_checksum_complete(struct sk_buff *skb)
if (likely(!sum)) {
if (unlikely(skb->ip_summed == CHECKSUM_COMPLETE) &&
!skb->csum_complete_sw)
- netdev_rx_csum_fault(skb->dev);
+ netdev_rx_csum_fault(skb->dev, skb);
}
NAPI_GRO_CB(skb)->csum = wsum;
diff --git a/net/sunrpc/socklib.c b/net/sunrpc/socklib.c
index 9062967575c4..7e55cfc69697 100644
--- a/net/sunrpc/socklib.c
+++ b/net/sunrpc/socklib.c
@@ -175,7 +175,7 @@ int csum_partial_copy_to_xdr(struct xdr_buf *xdr, struct sk_buff *skb)
return -1;
if (unlikely(skb->ip_summed == CHECKSUM_COMPLETE) &&
!skb->csum_complete_sw)
- netdev_rx_csum_fault(skb->dev);
+ netdev_rx_csum_fault(skb->dev, skb);
return 0;
no_checksum:
if (xdr_partial_copy_from_skb(xdr, 0, &desc, xdr_skb_read_bits) < 0)
--
2.19.1
^ permalink raw reply related
* Re: [PATCH bpf-next v2 2/3] bpf: Support socket lookup in CGROUP_SOCK_ADDR progs
From: Martin Lau @ 2018-11-09 19:30 UTC (permalink / raw)
To: Andrey Ignatov
Cc: netdev@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net,
joe@wand.net.nz, Kernel Team
In-Reply-To: <53ea8fee1f9a7c0a862e489799c3fd80b5b0034a.1541789512.git.rdna@fb.com>
On Fri, Nov 09, 2018 at 10:54:01AM -0800, Andrey Ignatov wrote:
> Make bpf_sk_lookup_tcp, bpf_sk_lookup_udp and bpf_sk_release helpers
> available in programs of type BPF_PROG_TYPE_CGROUP_SOCK_ADDR.
>
> Such programs operate on sockets and have access to socket and struct
> sockaddr passed by user to system calls such as sys_bind, sys_connect,
> sys_sendmsg.
>
> It's useful to be able to lookup other sockets from these programs.
> E.g. sys_connect may lookup IP:port endpoint and if there is a server
> socket bound to that endpoint ("server" can be defined by saddr & sport
> being zero), redirect client connection to it by rewriting IP:port in
> sockaddr passed to sys_connect.
Acked-by: Martin KaFai Lau <kafai@fb.com>
^ permalink raw reply
* [PATCH bpf-next v2 1/3] bpf: Fix IPv6 dport byte order in bpf_sk_lookup_udp
From: Andrey Ignatov @ 2018-11-09 18:54 UTC (permalink / raw)
To: netdev; +Cc: Andrey Ignatov, ast, daniel, joe, kafai, kernel-team
In-Reply-To: <cover.1541789512.git.rdna@fb.com>
Lookup functions in sk_lookup have different expectations about byte
order of provided arguments.
Specifically __inet_lookup, __udp4_lib_lookup and __udp6_lib_lookup
expect dport to be in network byte order and do ntohs(dport) internally.
At the same time __inet6_lookup expects dport to be in host byte order
and correspondingly name the argument hnum.
sk_lookup works correctly with __inet_lookup, __udp4_lib_lookup and
__inet6_lookup with regard to dport. But in __udp6_lib_lookup case it
uses host instead of expected network byte order. It makes result
returned by bpf_sk_lookup_udp for IPv6 incorrect.
The patch fixes byte order of dport passed to __udp6_lib_lookup.
Originally sk_lookup properly handled UDPv6, but not TCPv6. 5ef0ae84f02a
fixes TCPv6 but breaks UDPv6.
Fixes: 5ef0ae84f02a ("bpf: Fix IPv6 dport byte-order in bpf_sk_lookup")
Signed-off-by: Andrey Ignatov <rdna@fb.com>
Acked-by: Joe Stringer <joe@wand.net.nz>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
net/core/filter.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/net/core/filter.c b/net/core/filter.c
index 53d50fb75ea1..f4ae933edf61 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -4867,17 +4867,16 @@ static struct sock *sk_lookup(struct net *net, struct bpf_sock_tuple *tuple,
} else {
struct in6_addr *src6 = (struct in6_addr *)&tuple->ipv6.saddr;
struct in6_addr *dst6 = (struct in6_addr *)&tuple->ipv6.daddr;
- u16 hnum = ntohs(tuple->ipv6.dport);
if (proto == IPPROTO_TCP)
sk = __inet6_lookup(net, &tcp_hashinfo, NULL, 0,
src6, tuple->ipv6.sport,
- dst6, hnum,
+ dst6, ntohs(tuple->ipv6.dport),
dif, sdif, &refcounted);
else if (likely(ipv6_bpf_stub))
sk = ipv6_bpf_stub->udp6_lib_lookup(net,
src6, tuple->ipv6.sport,
- dst6, hnum,
+ dst6, tuple->ipv6.dport,
dif, sdif,
&udp_table, NULL);
#endif
--
2.17.1
^ permalink raw reply related
* [PATCH bpf-next v2 3/3] selftest/bpf: Use bpf_sk_lookup_{tcp,udp} in test_sock_addr
From: Andrey Ignatov @ 2018-11-09 18:54 UTC (permalink / raw)
To: netdev; +Cc: Andrey Ignatov, ast, daniel, joe, kafai, kernel-team
In-Reply-To: <cover.1541789512.git.rdna@fb.com>
Use bpf_sk_lookup_tcp, bpf_sk_lookup_udp and bpf_sk_release helpers from
test_sock_addr programs to make sure they're available and can lookup
and release socket properly for IPv4/IPv4, TCP/UDP.
Reading from a few fields of returned struct bpf_sock is also tested.
Signed-off-by: Andrey Ignatov <rdna@fb.com>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
tools/testing/selftests/bpf/connect4_prog.c | 43 ++++++++++++----
tools/testing/selftests/bpf/connect6_prog.c | 56 ++++++++++++++++-----
2 files changed, 78 insertions(+), 21 deletions(-)
diff --git a/tools/testing/selftests/bpf/connect4_prog.c b/tools/testing/selftests/bpf/connect4_prog.c
index 5a88a681d2ab..b8395f3c43e9 100644
--- a/tools/testing/selftests/bpf/connect4_prog.c
+++ b/tools/testing/selftests/bpf/connect4_prog.c
@@ -21,23 +21,48 @@ int _version SEC("version") = 1;
SEC("cgroup/connect4")
int connect_v4_prog(struct bpf_sock_addr *ctx)
{
+ struct bpf_sock_tuple tuple = {};
struct sockaddr_in sa;
+ struct bpf_sock *sk;
+
+ /* Verify that new destination is available. */
+ memset(&tuple.ipv4.saddr, 0, sizeof(tuple.ipv4.saddr));
+ memset(&tuple.ipv4.sport, 0, sizeof(tuple.ipv4.sport));
+
+ tuple.ipv4.daddr = bpf_htonl(DST_REWRITE_IP4);
+ tuple.ipv4.dport = bpf_htons(DST_REWRITE_PORT4);
+
+ if (ctx->type != SOCK_STREAM && ctx->type != SOCK_DGRAM)
+ return 0;
+ else if (ctx->type == SOCK_STREAM)
+ sk = bpf_sk_lookup_tcp(ctx, &tuple, sizeof(tuple.ipv4), 0, 0);
+ else
+ sk = bpf_sk_lookup_udp(ctx, &tuple, sizeof(tuple.ipv4), 0, 0);
+
+ if (!sk)
+ return 0;
+
+ if (sk->src_ip4 != tuple.ipv4.daddr ||
+ sk->src_port != DST_REWRITE_PORT4) {
+ bpf_sk_release(sk);
+ return 0;
+ }
+
+ bpf_sk_release(sk);
/* Rewrite destination. */
ctx->user_ip4 = bpf_htonl(DST_REWRITE_IP4);
ctx->user_port = bpf_htons(DST_REWRITE_PORT4);
- if (ctx->type == SOCK_DGRAM || ctx->type == SOCK_STREAM) {
- ///* Rewrite source. */
- memset(&sa, 0, sizeof(sa));
+ /* Rewrite source. */
+ memset(&sa, 0, sizeof(sa));
- sa.sin_family = AF_INET;
- sa.sin_port = bpf_htons(0);
- sa.sin_addr.s_addr = bpf_htonl(SRC_REWRITE_IP4);
+ sa.sin_family = AF_INET;
+ sa.sin_port = bpf_htons(0);
+ sa.sin_addr.s_addr = bpf_htonl(SRC_REWRITE_IP4);
- if (bpf_bind(ctx, (struct sockaddr *)&sa, sizeof(sa)) != 0)
- return 0;
- }
+ if (bpf_bind(ctx, (struct sockaddr *)&sa, sizeof(sa)) != 0)
+ return 0;
return 1;
}
diff --git a/tools/testing/selftests/bpf/connect6_prog.c b/tools/testing/selftests/bpf/connect6_prog.c
index 8ea3f7d12dee..25f5dc7b7aa0 100644
--- a/tools/testing/selftests/bpf/connect6_prog.c
+++ b/tools/testing/selftests/bpf/connect6_prog.c
@@ -29,7 +29,41 @@ int _version SEC("version") = 1;
SEC("cgroup/connect6")
int connect_v6_prog(struct bpf_sock_addr *ctx)
{
+ struct bpf_sock_tuple tuple = {};
struct sockaddr_in6 sa;
+ struct bpf_sock *sk;
+
+ /* Verify that new destination is available. */
+ memset(&tuple.ipv6.saddr, 0, sizeof(tuple.ipv6.saddr));
+ memset(&tuple.ipv6.sport, 0, sizeof(tuple.ipv6.sport));
+
+ tuple.ipv6.daddr[0] = bpf_htonl(DST_REWRITE_IP6_0);
+ tuple.ipv6.daddr[1] = bpf_htonl(DST_REWRITE_IP6_1);
+ tuple.ipv6.daddr[2] = bpf_htonl(DST_REWRITE_IP6_2);
+ tuple.ipv6.daddr[3] = bpf_htonl(DST_REWRITE_IP6_3);
+
+ tuple.ipv6.dport = bpf_htons(DST_REWRITE_PORT6);
+
+ if (ctx->type != SOCK_STREAM && ctx->type != SOCK_DGRAM)
+ return 0;
+ else if (ctx->type == SOCK_STREAM)
+ sk = bpf_sk_lookup_tcp(ctx, &tuple, sizeof(tuple.ipv6), 0, 0);
+ else
+ sk = bpf_sk_lookup_udp(ctx, &tuple, sizeof(tuple.ipv6), 0, 0);
+
+ if (!sk)
+ return 0;
+
+ if (sk->src_ip6[0] != tuple.ipv6.daddr[0] ||
+ sk->src_ip6[1] != tuple.ipv6.daddr[1] ||
+ sk->src_ip6[2] != tuple.ipv6.daddr[2] ||
+ sk->src_ip6[3] != tuple.ipv6.daddr[3] ||
+ sk->src_port != DST_REWRITE_PORT6) {
+ bpf_sk_release(sk);
+ return 0;
+ }
+
+ bpf_sk_release(sk);
/* Rewrite destination. */
ctx->user_ip6[0] = bpf_htonl(DST_REWRITE_IP6_0);
@@ -39,21 +73,19 @@ int connect_v6_prog(struct bpf_sock_addr *ctx)
ctx->user_port = bpf_htons(DST_REWRITE_PORT6);
- if (ctx->type == SOCK_DGRAM || ctx->type == SOCK_STREAM) {
- /* Rewrite source. */
- memset(&sa, 0, sizeof(sa));
+ /* Rewrite source. */
+ memset(&sa, 0, sizeof(sa));
- sa.sin6_family = AF_INET6;
- sa.sin6_port = bpf_htons(0);
+ sa.sin6_family = AF_INET6;
+ sa.sin6_port = bpf_htons(0);
- sa.sin6_addr.s6_addr32[0] = bpf_htonl(SRC_REWRITE_IP6_0);
- sa.sin6_addr.s6_addr32[1] = bpf_htonl(SRC_REWRITE_IP6_1);
- sa.sin6_addr.s6_addr32[2] = bpf_htonl(SRC_REWRITE_IP6_2);
- sa.sin6_addr.s6_addr32[3] = bpf_htonl(SRC_REWRITE_IP6_3);
+ sa.sin6_addr.s6_addr32[0] = bpf_htonl(SRC_REWRITE_IP6_0);
+ sa.sin6_addr.s6_addr32[1] = bpf_htonl(SRC_REWRITE_IP6_1);
+ sa.sin6_addr.s6_addr32[2] = bpf_htonl(SRC_REWRITE_IP6_2);
+ sa.sin6_addr.s6_addr32[3] = bpf_htonl(SRC_REWRITE_IP6_3);
- if (bpf_bind(ctx, (struct sockaddr *)&sa, sizeof(sa)) != 0)
- return 0;
- }
+ if (bpf_bind(ctx, (struct sockaddr *)&sa, sizeof(sa)) != 0)
+ return 0;
return 1;
}
--
2.17.1
^ permalink raw reply related
* [PATCH bpf-next v2 0/3] bpf: Support socket lookup in CGROUP_SOCK_ADDR progs
From: Andrey Ignatov @ 2018-11-09 18:53 UTC (permalink / raw)
To: netdev; +Cc: Andrey Ignatov, ast, daniel, joe, kafai, kernel-team
This patch set makes bpf_sk_lookup_tcp, bpf_sk_lookup_udp and
bpf_sk_release helpers available in programs of type
BPF_PROG_TYPE_CGROUP_SOCK_ADDR.
Patch 1 is a fix for bpf_sk_lookup_udp that was already merged to bpf
(stable) tree. Here it's prerequisite for patch 3.
Patch 2 is the main patch in the set, it makes the helpers available for
BPF_PROG_TYPE_CGROUP_SOCK_ADDR and provides more details about use-case.
Patch 3 adds selftest for new functionality.
v1->v2:
- remove "Split bpf_sk_lookup" patch since it was already split by:
commit c8123ead13a5 ("bpf: Extend the sk_lookup() helper to XDP
hookpoint.");
- avoid unnecessary bpf_sock_addr_sk_lookup function.
Andrey Ignatov (3):
bpf: Fix IPv6 dport byte order in bpf_sk_lookup_udp
bpf: Support socket lookup in CGROUP_SOCK_ADDR progs
selftest/bpf: Use bpf_sk_lookup_{tcp,udp} in test_sock_addr
net/core/filter.c | 50 ++++++++++++++++--
tools/testing/selftests/bpf/connect4_prog.c | 43 ++++++++++++----
tools/testing/selftests/bpf/connect6_prog.c | 56 ++++++++++++++++-----
3 files changed, 125 insertions(+), 24 deletions(-)
--
2.17.1
^ permalink raw reply
* [PATCH bpf-next v2 2/3] bpf: Support socket lookup in CGROUP_SOCK_ADDR progs
From: Andrey Ignatov @ 2018-11-09 18:54 UTC (permalink / raw)
To: netdev; +Cc: Andrey Ignatov, ast, daniel, joe, kafai, kernel-team
In-Reply-To: <cover.1541789512.git.rdna@fb.com>
Make bpf_sk_lookup_tcp, bpf_sk_lookup_udp and bpf_sk_release helpers
available in programs of type BPF_PROG_TYPE_CGROUP_SOCK_ADDR.
Such programs operate on sockets and have access to socket and struct
sockaddr passed by user to system calls such as sys_bind, sys_connect,
sys_sendmsg.
It's useful to be able to lookup other sockets from these programs.
E.g. sys_connect may lookup IP:port endpoint and if there is a server
socket bound to that endpoint ("server" can be defined by saddr & sport
being zero), redirect client connection to it by rewriting IP:port in
sockaddr passed to sys_connect.
Signed-off-by: Andrey Ignatov <rdna@fb.com>
Acked-by: Alexei Starovoitov <ast@kernel.org>
---
net/core/filter.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 45 insertions(+)
diff --git a/net/core/filter.c b/net/core/filter.c
index f4ae933edf61..f6ca38a7d433 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -5042,6 +5042,43 @@ static const struct bpf_func_proto bpf_xdp_sk_lookup_tcp_proto = {
.arg4_type = ARG_ANYTHING,
.arg5_type = ARG_ANYTHING,
};
+
+BPF_CALL_5(bpf_sock_addr_sk_lookup_tcp, struct bpf_sock_addr_kern *, ctx,
+ struct bpf_sock_tuple *, tuple, u32, len, u64, netns_id, u64, flags)
+{
+ return __bpf_sk_lookup(NULL, tuple, len, sock_net(ctx->sk), 0,
+ IPPROTO_TCP, netns_id, flags);
+}
+
+static const struct bpf_func_proto bpf_sock_addr_sk_lookup_tcp_proto = {
+ .func = bpf_sock_addr_sk_lookup_tcp,
+ .gpl_only = false,
+ .ret_type = RET_PTR_TO_SOCKET_OR_NULL,
+ .arg1_type = ARG_PTR_TO_CTX,
+ .arg2_type = ARG_PTR_TO_MEM,
+ .arg3_type = ARG_CONST_SIZE,
+ .arg4_type = ARG_ANYTHING,
+ .arg5_type = ARG_ANYTHING,
+};
+
+BPF_CALL_5(bpf_sock_addr_sk_lookup_udp, struct bpf_sock_addr_kern *, ctx,
+ struct bpf_sock_tuple *, tuple, u32, len, u64, netns_id, u64, flags)
+{
+ return __bpf_sk_lookup(NULL, tuple, len, sock_net(ctx->sk), 0,
+ IPPROTO_UDP, netns_id, flags);
+}
+
+static const struct bpf_func_proto bpf_sock_addr_sk_lookup_udp_proto = {
+ .func = bpf_sock_addr_sk_lookup_udp,
+ .gpl_only = false,
+ .ret_type = RET_PTR_TO_SOCKET_OR_NULL,
+ .arg1_type = ARG_PTR_TO_CTX,
+ .arg2_type = ARG_PTR_TO_MEM,
+ .arg3_type = ARG_CONST_SIZE,
+ .arg4_type = ARG_ANYTHING,
+ .arg5_type = ARG_ANYTHING,
+};
+
#endif /* CONFIG_INET */
bool bpf_helper_changes_pkt_data(void *func)
@@ -5148,6 +5185,14 @@ sock_addr_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)
return &bpf_get_socket_cookie_sock_addr_proto;
case BPF_FUNC_get_local_storage:
return &bpf_get_local_storage_proto;
+#ifdef CONFIG_INET
+ case BPF_FUNC_sk_lookup_tcp:
+ return &bpf_sock_addr_sk_lookup_tcp_proto;
+ case BPF_FUNC_sk_lookup_udp:
+ return &bpf_sock_addr_sk_lookup_udp_proto;
+ case BPF_FUNC_sk_release:
+ return &bpf_sk_release_proto;
+#endif /* CONFIG_INET */
default:
return bpf_base_func_proto(func_id);
}
--
2.17.1
^ permalink raw reply related
* Re: [PATCH bpf-next] filter: add BPF_ADJ_ROOM_DATA mode to bpf_skb_adjust_room()
From: Martin Lau @ 2018-11-09 18:51 UTC (permalink / raw)
To: Nicolas Dichtel
Cc: ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net,
netdev@vger.kernel.org
In-Reply-To: <20181108151137.3975-1-nicolas.dichtel@6wind.com>
On Thu, Nov 08, 2018 at 04:11:37PM +0100, Nicolas Dichtel wrote:
> This new mode enables to add or remove an l2 header in a programmatic way
> with cls_bpf.
> For example, it enables to play with mpls headers.
>
> Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
> ---
> include/uapi/linux/bpf.h | 3 ++
> net/core/filter.c | 54 ++++++++++++++++++++++++++++++++++
> tools/include/uapi/linux/bpf.h | 3 ++
> 3 files changed, 60 insertions(+)
>
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index 852dc17ab47a..47407fd5162b 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -1467,6 +1467,8 @@ union bpf_attr {
> *
> * * **BPF_ADJ_ROOM_NET**: Adjust room at the network layer
> * (room space is added or removed below the layer 3 header).
> + * * **BPF_ADJ_ROOM_DATA**: Adjust room at the beginning of the
> + * packet (room space is added or removed below skb->data).
> *
> * All values for *flags* are reserved for future usage, and must
> * be left at zero.
> @@ -2408,6 +2410,7 @@ enum bpf_func_id {
> /* Mode for BPF_FUNC_skb_adjust_room helper. */
> enum bpf_adj_room_mode {
> BPF_ADJ_ROOM_NET,
> + BPF_ADJ_ROOM_DATA,
> };
>
> /* Mode for BPF_FUNC_skb_load_bytes_relative helper. */
> diff --git a/net/core/filter.c b/net/core/filter.c
> index e521c5ebc7d1..e699849b269d 100644
> --- a/net/core/filter.c
> +++ b/net/core/filter.c
> @@ -2884,6 +2884,58 @@ static int bpf_skb_adjust_net(struct sk_buff *skb, s32 len_diff)
> return ret;
> }
>
> +static int bpf_skb_data_shrink(struct sk_buff *skb, u32 len)
> +{
> + unsigned short hhlen = skb->dev->header_ops ?
> + skb->dev->hard_header_len : 0;
> + int ret;
> +
> + ret = skb_unclone(skb, GFP_ATOMIC);
> + if (unlikely(ret < 0))
> + return ret;
> +
> + __skb_pull(skb, len);
> + skb_reset_mac_header(skb);
> + skb_reset_network_header(skb);
> + skb->network_header += hhlen;
> + skb_reset_transport_header(skb);
hmm...why transport_header does not need += hhlen here
while network_header does?
> + return 0;
> +}
> +
> +static int bpf_skb_data_grow(struct sk_buff *skb, u32 len)
> +{
> + unsigned short hhlen = skb->dev->header_ops ?
> + skb->dev->hard_header_len : 0;
> + int ret;
> +
> + ret = skb_cow(skb, len);
> + if (unlikely(ret < 0))
> + return ret;
> +
> + skb_push(skb, len);
> + skb_reset_mac_header(skb);
> + return 0;
> +}
> +
> +static int bpf_skb_adjust_data(struct sk_buff *skb, s32 len_diff)
> +{
> + u32 len_diff_abs = abs(len_diff);
> + bool shrink = len_diff < 0;
> + int ret;
> +
> + if (unlikely(len_diff_abs > 0xfffU))
> + return -EFAULT;
> +
> + if (shrink && len_diff_abs >= skb_headlen(skb))
> + return -EFAULT;
> +
> + ret = shrink ? bpf_skb_data_shrink(skb, len_diff_abs) :
> + bpf_skb_data_grow(skb, len_diff_abs);
> +
> + bpf_compute_data_pointers(skb);
> + return ret;
> +}
> +
> BPF_CALL_4(bpf_skb_adjust_room, struct sk_buff *, skb, s32, len_diff,
> u32, mode, u64, flags)
> {
> @@ -2891,6 +2943,8 @@ BPF_CALL_4(bpf_skb_adjust_room, struct sk_buff *, skb, s32, len_diff,
> return -EINVAL;
> if (likely(mode == BPF_ADJ_ROOM_NET))
> return bpf_skb_adjust_net(skb, len_diff);
> + if (likely(mode == BPF_ADJ_ROOM_DATA))
> + return bpf_skb_adjust_data(skb, len_diff);
>
> return -ENOTSUPP;
> }
> diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
> index 852dc17ab47a..47407fd5162b 100644
> --- a/tools/include/uapi/linux/bpf.h
> +++ b/tools/include/uapi/linux/bpf.h
> @@ -1467,6 +1467,8 @@ union bpf_attr {
> *
> * * **BPF_ADJ_ROOM_NET**: Adjust room at the network layer
> * (room space is added or removed below the layer 3 header).
> + * * **BPF_ADJ_ROOM_DATA**: Adjust room at the beginning of the
> + * packet (room space is added or removed below skb->data).
> *
> * All values for *flags* are reserved for future usage, and must
> * be left at zero.
> @@ -2408,6 +2410,7 @@ enum bpf_func_id {
> /* Mode for BPF_FUNC_skb_adjust_room helper. */
> enum bpf_adj_room_mode {
> BPF_ADJ_ROOM_NET,
> + BPF_ADJ_ROOM_DATA,
> };
>
> /* Mode for BPF_FUNC_skb_load_bytes_relative helper. */
> --
> 2.18.0
>
^ permalink raw reply
* Re: [PATCH v3] net: phy: leds: Don't make our own link speed names
From: David Miller @ 2018-11-10 4:14 UTC (permalink / raw)
To: kyle.roeschley; +Cc: andrew, f.fainelli, netdev, linux-kernel
In-Reply-To: <20181109184803.15588-1-kyle.roeschley@ni.com>
From: Kyle Roeschley <kyle.roeschley@ni.com>
Date: Fri, 9 Nov 2018 12:48:03 -0600
> The phy core provides a handy phy_speed_to_str() helper, so use that
> instead of doing our own formatting of the different known link speeds.
> To do this, increase PHY_LED_TRIGGER_SPEED_SUFFIX_SIZE to 11 so we can fit
> 'Unsupported' if necessary.
>
> Signed-off-by: Kyle Roeschley <kyle.roeschley@ni.com>
Applied to net-next.
^ permalink raw reply
* Re: [PATCH] net: mvneta: correct typo
From: David Miller @ 2018-11-10 4:10 UTC (permalink / raw)
To: alexandre.belloni
Cc: thomas.petazzoni, maxime.chevallier, netdev, linux-kernel
In-Reply-To: <20181109163720.17266-1-alexandre.belloni@bootlin.com>
From: Alexandre Belloni <alexandre.belloni@bootlin.com>
Date: Fri, 9 Nov 2018 17:37:20 +0100
> The reserved variable should be named reserved1.
>
> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Applied.
^ permalink raw reply
* Re: bring back IPX and NCPFS, please!
From: Willy Tarreau @ 2018-11-09 18:20 UTC (permalink / raw)
To: Johannes C. Schulz; +Cc: netdev
In-Reply-To: <CAFfL_ozyvDbOL0Z-E6nLkhVF3oDQHFqAHVe1SvuUk2=kHHre7w@mail.gmail.com>
On Fri, Nov 09, 2018 at 06:30:14PM +0100, Johannes C. Schulz wrote:
> Hello Willy, hello Stephen
>
> Thankyou for your reply.
> But I'm not able to maintain or code these modules. I'm just a bloody
> user/webdev.
That's what we've all claimed before taking over something many years
ago you know :-) The most important is time and willingness to try to
do it.
You could first look at the latest kernel supporting those, check if
they still used to work fine in your environment (not everyone has
access to these ones anymore), and if so, then try to copy that code
over newer kernels. Sometimes it will not build with an obvious error
that you'll be able to fix by yourself, sometimes it will be harder
and you'll have to ask for help and/or figure API changes in "git log".
After working many hours on this you'll be much more at ease with this
code and you'll possibly be able to make it work on your kernel version.
This is already a huge step because even if you don't consider it as
being in a mergeable state (too hackish, dirty etc), you have the
option to run it as your own patch for a while.
After this you'll seek some more help about the process needed to get
these merged back and to maintain them as long as you estimate you can
(possibly mark it deprecated and keep it as long as you can). And who
knows, given nothing changes in this area these days, maybe it will be
trivial to maintain this FS for another decade and you'll have learned
something fun and useful.
> It would be really nice if these modules will find a good
> maintainer!
Just think again about the advantages you have over many other people :
- access to the environment
- real use case for the feature
There's nothing wrong with trying and failing multiple times, even giving
up if you find the task too hard. But giving up before trying is quite
sad in your situation.
Cheers,
Willy
^ permalink raw reply
* [PATCH bpf-next] selftests/bpf: Fix uninitialized duration warning
From: Joe Stringer @ 2018-11-09 18:18 UTC (permalink / raw)
To: daniel; +Cc: netdev
Daniel Borkmann reports:
test_progs.c: In function ‘main’:
test_progs.c:81:3: warning: ‘duration’ may be used uninitialized in this function [-Wmaybe-uninitialized]
printf("%s:PASS:%s %d nsec\n", __func__, tag, duration);\
^~~~~~
test_progs.c:1706:8: note: ‘duration’ was declared here
__u32 duration;
^~~~~~~~
Signed-off-by: Joe Stringer <joe@wand.net.nz>
---
I'm actually not able to reproduce this with GCC 7.3 or 8.2, so I'll
rely on review to establish that this patch works as intended.
---
tools/testing/selftests/bpf/test_progs.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c
index 2d3c04f45530..c1e688f61061 100644
--- a/tools/testing/selftests/bpf/test_progs.c
+++ b/tools/testing/selftests/bpf/test_progs.c
@@ -1703,7 +1703,7 @@ static void test_reference_tracking()
const char *file = "./test_sk_lookup_kern.o";
struct bpf_object *obj;
struct bpf_program *prog;
- __u32 duration;
+ __u32 duration = 0;
int err = 0;
obj = bpf_object__open(file);
--
2.17.1
^ permalink raw reply related
* RE: [PATCH net-next 7/8] ixgbe: extend PTP gettime function to read system clock
From: Keller, Jacob E @ 2018-11-09 18:14 UTC (permalink / raw)
To: Miroslav Lichvar, netdev@vger.kernel.org
Cc: Richard Cochran, Kirsher, Jeffrey T
In-Reply-To: <20181109101449.15398-8-mlichvar@redhat.com>
> -----Original Message-----
> From: Miroslav Lichvar [mailto:mlichvar@redhat.com]
> Sent: Friday, November 09, 2018 2:15 AM
> To: netdev@vger.kernel.org
> Cc: Richard Cochran <richardcochran@gmail.com>; Keller, Jacob E
> <jacob.e.keller@intel.com>; Miroslav Lichvar <mlichvar@redhat.com>; Kirsher,
> Jeffrey T <jeffrey.t.kirsher@intel.com>
> Subject: [PATCH net-next 7/8] ixgbe: extend PTP gettime function to read system
> clock
>
> -static int ixgbe_ptp_gettime(struct ptp_clock_info *ptp, struct timespec64 *ts)
> +static int ixgbe_ptp_gettimex(struct ptp_clock_info *ptp,
> + struct timespec64 *ts,
> + struct ptp_system_timestamp *sts)
> {
> struct ixgbe_adapter *adapter =
> container_of(ptp, struct ixgbe_adapter, ptp_caps);
> + struct ixgbe_hw *hw = &adapter->hw;
> unsigned long flags;
> - u64 ns;
> + u64 ns, stamp;
>
> spin_lock_irqsave(&adapter->tmreg_lock, flags);
> - ns = timecounter_read(&adapter->hw_tc);
> +
> + switch (adapter->hw.mac.type) {
> + case ixgbe_mac_X550:
> + case ixgbe_mac_X550EM_x:
> + case ixgbe_mac_x550em_a:
> + /* Upper 32 bits represent billions of cycles, lower 32 bits
> + * represent cycles. However, we use timespec64_to_ns for the
> + * correct math even though the units haven't been corrected
> + * yet.
> + */
> + ptp_read_system_prets(sts);
> + IXGBE_READ_REG(hw, IXGBE_SYSTIMR);
> + ptp_read_system_postts(sts);
> + ts->tv_nsec = IXGBE_READ_REG(hw, IXGBE_SYSTIML);
> + ts->tv_sec = IXGBE_READ_REG(hw, IXGBE_SYSTIMH);
> + stamp = timespec64_to_ns(ts);
> + break;
> + default:
> + ptp_read_system_prets(sts);
> + stamp = IXGBE_READ_REG(hw, IXGBE_SYSTIML);
> + ptp_read_system_postts(sts);
> + stamp |= (u64)IXGBE_READ_REG(hw, IXGBE_SYSTIMH) << 32;
> + break;
> + }
> +
> + ns = timecounter_cyc2time(&adapter->hw_tc, stamp);
> +
At first, I was confused by this entire block of code, but then realized that we can't update the timecounter_read method, so we instead have to break this out so that our calls to ptp_read_system_prets() and ptp_read_system_postts() can be added between the register reads.
Ok, that makes sense.
> spin_unlock_irqrestore(&adapter->tmreg_lock, flags);
>
> *ts = ns_to_timespec64(ns);
> @@ -567,10 +597,14 @@ void ixgbe_ptp_overflow_check(struct ixgbe_adapter
> *adapter)
> {
> bool timeout = time_is_before_jiffies(adapter->last_overflow_check +
> IXGBE_OVERFLOW_PERIOD);
> - struct timespec64 ts;
> + unsigned long flags;
>
> if (timeout) {
> - ixgbe_ptp_gettime(&adapter->ptp_caps, &ts);
> + /* Update the timecounter */
> + spin_lock_irqsave(&adapter->tmreg_lock, flags);
> + timecounter_read(&adapter->hw_tc);
> + spin_unlock_irqrestore(&adapter->tmreg_lock, flags);
> +
This also explains this change where we now have to update the timecounter during the overflow check.
Ok, this makes sense to me.
Thanks,
Jake
^ permalink raw reply
* [PATCH] add an initial version of snmp_counter.rst
From: yupeng @ 2018-11-09 18:13 UTC (permalink / raw)
To: netdev, xiyou.wangcong
The snmp_counter.rst run a set of simple experiments, explains the
meaning of snmp counters depend on the experiments' results. This is
an initial version, only covers a small part of the snmp counters.
Signed-off-by: yupeng <yupeng0921@gmail.com>
---
Documentation/networking/index.rst | 1 +
Documentation/networking/snmp_counter.rst | 963 ++++++++++++++++++++++
2 files changed, 964 insertions(+)
create mode 100644 Documentation/networking/snmp_counter.rst
diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst
index bd89dae8d578..6a47629ef8ed 100644
--- a/Documentation/networking/index.rst
+++ b/Documentation/networking/index.rst
@@ -31,6 +31,7 @@ Contents:
net_failover
alias
bridge
+ snmp_counter
.. only:: subproject
diff --git a/Documentation/networking/snmp_counter.rst b/Documentation/networking/snmp_counter.rst
new file mode 100644
index 000000000000..2939c5acf675
--- /dev/null
+++ b/Documentation/networking/snmp_counter.rst
@@ -0,0 +1,963 @@
+====================
+snmp counter tutorial
+====================
+
+This document explains the meaning of snmp counters. For understanding
+their meanings better, this document doesn't explain the counters one
+by one, but creates a set of experiments, and explains the counters
+depend on the experiments' results. The experiments are on one or two
+virtual machines. Except for the test commands we use in the experiments,
+the virtual machines have no other network traffic. We use the 'nstat'
+command to get the values of snmp counters, before every test, we run
+'nstat -n' to update the history, so the 'nstat' output would only
+show the changes of the snmp counters. For more information about
+nstat, please refer:
+
+http://man7.org/linux/man-pages/man8/nstat.8.html
+
+icmp ping
+========
+
+Run the ping command against the public dns server 8.8.8.8::
+
+ nstatuser@nstat-a:~$ ping 8.8.8.8 -c 1
+ PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
+ 64 bytes from 8.8.8.8: icmp_seq=1 ttl=119 time=17.8 ms
+
+ --- 8.8.8.8 ping statistics ---
+ 1 packets transmitted, 1 received, 0% packet loss, time 0ms
+ rtt min/avg/max/mdev = 17.875/17.875/17.875/0.000 ms
+
+The nstayt result::
+
+ nstatuser@nstat-a:~$ nstat
+ #kernel
+ IpInReceives 1 0.0
+ IpInDelivers 1 0.0
+ IpOutRequests 1 0.0
+ IcmpInMsgs 1 0.0
+ IcmpInEchoReps 1 0.0
+ IcmpOutMsgs 1 0.0
+ IcmpOutEchos 1 0.0
+ IcmpMsgInType0 1 0.0
+ IcmpMsgOutType8 1 0.0
+ IpExtInOctets 84 0.0
+ IpExtOutOctets 84 0.0
+ IpExtInNoECTPkts 1 0.0
+
+The nstat output could be divided into two part: one with the 'Ext'
+keyword, another without the 'Ext' keyword. If the counter name
+doesn't have 'Ext', it is defined by one of snmp rfc, if it has 'Ext',
+it is a kernel extent counter. Below we explain them one by one.
+
+The rfc defined counters
+----------------------
+
+* IpInReceives
+The total number of input datagrams received from interfaces,
+including those received in error.
+
+https://tools.ietf.org/html/rfc1213#page-26
+
+* IpInDelivers
+The total number of input datagrams successfully delivered to IP
+user-protocols (including ICMP).
+
+https://tools.ietf.org/html/rfc1213#page-28
+
+* IpOutRequests
+The total number of IP datagrams which local IP user-protocols
+(including ICMP) supplied to IP in requests for transmission. Note
+that this counter does not include any datagrams counted in
+ipForwDatagrams.
+
+https://tools.ietf.org/html/rfc1213#page-28
+
+* IcmpInMsgs
+The total number of ICMP messages which the entity received. Note
+that this counter includes all those counted by icmpInErrors.
+
+https://tools.ietf.org/html/rfc1213#page-41
+
+* IcmpInEchoReps
+The number of ICMP Echo Reply messages received.
+
+https://tools.ietf.org/html/rfc1213#page-42
+
+* IcmpOutMsgs
+The total number of ICMP messages which this entity attempted to send.
+Note that this counter includes all those counted by icmpOutErrors.
+
+https://tools.ietf.org/html/rfc1213#page-43
+
+* IcmpOutEchos
+The number of ICMP Echo (request) messages sent.
+
+https://tools.ietf.org/html/rfc1213#page-45
+
+IcmpMsgInType0 and IcmpMsgOutType8 are not defined by any snmp related
+RFCs, but their meaning are quite straightforward, they count the
+packet number of specific icmp packet types. We could find the icmp
+types here:
+
+https://www.iana.org/assignments/icmp-parameters/icmp-parameters.xhtml
+
+Type 8 is echo, type 0 is echo reply.
+
+Until now, we can easily explain these items of the nstat: We sent an
+icmp echo request, so IpOutRequests, IcmpOutMsgs, IcmpOutEchos and
+IcmpMsgOutType8 were increased 1. We got icmp echo reply from 8.8.8.8,
+so IpInReceives, IcmpInMsgs, IcmpInEchoReps, IcmpMsgInType0 were
+increased 1. The icmp echo reply was passed to icmp layer via ip
+layer, so IpInDelivers was increased 1.
+
+Please note, these metrics don't aware LRO/GRO, e.g., IpOutRequests
+might count 1 packet, but hardware splits it to 2, and sends them
+separately.
+
+IpExtInOctets and IpExtOutOctets
+------------------------------
+They are linux kernel extensions, no rfc definitions. Please note,
+rfc1213 indeed defines ifInOctets and ifOutOctets, but they
+are different things. The ifInOctets and ifOutOctets are packets
+size which includes the mac layer. But IpExtInOctets and IpExtOutOctets
+are only ip layer sizes.
+
+In our example, an ICMP echo request has four parts:
+* 14 bytes mac header
+* 20 bytes ip header
+* 16 bytes icmp header
+* 48 bytes data (default value of the ping command)
+
+So IpExtInOctets value is 20+16+48=84. The IpExtOutOctets is similar.
+
+IpExtInNoECTPkts
+---------------
+We could find IpExtInNoECTPkts in the nstat output, but kernel provide
+four similar counters, we explain them together, they are:
+* IpExtInNoECTPkts
+* IpExtInECT1Pkts
+* IpExtInECT0Pkts
+* IpExtInCEPkts
+
+They indicate four kinds of ECN IP packets, they are defined here:
+
+https://tools.ietf.org/html/rfc3168#page-6
+
+These 4 counters calculate how many packets received per ECN
+status. They count the real frame number regardless the LRO/GRO. So
+for the same packet, you might find that IpInReceives count 1, but
+IpExtInNoECTPkts counts 2 or more.
+
+additional explain
+-----------------
+The ip layer counters are recorded by the ip layer code in the kernel. I mean, if you send a packet to a lower layer directly, Linux
+kernel won't record it. For example, tcpreplay will open an
+AF_PACKET socket, and send the packet to layer 2, although it could send
+an IP packet, you can't find it from the nstat output. Here is an
+example:
+
+We capture the ping packet by tcpdump::
+
+ nstatuser@nstat-a:~$ sudo tcpdump -w /tmp/ping.pcap dst 8.8.8.8
+
+Then run ping command::
+
+ nstatuser@nstat-a:~$ ping 8.8.8.8 -c 1
+
+Terminate tcpdump by Ctrl-C, and run 'nstat -n' to update the nstat
+history. Then run tcpreplay::
+
+ nstatuser@nstat-a:~$ sudo tcpreplay --intf1=ens3 /tmp/ping.pcap
+ Actual: 1 packets (98 bytes) sent in 0.000278 seconds
+ Rated: 352517.9 Bps, 2.82 Mbps, 3597.12 pps
+ Flows: 1 flows, 3597.12 fps, 1 flow packets, 0 non-flow
+ Statistics for network device: ens3
+ Successful packets: 1
+ Failed packets: 0
+ Truncated packets: 0
+ Retried packets (ENOBUFS): 0
+ Retried packets (EAGAIN): 0
+
+Check the nstat output::
+
+ nstatuser@nstat-a:~$ nstat
+ #kernel
+ IpInReceives 1 0.0
+ IpInDelivers 1 0.0
+ IcmpInMsgs 1 0.0
+ IcmpInEchoReps 1 0.0
+ IcmpMsgInType0 1 0.0
+ IpExtInOctets 84 0.0
+ IpExtInNoECTPkts 1 0.0
+
+We can see, nstat only show the received packet, because the IP layer
+of kernel only know the reply of 8.8.8.8, it doesn't know what
+tcpreplay sent.
+
+At the same time, when you use AF_INET socket, even you use the
+SOCK_RAW option, the IP layer will still try to verify whether the
+packet is an ICMP packet, if it is, kernel will still count it to its
+counters and you can find it in the output of nstat.
+
+tcp 3 way handshake
+==================
+
+On server side, we run::
+
+ nstatuser@nstat-b:~$ nc -lknv 0.0.0.0 9000
+ Listening on [0.0.0.0] (family 0, port 9000)
+
+On client side, we run::
+
+ nstatuser@nstat-a:~$ nc -nv 192.168.122.251 9000
+ Connection to 192.168.122.251 9000 port [tcp/*] succeeded!
+
+The server listened on tcp 9000 port, the client connected to it, they
+completed the 3-way handshake.
+
+On server side, we can find below nstat output::
+
+ nstatuser@nstat-b:~$ nstat | grep -i tcp
+ TcpPassiveOpens 1 0.0
+ TcpInSegs 2 0.0
+ TcpOutSegs 1 0.0
+ TcpExtTCPPureAcks 1 0.0
+
+On client side, we can find below nstat output::
+
+ nstatuser@nstat-a:~$ nstat | grep -i tcp
+ TcpActiveOpens 1 0.0
+ TcpInSegs 1 0.0
+ TcpOutSegs 2 0.0
+
+Except for TcpExtTCPPureAcks, all other counters are defined by rfc1213
+
+* TcpActiveOpens
+The number of times TCP connections have made a direct transition to
+the SYN-SENT state from the CLOSED state.
+
+https://tools.ietf.org/html/rfc1213#page-47
+
+* TcpPassiveOpens
+The number of times TCP connections have made a direct transition to
+the SYN-RCVD state from the LISTEN state.
+
+https://tools.ietf.org/html/rfc1213#page-47
+
+* TcpInSegs
+The total number of segments received, including those received in
+error. This count includes segments received on currently established
+connections.
+
+https://tools.ietf.org/html/rfc1213#page-48
+
+* TcpOutSegs
+The total number of segments sent, including those on current
+connections but excluding those containing only retransmitted octets.
+
+https://tools.ietf.org/html/rfc1213#page-48
+
+
+The TcpExtTCPPureAcks is an extension in linux kernel. When kernel
+receives a TCP packet which set ACK flag and with no data, either
+TcpExtTCPPureAcks or TcpExtTCPHPAcks will increase 1. We will discuss
+it in a later section.
+
+Now we can easily explain the nstat outputs on the server side and client
+side.
+
+When the server received the first syn, it replied a syn+ack, and came into
+SYN-RCVD state, so TcpPassiveOpens increased 1. The server received
+syn, sent syn+ack, received ack, so server sent 1 packet, received 2
+packets, TcpInSegs increased 2, TcpOutSegs increased 1. The last ack
+of the 3-way handshake is a pure ack without data, so
+TcpExtTCPPureAcks increased 1.
+
+When the client sent syn, the client came into the SYN-SENT state, so
+TcpActiveOpens increased 1, client sent syn, received syn+ack, sent
+ack, so client sent 2 packets, received 1 packet, TcpInSegs increased
+1, TcpOutSegs increased 2.
+
+Note: about TcpInSegs and TcpOutSegs, rfc1213 doesn't define the
+behaviors when gso/gro/tso are enabled on the NIC (network interface
+card). On current linux implementation, TcpOutSegs awares gso/tso, but
+TcpInSegs doesn't aware gro. So TcpOutSegs will count the actual
+packet number even only 1 packet is sent via tcp layer. If multiple
+packets arrived at a NIC, and they are merged into 1 packet, TcpInSegs
+will only count 1.
+
+tcp disconnect
+=============
+
+Continue our previous example, on the server side, we have run::
+
+ nstatuser@nstat-b:~$ nc -lknv 0.0.0.0 9000
+ Listening on [0.0.0.0] (family 0, port 9000)
+
+On client side, we have run::
+
+ nstatuser@nstat-a:~$ nc -nv 192.168.122.251 9000
+ Connection to 192.168.122.251 9000 port [tcp/*] succeeded!
+
+Now we type Ctrl-C on the client side, stop the tcp connection between the
+two nc command. Then we check the nstat output.
+
+On server side::
+
+ nstatuser@nstat-b:~$ nstat | grep -i tcp
+ TcpInSegs 2 0.0
+ TcpOutSegs 1 0.0
+ TcpExtTCPPureAcks 1 0.0
+ TcpExtTCPOrigDataSent 1 0.0
+
+On client side::
+
+ nstatuser@nstat-b:~$ nstat | grep -i tcp
+ TcpInSegs 2 0.0
+ TcpOutSegs 1 0.0
+ TcpExtTCPPureAcks 1 0.0
+ TcpExtTCPOrigDataSent 1 0.0
+
+Wait for more than 1 minute, run nstat on client again::
+
+ nstatuser@nstat-a:~$ nstat | grep -i tcp
+ TcpExtTW 1 0.0
+
+Most of the counters are explained in the previous section except
+two: TcpExtTCPOrigDataSent and TcpExtTW. Both of them are linux kernel
+extensions.
+
+TcpExtTW means a tcp connection is closed normally via
+time wait stage, not via tcp reuse process.
+
+About TcpExtTCPOrigDataSent, Below kernel patch has a good explanation:
+
+https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f19c29e3e391a66a273e9afebaf01917245148cd
+
+I pasted it here::
+
+ TCPOrigDataSent: number of outgoing packets with original data
+ (excluding retransmission but including data-in-SYN). This counter is
+ different from TcpOutSegs because TcpOutSegs also tracks pure
+ ACKs. TCPOrigDataSent is more useful to track the TCP retransmission rate.
+
+the effect of gso and gro
+=======================
+
+The Generic Segmentation Offload (GSO) and Generic Receive Offload
+would affect the metrics of the packet in/out on both ip and tcp
+layer. Here is an iperf example. Before the test, run below command to
+make sure both gso and gro are enabled on the NIC::
+
+ $ sudo ethtool -k ens3 | egrep '(generic-segmentation-offload|generic-receive-offload)'
+ generic-segmentation-offload: on
+ generic-receive-offload: on
+
+On server side, run::
+
+ iperf3 -s -p 9000
+
+On client side, run::
+
+ iperf3 -c 192.168.122.251 -p 9000 -t 5 -P 10
+
+The server listened on tcp port 9000, the client connected to the server,
+created 10 threads parallel, run 5 seconds. After the pierf3 stopped, we
+run nstat on both the server and client.
+
+On server side::
+
+ nstatuser@nstat-b:~$ nstat
+ #kernel
+ IpInReceives 36346 0.0
+ IpInDelivers 36346 0.0
+ IpOutRequests 33836 0.0
+ TcpPassiveOpens 11 0.0
+ TcpEstabResets 2 0.0
+ TcpInSegs 36346 0.0
+ TcpOutSegs 33836 0.0
+ TcpOutRsts 20 0.0
+ TcpExtDelayedACKs 26 0.0
+ TcpExtTCPHPHits 32120 0.0
+ TcpExtTCPPureAcks 16 0.0
+ TcpExtTCPHPAcks 5 0.0
+ TcpExtTCPAbortOnData 5 0.0
+ TcpExtTCPAbortOnClose 2 0.0
+ TcpExtTCPRcvCoalesce 7306 0.0
+ TcpExtTCPOFOQueue 1354 0.0
+ TcpExtTCPOrigDataSent 15 0.0
+ IpExtInOctets 311732432 0.0
+ IpExtOutOctets 1785119 0.0
+ IpExtInNoECTPkts 214032 0.0
+
+Client side::
+
+ nstatuser@nstat-a:~$ nstat
+ #kernel
+ IpInReceives 33836 0.0
+ IpInDelivers 33836 0.0
+ IpOutRequests 43786 0.0
+ TcpActiveOpens 11 0.0
+ TcpEstabResets 10 0.0
+ TcpInSegs 33836 0.0
+ TcpOutSegs 214072 0.0
+ TcpRetransSegs 3876 0.0
+ TcpExtDelayedACKs 7 0.0
+ TcpExtTCPHPHits 5 0.0
+ TcpExtTCPPureAcks 2719 0.0
+ TcpExtTCPHPAcks 31071 0.0
+ TcpExtTCPSackRecovery 607 0.0
+ TcpExtTCPSACKReorder 61 0.0
+ TcpExtTCPLostRetransmit 90 0.0
+ TcpExtTCPFastRetrans 3806 0.0
+ TcpExtTCPSlowStartRetrans 62 0.0
+ TcpExtTCPLossProbes 38 0.0
+ TcpExtTCPSackRecoveryFail 8 0.0
+ TcpExtTCPSackShifted 203 0.0
+ TcpExtTCPSackMerged 778 0.0
+ TcpExtTCPSackShiftFallback 700 0.0
+ TcpExtTCPSpuriousRtxHostQueues 4 0.0
+ TcpExtTCPAutoCorking 14 0.0
+ TcpExtTCPOrigDataSent 214038 0.0
+ TcpExtTCPHystartTrainDetect 8 0.0
+ TcpExtTCPHystartTrainCwnd 172 0.0
+ IpExtInOctets 1785227 0.0
+ IpExtOutOctets 317789680 0.0
+ IpExtInNoECTPkts 33836 0.0
+
+The TcpOutSegs and IpOutRequests on the server are 33836, exactly the
+same as IpExtInNoECTPkts, IpInReceives, IpInDelivers and TcpInSegs on
+the client side. During iperf3 test, the server only reply very short
+packets, so gso and gro has no effect on the server's reply.
+
+On the client side, TcpOutSegs is 214072, IpOutRequests is 43786, the
+tcp layer packet out is larger than ip layer packet out, because
+TcpOutSegs count the packet number after gso, but IpOutRequests
+doesn't. On the server side, IpExtInNoECTPkts is 214032, this number
+is smaller a little than the TcpOutSegs on the client side (214072), it
+might cause by the packet loss. The IpInReceives, IpInDelivers and
+TcpInSegs are obviously smaller than the TcpOutSegs on the client side,
+because these counters calculate the packet after gro.
+
+tcp counters in established state
+================================
+
+Run nc on server::
+
+ nstatuser@nstat-b:~$ nc -lkv 0.0.0.0 9000
+ Listening on [0.0.0.0] (family 0, port 9000)
+
+Run nc on client:
+
+ nstatuser@nstat-a:~$ nc -v nstat-b 9000
+ Connection to nstat-b 9000 port [tcp/*] succeeded!
+
+Input a string in the nc client ('hello' in our example):
+
+ nstatuser@nstat-a:~$ nc -v nstat-b 9000
+ Connection to nstat-b 9000 port [tcp/*] succeeded!
+ hello
+
+The client side nstat output:
+
+ nstatuser@nstat-a:~$ nstat
+ #kernel
+ IpInReceives 1 0.0
+ IpInDelivers 1 0.0
+ IpOutRequests 1 0.0
+ TcpInSegs 1 0.0
+ TcpOutSegs 1 0.0
+ TcpExtTCPPureAcks 1 0.0
+ TcpExtTCPOrigDataSent 1 0.0
+ IpExtInOctets 52 0.0
+ IpExtOutOctets 58 0.0
+ IpExtInNoECTPkts 1 0.0
+
+The server side nstat output:
+
+ nstatuser@nstat-b:~$ nstat
+ #kernel
+ IpInReceives 1 0.0
+ IpInDelivers 1 0.0
+ IpOutRequests 1 0.0
+ TcpInSegs 1 0.0
+ TcpOutSegs 1 0.0
+ IpExtInOctets 58 0.0
+ IpExtOutOctets 52 0.0
+ IpExtInNoECTPkts 1 0.0
+
+Input a string in nc client side again ('world' in our exmaple):
+
+ nstatuser@nstat-a:~$ nc -v nstat-b 9000
+ Connection to nstat-b 9000 port [tcp/*] succeeded!
+ hello
+ world
+
+Client side nstat output:
+
+ nstatuser@nstat-a:~$ nstat
+ #kernel
+ IpInReceives 1 0.0
+ IpInDelivers 1 0.0
+ IpOutRequests 1 0.0
+ TcpInSegs 1 0.0
+ TcpOutSegs 1 0.0
+ TcpExtTCPHPAcks 1 0.0
+ TcpExtTCPOrigDataSent 1 0.0
+ IpExtInOctets 52 0.0
+ IpExtOutOctets 58 0.0
+ IpExtInNoECTPkts 1 0.0
+
+
+Server side nstat output:
+
+ nstatuser@nstat-b:~$ nstat
+ #kernel
+ IpInReceives 1 0.0
+ IpInDelivers 1 0.0
+ IpOutRequests 1 0.0
+ TcpInSegs 1 0.0
+ TcpOutSegs 1 0.0
+ TcpExtTCPHPHits 1 0.0
+ IpExtInOctets 58 0.0
+ IpExtOutOctets 52 0.0
+ IpExtInNoECTPkts 1 0.0
+
+Compare the first client side output and the second client side
+output, we could find one difference: the first one had a
+'TcpExtTCPPureAcks', but the second one had a
+'TcpExtTCPHPAcks'. The first server side output and the second server
+side output had a difference too: the second server side output had a
+TcpExtTCPHPHits, but the first server side output didn't have it. The
+network traffic patterns were exactly the same: the client sent a packet to the server, the server replied an ack. But kernel handled them in different
+ways. When kernel receives a tpc packet in the established status,
+kernel has two paths to handle the packet, one is fast path, another
+is slow path. The comment in kernel code provides a good explanation of
+them, I paste them below:
+
+ It is split into a fast path and a slow path. The fast path is
+ disabled when:
+ - A zero window was announced from us - zero window probing
+ is only handled properly on the slow path.
+ - Out of order segments arrived.
+ - Urgent data is expected.
+ - There is no buffer space left
+ - Unexpected TCP flags/window values/header lengths are received
+ (detected by checking the TCP header against pred_flags)
+ - Data is sent in both directions. The fast path only supports pure senders
+ or pure receivers (this means either the sequence number or the ack
+ value must stay constant)
+ - Unexpected TCP option.
+
+Kernel will try to use fast path unless any of the above conditions
+are satisfied. If the packets are out of order, kernel will handle
+them in slow path, which means the performance might be not very
+good. Kernel would also come into slow path if the "Delayed ack" is
+used, because when using "Delayed ack", the data is sent in both
+directions. When the tcp window scale option is not used, kernel will
+try to enable fast path immediately when the connection comes into the established
+state, but if the tcp window scale option is used, kernel will disable
+the fast path at first, and try to enable it after kerenl receives
+packets. We could use the 'ss' command to verify whether the window
+scale option is used. e.g. run below command on either server or
+client:
+
+ nstatuser@nstat-a:~$ ss -o state established -i '( dport = :9000 or sport = :9000 )
+ Netid Recv-Q Send-Q Local Address:Port Peer Address:Port
+ tcp 0 0 192.168.122.250:40654 192.168.122.251:9000
+ ts sack cubic wscale:7,7 rto:204 rtt:0.98/0.49 mss:1448 pmtu:1500 rcvmss:536 advmss:1448 cwnd:10 bytes_acked:1 segs_out:2 segs_in:1 send 118.2Mbps lastsnd:46572 lastrcv:46572 lastack:46572 pacing_rate 236.4Mbps rcv_space:29200 rcv_ssthresh:29200 minrtt:0.98
+
+The 'wscale:7,7' means both server and client set the window scale
+option to 7. Now we could explain the nstat output in our test:
+
+In the first nstat output of client side, the client sent a packet, server
+reply an ack, when kernel handled this ack, the fast path was not
+enabled, so the ack was counted into 'TcpExtTCPPureAcks'.
+In the second nstat output of client side, the client sent a packet again,
+and received another ack from the server, this time, the fast path is
+enabled, and the ack was qualified for fast path, so it was handled by
+the fast path, so this ack was counted into TcpExtTCPHPAcks.
+In the first nstat output of server side, the fast path was not enabled,
+so there was no 'TcpExtTCPHPHits'.
+In the second nstat output of server side, the fast path was enabled,
+and the packet received from client qualified for fast path, so it
+was counted into 'TcpExtTCPHPHits'.
+
+tcp abort
+========
+
+Some counters indicate the reaons why tcp layer want to send a rst,
+they are:
+* TcpExtTCPAbortOnData
+* TcpExtTCPAbortOnClose
+* TcpExtTCPAbortOnMemory
+* TcpExtTCPAbortOnTimeout
+* TcpExtTCPAbortOnLinger
+* TcpExtTCPAbortFailed
+
+TcpExtTCPAbortOnData
+-------------------
+
+It means tcp layer has data in flight, but need to close the
+connection. So tcp layer sends a rst to the other side, indicate the
+connection is not closed very graceful. An easy way to increase this
+counter is using the SO_LINGER option. Please refer to the SO_LINGER
+section of the socket man page:
+
+http://man7.org/linux/man-pages/man7/socket.7.html).
+
+By default, when an application closes a connection, the close function
+will return immediately and kernel will try to send the in-flight data
+async. If you use the SO_LINGER option, set l_onoff to 1, and l_linger
+to a positive number, the close function won't return immediately, but
+wait for the in-flight data are acked by the other side, the max wait
+time is l_linger seconds. If set l_onoff to 1 and set l_linger to 0,
+when the application closes a connection, kernel will send an rst
+immediately, and increase the TcpExtTCPAbortOnData counter.
+
+We run nc on the server side::
+
+ nstatuser@nstat-b:~$ nc -lkv 0.0.0.0 9000
+ Listening on [0.0.0.0] (family 0, port 9000)
+
+Run below python code on the client side::
+
+ import socket
+ import struct
+
+ server = 'nstat-b' # server address
+ port = 9000
+
+ s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ s.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, struct.pack('ii', 1, 0))
+ s.connect((server, port))
+ s.close()
+
+On client side, we could see TcpExtTCPAbortOnData increased::
+
+ nstatuser@nstat-a:~$ nstat | grep -i abort
+ TcpExtTCPAbortOnData 1 0.0
+
+If we capture packet by tcpdump, we could see the client send rst
+instead of fin.
+
+
+TcpExtTCPAbortOnClose
+--------------------
+
+This counter means the tcp layer has unread data when an application
+want to close a connection.
+
+On the server side, we run below python script:
+
+ import socket
+ import time
+
+ port = 9000
+
+ s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ s.bind(('0.0.0.0', port))
+ s.listen(1)
+ sock, addr = s.accept()
+ while True:
+ time.sleep(9999999)
+
+This python script listen on 9000 port, but doesn't read anything from
+the connection.
+
+On the client side, we send the string "hello" by nc:
+
+ nstatuser@nstat-a:~$ echo "hello" | nc nstat-b 9000
+
+Then, we come back to the server side, the server has received the "hello"
+packet, and tcp layer has acked this packet, but the application didn't
+read it yet. We type Ctrl-C to terminate the server script. Then we
+could find TcpExtTCPAbortOnClose increased 1 on the server side:
+
+ nstatuser@nstat-b:~$ nstat | grep -i abort
+ TcpExtTCPAbortOnClose 1 0.0
+
+If we run tcpdump on the server side, we could find the server sent a
+rst after we type Ctrl-C.
+
+TcpExtTCPAbortOnMemory
+--------------------
+
+When an application closes a tcp connection, kernel still need to track
+the connection, let it complete the tcp disconnect process. E.g. an
+app calls the close method of a socket, kernel sends fin to the other
+side of the connection, then the app has no relationship with the
+socket any more, but kernel need to keep the socket, this socket
+becomes an orphan socket, kernel waits for the reply of the other side,
+and would come to the TIME_WAIT state finally. When kernel has no
+enough memory to keep the orphan socket, kernel would send an rst to
+the other side, and delete the socket, in such situation, kernel will
+increase 1 to the TcpExtTCPAbortOnMemory. Two conditions would trigger
+TcpExtTCPAbortOnMemory:
+
+* the memory used by tcp protocol is higher than the third value of
+the tcp_mem. Please refer the tcp_mem section in the tcp man page:
+
+http://man7.org/linux/man-pages/man7/tcp.7.html
+
+* the orphan socket count is higher than net.ipv4.tcp_max_orphans
+
+Below is an example which let the orphan socket count be higher than
+net.ipv4.tcp_max_orphans.
+
+Change tcp_max_orphans to a smaller value on client::
+
+ sudo bash -c "echo 10 > /proc/sys/net/ipv4/tcp_max_orphans"
+
+Client code (create 64 connection to server)::
+
+ nstatuser@nstat-a:~$ cat client_orphan.py
+ import socket
+ import time
+
+ server = 'nstat-b' # server address
+ port = 9000
+
+ count = 64
+
+ connection_list = []
+
+ for i in range(64):
+ s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ s.connect((server, port))
+ connection_list.append(s)
+ print("connection_count: %d" % len(connection_list))
+
+ while True:
+ time.sleep(99999)
+
+Server code (accept 64 connection from client)::
+
+ nstatuser@nstat-b:~$ cat server_orphan.py
+ import socket
+ import time
+
+ port = 9000
+ count = 64
+
+ s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ s.bind(('0.0.0.0', port))
+ s.listen(count)
+ connection_list = []
+ while True:
+ sock, addr = s.accept()
+ connection_list.append((sock, addr))
+ print("connection_count: %d" % len(connection_list))
+
+Run the python scripts on server and client.
+
+On server::
+
+ python3 server_orphan.py
+
+On client::
+
+ python3 client_orphan.py
+
+Run iptables on server::
+
+ sudo iptables -A INPUT -i ens3 -p tcp --destination-port 9000 -j DROP
+
+Type Ctrl-C on client, stop client_orphan.py.
+
+Check TcpExtTCPAbortOnMemory on client::
+
+ nstatuser@nstat-a:~$ nstat | grep -i abort
+ TcpExtTCPAbortOnMemory 54 0.0
+
+Check orphane socket count on client::
+
+ nstatuser@nstat-a:~$ ss -s
+ Total: 131 (kernel 0)
+ TCP: 14 (estab 1, closed 0, orphaned 10, synrecv 0, timewait 0/0), ports 0
+
+ Transport Total IP IPv6
+ * 0 - -
+ RAW 1 0 1
+ UDP 1 1 0
+ TCP 14 13 1
+ INET 16 14 2
+ FRAG 0 0 0
+
+The explanation of the test: after run server_orphan.py and
+client_orphan.py, we set up 64 connections between server and
+client. Run the iptables command, the server will drop all packets from
+the client, type Ctrl-C on client_orphan.py, the system of the client
+would try to close these connections, and before they are closed
+gracefully, these connections became orphan sockets. As the iptables
+of the server blocked packets from the client, the server won't receive fin
+from the client, so all connection on clients would be stuck on FIN_WAIT_1
+stage, so they will keep as orphan sockets until timeout. We have echo
+10 to /proc/sys/net/ipv4/tcp_max_orphans, so the client system would
+only keep 10 orphan sockets, for all other orphan sockets, the client
+system sent rst for them and delete them. We have 64 connections, so
+the 'ss -s' command shows the system has 10 orphan sockets, and the
+value of TcpExtTCPAbortOnMemory was 54.
+
+An additional explanation about orphan socket count: You could find the
+exactly orphan socket count by the 'ss -s' command, but when kernel
+decide whither increases TcpExtTCPAbortOnMemory and sends rst, kernel
+doesn't always check the exactly orphan socket count. For increasing
+performance, kernel checks an approximate count firstly, if the
+approximate count is more than tcp_max_orphans, kernel checks the
+exact count again. So if the approximate count is less than
+tcp_max_orphans, but exactly count is more than tcp_max_orphans, you
+would find TcpExtTCPAbortOnMemory is not increased at all. If
+tcp_max_orphans is large enough, it won't occur, but if you decrease
+tcp_max_orphans to a small value like our test, you might find this
+issue. So in our test, the client set up 64 connections although the
+tcp_max_orphans is 10. If the client only set up 11 connections, we
+can't find the change of TcpExtTCPAbortOnMemory.
+
+TcpExtTCPAbortOnTimeout
+----------------------
+This counter will increase when any of the tcp timers expire. In this
+situation, kernel won't send rst, just give up the connection.
+Continue the previous test, we wait for several minutes, because the
+iptables on the server blocked the traffic, the server wouldn't receive
+fin, and all the client's orphan sockets would timeout on the
+FIN_WAIT_1 state finally. So we wait for a few minutes, we could find
+10 timeout on the client::
+
+ nstatuser@nstat-a:~$ nstat | grep -i abort
+ TcpExtTCPAbortOnTimeout 10 0.0
+
+TcpExtTCPAbortOnLinger
+---------------------
+When a tcp connection comes into FIN_WAIT_2 state, instead of waiting
+for the fin packet from the other side, kernel could send a rst and
+delete the socket immediately. This is not the default behavior of
+linux kernel tcp stack, but after configuring socket option, you could
+let kernel follow this behavior. Below is an example.
+
+The server side code::
+
+ nstatuser@nstat-b:~$ cat server_linger.py
+ import socket
+ import time
+
+ port = 9000
+
+ s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ s.bind(('0.0.0.0', port))
+ s.listen(1)
+ sock, addr = s.accept()
+ while True:
+ time.sleep(9999999)
+
+The client side code::
+
+ nstatuser@nstat-a:~$ cat client_linger.py
+ import socket
+ import struct
+
+ server = 'nstat-b' # server address
+ port = 9000
+
+ s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ s.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, struct.pack('ii', 1, 10))
+ s.setsockopt(socket.SOL_TCP, socket.TCP_LINGER2, struct.pack('i', -1))
+ s.connect((server, port))
+ s.close()
+
+Run server_linger.py on server::
+
+ nstatuser@nstat-b:~$ python3 server_linger.py
+
+Run client_linger.py on client::
+
+ nstatuser@nstat-a:~$ python3 client_linger.py
+
+After run client_linger.py, check the output of nstat::
+
+ nstatuser@nstat-a:~$ nstat | grep -i abort
+ TcpExtTCPAbortOnLinger 1 0.0
+
+TcpExtTCPAbortFailed
+-------------------
+The kernel tcp layer will send rst if the RFC 2525 2.17 section is satisfied:
+
+https://tools.ietf.org/html/rfc2525#page-50
+
+If an internal error occurs during this process, TcpExtTCPAbortFailed
+will be increased.
+
+TcpExtListenOverflows and TcpExtListenDrops
+========================================
+When kernel receive a syn from a client, and if the tcp accept queue
+is full, kernel will drop the syn and add 1 to TcpExtListenOverflows.
+At the same time kernel will also add 1 to TcpExtListenDrops. When
+a tcp socket is in LISTEN state, and kernel need to drop a packet,
+kernel would always add 1 to TcpExtListenDrops. So increase
+TcpExtListenOverflows would let TcpExtListenDrops increasing at the
+same time, but TcpExtListenDrops would also increase without
+TcpExtListenOverflows increasing, e.g. a memory allocation fail would
+also let TcpExtListenDrops increase.
+
+Note: The above explain bases on kernel 4.15 or above version, on an
+old kernel, the tcp stack has different behavior when tcp accept queue
+is full. On the old kernel, tcp stack won't drop the syn, it would
+complete the 3-way handshake, but as the accept queue is full, tcp
+stack will keep the socket in the tcp half-open queue. As it is in the
+half open queue, tcp stack will send syn+ack on an exponential backoff
+timer, after client replies ack, tcp stack checks whether the accept
+queue is still full, if it is not full, move the socket to accept
+queue, if it is full, keeps the socket in the half-open queue, at next
+time client replies ack, this socket will get another chance to move
+to the accept queue.
+
+Here is an example:
+
+On server, run the nc command, listen on port 9000::
+
+ nstatuser@nstat-b:~$ nc -lkv 0.0.0.0 9000
+ Listening on [0.0.0.0] (family 0, port 9000)
+
+On client, run 3 nc commands in different terminals::
+
+ nstatuser@nstat-a:~$ nc -v nstat-b 9000
+ Connection to nstat-b 9000 port [tcp/*] succeeded!
+
+The nc command only accepts 1 connection, and the accept queue length
+is 1. On current linux implementation, set queue length to n means the
+actual queue length is n+1. Now we create 3 connections, 1 is accepted
+by nc, 2 in accepted queue, so the accept queue is full.
+
+Before running the 4th nc, we clean the nstat history on the server:
+
+ nstatuser@nstat-b:~$ nstat -n
+
+Run the 4th nc on the client:
+
+ nstatuser@nstat-a:~$ nc -v nstat-b 9000
+
+If the nc server is running on kernel 4.15 or higher version, you
+won't see the "Connection to ... succeeded!" string, because kernel
+will drop the syn if the accept queue is full. If the nc client is running
+on an old kernel, you could see that the connection is succeeded,
+because kernel would complete the 3-way handshake and keep the socket
+on the half-open queue.
+
+Our test is on kernel 4.15, run nstat on the server:
+
+ nstatuser@nstat-b:~$ nstat
+ #kernel
+ IpInReceives 4 0.0
+ IpInDelivers 4 0.0
+ TcpInSegs 4 0.0
+ TcpExtListenOverflows 4 0.0
+ TcpExtListenDrops 4 0.0
+ IpExtInOctets 240 0.0
+ IpExtInNoECTPkts 4 0.0
+
+We can see both TcpExtListenOverflows and TcpExtListenDrops are 4. If
+the time between the 4th nc and the nstat is longer, the value of
+TcpExtListenOverflows and TcpExtListenDrops will be larger, because
+the syn of the 4th nc is dropped, it keeps retrying.
+
--
2.17.1
^ permalink raw reply related
* RE: [PATCH net-next 0/8] More accurate PHC<->system clock synchronization
From: Keller, Jacob E @ 2018-11-09 18:12 UTC (permalink / raw)
To: Miroslav Lichvar, netdev@vger.kernel.org
Cc: Richard Cochran, Marcelo Tosatti, Kirsher, Jeffrey T,
Michael Chan
In-Reply-To: <20181109101449.15398-1-mlichvar@redhat.com>
> -----Original Message-----
> From: Miroslav Lichvar [mailto:mlichvar@redhat.com]
> Sent: Friday, November 09, 2018 2:15 AM
> To: netdev@vger.kernel.org
> Cc: Richard Cochran <richardcochran@gmail.com>; Keller, Jacob E
> <jacob.e.keller@intel.com>; Miroslav Lichvar <mlichvar@redhat.com>; Marcelo
> Tosatti <mtosatti@redhat.com>; Kirsher, Jeffrey T <jeffrey.t.kirsher@intel.com>;
> Michael Chan <michael.chan@broadcom.com>
> Subject: [PATCH net-next 0/8] More accurate PHC<->system clock synchronization
>
> RFC->v1:
> - added new patches
> - separated PHC timestamp from ptp_system_timestamp
> - fixed memory leak in PTP_SYS_OFFSET_EXTENDED
> - changed PTP_SYS_OFFSET_EXTENDED to work with array of arrays
> - fixed PTP_SYS_OFFSET_EXTENDED to break correctly from loop
> - fixed timecounter updates in drivers
> - split gettimex in igb driver
> - fixed ptp_read_* functions to be available without
> CONFIG_PTP_1588_CLOCK
>
> This series enables a more accurate synchronization between PTP hardware
> clocks and the system clock.
Thanks for doing this, Miroslav!
>
> The first two patches are minor cleanup/bug fixes.
>
> The third patch adds an extended version of the PTP_SYS_OFFSET ioctl,
> which returns three timestamps for each measurement. The idea is to
> shorten the interval between the system timestamps to contain just the
> reading of the lowest register of the PHC in order to reduce the error
> in the measured offset and get a smaller upper bound on the maximum
> error.
>
> The fourth patch deprecates the original gettime function.
>
> The remaining patches update the gettime function in order to support
> the new ioctl in the e1000e, igb, ixgbe, and tg3 drivers.
>
> Tests with few different NICs in different machines show that:
> - with an I219 (e1000e) the measured delay was reduced from 2500 to 1300
> ns and the error in the measured offset, when compared to the cross
> timestamping supported by the driver, was reduced by a factor of 5
> - with an I210 (igb) the delay was reduced from 5100 to 1700 ns
> - with an I350 (igb) the delay was reduced from 2300 to 750 ns
> - with an X550 (ixgbe) the delay was reduced from 1950 to 650 ns
> - with a BCM5720 (tg3) the delay was reduced from 2400 to 1200 ns
>
Impressive results!
For the main portions and the Intel driver changes this is
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Regards,
Jake
>
> Miroslav Lichvar (8):
> ptp: reorder declarations in ptp_ioctl()
> ptp: check gettime64 return code in PTP_SYS_OFFSET ioctl
> ptp: add PTP_SYS_OFFSET_EXTENDED ioctl
> ptp: deprecate gettime64() in favor of gettimex64()
> e1000e: extend PTP gettime function to read system clock
> igb: extend PTP gettime function to read system clock
> ixgbe: extend PTP gettime function to read system clock
> tg3: extend PTP gettime function to read system clock
>
> drivers/net/ethernet/broadcom/tg3.c | 19 ++++--
> drivers/net/ethernet/intel/e1000e/e1000.h | 3 +
> drivers/net/ethernet/intel/e1000e/netdev.c | 42 ++++++++++---
> drivers/net/ethernet/intel/e1000e/ptp.c | 16 +++--
> drivers/net/ethernet/intel/igb/igb_ptp.c | 65 +++++++++++++++++---
> drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c | 54 +++++++++++++---
> drivers/ptp/ptp_chardev.c | 55 ++++++++++++++---
> drivers/ptp/ptp_clock.c | 5 +-
> include/linux/ptp_clock_kernel.h | 33 ++++++++++
> include/uapi/linux/ptp_clock.h | 12 ++++
> 10 files changed, 253 insertions(+), 51 deletions(-)
>
> --
> 2.17.2
^ permalink raw reply
* [PATCH net-next 3/3] net: phy: improve and inline phy_change
From: Heiner Kallweit @ 2018-11-09 17:58 UTC (permalink / raw)
To: Andrew Lunn, Florian Fainelli, David Miller; +Cc: netdev@vger.kernel.org
In-Reply-To: <f4d8c0ea-07fe-d257-f392-6ddb7b686593@gmail.com>
Now that phy_mac_interrupt() doesn't call phy_change() any longer it's
called from phy_interrupt() only. Therefore phy_interrupt_is_valid()
returns true always and the check can be removed.
In case of PHY_HALTED phy_interrupt() bails out immediately,
therefore the second check for PHY_HALTED including the call to
phy_disable_interrupts() can be removed.
Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
---
drivers/net/phy/phy.c | 47 ++++++++++++++-----------------------------
1 file changed, 15 insertions(+), 32 deletions(-)
diff --git a/drivers/net/phy/phy.c b/drivers/net/phy/phy.c
index ce1e8130a..083977d2f 100644
--- a/drivers/net/phy/phy.c
+++ b/drivers/net/phy/phy.c
@@ -722,41 +722,12 @@ static int phy_disable_interrupts(struct phy_device *phydev)
return phy_clear_interrupt(phydev);
}
-/**
- * phy_change - Called by the phy_interrupt to handle PHY changes
- * @phydev: phy_device struct that interrupted
- */
-static irqreturn_t phy_change(struct phy_device *phydev)
-{
- if (phy_interrupt_is_valid(phydev)) {
- if (phydev->drv->did_interrupt &&
- !phydev->drv->did_interrupt(phydev))
- return IRQ_NONE;
-
- if (phydev->state == PHY_HALTED)
- if (phy_disable_interrupts(phydev))
- goto phy_err;
- }
-
- /* reschedule state queue work to run as soon as possible */
- phy_trigger_machine(phydev);
-
- if (phy_interrupt_is_valid(phydev) && phy_clear_interrupt(phydev))
- goto phy_err;
- return IRQ_HANDLED;
-
-phy_err:
- phy_error(phydev);
- return IRQ_NONE;
-}
-
/**
* phy_interrupt - PHY interrupt handler
* @irq: interrupt line
* @phy_dat: phy_device pointer
*
- * Description: When a PHY interrupt occurs, the handler disables
- * interrupts, and uses phy_change to handle the interrupt.
+ * Description: Handle PHY interrupt
*/
static irqreturn_t phy_interrupt(int irq, void *phy_dat)
{
@@ -765,7 +736,19 @@ static irqreturn_t phy_interrupt(int irq, void *phy_dat)
if (PHY_HALTED == phydev->state)
return IRQ_NONE; /* It can't be ours. */
- return phy_change(phydev);
+ if (phydev->drv->did_interrupt && !phydev->drv->did_interrupt(phydev))
+ return IRQ_NONE;
+
+ /* reschedule state queue work to run as soon as possible */
+ phy_trigger_machine(phydev);
+
+ if (phy_clear_interrupt(phydev))
+ goto phy_err;
+ return IRQ_HANDLED;
+
+phy_err:
+ phy_error(phydev);
+ return IRQ_NONE;
}
/**
@@ -846,7 +829,7 @@ void phy_stop(struct phy_device *phydev)
phy_state_machine(&phydev->state_queue.work);
/* Cannot call flush_scheduled_work() here as desired because
- * of rtnl_lock(), but PHY_HALTED shall guarantee phy_change()
+ * of rtnl_lock(), but PHY_HALTED shall guarantee irq handler
* will not reenable interrupts.
*/
}
--
2.19.1
^ permalink raw reply related
* [PATCH net-next 2/3] net: phy: simplify phy_mac_interrupt and related functions
From: Heiner Kallweit @ 2018-11-09 17:56 UTC (permalink / raw)
To: Andrew Lunn, Florian Fainelli, David Miller; +Cc: netdev@vger.kernel.org
In-Reply-To: <f4d8c0ea-07fe-d257-f392-6ddb7b686593@gmail.com>
When using phy_mac_interrupt() the irq number is set to
PHY_IGNORE_INTERRUPT, therefore phy_interrupt_is_valid() returns false.
As a result phy_change() effectively just calls phy_trigger_machine()
when called from phy_mac_interrupt() via phy_change_work(). So we can
call phy_trigger_machine() from phy_mac_interrupt() directly and
remove some now unneeded code.
Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
---
drivers/net/phy/phy.c | 14 +-------------
drivers/net/phy/phy_device.c | 1 -
include/linux/phy.h | 3 ---
3 files changed, 1 insertion(+), 17 deletions(-)
diff --git a/drivers/net/phy/phy.c b/drivers/net/phy/phy.c
index da41420df..ce1e8130a 100644
--- a/drivers/net/phy/phy.c
+++ b/drivers/net/phy/phy.c
@@ -750,18 +750,6 @@ static irqreturn_t phy_change(struct phy_device *phydev)
return IRQ_NONE;
}
-/**
- * phy_change_work - Scheduled by the phy_mac_interrupt to handle PHY changes
- * @work: work_struct that describes the work to be done
- */
-void phy_change_work(struct work_struct *work)
-{
- struct phy_device *phydev =
- container_of(work, struct phy_device, phy_queue);
-
- phy_change(phydev);
-}
-
/**
* phy_interrupt - PHY interrupt handler
* @irq: interrupt line
@@ -1005,7 +993,7 @@ void phy_state_machine(struct work_struct *work)
void phy_mac_interrupt(struct phy_device *phydev)
{
/* Trigger a state machine change */
- queue_work(system_power_efficient_wq, &phydev->phy_queue);
+ phy_trigger_machine(phydev);
}
EXPORT_SYMBOL(phy_mac_interrupt);
diff --git a/drivers/net/phy/phy_device.c b/drivers/net/phy/phy_device.c
index 00a46218c..0f56d408b 100644
--- a/drivers/net/phy/phy_device.c
+++ b/drivers/net/phy/phy_device.c
@@ -587,7 +587,6 @@ struct phy_device *phy_device_create(struct mii_bus *bus, int addr, int phy_id,
mutex_init(&dev->lock);
INIT_DELAYED_WORK(&dev->state_queue, phy_state_machine);
- INIT_WORK(&dev->phy_queue, phy_change_work);
/* Request the appropriate module unconditionally; don't
* bother trying to do so only if it isn't already loaded,
diff --git a/include/linux/phy.h b/include/linux/phy.h
index 7db07e69c..17d1f6472 100644
--- a/include/linux/phy.h
+++ b/include/linux/phy.h
@@ -369,7 +369,6 @@ struct phy_c45_device_ids {
* giving up on the current attempt at acquiring a link
* irq: IRQ number of the PHY's interrupt (-1 if none)
* phy_timer: The timer for handling the state machine
- * phy_queue: A work_queue for the phy_mac_interrupt
* attached_dev: The attached enet driver's device instance ptr
* adjust_link: Callback for the enet controller to respond to
* changes in the link state.
@@ -454,7 +453,6 @@ struct phy_device {
void *priv;
/* Interrupt and Polling infrastructure */
- struct work_struct phy_queue;
struct delayed_work state_queue;
struct mutex lock;
@@ -1029,7 +1027,6 @@ int phy_driver_register(struct phy_driver *new_driver, struct module *owner);
int phy_drivers_register(struct phy_driver *new_driver, int n,
struct module *owner);
void phy_state_machine(struct work_struct *work);
-void phy_change_work(struct work_struct *work);
void phy_mac_interrupt(struct phy_device *phydev);
void phy_start_machine(struct phy_device *phydev);
void phy_stop_machine(struct phy_device *phydev);
--
2.19.1
^ permalink raw reply related
* [PATCH net-next 1/3] net: phy: don't set state PHY_CHANGELINK in phy_change
From: Heiner Kallweit @ 2018-11-09 17:55 UTC (permalink / raw)
To: Andrew Lunn, Florian Fainelli, David Miller; +Cc: netdev@vger.kernel.org
In-Reply-To: <f4d8c0ea-07fe-d257-f392-6ddb7b686593@gmail.com>
State PHY_CHANGELINK isn't needed here, we can call the state machine
directly. We just have to remove the check for phy_polling_mode() to
make this work also in interrupt mode. Removing this check doesn't
cause any overhead because when not polling the state machine is
called only if required by some event.
Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
---
drivers/net/phy/phy.c | 8 --------
include/linux/phy.h | 7 ++-----
2 files changed, 2 insertions(+), 13 deletions(-)
diff --git a/drivers/net/phy/phy.c b/drivers/net/phy/phy.c
index 8dac890f3..da41420df 100644
--- a/drivers/net/phy/phy.c
+++ b/drivers/net/phy/phy.c
@@ -738,11 +738,6 @@ static irqreturn_t phy_change(struct phy_device *phydev)
goto phy_err;
}
- mutex_lock(&phydev->lock);
- if ((PHY_RUNNING == phydev->state) || (PHY_NOLINK == phydev->state))
- phydev->state = PHY_CHANGELINK;
- mutex_unlock(&phydev->lock);
-
/* reschedule state queue work to run as soon as possible */
phy_trigger_machine(phydev);
@@ -946,9 +941,6 @@ void phy_state_machine(struct work_struct *work)
break;
case PHY_NOLINK:
case PHY_RUNNING:
- if (!phy_polling_mode(phydev))
- break;
- /* fall through */
case PHY_CHANGELINK:
case PHY_RESUMING:
err = phy_check_link_status(phydev);
diff --git a/include/linux/phy.h b/include/linux/phy.h
index 59bb31ee1..7db07e69c 100644
--- a/include/linux/phy.h
+++ b/include/linux/phy.h
@@ -298,7 +298,7 @@ struct phy_device *mdiobus_scan(struct mii_bus *bus, int addr);
* - timer moves to NOLINK or RUNNING
*
* NOLINK: PHY is up, but not currently plugged in.
- * - If the timer notes that the link comes back, we move to RUNNING
+ * - irq or timer will set RUNNING if link comes back
* - phy_stop moves to HALTED
*
* FORCING: PHY is being configured with forced settings
@@ -309,10 +309,7 @@ struct phy_device *mdiobus_scan(struct mii_bus *bus, int addr);
*
* RUNNING: PHY is currently up, running, and possibly sending
* and/or receiving packets
- * - timer will set CHANGELINK if we're polling (this ensures the
- * link state is polled every other cycle of this state machine,
- * which makes it every other second)
- * - irq will set CHANGELINK
+ * - irq or timer will set NOLINK if link goes down
* - phy_stop moves to HALTED
*
* CHANGELINK: PHY experienced a change in link state
--
2.19.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox