[Bug 1053] ConnectX6 / mlx5 DPDK - bad RSS/ rte_flow performance on mixed traffic ( rxq_cqe_comp_en=4 )

[bugzilla@dpdk.org]

https://bugs.dpdk.org/show_bug.cgi?id=1053

            Bug ID: 1053
           Summary: ConnectX6 / mlx5 DPDK - bad RSS/ rte_flow performance
                    on mixed traffic (  rxq_cqe_comp_en=4 )
           Product: DPDK
           Version: 21.11
          Hardware: x86
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: ethdev
          Assignee: dev@dpdk.org
          Reporter: rtox@gmx.net
  Target Milestone: ---

Our team has been chasing major performance issues with ConnectX6 cards.

*Customer challenge:*
Flow stable ( symmetric RSS) load-balancing of flows to 8 worker lcores.

*Observation:*
Performance is fine up to 100Gbps using either tcp *or* udp-only traffic
profiles.
Mixed traffic drops down to 50% loss with all packets showing up as xstats:
rx_phy_discard_packets

card infos at end of email.


There appears to be a huge performance issue on mixed UDP/TCP using symmetric
load-balancing accross multiple workers.
E.g. compiling a DPDK v20.11 or newer test-pmd apps:


> sudo ./dpdk-testpmd -n 8 -l 4,6,8,10,12,14,16,18,20  -a >
> 0000:4b:00.0,rxq_cqe_comp_en=4  -a 0000:4b:00.1,rxq_cqe_comp_en=4  --
> --forward-> mode=mac --rxq=8 --txq=8 --nb-cores=8 --numa -i -a --disable-rss


and configuring:


> flow create 0 ingress pattern eth / ipv4 / tcp / end actions rss types
> ipv4-tcp > end queues 0 1 2 3 4 5 6 7 end key >
> 6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A
> / > end

> flow create 0 ingress pattern eth / ipv4 / udp / end actions rss types
> ipv4-udp > end queues 0 1 2 3 4 5 6 7 end key >
> 6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A
> / > end

> flow create 1 ingress pattern eth / ipv4 / tcp / end actions rss types
> ipv4-tcp > end queues 0 1 2 3 4 5 6 7 end key >
> 6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A
> / > end

> flow create 1 ingress pattern eth / ipv4 / udp / end actions rss types
> ipv4-udp > end queues 0 1 2 3 4 5 6 7 end key >
> 6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A
> / > end

will see *significant* packet drops at a load > 50 Gbps on any type of mixed
UDP/TCP traffic. E.g. 

https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/cap2/sfr3.yaml
Whenever those packet drops occur, I see those in the xstats as
"rx_phy_discard_packets"


On the other hand using a TCP-or UDP-only traffic profile perfectly scales up
to 100Gbps w/o drops.


Thanks for your help!



> {code}
> ConnectX6DX



> <Devices>
>     <Device pciName="0000:4b:00.0" type="ConnectX6DX" psid="DEL0000000027" >
>     partNumber="0F6FXM_08P2T2_Ax">
>       <Versions>
>         <FW current="22.31.1014" available="N/A"/>
>         <PXE current="3.6.0403" available="N/A"/>
>         <UEFI current="14.24.0013" available="N/A"/>
>      </Versions>
> {code}

-- 
You are receiving this mail because:
You are the assignee for the bug.