[Bug 188] significant multi-core performance degradation with SR-IOV/macswap in some cases

[bugzilla@dpdk.org]

https://bugs.dpdk.org/show_bug.cgi?id=188

            Bug ID: 188
           Summary: significant multi-core performance degradation with
                    SR-IOV/macswap in some cases
           Product: DPDK
           Version: 19.02
          Hardware: x86
                OS: Linux
            Status: CONFIRMED
          Severity: critical
          Priority: Normal
         Component: testpmd
          Assignee: dev@dpdk.org
          Reporter: alialnu@mellanox.com
  Target Milestone: ---

Hi,

I'm seeing a performance degradation with testpmd running with multiple cores.
I saw the issue with ConnectX-3, ConnectX-4 Lx, and ConnectX-5.

### For ConnectX-3:
- With 2 Cores, I see a degradation from 19.2 mpps to 13.5 mpps.

testpmd command:
`MLX4_INLINE_RECV_SIZE=64 ./build/app/test-pmd/testpmd -c 0x7 -n 4 -w
0002:00:02.0 --vdev=net_vdev_netvsc,ignore=1 --socket-mem=2048 --
--port-numa-config=0,0 --socket-num=0 --burst=64 --txd=256 --rxd=256
--mbcache=512  --rxq=2 --txq=2 --nb-cores=2 -i -a --forward-mode=macswap`


- With 4 Cores, I see a degradation from 19.2 mpps - 13.9 mpps.

testpmd command:
`MLX4_INLINE_RECV_SIZE=64 ./build/app/test-pmd/testpmd -c 0x1f -n 4 -w
0002:00:02.0 --vdev=net_vdev_netvsc,ignore=1 --socket-mem=2048 --
--port-numa-config=0,0 --socket-num=0 --burst=64 --txd=256 --rxd=256
--mbcache=512  --rxq=2 --txq=2 --nb-cores=4 -i -a --forward-mode=macswap`


### For ConnectX-4 Lx:

- With 2 Cores, I see a degradation from 32.9 mpps to 26.8 mpps.

testpmd command:
`./build/app/test-pmd/testpmd -c 0x7 -n 4 -w 0003:00:02.0
--vdev="net_vdev_netvsc,ignore=1" -- --burst=64 --txd=1024 --rxd=256
--mbcache=512 --rxq=2 --txq=2 --nb-cores=2 --rss-udp --port-topology=chained
--forward-mode=macswap -i -a`

- With 4 Cores, I see a degradation from 34.8 mpps to 19 mpps.

testpmd command:
`./build/app/test-pmd/testpmd -c 0x1f -n 4 -w 0003:00:02.0
--vdev="net_vdev_netvsc,ignore=1" -- --burst=64 --txd=8192 --rxd=256
--mbcache=512 --rxq=4 --txq=4 --nb-cores=4 --rss-udp --port-topology=chained
--forward-mode=macswap -i -a`


Server specs:
OS: Ubuntu 16.04.2 LTS
Kernel: 4.15.0-rc3

I'm using IXIA as a traffic generator, and to get the forwarding performance.

DPDK version: master:a89fbda

The cause of the degradation is the patch:

```
commit 62b52877adbe8d3ad19615b0827e7725e51feadc
Author: Qi Zhang <qi.z.zhang@intel.com>
Date:   Sun Dec 16 08:58:35 2018 +0800

    app/testpmd: batch MAC swap for performance on x86

    Do four packets macswap in same loop iterate to squeeze more
    CPU cycles.

    Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
    Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
    Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
```

-- 
You are receiving this mail because:
You are the assignee for the bug.