Re: [bug] cxgb4: vrf stopped working with cxgb4 card

[Ganesh Goudar <ganeshgr@chelsio.com>]

On Saturday, June 06/09/18, 2018 at 18:47:55 -0600, David Ahern wrote:
> Ganesh:
> 
> On 6/4/18 9:03 AM, AMG Zollner Robert wrote:
> > I have noticed that vrf is not working with kernel v4.15.0 but was
> > working with v4.13.0 when using cxgb4 Chelsio driver (T520-cr)
> > 
> > Setup:
> > Two metal servers with a T520-cr card each, directly connected without a
> > switch in between.
> > 
> >        SVR1  only ipfwd                 SVR2     with vrf
> > .----------------------------. .----------------------------------.
> > |                            |         |             |
> > |    192.168.8.1 [  ens2f4]--|---------|--[ens1f4] 192.168.8.2   |
> > |    192.168.9.1 [ens2f4d1]--|---------|--<ens1f4d1> 192.168.9.2 VRF=10   |
> > `----------------------------' `----------------------------------'
> > 
> > When vrf is not working there are no error messages (dmesg or iproute
> > commands), tcpdump on the interface (SVR2.ens1f4d1) enslaved in vrf 10
> > shows packets(arp req/reply) coming in and going out, but outgoing
> > packets(arp reply) do not reach the other server SVR1.ens2f4d1
> > 
> > 
> > Bisect:
> > Found this commit to be the problem after doing a git bisect between
> > v4.13..v4.15:
> > 
> > commit ba581f77df23c8ee70b372966e69cf10bc5453d8
> > Author: Ganesh Goudar <ganeshgr@chelsio.com>
> > Date:   Sat Sep 23 16:07:28 2017 +0530
> > 
> >     cxgb4: do DCB state reset in couple of places
> > 
> >     reset the driver's DCB state in couple of places
> >     where it was missing.
> 
> Are you working on a fix for this or should a revert of the above patch
> be sent?
Will look into it and fix/revert it soon, Thanks for responding to Robert.
> 
> 
> > 
> > 
> > A bisect step was considered good when:
> > - successful ping from SVR1 to SVR2.ens1f4d1 vrf interface
> > - successful ping from SVR2 global to SVR2 vrf interface trough SVR1(l3
> > forwarding) (this check was redundant,both tests fail or pass simultaneous)
> > 
> > The problem is still present on recent kernels also, checked v4.16.0 and
> > v4.17.rc7
> > 
> > Disabling DCB for the card support fixes the problem ( Compiling kernel
> > with "CONFIG_CHELSIO_T4_DCB=n")
> > 
> > 
> > 
> > This is my first time reporting a bug to the linux kernel and hope I
> > have included the right amount of information. Please let me know if I
> > have missed something.
> > 
> > 
> > 
> > Thank you,
> > Zollner Robert
> > 
> > 
> > --------
> > Logs:
> > 
> > VRF configured using folowing commands:
> > 
> > #!/bin/sh
> > 
> > CHDEV=ens1f4
> > VRF=vrf-recv
> > 
> > sysctl -w net.ipv4.tcp_l3mdev_accept=1
> > sysctl -w net.ipv4.udp_l3mdev_accept=1
> > sysctl -w net.ipv4.conf.all.accept_local=1
> > 
> > ifconfig ${CHDEV}   192.168.8.2/24
> > ifconfig ${CHDEV}d1 192.168.9.2/24
> > 
> > ip link add ${VRF} type vrf table 10
> > ip link set dev ${VRF} up
> > 
> > ip rule add pref 32765 table local
> > ip rule del pref 0
> > 
> > ip route add table 10 unreachable default metric 4278198272
> > 
> > ip link set dev ${CHDEV}d1 master ${VRF}
> > 
> > ip route add table 10 default via 192.168.9.1
> > ip route add 192.168.9.0/24 via 192.168.8.1
> > 
> > 
> > 
> > 
>