ixgbe: driver drops packets routed from an IPSec interface with a "bad sa_idx" error

[Michael Marley <michael@michaelmarley.com>]

(This is also reported at 
https://bugzilla.kernel.org/show_bug.cgi?id=204551, but it was 
recommended that I send it to this list as well.)

I have a put together a router that routes traffic from several local 
subnets from a switch attached to an i82599ES card through an IPSec VPN 
interface set up with StrongSwan.  (The VPN is running on an unrelated 
second interface with a different driver.)  Traffic from the local 
interfaces to the VPN works as it should and eventually makes it through 
the VPN server and out to the Internet.  The return traffic makes it 
back to the router and tcpdump shows it leaving by the i82599, but the 
traffic never actually makes it onto the wire and I instead get one of

enp1s0: ixgbe_ipsec_tx: bad sa_idx=64512 handle=0

for each packet that should be transmitted.  (The sa_idx and handle 
values are always the same.)

I realized this was probably related to ixgbe's IPSec offloading 
feature, so I tried with the motherboard's integrated e1000e device and 
didn't have the problem.  I tried using ethtool to disable all the 
IPSec-related offloads (tx-esp-segmentation, esp-hw-offload, 
esp-tx-csum-hw-offload), but the problem persisted.  I then tried 
recompiling the kernel with CONFIG_IXGBE_IPSEC=n and that worked around 
the problem.

I was also able to find another instance of the same problem reported in 
Debian at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=930443.  
That person seems to be having exactly the same issue as me, down to the 
sa_idx and handle values being the same.

If there are any more details I can provide to make this easier to track 
down, please let me know.

Thanks,

Michael Marley