Re: [PATCH xfrm-next v2 0/6] Extend XFRM core to allow full offload configuration

[Leon Romanovsky <leon@kernel.org>]

On Mon, Aug 22, 2022 at 10:41:05AM +0200, Steffen Klassert wrote:
> On Fri, Aug 19, 2022 at 10:53:56AM -0700, Jakub Kicinski wrote:
> > On Fri, 19 Aug 2022 13:01:22 -0300 Jason Gunthorpe wrote:
> > > Regardless, RDMA doesn't really intersect with this netdev work for
> > > XFRM beyond the usual ways that RDMA IP traffic can be captured by or
> > > run parallel to netdev.
> > > 
> > > A significant use case here is for switchdev modes where the switch
> > > will subject traffic from a switch port to ESP, not unlike it already
> > > does with vlan, vxlan, etc and other already fully offloaded switching
> > > transforms.
> > 
> > Yup, that's what I thought you'd say. Can't argue with that use case 
> > if Steffen is satisfied with the technical aspects.
> 
> Yes, everything that can help to overcome the performance problems
> can help and I'm interested in this type of offload. But we need to
> make sure the API is usable by the whole community, so I don't
> want an API for some special case one of the NIC vendors is
> interested in.

BTW, we have a performance data, I planned to send it as part of cover
letter for v3, but it is worth to share it now.

 ================================================================================
 Performance results:

 TCP multi-stream, using iperf3 instance per-CPU.
 +----------------------+--------+--------+--------+--------+---------+---------+
 |                      | 1 CPU  | 2 CPUs | 4 CPUs | 8 CPUs | 16 CPUs | 32 CPUs |
 |                      +--------+--------+--------+--------+---------+---------+
 |                      |                   BW (Gbps)                           |
 +----------------------+--------+--------+-------+---------+---------+---------+
 | Baseline             | 27.9   | 59     | 93.1  | 92.8    | 93.7    | 94.4    |
 +----------------------+--------+--------+-------+---------+---------+---------+
 | Software IPsec       | 6      | 11.9   | 23.3  | 45.9    | 83.8    | 91.8    |
 +----------------------+--------+--------+-------+---------+---------+---------+
 | IPsec crypto offload | 15     | 29.7   | 58.5  | 89.6    | 90.4    | 90.8    |
 +----------------------+--------+--------+-------+---------+---------+---------+
 | IPsec full offload   | 28     | 57     | 90.7  | 91      | 91.3    | 91.9    |
 +----------------------+--------+--------+-------+---------+---------+---------+

 IPsec full offload mode behaves as baseline and reaches linerate with same amount
 of CPUs.

 Setups details (similar for both sides):
 * NIC: ConnectX6-DX dual port, 100 Gbps each.
   Single port used in the tests.
 * CPU: Intel(R) Xeon(R) Platinum 8380 CPU @ 2.30GHz

Thanks