Re: ipsec impact on performance

[David Ahern <dsa@cumulusnetworks.com>]

On 12/1/15 10:17 AM, Rick Jones wrote:
> On 12/01/2015 09:59 AM, Sowmini Varadhan wrote:
>> But these are all still relatively small things - tweaking them
>> doesnt get me significantly past the 3 Gbps limit. Any suggestions
>> on how to make this budge (or design criticism of the patch) would
>> be welcome.
>
> What do the perf profiles show?  Presumably, loss of TSO/GSO means an
> increase in the per-packet costs, but if the ipsec path significantly
> increases the per-byte costs...
>
> Short of a perf profile, I suppose one way to probe for per-packet
> versus per-byte would be to up the MTU.  That should reduce the
> per-packet costs while keeping the per-byte roughly the same.

Using iperf3 and AH with NULL algorithm between 2 peers connected by a 
10G link.

Without AH configured I get a steady 9.9 Gbps with iperf3 consuming 
about 55% cpu.

With AH I get ~1.5 Gbps with MTU at 1500:

[  4]   0.00-1.01   sec   160 MBytes  1.33 Gbits/sec   23    905 KBytes
[  4]   1.01-2.00   sec   211 MBytes  1.79 Gbits/sec    0    996 KBytes

iperf3 runs about 60% CPU and ksoftirqd/2 is at 86%.


Bumping the MTU to 9000:

[  4]   3.00-4.00   sec   914 MBytes  7.67 Gbits/sec  260   1.01 MBytes
[  4]   4.00-5.00   sec  1012 MBytes  8.49 Gbits/sec    0   1.23 MBytes
[  4]   5.00-6.00   sec  1.15 GBytes  9.88 Gbits/sec    0   1.23 MBytes

At this rate iperf3 was at 95% CPU and ksoftirqd was not relevant.