[bpf?] [net?] Questions about the impact of ebpf sockmap/redirection on socket performance improvement

[bpf?] [net?] Questions about the impact of ebpf sockmap/redirection on socket performance improvement

[kongweibin]

I'm using ebpf sockmap/redirection to bypass the TCP/IP stack to improve
socket communication performance (throughput, latency) between different
PODs on the same machine. As concurrent connections or payloads increase,
there may be unconspicuous performance improvement or even performance
degradation compared to the TCP/IP stack. I have retrieved some performance
data, but it does not seem to involve high concurrency and high payloads,
I would like to know if community have conducted relevant tests on these
scenarios and have any opinions on the poor performance improvement in these
scenarios.

RE: [bpf?] [net?] Questions about the impact of ebpf sockmap/redirection on socket performance improvement

[John Fastabend]

kongweibin wrote:
> I'm using ebpf sockmap/redirection to bypass the TCP/IP stack to improve
> socket communication performance (throughput, latency) between different
> PODs on the same machine. As concurrent connections or payloads increase,
> there may be unconspicuous performance improvement or even performance
> degradation compared to the TCP/IP stack. I have retrieved some performance
> data, but it does not seem to involve high concurrency and high payloads,
> I would like to know if community have conducted relevant tests on these
> scenarios and have any opinions on the poor performance improvement in these
> scenarios.

Not fully following above, what is your performance test/tools and what
were the numbers?

[bpf?] [net?] socket test data of ebpf sockmap/redirection and TCP/IP stack

[kongweibin]

Thank you for your attention!

The testing tool is Fortio.First,start a Fortio server container,and then
start a Fortio client container to send messages to the server container.
Compare the performance indicators of using ebpf sockmap/redirect and TCP/IP
stack.

test cmd:
fortio server:docker run -it -d --name=fortio-server fortio/fortio server
fortio client:docker run -it --rm --name=fortio-client fortio/fortio load 
              -c $connum -t 30s -qps 0 -jitter=true -payload-size $payload 
			  -quiet http://172.17.0.2:8080

As I increase `connum` or `payload`,the performance improvement gradually
becomes less significant or even decreases.

Host Configuration:16 CPUs(x86_64),64G mem
OS kernel version:5.10

here are the test datas:

|----------------------------------------------------------------------------
|connum	|payload|   qps	  |avg(ms)|   qps   |avg(ms)|qps improve|avg reduce |
|----------------------------------------------------------------------------
|       |       |     TCP/IP  	  |		  ebpf      |           |           |
|1		|0k		|25819.43 |0.039  |31060.83 |0.032 	|20.30%		|17.24%     |
|1		|1k		|20697.53 |0.048  |27073.13 |0.037 	|30.80%		|23.45%     |
|1		|10k	|11086.70 |0.090  |14656.67 |0.068 	|32.20%		|24.44%     |
|1		|100k	|2993.60  |0.334  |3746.63  |0.267 	|25.15%		|20.16%     |
|10		|0k		|145297.5 |0.069  |170481.7 |0.058 	|17.33%		|15.53%     |
|10		|1k		|80145.87 |0.124  |94578.50 |0.105 	|18.01%		|15.28%     |
|10		|10k	|20523.40 |0.487  |23920.67 |0.418 	|16.55%		|14.11%     |
|10		|100k	|6532.70  |1.531  |6837.87  |1.462 	|4.67%		|4.49%      |
|20		|0k		|218367.3 |0.091  |258349.1 |0.077 	|18.31%		|15.38%     |
|20		|1k		|90609.27 |0.221  |98366.67 |0.203 	|8.65%		|7.93%      |
|20		|10k	|20363.63 |0.983  |22731.00 |0.880 	|11.66%		|10.45%     |
|20		|100k	|7391.83  |2.705  |7374.20  |2.712 	|-0.24%		|-0.25%     |
|50		|0k		|221913.6 |0.225  |255631.1 |0.195 	|15.20%		|13.18%     |
|50		|1k		|110390.9 |0.453  |123547.0 |0.404 	|12.00%		|10.74%     |
|50		|10k	|28793.87 |1.737  |28751.90 |1.739 	|-0.08%		|-0.15%     |
|50		|100k	|9590.40  |5.213  |9565.83  |5.227 	|-0.26%		|-0.26%     |
|100	|0k		|286830.5 |0.349  |348999.5 |0.287 	|21.67%		|17.78%     |
|100	|1k		|147858.3 |0.676  |170042.6 |0.588 	|15.01%		|13.01%     |
|100	|10k	|50195.63 |1.992  |52431.73 |1.907 	|4.49%		|4.25%      |
|100	|100k	|11895.37 |8.405  |11982.83 |8.344 	|0.74%		|0.73%      |
|----------------------------------------------------------------------------