Re: [PATCH net-next v5 0/3] Introduce IPPROTO_SMC

[Niklas Schnelle <schnelle@linux.ibm.com>]

On Thu, 2024-05-30 at 18:14 +0800, D. Wythe wrote:
> 
> On 5/30/24 5:30 PM, D. Wythe wrote:
> > From: "D. Wythe" <alibuda@linux.alibaba.com>
> > 
> > This patch allows to create smc socket via AF_INET,
> > similar to the following code,
> > 
> > /* create v4 smc sock */
> > v4 = socket(AF_INET, SOCK_STREAM, IPPROTO_SMC);
> > 
> > /* create v6 smc sock */
> > v6 = socket(AF_INET6, SOCK_STREAM, IPPROTO_SMC);
> 
> Welcome everyone to try out the eBPF based version of smc_run during 
> testing, I have added a separate command called smc_run.bpf,
> it was equivalent to normal smc_run but with IPPROTO_SMC via eBPF.
> 
> You can obtain the code and more info from: 
> https://github.com/D-Wythe/smc-tools
> 
> Usage:
> 
> smc_run.bpf
> An eBPF implemented smc_run based on IPPROTO_SMC:
> 
> 1. Support to transparent replacement based on command (Just like smc_run).
> 2. Supprot to transparent replacement based on pid configuration. And 
> supports the inheritance of this capability between parent and child 
> processes.
> 3. Support to transparent replacement based on per netns configuration.
> 
> smc_run.bpf COMMAND
> 
> 1. Equivalent to smc_run but with IPPROTO_SMC via eBPF
> 
> smc_run.bpf -p pid
> 
>   1. Add the process with target pid to the map. Afterward, all socket() 
> calls of the process and its descendant processes will be replaced from 
> IPPROTO_TCP to IPPROTO_SMC.
>   2. Mapping will be automatically deleted when process exits.
>   3. Specifically, COMMAND mode is actually works like following:
> 
>      smc_run.bpf -p $$
>      COMMAND
>      exit
> 
> smc_run.bpf -n 1
> 
>   1. Make all socket() calls of the current netns to be replaced from 
> IPPROTO_TCP to IPPROTO_SMC.
>   2. Turn off it by smc_run.bpf -n 0
> 
> 

Hi D. Wythe,

I gave this series plus your smc_run.bpf and SMC_LO based SMC-D a test
run on my Ryzen 3900X workstation and I have to say I'm quite
impressed. I first tried the SMC_LO feature as merged in v6.10-rc1 with
the classic LD_PRELOAD based smc_run and iperf3, and qperf …
tcp_bw/tcp_lat both with normal localhost and between docker
containers. For this to work I of course had to initially set my UEID
as x86_64 unlike s390x doesn't get an SEID set. I used the following
script for this.


#!/usr/bin/sh
machine_id_upper=$(cat /etc/machine-id | tr '[:lower:]' '[:upper:]')
machine_id_suffix=$(echo "$machine_id_upper" | head -c 27)
ueid="MID-$machine_id_suffix"
smcd ueid add "$ueid"


The performance is pretty impressive:
* iperf3 with 12 parallel connections (matching core count) results in
  ~152 Gbit/s on normal loopback and ~312 Gbit/s with SMC_LO.
* qperf … tcp_bw (single thread) results in ~46 Gbit/s on normal loopback
  and ~58 Gbit/s with SMC_LO
* qperf … tcp_lat latency test results in 5-9 us with normal loopback
  and around 3-4 us with SMC_LO

Then I applied this series on top of v6.10-rc1 and tried it with your
smc_run.bpf. The performance is of course in-line with the above but
thanks to being able to enable SMC on a per-netns basis I was able to
try a few more thing. First I tried just enabling it in my default
netns and verified that after restarting sshd new ssh connections to
localhost used SMC-D through SMC_LO. Then I started Chrome and
confirmed that its TCP connections also registered with SMC and
successfully fell back to TCP mode. I had no trouble with normal
browsing though I guess especially Google stuff often uses HTTP/3 so
isn't affected. Still nice to see I didn't get breakage.

Secondly I tried smc_run.bpf with docker containers using the following
trick:

docker inspect --format '{{.State.Pid}}' <my_container_name>
34651
nsenter -t 34651 -n smc_run.bpf -n 1

Sadly this only works for commands started in the container after
loading the BPF. So I wonder if you know of a good way to either
automatically execute smc_run.bpf on container start or maybe use it on
the docker daemon such that all namespaces created by docker get the
IPPROTO_SMC override. I'd then definitely consider using SMC-D with
SMC_LO between my home lab containers even if just for bragging rights
;-)

Feel free to add for the IPPROTO_SMC series:

Tested-by: Niklas Schnelle <schnelle@linux.ibm.com>

Thanks,
Niklas