Re: [PATCH net 4/4] net/smc: Fix wq mismatch issue caused by smc fallback

[Wen Gu <guwen@linux.alibaba.com>]

On 2021/11/2 5:25 pm, Karsten Graul wrote:
> On 01/11/2021 07:15, Wen Gu wrote:
>> Before explaining my intentions, I thought it would be better to describe the issue I encountered first：
>>
>> In nginx/wrk tests, when nginx uses TCP and wrk uses SMC to replace TCP, wrk should fall back to TCP and get correct results theoretically, But in fact it only got all zeros.
> 
> Thank you for the very detailed description, I now understand the situation.
> 
> The fix is not obvious and not easy to understand for the reader of the code,
> did you think about a fix that uses own sk_data_ready / sk_write_space
> implementations on the SMC socket to forward the call to the clcsock in the
> fallback situation?
> 
> I.e. we already have smc_tx_write_space(), and there is smc_clcsock_data_ready()
> which is right now only used for the listening socket case.
> 
> If this works this would be a much cleaner and more understandable way to fix this issue.
> 

Thanks for your suggestions about implementing SMC own sk_data_ready / 
sk_write_space and forwarding call to clcsock. It's a great idea. But I 
found some difficulties here in implementation process:

In my humble opinion, SMC own sk_write_space implementation should be 
called by clcsk->sk_write_space and complete the following steps:

1) Get smc_sock through clcsk->sk_user_data, like what did in 
smc_clcsock_data_ready().

2) Forward call to original clcsk->sk_write_space, it MIGHT wake up 
clcsk->sk_wq, depending on whether certain conditions are met.

3) Wake up smc sk->sk_wq to nodify application if clcsk->sk_write_space 
acctually wakes up clcsk->sk_wq.

In step 3), it seems a bit troublesome for SMC to know whether 
clcsk->sk_write_space acctually wake up clcsk->sk_wq, which is a black 
box to SMC.

There might be a feasible way that add a wait_queue_head_t to 
clcsk->sk_wq and report to SMC when clcsk->sk_wq is waked up. Then SMC 
can report to application by waking up smc sk->sk_wq. But that seems to 
be complex and redundancy.

I'm looking forward to hear your opinion about it. Thank you!


cheers,
Wen Gu