Re: [PATCH net 4/4] net/smc: Fix wq mismatch issue caused by smc fallback

Wen Gu
2021-11-04T04:39:51Z

On 2021/11/2 5:25 pm, Karsten Graul wrote:
> On 01/11/2021 07:15, Wen Gu wrote:
>> Before explaining my intentions, I thought it would be better to describe the issue I encountered first：
>>
>> In nginx/wrk tests, when nginx uses TCP and wrk uses SMC to replace TCP, wrk should fall back to TCP and get correct results theoretically, But in fact it only got all zeros.
>
> Thank you for the very detailed description, I now understand the situation.
>
> The fix is not obvious and not easy to understand for the reader of the code,
> did you think about a fix that uses own sk_data_ready / sk_write_space
> implementations on the SMC socket to forward the call to the clcsock in the
> fallback situation?
>
> I.e. we already have smc_tx_write_space(), and there is smc_clcsock_data_ready()
> which is right now only used for the listening socket case.
>
> If this works this would be a much cleaner and more understandable way to fix this issue.
>

Thanks for your suggestions about implementing SMC own sk_data_ready /
sk_write_space and forwarding call to clcsock. It's a great idea. But I
found some difficulties here in implementation process:

In my humble opinion, SMC own sk_write_space implementation should be
called by clcsk->sk_write_space and complete the following steps:

1) Get smc_sock through clcsk->sk_user_data, like what did in
smc_clcsock_data_ready().

2) Forward call to original clcsk->sk_write_space, it MIGHT wake up
clcsk->sk_wq, depending on whether certain conditions are met.

3) Wake up smc sk->sk_wq to nodify application if clcsk->sk_write_space
acctually wakes up clcsk->sk_wq.

In step 3), it seems a bit troublesome for SMC to know whether
clcsk->sk_write_space acctually wake up clcsk->sk_wq, which is a black
box to SMC.

There might be a feasible way that add a wait_queue_head_t to
clcsk->sk_wq and report to SMC when clcsk->sk_wq is waked up. Then SMC
can report to application by waking up smc sk->sk_wq. But that seems to
be complex and redundancy.

I'm looking forward to hear your opinion about it. Thank you!

cheers,
Wen Gu