Re: [net-next 1/2] net/smc: allow confirm/delete rkey response deliver multiplex

["D. Wythe" <alibuda@linux.alibaba.com>]

On 2/8/23 7:15 AM, Wenjia Zhang wrote:
> 
> 
> On 07.02.23 08:36, D. Wythe wrote:
>> From: "D. Wythe" <alibuda@linux.alibaba.com>
>>
>> We know that all flows except confirm_rkey and delete_rkey are exclusive,
>> confirm/delete rkey flows can run concurrently (local and remote).
>>
>> Although the protocol allows, all flows are actually mutually exclusive
>> in implementation, dues to waiting for LLC messages is in serial.
>>
>> This aggravates the time for establishing or destroying a SMC-R
>> connections, connections have to be queued in smc_llc_wait.
>>
>> We use rtokens or rkey to correlate a confirm/delete rkey message
>> with its response.
>>
>> Before sending a message, we put context with rtokens or rkey into
>> wait queue. When a response message received, we wakeup the context
>> which with the same rtokens or rkey against the response message.
>>
>> Signed-off-by: D. Wythe <alibuda@linux.alibaba.com>
>> ---
>>   net/smc/smc_llc.c | 174 +++++++++++++++++++++++++++++++++++++++++-------------
>>   net/smc/smc_wr.c  |  10 ----
>>   net/smc/smc_wr.h  |  10 ++++
>>   3 files changed, 143 insertions(+), 51 deletions(-)
>>
> 
> [...]
> 
>> +static int smc_llc_rkey_response_wake_function(struct wait_queue_entry *wq_entry,
>> +                           unsigned int mode, int sync, void *key)
>> +{
>> +    struct smc_llc_qentry *except, *incoming;
>> +    u8 except_llc_type;
>> +
>> +    /* not a rkey response */
>> +    if (!key)
>> +        return 0;
>> +
>> +    except = wq_entry->private;
>> +    incoming = key;
>> +
>> +    except_llc_type = except->msg.raw.hdr.common.llc_type;
>> +
>> +    /* except LLC MSG TYPE mismatch */
>> +    if (except_llc_type != incoming->msg.raw.hdr.common.llc_type)
>> +        return 0;
>> +
>> +    switch (except_llc_type) {
>> +    case SMC_LLC_CONFIRM_RKEY:
>> +        if (memcmp(except->msg.confirm_rkey.rtoken, incoming->msg.confirm_rkey.rtoken,
>> +               sizeof(struct smc_rmb_rtoken) *
>> +               except->msg.confirm_rkey.rtoken[0].num_rkeys))
>> +            return 0;
>> +        break;
>> +    case SMC_LLC_DELETE_RKEY:
>> +        if (memcmp(except->msg.delete_rkey.rkey, incoming->msg.delete_rkey.rkey,
>> +               sizeof(__be32) * except->msg.delete_rkey.num_rkeys))
>> +            return 0;
>> +        break;
>> +    default:
>> +        pr_warn("smc: invalid except llc msg %d.\n", except_llc_type);
>> +        return 0;
>> +    }
>> +
>> +    /* match, save hdr */
>> +    memcpy(&except->msg.raw.hdr, &incoming->msg.raw.hdr, sizeof(except->msg.raw.hdr));
>> +
>> +    wq_entry->private = except->private;
>> +    return woken_wake_function(wq_entry, mode, sync, NULL);
>> +}
>> +
> 
> s/except/expect/ ?
> Just kind of confusing
> 
> [...]

Hi, Wenjia

Except is what I want to express.
It means that only the confirm and delete rkey can be processed in parallel. :-)

Thanks
D. Wythe