Re: RDMA (smbdirect) testing

[Tom Talpey <tom@talpey.com>]

On 6/2/2022 8:07 PM, Long Li wrote:
>> Long, My Chelsio(iWARP) NIC reports this value as 4. When I set it with hw
>> report value in cifs.ko, There is kernel oops in cifs.ko. Have you checked smb-
>> direct of cifs.ko with Chelsio and any iWARP NICs before ? or only Mellanox
>> NICs ?
>>
>> Thanks!
> 
> Yes, I have tested on Chelsio. I didn't see kernel panic. In fact, I can pass a larger value (8) and successfully create a QP on Chelsio.

There are many generations of Chelsio RDMA adapters, and this number
is very likely to be different. You both should be sure you're testing
with multiple configurations (and don't forget all the other in-kernel
RDMA NICs). But a constant value of "8" is still arbitrary.

The kernel should definitely not throw an oops when the NIC doesn't
support some precompiled constant value. Namjae, what oops, exactly?

Tom.

> Can you paste your kernel panic trace?
> 
>>>>> If the CIFS upper layer ever sends data with larger number of SGEs,
>>>>> the send will fail.
>>>>>
>>>>> Long
>>>>>
>>>>>>
>>>>>> Tom.
>>>>>>
>>>>>>> diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h index
>>>>>>> a87fca82a796..7003722ab004 100644
>>>>>>> --- a/fs/cifs/smbdirect.h
>>>>>>> +++ b/fs/cifs/smbdirect.h
>>>>>>> @@ -226,7 +226,7 @@ struct smbd_buffer_descriptor_v1 {
>>>>>>>    } __packed;
>>>>>>>
>>>>>>>    /* Default maximum number of SGEs in a RDMA send/recv */
>>>>>>> -#define SMBDIRECT_MAX_SGE      16
>>>>>>> +#define SMBDIRECT_MAX_SGE      6
>>>>>>>    /* The context for a SMBD request */
>>>>>>>    struct smbd_request {
>>>>>>>           struct smbd_connection *info; diff --git
>>>>>>> a/fs/ksmbd/transport_rdma.c b/fs/ksmbd/transport_rdma.c index
>>>>>>> e646d79554b8..70662b3bd590 100644
>>>>>>> --- a/fs/ksmbd/transport_rdma.c
>>>>>>> +++ b/fs/ksmbd/transport_rdma.c
>>>>>>> @@ -42,7 +42,7 @@
>>>>>>>    /* SMB_DIRECT negotiation timeout in seconds */
>>>>>>>    #define SMB_DIRECT_NEGOTIATE_TIMEOUT           120
>>>>>>>
>>>>>>> -#define SMB_DIRECT_MAX_SEND_SGES               8
>>>>>>> +#define SMB_DIRECT_MAX_SEND_SGES               6
>>>>>>>    #define SMB_DIRECT_MAX_RECV_SGES               1
>>>>>>>
>>>>>>>    /*
>>>>>>>
>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> Tom.
>>>>>>>>
>>>>>>>
>>>>>
>>>