Re: Problem with smbdirect rw credits and initiator_depth

[Stefan Metzmacher <metze@samba.org>]

Am 17.01.26 um 00:08 schrieb Stefan Metzmacher:
> Am 15.01.26 um 10:50 schrieb Stefan Metzmacher:
>> Am 15.01.26 um 03:01 schrieb Namjae Jeon:
>>> On Thu, Jan 15, 2026 at 3:13 AM Stefan Metzmacher <metze@samba.org> wrote:
>>>>
>>>> Am 15.12.25 um 21:17 schrieb Stefan Metzmacher:
>>>>> Am 14.12.25 um 23:56 schrieb Stefan Metzmacher:
>>>>>> Am 13.12.25 um 03:14 schrieb Namjae Jeon:
>>>>>>>> I've put these changes a long with rw credit fixes into my
>>>>>>>> for-6.18/ksmbd-smbdirect-regression-v4 branch, are you able to
>>>>>>>> test this?
>>>>>>> Problems still occur. See:
>>>>>>
>>>>>> :-( Would you be able to use rxe and cake a network capture?
>>>>>>
>>>>>> Using test files with all zeros, e.g.
>>>>>> dd if=/dev/zero of=/tmp/4096MBzeros-sparse.dat seek=4096MB bs=1 count=1
>>>>>> would allow gzip --best on the capture file to compress well...
>>>>>
>>>>> I think I found something that explains it and
>>>>> I was able to reproduce and what I have in mind.
>>>>>
>>>>> We increment recv_io.posted.count after ib_post_recv()
>>>>>
>>>>> And manage_credits_prior_sending() uses
>>>>>
>>>>> new_credits = recv_io.posted.count - recv_io.credits.count
>>>>>
>>>>> But there is a race between the hardware receiving a message
>>>>> and recv_done being called in order to decrement recv_io.posted.count
>>>>> again. During that race manage_credits_prior_sending() might grant
>>>>> too much credits.
>>>>>
>>>>> Please test my for-6.18/ksmbd-smbdirect-regression-v5 branch,
>>>>> I haven't tested this branch yet, I'm running out of time
>>>>> for the day.
>>>>>
>>>>> But I tested it with smbclient and having a similar
>>>>> logic in fs/smb/common/smbdirect/smbdirect_connection.c
>>>>
>>>> I was able to reproduce the problem and the fix I created
>>>> for-6.18/ksmbd-smbdirect-regression-v5 was not correct.
>>>>
>>>> I needed to use
>>>>
>>>> available = atomic_xchg(&sc->recv_io.credits.available, 0);
>>>>
>>>> instead of
>>>>
>>>> available = atomic_read(&sc->recv_io.credits.available);
>>>> atomic_sub(new_credits, &sc->recv_io.credits.available);
>>>>
>>>> This following branch works for me:
>>>> for-6.18/ksmbd-smbdirect-regression-v7
>>>> and with the fixes again master this should also work:
>>>> for-6.19/ksmbd-smbdirect-regression-v1
>>>>
>>>> I'll post real patches tomorrow.
>>>>
>>>> Please check.
>>> Okay, I will test it with two branches.
>>> I'll try it too, but I recommend running frametest for performance
>>> difference and stress testing.
>>>
>>> https://support.dvsus.com/hc/en-us/articles/212925466-How-to-use-frametest
>>>
>>> ex) frametest.exe -w 4k -t 20 -n 2000
>>
>> That works fine, but
>>
>>   frametest.exe -r 4k -t 20 -n 2000
>>
>> generates a continues stream of such messages:
>> ksmbd: Failed to send message: -107
>>
>> Both with 6.17.2 and for-6.19/ksmbd-smbdirect-regression-v1,
>> so this is not a regression.
>>
>> I'll now check if the is related to the other problems
>> I found and fixes in for-6.18/ksmbd-smbdirect-regression-v5
> 
> Ok, I found the problem.
> 
> On send we are not allowed to consume the last send credit
> without granting any credit to the peer.
> 
>      MS-SMBD 3.1.5.1 Sending Upper Layer Messages
> 
>      ...
>      If Connection.SendCredits is 1 and the CreditsGranted field of the message is 0, stop
>      processing.
>      ...
> 
>      MS-SMBD 3.1.5.9 Managing Credits Prior to Sending
> 
>      ...
>      If Connection.ReceiveCredits is zero, or if Connection.SendCredits is one and the
>      Connection.SendQueue is not empty, the sender MUST allocate and post at least one receive of size
>      Connection.MaxReceiveSize and MUST increment Connection.ReceiveCredits by the number
>      allocated and posted. If no receives are posted, the processing MUST return a value of zero to indicate
>      to the caller that no Send message can be currently performed.
>      ...
> 
> It works in my master-ipproto-smbdirect branch, see the top commit.
> 
> I'll backport the related logic to ksmbd on top of
> for-6.19/ksmbd-smbdirect-regression-v1 tomorrow.

for-6.19/ksmbd-smbdirect-regression-v2 has the fixes and works for
me, I'll prepare official patches (most likely) on Monday.

metze