Re: Problem with smbdirect rw credits and initiator_depth

[Stefan Metzmacher <metze@samba.org>]

Am 15.01.26 um 10:50 schrieb Stefan Metzmacher:
> Am 15.01.26 um 03:01 schrieb Namjae Jeon:
>> On Thu, Jan 15, 2026 at 3:13 AM Stefan Metzmacher <metze@samba.org> wrote:
>>>
>>> Am 15.12.25 um 21:17 schrieb Stefan Metzmacher:
>>>> Am 14.12.25 um 23:56 schrieb Stefan Metzmacher:
>>>>> Am 13.12.25 um 03:14 schrieb Namjae Jeon:
>>>>>>> I've put these changes a long with rw credit fixes into my
>>>>>>> for-6.18/ksmbd-smbdirect-regression-v4 branch, are you able to
>>>>>>> test this?
>>>>>> Problems still occur. See:
>>>>>
>>>>> :-( Would you be able to use rxe and cake a network capture?
>>>>>
>>>>> Using test files with all zeros, e.g.
>>>>> dd if=/dev/zero of=/tmp/4096MBzeros-sparse.dat seek=4096MB bs=1 count=1
>>>>> would allow gzip --best on the capture file to compress well...
>>>>
>>>> I think I found something that explains it and
>>>> I was able to reproduce and what I have in mind.
>>>>
>>>> We increment recv_io.posted.count after ib_post_recv()
>>>>
>>>> And manage_credits_prior_sending() uses
>>>>
>>>> new_credits = recv_io.posted.count - recv_io.credits.count
>>>>
>>>> But there is a race between the hardware receiving a message
>>>> and recv_done being called in order to decrement recv_io.posted.count
>>>> again. During that race manage_credits_prior_sending() might grant
>>>> too much credits.
>>>>
>>>> Please test my for-6.18/ksmbd-smbdirect-regression-v5 branch,
>>>> I haven't tested this branch yet, I'm running out of time
>>>> for the day.
>>>>
>>>> But I tested it with smbclient and having a similar
>>>> logic in fs/smb/common/smbdirect/smbdirect_connection.c
>>>
>>> I was able to reproduce the problem and the fix I created
>>> for-6.18/ksmbd-smbdirect-regression-v5 was not correct.
>>>
>>> I needed to use
>>>
>>> available = atomic_xchg(&sc->recv_io.credits.available, 0);
>>>
>>> instead of
>>>
>>> available = atomic_read(&sc->recv_io.credits.available);
>>> atomic_sub(new_credits, &sc->recv_io.credits.available);
>>>
>>> This following branch works for me:
>>> for-6.18/ksmbd-smbdirect-regression-v7
>>> and with the fixes again master this should also work:
>>> for-6.19/ksmbd-smbdirect-regression-v1
>>>
>>> I'll post real patches tomorrow.
>>>
>>> Please check.
>> Okay, I will test it with two branches.
>> I'll try it too, but I recommend running frametest for performance
>> difference and stress testing.
>>
>> https://support.dvsus.com/hc/en-us/articles/212925466-How-to-use-frametest
>>
>> ex) frametest.exe -w 4k -t 20 -n 2000
> 
> That works fine, but
> 
>   frametest.exe -r 4k -t 20 -n 2000
> 
> generates a continues stream of such messages:
> ksmbd: Failed to send message: -107
> 
> Both with 6.17.2 and for-6.19/ksmbd-smbdirect-regression-v1,
> so this is not a regression.
> 
> I'll now check if the is related to the other problems
> I found and fixes in for-6.18/ksmbd-smbdirect-regression-v5

Ok, I found the problem.

On send we are not allowed to consume the last send credit
without granting any credit to the peer.

     MS-SMBD 3.1.5.1 Sending Upper Layer Messages

     ...
     If Connection.SendCredits is 1 and the CreditsGranted field of the message is 0, stop
     processing.
     ...

     MS-SMBD 3.1.5.9 Managing Credits Prior to Sending

     ...
     If Connection.ReceiveCredits is zero, or if Connection.SendCredits is one and the
     Connection.SendQueue is not empty, the sender MUST allocate and post at least one receive of size
     Connection.MaxReceiveSize and MUST increment Connection.ReceiveCredits by the number
     allocated and posted. If no receives are posted, the processing MUST return a value of zero to indicate
     to the caller that no Send message can be currently performed.
     ...

It works in my master-ipproto-smbdirect branch, see the top commit.

I'll backport the related logic to ksmbd on top of
for-6.19/ksmbd-smbdirect-regression-v1 tomorrow.

metze