Re: Problem with smbdirect rw credits and initiator_depth

[Stefan Metzmacher <metze@samba.org>]

Am 05.12.25 um 13:21 schrieb Namjae Jeon:
>>> Can you at least post the dmesg output generated by this:
>>> https://git.samba.org/?p=metze/linux/wip.git;a=commitdiff;h=7e724ebc58e986f4e101a55f4ab5e96912239918
>>> Assuming that this wasn't triggered:
>>> if (WARN_ONCE(needed > max_possible, "needed:%u > max:%u\n", needed, max_possible))
>> I didn't know you wanted it. I will share it after office.
> I have attached v2 and v3 logs. Let me know if you need something more,
>>>
>>> Did you run the bpftrace command? Did it print a lot of
>>> 'smb_direct_rdma_xmit' message over the whole time of the file copy?
>> No, I didn't check it. but I will try this.
> /mnt# bpftrace ksmbd-rdma-xmit.bt
> Attaching 1 probe...
> 
> The absence of any output after Attaching 1 probe... indicates that
> the smb_direct_rdma_xmit function has not been called ?

Assuming the client requires signing, I may found the
reason for a recv credit problem.

ksmbd uses this:

smb_direct_max_fragmented_recv_size = 1024 * 1024
smb_direct_max_receive_size = 1364;
smb_direct_receive_credit_max = 255;

In order for the client to fill the full eassembly buffer,
all our recv buffers are moved into it, which means
255 * (1364 - 24) = 341700 (0x536C4) bytes of payload,
after that we no longer able to grant and new recv credits to
the peer, which tries to send up to 1048576 (0x100000).

I found this using smbclient to download a large file
from a Windows server without using rdma offload.

So I guess you are seeing the problem when Windows
tries to copy a file to ksmbd.

For smbclient I made it work by changing
max_fragmented_recv_size to the minimum value of
131072 (0x20000), this value is smaller than
all local recv buffers 255 * (1364 - 24) = 341700 (0x536C4).

I try to find what difference we have between 6.17.9
and 6.18 tomorrow.

In the meantime you may want to test if 6.18 with
smb_direct_max_fragmented_recv_size = 131072 works
for you, or change smb_direct_receive_credit_max = 1024.

metze