Re: [netfs/cifs - Linux 6.14] loop on file cat + file copy when files are on CIFS share

[Paulo Alcantara <pc@manguebit.com>]

Hi Nicolas,

Thanks for the very detailed information and testing.

Nicolas Baranger <nicolas.baranger@3xo.fr> writes:

> In fact, I think there is somethings wrong:
>
> After a remount, I sucessfully get the good buffers size values in 
> /proc/mounts (those defined in /etc/fstab).
>
> grep cifs /proc/mounts
> //10.0.10.100/FBX24T /mnt/fbx/FBX-24T cifs 
> rw,nosuid,nodev,noexec,relatime,vers=3.1.1,cache=none,upcall_target=app,username=*****,domain=*****,uid=0,noforceuid,gid=0,noforcegid,addr=10.0.10.100,file_mode=0666,dir_mode=0755,iocharset=utf8,soft,nounix,serverino,mapposix,mfsymlinks,reparse=nfs,rsize=4194304,wsize=4194304,bsize=16777216,retrans=1,echo_interval=60,actimeo=1,closetimeo=1 
> 0 0

Interesting.  When you do 'mount -o remount ...' but don't pass rsize=
and wsize=, the client is suppposed to reuse the existing values of
rsize and wsize set in the current superblock.  The above values of
rsize, wsize and bsize are also the default ones in case you don't pass
them at all.

I'll look into that when time allows it.

> But here is what I constat: a 'dd' with a block size smaller than 65536 
> is working fine:
> LANG=en_US.UTF-8
>
> dd if=/dev/urandom of=/mnt/fbx/FBX-24T/dd.test3 bs=65536 status=progress 
> conv=notrunc oflag=direct count=128
> 128+0 records in
> 128+0 records out
> 8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.100398 s, 83.6 MB/s
>
>
>
> But a 'dd' with a block size bigger than 65536 is not working:
> LANG=en_US.UTF-8
>
> dd if=/dev/urandom of=/mnt/fbx/FBX-24T/dd.test3 bs=65537 status=progress 
> conv=notrunc oflag=direct count=128
> dd: error writing '/mnt/fbx/FBX-24T/dd.test3'
> dd: closing output file '/mnt/fbx/FBX-24T/dd.test3': Invalid argument
>
> And kernel report:
> Apr 24 10:01:37 14RV-SERVER.14rv.lan kernel: CIFS: VFS: \\10.0.10.100 
> Error -32 sending data on socket to server

This seems related to unaligned DIO reads and writes.  With O_DIRECT,
the client will set FILE_NO_INTERMEDIATE_BUFFERING when opening the
file, telling the server to not do any buffering when reading from or
writing to the file.  Some servers will fail the read or write request
if the file offset or length isn't a multiple of block size, where the
block size is >= 512 && <= PAGE_SIZE, as specified in MS-FSA 2.1.5.[34].

Since you're passing bs= with a value that is not multiple of block
size, then the server is failing the request with
STATUS_INVALID_PARAMETER as specified in MS-FSA.

I've tested it against Windows Server 2022 and it seems to enforce the
alignment only for DIO reads.  While samba doesn't enforce it at all.

win2k22:

$ dd if=/mnt/1/foo of=/dev/null status=none iflag=direct count=128 bs=65536
$ dd if=/mnt/1/foo of=/dev/null status=none iflag=direct count=128 bs=65537
dd: error reading '/mnt/1/foo': Invalid argument
$ dd if=/mnt/1/foo of=/dev/null status=none iflag=direct count=128 bs=$((65536+512))

$ xfs_io -d -f -c "pread 0 4096" /mnt/1/foo
read 4096/4096 bytes at offset 0
4 KiB, 1 ops; 0.0009 sec (4.260 MiB/sec and 1090.5125 ops/sec)
$ xfs_io -d -f -c "pread 1 4096" /mnt/1/foo
pread: Invalid argument

samba:

$ dd if=/mnt/1/foo of=/dev/null status=none iflag=direct count=128 bs=65536
$ dd if=/mnt/1/foo of=/dev/null status=none iflag=direct count=128 bs=65537
$ dd if=/mnt/1/foo of=/dev/null status=none iflag=direct count=128 bs=$((65536+512))

$ xfs_io -d -f -c "pread 0 4096" /mnt/1/foo
read 4096/4096 bytes at offset 0
4 KiB, 1 ops; 0.0071 sec (557.880 KiB/sec and 139.4700 ops/sec)
$ xfs_io -d -f -c "pread 1 4096" /mnt/1/foo
read 4096/4096 bytes at offset 1
4 KiB, 1 ops; 0.0010 sec (3.864 MiB/sec and 989.1197 ops/sec)

Note that the netfslib fix is for short DIO reads, so this bug is
related to unaligned DIO reads and writes and need to be fixed in the
client.  I'll let you know when I have patches for that.