Re: Locking issue between NFSv4 and SMB client

dai.ngo
2021-09-23T22:40:01Z

On 9/23/21 2:50 PM, Bruce Fields wrote:
> On Thu, Jul 15, 2021 at 04:45:22PM -0700, dai.ngo@oracle.com wrote:
>> Hi Bruce,
> Oops, sorry for neglecting this.
>
>> I'm doing some locking testing between NFSv4 and SMB client and
>> think there are some issues on the server that allows both clients
>> to lock the same file at the same time.
> It's not too surprising to me that getting consistent locks between the
> two would be hard.
>
> Did you get any review from a Samba expert? I seem to recall it having
> a lot of options, and I wonder if it's configured correctly for this
> case.

No, I have not heard from any Samba expert.

>
> It sounds like Samba may be giving out oplocks without getting a lease
> from the kernel.

I will have to circle back to this when we're done with the 1st
phase of courteous server.

-Dai

>
> --b.
>
>> Here is what I did:
>>
>> NOTE: lck is a simple program that use lockf(3) to lock a file from
>> offset 0 to the length specified by '-l'.
>>
>> On NFSv4 client
>> ---------------
>>
>> [root@nfsvmd07 ~]# nfsstat -m
>> /tmp/mnt from nfsvmf24:/root/smb_share
>> Flags: rw,relatime,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,
>> proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.80.62.47,
>> local_lock=none,addr=10.80.111.94
>> [root@nfsvmd07 ~]#
>>
>>
>> [root@nfsvmd07 ~]# ./lck -p /tmp/mnt/messages -W -l 100000000
>> Lck/file: 1, Maxlocks: 10000000
>> Locking[/tmp/mnt/messages] Offset[0] Len[100000000] N[0]...doing F_LOCK..
>> LOCKED...
>>
>> Locks[1] files[1] took[2.000s] sleep waiting...Hit Control-C to stop
>>
>> [NFS client successfully locks the file]
>>
>> On SMB client
>> -------------
>>
>> [root@nfsvme24 ~]# mount |grep cifs
>> //nfsvmf24/smb_share on /tmp/mnt type cifs (rw,relatime,vers=3.1.1,cache=strict,username=root,uid=0,noforceuid,gid=0,noforcegid,addr=10.80.111.94,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1)
>> [root@nfsvme24 ~]#
>>
>> [root@nfsvme24 ~]# smbclient -L nfsvmf24
>> Enter SAMBA\root's password:
>>
>> Sharename Type Comment
>> --------- ---- -------
>> print$ Disk Printer Drivers
>> smb_share Disk Test Samba Share <<===== share to mount
>> IPC$ IPC IPC Service (Samba 4.10.16)
>> root Disk Home Directories
>> Reconnecting with SMB1 for workgroup listing.
>>
>> Server Comment
>> --------- -------
>>
>> Workgroup Master
>> --------- -------
>> [root@nfsvme24 ~]#
>>
>> [root@nfsvme24 ~]# ./lck -p /tmp/mnt/messages -W -l 100000000
>> Lck/file: 1, Maxlocks: 10000000
>> Locking[/tmp/mnt/messages] Offset[0] Len[100000000] N[0]...doing F_LOCK..
>> LOCKED...
>>
>> Locks[1] files[1] took[2.000s] sleep waiting...Hit Control-C to stop
>>
>> [SMB client successfully locks the file]
>>
>> The same issue happens when either client locks the file first.
>> I think this is what has happened:
>>
>> 1. NFSv4 client opens and locks the file first
>>
>> . NFSv4 client send OPEN and LOCK to server, server replies
>> OK on both requests.
>>
>> . SMB client sends create request with Oplock==Lease for
>> the same file.
>>
>> . server holds off on replying to SMB client's create request,
>> recalls delegation from NFSv4 client, waits for NFSv4 client
>> to return the delegation then replies success to SMB client's
>> create request with lease granted (Oplock==Lease).
>>
>> NOTE: I think SMB server should replies the create request
>> with Oplock==None to force the SMB client to sends the
>> lock request.
>>
>> . Once SMB client receives the reply of the create with
>> 'Oplock==Lease', it assumes it has full control of the file
>> therefor it does not need to send the lock request.
>>
>> . both NFSv4 and SMB client now think they have locked the file.
>>
>> pcap: nfs_lock_smb_lock.pcap
>>
>> 2. SMB client creates the file with 'Oplock==Lease' first
>>
>> . SMB sends create request with 'Oplock==Lease' to server,
>> server replies OK with 'Oplock==Lease'. SMB client skips
>> sending lock request since it assumes it has full control
>> of the file with the lease.
>>
>> . NFSv4 client sends OPEN to server, server replies OK with
>> delagation is none. NFSv4 client sends LOCK request, since
>> no lock was created in the kernel for the SMB client, the
>> lock was granted to the NFSv4 client.
>>
>> NOTE: I think the SMB server should send lease break
>> notification to the SMB client, wait for the lease break
>> acknowledgment from SMB client before replying to the
>> OPEN of the NFSv4 client. This will force the SMB client
>> to send the lock request to the server.
>>
>> . both NFSv4 and SMB client now think they have locked the file.
>>
>> Your thought?
>>
>> Thanks,
>>
>> -Dai