Parallel shared to exclusive flock conversion blocks forever on single NFS client

[Tycho Kirchner <tychokirchner@mail.de>]

Dear NFS kernel developers,
In `man 2 flock` it is documented, that an existing lock can be 
converted to a new lock mode. Multiple processes on the *same* client 
converting their LOCK_SH to LOCK_EX quickly results in a deadlock of the 
client processes. This can already be reproduced on a single physical 
machine, with for instance the NFS server running in a VM and the host 
machine connecting to it as a client.

Steps to reproduce:
- Setup a virtual machine with Virtualbox and install NFS-server
- Create an /etc/export: /home/VMUSER/nfs  10.0.2.2(rw,async)
- Create a NAT firewall rule forwarding NFS port 2049 to the VM
- Mount the export on the host, chdir it and create an empty file:
   $ sudo mount -t nfs 127.0.0.1:/home/VMUSER/nfs  /somedir
   $ cd /somedir
   $ touch foo
- Execute below attached ~/locktest.py in parallel on the client:
   $ for i in {1..10}; do ~/locktest.py foo & done; wait
- Wait half a minute. The command does not terminate. Ever.
- Abort execution with Ctrl+C and kill leftovers: pkill -f locktest.py

Notes:
- According to my tests, from three concurrent client-processes onwards, 
the block quickly occurs.
- Placing a `fcntl.flock(a, fcntl.LOCK_UN)` before fcntl.LOCK_EX is 
enough, so the deadlock never occurs.
- OR'ing `| fcntl.LOCK_NB` quickly results in endless »BlockingIOError« 
exceptions with no client process making any progress. See the also 
attached ~/locktest_NB.py.
- Multiple distributions, Kernelversions and combinations tested, e.g. 
NFS-client KVER 6.6.67 on Debian12 and KVER 6.12.17-amd64 on 
DebianTesting, or KVER 6.4.0-150600.23.38-default on openSUSE Leap 15.6. 
The error was always and quickly reproducible.

Kind regards
Tycho

###___ ~/locktest.py ___###

#!/usr/bin/env python3
import fcntl
import sys
import time

a = open(sys.argv[1], 'r+')
fcntl.flock(a, fcntl.LOCK_SH)
fcntl.flock(a, fcntl.LOCK_EX)
time.sleep(1)

___________________________


###___ ~/locktest_NB.py ___###

#!/usr/bin/env python3
import fcntl
import sys
import time

def lock_nb(lockfile, l_mode):
     for i in range(20):
         try:
             fcntl.flock(lockfile.fileno(), l_mode | fcntl.LOCK_NB)
             return
         except BlockingIOError as e:
             time.sleep(1)
             continue
     print("gave up waiting for lock...", file=sys.stderr)

a = open(sys.argv[1], 'r+')
lock_nb(a, fcntl.LOCK_SH)
lock_nb(a, fcntl.LOCK_EX)
time.sleep(1)