Re: [PATCH v7 6/9] vfio/pci: Convert BAR mmap() to use a DMABUF

[Alex Mastro <amastro@fb.com>]

I used [1] to validate that the problematic sequence that deadlocks
in v5 is resolved by this v7 series.

The test stands up a spoofed nvgrace device [2] in QEMU and runs a test [3]
which:

1. Launch reader thread: pread() from vfio device fd into a userfaultfd-managed
   page.
2. Wait until the uffd fires, but do not resolve the fault. Now pread()
   from 1. is blocked, holding memory_lock(R) inside nvgrace_gpu_read_mem().
3. Launch writer thread: pwrite() the PCI command word, queuing for
   memory_lock(W) inside vfio_basic_config_write().
4. Launch mmap thread: mmap() on the vfio device fd, taking mmap_lock(W)
   and triggering the internal dma-buf export that backs the mapping.
5. Resolve the uffd fault.
6. Join the threads.

On v5 the dma-buf export in step 4 takes memory_lock(W) and blocks behind the
reader and the queued writer. The reader's fault path nests mmap_lock(R) inside
memory_lock(R), while the mapper holds mmap_lock(W) across the mmap handler and
waits on memory_lock (lockdep reports the cycle).

On v7 the export takes only the new dmabuf_lock, which neither the reader nor
the writer holds, so step 4 completes while the uffd fault is unresolved.

Tested-by: Alex Mastro <amastro@fb.com>
Reviewed-by: Alex Mastro <amastro@fb.com>

[1] https://github.com/opsound/vfio-dmabuf-lab.git
[2] https://github.com/opsound/qemu/commit/349f0b12ca633fc42532aeda33273079b2b8e5d4
[3] https://github.com/opsound/vfio-dmabuf-lab/blob/cfe762123076c4543fdc4e8d5e9b62ad6bf998d9/tests/nvgrace_uaccess_test.c#L309

Alex