tests: CUDA DMA-BUF resources leave exported descriptors open

tests: CUDA DMA-BUF resources leave exported descriptors open

[Dezhen Lu]

Hello,

At rdma-core master a70151dfec390eb5f7fc5b266f51d601da2a7778,
the CUDA DMA-BUF test resources obtain an FD from
cuMemGetHandleForAddressRange but do not close it after registration.
This affects DmabufCudaRes, MRExDmabufCudaRes,
MRExDmabufDmaHCudaRes, and Mlx5DmabufCudaRes. It occurs on successful
registration and on failures or skips after the FD is obtained. The
integer FD passed to the pyverbs MR constructor is caller-owned.

The following uses a real process FD and a mocked constructor to
exercise the first resource method without requiring a CUDA GPU or RDMA
device. Run it from a checkout with an in-place pyverbs build:

```sh
PYTHONPATH=build/python:tests python3 - <<'PY'
import errno
import os
import sys
from types import SimpleNamespace
from unittest.mock import patch

sys.path.append(sys.path.pop(0))  # Prefer the built pyverbs over the source directory.
import tests.test_cuda_dmabuf as test

fd = os.open('/dev/null', os.O_RDONLY)
res = SimpleNamespace(pd=object(), msg_size=64, mr_access=1,
                      dmabuf_cuda_init=lambda: fd)
try:
    with patch.object(test, 'DmaBufMR', return_value=object()):
        test.DmabufCudaRes.create_mr(res)
    try:
        os.fstat(fd)
    except OSError as exc:
        if exc.errno != errno.EBADF:
            raise
        print('descriptor closed')
    else:
        print('descriptor still open')
finally:
    try:
        os.close(fd)
    except OSError as exc:
        if exc.errno != errno.EBADF:
            raise
PY
```

Observed: descriptor still open. Expected: descriptor closed after
registration returns, including on error. A proposed fix closes it in a
finally block around registration and, for the DMAHandle path, around
handle creation as well. On Linux the RDMA DMA-BUF registration path
acquires its own buffer reference.

I reproduced the behavior on Linux aarch64 with Python 3.12 and a local
rdma-core/pyverbs build. Mock-based FD tests cover the four registration
methods, unsupported and error paths, and DMAHandle creation failure;
they fail on the base and pass with the fix. No CUDA or RDMA traffic was
run for this comparison.

Re: tests: CUDA DMA-BUF resources leave exported descriptors open

[Leon Romanovsky]

On Wed, Nov 04, 2026 at 08:48:09PM +0100, Dezhen Lu wrote:
> Hello,
> 
> At rdma-core master a70151dfec390eb5f7fc5b266f51d601da2a7778,
> the CUDA DMA-BUF test resources obtain an FD from
> cuMemGetHandleForAddressRange but do not close it after registration.
> This affects DmabufCudaRes, MRExDmabufCudaRes,
> MRExDmabufDmaHCudaRes, and Mlx5DmabufCudaRes. It occurs on successful
> registration and on failures or skips after the FD is obtained. The
> integer FD passed to the pyverbs MR constructor is caller-owned.
> 
> The following uses a real process FD and a mocked constructor to
> exercise the first resource method without requiring a CUDA GPU or RDMA
> device. Run it from a checkout with an in-place pyverbs build:
> 
> ```sh
> PYTHONPATH=build/python:tests python3 - <<'PY'
> import errno
> import os
> import sys
> from types import SimpleNamespace
> from unittest.mock import patch
> 
> sys.path.append(sys.path.pop(0))  # Prefer the built pyverbs over the source directory.
> import tests.test_cuda_dmabuf as test
> 
> fd = os.open('/dev/null', os.O_RDONLY)
> res = SimpleNamespace(pd=object(), msg_size=64, mr_access=1,
>                       dmabuf_cuda_init=lambda: fd)
> try:
>     with patch.object(test, 'DmaBufMR', return_value=object()):
>         test.DmabufCudaRes.create_mr(res)
>     try:
>         os.fstat(fd)
>     except OSError as exc:
>         if exc.errno != errno.EBADF:
>             raise
>         print('descriptor closed')
>     else:
>         print('descriptor still open')
> finally:
>     try:
>         os.close(fd)
>     except OSError as exc:
>         if exc.errno != errno.EBADF:
>             raise
> PY
> ```
> 
> Observed: descriptor still open. Expected: descriptor closed after
> registration returns, including on error. A proposed fix closes it in a
> finally block around registration and, for the DMAHandle path, around
> handle creation as well. On Linux the RDMA DMA-BUF registration path
> acquires its own buffer reference.

Please open PR against rdma-core with your fix for both test reports.

Thanks