Re: [RFC PATCH] misc: fgds: enable GPU-NVMe direct I/O via POSIX and io_uring

[Pavel Begunkov <asml.silence@gmail.com>]

On 9/10/26 04:57, Li Wang wrote:
> Hi Christian,
> 
>>>>> Nor why "fgds" is the name, that's going to be hard to remember, does it
>>>>> stand for something?
>>>>>
>>>> "FGDS" stands for Fast GPUDirect Storage. GPUDirect Storage (GDS) is NVIDIA's
>>>> technology enabling direct I/O between GPU memory and files on NVMe,
>>>> widely used in LLM workloads to bypass CPU overhead.
>>>
>>> That's nvidia's specific solution, but this works on other devices,
>>> right?  Or just for that one platform?
>>
>> That was nvidia's specific and very hacky out of tree solution
> Exactly, I completely agree with your viewpoint. In fact, we previously conducted
> a deep analysis of NVIDIA's GDS implementation code, which was also one of the
> motivations for us to develop fgds.
> 
> Please allow me to introduce NVIDIA's GDS implementation briefly:
> NVIDIA introduced two kernel modules: one module called nvidia-fs, and the other module
> is a customized NVMe driver replacing the Linux kernel's default NVMe driver.
> nvidia-fs creates a device file for every NVIDIA GPU on the machine: /dev/nvidia-fs<gpu-id>.

I'm not an nvidia expert, but IIRC they switched to P2PDMA, which
is supported by the upstream nvme driver.

-- 
Pavel Begunkov