Re: [PATCH RFC v6 05/26] nova-core: mm: Add support to use PRAMIN windows to write to VRAM

[Joel Fernandes <joelagnelf@nvidia.com>]

On 2/2/2026 4:12 AM, Christian König wrote:
> On 1/31/26 04:00, Dave Airlie wrote:
>> On Sat, 31 Jan 2026 at 07:14, Joel Fernandes <joelagnelf@nvidia.com> wrote:
>>> On 1/29/2026 10:38 PM, John Hubbard wrote:
[...]
>>>> For the deadlock above, I think a good way to break that deadlock is
>>>> to not allow taking that lock in a fence signaling calling path.
>>>>
>>>> So during an unmap, instead of "lock, unmap/free, unlock" it should
>>>> move the item to a deferred-free list, which is processed separately.
>>>> Of course, this is a little complex, because the allocation and reclaim
>>>> has to be aware of such lists if they get large.
>>> Yes, also avoiding GFP_KERNEL allocations while holding any of these mm locks
>>> (whichever we take during map). The GPU buddy actually does GFP_KERNEL
>>> allocations internally which is problematic.
>>>
>>> Some solutions / next steps:
>>>
>>> 1. allocating (VRAM and system memory) outside mm locks just before acquiring them.
>>>
>>> 2. pre-allocating both VRAM and system memory needed, before the DMA fence
>>> critical paths (The issue is also to figure out how much memory to pre-allocate
>>> for the page table pages based on the VM_BIND request. I think we can analyze
>>> the page tables in the submit stage to make an estimate).
>>>
>>> 3. Unfortunately, I am using gpu-buddy when allocating a VA range in the Vmm
>>> (called virt_buddy), which itself does GFP_KERNEL memory allocations in the
>>> allocate path. I am not sure what do yet about this. ISTR the maple tree also
>>> has similar issues.
>>>
>>> 4. Using non-reclaimable memory allocations where pre-allocation or
>>> pre-allocated memory pools is not possible (I'd like to avoid this #4 so we
>>> don't fail allocations when memory is scarce).
>>>
>>> Will work on these issues for the v7. Thanks,
>>
>> The way this works on nouveau at least (and I haven't yet read the
>> nova code in depth).
>>
>> Is we have 4 stages of vmm page table mgmt.
>>
>> ref - locked with a ref lock - can allocate/free memory - just makes
>> sure the page tables exist and are reference counted
>> map - locked with a map lock - cannot allocate memory - fill in the
>> PTEs in the page table
>> unmap - locked with a map lock - cannot allocate memory - removes
>> entries in PTEs
>> unref - locked with a ref lock - can allocate/free memory - just drops
>> references and frees (not sure if it ever merges).
> 
> On amdgpu VM page tables are allocated and PTEs filled outside of the fence critical path.

Does that really work for async VM_BIND? If we're missing anything in nova-core
related to the timing of when the allocate and update of the page tables, it
would be good to know.

My understanding you have to write the PTEs at the run stage of the job in
question otherwise you may not know how to map? Are you saying amdgpu writes it
during the run stage but somehow before fence signaling?

> 
> Only invalidating PTEs to signal that a shader needs to be taken off the HW are inside the fence critical path and here no memory allocation is needed.
> 
> Keep in mind that you not only need to avoid having memory allocations inside the critical path, but also not take locks under which memory is allocated.

Yes, this part I was clear from Danilo's email and clear about the various
deadlocks. See my analysis where what you mention is in the cases I covered:
https://lore.kernel.org/all/20e04a3e-8d7d-47bc-9299-deadf8b9e992@nvidia.com/

> Simona added some dma_fence_begin_signalling() and dma_fence_end_signalling() helpers to add lockdep annotations to the fence signaling path. Those have proven to be extremely useful since they allow lockdep to point out mistakes immediately and not just after hours of running on a test system.
> 
Yeah, I looked. Nice! Thanks,

--
Joel Fernandes