Re: [RESEND RFC PATCH v3 1/6] drm/amdgpu: Change AMDGPU_VA_RESERVED_TRAP_SIZE to 2 PAGE_SIZE pages

["Christian König" <christian.koenig@amd.com>]

On 3/23/26 12:50, Donet Tom wrote:
> 
> On 3/23/26 3:41 PM, Christian König wrote:
> 
> Hi Christian
> 
>> On 3/23/26 05:28, Donet Tom wrote:
>>> Currently, AMDGPU_VA_RESERVED_TRAP_SIZE is hardcoded to 8KB, while
>>> KFD_CWSR_TBA_TMA_SIZE is defined as 2 * PAGE_SIZE. On systems with
>>> 4K pages, both values match (8KB), so allocation and reserved space
>>> are consistent.
>>>
>>> However, on 64K page-size systems, KFD_CWSR_TBA_TMA_SIZE becomes 128KB,
>>> while the reserved trap area remains 8KB. This mismatch causes the
>>> kernel to crash when running rocminfo or rccl unit tests.
>>>
>>> Kernel attempted to read user page (2) - exploit attempt? (uid: 1001)
>>> BUG: Kernel NULL pointer dereference on read at 0x00000002
>>> Faulting instruction address: 0xc0000000002c8a64
>>> Oops: Kernel access of bad area, sig: 11 [#1]
>>> LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries
>>> CPU: 34 UID: 1001 PID: 9379 Comm: rocminfo Tainted: G E
>>> 6.19.0-rc4-amdgpu-00320-gf23176405700 #56 VOLUNTARY
>>> Tainted: [E]=UNSIGNED_MODULE
>>> Hardware name: IBM,9105-42A POWER10 (architected) 0x800200 0xf000006
>>> of:IBM,FW1060.30 (ML1060_896) hv:phyp pSeries
>>> NIP:  c0000000002c8a64 LR: c00000000125dbc8 CTR: c00000000125e730
>>> REGS: c0000001e0957580 TRAP: 0300 Tainted: G E
>>> MSR:  8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 24008268
>>> XER: 00000036
>>> CFAR: c00000000125dbc4 DAR: 0000000000000002 DSISR: 40000000
>>> IRQMASK: 1
>>> GPR00: c00000000125d908 c0000001e0957820 c0000000016e8100
>>> c00000013d814540
>>> GPR04: 0000000000000002 c00000013d814550 0000000000000045
>>> 0000000000000000
>>> GPR08: c00000013444d000 c00000013d814538 c00000013d814538
>>> 0000000084002268
>>> GPR12: c00000000125e730 c000007e2ffd5f00 ffffffffffffffff
>>> 0000000000020000
>>> GPR16: 0000000000000000 0000000000000002 c00000015f653000
>>> 0000000000000000
>>> GPR20: c000000138662400 c00000013d814540 0000000000000000
>>> c00000013d814500
>>> GPR24: 0000000000000000 0000000000000002 c0000001e0957888
>>> c0000001e0957878
>>> GPR28: c00000013d814548 0000000000000000 c00000013d814540
>>> c0000001e0957888
>>> NIP [c0000000002c8a64] __mutex_add_waiter+0x24/0xc0
>>> LR [c00000000125dbc8] __mutex_lock.constprop.0+0x318/0xd00
>>> Call Trace:
>>> 0xc0000001e0957890 (unreliable)
>>> __mutex_lock.constprop.0+0x58/0xd00
>>> amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu+0x6fc/0xb60 [amdgpu]
>>> kfd_process_alloc_gpuvm+0x54/0x1f0 [amdgpu]
>>> kfd_process_device_init_cwsr_dgpu+0xa4/0x1a0 [amdgpu]
>>> kfd_process_device_init_vm+0xd8/0x2e0 [amdgpu]
>>> kfd_ioctl_acquire_vm+0xd0/0x130 [amdgpu]
>>> kfd_ioctl+0x514/0x670 [amdgpu]
>>> sys_ioctl+0x134/0x180
>>> system_call_exception+0x114/0x300
>>> system_call_vectored_common+0x15c/0x2ec
>>>
>>> This patch changes AMDGPU_VA_RESERVED_TRAP_SIZE to 2 * PAGE_SIZE,
>>> ensuring that the reserved trap area matches the allocation size
>>> across all page sizes.
>>>
>>> cc: stable@vger.kernel.org
>>> Fixes: 34a1de0f7935 ("drm/amdkfd: Relocate TBA/TMA to opposite side of VM hole")
>>> Reviewed-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
>>> Signed-off-by: Donet Tom <donettom@linux.ibm.com>
>>> ---
>>>  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
>>> index 139642eacdd0..a5eae49f9471 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
>>> @@ -173,7 +173,7 @@ struct amdgpu_bo_vm;
>>>  #define AMDGPU_VA_RESERVED_SEQ64_SIZE		(2ULL << 20)
>>>  #define AMDGPU_VA_RESERVED_SEQ64_START(adev)	(AMDGPU_VA_RESERVED_CSA_START(adev) \
>>>  						 - AMDGPU_VA_RESERVED_SEQ64_SIZE)
>>> -#define AMDGPU_VA_RESERVED_TRAP_SIZE		(2ULL << 12)
>>> +#define AMDGPU_VA_RESERVED_TRAP_SIZE		(2ULL << PAGE_SHIFT)
>> Well using PAGE_SHIFT in amdgpu_vm.h looks quite broken to me.
>>
>> That makes the GPU VA reservation depend on the CPU page size and that is clearly not something we want to have.
>>
>> Where is KFD_CWSR_TBA_TMA_SIZE defined?
>>
> 
> Thanks Christian for reviewing this patch.
> 
> It is defined in kfd_priv.h.
> 
> /*
>  * Size of the per-process TBA+TMA buffer: 2 pages
>  *
>  * The first chunk is the TBA used for the CWSR ISA code. The second
>  * chunk is used as TMA for user-mode trap handler setup in daisy-chain mode.
>  */
> #define KFD_CWSR_TBA_TMA_SIZE (PAGE_SIZE * 2)
> 
> 
> 
> Could you please suggest the correct way to fix this issue?

I'm only looking from the POV of the VM code on this, but my educated guess is that KFD_CWSR_TBA_TMA_SIZE should be 8k independent of the CPU page size.

Background is that this is written by the shader trap handler and that byte code doesn't care what CPU architecture you have.

But I think only the engineers working on that trap handler can really answer this. @Felix / @Philip?

Regards,
Christian.

> 
> -Donet
> 
>> Regards,
>> Christian.
>>
>>>  #define AMDGPU_VA_RESERVED_TRAP_START(adev)	(AMDGPU_VA_RESERVED_SEQ64_START(adev) \
>>>  						 - AMDGPU_VA_RESERVED_TRAP_SIZE)
>>>  #define AMDGPU_VA_RESERVED_BOTTOM		(1ULL << 16)