[PATCH 3/3] drm/amdgpu: program the SH_MEM registers of the SPM VMID

[Jesse Zhang <Jesse.Zhang@amd.com>]

Commit 02232367be23 ("drm/amdgpu: add spm_vmid and exclude it from the
MES, KFD and kq pools") took the SPM VMID out of the kernel queue and the
user queue VMID masks, and since commit 8a9263fc8812 ("drm/amdgpu: use
dedicated SPM VMID from the reserved path") the kernel queue jobs of a VM
with a reserved VMID run on it. The gfx constants_init() only programs
SH_MEM_CONFIG and SH_MEM_BASES for VMID 0 and the kernel queue mask
though, and init_compute_vmid() for the user queue mask, so the SPM VMID
keeps the reset defaults: no unaligned access mode and no apertures.

On navi48 VMID 15 has SH_MEM_CONFIG 0xc000 and SH_MEM_BASES 0 while the
other graphics VMIDs have 0xc00c and 0x20001000. A shader storing a dword
to an address with a byte offset of 1 writes it there in a normal
process, but at the dword aligned address in one which reserved a VMID,
e.g. for profiling with SPM:

  $ umr -i 0 -sb 0 0 0 15 -r '*.*.regSH_MEM_CONFIG'
  gfx1201.regSH_MEM_CONFIG => 0x0000c000
  $ amd_pmc --run-subtest reserved-vmid-unaligned-store
  unaligned store: 0x0000000000002a00 normally, 0x000000000000002a with a reserved VMID

Include the SPM VMID in for_each_vmid_and_zero() for the GFXHUB so the
constants_init() of all gfx generations program it like the other
kernel queue VMIDs.

Fixes: 8a9263fc8812 ("drm/amdgpu: use dedicated SPM VMID from the reserved path")
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h
index c134000eecfd..6d0f302c1344 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.h
@@ -45,14 +45,17 @@
  * Equivalent to OR-ing BIT(0) into id_mgr[hub].vmid_mask and walking
  * the result with for_each_set_bit(). Used by the gfx_v*_constants_init()
  * paths which must always touch VMID 0 (system) in addition to the regular
- * VMIDs owned by the hub.
+ * VMIDs owned by the hub. On the GFXHUB this includes the SPM VMID, which is
+ * kept out of the pool but runs the kernel queue jobs of the VM reserving it.
  *
  * Relies on AMDGPU_NUM_VMID (16) fitting in a single unsigned long, i.e.
  * vmid_mask[] being a 1-element bitmap.
  */
 #define for_each_vmid_and_zero(vmid, adev, hub)				\
 	for (unsigned long __vmid_mask =				\
-		(adev)->vm_manager.id_mgr[(hub)].vmid_mask[0] | BIT(0);	\
+		(adev)->vm_manager.id_mgr[(hub)].vmid_mask[0] | BIT(0) |	\
+		((hub) == AMDGPU_GFXHUB(0) && (adev)->vm_manager.spm_vmid ?	\
+		 BIT((adev)->vm_manager.spm_vmid) : 0);			\
 	     __vmid_mask; __vmid_mask = 0)				\
 		for_each_set_bit((vmid), &__vmid_mask, AMDGPU_NUM_VMID)
 
-- 
2.50.1