[PATCH] drm/amdkfd: Fix GPU mappings for APU after prefetch

[PATCH] drm/amdkfd: Fix GPU mappings for APU after prefetch

[Harish Kasiviswanathan]

Fix the following corner case:-
 Consider a 2M huge page SVM allocation, followed by prefetch call for
the first 4K page. The whole range is initially mapped with single PTE.
After the prefetch, this range gets split to first page + rest of the
pages. Currently, the first page mapping is not updated on MI300A (APU)
since page hasn't migrated. However, after range split PTE mapping it not
valid.

Fix this by forcing page table update for the whole range when prefetch
is called.  Calling prefetch on APU doesn't improve performance. If all
it deteriotes. However, functionality has to be supported.

Suggested-by: Philip Yang <Philip.Yang@amd.com>
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
---
 drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
index c30dfb8ec236..d3d4f0d01653 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
@@ -766,14 +766,21 @@ svm_range_apply_attrs(struct kfd_process *p, struct svm_range *prange,
 {
 	uint32_t i;
 	int gpuidx;
+	struct kfd_node *node;
 
 	for (i = 0; i < nattr; i++) {
 		switch (attrs[i].type) {
 		case KFD_IOCTL_SVM_ATTR_PREFERRED_LOC:
 			prange->preferred_loc = attrs[i].value;
+			node = svm_range_get_node_by_id(prange, attrs[i].value);
+			if (node && node->adev->flags & AMD_IS_APU && !p->xnack_enabled)
+				*update_mapping = true;
 			break;
 		case KFD_IOCTL_SVM_ATTR_PREFETCH_LOC:
 			prange->prefetch_loc = attrs[i].value;
+			node = svm_range_get_node_by_id(prange, attrs[i].value);
+			if (node && node->adev->flags & AMD_IS_APU && !p->xnack_enabled)
+				*update_mapping = true;
 			break;
 		case KFD_IOCTL_SVM_ATTR_ACCESS:
 		case KFD_IOCTL_SVM_ATTR_ACCESS_IN_PLACE:
-- 
2.34.1

Re: [PATCH] drm/amdkfd: Fix GPU mappings for APU after prefetch

[Philip Yang]

On 2025-10-28 21:31, Harish Kasiviswanathan wrote:
> Fix the following corner case:-
>   Consider a 2M huge page SVM allocation, followed by prefetch call for
> the first 4K page. The whole range is initially mapped with single PTE.
> After the prefetch, this range gets split to first page + rest of the
> pages. Currently, the first page mapping is not updated on MI300A (APU)
> since page hasn't migrated. However, after range split PTE mapping it not
> valid.
>
> Fix this by forcing page table update for the whole range when prefetch
> is called.  Calling prefetch on APU doesn't improve performance. If all
> it deteriotes. However, functionality has to be supported.
>
> Suggested-by: Philip Yang <Philip.Yang@amd.com>
> Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
> ---
>   drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 7 +++++++
>   1 file changed, 7 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
> index c30dfb8ec236..d3d4f0d01653 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
> @@ -766,14 +766,21 @@ svm_range_apply_attrs(struct kfd_process *p, struct svm_range *prange,
>   {
>   	uint32_t i;
>   	int gpuidx;
> +	struct kfd_node *node;
>   
>   	for (i = 0; i < nattr; i++) {
>   		switch (attrs[i].type) {
>   		case KFD_IOCTL_SVM_ATTR_PREFERRED_LOC:
>   			prange->preferred_loc = attrs[i].value;
> +			node = svm_range_get_node_by_id(prange, attrs[i].value);
> +			if (node && node->adev->flags & AMD_IS_APU && !p->xnack_enabled)

Maybe check adev->apu_prefer_gtt, seems specific for MI300A? adev->flags 
& AMD_IS_APU includes MI300A and system with caved out VRAM, on which 
prefetch will trigger migration.

Regards,

Philip

> +				*update_mapping = true;
>   			break;
>   		case KFD_IOCTL_SVM_ATTR_PREFETCH_LOC:
>   			prange->prefetch_loc = attrs[i].value;
> +			node = svm_range_get_node_by_id(prange, attrs[i].value);
> +			if (node && node->adev->flags & AMD_IS_APU && !p->xnack_enabled)
> +				*update_mapping = true;
>   			break;
>   		case KFD_IOCTL_SVM_ATTR_ACCESS:
>   		case KFD_IOCTL_SVM_ATTR_ACCESS_IN_PLACE: