All of lore.kernel.org
 help / color / mirror / Atom feed
From: Felix Kuehling <felix.kuehling@amd.com>
To: William Palacek <William.Palacek@amd.com>, amd-gfx@lists.freedesktop.org
Cc: Harish.Kasiviswanathan@amd.com, christian.koenig@amd.com,
	philip.yang@amd.com
Subject: Re: [PATCH 1/4] drm/amdgpu: limit userptr sg segments to half of sg->length
Date: Fri, 20 Nov 2026 16:28:24 -0500	[thread overview]
Message-ID: <9dacfc65-7554-466e-bb71-63b853386278@amd.com> (raw)
In-Reply-To: <5a6eb18599b44e0457abdc9d54a00dbb0060f10c.1795014593.git.William.Palacek@amd.com>


On 2026-11-18 10:10, William Palacek wrote:
> Userptr sg tables are built with sg_alloc_table_from_pages(), which
> merges physically contiguous pages into segments of up to
> 0xfffff000 bytes. iommu_dma_map_sg() pads a segment so that the next
> one does not cross the DMA segment boundary, and adds the padding to
> the 32-bit sg->length. With two large contiguous segments the padded
> length reaches 4 GiB and wraps to 0.
>
> On kernels without commit 1c8f6308e557 ("iommu/dma: Catch scatterlist
> length overflows") the wrapped length makes iommu_map_sg() map less

I recommend you don't mention upstream bugs that are already fixed, 
since this commit is meant for upstream. Your patch seems to be 
applicable regardless.


> than the allocated IOVA range. The IOVA is then freed without
> unmapping it, and a later map at that address fails with -EADDRINUSE.
> On kernels with that commit, iommu_dma_map_sg() returns -EOVERFLOW
> instead. In both cases a large userptr allocation fails to pin. On
> MI300X with the default DMA-FQ IOMMU domain, the first 32 GiB
> hipHostMalloc after boot built 16 segments of up to 3.7 GiB, mapped
> only 15.7 GiB and failed.
>
> The padding is shorter than the next segment, so a padded segment
> still fits sg->length when no segment exceeds half of it.

This statement doesn't make sense. sg->length is per scatterlist entry. 
Do you mean "no unpadded segment exceeds half of the 4GB maximum size"?

The commit headline is equally confusing and should probably also be 
reworded.


>   Limit both
> userptr sg tables to that, as xe does in commit 1b1d3710380d
> ("drm/xe: Apply upper limit to sg element size").

The XE commit also change a call to dma_set_max_seg_size. You're leaving 
the call in amdgpu_ttm_init unchanged with UINT_MAX. Is that an oversight?

Regards,
   Felix


>
> Signed-off-by: William Palacek <William.Palacek@amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 8 ++++----
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c          | 7 ++++---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h          | 7 +++++++
>   3 files changed, 15 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> index f060b206c255..565ecd7e012a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> @@ -589,10 +589,10 @@ kfd_mem_dmamap_userptr(struct kgd_mem *mem,
>   		return -ENOMEM;
>   
>   	/* Same sequence as in amdgpu_ttm_tt_pin_userptr */
> -	ret = sg_alloc_table_from_pages(ttm->sg, src_ttm->pages,
> -					ttm->num_pages, 0,
> -					(u64)ttm->num_pages << PAGE_SHIFT,
> -					GFP_KERNEL);
> +	ret = sg_alloc_table_from_pages_segment(ttm->sg, src_ttm->pages,
> +						ttm->num_pages, 0,
> +						(u64)ttm->num_pages << PAGE_SHIFT,
> +						AMDGPU_SG_MAX_SEGMENT, GFP_KERNEL);
>   	if (unlikely(ret))
>   		goto free_sg;
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index 8dfa92967b63..b0672f1747ee 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -837,9 +837,10 @@ static int amdgpu_ttm_tt_pin_userptr(struct ttm_device *bdev,
>   	int r;
>   
>   	/* Allocate an SG array and squash pages into it */
> -	r = sg_alloc_table_from_pages(ttm->sg, ttm->pages, ttm->num_pages, 0,
> -				      (u64)ttm->num_pages << PAGE_SHIFT,
> -				      GFP_KERNEL);
> +	r = sg_alloc_table_from_pages_segment(ttm->sg, ttm->pages,
> +					      ttm->num_pages, 0,
> +					      (u64)ttm->num_pages << PAGE_SHIFT,
> +					      AMDGPU_SG_MAX_SEGMENT, GFP_KERNEL);
>   	if (r)
>   		goto release_sg;
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
> index 9856a2fc2d08..da7d7621aad1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
> @@ -44,6 +44,13 @@
>   
>   #define AMDGPU_GTT_MAX_TRANSFER_SIZE	(1ULL << 22)
>   
> +/*
> + * iommu_dma_map_sg() pads a segment so that the next one does not cross the
> + * DMA segment boundary. The padding is shorter than the next segment, so the
> + * padded length still fits sg->length if no segment exceeds half of it.
> + */
> +#define AMDGPU_SG_MAX_SEGMENT	round_down(UINT_MAX / 2, PAGE_SIZE)
> +
>   extern const struct attribute_group amdgpu_vram_mgr_attr_group;
>   extern const struct attribute_group amdgpu_gtt_mgr_attr_group;
>   

  reply	other threads:[~2026-11-20 21:28 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 15:10 [PATCH 0/4] drm/amdkfd: don't stall other threads during large userptr registration William Palacek
2026-11-18 15:10 ` [PATCH 1/4] drm/amdgpu: limit userptr sg segments to half of sg->length William Palacek
2026-11-20 21:28   ` Felix Kuehling [this message]
2026-11-18 15:10 ` [PATCH 2/4] drm/amdkfd: don't take p->mutex in AMDKFD_IOC_GET_CLOCK_COUNTERS William Palacek
2026-11-20 21:35   ` Felix Kuehling
2026-11-18 15:10 ` [PATCH 3/4] drm/amdgpu: drop mmap_lock between chunks when faulting userptr pages William Palacek
2026-11-20 21:48   ` Felix Kuehling
2026-11-18 15:10 ` [PATCH 4/4] drm/amdkfd: register userptr pages without holding KFD process locks William Palacek
2026-11-19 14:26 ` [PATCH 0/4] drm/amdkfd: don't stall other threads during large userptr registration Christian König

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9dacfc65-7554-466e-bb71-63b853386278@amd.com \
    --to=felix.kuehling@amd.com \
    --cc=Harish.Kasiviswanathan@amd.com \
    --cc=William.Palacek@amd.com \
    --cc=amd-gfx@lists.freedesktop.org \
    --cc=christian.koenig@amd.com \
    --cc=philip.yang@amd.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.