All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Chen, Xiaogang" <xiaogang.chen@amd.com>
To: Philip Yang <yangp@amd.com>, amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH v3 1/3] drm/amdkfd: Add awareness of THP of device and system RAM in kfd svm driver
Date: Mon, 16 Nov 2026 09:54:20 -0600	[thread overview]
Message-ID: <b6c80e1b-e6ca-4eac-9fe7-8f67e766192c@amd.com> (raw)
In-Reply-To: <2740efce-be38-4a82-98cf-4ddb6cc1212f@amd.com>


On 11/13/2026 5:28 PM, Philip Yang wrote:
>
>
> On 2026-10-19 09:38, Xiaogang.Chen wrote:
>> From: Xiaogang Chen<xiaogang.chen@amd.com>
>>
>> Extend kfd/svm functions to allocate HPAGE_PMD_SIZE based device 
>> memory by
>> buddy allocator, each drm_buddy_block is HPAGE_PMD_SIZE aligned; and to
>> allocate THP system ram by vma_alloc_folio.
>>
>> Introduce SVM_RANGE_DMA_THP flag that indicates dma map of THP. THP dma
>> addresss will use this flag.
>>
>> Add dev_pagemap_ops->folio_split callback that is called by 
>> folio_split when
>> core MM splits device memory folio.
>>
>> Reset compound folio of device memory after it got released from core 
>> HMM.
>> So, TTM can reuse this VRAM as 4K pages on next range.
>>
>> These are preparations for following support for (THP) migration of zone
>> device-private memory, no function change.
>>
>> Signed-off-by: Xiaogang Chen<xiaogang.chen@amd.com>
>> ---
>>   drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 84 ++++++++++++++++++++----
>>   drivers/gpu/drm/amd/amdkfd/kfd_svm.c     | 37 +++++++++--
>>   drivers/gpu/drm/amd/amdkfd/kfd_svm.h     |  1 +
>>   3 files changed, 104 insertions(+), 18 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c 
>> b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
>> index 8b96e1fe741a..7a129c9954fc 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
>> @@ -218,17 +218,45 @@ svm_migrate_addr_to_pfn(struct amdgpu_device 
>> *adev, unsigned long addr)
>>       return (addr + adev->kfd.pgmap.range.start) >> PAGE_SHIFT;
>>   }
>>   +/*
>> + * TTM can reuse a 2MB VRAM block that was previously a device THP.
>> + * Reset unused pages so the next 4K init does not see leftover 
>> head/tail.
>> + */
>> +static void svm_migrate_clear_compound_pages(struct page *page, 
>> unsigned int nr,
>> +                          struct dev_pagemap *pgmap)
>> +{
>> +    unsigned int i;
>> +
>> +    for (i = 0; i < nr; i++, page++) {
>> +        struct folio *new_folio = (struct folio *)page;
>> +
>> +        if (page_count(page))
>> +            continue;
>> +
>> +        page->flags.f &= ~0xffUL;
>> +#ifdef NR_PAGES_IN_LARGE_FOLIO
>> +        ((struct folio *)(page - 1))->_nr_pages = 0;
>> +#endif
>> +        new_folio->mapping = NULL;
>> +        new_folio->pgmap = pgmap;
>> +        new_folio->share = 0;
>> +        page->zone_device_data = NULL;
>> +    }
>> +}
>> +
> This is called after the device page is freed. When migrating, 
> svm_migrate_get_vram_page() uses the VRAM page and 
> zone_device_folio_init() does the same initialization, so this looks 
> duplicated. Is it?
svm_migrate_clear_compound_pages() runs because freeing the device THP 
only releases the head folio. The other 511 struct pages stay linked to 
that head, and TTM can hand those same VRAM pages out later as ordinary 
4K pages.

free_zone_device_folio() clears folio->mapping on the head and calls 
folio_free. It does not walk the tails. Each tail still has its 
compound-tail marker, so page_folio() follows it back to the old head, 
and page_pgmap() is implemented as page_folio(page)->pgmap. The tail's 
own pgmap slot held the head pointer, not the dev_pagemap.

The next 4K migration initializes that exact pfn:

svm_migrate_get_vram_page(struct svm_range *prange, unsigned long pfn,
                int order)
{
     struct page *page = pfn_to_page(pfn);

     /* Init this pfn, not page_folio() ? that follows a leftover tail 
to the old head. */
     zone_device_folio_init((struct folio *)page, page_pgmap(page), order);

page_pgmap() runs before that init. On a leftover tail it returns the 
old head's pgmap, or a stale one if that head was already reused.

The clear walks every page of the freed folio, drops the head/tail flag 
bits and _nr_pages, and stores the real pgmap on each page. After that, 
each page is a standalone zone-device page and the next order-0 init 
does not follow the old THP.

>>   static void
>> -svm_migrate_get_vram_page(struct svm_range *prange, unsigned long pfn)
>> +svm_migrate_get_vram_page(struct svm_range *prange, unsigned long pfn,
>> +               int order)
>>   {
>> -    struct page *page;
>> +    struct page *page = pfn_to_page(pfn);
>>   -    page = pfn_to_page(pfn);
>> -    svm_range_bo_ref(prange->svm_bo);
>> +    /* Init this pfn, not page_folio() — that follows a leftover 
>> tail to the old head. */
>> +    zone_device_folio_init((struct folio *)page, page_pgmap(page), 
>> order);
>>       page->zone_device_data = prange->svm_bo;
>> -    zone_device_page_init(page, page_pgmap(page), 0);
>> +    svm_range_bo_ref(prange->svm_bo);
>>   }
>>   +
>>   static void
>>   svm_migrate_put_vram_page(struct amdgpu_device *adev, unsigned long 
>> addr)
>>   {
>> @@ -249,11 +277,17 @@ svm_migrate_addr(struct amdgpu_device *adev, 
>> struct page *page)
>>   }
>>     static struct page *
>> -svm_migrate_get_sys_page(struct vm_area_struct *vma, unsigned long 
>> addr)
>> +svm_migrate_get_sys_page(struct vm_area_struct *vma, unsigned long 
>> addr,
>> +              unsigned long order)
>>   {
>>       struct page *page;
>>   -    page = alloc_page_vma(GFP_HIGHUSER, vma, addr);
>> +    if (order)
>> +        page = folio_page(vma_alloc_folio(GFP_HIGHUSER,
>> +                          order, vma, addr), 0);
> vma_alloc_folio could fail, then folio_page has NULL access, should 
> check vma_alloc_folio return value. if vma_alloc_folio() fails for 
> order 9, this fails the entire migration with -ENOMEM and the CPU 
> fault becomes SIGBUS. migrate_vma_pages() already handles a compound 
> source with a non-compound destination by splitting the source, so 
> falling back to a 4K page here would be enough. Where is that handled?
yes, that needs fallback to alloc_page_vma if alloc sys THP fail.
>> +    else
>> +        page = alloc_page_vma(GFP_HIGHUSER, vma, addr);
>> +
>>       if (page)
>>           lock_page(page);
>>   @@ -267,8 +301,13 @@ static unsigned long 
>> svm_migrate_successful_pages(struct migrate_vma *migrate)
>>         for (i = 0; i < migrate->npages; i++) {
>>           if (migrate->dst[i] & MIGRATE_PFN_VALID &&
>> -            migrate->src[i] & MIGRATE_PFN_MIGRATE)
>> -            mpages++;
>> +            migrate->src[i] & MIGRATE_PFN_MIGRATE) {
>> +
>> +            if (migrate->dst[i] & MIGRATE_PFN_COMPOUND)
>> +                mpages += HPAGE_PMD_NR;
>> +            else
>> +                mpages++;
>> +        }
>>       }
>>       return mpages;
>>   }
>> @@ -302,7 +341,7 @@ svm_migrate_copy_to_vram(struct kfd_node *node, 
>> struct svm_range *prange,
>>           if (migrate->src[i] & MIGRATE_PFN_MIGRATE) {
>>               dst[i] = cursor.start + (j << PAGE_SHIFT);
>>               migrate->dst[i] = svm_migrate_addr_to_pfn(adev, dst[i]);
>> -            svm_migrate_get_vram_page(prange, migrate->dst[i]);
>> +            svm_migrate_get_vram_page(prange, migrate->dst[i], 0);
>>               migrate->dst[i] = migrate_pfn(migrate->dst[i]);
>>               mpages++;
>>           }
>> @@ -616,15 +655,22 @@ svm_migrate_ram_to_vram(struct svm_range 
>> *prange, uint32_t best_loc,
>>       return r < 0 ? r : 0;
>>   }
>>   +/* folio can be compound folio or single page */
>>   static void svm_migrate_folio_free(struct folio *folio)
>>   {
>>       struct page *page = &folio->page;
>>       struct svm_range_bo *svm_bo = page->zone_device_data;
>> +    struct dev_pagemap *pgmap = folio->pgmap;
>> +    unsigned int order = folio_order(folio);
>>         if (svm_bo) {
>>           pr_debug_ratelimited("ref: %d\n", kref_read(&svm_bo->kref));
>>           svm_range_bo_unref_async(svm_bo);
>>       }
>> +
>> +    /* So TTM can reuse this VRAM as 4K pages on the next range. */
>> +    if (order)
>> +        svm_migrate_clear_compound_pages(page, 1U << order, pgmap);
>>   }
>>     static int
>> @@ -678,7 +724,7 @@ svm_migrate_copy_to_ram(struct amdgpu_device 
>> *adev, struct svm_range *prange,
>>               j = 0;
>>           }
>>   -        dpage = svm_migrate_get_sys_page(migrate->vma, addr);
>> +        dpage = svm_migrate_get_sys_page(migrate->vma, addr, 0);
>>           if (!dpage) {
>>               pr_debug("failed get page svms 0x%p [0x%lx 0x%lx]\n",
>>                    prange->svms, prange->start, prange->last);
>> @@ -1081,9 +1127,25 @@ static vm_fault_t svm_migrate_to_ram(struct 
>> vm_fault *vmf)
>>       return r ? VM_FAULT_SIGBUS : 0;
>>   }
>>   +static void svm_migrate_folio_split(struct folio *head, struct 
>> folio *tail)
>> +{
>> +    struct svm_range_bo *svm_bo;
>> +
>> +    if (tail == NULL)
>> +        return;
>> +
>> +    tail->pgmap = head->pgmap;
>> +    tail->mapping = head->mapping;
>> +
>> +    svm_bo = folio_zone_device_data(head);
>> +    folio_set_zone_device_data(tail, svm_bo);
>> +    svm_range_bo_ref(svm_bo);
>> +}
>> +
>>   static const struct dev_pagemap_ops svm_migrate_pgmap_ops = {
>>       .folio_free        = svm_migrate_folio_free,
>>       .migrate_to_ram        = svm_migrate_to_ram,
>> +    .folio_split        = svm_migrate_folio_split,
>>   };
>>     /* Each VRAM page uses sizeof(struct page) on system memory */
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c 
>> b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
>> index a99dbedf3cd4..8e1fb5df8abd 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c
>> @@ -245,7 +245,16 @@ void svm_range_dma_unmap_dev(struct device *dev, 
>> dma_addr_t *dma_addr,
>>           if (!svm_is_valid_dma_mapping_addr(dev, dma_addr[i]))
>>               continue;
>>           pr_debug_ratelimited("unmap 0x%llx\n", dma_addr[i] >> 
>> PAGE_SHIFT);
>> -        dma_unmap_page(dev, dma_addr[i], PAGE_SIZE, dir);
>> +
>> +        /* dma unmap of THP */
>> +        if (dma_addr[i] & SVM_RANGE_DMA_THP) {
>> +            dma_addr[i] &= ~SVM_RANGE_DMA_THP;
>> +            dma_unmap_page(dev, dma_addr[i], PAGE_SIZE*HPAGE_PMD_NR,
>> +                       DMA_BIDIRECTIONAL);
>> +        } else {
>> +            dma_unmap_page(dev, dma_addr[i], PAGE_SIZE, dir);
>> +        }
>> +
>>           dma_addr[i] = 0;
>>       }
>>   }
>> @@ -578,10 +587,15 @@ svm_range_vram_node_new(struct kfd_node *node, 
>> struct svm_range *prange,
>>       }
>>         memset(&bp, 0, sizeof(bp));
>> -    bp.size = prange->npages * PAGE_SIZE;
>> +    if (prange->npages >= HPAGE_PMD_NR) {
>> +        bp.size = ALIGN(prange->npages * PAGE_SIZE, HPAGE_PMD_SIZE);
>> +        bp.byte_align = HPAGE_PMD_SIZE;
>> +    } else {
>> +        bp.size = prange->npages * PAGE_SIZE;
>> +        bp.byte_align = PAGE_SIZE;
>> +    }
> seems don't need align size to 2MB, this will alloc vram that never 
> used. For example size 3MB, byte_align to 2MB, gpu buddy allocator 
> will return first block with 2MB, the second block 1MB

The fallback is align to PAGA_SIZE. Do you mean the case above? That is 
aligned to 2MB.

Regards

Xiaogang

> . Regards, Philip
>>       bp.bo_ptr_size = sizeof(struct svm_range_bo);
>>       bp.destroy = svm_range_bo_destroy;
>> -    bp.byte_align = PAGE_SIZE;
>>       bp.domain = AMDGPU_GEM_DOMAIN_VRAM;
>>       bp.flags = AMDGPU_GEM_CREATE_NO_CPU_ACCESS;
>>       bp.flags |= clear ? AMDGPU_GEM_CREATE_VRAM_CLEARED : 0;
>> @@ -593,9 +607,17 @@ svm_range_vram_node_new(struct kfd_node *node, 
>> struct svm_range *prange,
>>         r = amdgpu_bo_create(node->adev, &bp, &bo);
>>       if (r) {
>> -        pr_debug("failed %d to create bo\n", r);
>> -        mmput(mm);
>> -        goto create_bo_failed;
>> +        if (prange->npages >= HPAGE_PMD_NR) {
>> +            /* fall back PAGE_SIZE granularity */
>> +            bp.size = prange->npages * PAGE_SIZE;
>> +            bp.byte_align = PAGE_SIZE;
>> +            r = amdgpu_bo_create(node->adev, &bp, &bo);
>> +        }
>> +        if (r) {
>> +            pr_debug("failed %d to create bo\n", r);
>> +            mmput(mm);
>> +            goto create_bo_failed;
>> +        }
>>       }
>>         svm_bo = to_svm_range_bo(bo);
>> @@ -956,6 +978,7 @@ svm_range_copy_array(void *psrc, size_t size, 
>> uint64_t num_elements,
>>       *vram_pages = 0;
>>       for (i = 0; i < num_elements; i++) {
>>           dma_addr_t *temp;
>> +
>>           temp = (dma_addr_t *)dst + i;
>>           *temp = *((dma_addr_t *)src + i);
>>           if (*temp&SVM_RANGE_VRAM_DOMAIN)
>> @@ -1628,7 +1651,7 @@ static int svm_range_reserve_bos(struct 
>> svm_validate_context *ctx, bool intr)
>>       uint32_t gpuidx;
>>       int r;
>>   -    drm_exec_init(&ctx->exec, intr ? DRM_EXEC_INTERRUPTIBLE_WAIT: 
>> 0, 0);
>> +    drm_exec_init(&ctx->exec, intr ? DRM_EXEC_INTERRUPTIBLE_WAIT : 
>> 0, 0);
>>       drm_exec_until_all_locked(&ctx->exec) {
>>           for_each_set_bit(gpuidx, ctx->bitmap, MAX_GPU_INSTANCE) {
>>               pdd = kfd_process_device_from_gpuidx(ctx->process, 
>> gpuidx);
>> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.h 
>> b/drivers/gpu/drm/amd/amdkfd/kfd_svm.h
>> index c7d7adae4476..f2b3a05cd8cf 100644
>> --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.h
>> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.h
>> @@ -35,6 +35,7 @@
>>   #include "kfd_priv.h"
>>     #define SVM_RANGE_VRAM_DOMAIN (1UL << 0)
>> +#define SVM_RANGE_DMA_THP (1UL << 1)
>>   #define SVM_ADEV_PGMAP_OWNER(adev)\
>>               ((adev)->hive ? (void *)(adev)->hive : (void *)(adev))
>

  reply	other threads:[~2026-11-16 15:54 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-19 13:38 [PATCH v3 0/3] drm/amdkfd: Enable device private memory THP support in kfd svm driver Xiaogang.Chen
2026-10-19 13:38 ` [PATCH v3 1/3] drm/amdkfd: Add awareness of THP of device and system RAM " Xiaogang.Chen
2026-11-13 23:28   ` Philip Yang
2026-11-16 15:54     ` Chen, Xiaogang [this message]
2026-10-19 13:38 ` [PATCH v3 2/3] drm/amdkfd: Do not migrate partial THP memory at CPU/GPU page fault handler Xiaogang.Chen
2026-10-19 13:38 ` [PATCH v3 3/3] drm/amdkfd: Enable HMM device-private THP migration in KFD SVM driver Xiaogang.Chen
2026-11-13 19:29   ` Philip Yang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=b6c80e1b-e6ca-4eac-9fe7-8f67e766192c@amd.com \
    --to=xiaogang.chen@amd.com \
    --cc=amd-gfx@lists.freedesktop.org \
    --cc=yangp@amd.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.