From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A5A5FFF8861 for ; Mon, 27 Apr 2026 10:05:58 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 13D2B10E686; Mon, 27 Apr 2026 10:05:58 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (1024-bit key; unprotected) header.d=amd.com header.i=@amd.com header.b="wr4CyjkP"; dkim-atps=neutral Received: from CY3PR05CU001.outbound.protection.outlook.com (mail-westcentralusazon11013045.outbound.protection.outlook.com [40.93.201.45]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7520A10E686; Mon, 27 Apr 2026 10:05:56 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gyPev4V0tdCUIpkOLHr5MlLu9BGbg2Xn2wIIcWpVwc6c3mKFY7uspUJeVunPUuAccMcmd56oMEn6TY87aLBzx32m5NJIstXDtWqqq3UgR3Rqmjn2XhaTi+CZZZWWEUxcxWWlOA8W3YmOrYoTWmlxwEhgTL3My7M73F6KXLTsXYkDTYmn7FS7Z2nkOEW+TALoIkc0clJVqcNy+iQh+SBUd2KiuO+ojwEhse5V88w9S67UwBmgdijDDdiDHF30HQ/kSBMfOvFwJ3KfyPpUenfOcrA952ElBuNilttKWXuEHNzvm5hu8lh9XEhsdtER2RZfFTT016HgrZW3uIgm3m5afA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cJEcST48QJ6MtcBwjvOM7yQgQqSBEcUJFbjjfw++JzI=; b=xLBN7BsEYEs5cVQ3+p6gmj3Ow1O3smsy89kpKUwNEgJUFt5Q/WKZ5RjjJmuoDb7C/CO2yKoEDJ8ZAqPoZZk1RiCYoXYCOZrgdBMuCl5d12PfQf2aMmlrkjYqDMm62y5UO3HASKEl2SOCs/WTHnDj6Ro4PKiIOX33GTUX6v7yw7pi/WBqQx7dsJN8o4M8Rt+8YZOUJD8J7XcZMj3zvmFk4b7nhhOwsTu0xoL2lG0JLI6x0XAS9PMNlMN+Szso+6HhCVEzH9td3w1qXx5w+wAw6ZNBEzNsZIshEGU+709IBY8kLF3SyhMv0mRDiuJ5ID3kWXYO6bnsMd7canJ05lUGQA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cJEcST48QJ6MtcBwjvOM7yQgQqSBEcUJFbjjfw++JzI=; b=wr4CyjkPj3FgOAuE4TBsnLht4WlLwp20H7qslDnzxkGRyfoxI27x8uOLIL2vmfWbmSSBZAvcf22LoUlCaJG7F/fikzIroALl+pTFhihxSG9ThZN4vGFPEck5CGPKRbCK3K4Dx49vwZzkRnk5yEHByGKjTZBzbLKszweyDKqKstY= Received: from PH7P220CA0108.NAMP220.PROD.OUTLOOK.COM (2603:10b6:510:32d::27) by PH7PR12MB5927.namprd12.prod.outlook.com (2603:10b6:510:1da::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.15; Mon, 27 Apr 2026 10:05:45 +0000 Received: from MW1PEPF0001615B.namprd21.prod.outlook.com (2603:10b6:510:32d:cafe::a2) by PH7P220CA0108.outlook.office365.com (2603:10b6:510:32d::27) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.26 via Frontend Transport; Mon, 27 Apr 2026 10:05:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb08.amd.com; pr=C Received: from satlexmb08.amd.com (165.204.84.17) by MW1PEPF0001615B.mail.protection.outlook.com (10.167.249.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9891.0 via Frontend Transport; Mon, 27 Apr 2026 10:05:45 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.2562.17; Mon, 27 Apr 2026 05:05:34 -0500 Received: from satlexmb07.amd.com (10.181.42.216) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 27 Apr 2026 05:05:34 -0500 Received: from junhua-PC.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Mon, 27 Apr 2026 05:05:31 -0500 From: Junhua Shen To: , , , , , , , , , CC: , , Junhua Shen Subject: [PATCH v3 1/5] drm/amdgpu: add VRAM migration infrastructure for drm_pagemap Date: Mon, 27 Apr 2026 18:05:18 +0800 Message-ID: <20260427100522.7014-2-Junhua.Shen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260427100522.7014-1-Junhua.Shen@amd.com> References: <20260427100522.7014-1-Junhua.Shen@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit Received-SPF: None (SATLEXMB04.amd.com: Junhua.Shen@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW1PEPF0001615B:EE_|PH7PR12MB5927:EE_ X-MS-Office365-Filtering-Correlation-Id: a539742b-e5b3-409d-0792-08dea4448c6f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|376014|82310400026|36860700016|921020|56012099003|22082099003|18002099003|18096099003; X-Microsoft-Antispam-Message-Info: ibgWZypc3jl3iXONHEX+3pc8x84WamOLPOqaYD0M2jF+mGk/HsUuinYEAhF6AxkA3v52RyOq65hXnm6dAmepK3x1KAZaOaTZTJagLJesRbRLxzBlSDCH32e7k/iI4QUvuQ9CHmYamzuASp4kQdwWrWu5AAKfOPwbZPAKDblB50VhEqJUioQvOgMJ479JbgmysnVf+kPto+KeCpqzfyZXa47n9yhb4pq/U3iwqva2lIfpSSnVAPJ02nSGYJ/Ouc4oydePMWIptjl4pa43PBOMkmcDL7ipB3roNbxB4s3uptrUKu4txI1VmiJs0AADsTx4P0Umcp/9/hw7GslvMj+R2bCbLvS3luVNt9rdd4Y/MJQOnufk8BeXUg1j02mEPKo4sozHgP22SE+cRttUX4VFSRoosey8jjTn66012AZP0BLmR689oD0Z/7KFpjTDKnSgH4A5kmCQJB0yiUYyxXtEg6gZF7uXsAG6UA+iCKv1rqK+eHnnEMpJvJb8eR11U13KG7s5u0VSaCBnw23luHEsp35mr0zraoFPZfnL8qj9zuPjyXTvemLEQpIxF2MgXBe47G19vsNWm5JefYqQRKFH3nZsiImcu+YOhcAZARAdtPYZGtzSGlD6MMUQ3MxhqXjjc62eQ3kneZt8h7xcMR1MTYovO+RTDnNDN3JJhdHVkjU5vJOdy+ygoDuavYeAqts91mHNF029JOfoKxWta5zNAVcKPmRI93c33H/iGrKUxyRERMoh2T2Mja1oqwmweCBTg5gjD3AqIFUXTh0/IyNmCD9EpE4j5SRocwWbHESr0h98eMalhKr9tAXpHIExoMVS X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:satlexmb08.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(376014)(82310400026)(36860700016)(921020)(56012099003)(22082099003)(18002099003)(18096099003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 43j+/FqYDJap7VJ4AYvRIrMn3xJ9DH51E3V1ho065sWOcY6U11ZLOYdCQ02LadlXE4ii/19wUiHFmZCkP2b7ut9awv/cgj8ixPK7ft5bfehvcrMjX9GH8kIpv+SC0lzoYHtReNwxFrbkwCJoH5xrCIn225aehZVUS0DPcHONf2GizR2s1lhOQYIbsZvMPTdGlM14SHsYO6OIC6JmuaZdBpdpWrgxZgE3ugKe23dy1LCwHnCJDuvp4YiTwXCEJziDGFmw9FSLs4ix0PGPWiU7CNlUKrPZ0Cd7HOOoOYK758IRrUntZCq0GHc0NyfKCdyR27+NrjSyR0AxESmH9aAn2iRBVDaLf2jaWxLjzKukAR1URpYFEGWptY74Onl78ycIGDZsAoYY0TVDST7S4s7M0igGEysyKnKOBvqlPhqUYACXsW85GkuNbzP0nQVax5B7 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Apr 2026 10:05:45.2615 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a539742b-e5b3-409d-0792-08dea4448c6f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[satlexmb08.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MW1PEPF0001615B.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5927 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Add the drm_pagemap-based VRAM migration infrastructure: - Define struct amdgpu_pagemap wrapping dev_pagemap + drm_pagemap - Define AMDGPU_SVM_PGMAP_OWNER() and AMDGPU_INTERCONNECT_VRAM macros - Implement amdgpu_svm_migration_init() to register ZONE_DEVICE via devm_memremap_pages() and initialize the drm_pagemap - Add amdgpu_pagemap pointer (apagemap) to struct amdgpu_device Signed-off-by: Junhua Shen --- drivers/gpu/drm/amd/amdgpu/Makefile | 6 +- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 8 + drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c | 179 ++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h | 98 +++++++++++ 4 files changed, 289 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile b/drivers/gpu/drm/amd/amdgpu/Makefile index 7700f81a246e..e64abb5c8ab8 100644 --- a/drivers/gpu/drm/amd/amdgpu/Makefile +++ b/drivers/gpu/drm/amd/amdgpu/Makefile @@ -323,12 +323,14 @@ amdgpu-$(CONFIG_HMM_MIRROR) += amdgpu_hmm.o # svm support amdgpu-$(CONFIG_DRM_AMDGPU_SVM) += amdgpu_svm.o amdgpu_svm_attr.o \ - amdgpu_svm_range.o + amdgpu_svm_range.o amdgpu_migrate.o .PHONY: clean-svm clean-svm: rm -f $(obj)/amdgpu_svm.o $(obj)/amdgpu_svm_attr.o $(obj)/amdgpu_svm_range.o \ - $(obj)/.amdgpu_svm.o.cmd $(obj)/.amdgpu_svm_attr.o.cmd $(obj)/.amdgpu_svm_range.o.cmd + $(obj)/amdgpu_migrate.o \ + $(obj)/.amdgpu_svm.o.cmd $(obj)/.amdgpu_svm_attr.o.cmd $(obj)/.amdgpu_svm_range.o.cmd \ + $(obj)/.amdgpu_migrate.o.cmd include $(FULL_AMD_PATH)/pm/Makefile diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 49e7881750fa..fe6ba9911d9f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -325,6 +325,7 @@ struct amdgpu_fpriv; struct amdgpu_bo_va_mapping; struct kfd_vm_fault_info; struct amdgpu_hive_info; +struct amdgpu_pagemap; struct amdgpu_reset_context; struct amdgpu_reset_control; struct amdgpu_coredump_info; @@ -1200,6 +1201,13 @@ struct amdgpu_device { struct amdgpu_uma_carveout_info uma_info; +#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM) + /* SVM VRAM migration via drm_pagemap (drm_gpusvm path). + * Allocated in amdgpu_svm_migration_init(), NULL if SVM disabled. + */ + struct amdgpu_pagemap *apagemap; +#endif + /* KFD * Must be last --ends in a flexible-array member. */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c new file mode 100644 index 000000000000..170e2eadc106 --- /dev/null +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.c @@ -0,0 +1,179 @@ +/* SPDX-License-Identifier: GPL-2.0 OR MIT */ +/* + * Copyright 2026 Advanced Micro Devices, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + */ + +/** + * DOC: AMDGPU SVM Migration + * + * This file implements the drm_pagemap-based migration infrastructure for + * AMDGPU SVM. It provides the callbacks that the DRM GPUSVM / drm_pagemap + * framework needs to: + * + * 1. Map ZONE_DEVICE pages to GPU-visible VRAM MC addresses (device_map) + * 2. Allocate VRAM and migrate pages from system memory (populate_mm) + * 3. Copy data between RAM and VRAM using SDMA (copy_to_devmem / copy_to_ram) + * 4. Release VRAM backing when pages migrate back to system memory (devmem_release) + * + * Architecture overview:: + * + * adev->apagemap->dpagemap (struct drm_pagemap) + * .ops = &amdgpu_svm_drm_pagemap_ops + * | + * +---+-------------------+ + * | | + * .populate_mm .device_map + * (alloc BO + migrate) (page -> VRAM MC addr) + * | + * v + * drm_pagemap_devmem_ops (per-BO migration mechanics) + * .populate_devmem_pfn -> BO buddy blocks -> PFN array + * .copy_to_devmem -> SDMA copy RAM -> VRAM + * .copy_to_ram -> SDMA copy VRAM -> RAM + * .devmem_release -> release BO reference + * + * The three address spaces involved: + * + * VRAM offset [0, real_vram_size) - buddy allocator managed + * + hpa_base + * HPA / PFN [hpa_base, hpa_base+..) - ZONE_DEVICE struct page management + * + vm_manager.vram_base_offset + * PTE address [vram_base_offset, ..] - GPU page table entries (from MMHUB FB_OFFSET) + */ + +#include +#include +#include + +#include "amdgpu_amdkfd.h" +#include "amdgpu_migrate.h" +#include "amdgpu.h" + +static inline struct amdgpu_pagemap * +dpagemap_to_apagemap(struct drm_pagemap *dpagemap) +{ + return container_of(dpagemap, struct amdgpu_pagemap, dpagemap); +} + +static inline struct amdgpu_device * +dpagemap_to_adev(struct drm_pagemap *dpagemap) +{ + return drm_to_adev(dpagemap->drm); +} + +/** + * amdgpu_svm_page_to_apagemap - Get amdgpu_pagemap from a ZONE_DEVICE page + * @page: A ZONE_DEVICE page backed by VRAM + * + * Follows: page -> pgmap -> container_of(apagemap) + */ +static inline struct amdgpu_pagemap * +amdgpu_svm_page_to_apagemap(struct page *page) +{ + struct dev_pagemap *pgmap = page_pgmap(page); + + return container_of(pgmap, struct amdgpu_pagemap, pgmap); +} + + +const struct drm_pagemap_ops amdgpu_svm_drm_pagemap_ops = { }; + +/** + * amdgpu_svm_migration_init - Register ZONE_DEVICE and initialize drm_pagemap + * @adev: AMDGPU device to set up VRAM migration for + * + * Allocates a ZONE_DEVICE region covering the GPU's VRAM, registers it + * via devm_memremap_pages() with drm_pagemap's generic dev_pagemap_ops, + * and then initializes the drm_pagemap (dpagemap) that provides the + * device_map / populate_mm callbacks for the DRM GPUSVM migration path. + * + * For XGMI-connected CPUs, uses the device's aperture directly + * (MEMORY_DEVICE_COHERENT). For discrete GPUs, requests a free + * iomem region for MEMORY_DEVICE_PRIVATE pages. + * + * Return: 0 on success, -EINVAL if GPU IP too old, negative error on failure + */ +int amdgpu_svm_migration_init(struct amdgpu_device *adev) +{ + struct amdgpu_pagemap *svm_dm; + struct dev_pagemap *pgmap; + struct resource *res = NULL; + unsigned long size; + void *r; + + if (amdgpu_ip_version(adev, GC_HWIP, 0) < IP_VERSION(9, 0, 1)) + return -EINVAL; + + if (adev->apu_prefer_gtt) + return 0; + + if (adev->apagemap && adev->apagemap->initialized) + return 0; + + svm_dm = devm_kzalloc(adev->dev, sizeof(*svm_dm), GFP_KERNEL); + if (!svm_dm) + return -ENOMEM; + + pgmap = &svm_dm->pgmap; + + size = ALIGN(adev->gmc.real_vram_size, 2ULL << 20); + if (adev->gmc.xgmi.connected_to_cpu) { + pgmap->range.start = adev->gmc.aper_base; + pgmap->range.end = adev->gmc.aper_base + adev->gmc.aper_size - 1; + pgmap->type = MEMORY_DEVICE_COHERENT; + } else { + res = devm_request_free_mem_region(adev->dev, &iomem_resource, size); + if (IS_ERR(res)) + return PTR_ERR(res); + pgmap->range.start = res->start; + pgmap->range.end = res->end; + pgmap->type = MEMORY_DEVICE_PRIVATE; + } + + pgmap->nr_range = 1; + pgmap->flags = 0; + pgmap->ops = drm_pagemap_pagemap_ops_get(); + pgmap->owner = AMDGPU_SVM_PGMAP_OWNER(adev); + + r = devm_memremap_pages(adev->dev, pgmap); + if (IS_ERR(r)) { + dev_err(adev->dev, "SVM: failed to register HMM device memory\n"); + if (pgmap->type == MEMORY_DEVICE_PRIVATE && res) + devm_release_mem_region(adev->dev, res->start, resource_size(res)); + pgmap->type = 0; + return PTR_ERR(r); + } + + if (drm_pagemap_init(&svm_dm->dpagemap, pgmap, adev_to_drm(adev), + &amdgpu_svm_drm_pagemap_ops)) { + dev_err(adev->dev, "SVM: failed to init drm_pagemap\n"); + return -EINVAL; + } + svm_dm->adev = adev; + svm_dm->hpa_base = pgmap->range.start; + svm_dm->initialized = true; + adev->apagemap = svm_dm; + + dev_info(adev->dev, "SVM: registered %ldMB device memory, hpa_base=0x%llx\n", + size >> 20, svm_dm->hpa_base); + return 0; +} diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h new file mode 100644 index 000000000000..e20698fb1597 --- /dev/null +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_migrate.h @@ -0,0 +1,98 @@ +/* SPDX-License-Identifier: GPL-2.0 OR MIT */ +/* + * Copyright 2026 Advanced Micro Devices, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + */ + +#ifndef __AMDGPU_MIGRATE_H__ +#define __AMDGPU_MIGRATE_H__ + +#include +#include + +struct amdgpu_device; + +/* + * AMDGPU_INTERCONNECT_VRAM - Protocol identifier for local VRAM access. + * + * Used in drm_pagemap_addr to distinguish device-local VRAM addresses from + * DMA-mapped system memory addresses. drm_gpusvm_get_pages() uses this to + * identify pages that are already in device memory and need no DMA mapping. + * + * Value must be non-zero (0 == DRM_INTERCONNECT_SYSTEM). + */ +#define AMDGPU_INTERCONNECT_VRAM 1 + +/* + * AMDGPU_SVM_PGMAP_OWNER - Unique owner token for dev_pagemap registration. + * + * migrate_vma_setup() uses pgmap->owner to distinguish "own" device pages + * from "foreign" device pages (e.g., another GPU in an XGMI hive). + * Pages whose page->pgmap->owner matches the migration source are skipped + * (they're already in the right place). + * + * For XGMI hive: all GPUs in the hive share the same owner (the hive pointer) + * so intra-hive pages are treated as local. + * For standalone GPU: use the adev pointer itself as a unique per-device token. + */ +#define AMDGPU_SVM_PGMAP_OWNER(adev) \ + ((adev)->hive ? (void *)(adev)->hive : (void *)(adev)) + +/** + * struct amdgpu_pagemap - VRAM migration infrastructure for drm_pagemap + * @dpagemap: DRM pagemap wrapper providing device_map/populate_mm callbacks + * @adev: back-pointer to the owning amdgpu_device + * @hpa_base: host physical address base of the ZONE_DEVICE region + * @initialized: set to true after successful registration + * @pgmap: the dev_pagemap registered with devm_memremap_pages(); + * must be last — contains a flexible-array member (ranges[]) + * + * Allocated with devm_kzalloc() in amdgpu_svm_migration_init() and stored + * as adev->apagemap. Lifetime is tied to the device via devres. + */ +struct amdgpu_pagemap { + struct drm_pagemap dpagemap; + struct amdgpu_device *adev; + resource_size_t hpa_base; + bool initialized; + struct dev_pagemap pgmap; /* must be last — flex-array */ +}; + +#if IS_ENABLED(CONFIG_DRM_AMDGPU_SVM) +int amdgpu_svm_migration_init(struct amdgpu_device *adev); +#else +static inline +int amdgpu_svm_migration_init(struct amdgpu_device *adev) +{ + return 0; +} +#endif + +/** + * amdgpu_svm_drm_pagemap_ops - drm_pagemap_ops for AMDGPU VRAM migration + * + * Provides: + * .device_map - Convert ZONE_DEVICE page to VRAM address + * .populate_mm - Allocate VRAM BO and migrate pages from system memory + */ +extern const struct drm_pagemap_ops amdgpu_svm_drm_pagemap_ops; + +#endif /* __AMDGPU_MIGRATE_H__ */ -- 2.34.1