From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5AA69C4345F for ; Mon, 15 Apr 2024 09:13:09 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 14ED41123D1; Mon, 15 Apr 2024 09:13:09 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="TIAwk0lS"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.18]) by gabe.freedesktop.org (Postfix) with ESMTPS id DC3461123D1 for ; Mon, 15 Apr 2024 09:13:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1713172388; x=1744708388; h=message-id:date:subject:to:references:from:in-reply-to: mime-version; bh=EYAKaZjV6nICdHjaoLegvn0+hVeWIgJV5rvm7YqE/M0=; b=TIAwk0lS+n+zayMcLpQGQ36is+yrRVJjBKi79enQWkXigK1QiiVNCjRp oJLxgSCR2epa5unxUUI5BKtdXU1XHfeoD1baLdGSiokIA3iWFb54kunDL AydxOC+ilkbBFfOAlPyryyrVvr6ZUlProGHSELdXXvfCNIKiXiFDonLLS xq3UFd11rYda3XsCJ3cAiHV5sfEUlJ7g9fB6mTlbmyJUZ2dK4d2cYAUgr FwKSDn1PLnUVoOVHAUB3jr3vbUQi5BolAXAEIfbJq/5lmKZeDKGpqT1O0 iXQqJvoUeLyZl29peZG93nC+hy45aB9furspRMh18HisEzL0HNPLwXagw g==; X-CSE-ConnectionGUID: K6SoeIP0S2KaZwRPXTloAA== X-CSE-MsgGUID: T/ff9il7TWa6EqHb+Ph15w== X-IronPort-AV: E=McAfee;i="6600,9927,11044"; a="8721317" X-IronPort-AV: E=Sophos;i="6.07,202,1708416000"; d="scan'208,217";a="8721317" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa110.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Apr 2024 02:13:08 -0700 X-CSE-ConnectionGUID: RLCp/Up/TWi3wxPQ05hUwg== X-CSE-MsgGUID: xZwzaoQeT1yLZs9kk7ZPJA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,202,1708416000"; d="scan'208,217";a="26411806" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by fmviesa003.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 15 Apr 2024 02:13:07 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 15 Apr 2024 02:13:06 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 15 Apr 2024 02:13:06 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Mon, 15 Apr 2024 02:13:06 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.170) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Mon, 15 Apr 2024 02:13:05 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K4aU1sgptO7EXeDUbk58UhDZjXB4YnbsIEPAusobsTw9Qpxy2+60QNFZrw8NQtTXpafjLup6R2E9OaMbNtZfby4Y8JCm8PQIhLiLd4MJrhEBMXY0waRrFmZhKXO5lPHN1GpspQlcrdzdV7B8Ih3QK/EGVfmqSAmNHH47Ks3iFKbAiiJfnE5huH8s2JmEPYs109LSkkEa/z+JVc49nztCOdS5sZmiFuOgPH5G0NjN0CJEtqPk6MIdk45SajpHiaTau5XdEsHXEhL26OWvqqJK+i5p9HWaR5Kh4pWN6OhPVEWs5kU+d3yW8kxR/1Q0k/0aKShhy6eaLWdwsEl4EWv12Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Qwqr4eJCdnbgqzcaEVIXFB5woAu6po0nY+B8dudfiAg=; b=YMDVIvsuRRGQ7PiJYZVZk7Lh4hnkO+Pc0L4Wtn8MLsp269E92f42KGBEno++7jDsC7WOhPjOW0zN8jE+JpzqGGLtJDWkzEnc4VkpxzTx182Ih/uPORzIAeEpggUFzOhml/Sztfo6pdepwqLaC7ftgcgY/SMS3aut9lds9o50N+N1nWlO1EOt0I4mIQwtK7k5TnDxuy9eI3h656f16hPGUrO7Az2/WFgyq3a6DsKJ/4rBtH/abGp6B0ovYFnMiy/fkJVoO+em13pf6Ethw4UQ7eOb4di4IhaNRSPL4FNUTRUpcUhphQiIHZ/Fa9kYkwMzx8bMsjZhTwR6XS3KUpflhg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MW4PR11MB7056.namprd11.prod.outlook.com (2603:10b6:303:21a::12) by DS0PR11MB8719.namprd11.prod.outlook.com (2603:10b6:8:1a8::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7472.25; Mon, 15 Apr 2024 09:13:03 +0000 Received: from MW4PR11MB7056.namprd11.prod.outlook.com ([fe80::ff2a:1235:d1ba:4f93]) by MW4PR11MB7056.namprd11.prod.outlook.com ([fe80::ff2a:1235:d1ba:4f93%3]) with mapi id 15.20.7472.025; Mon, 15 Apr 2024 09:12:57 +0000 Content-Type: multipart/alternative; boundary="------------LjjMg38QTjTK5A7Ny8l6RNSL" Message-ID: <0d5a5db5-6eee-4def-bbee-0071b376f142@intel.com> Date: Mon, 15 Apr 2024 14:42:51 +0530 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 2/6] drm/xe: Allow to assign GGTT region to the VF To: Michal Wajdeczko , References: <20240414190137.1243-1-michal.wajdeczko@intel.com> <20240414190137.1243-3-michal.wajdeczko@intel.com> Content-Language: en-US From: "Ghimiray, Himal Prasad" In-Reply-To: X-ClientProxiedBy: PN3PR01CA0145.INDPRD01.PROD.OUTLOOK.COM (2603:1096:c01:c8::23) To MW4PR11MB7056.namprd11.prod.outlook.com (2603:10b6:303:21a::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW4PR11MB7056:EE_|DS0PR11MB8719:EE_ X-MS-Office365-Filtering-Correlation-Id: 8d9ac306-938a-4731-b34c-08dc5d2c3da5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VYqEqBxmQGPyeujtLwxGRylHRj2VMPM+/0EFMjov0AsxFmgK/22VYgSoMJBPu62l9d6lWAGqWX8UfbGlnGMhbg03ULWpqGMcP6UAgYlbHurRprPsPrS+LJBO9ro+s7CyA3LD7AayO/cecEe3GqdCn4B5XZNT3bTlx5QZwaWXX2mI4kIt2vms9fKianJf5IFIviKUidCxfsKhsYkIlbVha/uZ+MvMT86ORA4OefnDzi3FbZjGlQ5+eXeV6L1fNLjyD7dui+91HHJ/n97NUzPpzw6rhA7d2Q2EnkUMXBF8h6zwXYOfsnW8U7z57QXwFvF5D2DgZXqXzopmPYh3FY5btveNXLYgU8Aew28wSaA2YRqv1avRKGJPmBFNC4onQBp9+v1E7wLRc1BGseUrswX2sin0OEhkQ7ioLFzUgl6bOeVT+XCFsFAgEs9mgZimhHtWK3krfn6rKULRD9u/KFMWhYZ5zjPpOIoRDTi9c1jE3YX4Tx7aNzS3coEwPK0p03CQTKTXlkzbg2ybkcPIsCai85XJsX65qIUdpGgUWiEpx1+2aoulB90uP5aSa31YLDshBhQb1jhHeVo3/h87hPOBhJy3N4Xthk42U52CNZTUVkeqR3rebJtPPyGq96Roi53eap0A1iKgIbp38NaPt1emU4tSsB8rXBHFP2+gN55sMFk= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW4PR11MB7056.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(376005)(1800799015)(366007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?K1FkZHpsdHJzYnZkQnJDS1k1dkh2ZFdFbVFCTjVudHR5ekUwcHNPYzlnakln?= =?utf-8?B?ZVkwd2hvb0ZJYUc1NWhRam44MWppMmNrcGN6TERtL1BOeCtKbWI4MjV1M3dZ?= =?utf-8?B?ZWgrTDhUMk9hcHZubTFZdEt4eitrRERSQlhaZ0V1cExaMHlvS2lTRnlIZ2ZM?= =?utf-8?B?WE5vb1VDWVJsYWVtSGFEYlludGM5WERDN0hEMUhOZWlBYUhNUzR1N2NFeFBr?= =?utf-8?B?UFBSZllOZENYL1FSRWlSaFhMV29BT1BwbEdaR0poVVlMMUlsbHYwMkt3Wkdz?= =?utf-8?B?VEZzNmJ3OFNnRWRVVG5Yci9EVlJYdXFKR21SUjhFUTBWZnFwbE5NUGMxUzZx?= =?utf-8?B?NWlFYzQzVVRsSUxtTFM4NE95eVQwV3hJVGZYTG90eks2b0V2czhvd1d2Tjlj?= =?utf-8?B?NmNmNU5IOTF4ZHJjODhqay9IWXdyVTE3V0lLSmgxNnpHcGowbHNYYlJuQmNB?= =?utf-8?B?a1VlRCttbnA0ZVJjODFzaTNNSXlEMlFIOVAwaXhDVTFhRmNyVjkyV2hsVXla?= =?utf-8?B?TnJOb2kwOEtYcTlBZVdpQnpzSVNERTNkQWczb0ZtcmRiRVFiM2E4WG01cGlB?= =?utf-8?B?R0h0TGRXWGtYam1vVmRCdG1YSVFCOEdQeE9YSFFwcVpTbTNwMWdKaDZVc25S?= =?utf-8?B?OS9UbXJmWUg1eElwVG9CNTN3OXJrcFl5NmRyS1YxNUhEd1RFNXFVa1Q3U2ZX?= =?utf-8?B?RkxET1pVOTZKVkxuSVZ3Nmg4UkJSVG82NThQSFZBQXdDQmJYekU5bGhMRzFL?= =?utf-8?B?a0c1ZTNFNWVSUzhVTE13aHlDR3hraGRWZlkvOWtLRmFmVlRnUVJsdnVnRHhM?= =?utf-8?B?dGU3SHdGcWhFUyszZitTTWRHeHRXQi9tNUlrSUVVZGcxa3lpOEFtZlN2a2Zq?= =?utf-8?B?VEsxeXA5V0RtZzBIcGZkSGtwNHl4cUkxd0JWdlFOZWxDaGNqMVNTL2pNL2F6?= =?utf-8?B?aTQyZUU2NUswY21KWUptQ1ZFRkF6ZE56YndvcmM1M2RrQkg5N2hsNmNxYjVl?= =?utf-8?B?OEZHY0NheWtKNnpTaDBYSFYxUmxhUnlnZTVRUmV5QWZwV2RUNVY4NWE1U3JU?= =?utf-8?B?YUYwVGdxcEZZRHhFQUFOd2VNbEs4QThCL3JjQTFLM1NPZ0lLcnlmeXp1OEd5?= =?utf-8?B?ZEF4R21KRkswRE1NYnFad2czR1doUWxZQ3FjejdtM2M2aXVUd2xGTEhsV0xi?= =?utf-8?B?THhiNUdLWUt5aXRIKzFnNjB1ZnFlL2toRll6c0poSU9PSVRjNXBhMjUvalc3?= =?utf-8?B?SmZmVkxXQzROWHdEbUtSMmpPc3QzY1VtTlVoMEl2NTlDM3NxcU9aZ0FzN1pz?= =?utf-8?B?MEp4QlRJaFRxd2NadS9UZDBwM0o3MUZFZlpKY0dlVFQwSXUxcGYzYTZuZXda?= =?utf-8?B?TSs0L2lHblJ4ajRlWElQT0pSeFR2VEgwMitCNC9tLzVtVVFBTGNOZCtEQTlN?= =?utf-8?B?aUJjam1hR0hhQWl0NXZ1eit3cjFzSnJHTms3ajdpTmMvendOQVpaYzlGMzM1?= =?utf-8?B?dEVhTXk4a2dWTk9MQlFUMmg0eEtnc0h2ZlFuNnRMSm14NFRkb2pzQXdTTkhy?= =?utf-8?B?OHkrRzdONDlrTWdMM2hienVFTnJIeG1GVlNYbHlCSTY0a2JySitLY3dmS0Rh?= =?utf-8?B?ME9SeVRZVUJIRkl6RStrYS9Hamt1dnlsV0ZHTEtuZ3V6Q3lDenE3SWtWc3Ba?= =?utf-8?B?WnpZdExqOFpmWDF3VU1uOGgwcjZ6K2xIcFFRWVlpR2xZc2pqNnlRZWhTVVNL?= =?utf-8?B?NlN5bEw1MjYwUHpTY1lPT2xzU3FVRW1BMm5HWUhCWktFVVVZWmplbVRaWmdZ?= =?utf-8?B?MFl0SnVyNzRXRWhNMGFvNkVRSnZCcmROODhPK1RjNHBESVI1eGNGTW5JMkJ1?= =?utf-8?B?dDVXK2lxQ0NkU2JtWGVMN2NTZlJwakJtamFDdnRObXJZdlRWcjZYODB1S2th?= =?utf-8?B?dEQxakwxenpCTjc2THptZFBrSzVxRTVUbSt6OE1OemM3d2Y0QVVTc3htY2U2?= =?utf-8?B?djdxeHQ5eHFrbldFMDZGQVZ4TENDMEtBLzhHa1JhYnYrMGVuT0h4clpyRExF?= =?utf-8?B?NVo3aXZvbkpHSHZPeFQ2TGk2bFd3VWhoOUc5alFYWnhEMExWcG1VRkIyQWU3?= =?utf-8?B?TkJkNFFwTTVRQ1Q1ZmNzSjFhSDQzYjVzNmNqTmNLOHdQOXQyVFppdFhvQWtZ?= =?utf-8?B?Snc9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: 8d9ac306-938a-4731-b34c-08dc5d2c3da5 X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB7056.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Apr 2024 09:12:57.6861 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YWDSJLkoeBdcUL3bWvjrp8esOI7eRBIejprKpImzn5Z91r0WvnNT8YI3wJXPbD62BrZOuBKgFJRUTOweNUJ+0RjZ1cilC4hIimCggEcTMwE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB8719 X-OriginatorOrg: intel.com X-BeenThere: intel-xe@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Xe graphics driver List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-xe-bounces@lists.freedesktop.org Sender: "Intel-xe" --------------LjjMg38QTjTK5A7Ny8l6RNSL Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit On 15-04-2024 14:31, Michal Wajdeczko wrote: > > On 15.04.2024 06:49, Ghimiray, Himal Prasad wrote: >> On 15-04-2024 00:31, Michal Wajdeczko wrote: >>> VF's drivers can't modify GGTT PTEs except the range explicitly >>> assigned by the PF driver. To allow hardware enforcement of this >>> requirement, each GGTT PTE has a field with the VF number that >>> identifies which VF can modify that particular GGTT PTE entry. >>> >>> Only PF driver can modify this field and PF driver shall do that >>> before VF drivers will be loaded. Add function to prepare PTEs. >>> Since it will be used only by the PF driver, make it available >>> only for CONFIG_PCI_IOV=y. >>> >>> Bspec: 45015 >>> Signed-off-by: Michal Wajdeczko >>> --- >>>   drivers/gpu/drm/xe/regs/xe_gtt_defs.h |  2 ++ >>>   drivers/gpu/drm/xe/xe_ggtt.c          | 44 +++++++++++++++++++++++++++ >>>   drivers/gpu/drm/xe/xe_ggtt.h          |  4 +++ >>>   3 files changed, 50 insertions(+) >>> >>> diff --git a/drivers/gpu/drm/xe/regs/xe_gtt_defs.h >>> b/drivers/gpu/drm/xe/regs/xe_gtt_defs.h >>> index 558519ce48c7..4389e5a76f89 100644 >>> --- a/drivers/gpu/drm/xe/regs/xe_gtt_defs.h >>> +++ b/drivers/gpu/drm/xe/regs/xe_gtt_defs.h >>> @@ -9,6 +9,8 @@ >>>   #define XELPG_GGTT_PTE_PAT0    BIT_ULL(52) >>>   #define XELPG_GGTT_PTE_PAT1    BIT_ULL(53) >>>   +#define GGTT_PTE_VFID        GENMASK_ULL(11, 2) >> Patch looks good to me. >> >> Just to confrim, based on BSPEC, this MASK applies to GRAPHICS_VER >= >> 12.50. For versions before this, the applicable mask is (4, 2). I infer >> that since SRIOV support on XE is expected beyond GRAPHICS_VER version >> 12.50, you are using mask (11, 2). > true, but even on earlier versions, which we may want to use as SDV, > where real mask is (4, 2) this extended mask (11, 2) should be still > harmless as number of VFs would be then always less than 7 so it will > fit only into (4, 2) Thanks for confirming. > >>> + >>>   #define GUC_GGTT_TOP        0xFEE00000 >>>     #define XELPG_PPGTT_PTE_PAT3        BIT_ULL(62) >>> diff --git a/drivers/gpu/drm/xe/xe_ggtt.c b/drivers/gpu/drm/xe/xe_ggtt.c >>> index ff2239c0eda5..f090cab065b8 100644 >>> --- a/drivers/gpu/drm/xe/xe_ggtt.c >>> +++ b/drivers/gpu/drm/xe/xe_ggtt.c >>> @@ -460,6 +460,50 @@ void xe_ggtt_remove_bo(struct xe_ggtt *ggtt, >>> struct xe_bo *bo) >>>                   bo->flags & XE_BO_FLAG_GGTT_INVALIDATE); >>>   } >>>   +#ifdef CONFIG_PCI_IOV >>> +static u64 xe_encode_vfid_pte(u16 vfid) >>> +{ >>> +    return FIELD_PREP(GGTT_PTE_VFID, vfid) | XE_PAGE_PRESENT; >>> +} >>> + >>> +static void xe_ggtt_assign_locked(struct xe_ggtt *ggtt, const struct >>> drm_mm_node *node, u16 vfid) >>> +{ >>> +    u64 start = node->start; >>> +    u64 size = node->size; >>> +    u64 end = start + size - 1; >>> +    u64 pte = xe_encode_vfid_pte(vfid); >>> + >>> +    lockdep_assert_held(&ggtt->lock); >>> + >>> +    if (!drm_mm_node_allocated(node)) >>> +        return; >>> + >>> +    while (start < end) { >>> +        xe_ggtt_set_pte(ggtt, start, pte); >>> +        start += XE_PAGE_SIZE; >>> +    } >>> + >>> +    xe_ggtt_invalidate(ggtt); >>> +} >>> + >>> +/** >>> + * xe_ggtt_assign - assign a GGTT region to the VF >>> + * @ggtt: the &xe_ggtt where the node belongs >>> + * @node: the &drm_mm_node to update >>> + * @vfid: the VF identifier >>> + * >>> + * This function is used by the PF driver to assign a GGTT region to >>> the VF. >>> + * In addition to PTE's VFID bits 11:2 also PRESENT bit 0 is set as >>> on some >>> + * platforms VFs can't modify that either. >> This info regarding bit 0 not being modifiable by VF's on some platforms >> is missing in Bspec. > Piotr reminded me of another page: > > Bspec: 52395 Thanks. LGTM Reviewed-by: Himal Prasad Ghimiray > >>> + */ >>> +void xe_ggtt_assign(struct xe_ggtt *ggtt, const struct drm_mm_node >>> *node, u16 vfid) >>> +{ >>> +    mutex_lock(&ggtt->lock); >>> +    xe_ggtt_assign_locked(ggtt, node, vfid); >>> +    mutex_unlock(&ggtt->lock); >>> +} >>> +#endif >>> + >>>   int xe_ggtt_dump(struct xe_ggtt *ggtt, struct drm_printer *p) >>>   { >>>       int err; >>> diff --git a/drivers/gpu/drm/xe/xe_ggtt.h b/drivers/gpu/drm/xe/xe_ggtt.h >>> index 8306ef74abc6..4a41a1762358 100644 >>> --- a/drivers/gpu/drm/xe/xe_ggtt.h >>> +++ b/drivers/gpu/drm/xe/xe_ggtt.h >>> @@ -33,4 +33,8 @@ void xe_ggtt_remove_bo(struct xe_ggtt *ggtt, struct >>> xe_bo *bo); >>>     int xe_ggtt_dump(struct xe_ggtt *ggtt, struct drm_printer *p); >>>   +#ifdef CONFIG_PCI_IOV >>> +void xe_ggtt_assign(struct xe_ggtt *ggtt, const struct drm_mm_node >>> *node, u16 vfid); >>> +#endif >>> + >>>   #endif --------------LjjMg38QTjTK5A7Ny8l6RNSL Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: 8bit


On 15-04-2024 14:31, Michal Wajdeczko wrote:

On 15.04.2024 06:49, Ghimiray, Himal Prasad wrote:

On 15-04-2024 00:31, Michal Wajdeczko wrote:

VF's drivers can't modify GGTT PTEs except the range explicitly
assigned by the PF driver. To allow hardware enforcement of this
requirement, each GGTT PTE has a field with the VF number that
identifies which VF can modify that particular GGTT PTE entry.

Only PF driver can modify this field and PF driver shall do that
before VF drivers will be loaded. Add function to prepare PTEs.
Since it will be used only by the PF driver, make it available
only for CONFIG_PCI_IOV=y.

Bspec: 45015
Signed-off-by: Michal Wajdeczko<michal.wajdeczko@intel.com>
---
  drivers/gpu/drm/xe/regs/xe_gtt_defs.h |  2 ++
  drivers/gpu/drm/xe/xe_ggtt.c          | 44 +++++++++++++++++++++++++++
  drivers/gpu/drm/xe/xe_ggtt.h          |  4 +++
  3 files changed, 50 insertions(+)

diff --git a/drivers/gpu/drm/xe/regs/xe_gtt_defs.h
b/drivers/gpu/drm/xe/regs/xe_gtt_defs.h
index 558519ce48c7..4389e5a76f89 100644
--- a/drivers/gpu/drm/xe/regs/xe_gtt_defs.h
+++ b/drivers/gpu/drm/xe/regs/xe_gtt_defs.h
@@ -9,6 +9,8 @@
  #define XELPG_GGTT_PTE_PAT0    BIT_ULL(52)
  #define XELPG_GGTT_PTE_PAT1    BIT_ULL(53)
  +#define GGTT_PTE_VFID        GENMASK_ULL(11, 2)

Patch looks good to me.

Just to confrim, based on BSPEC, this MASK applies to GRAPHICS_VER >=
12.50. For versions before this, the applicable mask is (4, 2). I infer
that since SRIOV support on XE is expected beyond GRAPHICS_VER version
12.50, you are using mask (11, 2).

true, but even on earlier versions, which we may want to use as SDV,
where real mask is (4, 2) this extended mask (11, 2) should be still
harmless as number of VFs would be then always less than 7 so it will
fit only into (4, 2)
Thanks for confirming. 



        

          
+
  #define GUC_GGTT_TOP        0xFEE00000
    #define XELPG_PPGTT_PTE_PAT3        BIT_ULL(62)
diff --git a/drivers/gpu/drm/xe/xe_ggtt.c b/drivers/gpu/drm/xe/xe_ggtt.c
index ff2239c0eda5..f090cab065b8 100644
--- a/drivers/gpu/drm/xe/xe_ggtt.c
+++ b/drivers/gpu/drm/xe/xe_ggtt.c
@@ -460,6 +460,50 @@ void xe_ggtt_remove_bo(struct xe_ggtt *ggtt,
struct xe_bo *bo)
                  bo->flags & XE_BO_FLAG_GGTT_INVALIDATE);
  }
  +#ifdef CONFIG_PCI_IOV
+static u64 xe_encode_vfid_pte(u16 vfid)
+{
+    return FIELD_PREP(GGTT_PTE_VFID, vfid) | XE_PAGE_PRESENT;
+}
+
+static void xe_ggtt_assign_locked(struct xe_ggtt *ggtt, const struct
drm_mm_node *node, u16 vfid)
+{
+    u64 start = node->start;
+    u64 size = node->size;
+    u64 end = start + size - 1;
+    u64 pte = xe_encode_vfid_pte(vfid);
+
+    lockdep_assert_held(&ggtt->lock);
+
+    if (!drm_mm_node_allocated(node))
+        return;
+
+    while (start < end) {
+        xe_ggtt_set_pte(ggtt, start, pte);
+        start += XE_PAGE_SIZE;
+    }
+
+    xe_ggtt_invalidate(ggtt);
+}
+
+/**
+ * xe_ggtt_assign - assign a GGTT region to the VF
+ * @ggtt: the &xe_ggtt where the node belongs
+ * @node: the &drm_mm_node to update
+ * @vfid: the VF identifier
+ *
+ * This function is used by the PF driver to assign a GGTT region to
the VF.
+ * In addition to PTE's VFID bits 11:2 also PRESENT bit 0 is set as
on some
+ * platforms VFs can't modify that either.


        

        
This info regarding bit 0 not being modifiable by VF's on some platforms
is missing in Bspec.


      

      
Piotr reminded me of another page:

Bspec: 52395

    

    
Thanks.

    
LGTM

    

    

    
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>

    

      



      

        

        

          
+ */
+void xe_ggtt_assign(struct xe_ggtt *ggtt, const struct drm_mm_node
*node, u16 vfid)
+{
+    mutex_lock(&ggtt->lock);
+    xe_ggtt_assign_locked(ggtt, node, vfid);
+    mutex_unlock(&ggtt->lock);
+}
+#endif
+
  int xe_ggtt_dump(struct xe_ggtt *ggtt, struct drm_printer *p)
  {
      int err;
diff --git a/drivers/gpu/drm/xe/xe_ggtt.h b/drivers/gpu/drm/xe/xe_ggtt.h
index 8306ef74abc6..4a41a1762358 100644
--- a/drivers/gpu/drm/xe/xe_ggtt.h
+++ b/drivers/gpu/drm/xe/xe_ggtt.h
@@ -33,4 +33,8 @@ void xe_ggtt_remove_bo(struct xe_ggtt *ggtt, struct
xe_bo *bo);
    int xe_ggtt_dump(struct xe_ggtt *ggtt, struct drm_printer *p);
  +#ifdef CONFIG_PCI_IOV
+void xe_ggtt_assign(struct xe_ggtt *ggtt, const struct drm_mm_node
*node, u16 vfid);
+#endif
+
  #endif


        

      

    

  


--------------LjjMg38QTjTK5A7Ny8l6RNSL--