From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E725FC4345F for ; Tue, 16 Apr 2024 17:17:00 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rwmPd-00073E-J1; Tue, 16 Apr 2024 13:16:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rwmPT-000730-0T for qemu-devel@nongnu.org; Tue, 16 Apr 2024 13:15:55 -0400 Received: from mail-dm6nam10on20716.outbound.protection.outlook.com ([2a01:111:f400:7e88::716] helo=NAM10-DM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rwmPQ-0004Rz-TJ for qemu-devel@nongnu.org; Tue, 16 Apr 2024 13:15:54 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HuT3y/UbleQiUIvqCgHA8MS1p6iptjYPEiaxrFoqQ4QuJJAPHR6r+6F0t8eaL03D0xwrqpY75o91Bz/4OPoU7xG4QLlaX2Toe4J0etwo5NToWCrqGJMvBaL95sCK23qaFpdb74ck4u7KE4kJ00pjDS58SE2mHKSsFB/qypG9x9mmXbfja/3M3q95Kv9/aXvQUnR7WVPY+Idh8xu6ag/ZYUBD2Nj0LGt8KeBtmahX94THlUs480L4VPz7XV4odklzhPqdLsoiOjcHkWnzqxByC6ZAvqPbApYso9Ax/rtgCcCLEUoziO+Ru8Q4wAPFXYn8qySz3k/XmoCcclkpmd7wWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dYWF4ZNL035o0lPcryp3L78l7KmeFmCRvi0DstKzmu4=; b=dxKzNAU9Ke8pcBJ19k9vxTpn+6xnY0GZjXhPs4N90IORI4I2lai5YLji/RzMxYVd91VbQY+4pCqoD6dyvuPovRmuHahKEtKDbFNvMlWqBgUkIkrxsRc76n2Wwg+JW0FzG+JJB7QWidFAKVDUA5497ZbXfcaVdUzXFRKGF7b958MWMbFTQ7hO+UHxfAzTWx7K+fCe82x+32dvjECLUGKLFny/AMHqTAl2KzSIj51z1Auju14NmKciQXb/uHjONzZuAz1KdNdKd6Z93gH2BWsUUoFvoQItfwQyyHAFsg9FiOuOU1agEW8uvNrMIm1338lBVKtt79z+0O8da8ZRcTsLXA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=memverge.com; dmarc=pass action=none header.from=memverge.com; dkim=pass header.d=memverge.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=memverge.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dYWF4ZNL035o0lPcryp3L78l7KmeFmCRvi0DstKzmu4=; b=du8ajFklrnSdBukw0OrO2Wourgyckx8ks5UfhwZkFSO4tq+rzQkVuk37usU60HVCPpchfqax2Dzl46lIqBF2XaYGxjDSpzBmlONAyOUPisgxgZWMGOvAHTChhWYFTnRSIrqFvsHlOjGHK3JZxFlHMRVN/3tULP2WaYEVqESW/n0= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=memverge.com; Received: from SJ0PR17MB5512.namprd17.prod.outlook.com (2603:10b6:a03:394::19) by SA1PR17MB5459.namprd17.prod.outlook.com (2603:10b6:806:1c2::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7452.50; Tue, 16 Apr 2024 17:15:49 +0000 Received: from SJ0PR17MB5512.namprd17.prod.outlook.com ([fe80::5d53:b947:4cab:2cc8]) by SJ0PR17MB5512.namprd17.prod.outlook.com ([fe80::5d53:b947:4cab:2cc8%4]) with mapi id 15.20.7452.049; Tue, 16 Apr 2024 17:15:48 +0000 Date: Tue, 16 Apr 2024 13:15:44 -0400 From: Gregory Price To: Jonathan Cameron Cc: fan , qemu-devel@nongnu.org, linux-cxl@vger.kernel.org, ira.weiny@intel.com, dan.j.williams@intel.com, a.manzanares@samsung.com, dave@stgolabs.net, nmtadam.samsung@gmail.com, jim.harris@samsung.com, Jorgen.Hansen@wdc.com, wj28.lee@gmail.com, Fan Ni Subject: Re: [PATCH v6 10/12] hw/mem/cxl_type3: Add dpa range validation for accesses to DC regions Message-ID: References: <20240325190339.696686-1-nifan.cxl@gmail.com> <20240325190339.696686-11-nifan.cxl@gmail.com> <20240416160056.0000325c@Huawei.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240416160056.0000325c@Huawei.com> X-ClientProxiedBy: SJ0PR13CA0186.namprd13.prod.outlook.com (2603:10b6:a03:2c3::11) To SJ0PR17MB5512.namprd17.prod.outlook.com (2603:10b6:a03:394::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR17MB5512:EE_|SA1PR17MB5459:EE_ X-MS-Office365-Filtering-Correlation-Id: 2379e9f3-ead5-462b-7628-08dc5e38dc16 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Q4XXbM7O479mwbEP6mCz81qR/CvZ/KZEXjMkREsW9f5//YXYbbEz35BM3QAjepKslp7N7YjtSOoHF3jO372mDSJMwG8Rm9p66XKbjkb1rzRZdF46MeP8CXWXcq6D5lcN5HIhr1ZIiwp+6fgW+tezMukHJmNxPiPmVUXslyoAVFQnI6+m5u6Hwt6kbILLWfi68igwaUZcxOIwMOja1iSyVNm1RHm4yyjvcOZP4W1DUX+S2gRPe2GnvSrGl2filHypIKYkYm/CE0PLgZ3StU9o8b6K2E1ImWLrhOCoDmXbvMcjmKzd4SwHUqof5LovHMNb0CP8D+b1N7mDvkqkTKDBkGUU0n3d8UaUP+e/O+MuRujQY9wFCU+Na6CwO9eNkOci0YGfhVbc4hW6nXnk3sQheLnkQDH6aqzk24jhNbBKOT5SKgFyZEL5PUwcg9WO1wWqxjh0vPm2J32hIBXfIUWRot3hFb4ngHKp0wI5RD3gNYqBDrzV4G1z2KIFiSbsRt6FY8Id2UQTbQPOAo1ALwqmz5WLp3fdXp+vLOETU06FgGfqLVDKDp21GYHiPubskazzgU4TH0QINk4Yrg+S9jX7VNFV0/SAgXO2XR2JaZPStI/r651YVNju2OeHKYMPCBespzIQ1gXD2ZXN8jyA/0aIYDob29DZtK8lUAdtEP5gWr0= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR17MB5512.namprd17.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366007)(1800799015)(376005)(7416005); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?HoRx1lVqcL2VRyV5c4+gIL0J2sEM8hq69+VwgOH1pv7JAYZCVs7KrelHTP9B?= =?us-ascii?Q?NDPTyJBKVikoW7iGhYbfSFQCJSv8SgR8UhDT96hmhGFTRkKbrXurAgFbcx/8?= =?us-ascii?Q?BADH+hl31nHAff+buytuywyIhtIpvwcjl3iTajUd3TL2i9gLBNKhuwmsomqr?= =?us-ascii?Q?G37b4YBUuQwMfFNBBw5npC7qYQawDkihAZUx2uYLOTu1Nj++eIB4Vbdnx4iI?= =?us-ascii?Q?2yd0OjgbYghTX6ccrCbHmSvI3VKZaYOcMwOU/4ogcqwmikmLg/FHpBaP+gVT?= =?us-ascii?Q?ljFanA+Ha89mAAfociuqfFNjajhABzNjOJrvm0I2/iRJnK9qtN2wM4mty7fZ?= =?us-ascii?Q?BVgNCxyhAjHN12aLbHmF+CdW2AUI7tX15WD3c4KjVBEg3cP2wUH84DqAdNYW?= =?us-ascii?Q?RN1lHvQGPGbeWQjti1FYyCoBTn6j2qJixELWQcLzORDI1HaWNZJgBXFlFITL?= =?us-ascii?Q?ztoRnjJyFQ93RY8c1ZjdaDp9qG0xBMTbk4YaUdcjV10Dfq70jP0xUfpLQc/V?= =?us-ascii?Q?dDr6lzQ6I+OC0u+1yDGHxZ/nd17A1sfqGYg8eu6HNQuicPvG9LPUifZzIWzD?= =?us-ascii?Q?3ONgEuAWpdb77+Abgh7jXiTfo2Tn1E4ErQJlKoXNSXa1DHmfk15wmdU7E5Hd?= =?us-ascii?Q?PcV1KKeQC7RRQtt2ZA9PfLT8BfVrmkZ3pyvpQ1FDCeYtyMMdRgC4J9r2gXrA?= =?us-ascii?Q?esUuQ7UrioL0QoYkQcIx5L8pP7N+vtfplpEmW2BVyBws6jPG9EAtR8H2IliK?= =?us-ascii?Q?BLeJ3SgzJVg1klSMfDjYYLd67HDlHNYy+fZfqFJ4N9iv9kP7Z4D/IdLncXVG?= =?us-ascii?Q?DlUZPqmTiNb2O/eASJnPQEDI8wYr9bqn2INJUIGf4v0obm/c5/g1XlrWp8Nu?= =?us-ascii?Q?pyJWS3y6b/f3iJ1obAGIBIXMzeU38z6ee4pkA6tEZ8+Pu5Hnxj/tcOwq7zSQ?= =?us-ascii?Q?yvcS5vMCtXdwrakQ2wVOMCQU7op3m2Dy+j6e+9syAtn2yJ6T6KGZKTRSO7Is?= =?us-ascii?Q?aM992AR+GFC+5WRVvVvT26WfKtwajM6mc4YHOzM81rit6AXER1RKhkC3b1ew?= =?us-ascii?Q?LCN0MqmA3ECWKI2bSIfZLMwoz9hgLdeNyjEHdK3mi2L/0XwftmkScvyUC1Ci?= =?us-ascii?Q?Fj9GjY6C07mvwXN2bp3OXuNQUFU/a44pSaYUojWHsnm/U3SQv8F8kQ/TWP/K?= =?us-ascii?Q?7rs0+MqXBWIyoKLjlFrDmqlgICj8YNBciOEw5R38a7dOWQEWsIDn+KN/k6hQ?= =?us-ascii?Q?PZuIzslrpU85Us853OkrbJy89S6D16CQ987uoWB/uK5lhnNxljg10xoFv1G9?= =?us-ascii?Q?FjbK4BwFo2J3NLa3M9cpQCAUH+JHubZUBHm7XZ522L2dYbvMcNrPUaDHTHJN?= =?us-ascii?Q?PoTbnZrt5izs3JV7mvekQNDicDkz65JTJCcjBW4tbVp6uk8bSbl/AOvrQwBp?= =?us-ascii?Q?It8V6qdiE3FlgZzM80wNH2AmzCKeFhrEHqCbS7w2qSs8iPzXHBV/POfnKK7G?= =?us-ascii?Q?S/uaM4NbHM63y7yw0UlTZmtJq7EoFxS1zG9QOIVclj+vE1NPGUi3/zF0Yrcu?= =?us-ascii?Q?AP3Q+PCW/ArvQia6ULy6KpK3juIjCypOuTbZ/844PajpsacJZq7LdurNYa2k?= =?us-ascii?Q?1A=3D=3D?= X-OriginatorOrg: memverge.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2379e9f3-ead5-462b-7628-08dc5e38dc16 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Apr 2024 17:15:48.5000 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xLx31i3em8W2mL1UpfmT2yzD7QSoM/+7ektmXmVWcUFz3E0+g5Z4JyGKM4XOfgn5HcqD0Vt2Nx/ye5LB/3Z50FFvsECIc/Z6q32Ox6PoYDk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR17MB5459 Received-SPF: pass client-ip=2a01:111:f400:7e88::716; envelope-from=gregory.price@memverge.com; helo=NAM10-DM6-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org On Tue, Apr 16, 2024 at 04:00:56PM +0100, Jonathan Cameron wrote: > On Mon, 15 Apr 2024 10:37:00 -0700 > fan wrote: > > > On Fri, Apr 12, 2024 at 06:54:42PM -0400, Gregory Price wrote: > > > On Mon, Mar 25, 2024 at 12:02:28PM -0700, nifan.cxl@gmail.com wrote: > > > > From: Fan Ni > > > > > > > > All dpa ranges in the DC regions are invalid to access until an extent > > > > covering the range has been added. Add a bitmap for each region to > > > > record whether a DC block in the region has been backed by DC extent. > > > > For the bitmap, a bit in the bitmap represents a DC block. When a DC > > > > extent is added, all the bits of the blocks in the extent will be set, > > > > which will be cleared when the extent is released. > > > > > > > > Reviewed-by: Jonathan Cameron > > > > Signed-off-by: Fan Ni > > > > --- > > > > hw/cxl/cxl-mailbox-utils.c | 6 +++ > > > > hw/mem/cxl_type3.c | 76 +++++++++++++++++++++++++++++++++++++ > > > > include/hw/cxl/cxl_device.h | 7 ++++ > > > > 3 files changed, 89 insertions(+) > > > > > > > > diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c > > > > index 7094e007b9..a0d2239176 100644 > > > > --- a/hw/cxl/cxl-mailbox-utils.c > > > > +++ b/hw/cxl/cxl-mailbox-utils.c > > > > @@ -1620,6 +1620,7 @@ static CXLRetCode cmd_dcd_add_dyn_cap_rsp(const struct cxl_cmd *cmd, > > > > > > > > cxl_insert_extent_to_extent_list(extent_list, dpa, len, NULL, 0); > > > > ct3d->dc.total_extent_count += 1; > > > > + ct3_set_region_block_backed(ct3d, dpa, len); > > > > > > > > ent = QTAILQ_FIRST(&ct3d->dc.extents_pending); > > > > cxl_remove_extent_from_extent_list(&ct3d->dc.extents_pending, ent); > > > > > > while looking at the MHD code, we had decided to "reserve" the blocks in > > > the bitmap in the call to `qmp_cxl_process_dynamic_capacity` in order to > > > prevent a potential double-allocation (basically we need to sanity check > > > that two hosts aren't reserving the region PRIOR to the host being > > > notified). > > > > > > I did not see any checks in the `qmp_cxl_process_dynamic_capacity` path > > > to prevent pending extents from being double-allocated. Is this an > > > explicit choice? > > > > > > I can see, for example, why you may want to allow the following in the > > > pending list: [Add X, Remove X, Add X]. I just want to know if this is > > > intentional or not. If not, you may consider adding a pending check > > > during the sanity check phase of `qmp_cxl_process_dynamic_capacity` > > > > > > ~Gregory > > > > First, for remove request, pending list is not involved. See cxl r3.1, > > 9.13.3.3. Pending basically means "pending to add". > > So for the above example, in the pending list, you can see [Add x, add x] if the > > event is not processed in time. > > Second, from the spec, I cannot find any text saying we cannot issue > > another add extent X if it is still pending. > > I think there is text saying that the capacity is not released for reuse > by the device until it receives a response from the host. Whilst > it's not explicit on offers to the same host, I'm not sure that matters. > So I don't think it is suppose to queue multiple extents... > > It definitely should not release capacity until it receives a response, because the host could tell the device to kick rocks (which would be reasonable under a variety of circumstances). ~Gregory