From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2065.outbound.protection.outlook.com [40.107.92.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FB342248AC; Wed, 30 Jul 2025 21:51:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.65 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753912278; cv=fail; b=LmKPEZc8CFTCP/wiTZ2nlZwFlaNelupfkk9XUNYU9PsRXXnA50bF+tn25ssY3WmGczc3Fh3BDIL2ZanrErokFqLBPmGSn47b3rlNupRz547gZJh+jDM/4wqMwV2b8nfx9e31Ne+m6k+wZsCfZ9GEVKKoNsY/UVFM0qfLh8V0o98= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753912278; c=relaxed/simple; bh=tFZknpRsn9bbSM2vVM5NXw2q77TZbpbBSw5CxOr7324=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=TzGCkv5ro68bRfcF7TlLfykH7TRKbfclWYilo4jarO6X/GQ2UQIpDrFd5PudEmKff8Ae4dsh+u1sMmEUWdxYl3qhZgboCH6qQ1Cc062ZDhxwxHMC9Yb4QOOcrwCxbBw/irREqn7nArXLJBD8CjkiPUqF/O02/y45aEx0fw1IDc8= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Ses1iKpg; arc=fail smtp.client-ip=40.107.92.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Ses1iKpg" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CmPiEzQNEk78TTHhHNrHgiQuMCKXqS5PkcVQLELbBRaVj3H8NdAyCZ12SRTXu1QybuN8AM5m5CWstKwfgw9QGsXTlUH1b8SqhR5Kow0WdK8JIJmH0OMKrva0qjjLVeUN0WVmPCMqlSpCx9yhCq3tKjeQSpDhkLx3DswAUUKj9JtUBST00jue96iL2CoVcp7IT6OFWYYocDlX7TE4WmbeQ/juui0Q59ZShbfI8HyoqEkDfE4XXJOB+Ntxhet0rdFCxyzMMe7aslJ+VwAwJlDGPxZhHVmBWePnnGHM10c0IXPaf42ii8ABUp5TaK0NPqCMUdkx+0NSJmpVP+cfobOOCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JXdgrMRONTkLSLDw2UyXJ+BXEsnbtgjMqsw9XW4ClNc=; b=JM8uIOYFH8Bm3rvCAY3svQz3ksepqQ4etNC2MJtXl49dlnyvLvPNXEO7XpprIdWfg478R4Z4480FY/U2rl2qXFFm0Wy3AU0TDTWBkjmac4UoEw/KYzf5FoUv4yk+fnDuKJe2meFwcwzctfdx6VPFHNBzM+FAPrdbDn6e4ff9VouLOho8AlsWWSVR0lJSeiURmBT07j54tS0Nurfb984ug1+cqeytW06Bbj2UwpU3gMSYwaHrH7sYl866Tc+nnHmNH75gy+7tGoVzrNfnF4CG9iLMCbM8E0EpHiNq4ypRIsYjDeL/pKmtfulMnPGuMe+BHs3CbF7HuwdDhpBscO1gHg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JXdgrMRONTkLSLDw2UyXJ+BXEsnbtgjMqsw9XW4ClNc=; b=Ses1iKpgN1ABSqyXQG5BhGCLJ1SLTvQU56TXGnnJ4A5MLKHggzgvyFFN8kNwDgCCoOcTPS/Eu7GtBvicyBNdKZZzG3gfz5pHCQUfpyEWhVUxLf0eiZfRyh9RlkvmkqNyfMhTXKWVIMDE3PCUeerGRXFXdSdHuUvSz3ivvkmk38o= Received: from BL1PR13CA0271.namprd13.prod.outlook.com (2603:10b6:208:2bc::6) by SA1PR12MB6701.namprd12.prod.outlook.com (2603:10b6:806:251::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8989.11; Wed, 30 Jul 2025 21:51:12 +0000 Received: from MN1PEPF0000F0E5.namprd04.prod.outlook.com (2603:10b6:208:2bc:cafe::99) by BL1PR13CA0271.outlook.office365.com (2603:10b6:208:2bc::6) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9009.4 via Frontend Transport; Wed, 30 Jul 2025 21:51:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E5.mail.protection.outlook.com (10.167.242.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8989.10 via Frontend Transport; Wed, 30 Jul 2025 21:51:11 +0000 Received: from SCS-L-bcheatha.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 30 Jul 2025 16:51:03 -0500 From: Ben Cheatham To: , , CC: Ben Cheatham Subject: [PATCH 16/16] cxl/core, cxl/acpi: Add CXL isolation notify handler Date: Wed, 30 Jul 2025 16:47:18 -0500 Message-ID: <20250730214718.10679-17-Benjamin.Cheatham@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250730214718.10679-1-Benjamin.Cheatham@amd.com> References: <20250730214718.10679-1-Benjamin.Cheatham@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E5:EE_|SA1PR12MB6701:EE_ X-MS-Office365-Filtering-Correlation-Id: d5bde3a0-c35d-43a2-972b-08ddcfb33316 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|82310400026|376014|13003099007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ue7NFwbfXipoSudG61Ufm4XLkY8DJGP959ruv3zOEQEnElMr1s/CgXJ+Indc?= =?us-ascii?Q?1Uzs76JJxSf6maFcEglKainSHx7d841YJv6ZV7aYbXWpUf75hLJGdbwVHomP?= =?us-ascii?Q?zMQKFpQ5jBJYx5/mJQ8maWKbEdhBDht6SAnFCRpwS1GRqTh/6COFI69/tQL1?= =?us-ascii?Q?iR9bw8PE+AF6Xa2m/2phnUtWv2YnFTMLMFmeSzygjc9kSR8IDzRS5J7S3yfx?= =?us-ascii?Q?yjdfmO30yrskFHIoUUa4cXW6ZmvRbtyki3zzev7UOV/HGyLPZbHelr0umkf7?= =?us-ascii?Q?ujxdPrZyNvT9OwnT3J7AtDY8iqtObdJMWg6a8UTbWHu+oPZ/200wq1F0iRPX?= =?us-ascii?Q?PKwGZnaieBdPs8W9h6eHyAZDLVSOch1tLtJfhiSpI+i6/PnE1SE/dBrcyzwD?= =?us-ascii?Q?ovJhv6QrBtWiqC/Ktpk7JLFnCI73FdnD5PFmll4jMqI9DsIsrpK6tpFMda1W?= =?us-ascii?Q?kmz1CE3PWz3t9SakZG7Lf76KnN08hOspqYr3DoDCvURr3/nfdRo294WDLp8w?= =?us-ascii?Q?ru4JpvSf97j5tpowffTa6/RsyDmGDIfMhafUGh7DwUGlwoJfYYe6+2Mw8xgI?= =?us-ascii?Q?8bU58C5wZ/V95Jy06cjDHJtpPOB0GOnN6QQNctb1gmsk68PaRZGSTfQwM1os?= =?us-ascii?Q?Vx2UlviloWzKRHWAUZFZajmIfbZG3YqqWnCq59YFuLCr9WkMB78eOgrh0YtH?= =?us-ascii?Q?4TFF2yBSQriQx9m4l85qOlx6PeiAe/2pROXNGvtLqBKO9xlhOHlOLjHa/5xj?= =?us-ascii?Q?BBZuj6wJdOHP6R5DWigvHjvDCwvXLbu3DAWdD9dp4HRxAq7FumT9OJxDhb5j?= =?us-ascii?Q?tAo2lztt2UawP9dBaTkVKpCfsy95mSRNaHrRbCOj+4oY/Qswwpn0LAh5feW8?= =?us-ascii?Q?IiQSR2oVPI8dq2+UXNRTmI2xrV3nJ1nQx935kA4OjVSXcIu5t7OP5a8JG8NI?= =?us-ascii?Q?0+H/jPacbVAd0/5PTPkf4C2WBWgFC4mTvwjPAcDAJcTYOpw/2I0Mw0LlLR8c?= =?us-ascii?Q?est4mFIlHKtRQyUO3LekviP7uQ7HAyWgWk0mOvxgeJJqBF+EmdDgkQj0PLGo?= =?us-ascii?Q?8+j7G7FL4coMbfTVKZCqBdk9Wg0Q7xDizSh0HciYxdEpUJKlrrXdZGTVeUxP?= =?us-ascii?Q?KDAqQASmyHcyPEEzYCRlrfNCEDZw5kfx0oYx7D404xpuNwVHLgQKUcqcCTPJ?= =?us-ascii?Q?sNXz5gC0MXMzqo061JJDUyxNhkh8JFAuLd6x+qfT2tZU0xEWJAXWgnPPa5CY?= =?us-ascii?Q?knPxyw88QsK41P/bvdfR5a5M/NgqNMTqsFTjSXpb9Is5roUDTH5rHK7NwV75?= =?us-ascii?Q?LsoLJj720ZVGSja1hDf8fm1wOvPjr+GdTf+BTDQbqZ5amb6c+9F73Zt4uMp7?= =?us-ascii?Q?KIt6cwj+soxTGxDCvf3XH0t4insCrACPn/Nspj+0T7OgrdR4/oJDyMtdeXQW?= =?us-ascii?Q?nSDL5EoiuQMpLBJkHNDW1TkzTXTnNY3SoeHPpZDe5QaDf/KHrdF8WA=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(82310400026)(376014)(13003099007);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jul 2025 21:51:11.9169 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d5bde3a0-c35d-43a2-972b-08ddcfb33316 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6701 Install ACPI 0x80 notify handler for CXL isolation, based on the result of the _OSC negotiation. The handler is once installed per CXL host bridge (HID of "ACPI0016") as part of isolation set up. A link for the ECN (expected in CXL 4.0 spec) that introduces the relevant parts of the _OSC method (CXL 3.2 9.18.2) can be found below. spec). This link is only accesible to CXL SSWG members, so here's a brief overview: The ECN introduces a field in the _OSC method to control how the OSPM is notified isolation has occurred. If the ERR_COR Signaling Supported bit in the isolation capability register (CXL 3.2 8.2.4.24.1) is NOT set, this portion of the _OSC can be ignored. If the OSPM is given control of isolation notification, the mechanism to be used when isolation occurs is an MSI/-X interrupt (pre-ECN behavior). If the platorm firmware reserves control, the OSPM will be notified through a ACPI 0x80 notify on the CXL host bridge ACPI device (ACPI HID: "ACPI0016"). Link: https://members.computeexpresslink.org/wg/software_systems/document/3118 Signed-off-by: Ben Cheatham --- drivers/cxl/acpi.c | 51 +++++++++++++++++++++++++++++++++++++++++ drivers/cxl/core/port.c | 41 +++++++++++++++++++++++++-------- drivers/cxl/cxl.h | 3 +++ 3 files changed, 86 insertions(+), 9 deletions(-) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index b964f02fb56b..145a03f15255 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -390,10 +390,61 @@ static void decode_isolation_osc(struct cxl_port *hb, u32 iso_cap) hb->isolation_caps |= CXL_ISOLATION_INTERRUPTS; } +static void isolation_notify_handler(acpi_handle handle, u32 event, void *data) +{ + struct cxl_port *hb = data; + struct cxl_dport *dport; + unsigned long index; + + guard(device)(&hb->dev); + xa_for_each(&hb->dports, index, dport) { + if (dport->regs.isolation) + cxl_isolation_interrupt_handler(dport); + } +} + +static void cxl_remove_iso_handler(void *data) +{ + acpi_handle handle = data; + + acpi_remove_notify_handler(handle, ACPI_DEVICE_NOTIFY, + isolation_notify_handler); +} + +static int cxl_register_iso_handler(struct cxl_port *hb) +{ + struct acpi_device *adev = ACPI_COMPANION(hb->uport_dev); + acpi_handle handle; + int rc; + + if (!adev) + return -EINVAL; + + handle = acpi_device_handle(adev); + + guard(device)(&hb->dev); + if (devm_is_action_added(&hb->dev, cxl_remove_iso_handler, handle)) + return 0; + + rc = acpi_install_notify_handler(handle, ACPI_DEVICE_NOTIFY, + isolation_notify_handler, hb); + if (rc != AE_OK) + return -ENXIO; + + rc = devm_add_action_or_reset(&hb->dev, cxl_remove_iso_handler, + handle); + if (rc) + return rc; + + dev_dbg(&hb->dev, "Installed CXL isolation notify handler\n"); + return 0; +} + static const struct cxl_root_ops acpi_root_ops = { .qos_class = cxl_acpi_qos_class, .setup_hostbridge_uport = cxl_acpi_setup_hostbridge_uport, .get_isolation_caps = decode_isolation_osc, + .register_hb_isolation_handler = cxl_register_iso_handler, }; static void del_cxl_resource(struct resource *res) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index e9eb7a8a5f72..ece667f3aaf5 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1207,11 +1207,9 @@ struct isolation_intr_data { struct cxl_port *port; }; -static irqreturn_t cxl_isolation_thread(int irq, void *_data) +int cxl_isolation_interrupt_handler(struct cxl_dport *dport) { - struct isolation_intr_data *data = _data; - struct cxl_dport *dport = data->dport; - struct cxl_port *port = data->port; + struct cxl_port *port = dport->port; enum cxl_err_results res, acc; struct cxl_dev_state *cxlds; struct cxl_memdev *cxlmd; @@ -1221,10 +1219,6 @@ static irqreturn_t cxl_isolation_thread(int irq, void *_data) bool lnk_down; u32 status; - if (!dport || !port) - return IRQ_NONE; - - guard(device)(&port->dev); if (!dport->regs.isolation) goto panic; @@ -1255,8 +1249,9 @@ static irqreturn_t cxl_isolation_thread(int irq, void *_data) panic: panic("%s: downstream devices could not recover from CXL.mem link down\n", dev_name(dport->dport_dev)); - return IRQ_NONE; + return -ENXIO; } +EXPORT_SYMBOL_NS_GPL(cxl_isolation_interrupt_handler, "CXL"); static void cxl_dport_free_interrupts(void *data) { @@ -1274,6 +1269,24 @@ static void cxl_dport_free_interrupts(void *data) devm_free_irq(info->dev, info->irq, dport); } +static irqreturn_t cxl_isolation_thread(int irq, void *_data) +{ + struct isolation_intr_data *data = _data; + struct cxl_dport *dport = data->dport; + struct cxl_port *port = data->port; + int rc; + + if (!dport || !port) + return IRQ_NONE; + + guard(device)(&port->dev); + rc = cxl_isolation_interrupt_handler(dport); + if (rc) + return IRQ_NONE; + + return IRQ_HANDLED; +} + static int cxl_dport_setup_interrupts(struct device *host, struct cxl_dport *dport) { @@ -1282,6 +1295,16 @@ static int cxl_dport_setup_interrupts(struct device *host, u32 cap; int rc; + if (!(dport->port->isolation_caps & CXL_ISOLATION_INTERRUPTS)) { + struct cxl_root *root __free(put_cxl_root) = + find_cxl_root(dport->port); + if (!root || !root->ops || + !root->ops->register_hb_isolation_handler) + return -ENXIO; + + return root->ops->register_hb_isolation_handler(dport->port); + } + cap = readl(dport->regs.isolation + CXL_ISOLATION_CAPABILITY_OFFSET); if (!(cap & CXL_ISOLATION_CAP_INTR_SUPP)) return -ENXIO; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index aa36eba79181..68074c0d78d1 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -684,8 +684,11 @@ struct cxl_root_ops { struct device *bridge_dev); void (*get_isolation_caps)(struct cxl_port *hb, u32 iso_cap); + int (*register_hb_isolation_handler)(struct cxl_port *hb); }; +int cxl_isolation_interrupt_handler(struct cxl_dport *dport); + static inline struct cxl_dport * cxl_find_dport_by_dev(struct cxl_port *port, const struct device *dport_dev) { -- 2.34.1