From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72884C433FE for ; Wed, 30 Nov 2022 14:32:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229551AbiK3OcR (ORCPT ); Wed, 30 Nov 2022 09:32:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230071AbiK3Obo (ORCPT ); Wed, 30 Nov 2022 09:31:44 -0500 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DACFA8138F for ; Wed, 30 Nov 2022 06:31:40 -0800 (PST) Received: from fraeml710-chm.china.huawei.com (unknown [172.18.147.200]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4NMhTZ0jb4z6H6n5; Wed, 30 Nov 2022 22:31:14 +0800 (CST) Received: from lhrpeml500005.china.huawei.com (7.191.163.240) by fraeml710-chm.china.huawei.com (10.206.15.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Wed, 30 Nov 2022 15:31:38 +0100 Received: from localhost (10.202.227.76) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Wed, 30 Nov 2022 14:31:37 +0000 Date: Wed, 30 Nov 2022 14:31:36 +0000 From: Jonathan Cameron To: CC: Dan Williams , Ira Weiny , Vishal Verma , Ben Widawsky , Dave Jiang , Subject: Re: [PATCH 1/5] cxl/memdev: Add support for the Inject Poison mailbox command Message-ID: <20221130143136.00003808@Huawei.com> In-Reply-To: <3c260749c833f51d5cad9ae3912debcdf8b82753.1669781852.git.alison.schofield@intel.com> References: <3c260749c833f51d5cad9ae3912debcdf8b82753.1669781852.git.alison.schofield@intel.com> Organization: Huawei Technologies Research and Development (UK) Ltd. X-Mailer: Claws Mail 4.1.0 (GTK 3.24.33; x86_64-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.202.227.76] X-ClientProxiedBy: lhrpeml100004.china.huawei.com (7.191.162.219) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org On Tue, 29 Nov 2022 20:34:33 -0800 alison.schofield@intel.com wrote: > From: Alison Schofield > > CXL devices optionally support the INJECT POISON mailbox command. Add > a sysfs attribute and memdev driver support for injecting poison. > > When a Device Physical Address (DPA) is written to the inject_poison > sysfs attribute send an inject poison command to the device for the > specified address. > > Per the CXL Specification (8.2.9.8.4.2), after receiving a valid > inject poison request, the device will return poison when the address > is accessed through the CXL.mem bus. Injecting poison adds the address > to the device's Poison List and the error source is set to injected > error. In addition, the device adds a poison creation event to its > internal Informational Event log, updates the Event Status register, > and if configured, interrupts the host. > > Also, per the CXL Specification, it is not an error to inject poison > into an address that already has poison present and no error is returned > from the device. The memdev driver performs basic sanity checking on the > address, however, it does not go as far as reading the poison list to see > if the address is on the list. That discovery is left to the device. > > The inject_poison attribute is only visible for devices supporting > the capability. > > Signed-off-by: Alison Schofield A few trivial things inline. With those fixes LGTM Reviewed-by: Jonathan Cameron > --- > Documentation/ABI/testing/sysfs-bus-cxl | 19 +++++++++ > drivers/cxl/core/memdev.c | 53 +++++++++++++++++++++++++ > drivers/cxl/cxlmem.h | 3 ++ > 3 files changed, 75 insertions(+) > > diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl > index b715a4609718..20db97f7a1aa 100644 > --- a/Documentation/ABI/testing/sysfs-bus-cxl > +++ b/Documentation/ABI/testing/sysfs-bus-cxl > @@ -416,3 +416,22 @@ Description: > if accessed, and the source of the poison. The retrieved > errors are logged as kernel trace events with the label > 'cxl_poison'. > + > + > +What: /sys/bus/cxl/devices/memX/inject_poison > +Date: December, 2022 > +KernelVersion: v6.2 > +Contact: linux-cxl@vger.kernel.org > +Description: > + (WO) When a Device Physical Address (DPA) is written to this > + attribute the memdev driver sends an inject poison command to > + the device for the specified address. If successful, the device > + returns poison when the address is accessed through the CXL.mem > + bus. Injecting poison adds the address to the device's Poison > + List and the error source is set to injected error. In addition, "set to Injected." perhaps to match spec naming in Media Error Record. > + the device adds a poison creation event to its internal > + Informational Event log, updates the Event Status register, and > + if configured, interrupts the host. It is not an error to inject > + poison into an address that already has poison present and no > + error is returned. The inject_poison attribute is only visible > + for devices supporting the capability. White space issues (spaces instead of tabs?) Add something about the masked bits / granularity of addresses that are accepted. > diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c > index d08b7295a01c..71130813030f 100644 > --- a/drivers/cxl/core/memdev.c > +++ b/drivers/cxl/core/memdev.c > @@ -142,6 +142,51 @@ static ssize_t trigger_poison_list_store(struct device *dev, > } > static DEVICE_ATTR_WO(trigger_poison_list); > > +static int cxl_validate_poison_dpa(struct cxl_dev_state *cxlds, u64 dpa) > +{ > + if (!resource_size(&cxlds->dpa_res)) { > + dev_dbg(cxlds->dev, "device has no dpa resource\n"); > + return -EINVAL; > + } > + if (dpa < cxlds->dpa_res.start || dpa > cxlds->dpa_res.end) { > + dev_dbg(cxlds->dev, "dpa:0x%llx not in resource:%pR\n", > + dpa, &cxlds->dpa_res); > + return -EINVAL; > + } > + if ((dpa & CXL_POISON_INJECT_RESERVED) != 0) { > + dev_dbg(cxlds->dev, "dpa reserve bit(s) [5:0] set 0x%llx\n", > + dpa); > + return -EINVAL; > + } > + return 0; > +} > + > +static ssize_t inject_poison_store(struct device *dev, > + struct device_attribute *attr, > + const char *buf, size_t len) > +{ > + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); > + struct cxl_dev_state *cxlds = cxlmd->cxlds; > + u64 dpa; > + int rc; > + > + rc = kstrtou64(buf, 0, &dpa); > + if (rc) > + return rc; > + rc = cxl_validate_poison_dpa(cxlds, dpa); > + if (rc) > + return rc; > + > + dpa = cpu_to_le64(dpa); > + rc = cxl_mbox_send_cmd(cxlds, CXL_MBOX_OP_INJECT_POISON, &dpa, Endianness? > + sizeof(dpa), NULL, cxlds->payload_size); > + if (rc) > + return rc; > + > + return len; > +} > +static DEVICE_ATTR_WO(inject_poison); > + > static struct attribute *cxl_memdev_attributes[] = { > &dev_attr_serial.attr, > &dev_attr_firmware_version.attr, > @@ -149,6 +194,7 @@ static struct attribute *cxl_memdev_attributes[] = { > &dev_attr_label_storage_size.attr, > &dev_attr_numa_node.attr, > &dev_attr_trigger_poison_list.attr, > + &dev_attr_inject_poison.attr, > NULL, > }; > > @@ -175,6 +221,13 @@ static umode_t cxl_memdev_visible(struct kobject *kobj, struct attribute *a, > to_cxl_memdev(dev)->cxlds->enabled_cmds)) > return 0; > } > + if (a == &dev_attr_inject_poison.attr) { > + struct device *dev = kobj_to_dev(kobj); > + > + if (!test_bit(CXL_MEM_COMMAND_ID_INJECT_POISON, > + to_cxl_memdev(dev)->cxlds->enabled_cmds)) > + return 0; > + } > return a->mode; > } > > diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h > index 19a9e545ac19..0d4c34be7335 100644 > --- a/drivers/cxl/cxlmem.h > +++ b/drivers/cxl/cxlmem.h > @@ -396,6 +396,9 @@ struct cxl_mbox_poison_payload_out { > #define CXL_POISON_SOURCE_INJECTED 3 > #define CXL_POISON_SOURCE_VENDOR 7 > > +/* Inject & Clear Poison CXL 3.0 Spec 8.2.9.8.4.2/3 */ > +#define CXL_POISON_INJECT_RESERVED GENMASK_ULL(5, 0) > + > /** > * struct cxl_mem_command - Driver representation of a memory device command > * @info: Command information as it exists for the UAPI