* Re: [PATCH v4 10/25] nvdimm: Add driver for OpenCAPI Persistent Memory
From: Matthew Wilcox @ 2020-03-29 2:56 UTC (permalink / raw)
To: Alastair D'Silva
Cc: Madhavan Srinivasan, Alexey Kardashevskiy, Masahiro Yamada,
Oliver O'Halloran, Mauro Carvalho Chehab, Ira Weiny,
Thomas Gleixner, Rob Herring, Dave Jiang, linux-nvdimm,
Aneesh Kumar K . V, Krzysztof Kozlowski, Anju T Sudhakar,
Mahesh Salgaonkar, Andrew Donnellan, Arnd Bergmann, Greg Kurz,
Nicholas Piggin, Cédric Le Goater, Dan Williams,
Hari Bathini, linux-mm, Greg Kroah-Hartman, linux-kernel,
Vishal Verma, Frederic Barrat, Paul Mackerras, Andrew Morton,
linuxppc-dev, David S. Miller
In-Reply-To: <20200327071202.2159885-11-alastair@d-silva.org>
On Fri, Mar 27, 2020 at 06:11:47PM +1100, Alastair D'Silva wrote:
> +static struct mutex minors_idr_lock;
> +static struct idr minors_idr;
...
> + mutex_lock(&minors_idr_lock);
> + minor = idr_alloc(&minors_idr, ocxlpmem, 0, NUM_MINORS, GFP_KERNEL);
> + mutex_unlock(&minors_idr_lock);
...
> + mutex_lock(&minors_idr_lock);
> + idr_remove(&minors_idr, MINOR(ocxlpmem->dev.devt));
> + mutex_unlock(&minors_idr_lock);
...
> + mutex_init(&minors_idr_lock);
> + idr_init(&minors_idr);
Unless you look up ocxlpmem by minor number later in the patch series (and
most of the series didn't make it to my mailbox), this can just be an ida.
static DEFINE_IDA(minors);
...
minor = ida_alloc_max(&minors, NUM_MINORS, GFP_KERNEL);
...
ida_free(&minors, MINOR(ocxlpmem->dev.devt));
...
and you can drop the dynamic initialisation. And the mutex.
^ permalink raw reply
* [PATCH v4 09/25] ocxl: Save the device serial number in ocxl_fn
From: Alastair D'Silva @ 2020-03-27 7:11 UTC (permalink / raw)
To: alastair
Cc: Madhavan Srinivasan, Alexey Kardashevskiy, Masahiro Yamada,
Oliver O'Halloran, Mauro Carvalho Chehab, Ira Weiny,
Thomas Gleixner, Rob Herring, Dave Jiang, linux-nvdimm,
Aneesh Kumar K . V, Krzysztof Kozlowski, Anju T Sudhakar,
Mahesh Salgaonkar, Andrew Donnellan, Arnd Bergmann, Greg Kurz,
Nicholas Piggin, Cédric Le Goater, Dan Williams,
Hari Bathini, linux-mm, Greg Kroah-Hartman, linux-kernel,
Vishal Verma, Frederic Barrat, Paul Mackerras, Andrew Morton,
linuxppc-dev, David S. Miller
In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org>
This patch retrieves the serial number of the card and makes it available
to consumers of the ocxl driver via the ocxl_fn struct.
Signed-off-by: Alastair D'Silva <alastair@d-silva.org>
Acked-by: Frederic Barrat <fbarrat@linux.ibm.com>
Acked-by: Andrew Donnellan <ajd@linux.ibm.com>
---
drivers/misc/ocxl/config.c | 46 ++++++++++++++++++++++++++++++++++++++
include/misc/ocxl.h | 1 +
2 files changed, 47 insertions(+)
diff --git a/drivers/misc/ocxl/config.c b/drivers/misc/ocxl/config.c
index 69cca341d446..2b835e57daca 100644
--- a/drivers/misc/ocxl/config.c
+++ b/drivers/misc/ocxl/config.c
@@ -71,6 +71,51 @@ static int find_dvsec_afu_ctrl(struct pci_dev *dev, u8 afu_idx)
return 0;
}
+/**
+ * get_function_0() - Find a related PCI device (function 0)
+ * @device: PCI device to match
+ *
+ * Returns a pointer to the related device, or null if not found
+ */
+static struct pci_dev *get_function_0(struct pci_dev *dev)
+{
+ unsigned int devfn = PCI_DEVFN(PCI_SLOT(dev->devfn), 0);
+
+ return pci_get_domain_bus_and_slot(pci_domain_nr(dev->bus),
+ dev->bus->number, devfn);
+}
+
+static void read_serial(struct pci_dev *dev, struct ocxl_fn_config *fn)
+{
+ u32 low, high;
+ int pos;
+
+ pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DSN);
+ if (pos) {
+ pci_read_config_dword(dev, pos + 0x04, &low);
+ pci_read_config_dword(dev, pos + 0x08, &high);
+
+ fn->serial = low | ((u64)high) << 32;
+
+ return;
+ }
+
+ if (PCI_FUNC(dev->devfn) != 0) {
+ struct pci_dev *related = get_function_0(dev);
+
+ if (!related) {
+ fn->serial = 0;
+ return;
+ }
+
+ read_serial(related, fn);
+ pci_dev_put(related);
+ return;
+ }
+
+ fn->serial = 0;
+}
+
static void read_pasid(struct pci_dev *dev, struct ocxl_fn_config *fn)
{
u16 val;
@@ -208,6 +253,7 @@ int ocxl_config_read_function(struct pci_dev *dev, struct ocxl_fn_config *fn)
int rc;
read_pasid(dev, fn);
+ read_serial(dev, fn);
rc = read_dvsec_tl(dev, fn);
if (rc) {
diff --git a/include/misc/ocxl.h b/include/misc/ocxl.h
index d8b0b4d46bfb..b8514dc64bd0 100644
--- a/include/misc/ocxl.h
+++ b/include/misc/ocxl.h
@@ -46,6 +46,7 @@ struct ocxl_fn_config {
int dvsec_afu_info_pos; /* offset of the AFU information DVSEC */
s8 max_pasid_log;
s8 max_afu_index;
+ u64 serial;
};
enum ocxl_endian {
--
2.24.1
^ permalink raw reply related
* [PATCH v4 10/25] nvdimm: Add driver for OpenCAPI Persistent Memory
From: Alastair D'Silva @ 2020-03-27 7:11 UTC (permalink / raw)
To: alastair
Cc: Madhavan Srinivasan, Alexey Kardashevskiy, Masahiro Yamada,
Oliver O'Halloran, Mauro Carvalho Chehab, Ira Weiny,
Thomas Gleixner, Rob Herring, Dave Jiang, linux-nvdimm,
Aneesh Kumar K . V, Krzysztof Kozlowski, Anju T Sudhakar,
Mahesh Salgaonkar, Andrew Donnellan, Arnd Bergmann, Greg Kurz,
Nicholas Piggin, Cédric Le Goater, Dan Williams,
Hari Bathini, linux-mm, Greg Kroah-Hartman, linux-kernel,
Vishal Verma, Frederic Barrat, Paul Mackerras, Andrew Morton,
linuxppc-dev, David S. Miller
In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org>
This driver exposes LPC memory on OpenCAPI pmem cards
as an NVDIMM, allowing the existing nvram infrastructure
to be used.
Namespace metadata is stored on the media itself, so
scm_reserve_metadata() maps 1 section's worth of PMEM storage
at the start to hold this. The rest of the PMEM range is registered
with libnvdimm as an nvdimm. ndctl_config_read/write/size() provide
callbacks to libnvdimm to access the metadata.
Signed-off-by: Alastair D'Silva <alastair@d-silva.org>
---
drivers/nvdimm/Kconfig | 2 +
drivers/nvdimm/Makefile | 1 +
drivers/nvdimm/ocxl/Kconfig | 15 ++
drivers/nvdimm/ocxl/Makefile | 7 +
drivers/nvdimm/ocxl/main.c | 476 +++++++++++++++++++++++++++++++++
drivers/nvdimm/ocxl/ocxlpmem.h | 23 ++
6 files changed, 524 insertions(+)
create mode 100644 drivers/nvdimm/ocxl/Kconfig
create mode 100644 drivers/nvdimm/ocxl/Makefile
create mode 100644 drivers/nvdimm/ocxl/main.c
create mode 100644 drivers/nvdimm/ocxl/ocxlpmem.h
diff --git a/drivers/nvdimm/Kconfig b/drivers/nvdimm/Kconfig
index b7d1eb38b27d..368328637182 100644
--- a/drivers/nvdimm/Kconfig
+++ b/drivers/nvdimm/Kconfig
@@ -131,4 +131,6 @@ config NVDIMM_TEST_BUILD
core devm_memremap_pages() implementation and other
infrastructure.
+source "drivers/nvdimm/ocxl/Kconfig"
+
endif
diff --git a/drivers/nvdimm/Makefile b/drivers/nvdimm/Makefile
index 29203f3d3069..bc02be11c794 100644
--- a/drivers/nvdimm/Makefile
+++ b/drivers/nvdimm/Makefile
@@ -33,3 +33,4 @@ libnvdimm-$(CONFIG_NVDIMM_KEYS) += security.o
TOOLS := ../../tools
TEST_SRC := $(TOOLS)/testing/nvdimm/test
obj-$(CONFIG_NVDIMM_TEST_BUILD) += $(TEST_SRC)/iomap.o
+obj-$(CONFIG_LIBNVDIMM) += ocxl/
diff --git a/drivers/nvdimm/ocxl/Kconfig b/drivers/nvdimm/ocxl/Kconfig
new file mode 100644
index 000000000000..c5d927520920
--- /dev/null
+++ b/drivers/nvdimm/ocxl/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+if LIBNVDIMM
+
+config OCXL_PMEM
+ tristate "OpenCAPI Persistent Memory"
+ depends on LIBNVDIMM && PPC_POWERNV && PCI && EEH && ZONE_DEVICE && OCXL
+ help
+ Exposes devices that implement the OpenCAPI Storage Class Memory
+ specification as persistent memory regions. You may also want
+ DEV_DAX, DEV_DAX_PMEM & FS_DAX if you plan on using DAX devices
+ stacked on top of this driver.
+
+ Select N if unsure.
+
+endif
diff --git a/drivers/nvdimm/ocxl/Makefile b/drivers/nvdimm/ocxl/Makefile
new file mode 100644
index 000000000000..e0e8ade1987a
--- /dev/null
+++ b/drivers/nvdimm/ocxl/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0
+
+ccflags-$(CONFIG_PPC_WERROR) += -Werror
+
+obj-$(CONFIG_OCXL_PMEM) += ocxlpmem.o
+
+ocxlpmem-y := main.o
\ No newline at end of file
diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c
new file mode 100644
index 000000000000..c0066fedf9cc
--- /dev/null
+++ b/drivers/nvdimm/ocxl/main.c
@@ -0,0 +1,476 @@
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright 2020 IBM Corp.
+
+/*
+ * A driver for OpenCAPI devices that implement the Storage Class
+ * Memory specification.
+ */
+
+#include <linux/module.h>
+#include <misc/ocxl.h>
+#include <linux/ndctl.h>
+#include <linux/mm_types.h>
+#include <linux/memory_hotplug.h>
+#include "ocxlpmem.h"
+
+static const struct pci_device_id pci_tbl[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_IBM, 0x0625), },
+ { }
+};
+
+MODULE_DEVICE_TABLE(pci, pci_tbl);
+
+#define NUM_MINORS 256 // Total to reserve
+
+static dev_t ocxlpmem_dev;
+static struct class *ocxlpmem_class;
+static struct mutex minors_idr_lock;
+static struct idr minors_idr;
+
+/**
+ * ndctl_config_write() - Handle a ND_CMD_SET_CONFIG_DATA command from ndctl
+ * @ocxlpmem: the device metadata
+ * @command: the incoming data to write
+ * Return: 0 on success, negative on failure
+ */
+static int ndctl_config_write(struct ocxlpmem *ocxlpmem,
+ struct nd_cmd_set_config_hdr *command)
+{
+ if (command->in_offset + command->in_length > LABEL_AREA_SIZE)
+ return -EINVAL;
+
+ memcpy_flushcache(ocxlpmem->metadata_addr + command->in_offset,
+ command->in_buf, command->in_length);
+
+ return 0;
+}
+
+/**
+ * ndctl_config_read() - Handle a ND_CMD_GET_CONFIG_DATA command from ndctl
+ * @ocxlpmem: the device metadata
+ * @command: the read request
+ * Return: 0 on success, negative on failure
+ */
+static int ndctl_config_read(struct ocxlpmem *ocxlpmem,
+ struct nd_cmd_get_config_data_hdr *command)
+{
+ if (command->in_offset + command->in_length > LABEL_AREA_SIZE)
+ return -EINVAL;
+
+ memcpy_mcsafe(command->out_buf,
+ ocxlpmem->metadata_addr + command->in_offset,
+ command->in_length);
+
+ return 0;
+}
+
+/**
+ * ndctl_config_size() - Handle a ND_CMD_GET_CONFIG_SIZE command from ndctl
+ * @command: the read request
+ * Return: 0 on success, negative on failure
+ */
+static int ndctl_config_size(struct nd_cmd_get_config_size *command)
+{
+ command->status = 0;
+ command->config_size = LABEL_AREA_SIZE;
+ command->max_xfer = PAGE_SIZE;
+
+ return 0;
+}
+
+static int ndctl(struct nvdimm_bus_descriptor *nd_desc,
+ struct nvdimm *nvdimm,
+ unsigned int cmd, void *buf, unsigned int buf_len, int *cmd_rc)
+{
+ struct ocxlpmem *ocxlpmem = container_of(nd_desc,
+ struct ocxlpmem, bus_desc);
+
+ switch (cmd) {
+ case ND_CMD_GET_CONFIG_SIZE:
+ *cmd_rc = ndctl_config_size(buf);
+ return 0;
+
+ case ND_CMD_GET_CONFIG_DATA:
+ *cmd_rc = ndctl_config_read(ocxlpmem, buf);
+ return 0;
+
+ case ND_CMD_SET_CONFIG_DATA:
+ *cmd_rc = ndctl_config_write(ocxlpmem, buf);
+ return 0;
+
+ default:
+ return -ENOTTY;
+ }
+}
+
+/**
+ * reserve_metadata() - Reserve space for nvdimm metadata
+ * @ocxlpmem: the device metadata
+ * @lpc_mem: The resource representing the LPC memory of the OpenCAPI device
+ */
+static int reserve_metadata(struct ocxlpmem *ocxlpmem,
+ struct resource *lpc_mem)
+{
+ ocxlpmem->metadata_addr = devm_memremap(&ocxlpmem->dev, lpc_mem->start,
+ LABEL_AREA_SIZE, MEMREMAP_WB);
+ if (IS_ERR(ocxlpmem->metadata_addr))
+ return PTR_ERR(ocxlpmem->metadata_addr);
+
+ return 0;
+}
+
+/**
+ * register_lpc_mem() - Discover persistent memory on a device and register it with the NVDIMM subsystem
+ * @ocxlpmem: the device metadata
+ * Return: 0 on success
+ */
+static int register_lpc_mem(struct ocxlpmem *ocxlpmem)
+{
+ struct nd_region_desc region_desc;
+ struct nd_mapping_desc nd_mapping_desc;
+ struct resource *lpc_mem;
+ const struct ocxl_afu_config *config;
+ const struct ocxl_fn_config *fn_config;
+ int rc;
+ unsigned long nvdimm_cmd_mask = 0;
+ unsigned long nvdimm_flags = 0;
+ int target_node;
+ char serial[16 + 1];
+
+ // Set up the reserved metadata area
+ rc = ocxl_afu_map_lpc_mem(ocxlpmem->ocxl_afu);
+ if (rc < 0)
+ return rc;
+
+ lpc_mem = ocxl_afu_lpc_mem(ocxlpmem->ocxl_afu);
+ if (!lpc_mem || !lpc_mem->start)
+ return -EINVAL;
+
+ config = ocxl_afu_config(ocxlpmem->ocxl_afu);
+ fn_config = ocxl_function_config(ocxlpmem->ocxl_fn);
+
+ rc = reserve_metadata(ocxlpmem, lpc_mem);
+ if (rc)
+ return rc;
+
+ ocxlpmem->bus_desc.provider_name = "ocxlpmem";
+ ocxlpmem->bus_desc.ndctl = ndctl;
+ ocxlpmem->bus_desc.module = THIS_MODULE;
+
+ ocxlpmem->nvdimm_bus = nvdimm_bus_register(&ocxlpmem->dev,
+ &ocxlpmem->bus_desc);
+ if (!ocxlpmem->nvdimm_bus)
+ return -EINVAL;
+
+ ocxlpmem->pmem_res.start = (u64)lpc_mem->start + LABEL_AREA_SIZE;
+ ocxlpmem->pmem_res.end = (u64)lpc_mem->start + config->lpc_mem_size - 1;
+ ocxlpmem->pmem_res.name = "OpenCAPI persistent memory";
+
+ set_bit(ND_CMD_GET_CONFIG_SIZE, &nvdimm_cmd_mask);
+ set_bit(ND_CMD_GET_CONFIG_DATA, &nvdimm_cmd_mask);
+ set_bit(ND_CMD_SET_CONFIG_DATA, &nvdimm_cmd_mask);
+
+ set_bit(NDD_ALIASING, &nvdimm_flags);
+
+ snprintf(serial, sizeof(serial), "%llx", fn_config->serial);
+ nd_mapping_desc.nvdimm = nvdimm_create(ocxlpmem->nvdimm_bus, ocxlpmem,
+ NULL, nvdimm_flags,
+ nvdimm_cmd_mask, 0, NULL);
+ if (!nd_mapping_desc.nvdimm)
+ return -ENOMEM;
+
+ if (nvdimm_bus_check_dimm_count(ocxlpmem->nvdimm_bus, 1))
+ return -EINVAL;
+
+ nd_mapping_desc.start = ocxlpmem->pmem_res.start;
+ nd_mapping_desc.size = resource_size(&ocxlpmem->pmem_res);
+ nd_mapping_desc.position = 0;
+
+ ocxlpmem->nd_set.cookie1 = fn_config->serial;
+ ocxlpmem->nd_set.cookie2 = fn_config->serial;
+
+ target_node = of_node_to_nid(ocxlpmem->pdev->dev.of_node);
+
+ memset(®ion_desc, 0, sizeof(region_desc));
+ region_desc.res = &ocxlpmem->pmem_res;
+ region_desc.numa_node = NUMA_NO_NODE;
+ region_desc.target_node = target_node;
+ region_desc.num_mappings = 1;
+ region_desc.mapping = &nd_mapping_desc;
+ region_desc.nd_set = &ocxlpmem->nd_set;
+
+ set_bit(ND_REGION_PAGEMAP, ®ion_desc.flags);
+ /*
+ * NB: libnvdimm copies the data from ndr_desc into it's own
+ * structures so passing a stack pointer is fine.
+ */
+ ocxlpmem->nd_region = nvdimm_pmem_region_create(ocxlpmem->nvdimm_bus,
+ ®ion_desc);
+ if (!ocxlpmem->nd_region)
+ return -EINVAL;
+
+ dev_info(&ocxlpmem->dev,
+ "Onlining %lluMB of persistent memory\n",
+ nd_mapping_desc.size / SZ_1M);
+
+ return 0;
+}
+
+/**
+ * allocate_minor() - Allocate a minor number to use for an OpenCAPI pmem device
+ * @ocxlpmem: the device metadata
+ * Return: the allocated minor number
+ */
+static int allocate_minor(struct ocxlpmem *ocxlpmem)
+{
+ int minor;
+
+ mutex_lock(&minors_idr_lock);
+ minor = idr_alloc(&minors_idr, ocxlpmem, 0, NUM_MINORS, GFP_KERNEL);
+ mutex_unlock(&minors_idr_lock);
+ return minor;
+}
+
+static void free_minor(struct ocxlpmem *ocxlpmem)
+{
+ mutex_lock(&minors_idr_lock);
+ idr_remove(&minors_idr, MINOR(ocxlpmem->dev.devt));
+ mutex_unlock(&minors_idr_lock);
+}
+
+/**
+ * free_ocxlpmem() - Free all members of an ocxlpmem struct
+ * @ocxlpmem: the device struct to clear
+ */
+static void free_ocxlpmem(struct ocxlpmem *ocxlpmem)
+{
+ int rc;
+
+ free_minor(ocxlpmem);
+
+ if (ocxlpmem->ocxl_context) {
+ rc = ocxl_context_detach(ocxlpmem->ocxl_context);
+ if (rc == -EBUSY)
+ dev_warn(&ocxlpmem->dev, "Timeout detaching ocxl context\n");
+ else
+ ocxl_context_free(ocxlpmem->ocxl_context);
+ }
+
+ if (ocxlpmem->ocxl_afu)
+ ocxl_afu_put(ocxlpmem->ocxl_afu);
+
+ if (ocxlpmem->ocxl_fn)
+ ocxl_function_close(ocxlpmem->ocxl_fn);
+
+ pci_dev_put(ocxlpmem->pdev);
+
+ kfree(ocxlpmem);
+}
+
+/**
+ * free_ocxlpmem_dev() - Free an OpenCAPI persistent memory device
+ * @dev: The device struct
+ */
+static void free_ocxlpmem_dev(struct device *dev)
+{
+ struct ocxlpmem *ocxlpmem = container_of(dev, struct ocxlpmem, dev);
+
+ free_ocxlpmem(ocxlpmem);
+}
+
+/**
+ * ocxlpmem_register() - Register an OpenCAPI pmem device with the kernel
+ * @ocxlpmem: the device metadata
+ * Return: 0 on success, negative on failure
+ */
+static int ocxlpmem_register(struct ocxlpmem *ocxlpmem)
+{
+ int rc;
+ int minor = allocate_minor(ocxlpmem);
+
+ if (minor < 0)
+ return minor;
+
+ ocxlpmem->dev.release = free_ocxlpmem_dev;
+ rc = dev_set_name(&ocxlpmem->dev, "ocxlpmem%d", minor);
+ if (rc < 0)
+ return rc;
+
+ ocxlpmem->dev.devt = MKDEV(MAJOR(ocxlpmem_dev), minor);
+ ocxlpmem->dev.class = ocxlpmem_class;
+ ocxlpmem->dev.parent = &ocxlpmem->pdev->dev;
+
+ return device_register(&ocxlpmem->dev);
+}
+
+/**
+ * ocxlpmem_remove() - Free an OpenCAPI persistent memory device
+ * @pdev: the PCI device information struct
+ */
+static void remove(struct pci_dev *pdev)
+{
+ if (PCI_FUNC(pdev->devfn) == 0) {
+ struct ocxl_fn *func0 = pci_get_drvdata(pdev);
+
+ if (func0)
+ ocxl_function_close(func0);
+ } else {
+ struct ocxlpmem *ocxlpmem = pci_get_drvdata(pdev);
+
+ if (!ocxlpmem)
+ return;
+
+ if (ocxlpmem->nvdimm_bus)
+ nvdimm_bus_unregister(ocxlpmem->nvdimm_bus);
+
+ device_unregister(&ocxlpmem->dev);
+ }
+}
+
+/**
+ * probe_function0() - Set up function 0 for an OpenCAPI persistent memory device
+ * This is important as it enables templates higher than 0 across all other
+ * functions, which in turn enables higher bandwidth accesses
+ * @pdev: the PCI device information struct
+ * Return: 0 on success, negative on failure
+ */
+static int probe_function0(struct pci_dev *pdev)
+{
+ struct ocxl_fn *fn;
+
+ fn = ocxl_function_open(pdev);
+ if (IS_ERR(fn)) {
+ dev_err(&pdev->dev, "failed to open OCXL function\n");
+ return PTR_ERR(fn);
+ }
+
+ pci_set_drvdata(pdev, fn);
+
+ return 0;
+}
+
+/**
+ * probe() - Init an OpenCAPI persistent memory device
+ * @pdev: the PCI device information struct
+ * @ent: The entry from ocxlpmem_pci_tbl
+ * Return: 0 on success, negative on failure
+ */
+static int probe(struct pci_dev *pdev, const struct pci_device_id *ent)
+{
+ struct ocxlpmem *ocxlpmem;
+ int rc;
+
+ if (PCI_FUNC(pdev->devfn) == 0)
+ return probe_function0(pdev);
+ else if (PCI_FUNC(pdev->devfn) != 1)
+ return 0;
+
+ ocxlpmem = kzalloc(sizeof(*ocxlpmem), GFP_KERNEL);
+ if (!ocxlpmem) {
+ rc = -ENOMEM;
+ goto err;
+ }
+
+ ocxlpmem->pdev = pci_dev_get(pdev);
+
+ pci_set_drvdata(pdev, ocxlpmem);
+
+ ocxlpmem->ocxl_fn = ocxl_function_open(pdev);
+ if (IS_ERR(ocxlpmem->ocxl_fn)) {
+ rc = PTR_ERR(ocxlpmem->ocxl_fn);
+ dev_err(&pdev->dev, "failed to open OCXL function\n");
+ goto err_unregistered;
+ }
+
+ ocxlpmem->ocxl_afu = ocxl_function_fetch_afu(ocxlpmem->ocxl_fn, 0);
+ if (!ocxlpmem->ocxl_afu) {
+ rc = -ENXIO;
+ dev_err(&pdev->dev, "Could not get OCXL AFU from function\n");
+ goto err_unregistered;
+ }
+
+ ocxl_afu_get(ocxlpmem->ocxl_afu);
+
+ // Resources allocated below here are cleaned up in the release handler
+
+ rc = ocxlpmem_register(ocxlpmem);
+ if (rc) {
+ dev_err(&pdev->dev,
+ "Could not register OpenCAPI persistent memory device with the kernel\n");
+ goto err;
+ }
+
+ rc = ocxl_context_alloc(&ocxlpmem->ocxl_context, ocxlpmem->ocxl_afu,
+ NULL);
+ if (rc) {
+ dev_err(&pdev->dev, "Could not allocate OCXL context\n");
+ goto err;
+ }
+
+ rc = ocxl_context_attach(ocxlpmem->ocxl_context, 0, NULL);
+ if (rc) {
+ dev_err(&pdev->dev, "Could not attach ocxl context\n");
+ goto err;
+ }
+
+ rc = register_lpc_mem(ocxlpmem);
+ if (rc) {
+ dev_err(&pdev->dev,
+ "Could not register OpenCAPI persistent memory with libnvdimm\n");
+ goto err;
+ }
+
+ return 0;
+
+err_unregistered:
+ if (!IS_ERR(ocxlpmem->ocxl_fn))
+ ocxl_function_close(ocxlpmem->ocxl_fn);
+ pci_dev_put(ocxlpmem->pdev);
+ kfree(ocxlpmem);
+ pci_set_drvdata(pdev, NULL);
+
+err:
+ /*
+ * Further cleanup is done in the release handler via free_ocxlpmem()
+ * This allows us to keep the character device live to handle IOCTLs to
+ * investigate issues if the card has an error
+ */
+
+ dev_err(&pdev->dev,
+ "Error detected, will not register OpenCAPI persistent memory\n");
+ return 0;
+}
+
+static struct pci_driver pci_driver = {
+ .name = "ocxlpmem",
+ .id_table = pci_tbl,
+ .probe = probe,
+ .remove = remove,
+ .shutdown = remove,
+};
+
+static int __init ocxlpmem_init(void)
+{
+ int rc;
+
+ mutex_init(&minors_idr_lock);
+ idr_init(&minors_idr);
+
+ rc = pci_register_driver(&pci_driver);
+ if (rc)
+ return rc;
+
+ return 0;
+}
+
+static void ocxlpmem_exit(void)
+{
+ pci_unregister_driver(&pci_driver);
+}
+
+module_init(ocxlpmem_init);
+module_exit(ocxlpmem_exit);
+
+MODULE_DESCRIPTION("OpenCAPI Persistent Memory");
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Alastair D'Silva <alastair@d-silva.org>");
diff --git a/drivers/nvdimm/ocxl/ocxlpmem.h b/drivers/nvdimm/ocxl/ocxlpmem.h
new file mode 100644
index 000000000000..03fe7a264281
--- /dev/null
+++ b/drivers/nvdimm/ocxl/ocxlpmem.h
@@ -0,0 +1,23 @@
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright 2020 IBM Corp.
+
+#include <linux/pci.h>
+#include <misc/ocxl.h>
+#include <linux/libnvdimm.h>
+#include <linux/mm.h>
+
+#define LABEL_AREA_SIZE BIT_ULL(PA_SECTION_SHIFT)
+
+struct ocxlpmem {
+ struct device dev;
+ struct pci_dev *pdev;
+ struct ocxl_fn *ocxl_fn;
+ struct nd_interleave_set nd_set;
+ struct nvdimm_bus_descriptor bus_desc;
+ struct nvdimm_bus *nvdimm_bus;
+ struct ocxl_afu *ocxl_afu;
+ struct ocxl_context *ocxl_context;
+ void *metadata_addr;
+ struct resource pmem_res;
+ struct nd_region *nd_region;
+};
--
2.24.1
^ permalink raw reply related
* [PATCH v4 13/25] nvdimm/ocxl: Read the capability registers & wait for device ready
From: Alastair D'Silva @ 2020-03-27 7:11 UTC (permalink / raw)
To: alastair
Cc: Madhavan Srinivasan, Alexey Kardashevskiy, Masahiro Yamada,
Oliver O'Halloran, Mauro Carvalho Chehab, Ira Weiny,
Thomas Gleixner, Rob Herring, Dave Jiang, linux-nvdimm,
Aneesh Kumar K . V, Krzysztof Kozlowski, Anju T Sudhakar,
Mahesh Salgaonkar, Andrew Donnellan, Arnd Bergmann, Greg Kurz,
Nicholas Piggin, Cédric Le Goater, Dan Williams,
Hari Bathini, linux-mm, Greg Kroah-Hartman, linux-kernel,
Vishal Verma, Frederic Barrat, Paul Mackerras, Andrew Morton,
linuxppc-dev, David S. Miller
In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org>
This patch reads timeouts & firmware version from the controller, and
uses those timeouts to wait for the controller to report that it is ready
before handing the memory over to libnvdimm.
Signed-off-by: Alastair D'Silva <alastair@d-silva.org>
---
drivers/nvdimm/ocxl/Makefile | 2 +-
drivers/nvdimm/ocxl/main.c | 85 +++++++++++++++++++++++++
drivers/nvdimm/ocxl/ocxlpmem.h | 29 +++++++++
drivers/nvdimm/ocxl/ocxlpmem_internal.c | 19 ++++++
4 files changed, 134 insertions(+), 1 deletion(-)
create mode 100644 drivers/nvdimm/ocxl/ocxlpmem_internal.c
diff --git a/drivers/nvdimm/ocxl/Makefile b/drivers/nvdimm/ocxl/Makefile
index e0e8ade1987a..bab97082e062 100644
--- a/drivers/nvdimm/ocxl/Makefile
+++ b/drivers/nvdimm/ocxl/Makefile
@@ -4,4 +4,4 @@ ccflags-$(CONFIG_PPC_WERROR) += -Werror
obj-$(CONFIG_OCXL_PMEM) += ocxlpmem.o
-ocxlpmem-y := main.o
\ No newline at end of file
+ocxlpmem-y := main.o ocxlpmem_internal.o
diff --git a/drivers/nvdimm/ocxl/main.c b/drivers/nvdimm/ocxl/main.c
index c0066fedf9cc..be76acd33d74 100644
--- a/drivers/nvdimm/ocxl/main.c
+++ b/drivers/nvdimm/ocxl/main.c
@@ -8,6 +8,7 @@
#include <linux/module.h>
#include <misc/ocxl.h>
+#include <linux/delay.h>
#include <linux/ndctl.h>
#include <linux/mm_types.h>
#include <linux/memory_hotplug.h>
@@ -327,6 +328,50 @@ static void remove(struct pci_dev *pdev)
}
}
+/**
+ * read_device_metadata() - Retrieve config information from the AFU and save it for future use
+ * @ocxlpmem: the device metadata
+ * Return: 0 on success, negative on failure
+ */
+static int read_device_metadata(struct ocxlpmem *ocxlpmem)
+{
+ u64 val;
+ int rc;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CCAP0,
+ OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ return rc;
+
+ ocxlpmem->scm_revision = val & 0xFFFF;
+ ocxlpmem->read_latency = (val >> 32) & 0xFFFF;
+ ocxlpmem->readiness_timeout = (val >> 48) & 0x0F;
+ ocxlpmem->memory_available_timeout = val >> 52;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CCAP1,
+ OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ return rc;
+
+ ocxlpmem->max_controller_dump_size = val & 0xFFFFFFFF;
+
+ // Extract firmware version text
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_FWVER,
+ OCXL_HOST_ENDIAN,
+ (u64 *)ocxlpmem->fw_version);
+ if (rc)
+ return rc;
+
+ ocxlpmem->fw_version[8] = '\0';
+
+ dev_info(&ocxlpmem->dev,
+ "Firmware version '%s' SCM revision %d:%d\n",
+ ocxlpmem->fw_version, ocxlpmem->scm_revision >> 4,
+ ocxlpmem->scm_revision & 0x0F);
+
+ return 0;
+}
+
/**
* probe_function0() - Set up function 0 for an OpenCAPI persistent memory device
* This is important as it enables templates higher than 0 across all other
@@ -359,6 +404,9 @@ static int probe(struct pci_dev *pdev, const struct pci_device_id *ent)
{
struct ocxlpmem *ocxlpmem;
int rc;
+ u64 chi;
+ u16 elapsed, timeout;
+ bool ready = false;
if (PCI_FUNC(pdev->devfn) == 0)
return probe_function0(pdev);
@@ -413,6 +461,43 @@ static int probe(struct pci_dev *pdev, const struct pci_device_id *ent)
goto err;
}
+ rc = read_device_metadata(ocxlpmem);
+ if (rc) {
+ dev_err(&pdev->dev, "Could not read metadata\n");
+ goto err;
+ }
+
+ elapsed = 0;
+ timeout = ocxlpmem->readiness_timeout +
+ ocxlpmem->memory_available_timeout;
+
+ while (true) {
+ rc = ocxlpmem_chi(ocxlpmem, &chi);
+ ready = (chi & (GLOBAL_MMIO_CHI_CRDY | GLOBAL_MMIO_CHI_MA)) ==
+ (GLOBAL_MMIO_CHI_CRDY | GLOBAL_MMIO_CHI_MA);
+
+ if (ready)
+ break;
+
+ if (elapsed++ > timeout) {
+ dev_err(&ocxlpmem->dev,
+ "OpenCAPI Persistent Memory ready timeout.\n");
+
+ if (!(chi & GLOBAL_MMIO_CHI_CRDY))
+ dev_err(&ocxlpmem->dev,
+ "controller is not ready.\n");
+
+ if (!(chi & GLOBAL_MMIO_CHI_MA))
+ dev_err(&ocxlpmem->dev,
+ "controller does not have memory available.\n");
+
+ rc = -ENXIO;
+ goto err;
+ }
+
+ msleep(1000);
+ }
+
rc = register_lpc_mem(ocxlpmem);
if (rc) {
dev_err(&pdev->dev,
diff --git a/drivers/nvdimm/ocxl/ocxlpmem.h b/drivers/nvdimm/ocxl/ocxlpmem.h
index 322387873b4b..3eadbe19f6d0 100644
--- a/drivers/nvdimm/ocxl/ocxlpmem.h
+++ b/drivers/nvdimm/ocxl/ocxlpmem.h
@@ -93,4 +93,33 @@ struct ocxlpmem {
void *metadata_addr;
struct resource pmem_res;
struct nd_region *nd_region;
+ char fw_version[8 + 1];
+
+ u32 max_controller_dump_size;
+ u16 scm_revision; // major/minor
+ u8 readiness_timeout; /* The worst case time (in seconds) that the host
+ * shall wait for the controller to become
+ * operational following a reset (CHI.CRDY).
+ */
+ u8 memory_available_timeout; /* The worst case time (in seconds) that
+ * the host shall wait for memory to
+ * become available following a reset
+ * (CHI.MA).
+ */
+
+ u16 read_latency; /* The nominal measure of latency (in nanoseconds)
+ * associated with an unassisted read of a memory
+ * block.
+ * This represents the capability of the raw media
+ * technology without assistance
+ */
};
+
+/**
+ * ocxlpmem_chi() - Get the value of the CHI register
+ * @ocxlpmem: the device metadata
+ * @chi: returns the CHI value
+ *
+ * Returns 0 on success, negative on error
+ */
+int ocxlpmem_chi(const struct ocxlpmem *ocxlpmem, u64 *chi);
diff --git a/drivers/nvdimm/ocxl/ocxlpmem_internal.c b/drivers/nvdimm/ocxl/ocxlpmem_internal.c
new file mode 100644
index 000000000000..5578169b7515
--- /dev/null
+++ b/drivers/nvdimm/ocxl/ocxlpmem_internal.c
@@ -0,0 +1,19 @@
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright 2020 IBM Corp.
+
+#include <misc/ocxl.h>
+#include <linux/delay.h>
+#include "ocxlpmem.h"
+
+int ocxlpmem_chi(const struct ocxlpmem *ocxlpmem, u64 *chi)
+{
+ u64 val;
+ int rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CHI,
+ OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ return rc;
+
+ *chi = val;
+
+ return 0;
+}
--
2.24.1
^ permalink raw reply related
* [PATCH v4 01/25] powerpc/powernv: Add OPAL calls for LPC memory alloc/release
From: Alastair D'Silva @ 2020-03-27 7:11 UTC (permalink / raw)
To: alastair
Cc: Madhavan Srinivasan, Alexey Kardashevskiy, Masahiro Yamada,
Oliver O'Halloran, Mauro Carvalho Chehab, Ira Weiny,
Thomas Gleixner, Rob Herring, Dave Jiang, linux-nvdimm,
Aneesh Kumar K . V, Krzysztof Kozlowski, Anju T Sudhakar,
Mahesh Salgaonkar, Andrew Donnellan, Arnd Bergmann, Greg Kurz,
Nicholas Piggin, Cédric Le Goater, Dan Williams,
Hari Bathini, linux-mm, Greg Kroah-Hartman, linux-kernel,
Vishal Verma, Frederic Barrat, Paul Mackerras, Andrew Morton,
linuxppc-dev, David S. Miller
In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org>
Add OPAL calls for LPC memory alloc/release
Signed-off-by: Alastair D'Silva <alastair@d-silva.org>
Acked-by: Andrew Donnellan <ajd@linux.ibm.com>
Acked-by: Frederic Barrat <fbarrat@linux.ibm.com>
---
arch/powerpc/include/asm/opal-api.h | 2 ++
arch/powerpc/include/asm/opal.h | 2 ++
arch/powerpc/platforms/powernv/opal-call.c | 2 ++
3 files changed, 6 insertions(+)
diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h
index c1f25a760eb1..9298e603001b 100644
--- a/arch/powerpc/include/asm/opal-api.h
+++ b/arch/powerpc/include/asm/opal-api.h
@@ -208,6 +208,8 @@
#define OPAL_HANDLE_HMI2 166
#define OPAL_NX_COPROC_INIT 167
#define OPAL_XIVE_GET_VP_STATE 170
+#define OPAL_NPU_MEM_ALLOC 171
+#define OPAL_NPU_MEM_RELEASE 172
#define OPAL_MPIPL_UPDATE 173
#define OPAL_MPIPL_REGISTER_TAG 174
#define OPAL_MPIPL_QUERY_TAG 175
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 9986ac34b8e2..301fea46c7ca 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -39,6 +39,8 @@ int64_t opal_npu_spa_clear_cache(uint64_t phb_id, uint32_t bdfn,
uint64_t PE_handle);
int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t bdfn, long cap,
uint64_t rate_phys, uint32_t size);
+int64_t opal_npu_mem_alloc(u64 phb_id, u32 bdfn, u64 size, __be64 *bar);
+int64_t opal_npu_mem_release(u64 phb_id, u32 bdfn);
int64_t opal_console_write(int64_t term_number, __be64 *length,
const uint8_t *buffer);
diff --git a/arch/powerpc/platforms/powernv/opal-call.c b/arch/powerpc/platforms/powernv/opal-call.c
index 5cd0f52d258f..f26e58b72c04 100644
--- a/arch/powerpc/platforms/powernv/opal-call.c
+++ b/arch/powerpc/platforms/powernv/opal-call.c
@@ -287,6 +287,8 @@ OPAL_CALL(opal_pci_set_pbcq_tunnel_bar, OPAL_PCI_SET_PBCQ_TUNNEL_BAR);
OPAL_CALL(opal_sensor_read_u64, OPAL_SENSOR_READ_U64);
OPAL_CALL(opal_sensor_group_enable, OPAL_SENSOR_GROUP_ENABLE);
OPAL_CALL(opal_nx_coproc_init, OPAL_NX_COPROC_INIT);
+OPAL_CALL(opal_npu_mem_alloc, OPAL_NPU_MEM_ALLOC);
+OPAL_CALL(opal_npu_mem_release, OPAL_NPU_MEM_RELEASE);
OPAL_CALL(opal_mpipl_update, OPAL_MPIPL_UPDATE);
OPAL_CALL(opal_mpipl_register_tag, OPAL_MPIPL_REGISTER_TAG);
OPAL_CALL(opal_mpipl_query_tag, OPAL_MPIPL_QUERY_TAG);
--
2.24.1
^ permalink raw reply related
* [PATCH v4 08/25] ocxl: Emit a log message showing how much LPC memory was detected
From: Alastair D'Silva @ 2020-03-27 7:11 UTC (permalink / raw)
To: alastair
Cc: Madhavan Srinivasan, Alexey Kardashevskiy, Masahiro Yamada,
Oliver O'Halloran, Mauro Carvalho Chehab, Ira Weiny,
Thomas Gleixner, Rob Herring, Dave Jiang, linux-nvdimm,
Aneesh Kumar K . V, Krzysztof Kozlowski, Anju T Sudhakar,
Mahesh Salgaonkar, Andrew Donnellan, Arnd Bergmann, Greg Kurz,
Nicholas Piggin, Cédric Le Goater, Dan Williams,
Hari Bathini, linux-mm, Greg Kroah-Hartman, linux-kernel,
Vishal Verma, Frederic Barrat, Paul Mackerras, Andrew Morton,
linuxppc-dev, David S. Miller
In-Reply-To: <20200327071202.2159885-1-alastair@d-silva.org>
This patch emits a message showing how much LPC memory & special purpose
memory was detected on an OCXL device.
Signed-off-by: Alastair D'Silva <alastair@d-silva.org>
Acked-by: Frederic Barrat <fbarrat@linux.ibm.com>
Acked-by: Andrew Donnellan <ajd@linux.ibm.com>
---
drivers/misc/ocxl/config.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/misc/ocxl/config.c b/drivers/misc/ocxl/config.c
index a62e3d7db2bf..69cca341d446 100644
--- a/drivers/misc/ocxl/config.c
+++ b/drivers/misc/ocxl/config.c
@@ -568,6 +568,10 @@ static int read_afu_lpc_memory_info(struct pci_dev *dev,
afu->special_purpose_mem_size =
total_mem_size - lpc_mem_size;
}
+
+ dev_info(&dev->dev, "Probed LPC memory of %#llx bytes and special purpose memory of %#llx bytes\n",
+ afu->lpc_mem_size, afu->special_purpose_mem_size);
+
return 0;
}
--
2.24.1
^ permalink raw reply related
* Re: [PATCH v3 0/5] mm: Enable CONFIG_NODES_SPAN_OTHER_NODES by default for NUMA
From: Baoquan He @ 2020-03-29 0:19 UTC (permalink / raw)
To: Hoan Tran
Cc: mmorana, Michal Hocko, Catalin Marinas, Heiko Carstens,
open list:MEMORY MANAGEMENT, Paul Mackerras, H. Peter Anvin,
sparclinux, Alexander Duyck, linux-s390, x86, Mike Rapoport,
Christian Borntraeger, Ingo Molnar, Vlastimil Babka,
Pavel Tatashin, lho, Vasily Gorbik, Will Deacon, Borislav Petkov,
Thomas Gleixner, linux-arm-kernel, Oscar Salvador, linux-kernel,
Andrew Morton, linuxppc-dev, David S. Miller
In-Reply-To: <1585420282-25630-1-git-send-email-Hoan@os.amperecomputing.com>
On 03/28/20 at 11:31am, Hoan Tran wrote:
> In NUMA layout which nodes have memory ranges that span across other nodes,
> the mm driver can detect the memory node id incorrectly.
>
> For example, with layout below
> Node 0 address: 0000 xxxx 0000 xxxx
> Node 1 address: xxxx 1111 xxxx 1111
Sorry, I read this example several times, but still don't get what it
means. Can it be given with real hex number address as an exmaple? I
mean just using the memory layout you have seen from some systems. The
change looks interesting though.
>
> Note:
> - Memory from low to high
> - 0/1: Node id
> - x: Invalid memory of a node
>
> When mm probes the memory map, without CONFIG_NODES_SPAN_OTHER_NODES
> config, mm only checks the memory validity but not the node id.
> Because of that, Node 1 also detects the memory from node 0 as below
> when it scans from the start address to the end address of node 1.
>
> Node 0 address: 0000 xxxx xxxx xxxx
> Node 1 address: xxxx 1111 1111 1111
>
> This layout could occur on any architecture. Most of them enables
> this config by default with CONFIG_NUMA. This patch, by default, enables
> CONFIG_NODES_SPAN_OTHER_NODES or uses early_pfn_in_nid() for NUMA.
>
> v3:
> * Revise the patch description
>
> V2:
> * Revise the patch description
>
> Hoan Tran (5):
> mm: Enable CONFIG_NODES_SPAN_OTHER_NODES by default for NUMA
> powerpc: Kconfig: Remove CONFIG_NODES_SPAN_OTHER_NODES
> x86: Kconfig: Remove CONFIG_NODES_SPAN_OTHER_NODES
> sparc: Kconfig: Remove CONFIG_NODES_SPAN_OTHER_NODES
> s390: Kconfig: Remove CONFIG_NODES_SPAN_OTHER_NODES
>
> arch/powerpc/Kconfig | 9 ---------
> arch/s390/Kconfig | 8 --------
> arch/sparc/Kconfig | 9 ---------
> arch/x86/Kconfig | 9 ---------
> mm/page_alloc.c | 2 +-
> 5 files changed, 1 insertion(+), 36 deletions(-)
>
> --
> 1.8.3.1
>
>
^ permalink raw reply
* [RFC PATCH v1 40/50] arch/*/include/asm/stackprotector.h: Use get_random_canary() consistently
From: George Spelvin @ 2019-12-10 5:35 UTC (permalink / raw)
To: linux-kernel, lkml
Cc: Rich Felker, linux-sh, Max Filippov, Paul Mackerras,
H. Peter Anvin, Will Deacon, Yoshinori Sato, x86, Russell King,
Ingo Molnar, Catalin Marinas, James Hogan, linux-xtensa,
Borislav Petkov, Thomas Gleixner, linux-arm-kernel, Chris Zankel,
Paul Burton, linux-mips, Ralf Baechle, linuxppc-dev
... in boot_init_stack_canary().
This is the archetypical example of where the extra security of
get_random_bytes() is wasted. The canary is only important as
long as it's stored in __stack_chk_guard.
It's also a great example of code that has been copied around
a lot and not updated.
Remove the XOR with LINUX_VERSION_CODE as it's pointless; the inclusion
of utsname() in init_std_data in the random seeding obviates it.
The XOR with the TSC on x86 and mtfb() on powerPC were left in,
as I haven't proved them redundant yet. For those, we call
get_random_long(), xor, and mask manually.
FUNCTIONAL CHANGE: mips and xtensa were changed from 64-bit
get_random_long() to 56-bit get_random_canary() to match the
others, in accordance with the logic in CANARY_MASK.
(We could do 1 bit better and zero *one* of the two high bytes.)
Signed-off-by: George Spelvin <lkml@sdf.org>
Cc: Russell King <linux@armlinux.org.uk>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paulburton@kernel.org>
Cc: James Hogan <jhogan@kernel.org>
Cc: linux-mips@vger.kernel.org
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: linuxppc-dev@lists.ozlabs.org
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Rich Felker <dalias@libc.org>
Cc: linux-sh@vger.kernel.org
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org
Cc: Chris Zankel <chris@zankel.net>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Cc: linux-xtensa@linux-xtensa.org
---
arch/arm/include/asm/stackprotector.h | 9 +++------
arch/arm64/include/asm/stackprotector.h | 8 ++------
arch/mips/include/asm/stackprotector.h | 7 ++-----
arch/powerpc/include/asm/stackprotector.h | 6 ++----
arch/sh/include/asm/stackprotector.h | 8 ++------
arch/x86/include/asm/stackprotector.h | 4 ++--
arch/xtensa/include/asm/stackprotector.h | 7 ++-----
7 files changed, 15 insertions(+), 34 deletions(-)
diff --git a/arch/arm/include/asm/stackprotector.h b/arch/arm/include/asm/stackprotector.h
index 72a20c3a0a90b..88c66fec1b5f4 100644
--- a/arch/arm/include/asm/stackprotector.h
+++ b/arch/arm/include/asm/stackprotector.h
@@ -30,17 +30,14 @@ extern unsigned long __stack_chk_guard;
*/
static __always_inline void boot_init_stack_canary(void)
{
- unsigned long canary;
-
/* Try to get a semi random initial value. */
- get_random_bytes(&canary, sizeof(canary));
- canary ^= LINUX_VERSION_CODE;
+ unsigned long canary = get_random_canary();
current->stack_canary = canary;
#ifndef CONFIG_STACKPROTECTOR_PER_TASK
- __stack_chk_guard = current->stack_canary;
+ __stack_chk_guard = canary;
#else
- current_thread_info()->stack_canary = current->stack_canary;
+ current_thread_info()->stack_canary = canary;
#endif
}
diff --git a/arch/arm64/include/asm/stackprotector.h b/arch/arm64/include/asm/stackprotector.h
index 5884a2b028277..705f60b9df85e 100644
--- a/arch/arm64/include/asm/stackprotector.h
+++ b/arch/arm64/include/asm/stackprotector.h
@@ -26,16 +26,12 @@ extern unsigned long __stack_chk_guard;
*/
static __always_inline void boot_init_stack_canary(void)
{
- unsigned long canary;
-
/* Try to get a semi random initial value. */
- get_random_bytes(&canary, sizeof(canary));
- canary ^= LINUX_VERSION_CODE;
- canary &= CANARY_MASK;
+ unsigned long canary = get_random_canary();
current->stack_canary = canary;
if (!IS_ENABLED(CONFIG_STACKPROTECTOR_PER_TASK))
- __stack_chk_guard = current->stack_canary;
+ __stack_chk_guard = canary;
}
#endif /* _ASM_STACKPROTECTOR_H */
diff --git a/arch/mips/include/asm/stackprotector.h b/arch/mips/include/asm/stackprotector.h
index 68d4be9e12547..6d1e4652152bc 100644
--- a/arch/mips/include/asm/stackprotector.h
+++ b/arch/mips/include/asm/stackprotector.h
@@ -28,14 +28,11 @@ extern unsigned long __stack_chk_guard;
*/
static __always_inline void boot_init_stack_canary(void)
{
- unsigned long canary;
-
/* Try to get a semi random initial value. */
- get_random_bytes(&canary, sizeof(canary));
- canary ^= LINUX_VERSION_CODE;
+ unsigned long canary = get_random_canary();
current->stack_canary = canary;
- __stack_chk_guard = current->stack_canary;
+ __stack_chk_guard = canary;
}
#endif /* _ASM_STACKPROTECTOR_H */
diff --git a/arch/powerpc/include/asm/stackprotector.h b/arch/powerpc/include/asm/stackprotector.h
index 1c8460e235838..76577b72ef736 100644
--- a/arch/powerpc/include/asm/stackprotector.h
+++ b/arch/powerpc/include/asm/stackprotector.h
@@ -21,12 +21,10 @@
*/
static __always_inline void boot_init_stack_canary(void)
{
- unsigned long canary;
-
/* Try to get a semi random initial value. */
- canary = get_random_canary();
+ unsigned long canary = get_random_long();
+
canary ^= mftb();
- canary ^= LINUX_VERSION_CODE;
canary &= CANARY_MASK;
current->stack_canary = canary;
diff --git a/arch/sh/include/asm/stackprotector.h b/arch/sh/include/asm/stackprotector.h
index 35616841d0a1c..a9ef619c8a0ec 100644
--- a/arch/sh/include/asm/stackprotector.h
+++ b/arch/sh/include/asm/stackprotector.h
@@ -15,15 +15,11 @@ extern unsigned long __stack_chk_guard;
*/
static __always_inline void boot_init_stack_canary(void)
{
- unsigned long canary;
-
/* Try to get a semi random initial value. */
- get_random_bytes(&canary, sizeof(canary));
- canary ^= LINUX_VERSION_CODE;
- canary &= CANARY_MASK;
+ unsigned long canary = get_random_canary();
current->stack_canary = canary;
- __stack_chk_guard = current->stack_canary;
+ __stack_chk_guard = canary;
}
#endif /* __ASM_SH_STACKPROTECTOR_H */
diff --git a/arch/x86/include/asm/stackprotector.h b/arch/x86/include/asm/stackprotector.h
index 91e29b6a86a5e..af74fd3130cf4 100644
--- a/arch/x86/include/asm/stackprotector.h
+++ b/arch/x86/include/asm/stackprotector.h
@@ -72,9 +72,9 @@ static __always_inline void boot_init_stack_canary(void)
* there it already has some randomness on most systems. Later
* on during the bootup the random pool has true entropy too.
*/
- get_random_bytes(&canary, sizeof(canary));
+ canary = get_random_u64();
tsc = rdtsc();
- canary += tsc + (tsc << 32UL);
+ canary += tsc + (tsc << 32);
canary &= CANARY_MASK;
current->stack_canary = canary;
diff --git a/arch/xtensa/include/asm/stackprotector.h b/arch/xtensa/include/asm/stackprotector.h
index e368f94fd2af3..9807fd80e5a8e 100644
--- a/arch/xtensa/include/asm/stackprotector.h
+++ b/arch/xtensa/include/asm/stackprotector.h
@@ -27,14 +27,11 @@ extern unsigned long __stack_chk_guard;
*/
static __always_inline void boot_init_stack_canary(void)
{
- unsigned long canary;
-
/* Try to get a semi random initial value. */
- get_random_bytes(&canary, sizeof(canary));
- canary ^= LINUX_VERSION_CODE;
+ unsigned long canary = get_random_canary();
current->stack_canary = canary;
- __stack_chk_guard = current->stack_canary;
+ __stack_chk_guard = canary;
}
#endif /* _ASM_STACKPROTECTOR_H */
--
2.26.0
^ permalink raw reply related
* [PATCH v3 1/5] mm: Enable CONFIG_NODES_SPAN_OTHER_NODES by default for NUMA
From: Hoan Tran @ 2020-03-28 18:31 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Andrew Morton, Michal Hocko,
Vlastimil Babka, Oscar Salvador, Pavel Tatashin, Mike Rapoport,
Alexander Duyck, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
H. Peter Anvin, David S. Miller, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger
Cc: linux-s390, lho, mmorana, x86, linux-kernel,
open list:MEMORY MANAGEMENT, Hoan Tran, sparclinux, linuxppc-dev,
linux-arm-kernel
In-Reply-To: <1585420282-25630-1-git-send-email-Hoan@os.amperecomputing.com>
In NUMA layout which nodes have memory ranges that span across other nodes,
the mm driver can detect the memory node id incorrectly.
For example, with layout below
Node 0 address: 0000 xxxx 0000 xxxx
Node 1 address: xxxx 1111 xxxx 1111
Note:
- Memory from low to high
- 0/1: Node id
- x: Invalid memory of a node
When mm probes the memory map, without CONFIG_NODES_SPAN_OTHER_NODES
config, mm only checks the memory validity but not the node id.
Because of that, Node 1 also detects the memory from node 0 as below
when it scans from the start address to the end address of node 1.
Node 0 address: 0000 xxxx xxxx xxxx
Node 1 address: xxxx 1111 1111 1111
This layout could occur on any architecture. Most of them enables
this config by default with CONFIG_NUMA. This patch, by default, enables
CONFIG_NODES_SPAN_OTHER_NODES or uses early_pfn_in_nid() for NUMA.
Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
---
mm/page_alloc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index d047bf7..948c1c9 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -1467,7 +1467,7 @@ int __meminit early_pfn_to_nid(unsigned long pfn)
}
#endif
-#ifdef CONFIG_NODES_SPAN_OTHER_NODES
+#ifdef CONFIG_NUMA
/* Only safe to use early in boot when initialisation is single-threaded */
static inline bool __meminit early_pfn_in_nid(unsigned long pfn, int node)
{
--
1.8.3.1
^ permalink raw reply related
* [PATCH v3 5/5] s390: Kconfig: Remove CONFIG_NODES_SPAN_OTHER_NODES
From: Hoan Tran @ 2020-03-28 18:31 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Andrew Morton, Michal Hocko,
Vlastimil Babka, Oscar Salvador, Pavel Tatashin, Mike Rapoport,
Alexander Duyck, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
H. Peter Anvin, David S. Miller, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger
Cc: linux-s390, lho, mmorana, x86, linux-kernel,
open list:MEMORY MANAGEMENT, Hoan Tran, sparclinux, linuxppc-dev,
linux-arm-kernel
In-Reply-To: <1585420282-25630-1-git-send-email-Hoan@os.amperecomputing.com>
Remove CONFIG_NODES_SPAN_OTHER_NODES as it's enabled
by default with NUMA.
Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
---
arch/s390/Kconfig | 8 --------
1 file changed, 8 deletions(-)
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index bc88841..d86066e 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -449,14 +449,6 @@ config NR_CPUS
config HOTPLUG_CPU
def_bool y
-# Some NUMA nodes have memory ranges that span
-# other nodes. Even though a pfn is valid and
-# between a node's start and end pfns, it may not
-# reside on that node. See memmap_init_zone()
-# for details. <- They meant memory holes!
-config NODES_SPAN_OTHER_NODES
- def_bool NUMA
-
config NUMA
bool "NUMA support"
depends on SCHED_TOPOLOGY
--
1.8.3.1
^ permalink raw reply related
* [PATCH v3 4/5] sparc: Kconfig: Remove CONFIG_NODES_SPAN_OTHER_NODES
From: Hoan Tran @ 2020-03-28 18:31 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Andrew Morton, Michal Hocko,
Vlastimil Babka, Oscar Salvador, Pavel Tatashin, Mike Rapoport,
Alexander Duyck, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
H. Peter Anvin, David S. Miller, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger
Cc: linux-s390, lho, mmorana, x86, linux-kernel,
open list:MEMORY MANAGEMENT, Hoan Tran, sparclinux, linuxppc-dev,
linux-arm-kernel
In-Reply-To: <1585420282-25630-1-git-send-email-Hoan@os.amperecomputing.com>
Remove CONFIG_NODES_SPAN_OTHER_NODES as it's enabled
by default with NUMA.
Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
---
arch/sparc/Kconfig | 9 ---------
1 file changed, 9 deletions(-)
diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
index eb24cb1..6fc615e 100644
--- a/arch/sparc/Kconfig
+++ b/arch/sparc/Kconfig
@@ -292,15 +292,6 @@ config NODES_SHIFT
Specify the maximum number of NUMA Nodes available on the target
system. Increases memory reserved to accommodate various tables.
-# Some NUMA nodes have memory ranges that span
-# other nodes. Even though a pfn is valid and
-# between a node's start and end pfns, it may not
-# reside on that node. See memmap_init_zone()
-# for details.
-config NODES_SPAN_OTHER_NODES
- def_bool y
- depends on NEED_MULTIPLE_NODES
-
config ARCH_SPARSEMEM_ENABLE
def_bool y if SPARC64
select SPARSEMEM_VMEMMAP_ENABLE
--
1.8.3.1
^ permalink raw reply related
* [PATCH v3 3/5] x86: Kconfig: Remove CONFIG_NODES_SPAN_OTHER_NODES
From: Hoan Tran @ 2020-03-28 18:31 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Andrew Morton, Michal Hocko,
Vlastimil Babka, Oscar Salvador, Pavel Tatashin, Mike Rapoport,
Alexander Duyck, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
H. Peter Anvin, David S. Miller, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger
Cc: linux-s390, lho, mmorana, x86, linux-kernel,
open list:MEMORY MANAGEMENT, Hoan Tran, sparclinux, linuxppc-dev,
linux-arm-kernel
In-Reply-To: <1585420282-25630-1-git-send-email-Hoan@os.amperecomputing.com>
Remove CONFIG_NODES_SPAN_OTHER_NODES as it's enabled
by default. It is now enabled for x86(32 bit) configurations
and do not depend on X64_64_ACPI_NUMA config.
Because of that, on NUMA enabled system, early_pfn_in_nid()
function is called by memmap_init_zone() during boot-time.
It doesn't affect the performance at run-time.
Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
---
arch/x86/Kconfig | 9 ---------
1 file changed, 9 deletions(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 5e89499..a938738 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1581,15 +1581,6 @@ config X86_64_ACPI_NUMA
---help---
Enable ACPI SRAT based node topology detection.
-# Some NUMA nodes have memory ranges that span
-# other nodes. Even though a pfn is valid and
-# between a node's start and end pfns, it may not
-# reside on that node. See memmap_init_zone()
-# for details.
-config NODES_SPAN_OTHER_NODES
- def_bool y
- depends on X86_64_ACPI_NUMA
-
config NUMA_EMU
bool "NUMA emulation"
depends on NUMA
--
1.8.3.1
^ permalink raw reply related
* [PATCH v3 2/5] powerpc: Kconfig: Remove CONFIG_NODES_SPAN_OTHER_NODES
From: Hoan Tran @ 2020-03-28 18:31 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Andrew Morton, Michal Hocko,
Vlastimil Babka, Oscar Salvador, Pavel Tatashin, Mike Rapoport,
Alexander Duyck, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
H. Peter Anvin, David S. Miller, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger
Cc: linux-s390, lho, mmorana, x86, linux-kernel,
open list:MEMORY MANAGEMENT, Hoan Tran, sparclinux, linuxppc-dev,
linux-arm-kernel
In-Reply-To: <1585420282-25630-1-git-send-email-Hoan@os.amperecomputing.com>
Remove CONFIG_NODES_SPAN_OTHER_NODES as it's enabled by
default with NUMA.
Signed-off-by: Hoan Tran <Hoan@os.amperecomputing.com>
---
arch/powerpc/Kconfig | 9 ---------
1 file changed, 9 deletions(-)
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index e2a4121..4af2699 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -686,15 +686,6 @@ config ARCH_MEMORY_PROBE
def_bool y
depends on MEMORY_HOTPLUG
-# Some NUMA nodes have memory ranges that span
-# other nodes. Even though a pfn is valid and
-# between a node's start and end pfns, it may not
-# reside on that node. See memmap_init_zone()
-# for details.
-config NODES_SPAN_OTHER_NODES
- def_bool y
- depends on NEED_MULTIPLE_NODES
-
config STDBINUTILS
bool "Using standard binutils settings"
depends on 44x
--
1.8.3.1
^ permalink raw reply related
* [PATCH v3 0/5] mm: Enable CONFIG_NODES_SPAN_OTHER_NODES by default for NUMA
From: Hoan Tran @ 2020-03-28 18:31 UTC (permalink / raw)
To: Catalin Marinas, Will Deacon, Andrew Morton, Michal Hocko,
Vlastimil Babka, Oscar Salvador, Pavel Tatashin, Mike Rapoport,
Alexander Duyck, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
H. Peter Anvin, David S. Miller, Heiko Carstens, Vasily Gorbik,
Christian Borntraeger
Cc: linux-s390, lho, mmorana, x86, linux-kernel,
open list:MEMORY MANAGEMENT, Hoan Tran, sparclinux, linuxppc-dev,
linux-arm-kernel
In NUMA layout which nodes have memory ranges that span across other nodes,
the mm driver can detect the memory node id incorrectly.
For example, with layout below
Node 0 address: 0000 xxxx 0000 xxxx
Node 1 address: xxxx 1111 xxxx 1111
Note:
- Memory from low to high
- 0/1: Node id
- x: Invalid memory of a node
When mm probes the memory map, without CONFIG_NODES_SPAN_OTHER_NODES
config, mm only checks the memory validity but not the node id.
Because of that, Node 1 also detects the memory from node 0 as below
when it scans from the start address to the end address of node 1.
Node 0 address: 0000 xxxx xxxx xxxx
Node 1 address: xxxx 1111 1111 1111
This layout could occur on any architecture. Most of them enables
this config by default with CONFIG_NUMA. This patch, by default, enables
CONFIG_NODES_SPAN_OTHER_NODES or uses early_pfn_in_nid() for NUMA.
v3:
* Revise the patch description
V2:
* Revise the patch description
Hoan Tran (5):
mm: Enable CONFIG_NODES_SPAN_OTHER_NODES by default for NUMA
powerpc: Kconfig: Remove CONFIG_NODES_SPAN_OTHER_NODES
x86: Kconfig: Remove CONFIG_NODES_SPAN_OTHER_NODES
sparc: Kconfig: Remove CONFIG_NODES_SPAN_OTHER_NODES
s390: Kconfig: Remove CONFIG_NODES_SPAN_OTHER_NODES
arch/powerpc/Kconfig | 9 ---------
arch/s390/Kconfig | 8 --------
arch/sparc/Kconfig | 9 ---------
arch/x86/Kconfig | 9 ---------
mm/page_alloc.c | 2 +-
5 files changed, 1 insertion(+), 36 deletions(-)
--
1.8.3.1
^ permalink raw reply
* Re: [PATCH 1/9] powerpc/ps3: Remove duplicate error messages
From: Markus Elfring @ 2020-03-28 17:09 UTC (permalink / raw)
To: Geoff Levand, Michael Ellerman, linuxppc-dev
Cc: Geert Uytterhoeven, Dan Carpenter, Emmanuel Nicolet
In-Reply-To: <1bc5a16a22c487c478a204ebb7b80a22d2ad9cd0.1585340156.git.geoff@infradead.org>
> Remove duplicate memory allocation failure error messages.
A single message can be omitted here.
https://lkml.org/lkml/2017/10/17/870
https://lore.kernel.org/patchwork/patch/842101/
https://lore.kernel.org/linuxppc-dev/e16c8b7d-de3a-6c96-9af4-dd0551cca805@users.sourceforge.net/
Will this detail be reflected in the final commit message?
Regards,
Markus
^ permalink raw reply
* Re: [PATCH 0/2] powerpc: Remove support for ppc405/440 Xilinx platforms
From: Christian Lamparter @ 2020-03-28 15:06 UTC (permalink / raw)
To: linuxppc-dev, Benjamin Herrenschmidt, Enrico Weigelt, Mark Brown,
Thomas Gleixner
Cc: Sasha Levin, Geert Uytterhoeven, Arnd Bergmann, Jonathan Corbet,
Masahiro Yamada, Nick Desaulniers, linux-kernel, Nicholas Piggin,
ewald_comhaire, DTML, Rob Herring, Paul Mackerras,
Greg Kroah-Hartman, Mauro Carvalho Chehab, Andy Shevchenko,
David S. Miller
In-Reply-To: <b5adcc7a-9d10-d75f-50e3-9c150a7b4989@c-s.fr>
(Sorry for the bounces, yes my smarthost mail setup breaks from time to time
and I had to cut the CC since it complained about the length.)
On Saturday, 28 March 2020 12:17:58 CET Christophe Leroy wrote:
>
> Le 27/03/2020 à 15:14, Andy Shevchenko a écrit :
>> On Fri, Mar 27, 2020 at 02:22:55PM +0100, Arnd Bergmann wrote:
>>> On Fri, Mar 27, 2020 at 2:15 PM Andy Shevchenko
>>> <andriy.shevchenko@linux.intel.com> wrote:
>>>> On Fri, Mar 27, 2020 at 03:10:26PM +0200, Andy Shevchenko wrote:
>>>>> On Fri, Mar 27, 2020 at 01:54:33PM +0100, Arnd Bergmann wrote:
>>>>>> On Fri, Mar 27, 2020 at 1:12 PM Michal Simek
>>>>>> <michal.simek@xilinx.com> wrote:
>>>>>> It does raise a follow-up question about ppc40x though: is it time to
>>>>>> retire all of it?
>>>>>
>>>>> Who knows?
>>>>>
>>>>> I have in possession nice WD My Book Live, based on this
>>>>> architecture, and I won't it gone from modern kernel support.
>>>>> OTOH I understand that amount of real users not too big.
Hm, can't add much to Xilinx ppc405/440 removal patch debate.
But as for the APM82181 with it's PPC464:
The last time I checked was with 5.6-rc4, it worked fine on the APM82181
(a MyBook Live) device. I've made a "build your own powerpc debian sid"
image thing that takes the latest kernel git and up-to-date packages
from debian ports (they still make powerpc packages!):
<https://github.com/chunkeey/mbl-debian> .
Though, this is small potatoes. There exists a much more popular project
by Ewald Comhaire (CCed): <https://github.com/ewaldc/My-Book-Live>
that serves the largest userbase:
<https://community.wd.com/c/wd-legacy-products>
I guess we should think about upstreaming the MyBook Live DTS. Problem here
is that we deviated a bit from the canyonlands.dts/bluestone.dts structure
by having a skeleton apm82181.dtsi with most of the SoC defintition and a
wd-mybooklive.dts for the device.
<https://github.com/chunkeey/mbl-debian/blob/master/dts/apm82181.dtsi>
<https://github.com/chunkeey/mbl-debian/blob/master/dts/wd-mybooklive.dts>
Cheers,
Christian
^ permalink raw reply
* [PATCH v2 4/4] powerpc/papr_scm: Implement support for DSM_PAPR_SCM_HEALTH
From: Vaibhav Jain @ 2020-03-28 12:10 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alastair D'Silva, Aneesh Kumar K . V, Jeff Moyer,
Oliver O'Halloran, Vishal Verma, Vaibhav Jain,
Michael Ellerman, Dan Williams
In-Reply-To: <20200328121023.160363-1-vaibhav@linux.ibm.com>
This patch implements support for papr_scm command
'DSM_PAPR_SCM_HEALTH' that returns a newly introduced 'struct
nd_papr_scm_dimm_health_stat' instance containing dimm health
information back to user space in response to ND_CMD_CALL. This
functionality is implemented in newly introduced papr_scm_get_health()
that queries the scm-dimm health information and then copies these bitmaps
to the package payload whose layout is defined by 'struct
papr_scm_ndctl_health'.
The patch also introduces a new member a new member 'struct
papr_scm_priv.health' thats an instance of 'struct
nd_papr_scm_dimm_health_stat' to cache the health information of a
scm-dimm. As a result functions drc_pmem_query_health() and
papr_flags_show() are updated to populate and use this new struct
instead of two be64 integers that we earlier used.
Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
arch/powerpc/include/uapi/asm/papr_scm_dsm.h | 42 +++++++
arch/powerpc/platforms/pseries/papr_scm.c | 111 ++++++++++++++++---
2 files changed, 136 insertions(+), 17 deletions(-)
diff --git a/arch/powerpc/include/uapi/asm/papr_scm_dsm.h b/arch/powerpc/include/uapi/asm/papr_scm_dsm.h
index 1f3127a27bde..0644060f8e3d 100644
--- a/arch/powerpc/include/uapi/asm/papr_scm_dsm.h
+++ b/arch/powerpc/include/uapi/asm/papr_scm_dsm.h
@@ -132,6 +132,7 @@ struct nd_papr_scm_cmd_pkg {
*/
enum dsm_papr_scm {
DSM_PAPR_SCM_MIN = 0x10000,
+ DSM_PAPR_SCM_HEALTH,
DSM_PAPR_SCM_MAX,
};
@@ -158,4 +159,45 @@ static void *papr_scm_pcmd_to_payload(struct nd_papr_scm_cmd_pkg *pcmd)
else
return (void *)((u8 *) pcmd + pcmd->payload_offset);
}
+
+/* Various scm-dimm health indicators */
+enum dsm_papr_scm_dimm_health {
+ DSM_PAPR_SCM_DIMM_HEALTHY,
+ DSM_PAPR_SCM_DIMM_UNHEALTHY,
+ DSM_PAPR_SCM_DIMM_CRITICAL,
+ DSM_PAPR_SCM_DIMM_FATAL,
+};
+
+/*
+ * Struct exchanged between kernel & ndctl in for PAPR_DSM_PAPR_SMART_HEALTH
+ * Various bitflags indicate the health status of the dimm.
+ *
+ * dimm_unarmed : Dimm not armed. So contents wont persist.
+ * dimm_bad_shutdown : Previous shutdown did not persist contents.
+ * dimm_bad_restore : Contents from previous shutdown werent restored.
+ * dimm_scrubbed : Contents of the dimm have been scrubbed.
+ * dimm_locked : Contents of the dimm cant be modified until CEC reboot
+ * dimm_encrypted : Contents of dimm are encrypted.
+ * dimm_health : Dimm health indicator.
+ */
+struct nd_papr_scm_dimm_health_stat_v1 {
+ bool dimm_unarmed;
+ bool dimm_bad_shutdown;
+ bool dimm_bad_restore;
+ bool dimm_scrubbed;
+ bool dimm_locked;
+ bool dimm_encrypted;
+ enum dsm_papr_scm_dimm_health dimm_health;
+};
+
+/*
+ * Typedef the current struct for dimm_health so that any application
+ * or kernel recompiled after introducing a new version automatically
+ * supports the new version.
+ */
+#define nd_papr_scm_dimm_health_stat nd_papr_scm_dimm_health_stat_v1
+
+/* Current version number for the dimm health struct */
+#define ND_PAPR_SCM_DIMM_HEALTH_VERSION 1
+
#endif /* _UAPI_ASM_POWERPC_PAPR_SCM_DSM_H_ */
diff --git a/arch/powerpc/platforms/pseries/papr_scm.c b/arch/powerpc/platforms/pseries/papr_scm.c
index 9a2614aaae88..16746d55f0b7 100644
--- a/arch/powerpc/platforms/pseries/papr_scm.c
+++ b/arch/powerpc/platforms/pseries/papr_scm.c
@@ -47,8 +47,7 @@ struct papr_scm_priv {
struct mutex dimm_mutex;
/* Health information for the dimm */
- __be64 health_bitmap;
- __be64 health_bitmap_valid;
+ struct nd_papr_scm_dimm_health_stat health;
};
static int drc_pmem_bind(struct papr_scm_priv *p)
@@ -158,6 +157,7 @@ static int drc_pmem_query_health(struct papr_scm_priv *p)
{
unsigned long ret[PLPAR_HCALL_BUFSIZE];
int64_t rc;
+ __be64 health;
rc = plpar_hcall(H_SCM_HEALTH, ret, p->drc_index);
if (rc != H_SUCCESS) {
@@ -172,13 +172,41 @@ static int drc_pmem_query_health(struct papr_scm_priv *p)
return rc;
/* Store the retrieved health information in dimm platform data */
- p->health_bitmap = ret[0];
- p->health_bitmap_valid = ret[1];
+ health = ret[0] & ret[1];
dev_dbg(&p->pdev->dev,
"Queried dimm health info. Bitmap:0x%016llx Mask:0x%016llx\n",
- be64_to_cpu(p->health_bitmap),
- be64_to_cpu(p->health_bitmap_valid));
+ be64_to_cpu(ret[0]),
+ be64_to_cpu(ret[1]));
+
+ memset(&p->health, 0, sizeof(p->health));
+
+ /* Check for various masks in bitmap and set the buffer */
+ if (health & PAPR_SCM_DIMM_UNARMED_MASK)
+ p->health.dimm_unarmed = true;
+
+ if (health & PAPR_SCM_DIMM_BAD_SHUTDOWN_MASK)
+ p->health.dimm_bad_shutdown = true;
+
+ if (health & PAPR_SCM_DIMM_BAD_RESTORE_MASK)
+ p->health.dimm_bad_restore = true;
+
+ if (health & PAPR_SCM_DIMM_ENCRYPTED)
+ p->health.dimm_encrypted = true;
+
+ if (health & PAPR_SCM_DIMM_SCRUBBED_AND_LOCKED) {
+ p->health.dimm_locked = true;
+ p->health.dimm_scrubbed = true;
+ }
+
+ if (health & PAPR_SCM_DIMM_HEALTH_UNHEALTHY)
+ p->health.dimm_health = DSM_PAPR_SCM_DIMM_UNHEALTHY;
+
+ if (health & PAPR_SCM_DIMM_HEALTH_CRITICAL)
+ p->health.dimm_health = DSM_PAPR_SCM_DIMM_CRITICAL;
+
+ if (health & PAPR_SCM_DIMM_HEALTH_FATAL)
+ p->health.dimm_health = DSM_PAPR_SCM_DIMM_FATAL;
mutex_unlock(&p->dimm_mutex);
return 0;
@@ -340,6 +368,51 @@ static int cmd_to_func(struct nvdimm *nvdimm, unsigned int cmd, void *buf,
return pkg->hdr.nd_command;
}
+/* Fetch the DIMM health info and populate it in provided package. */
+static int papr_scm_get_health(struct papr_scm_priv *p,
+ struct nd_papr_scm_cmd_pkg *pkg)
+{
+ int rc;
+ size_t copysize = sizeof(p->health);
+
+ rc = drc_pmem_query_health(p);
+ if (rc)
+ goto out;
+ /*
+ * If the requested payload version is greater than one we know
+ * about, return the payload version we know about and let
+ * caller/userspace handle.
+ */
+ if (pkg->payload_version > ND_PAPR_SCM_DIMM_HEALTH_VERSION)
+ pkg->payload_version = ND_PAPR_SCM_DIMM_HEALTH_VERSION;
+
+ if (pkg->hdr.nd_size_out < copysize) {
+ dev_dbg(&p->pdev->dev, "%s Payload not large enough\n",
+ __func__);
+ dev_dbg(&p->pdev->dev, "%s Expected %lu, available %u\n",
+ __func__, copysize, pkg->hdr.nd_size_out);
+ rc = -ENOSPC;
+ goto out;
+ }
+
+ dev_dbg(&p->pdev->dev, "%s Copying payload size=%lu version=0x%x\n",
+ __func__, copysize, pkg->payload_version);
+
+ /* Copy a subset of health struct based on copysize */
+ memcpy(papr_scm_pcmd_to_payload(pkg), &p->health, copysize);
+ pkg->hdr.nd_fw_size = copysize;
+
+out:
+ /*
+ * Put the error in out package and return success from function
+ * so that errors if any are propogated back to userspace.
+ */
+ pkg->cmd_status = rc;
+ dev_dbg(&p->pdev->dev, "%s completion code = %d\n", __func__, rc);
+
+ return 0;
+}
+
int papr_scm_ndctl(struct nvdimm_bus_descriptor *nd_desc, struct nvdimm *nvdimm,
unsigned int cmd, void *buf, unsigned int buf_len, int *cmd_rc)
{
@@ -385,6 +458,11 @@ int papr_scm_ndctl(struct nvdimm_bus_descriptor *nd_desc, struct nvdimm *nvdimm,
*cmd_rc = 0;
break;
+ case DSM_PAPR_SCM_HEALTH:
+ call_pkg = nd_to_papr_cmd_pkg(buf);
+ *cmd_rc = papr_scm_get_health(p, call_pkg);
+ break;
+
default:
dev_dbg(&p->pdev->dev, "Unknown command = %d\n", cmd_in);
*cmd_rc = -EINVAL;
@@ -419,7 +497,6 @@ static ssize_t papr_flags_show(struct device *dev,
{
struct nvdimm *dimm = to_nvdimm(dev);
struct papr_scm_priv *p = nvdimm_provider_data(dimm);
- __be64 health;
int rc;
rc = drc_pmem_query_health(p);
@@ -431,26 +508,26 @@ static ssize_t papr_flags_show(struct device *dev,
if (rc)
return rc;
- health = p->health_bitmap & p->health_bitmap_valid;
-
- /* Check for various masks in bitmap and set the buffer */
- if (health & PAPR_SCM_DIMM_UNARMED_MASK)
+ if (p->health.dimm_unarmed)
rc += sprintf(buf, "not_armed ");
- if (health & PAPR_SCM_DIMM_BAD_SHUTDOWN_MASK)
+ if (p->health.dimm_bad_shutdown)
rc += sprintf(buf + rc, "save_fail ");
- if (health & PAPR_SCM_DIMM_BAD_RESTORE_MASK)
+ if (p->health.dimm_bad_restore)
rc += sprintf(buf + rc, "restore_fail ");
- if (health & PAPR_SCM_DIMM_ENCRYPTED)
+ if (p->health.dimm_encrypted)
rc += sprintf(buf + rc, "encrypted ");
- if (health & PAPR_SCM_DIMM_SMART_EVENT_MASK)
+ if (p->health.dimm_health)
rc += sprintf(buf + rc, "smart_notify ");
- if (health & PAPR_SCM_DIMM_SCRUBBED_AND_LOCKED)
- rc += sprintf(buf + rc, "scrubbed locked ");
+ if (p->health.dimm_scrubbed)
+ rc += sprintf(buf + rc, "scrubbed ");
+
+ if (p->health.dimm_locked)
+ rc += sprintf(buf + rc, "locked ");
if (rc > 0)
rc += sprintf(buf + rc, "\n");
--
2.24.1
^ permalink raw reply related
* [PATCH v2 0/4] powerpc/papr_scm: Add support for reporting nvdimm health
From: Vaibhav Jain @ 2020-03-28 12:10 UTC (permalink / raw)
To: linuxppc-dev, linux-nvdimm
Cc: Alastair D'Silva, Aneesh Kumar K . V, Jeff Moyer,
Oliver O'Halloran, Vishal Verma, Vaibhav Jain,
Michael Ellerman, Dan Williams
The PAPR standard[1][3] provides mechanisms to query the health and
performance stats of an NVDIMM via various hcalls as described in Ref[2].
Until now these stats were never available nor exposed to the user-space
tools like 'ndctl'. This is partly due to PAPR platform not having support
for ACPI and NFIT. Hence 'ndctl' is unable to query and report the dimm
health status and a user had no way to determine the current health status
of a NDVIMM.
To overcome this limitation, this patch-set updates papr_scm kernel module
to query and fetch nvdimm health stats using hcalls described in Ref[2].
This health and performance stats are then exposed to userspace via syfs
and Dimm-Specific-Methods(DSM) issued by libndctl.
These changes coupled with proposed ndtcl changes located at Ref[4] should
provide a way for the user to retrieve NVDIMM health status using ndtcl.
Below is a sample output using proposed kernel + ndctl for PAPR NVDIMM in
a emulation environment:
# ndctl list -DH
[
{
"dev":"nmem0",
"health":{
"health_state":"fatal",
"shutdown_state":"dirty"
}
}
]
PAPR Dimm-Specific-Methods(DSM)
================================
As the name suggests DSMs are used by vendor specific code in libndctl to
execute certain operations or fetch certain information for NVDIMMS. DSMs
can be sent to papr_scm module via libndctl (userspace) and libnvdimm
(kernel) using the ND_CMD_CALL ioctl which can be handled in the dimm
control function papr_scm_ndctl(). For PAPR this patchset proposes a single
DSM to retrieve DIMM health, defined in the newly introduced uapi header
named 'papr_scm_dsm.h'. Support for more DSMs will be added in future.
Structure of the patch-set
==========================
The patchset starts with implementing support for fetching nvdimm health
information from PHYP and partially exposing it to user-space via nvdimm
flags.
Second & Third patches deal with implementing support for servicing DSM
commands papr_scm.
Finally the Fourth patch implements support for servicing DSM
'DSM_PAPR_SCM_HEALTH' that returns the nvdimm health information to
libndctl.
Change-log
==========
v2..v1:
* Restructured the patch-set based on review comments on V1 patch-set to
simplify the patch review. Multiple small patches have been combined into
single patches to reduce cross referencing that was needed in earlier
patch-set. Hence most of the patches in this patch-set as now new. [Aneesh]
* Removed the initial work done for fetch nvdimm performance statistics.
These changes will be re-proposed in a separate patch-set. [Aneesh]
* Simplified handling of versioning of 'struct
nd_papr_scm_dimm_health_stat_v1' as only one version of the structure is
currently in existence.
References:
[1]: "Power Architecture Platform Reference"
https://en.wikipedia.org/wiki/Power_Architecture_Platform_Reference
[2]: commit 58b278f568f0
("powerpc: Provide initial documentation for PAPR hcalls")
[3]: "Linux on Power Architecture Platform Reference"
https://members.openpowerfoundation.org/document/dl/469
[4]: https://patchwork.kernel.org/project/linux-nvdimm/list/?series=244625
Vaibhav Jain (4):
powerpc/papr_scm: Fetch nvdimm health information from PHYP
UAPI: ndctl: Introduce NVDIMM_FAMILY_PAPR_SCM as a new NVDIMM DSM
family
powerpc/papr_scm,uapi: Add support for handling PAPR DSM commands
powerpc/papr_scm: Implement support for DSM_PAPR_SCM_HEALTH
arch/powerpc/include/asm/papr_scm.h | 49 ++++
arch/powerpc/include/uapi/asm/papr_scm_dsm.h | 203 ++++++++++++++
arch/powerpc/platforms/pseries/papr_scm.c | 269 ++++++++++++++++++-
include/uapi/linux/ndctl.h | 1 +
4 files changed, 513 insertions(+), 9 deletions(-)
create mode 100644 arch/powerpc/include/asm/papr_scm.h
create mode 100644 arch/powerpc/include/uapi/asm/papr_scm_dsm.h
--
2.24.1
^ permalink raw reply
* [PATCH v2 2/4] UAPI: ndctl: Introduce NVDIMM_FAMILY_PAPR_SCM as a new NVDIMM DSM family
From: Vaibhav Jain @ 2020-03-28 12:10 UTC (permalink / raw)
To: linuxppc-dev, linux-nvdimm
Cc: Alastair D'Silva, Aneesh Kumar K . V, Jeff Moyer,
Oliver O'Halloran, Vishal Verma, Vaibhav Jain,
Michael Ellerman, Dan Williams
In-Reply-To: <20200328121023.160363-1-vaibhav@linux.ibm.com>
Add PAPR-scm family of DSM command-set to the white list of NVDIMM
command sets.
Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
include/uapi/linux/ndctl.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/linux/ndctl.h b/include/uapi/linux/ndctl.h
index de5d90212409..99fb60600ef8 100644
--- a/include/uapi/linux/ndctl.h
+++ b/include/uapi/linux/ndctl.h
@@ -244,6 +244,7 @@ struct nd_cmd_pkg {
#define NVDIMM_FAMILY_HPE2 2
#define NVDIMM_FAMILY_MSFT 3
#define NVDIMM_FAMILY_HYPERV 4
+#define NVDIMM_FAMILY_PAPR_SCM 5
#define ND_IOCTL_CALL _IOWR(ND_IOCTL, ND_CMD_CALL,\
struct nd_cmd_pkg)
--
2.24.1
^ permalink raw reply related
* [PATCH v2 1/4] powerpc/papr_scm: Fetch nvdimm health information from PHYP
From: Vaibhav Jain @ 2020-03-28 12:10 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alastair D'Silva, Aneesh Kumar K . V, Jeff Moyer,
Oliver O'Halloran, Vishal Verma, Vaibhav Jain,
Michael Ellerman, Dan Williams
In-Reply-To: <20200328121023.160363-1-vaibhav@linux.ibm.com>
Implement support for fetching nvdimm health information via
H_SCM_HEALTH hcall as documented in Ref[1]. The hcall returns a pair
of 64-bit big-endian integers which are then stored in 'struct
papr_scm_priv' and subsequently partially exposed to user-space via
newly introduced dimm specific attribute 'papr_flags'. Also a new asm
header named 'papr-scm.h' is added that describes the interface
between PHYP and guest kernel.
Following flags are reported via 'papr_flags' sysfs attribute contents
of which are space separated string flags indicating various nvdimm
states:
* "not_armed" : Indicating that nvdimm contents wont survive a power
cycle.
* "save_fail" : Indicating that nvdimm contents couldn't be flushed
during last shutdown event.
* "restore_fail": Indicating that nvdimm contents couldn't be restored
during dimm initialization.
* "encrypted" : Dimm contents are encrypted.
* "smart_notify": There is health event for the nvdimm.
* "scrubbed" : Indicating that contents of the nvdimm have been
scrubbed.
* "locked" : Indicating that nvdimm contents cant be modified
until next power cycle.
[1]: commit 58b278f568f0 ("powerpc: Provide initial documentation for
PAPR hcalls")
Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
arch/powerpc/include/asm/papr_scm.h | 49 ++++++++++
arch/powerpc/platforms/pseries/papr_scm.c | 105 +++++++++++++++++++++-
2 files changed, 152 insertions(+), 2 deletions(-)
create mode 100644 arch/powerpc/include/asm/papr_scm.h
diff --git a/arch/powerpc/include/asm/papr_scm.h b/arch/powerpc/include/asm/papr_scm.h
new file mode 100644
index 000000000000..4712dfef8990
--- /dev/null
+++ b/arch/powerpc/include/asm/papr_scm.h
@@ -0,0 +1,49 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Structures and defines needed to manage nvdimms for spapr guests.
+ */
+#ifndef _ASM_POWERPC_PAPR_SCM_H_
+#define _ASM_POWERPC_PAPR_SCM_H_
+
+#include <linux/types.h>
+#include <asm/bitsperlong.h>
+
+/* DIMM health bitmap bitmap indicators */
+/* SCM device is unable to persist memory contents */
+#define PAPR_SCM_DIMM_UNARMED PPC_BIT(0)
+/* SCM device failed to persist memory contents */
+#define PAPR_SCM_DIMM_SHUTDOWN_DIRTY PPC_BIT(1)
+/* SCM device contents are persisted from previous IPL */
+#define PAPR_SCM_DIMM_SHUTDOWN_CLEAN PPC_BIT(2)
+/* SCM device contents are not persisted from previous IPL */
+#define PAPR_SCM_DIMM_EMPTY PPC_BIT(3)
+/* SCM device memory life remaining is critically low */
+#define PAPR_SCM_DIMM_HEALTH_CRITICAL PPC_BIT(4)
+/* SCM device will be garded off next IPL due to failure */
+#define PAPR_SCM_DIMM_HEALTH_FATAL PPC_BIT(5)
+/* SCM contents cannot persist due to current platform health status */
+#define PAPR_SCM_DIMM_HEALTH_UNHEALTHY PPC_BIT(6)
+/* SCM device is unable to persist memory contents in certain conditions */
+#define PAPR_SCM_DIMM_HEALTH_NON_CRITICAL PPC_BIT(7)
+/* SCM device is encrypted */
+#define PAPR_SCM_DIMM_ENCRYPTED PPC_BIT(8)
+/* SCM device has been scrubbed and locked */
+#define PAPR_SCM_DIMM_SCRUBBED_AND_LOCKED PPC_BIT(9)
+
+/* Bits status indicators for health bitmap indicating unarmed dimm */
+#define PAPR_SCM_DIMM_UNARMED_MASK (PAPR_SCM_DIMM_UNARMED | \
+ PAPR_SCM_DIMM_HEALTH_UNHEALTHY | \
+ PAPR_SCM_DIMM_HEALTH_NON_CRITICAL)
+
+/* Bits status indicators for health bitmap indicating unflushed dimm */
+#define PAPR_SCM_DIMM_BAD_SHUTDOWN_MASK (PAPR_SCM_DIMM_SHUTDOWN_DIRTY)
+
+/* Bits status indicators for health bitmap indicating unrestored dimm */
+#define PAPR_SCM_DIMM_BAD_RESTORE_MASK (PAPR_SCM_DIMM_EMPTY)
+
+/* Bit status indicators for smart event notification */
+#define PAPR_SCM_DIMM_SMART_EVENT_MASK (PAPR_SCM_DIMM_HEALTH_CRITICAL | \
+ PAPR_SCM_DIMM_HEALTH_FATAL | \
+ PAPR_SCM_DIMM_HEALTH_UNHEALTHY)
+
+#endif
diff --git a/arch/powerpc/platforms/pseries/papr_scm.c b/arch/powerpc/platforms/pseries/papr_scm.c
index 0b4467e378e5..aaf2e4ab1f75 100644
--- a/arch/powerpc/platforms/pseries/papr_scm.c
+++ b/arch/powerpc/platforms/pseries/papr_scm.c
@@ -14,6 +14,7 @@
#include <linux/delay.h>
#include <asm/plpar_wrappers.h>
+#include <asm/papr_scm.h>
#define BIND_ANY_ADDR (~0ul)
@@ -39,6 +40,13 @@ struct papr_scm_priv {
struct resource res;
struct nd_region *region;
struct nd_interleave_set nd_set;
+
+ /* Protect dimm data from concurrent access */
+ struct mutex dimm_mutex;
+
+ /* Health information for the dimm */
+ __be64 health_bitmap;
+ __be64 health_bitmap_valid;
};
static int drc_pmem_bind(struct papr_scm_priv *p)
@@ -144,6 +152,35 @@ static int drc_pmem_query_n_bind(struct papr_scm_priv *p)
return drc_pmem_bind(p);
}
+static int drc_pmem_query_health(struct papr_scm_priv *p)
+{
+ unsigned long ret[PLPAR_HCALL_BUFSIZE];
+ int64_t rc;
+
+ rc = plpar_hcall(H_SCM_HEALTH, ret, p->drc_index);
+ if (rc != H_SUCCESS) {
+ dev_err(&p->pdev->dev,
+ "Failed to query health information, Err:%lld\n", rc);
+ return -ENXIO;
+ }
+
+ /* Protect modifications to papr_scm_priv with the mutex */
+ rc = mutex_lock_interruptible(&p->dimm_mutex);
+ if (rc)
+ return rc;
+
+ /* Store the retrieved health information in dimm platform data */
+ p->health_bitmap = ret[0];
+ p->health_bitmap_valid = ret[1];
+
+ dev_dbg(&p->pdev->dev,
+ "Queried dimm health info. Bitmap:0x%016llx Mask:0x%016llx\n",
+ be64_to_cpu(p->health_bitmap),
+ be64_to_cpu(p->health_bitmap_valid));
+
+ mutex_unlock(&p->dimm_mutex);
+ return 0;
+}
static int papr_scm_meta_get(struct papr_scm_priv *p,
struct nd_cmd_get_config_data_hdr *hdr)
@@ -304,6 +341,67 @@ static inline int papr_scm_node(int node)
return min_node;
}
+static ssize_t papr_flags_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct nvdimm *dimm = to_nvdimm(dev);
+ struct papr_scm_priv *p = nvdimm_provider_data(dimm);
+ __be64 health;
+ int rc;
+
+ rc = drc_pmem_query_health(p);
+ if (rc)
+ return rc;
+
+ /* Protect against modifications to papr_scm_priv with the mutex */
+ rc = mutex_lock_interruptible(&p->dimm_mutex);
+ if (rc)
+ return rc;
+
+ health = p->health_bitmap & p->health_bitmap_valid;
+
+ /* Check for various masks in bitmap and set the buffer */
+ if (health & PAPR_SCM_DIMM_UNARMED_MASK)
+ rc += sprintf(buf, "not_armed ");
+
+ if (health & PAPR_SCM_DIMM_BAD_SHUTDOWN_MASK)
+ rc += sprintf(buf + rc, "save_fail ");
+
+ if (health & PAPR_SCM_DIMM_BAD_RESTORE_MASK)
+ rc += sprintf(buf + rc, "restore_fail ");
+
+ if (health & PAPR_SCM_DIMM_ENCRYPTED)
+ rc += sprintf(buf + rc, "encrypted ");
+
+ if (health & PAPR_SCM_DIMM_SMART_EVENT_MASK)
+ rc += sprintf(buf + rc, "smart_notify ");
+
+ if (health & PAPR_SCM_DIMM_SCRUBBED_AND_LOCKED)
+ rc += sprintf(buf + rc, "scrubbed locked ");
+
+ if (rc > 0)
+ rc += sprintf(buf + rc, "\n");
+
+ mutex_unlock(&p->dimm_mutex);
+ return rc;
+}
+DEVICE_ATTR_RO(papr_flags);
+
+/* papr_scm specific dimm attributes */
+static struct attribute *papr_scm_nd_attributes[] = {
+ &dev_attr_papr_flags.attr,
+ NULL,
+};
+
+static struct attribute_group papr_scm_nd_attribute_group = {
+ .attrs = papr_scm_nd_attributes,
+};
+
+static const struct attribute_group *papr_scm_dimm_attr_groups[] = {
+ &papr_scm_nd_attribute_group,
+ NULL,
+};
+
static int papr_scm_nvdimm_init(struct papr_scm_priv *p)
{
struct device *dev = &p->pdev->dev;
@@ -330,8 +428,8 @@ static int papr_scm_nvdimm_init(struct papr_scm_priv *p)
dimm_flags = 0;
set_bit(NDD_ALIASING, &dimm_flags);
- p->nvdimm = nvdimm_create(p->bus, p, NULL, dimm_flags,
- PAPR_SCM_DIMM_CMD_MASK, 0, NULL);
+ p->nvdimm = nvdimm_create(p->bus, p, papr_scm_dimm_attr_groups,
+ dimm_flags, PAPR_SCM_DIMM_CMD_MASK, 0, NULL);
if (!p->nvdimm) {
dev_err(dev, "Error creating DIMM object for %pOF\n", p->dn);
goto err;
@@ -415,6 +513,9 @@ static int papr_scm_probe(struct platform_device *pdev)
if (!p)
return -ENOMEM;
+ /* Initialize the dimm mutex */
+ mutex_init(&p->dimm_mutex);
+
/* optional DT properties */
of_property_read_u32(dn, "ibm,metadata-size", &metadata_size);
--
2.24.1
^ permalink raw reply related
* [PATCH v2 3/4] powerpc/papr_scm, uapi: Add support for handling PAPR DSM commands
From: Vaibhav Jain @ 2020-03-28 12:10 UTC (permalink / raw)
To: linuxppc-dev
Cc: Alastair D'Silva, Aneesh Kumar K . V, Jeff Moyer,
Oliver O'Halloran, Vishal Verma, Vaibhav Jain,
Michael Ellerman, Dan Williams
In-Reply-To: <20200328121023.160363-1-vaibhav@linux.ibm.com>
Implement support for handling PAPR DSM commands in papr_scm
module. We advertise support for ND_CMD_CALL for the dimm command mask
and implement necessary scaffolding in the module to handle ND_CMD_CALL
ioctl and DSM commands that we receive.
The layout of the DSM commands as we expect from libnvdimm/libndctl is
described in newly introduced uapi header 'papr_scm_dsm.h' which
defines a new 'struct nd_papr_scm_cmd_pkg' header. This header is used
to communicate the DSM command via 'nd_pkg_papr_scm->nd_command' and
size of payload that need to be sent/received for servicing the DSM.
The PAPR DSM commands are assigned indexes started from 0x10000 to
prevent them from overlapping ND_CMD_* values and also makes handling
dimm commands in papr_scm_ndctl() easier via a simplified switch-case
block. For this a new function cmd_to_func() is implemented that reads
the args to papr_scm_ndctl() , performs sanity tests on them and
converts them to PAPR DSM commands which can then be handled via the
switch-case block.
Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
arch/powerpc/include/uapi/asm/papr_scm_dsm.h | 161 +++++++++++++++++++
arch/powerpc/platforms/pseries/papr_scm.c | 87 +++++++++-
2 files changed, 241 insertions(+), 7 deletions(-)
create mode 100644 arch/powerpc/include/uapi/asm/papr_scm_dsm.h
diff --git a/arch/powerpc/include/uapi/asm/papr_scm_dsm.h b/arch/powerpc/include/uapi/asm/papr_scm_dsm.h
new file mode 100644
index 000000000000..1f3127a27bde
--- /dev/null
+++ b/arch/powerpc/include/uapi/asm/papr_scm_dsm.h
@@ -0,0 +1,161 @@
+/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
+/*
+ * PAPR SCM Device specific methods and struct for libndctl and ndctl
+ *
+ * (C) Copyright IBM 2020
+ *
+ * Author: Vaibhav Jain <vaibhav at linux.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _UAPI_ASM_POWERPC_PAPR_SCM_DSM_H_
+#define _UAPI_ASM_POWERPC_PAPR_SCM_DSM_H_
+
+#include <linux/types.h>
+
+#ifdef __KERNEL__
+#include <linux/ndctl.h>
+#else
+#include <ndctl.h>
+#endif
+
+/*
+ * DSM Envelope:
+ *
+ * The ioctl ND_CMD_CALL transfers data between user-space and kernel via
+ * 'envelopes' which consists of a header and user-defined payload sections.
+ * The header is described by 'struct nd_papr_scm_cmd_pkg' which expects a
+ * payload following it and offset of which relative to the struct is provided
+ * by 'nd_papr_scm_cmd_pkg.payload_offset'. *
+ *
+ * +-------------+---------------------+---------------------------+
+ * | 64-Bytes | 8-Bytes | Max 184-Bytes |
+ * +-------------+---------------------+---------------------------+
+ * | nd_papr_scm_cmd_pkg | |
+ * |-------------+ | |
+ * | nd_cmd_pkg | | |
+ * +-------------+---------------------+---------------------------+
+ * | nd_family | | |
+ * | nd_size_out | cmd_status | |
+ * | nd_size_in | payload_version | PAYLOAD |
+ * | nd_command | payload_offset -----> |
+ * | nd_fw_size | | |
+ * +-------------+---------------------+---------------------------+
+ *
+ * DSM Header:
+ *
+ * The header is defined as 'struct nd_papr_scm_cmd_pkg' which embeds a
+ * 'struct nd_cmd_pkg' instance. The DSM command is assigned to member
+ * 'nd_cmd_pkg.nd_command'. Apart from size information of the envelop which is
+ * contained in 'struct nd_cmd_pkg', the header also has members following
+ * members:
+ *
+ * 'cmd_status' : (Out) Errors if any encountered while servicing DSM.
+ * 'payload_version' : (In/Out) Version number associated with the payload.
+ * 'payload_offset' : (In)Relative offset of payload from start of envelope.
+ *
+ * DSM Payload:
+ *
+ * The layout of the DSM Payload is defined by various structs shared between
+ * papr_scm and libndctl so that contents of payload can be interpreted. During
+ * servicing of a DSM the papr_scm module will read input args from the payload
+ * field by casting its contents to an appropriate struct pointer based on the
+ * DSM command. Similarly the output of servicing the DSM command will be copied
+ * to the payload field using the same struct.
+ *
+ * 'libnvdimm' enforces a hard limit of 256 bytes on the envelope size, which
+ * leaves around 184 bytes for the envelope payload (ignoring any padding that
+ * the compiler may silently introduce).
+ *
+ * Payload Version:
+ *
+ * A 'payload_version' field is present in DSM header that indicates a specific
+ * version of the structure present in DSM Payload for a given DSM command. This
+ * provides backward compatibility in case the DSM Payload structure evolves
+ * and different structures are supported by 'papr_scm' and 'libndctl'.
+ *
+ * When sending a DSM Payload to 'papr_scm', 'libndctl' should send the version
+ * of the payload struct it supports via 'payload_version' field. The 'papr_scm'
+ * module when servicing the DSM envelop checks the 'payload_version' and then
+ * uses 'payload struct version' == MIN('payload_version field',
+ * 'max payload-struct-version supported by papr_scm') to service the DSM. After
+ * servicing the DSM, 'papr_scm' put the negotiated version of payload struct in
+ * returned 'payload_version' field.
+ *
+ * Libndctl on receiving the envelop back from papr_scm again checks the
+ * 'payload_version' field and based on it use the appropriate version dsm
+ * struct to parse the results.
+ *
+ * Backward Compatibility:
+ *
+ * Above scheme of exchanging different versioned DSM struct between libndctl
+ * and papr_scm should provide backward compatibility until following two
+ * assumptions/conditions when defining new DSM structs hold:
+ *
+ * Let T(X) = { set of attributes in DSM struct 'T' versioned X }
+ *
+ * 1. T(X) is a proper subset of T(Y) if X > Y.
+ * i.e Each new version of DSM struct should retain existing struct
+ * attributes from previous version
+ *
+ * 2. If an entity (libndctl or papr_scm) supports a DSM struct T(X) then
+ * it should also support T(1), T(2)...T(X - 1).
+ * i.e When adding support for new version of a DSM struct, libndctl
+ * and papr_scm should retain support of the existing DSM struct
+ * version they support.
+ */
+
+/* Papr-scm-header + payload expected with ND_CMD_CALL ioctl from libnvdimm */
+struct nd_papr_scm_cmd_pkg {
+ struct nd_cmd_pkg hdr; /* Package header containing sub-cmd */
+ int32_t cmd_status; /* Out: Sub-cmd status returned back */
+ uint16_t payload_offset; /* In: offset from start of struct */
+ uint16_t payload_version; /* In/Out: version of the payload */
+ uint8_t payload[]; /* In/Out: Sub-cmd data buffer */
+};
+
+/*
+ * Sub commands for ND_CMD_CALL. To prevent overlap from ND_CMD_*, values for
+ * these enums start at 0x10000. These values are then returned from
+ * cmd_to_func() making it easy to implement the switch-case block in
+ * papr_scm_ndctl(). These commands are sent to the kernel via
+ * 'nd_papr_scm_cmd_pkg.hdr.nd_command'
+ */
+enum dsm_papr_scm {
+ DSM_PAPR_SCM_MIN = 0x10000,
+ DSM_PAPR_SCM_MAX,
+};
+
+/* Helpers to evaluate the size of PAPR_SCM envelope */
+/* Calculate the papr_scm-header size */
+#define ND_PAPR_SCM_ENVELOPE_CONTENT_HDR_SIZE \
+ (sizeof(struct nd_papr_scm_cmd_pkg) - sizeof(struct nd_cmd_pkg))
+
+/* Given a type calculate envelope-content size (papr_scm-header + payload) */
+#define ND_PAPR_SCM_ENVELOPE_CONTENT_SIZE(_type_) \
+ (sizeof(_type_) + ND_PAPR_SCM_ENVELOPE_CONTENT_HDR_SIZE)
+
+/* Convert a libnvdimm nd_cmd_pkg to papr_scm specific pkg */
+static struct nd_papr_scm_cmd_pkg *nd_to_papr_cmd_pkg(struct nd_cmd_pkg *cmd)
+{
+ return (struct nd_papr_scm_cmd_pkg *) cmd;
+}
+
+/* Return the payload pointer for a given pcmd */
+static void *papr_scm_pcmd_to_payload(struct nd_papr_scm_cmd_pkg *pcmd)
+{
+ if (pcmd->hdr.nd_size_in == 0 && pcmd->hdr.nd_size_out == 0)
+ return NULL;
+ else
+ return (void *)((u8 *) pcmd + pcmd->payload_offset);
+}
+#endif /* _UAPI_ASM_POWERPC_PAPR_SCM_DSM_H_ */
diff --git a/arch/powerpc/platforms/pseries/papr_scm.c b/arch/powerpc/platforms/pseries/papr_scm.c
index aaf2e4ab1f75..9a2614aaae88 100644
--- a/arch/powerpc/platforms/pseries/papr_scm.c
+++ b/arch/powerpc/platforms/pseries/papr_scm.c
@@ -15,13 +15,15 @@
#include <asm/plpar_wrappers.h>
#include <asm/papr_scm.h>
+#include <asm/papr_scm_dsm.h>
#define BIND_ANY_ADDR (~0ul)
#define PAPR_SCM_DIMM_CMD_MASK \
((1ul << ND_CMD_GET_CONFIG_SIZE) | \
(1ul << ND_CMD_GET_CONFIG_DATA) | \
- (1ul << ND_CMD_SET_CONFIG_DATA))
+ (1ul << ND_CMD_SET_CONFIG_DATA) | \
+ (1ul << ND_CMD_CALL))
struct papr_scm_priv {
struct platform_device *pdev;
@@ -283,19 +285,82 @@ static int papr_scm_meta_set(struct papr_scm_priv *p,
return 0;
}
+/*
+ * Validate the input to dimm-control function and return papr_scm specific
+ * commands. This does sanity validation to ND_CMD_CALL sub-command packages.
+ */
+static int cmd_to_func(struct nvdimm *nvdimm, unsigned int cmd, void *buf,
+ unsigned int buf_len)
+{
+ unsigned long cmd_mask = PAPR_SCM_DIMM_CMD_MASK;
+ struct nd_papr_scm_cmd_pkg *pkg = nd_to_papr_cmd_pkg(buf);
+
+ /* Only dimm-specific calls are supported atm */
+ if (!nvdimm)
+ return -EINVAL;
+
+ if (!test_bit(cmd, &cmd_mask)) {
+ pr_debug("%s: Unsupported cmd=%u\n", __func__, cmd);
+ return -EINVAL;
+ } else if (cmd != ND_CMD_CALL) {
+ return cmd;
+ }
+
+ /* cmd == ND_CMD_CALL so verify the envelop package */
+
+ if (!buf || buf_len < sizeof(struct nd_papr_scm_cmd_pkg)) {
+ pr_debug("%s: Invalid pkg size=%u\n", __func__, buf_len);
+ return -EINVAL;
+ }
+
+ if (pkg->hdr.nd_family != NVDIMM_FAMILY_PAPR_SCM) {
+ pr_debug("%s: Invalid pkg family=0x%llx\n", __func__,
+ pkg->hdr.nd_family);
+ return -EINVAL;
+
+ }
+
+ if (pkg->hdr.nd_command <= DSM_PAPR_SCM_MIN ||
+ pkg->hdr.nd_command >= DSM_PAPR_SCM_MAX) {
+
+ /* for unknown subcommands return ND_CMD_CALL */
+ pr_debug("%s: Unknown sub-command=0x%llx\n", __func__,
+ pkg->hdr.nd_command);
+ return ND_CMD_CALL;
+ }
+
+ /* We except a payload with all DSM commands */
+ if (papr_scm_pcmd_to_payload(pkg) == NULL) {
+ pr_debug("%s: Empty patload for sub-command=0x%llx\n", __func__,
+ pkg->hdr.nd_command);
+ return -EINVAL;
+ }
+
+ /* Return the DSM_PAPR_SCM_* command */
+ return pkg->hdr.nd_command;
+}
+
int papr_scm_ndctl(struct nvdimm_bus_descriptor *nd_desc, struct nvdimm *nvdimm,
unsigned int cmd, void *buf, unsigned int buf_len, int *cmd_rc)
{
struct nd_cmd_get_config_size *get_size_hdr;
struct papr_scm_priv *p;
+ struct nd_papr_scm_cmd_pkg *call_pkg = NULL;
+ int cmd_in, rc;
- /* Only dimm-specific calls are supported atm */
- if (!nvdimm)
- return -EINVAL;
+ /* Use a local variable in case cmd_rc pointer is NULL */
+ if (cmd_rc == NULL)
+ cmd_rc = &rc;
+
+ cmd_in = cmd_to_func(nvdimm, cmd, buf, buf_len);
+ if (cmd_in < 0) {
+ pr_debug("%s: Invalid cmd=%u. Err=%d\n", __func__, cmd, cmd_in);
+ return cmd_in;
+ }
p = nvdimm_provider_data(nvdimm);
- switch (cmd) {
+ switch (cmd_in) {
case ND_CMD_GET_CONFIG_SIZE:
get_size_hdr = buf;
@@ -313,13 +378,21 @@ int papr_scm_ndctl(struct nvdimm_bus_descriptor *nd_desc, struct nvdimm *nvdimm,
*cmd_rc = papr_scm_meta_set(p, buf);
break;
+ case ND_CMD_CALL:
+ /* This happens if subcommand package sanity fails */
+ call_pkg = nd_to_papr_cmd_pkg(buf);
+ call_pkg->cmd_status = -ENOENT;
+ *cmd_rc = 0;
+ break;
+
default:
- return -EINVAL;
+ dev_dbg(&p->pdev->dev, "Unknown command = %d\n", cmd_in);
+ *cmd_rc = -EINVAL;
}
dev_dbg(&p->pdev->dev, "returned with cmd_rc = %d\n", *cmd_rc);
- return 0;
+ return *cmd_rc;
}
static inline int papr_scm_node(int node)
--
2.24.1
^ permalink raw reply related
* Re: [PATCH 0/2] powerpc: Remove support for ppc405/440 Xilinx platforms
From: Christophe Leroy @ 2020-03-28 11:17 UTC (permalink / raw)
To: Andy Shevchenko, Arnd Bergmann, Michael Ellerman
Cc: Kate Stewart, Mark Rutland, Desnes A. Nunes do Rosario,
Geert Uytterhoeven, open list:DOCUMENTATION,
ALSA Development Mailing List, dri-devel, Jaroslav Kysela,
Richard Fontana, Paul Mackerras, Miquel Raynal,
Mauro Carvalho Chehab, Fabio Estevam, Sasha Levin,
Stephen Rothwell, Jonathan Corbet, Masahiro Yamada, Takashi Iwai,
YueHaibing, Michal Simek, Krzysztof Kozlowski, Linux ARM,
Leonardo Bras, DTML, Andrew Donnellan, Bartlomiej Zolnierkiewicz,
Marc Zyngier, Alistair Popple, linuxppc-dev, Nicholas Piggin,
Alexios Zavras, Mark Brown, git, Linux Fbdev development list,
Jonathan Cameron, Thomas Gleixner, Allison Randal, Michal Simek,
Wei Hu, Christian Lamparter, Greg Kroah-Hartman, Nick Desaulniers,
linux-kernel@vger.kernel.org, Armijn Hemel, Rob Herring,
Enrico Weigelt, David S. Miller, Thiago Jung Bauermann
In-Reply-To: <20200327141434.GA1922688@smile.fi.intel.com>
Le 27/03/2020 à 15:14, Andy Shevchenko a écrit :
> On Fri, Mar 27, 2020 at 02:22:55PM +0100, Arnd Bergmann wrote:
>> On Fri, Mar 27, 2020 at 2:15 PM Andy Shevchenko
>> <andriy.shevchenko@linux.intel.com> wrote:
>>> On Fri, Mar 27, 2020 at 03:10:26PM +0200, Andy Shevchenko wrote:
>>>> On Fri, Mar 27, 2020 at 01:54:33PM +0100, Arnd Bergmann wrote:
>>>>> On Fri, Mar 27, 2020 at 1:12 PM Michal Simek <michal.simek@xilinx.com> wrote:
>
> ...
>
>>>>> It does raise a follow-up question about ppc40x though: is it time to
>>>>> retire all of it?
>>>>
>>>> Who knows?
>>>>
>>>> I have in possession nice WD My Book Live, based on this architecture, and I
>>>> won't it gone from modern kernel support. OTOH I understand that amount of real
>>>> users not too big.
>>>
>>> +Cc: Christian Lamparter, whom I owe for that WD box.
>>
>> According to https://openwrt.org/toh/wd/mybooklive, that one is based on
>> APM82181/ppc464, so it is about several generations newer than what I
>> asked about (ppc40x).
>>
>>>> Ah, and I have Amiga board, but that one is being used only for testing, so,
>>>> I don't care much.
>>
>> I think there are a couple of ppc440 based Amiga boards, but again, not 405
>> to my knowledge.
>
> Ah, you are right. No objections from ppc40x removal!
>
Removing 40x would help cleaning things a bit. For instance 40x is the
last platform still having PTE_ATOMIC_UPDATES. So if we can remove 40x
we can get rid of PTE_ATOMIC_UPDATES completely.
If no one objects, I can prepare a series to drop support for 40x
completely.
Michael, any thought ?
Christophe
^ permalink raw reply
* Re: [PATCH 1/1] ppc/crash: Skip spinlocks during crash
From: Christophe Leroy @ 2020-03-28 10:19 UTC (permalink / raw)
To: Leonardo Bras, Peter Zijlstra, Ingo Molnar, Will Deacon,
Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman,
Enrico Weigelt, Allison Randal, Thomas Gleixner
Cc: linuxppc-dev, linux-kernel
In-Reply-To: <56965ad674071181548d5ed4fb7c8fa08061b591.camel@linux.ibm.com>
Hi Leonardo,
On 03/27/2020 03:51 PM, Leonardo Bras wrote:
> Hello Christophe, thanks for the feedback.
>
> I noticed an error in this patch and sent a v2, that can be seen here:
> http://patchwork.ozlabs.org/patch/1262468/
>
> Comments inline::
>
> On Fri, 2020-03-27 at 07:50 +0100, Christophe Leroy wrote:
>>> @@ -142,6 +144,8 @@ static inline void arch_spin_lock(arch_spinlock_t *lock)
>>> if (likely(__arch_spin_trylock(lock) == 0))
>>> break;
>>> do {
>>> + if (unlikely(crash_skip_spinlock))
>>> + return;
>
> Complete function for reference:
> static inline void arch_spin_lock(arch_spinlock_t *lock)
> {
> while (1) {
> if (likely(__arch_spin_trylock(lock) == 0))
> break;
> do {
> if (unlikely(crash_skip_spinlock))
> return;
> HMT_low();
> if (is_shared_processor())
> splpar_spin_yield(lock);
> } while (unlikely(lock->slock != 0));
> HMT_medium();
> }
> }
>
>> You are adding a test that reads a global var in the middle of a so hot
>> path ? That must kill performance.
>
> I thought it would, in worst case scenario, increase a maximum delay of
> an arch_spin_lock() call 1 spin cycle. Here is what I thought:
>
> - If the lock is already free, it would change nothing,
> - Otherwise, the lock will wait.
> - Waiting cycle just got bigger.
> - Worst case scenario: running one more cycle, given lock->slock can
> turn to 0 just after checking.
>
> Could you please point where I failed to see the performance penalty?
> (I need to get better at this :) )
You are right that when the lock is free, it changes nothing. However
when it is not, it is not just one cycle.
Here is arch_spin_lock() without your patch:
00000440 <my_lock>:
440: 39 40 00 01 li r10,1
444: 7d 20 18 28 lwarx r9,0,r3
448: 2c 09 00 00 cmpwi r9,0
44c: 40 82 00 10 bne 45c <my_lock+0x1c>
450: 7d 40 19 2d stwcx. r10,0,r3
454: 40 a2 ff f0 bne 444 <my_lock+0x4>
458: 4c 00 01 2c isync
45c: 2f 89 00 00 cmpwi cr7,r9,0
460: 4d be 00 20 bclr+ 12,4*cr7+eq
464: 7c 21 0b 78 mr r1,r1
468: 81 23 00 00 lwz r9,0(r3)
46c: 2f 89 00 00 cmpwi cr7,r9,0
470: 40 be ff f4 bne cr7,464 <my_lock+0x24>
474: 7c 42 13 78 mr r2,r2
478: 7d 20 18 28 lwarx r9,0,r3
47c: 2c 09 00 00 cmpwi r9,0
480: 40 82 00 10 bne 490 <my_lock+0x50>
484: 7d 40 19 2d stwcx. r10,0,r3
488: 40 a2 ff f0 bne 478 <my_lock+0x38>
48c: 4c 00 01 2c isync
490: 2f 89 00 00 cmpwi cr7,r9,0
494: 40 be ff d0 bne cr7,464 <my_lock+0x24>
498: 4e 80 00 20 blr
Here is arch_spin_lock() with your patch. I enclose with === what comes
in addition:
00000440 <my_lock>:
440: 39 40 00 01 li r10,1
444: 7d 20 18 28 lwarx r9,0,r3
448: 2c 09 00 00 cmpwi r9,0
44c: 40 82 00 10 bne 45c <my_lock+0x1c>
450: 7d 40 19 2d stwcx. r10,0,r3
454: 40 a2 ff f0 bne 444 <my_lock+0x4>
458: 4c 00 01 2c isync
45c: 2f 89 00 00 cmpwi cr7,r9,0
460: 4d be 00 20 bclr+ 12,4*cr7+eq
=====================================================
464: 3d 40 00 00 lis r10,0
466: R_PPC_ADDR16_HA crash_skip_spinlock
468: 39 4a 00 00 addi r10,r10,0
46a: R_PPC_ADDR16_LO crash_skip_spinlock
46c: 39 00 00 01 li r8,1
470: 89 2a 00 00 lbz r9,0(r10)
474: 2f 89 00 00 cmpwi cr7,r9,0
478: 4c 9e 00 20 bnelr cr7
=====================================================
47c: 7c 21 0b 78 mr r1,r1
480: 81 23 00 00 lwz r9,0(r3)
484: 2f 89 00 00 cmpwi cr7,r9,0
488: 40 be ff f4 bne cr7,47c <my_lock+0x3c>
48c: 7c 42 13 78 mr r2,r2
490: 7d 20 18 28 lwarx r9,0,r3
494: 2c 09 00 00 cmpwi r9,0
498: 40 82 00 10 bne 4a8 <my_lock+0x68>
49c: 7d 00 19 2d stwcx. r8,0,r3
4a0: 40 a2 ff f0 bne 490 <my_lock+0x50>
4a4: 4c 00 01 2c isync
4a8: 2f 89 00 00 cmpwi cr7,r9,0
4ac: 40 be ff c4 bne cr7,470 <my_lock+0x30>
4b0: 4e 80 00 20 blr
Christophe
^ permalink raw reply
* Re: [PATCH v2] powerpc xmon: use `dcbf` inplace of `dcbi` instruction for 64bit Book3S
From: Balamuruhan S @ 2020-03-28 8:03 UTC (permalink / raw)
To: Christophe Leroy, mpe
Cc: ravi.bangoria, jniethe5, paulus, sandipan, naveen.n.rao,
linuxppc-dev
In-Reply-To: <9a3c084a-9e86-ff37-111c-6f1a8f0989fc@c-s.fr>
On Fri, 2020-03-27 at 16:12 +0100, Christophe Leroy wrote:
>
> Le 27/03/2020 à 10:03, Balamuruhan S a écrit :
> > On Fri, 2020-03-27 at 07:48 +0100, Christophe Leroy wrote:
> > > Le 26/03/2020 à 07:15, Balamuruhan S a écrit :
> > > > Data Cache Block Invalidate (dcbi) instruction was implemented back in
> > > > PowerPC
> > > > architecture version 2.03. It is obsolete and attempt to use of this
> > > > illegal
> > > > instruction results in a hypervisor emulation assistance interrupt. So,
> > > > ifdef
> > > > it out the option `i` in xmon for 64bit Book3S.
> > >
> > > I don't understand. You say two contradictory things:
> > > 1/ You say it _was_ added back.
> > > 2/ You say it _is_ obsolete.
> > >
> > > How can it be obsolete if it was added back ?
> >
> > I actually learnt it from P8 and P9 User Manual,
> >
> > The POWER8/POWER9 core does not provide support for the following optional
> > or
> > obsolete instructions (attempted use of these results in a hypervisor
> > emulation
> > assistance interrupt):
> > • tlbia - TLB invalidate all
> > • tlbiex - TLB invalidate entry by index (obsolete)
> > • slbiex - SLB invalidate entry by index (obsolete)
> > • dcba - Data cache block allocate (Book II; obsolete)
> > • dcbi - Data cache block invalidate (obsolete)
> > • rfi - Return from interrupt (32-bit; obsolete)
> >
>
> Then that's exactly what you have to say in the coming log.
Sure, I will change the commit log in next version along with your suggested
way to ifdef.
>
> Maybe you could also change invalidate_dcache_range():
>
> for (i = 0; i < size >> shift; i++, addr += bytes) {
> if (IS_ENABLED(CONFIG_PPC_BOOK3S_64))
> dcbf(addr);
> else
> dcbi(addr);
> }
I will leave this as is based on the discussion.
Thank you Christophe and Segher.
-- Bala
>
>
>
>
> Christophe
^ permalink raw reply
* Re: [PATCH 0/6] Kill setup_irq()
From: afzal mohammed @ 2020-03-28 7:32 UTC (permalink / raw)
To: Brian Cain
Cc: linux-s390, linux-samsung-soc, linux-ia64, linux-c6x-dev,
linux-parisc, linux-sh, linux-hexagon, x86, linux-kernel,
linux-mips, linux-m68k, linux-alpha, 'Thomas Gleixner',
linux-omap, linuxppc-dev, linux-arm-kernel
In-Reply-To: <059b01d604ab$637355b0$2a5a0110$@codeaurora.org>
Hi Brian,
On Fri, Mar 27, 2020 at 09:48:38PM -0500, Brian Cain wrote:
> > Note 2: hexagon final image creation fails even w/o my patch
> What's the nature of the failure in "Note 2"?
drivers/base/firmware_loader/main.o: In function `fw_is_builtin_firmware':
/devel/src/kernel6/drivers/base/firmware_loader/main.c:132:(.text+0xc8): relocation truncated to fit: R_HEX_16_X against symbol `__start_builtin_fw' defined in .modinfo section in .tmp_vmlinux1
Makefile:1077: recipe for target 'vmlinux' failed
make: *** [vmlinux] Error 1
[ i had deleted the toolchain earlier, since you asked, download again &
checked ]
Regards
afzal
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox