* [PATCH v3 12/41] cxlflash: Use IDR to manage adapter contexts
From: Uma Krishnan @ 2018-03-26 16:32 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522081759-57431-1-git-send-email-ukrishn@linux.vnet.ibm.com>
A range of PASIDs are used as identifiers for the adapter contexts. These
contexts may be destroyed and created randomly. Use an IDR to keep track
of contexts that are in use and assign a unique identifier to new ones.
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
---
drivers/scsi/cxlflash/ocxl_hw.c | 21 +++++++++++++++++++--
drivers/scsi/cxlflash/ocxl_hw.h | 2 ++
2 files changed, 21 insertions(+), 2 deletions(-)
diff --git a/drivers/scsi/cxlflash/ocxl_hw.c b/drivers/scsi/cxlflash/ocxl_hw.c
index cbe4d93..e8864a1 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.c
+++ b/drivers/scsi/cxlflash/ocxl_hw.c
@@ -12,6 +12,8 @@
* 2 of the License, or (at your option) any later version.
*/
+#include <linux/idr.h>
+
#include <misc/ocxl.h>
#include "backend.h"
@@ -60,14 +62,25 @@ static void *ocxlflash_dev_context_init(struct pci_dev *pdev, void *afu_cookie)
if (unlikely(!ctx)) {
dev_err(dev, "%s: Context allocation failed\n", __func__);
rc = -ENOMEM;
- goto err;
+ goto err1;
+ }
+
+ idr_preload(GFP_KERNEL);
+ rc = idr_alloc(&afu->idr, ctx, 0, afu->max_pasid, GFP_NOWAIT);
+ idr_preload_end();
+ if (unlikely(rc < 0)) {
+ dev_err(dev, "%s: idr_alloc failed rc=%d\n", __func__, rc);
+ goto err2;
}
+ ctx->pe = rc;
ctx->master = false;
ctx->hw_afu = afu;
out:
return ctx;
-err:
+err2:
+ kfree(ctx);
+err1:
ctx = ERR_PTR(rc);
goto out;
}
@@ -86,6 +99,7 @@ static int ocxlflash_release_context(void *ctx_cookie)
if (!ctx)
goto out;
+ idr_remove(&ctx->hw_afu->idr, ctx->pe);
kfree(ctx);
out:
return rc;
@@ -103,6 +117,7 @@ static void ocxlflash_destroy_afu(void *afu_cookie)
return;
ocxlflash_release_context(afu->ocxl_ctx);
+ idr_destroy(&afu->idr);
kfree(afu);
}
@@ -221,6 +236,7 @@ static void *ocxlflash_create_afu(struct pci_dev *pdev)
afu->pdev = pdev;
afu->dev = dev;
+ idr_init(&afu->idr);
rc = ocxlflash_config_fn(pdev, afu);
if (unlikely(rc)) {
@@ -248,6 +264,7 @@ static void *ocxlflash_create_afu(struct pci_dev *pdev)
out:
return afu;
err1:
+ idr_destroy(&afu->idr);
kfree(afu);
afu = NULL;
goto out;
diff --git a/drivers/scsi/cxlflash/ocxl_hw.h b/drivers/scsi/cxlflash/ocxl_hw.h
index f41ba0b..a5337b6 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.h
+++ b/drivers/scsi/cxlflash/ocxl_hw.h
@@ -26,6 +26,7 @@ struct ocxl_hw_afu {
int afu_actag_base; /* AFU acTag base */
int afu_actag_enabled; /* AFU acTag number enabled */
+ struct idr idr; /* IDR to manage contexts */
int max_pasid; /* Maximum number of contexts */
bool is_present; /* Function has AFUs defined */
};
@@ -33,4 +34,5 @@ struct ocxl_hw_afu {
struct ocxlflash_context {
struct ocxl_hw_afu *hw_afu; /* HW AFU back pointer */
bool master; /* Whether this is a master context */
+ int pe; /* Process element */
};
--
2.1.0
^ permalink raw reply related
* [PATCH v3 11/41] cxlflash: Adapter context support for OCXL
From: Uma Krishnan @ 2018-03-26 16:31 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522081759-57431-1-git-send-email-ukrishn@linux.vnet.ibm.com>
Add support to create and release the adapter contexts for OCXL and
provide means to specify certain contexts as a master.
The existing cxlflash core has a design requirement that each host will
have a single host context available by default. To satisfy this
requirement, one host adapter context is created when the hardware AFU is
initialized. This is returned by the get_context() fop.
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Reviewed-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
---
drivers/scsi/cxlflash/ocxl_hw.c | 90 +++++++++++++++++++++++++++++++++++++++++
drivers/scsi/cxlflash/ocxl_hw.h | 6 +++
2 files changed, 96 insertions(+)
diff --git a/drivers/scsi/cxlflash/ocxl_hw.c b/drivers/scsi/cxlflash/ocxl_hw.c
index 44688af..cbe4d93 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.c
+++ b/drivers/scsi/cxlflash/ocxl_hw.c
@@ -18,6 +18,80 @@
#include "ocxl_hw.h"
/**
+ * ocxlflash_set_master() - sets the context as master
+ * @ctx_cookie: Adapter context to set as master.
+ */
+static void ocxlflash_set_master(void *ctx_cookie)
+{
+ struct ocxlflash_context *ctx = ctx_cookie;
+
+ ctx->master = true;
+}
+
+/**
+ * ocxlflash_get_context() - obtains the context associated with the host
+ * @pdev: PCI device associated with the host.
+ * @afu_cookie: Hardware AFU associated with the host.
+ *
+ * Return: returns the pointer to host adapter context
+ */
+static void *ocxlflash_get_context(struct pci_dev *pdev, void *afu_cookie)
+{
+ struct ocxl_hw_afu *afu = afu_cookie;
+
+ return afu->ocxl_ctx;
+}
+
+/**
+ * ocxlflash_dev_context_init() - allocate and initialize an adapter context
+ * @pdev: PCI device associated with the host.
+ * @afu_cookie: Hardware AFU associated with the host.
+ *
+ * Return: returns the adapter context on success, ERR_PTR on failure
+ */
+static void *ocxlflash_dev_context_init(struct pci_dev *pdev, void *afu_cookie)
+{
+ struct ocxl_hw_afu *afu = afu_cookie;
+ struct device *dev = afu->dev;
+ struct ocxlflash_context *ctx;
+ int rc;
+
+ ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ if (unlikely(!ctx)) {
+ dev_err(dev, "%s: Context allocation failed\n", __func__);
+ rc = -ENOMEM;
+ goto err;
+ }
+
+ ctx->master = false;
+ ctx->hw_afu = afu;
+out:
+ return ctx;
+err:
+ ctx = ERR_PTR(rc);
+ goto out;
+}
+
+/**
+ * ocxlflash_release_context() - releases an adapter context
+ * @ctx_cookie: Adapter context to be released.
+ *
+ * Return: 0 on success, -errno on failure
+ */
+static int ocxlflash_release_context(void *ctx_cookie)
+{
+ struct ocxlflash_context *ctx = ctx_cookie;
+ int rc = 0;
+
+ if (!ctx)
+ goto out;
+
+ kfree(ctx);
+out:
+ return rc;
+}
+
+/**
* ocxlflash_destroy_afu() - destroy the AFU structure
* @afu_cookie: AFU to be freed.
*/
@@ -28,6 +102,7 @@ static void ocxlflash_destroy_afu(void *afu_cookie)
if (!afu)
return;
+ ocxlflash_release_context(afu->ocxl_ctx);
kfree(afu);
}
@@ -134,6 +209,7 @@ static int ocxlflash_config_afu(struct pci_dev *pdev, struct ocxl_hw_afu *afu)
static void *ocxlflash_create_afu(struct pci_dev *pdev)
{
struct device *dev = &pdev->dev;
+ struct ocxlflash_context *ctx;
struct ocxl_hw_afu *afu;
int rc;
@@ -159,6 +235,16 @@ static void *ocxlflash_create_afu(struct pci_dev *pdev)
__func__, rc);
goto err1;
}
+
+ ctx = ocxlflash_dev_context_init(pdev, afu);
+ if (IS_ERR(ctx)) {
+ rc = PTR_ERR(ctx);
+ dev_err(dev, "%s: ocxlflash_dev_context_init failed rc=%d\n",
+ __func__, rc);
+ goto err1;
+ }
+
+ afu->ocxl_ctx = ctx;
out:
return afu;
err1:
@@ -170,6 +256,10 @@ static void *ocxlflash_create_afu(struct pci_dev *pdev)
/* Backend ops to ocxlflash services */
const struct cxlflash_backend_ops cxlflash_ocxl_ops = {
.module = THIS_MODULE,
+ .set_master = ocxlflash_set_master,
+ .get_context = ocxlflash_get_context,
+ .dev_context_init = ocxlflash_dev_context_init,
+ .release_context = ocxlflash_release_context,
.create_afu = ocxlflash_create_afu,
.destroy_afu = ocxlflash_destroy_afu,
};
diff --git a/drivers/scsi/cxlflash/ocxl_hw.h b/drivers/scsi/cxlflash/ocxl_hw.h
index 46de75b..f41ba0b 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.h
+++ b/drivers/scsi/cxlflash/ocxl_hw.h
@@ -14,6 +14,7 @@
/* OCXL hardware AFU associated with the host */
struct ocxl_hw_afu {
+ struct ocxlflash_context *ocxl_ctx; /* Host context */
struct pci_dev *pdev; /* PCI device */
struct device *dev; /* Generic device */
@@ -28,3 +29,8 @@ struct ocxl_hw_afu {
int max_pasid; /* Maximum number of contexts */
bool is_present; /* Function has AFUs defined */
};
+
+struct ocxlflash_context {
+ struct ocxl_hw_afu *hw_afu; /* HW AFU back pointer */
+ bool master; /* Whether this is a master context */
+};
--
2.1.0
^ permalink raw reply related
* [PATCH v3 10/41] cxlflash: Setup AFU PASID
From: Uma Krishnan @ 2018-03-26 16:31 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522081759-57431-1-git-send-email-ukrishn@linux.vnet.ibm.com>
Per the OCXL specification, the maximum PASID supported by the AFU is
indicated by a field within the configuration space. Similar to acTags,
implementations can choose to use any sub-range of PASID within their
assigned range. For cxlflash, the entire range is used.
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Reviewed-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
---
drivers/scsi/cxlflash/ocxl_hw.c | 3 +++
drivers/scsi/cxlflash/ocxl_hw.h | 1 +
2 files changed, 4 insertions(+)
diff --git a/drivers/scsi/cxlflash/ocxl_hw.c b/drivers/scsi/cxlflash/ocxl_hw.c
index 105712b..44688af 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.c
+++ b/drivers/scsi/cxlflash/ocxl_hw.c
@@ -118,6 +118,9 @@ static int ocxlflash_config_afu(struct pci_dev *pdev, struct ocxl_hw_afu *afu)
dev_dbg(dev, "%s: acTag base=%d enabled=%d\n", __func__, base, count);
afu->afu_actag_base = base;
afu->afu_actag_enabled = count;
+ afu->max_pasid = 1 << acfg->pasid_supported_log;
+
+ ocxl_config_set_afu_pasid(pdev, pos, 0, acfg->pasid_supported_log);
out:
return rc;
}
diff --git a/drivers/scsi/cxlflash/ocxl_hw.h b/drivers/scsi/cxlflash/ocxl_hw.h
index 21803b5..46de75b 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.h
+++ b/drivers/scsi/cxlflash/ocxl_hw.h
@@ -25,5 +25,6 @@ struct ocxl_hw_afu {
int afu_actag_base; /* AFU acTag base */
int afu_actag_enabled; /* AFU acTag number enabled */
+ int max_pasid; /* Maximum number of contexts */
bool is_present; /* Function has AFUs defined */
};
--
2.1.0
^ permalink raw reply related
* [PATCH v3 09/41] cxlflash: Setup AFU acTag range
From: Uma Krishnan @ 2018-03-26 16:31 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522081759-57431-1-git-send-email-ukrishn@linux.vnet.ibm.com>
The OCXL specification supports distributing acTags amongst different
AFUs and functions on the link. As cxlflash devices are expected to only
support a single AFU per function, the entire range that was assigned to
the function is also assigned to the AFU.
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Reviewed-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
---
drivers/scsi/cxlflash/ocxl_hw.c | 13 +++++++++++++
drivers/scsi/cxlflash/ocxl_hw.h | 2 ++
2 files changed, 15 insertions(+)
diff --git a/drivers/scsi/cxlflash/ocxl_hw.c b/drivers/scsi/cxlflash/ocxl_hw.c
index 11399cf..105712b 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.c
+++ b/drivers/scsi/cxlflash/ocxl_hw.c
@@ -92,6 +92,9 @@ static int ocxlflash_config_afu(struct pci_dev *pdev, struct ocxl_hw_afu *afu)
struct ocxl_afu_config *acfg = &afu->acfg;
struct ocxl_fn_config *fcfg = &afu->fcfg;
struct device *dev = &pdev->dev;
+ int count;
+ int base;
+ int pos;
int rc = 0;
/* This HW AFU function does not have any AFUs defined */
@@ -105,6 +108,16 @@ static int ocxlflash_config_afu(struct pci_dev *pdev, struct ocxl_hw_afu *afu)
__func__, rc);
goto out;
}
+
+ /* Only one AFU per function is supported, so actag_base is same */
+ base = afu->fn_actag_base;
+ count = min_t(int, acfg->actag_supported, afu->fn_actag_enabled);
+ pos = acfg->dvsec_afu_control_pos;
+
+ ocxl_config_set_afu_actag(pdev, pos, base, count);
+ dev_dbg(dev, "%s: acTag base=%d enabled=%d\n", __func__, base, count);
+ afu->afu_actag_base = base;
+ afu->afu_actag_enabled = count;
out:
return rc;
}
diff --git a/drivers/scsi/cxlflash/ocxl_hw.h b/drivers/scsi/cxlflash/ocxl_hw.h
index c9b2843..21803b5 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.h
+++ b/drivers/scsi/cxlflash/ocxl_hw.h
@@ -22,6 +22,8 @@ struct ocxl_hw_afu {
int fn_actag_base; /* Function acTag base */
int fn_actag_enabled; /* Function acTag number enabled */
+ int afu_actag_base; /* AFU acTag base */
+ int afu_actag_enabled; /* AFU acTag number enabled */
bool is_present; /* Function has AFUs defined */
};
--
2.1.0
^ permalink raw reply related
* [PATCH v3 08/41] cxlflash: Read host AFU configuration
From: Uma Krishnan @ 2018-03-26 16:31 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522081759-57431-1-git-send-email-ukrishn@linux.vnet.ibm.com>
The host AFU configuration is read on the initialization path to identify
the features and configuration of the AFU. This data is cached for use in
later configuration steps.
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Reviewed-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
---
drivers/scsi/cxlflash/ocxl_hw.c | 38 ++++++++++++++++++++++++++++++++++++++
drivers/scsi/cxlflash/ocxl_hw.h | 1 +
2 files changed, 39 insertions(+)
diff --git a/drivers/scsi/cxlflash/ocxl_hw.c b/drivers/scsi/cxlflash/ocxl_hw.c
index 7783842..11399cf 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.c
+++ b/drivers/scsi/cxlflash/ocxl_hw.c
@@ -79,6 +79,37 @@ static int ocxlflash_config_fn(struct pci_dev *pdev, struct ocxl_hw_afu *afu)
}
/**
+ * ocxlflash_config_afu() - configure the host AFU
+ * @pdev: PCI device associated with the host.
+ * @afu: AFU associated with the host.
+ *
+ * Must be called _after_ host function configuration.
+ *
+ * Return: 0 on success, -errno on failure
+ */
+static int ocxlflash_config_afu(struct pci_dev *pdev, struct ocxl_hw_afu *afu)
+{
+ struct ocxl_afu_config *acfg = &afu->acfg;
+ struct ocxl_fn_config *fcfg = &afu->fcfg;
+ struct device *dev = &pdev->dev;
+ int rc = 0;
+
+ /* This HW AFU function does not have any AFUs defined */
+ if (!afu->is_present)
+ goto out;
+
+ /* Read AFU config at index 0 */
+ rc = ocxl_config_read_afu(pdev, fcfg, acfg, 0);
+ if (unlikely(rc)) {
+ dev_err(dev, "%s: ocxl_config_read_afu failed rc=%d\n",
+ __func__, rc);
+ goto out;
+ }
+out:
+ return rc;
+}
+
+/**
* ocxlflash_create_afu() - create the AFU for OCXL
* @pdev: PCI device associated with the host.
*
@@ -105,6 +136,13 @@ static void *ocxlflash_create_afu(struct pci_dev *pdev)
__func__, rc);
goto err1;
}
+
+ rc = ocxlflash_config_afu(pdev, afu);
+ if (unlikely(rc)) {
+ dev_err(dev, "%s: AFU configuration failed rc=%d\n",
+ __func__, rc);
+ goto err1;
+ }
out:
return afu;
err1:
diff --git a/drivers/scsi/cxlflash/ocxl_hw.h b/drivers/scsi/cxlflash/ocxl_hw.h
index 55e833cb..c9b2843 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.h
+++ b/drivers/scsi/cxlflash/ocxl_hw.h
@@ -18,6 +18,7 @@ struct ocxl_hw_afu {
struct device *dev; /* Generic device */
struct ocxl_fn_config fcfg; /* DVSEC config of the function */
+ struct ocxl_afu_config acfg; /* AFU configuration data */
int fn_actag_base; /* Function acTag base */
int fn_actag_enabled; /* Function acTag number enabled */
--
2.1.0
^ permalink raw reply related
* [PATCH v3 07/41] cxlflash: Setup function acTag range
From: Uma Krishnan @ 2018-03-26 16:31 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522081759-57431-1-git-send-email-ukrishn@linux.vnet.ibm.com>
The OCXL specification supports distributing acTags amongst different
AFUs and functions on the link. The platform-specific acTag range for the
link is obtained using the OCXL provider services and then assigned to the
host function based on implementation.
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Reviewed-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
---
drivers/scsi/cxlflash/ocxl_hw.c | 15 +++++++++++++++
drivers/scsi/cxlflash/ocxl_hw.h | 3 +++
2 files changed, 18 insertions(+)
diff --git a/drivers/scsi/cxlflash/ocxl_hw.c b/drivers/scsi/cxlflash/ocxl_hw.c
index 85c0f84..7783842 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.c
+++ b/drivers/scsi/cxlflash/ocxl_hw.c
@@ -42,6 +42,7 @@ static int ocxlflash_config_fn(struct pci_dev *pdev, struct ocxl_hw_afu *afu)
{
struct ocxl_fn_config *fcfg = &afu->fcfg;
struct device *dev = &pdev->dev;
+ u16 base, enabled, supported;
int rc = 0;
/* Read DVSEC config of the function */
@@ -59,6 +60,20 @@ static int ocxlflash_config_fn(struct pci_dev *pdev, struct ocxl_hw_afu *afu)
dev_warn(dev, "%s: Unexpected AFU index value %d\n",
__func__, fcfg->max_afu_index);
}
+
+ rc = ocxl_config_get_actag_info(pdev, &base, &enabled, &supported);
+ if (unlikely(rc)) {
+ dev_err(dev, "%s: ocxl_config_get_actag_info failed rc=%d\n",
+ __func__, rc);
+ goto out;
+ }
+
+ afu->fn_actag_base = base;
+ afu->fn_actag_enabled = enabled;
+
+ ocxl_config_set_actag(pdev, fcfg->dvsec_function_pos, base, enabled);
+ dev_dbg(dev, "%s: Function acTag range base=%u enabled=%u\n",
+ __func__, base, enabled);
out:
return rc;
}
diff --git a/drivers/scsi/cxlflash/ocxl_hw.h b/drivers/scsi/cxlflash/ocxl_hw.h
index 356a6a3..55e833cb 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.h
+++ b/drivers/scsi/cxlflash/ocxl_hw.h
@@ -19,5 +19,8 @@ struct ocxl_hw_afu {
struct ocxl_fn_config fcfg; /* DVSEC config of the function */
+ int fn_actag_base; /* Function acTag base */
+ int fn_actag_enabled; /* Function acTag number enabled */
+
bool is_present; /* Function has AFUs defined */
};
--
2.1.0
^ permalink raw reply related
* [PATCH v3 06/41] cxlflash: Read host function configuration
From: Uma Krishnan @ 2018-03-26 16:31 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522081759-57431-1-git-send-email-ukrishn@linux.vnet.ibm.com>
Per the OCXL specification, the underlying host can have multiple AFUs
per function with each function supporting its own configuration. The host
function configuration is read on the initialization path to evaluate the
number of functions present and identify the features and configuration of
the functions present. This data is cached for use in later configuration
steps. Note that for the OCXL hardware supported by the cxlflash driver,
only one AFU per function is expected.
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
Reviewed-by: Frederic Barrat <fbarrat@linux.vnet.ibm.com>
---
drivers/scsi/cxlflash/ocxl_hw.c | 44 +++++++++++++++++++++++++++++++++++++++++
drivers/scsi/cxlflash/ocxl_hw.h | 4 ++++
2 files changed, 48 insertions(+)
diff --git a/drivers/scsi/cxlflash/ocxl_hw.c b/drivers/scsi/cxlflash/ocxl_hw.c
index e3a0a9b..85c0f84 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.c
+++ b/drivers/scsi/cxlflash/ocxl_hw.c
@@ -32,6 +32,38 @@ static void ocxlflash_destroy_afu(void *afu_cookie)
}
/**
+ * ocxlflash_config_fn() - configure the host function
+ * @pdev: PCI device associated with the host.
+ * @afu: AFU associated with the host.
+ *
+ * Return: 0 on success, -errno on failure
+ */
+static int ocxlflash_config_fn(struct pci_dev *pdev, struct ocxl_hw_afu *afu)
+{
+ struct ocxl_fn_config *fcfg = &afu->fcfg;
+ struct device *dev = &pdev->dev;
+ int rc = 0;
+
+ /* Read DVSEC config of the function */
+ rc = ocxl_config_read_function(pdev, fcfg);
+ if (unlikely(rc)) {
+ dev_err(dev, "%s: ocxl_config_read_function failed rc=%d\n",
+ __func__, rc);
+ goto out;
+ }
+
+ /* Check if function has AFUs defined, only 1 per function supported */
+ if (fcfg->max_afu_index >= 0) {
+ afu->is_present = true;
+ if (fcfg->max_afu_index != 0)
+ dev_warn(dev, "%s: Unexpected AFU index value %d\n",
+ __func__, fcfg->max_afu_index);
+ }
+out:
+ return rc;
+}
+
+/**
* ocxlflash_create_afu() - create the AFU for OCXL
* @pdev: PCI device associated with the host.
*
@@ -41,6 +73,7 @@ static void *ocxlflash_create_afu(struct pci_dev *pdev)
{
struct device *dev = &pdev->dev;
struct ocxl_hw_afu *afu;
+ int rc;
afu = kzalloc(sizeof(*afu), GFP_KERNEL);
if (unlikely(!afu)) {
@@ -50,8 +83,19 @@ static void *ocxlflash_create_afu(struct pci_dev *pdev)
afu->pdev = pdev;
afu->dev = dev;
+
+ rc = ocxlflash_config_fn(pdev, afu);
+ if (unlikely(rc)) {
+ dev_err(dev, "%s: Function configuration failed rc=%d\n",
+ __func__, rc);
+ goto err1;
+ }
out:
return afu;
+err1:
+ kfree(afu);
+ afu = NULL;
+ goto out;
}
/* Backend ops to ocxlflash services */
diff --git a/drivers/scsi/cxlflash/ocxl_hw.h b/drivers/scsi/cxlflash/ocxl_hw.h
index c7e5c4d..356a6a3 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.h
+++ b/drivers/scsi/cxlflash/ocxl_hw.h
@@ -16,4 +16,8 @@
struct ocxl_hw_afu {
struct pci_dev *pdev; /* PCI device */
struct device *dev; /* Generic device */
+
+ struct ocxl_fn_config fcfg; /* DVSEC config of the function */
+
+ bool is_present; /* Function has AFUs defined */
};
--
2.1.0
^ permalink raw reply related
* [PATCH v3 05/41] cxlflash: Hardware AFU for OCXL
From: Uma Krishnan @ 2018-03-26 16:31 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522081759-57431-1-git-send-email-ukrishn@linux.vnet.ibm.com>
When an adapter is initialized, transport specific configuration and MMIO
mapping details need to be saved. For CXL, this data is managed by the
underlying kernel module. To maintain a separation between the cxlflash
core and underlying transports, introduce a new structure to store data
specific to the OCXL AFU.
Initially only the pointers to underlying PCI and generic devices are
added to this new structure - it will be expanded further in future
commits. Services to create and destroy this hardware AFU are added and
integrated in the probe and exit paths of the driver.
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
---
drivers/scsi/cxlflash/backend.h | 1 +
drivers/scsi/cxlflash/cxl_hw.c | 6 ++++++
drivers/scsi/cxlflash/main.c | 9 +++++++--
drivers/scsi/cxlflash/ocxl_hw.c | 40 ++++++++++++++++++++++++++++++++++++++++
drivers/scsi/cxlflash/ocxl_hw.h | 19 +++++++++++++++++++
5 files changed, 73 insertions(+), 2 deletions(-)
create mode 100644 drivers/scsi/cxlflash/ocxl_hw.h
diff --git a/drivers/scsi/cxlflash/backend.h b/drivers/scsi/cxlflash/backend.h
index a60f051..f675bcb 100644
--- a/drivers/scsi/cxlflash/backend.h
+++ b/drivers/scsi/cxlflash/backend.h
@@ -36,6 +36,7 @@ struct cxlflash_backend_ops {
int (*allocate_afu_irqs)(void *ctx_cookie, int num);
void (*free_afu_irqs)(void *ctx_cookie);
void * (*create_afu)(struct pci_dev *dev);
+ void (*destroy_afu)(void *afu_cookie);
struct file * (*get_fd)(void *ctx_cookie, struct file_operations *fops,
int *fd);
void * (*fops_get_context)(struct file *file);
diff --git a/drivers/scsi/cxlflash/cxl_hw.c b/drivers/scsi/cxlflash/cxl_hw.c
index db1cada..a1d6d12 100644
--- a/drivers/scsi/cxlflash/cxl_hw.c
+++ b/drivers/scsi/cxlflash/cxl_hw.c
@@ -110,6 +110,11 @@ static void *cxlflash_create_afu(struct pci_dev *dev)
return cxl_pci_to_afu(dev);
}
+static void cxlflash_destroy_afu(void *afu)
+{
+ /* Dummy fop for cxl */
+}
+
static struct file *cxlflash_get_fd(void *ctx_cookie,
struct file_operations *fops, int *fd)
{
@@ -160,6 +165,7 @@ const struct cxlflash_backend_ops cxlflash_cxl_ops = {
.allocate_afu_irqs = cxlflash_allocate_afu_irqs,
.free_afu_irqs = cxlflash_free_afu_irqs,
.create_afu = cxlflash_create_afu,
+ .destroy_afu = cxlflash_destroy_afu,
.get_fd = cxlflash_get_fd,
.fops_get_context = cxlflash_fops_get_context,
.start_work = cxlflash_start_work,
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index b83a55a..5d754d1 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -971,6 +971,7 @@ static void cxlflash_remove(struct pci_dev *pdev)
case INIT_STATE_AFU:
term_afu(cfg);
case INIT_STATE_PCI:
+ cfg->ops->destroy_afu(cfg->afu_cookie);
pci_disable_device(pdev);
case INIT_STATE_NONE:
free_mem(cfg);
@@ -3689,8 +3690,6 @@ static int cxlflash_probe(struct pci_dev *pdev,
pci_set_drvdata(pdev, cfg);
- cfg->afu_cookie = cfg->ops->create_afu(pdev);
-
rc = init_pci(cfg);
if (rc) {
dev_err(dev, "%s: init_pci failed rc=%d\n", __func__, rc);
@@ -3698,6 +3697,12 @@ static int cxlflash_probe(struct pci_dev *pdev,
}
cfg->init_state = INIT_STATE_PCI;
+ cfg->afu_cookie = cfg->ops->create_afu(pdev);
+ if (unlikely(!cfg->afu_cookie)) {
+ dev_err(dev, "%s: create_afu failed\n", __func__);
+ goto out_remove;
+ }
+
rc = init_afu(cfg);
if (rc && !wq_has_sleeper(&cfg->reset_waitq)) {
dev_err(dev, "%s: init_afu failed rc=%d\n", __func__, rc);
diff --git a/drivers/scsi/cxlflash/ocxl_hw.c b/drivers/scsi/cxlflash/ocxl_hw.c
index 58a3182..e3a0a9b 100644
--- a/drivers/scsi/cxlflash/ocxl_hw.c
+++ b/drivers/scsi/cxlflash/ocxl_hw.c
@@ -15,8 +15,48 @@
#include <misc/ocxl.h>
#include "backend.h"
+#include "ocxl_hw.h"
+
+/**
+ * ocxlflash_destroy_afu() - destroy the AFU structure
+ * @afu_cookie: AFU to be freed.
+ */
+static void ocxlflash_destroy_afu(void *afu_cookie)
+{
+ struct ocxl_hw_afu *afu = afu_cookie;
+
+ if (!afu)
+ return;
+
+ kfree(afu);
+}
+
+/**
+ * ocxlflash_create_afu() - create the AFU for OCXL
+ * @pdev: PCI device associated with the host.
+ *
+ * Return: AFU on success, NULL on failure
+ */
+static void *ocxlflash_create_afu(struct pci_dev *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct ocxl_hw_afu *afu;
+
+ afu = kzalloc(sizeof(*afu), GFP_KERNEL);
+ if (unlikely(!afu)) {
+ dev_err(dev, "%s: HW AFU allocation failed\n", __func__);
+ goto out;
+ }
+
+ afu->pdev = pdev;
+ afu->dev = dev;
+out:
+ return afu;
+}
/* Backend ops to ocxlflash services */
const struct cxlflash_backend_ops cxlflash_ocxl_ops = {
.module = THIS_MODULE,
+ .create_afu = ocxlflash_create_afu,
+ .destroy_afu = ocxlflash_destroy_afu,
};
diff --git a/drivers/scsi/cxlflash/ocxl_hw.h b/drivers/scsi/cxlflash/ocxl_hw.h
new file mode 100644
index 0000000..c7e5c4d
--- /dev/null
+++ b/drivers/scsi/cxlflash/ocxl_hw.h
@@ -0,0 +1,19 @@
+/*
+ * CXL Flash Device Driver
+ *
+ * Written by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>, IBM Corporation
+ * Uma Krishnan <ukrishn@linux.vnet.ibm.com>, IBM Corporation
+ *
+ * Copyright (C) 2018 IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+/* OCXL hardware AFU associated with the host */
+struct ocxl_hw_afu {
+ struct pci_dev *pdev; /* PCI device */
+ struct device *dev; /* Generic device */
+};
--
2.1.0
^ permalink raw reply related
* [PATCH v3 04/41] cxlflash: Introduce OCXL backend
From: Uma Krishnan @ 2018-03-26 16:30 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522081759-57431-1-git-send-email-ukrishn@linux.vnet.ibm.com>
Add initial infrastructure to support a new cxlflash transport, OCXL.
Claim a dependency on OCXL and add a new file, ocxl_hw.c, which will host
the backend routines that are specific to OCXL.
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Reviewed-by: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
---
drivers/scsi/cxlflash/Kconfig | 2 +-
drivers/scsi/cxlflash/Makefile | 2 +-
drivers/scsi/cxlflash/backend.h | 1 +
drivers/scsi/cxlflash/ocxl_hw.c | 22 ++++++++++++++++++++++
4 files changed, 25 insertions(+), 2 deletions(-)
create mode 100644 drivers/scsi/cxlflash/ocxl_hw.c
diff --git a/drivers/scsi/cxlflash/Kconfig b/drivers/scsi/cxlflash/Kconfig
index a011c5d..e2a3a1b 100644
--- a/drivers/scsi/cxlflash/Kconfig
+++ b/drivers/scsi/cxlflash/Kconfig
@@ -4,7 +4,7 @@
config CXLFLASH
tristate "Support for IBM CAPI Flash"
- depends on PCI && SCSI && CXL && EEH
+ depends on PCI && SCSI && CXL && OCXL && EEH
select IRQ_POLL
default m
help
diff --git a/drivers/scsi/cxlflash/Makefile b/drivers/scsi/cxlflash/Makefile
index 7ec3f6b..5124c68 100644
--- a/drivers/scsi/cxlflash/Makefile
+++ b/drivers/scsi/cxlflash/Makefile
@@ -1,2 +1,2 @@
obj-$(CONFIG_CXLFLASH) += cxlflash.o
-cxlflash-y += main.o superpipe.o lunmgt.o vlun.o cxl_hw.o
+cxlflash-y += main.o superpipe.o lunmgt.o vlun.o cxl_hw.o ocxl_hw.o
diff --git a/drivers/scsi/cxlflash/backend.h b/drivers/scsi/cxlflash/backend.h
index 7b72149..a60f051 100644
--- a/drivers/scsi/cxlflash/backend.h
+++ b/drivers/scsi/cxlflash/backend.h
@@ -13,6 +13,7 @@
*/
extern const struct cxlflash_backend_ops cxlflash_cxl_ops;
+extern const struct cxlflash_backend_ops cxlflash_ocxl_ops;
struct cxlflash_backend_ops {
struct module *module;
diff --git a/drivers/scsi/cxlflash/ocxl_hw.c b/drivers/scsi/cxlflash/ocxl_hw.c
new file mode 100644
index 0000000..58a3182
--- /dev/null
+++ b/drivers/scsi/cxlflash/ocxl_hw.c
@@ -0,0 +1,22 @@
+/*
+ * CXL Flash Device Driver
+ *
+ * Written by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>, IBM Corporation
+ * Uma Krishnan <ukrishn@linux.vnet.ibm.com>, IBM Corporation
+ *
+ * Copyright (C) 2018 IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <misc/ocxl.h>
+
+#include "backend.h"
+
+/* Backend ops to ocxlflash services */
+const struct cxlflash_backend_ops cxlflash_ocxl_ops = {
+ .module = THIS_MODULE,
+};
--
2.1.0
^ permalink raw reply related
* [PATCH v3 03/41] cxlflash: Add argument identifier names
From: Uma Krishnan @ 2018-03-26 16:30 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522081759-57431-1-git-send-email-ukrishn@linux.vnet.ibm.com>
Checkpatch throws a warning when the argument identifier names are not
included in the function definitions.
To avoid these warnings, argument identifiers are added in the existing
function definitions.
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
---
drivers/scsi/cxlflash/backend.h | 47 ++++++++++++++++++++++-------------------
drivers/scsi/cxlflash/common.h | 4 ++--
2 files changed, 27 insertions(+), 24 deletions(-)
diff --git a/drivers/scsi/cxlflash/backend.h b/drivers/scsi/cxlflash/backend.h
index 339e42b..7b72149 100644
--- a/drivers/scsi/cxlflash/backend.h
+++ b/drivers/scsi/cxlflash/backend.h
@@ -16,26 +16,29 @@ extern const struct cxlflash_backend_ops cxlflash_cxl_ops;
struct cxlflash_backend_ops {
struct module *module;
- void __iomem * (*psa_map)(void *);
- void (*psa_unmap)(void __iomem *);
- int (*process_element)(void *);
- int (*map_afu_irq)(void *, int, irq_handler_t, void *, char *);
- void (*unmap_afu_irq)(void *, int, void *);
- int (*start_context)(void *);
- int (*stop_context)(void *);
- int (*afu_reset)(void *);
- void (*set_master)(void *);
- void * (*get_context)(struct pci_dev *, void *);
- void * (*dev_context_init)(struct pci_dev *, void *);
- int (*release_context)(void *);
- void (*perst_reloads_same_image)(void *, bool);
- ssize_t (*read_adapter_vpd)(struct pci_dev *, void *, size_t);
- int (*allocate_afu_irqs)(void *, int);
- void (*free_afu_irqs)(void *);
- void * (*create_afu)(struct pci_dev *);
- struct file * (*get_fd)(void *, struct file_operations *, int *);
- void * (*fops_get_context)(struct file *);
- int (*start_work)(void *, u64);
- int (*fd_mmap)(struct file *, struct vm_area_struct *);
- int (*fd_release)(struct inode *, struct file *);
+ void __iomem * (*psa_map)(void *ctx_cookie);
+ void (*psa_unmap)(void __iomem *addr);
+ int (*process_element)(void *ctx_cookie);
+ int (*map_afu_irq)(void *ctx_cookie, int num, irq_handler_t handler,
+ void *cookie, char *name);
+ void (*unmap_afu_irq)(void *ctx_cookie, int num, void *cookie);
+ int (*start_context)(void *ctx_cookie);
+ int (*stop_context)(void *ctx_cookie);
+ int (*afu_reset)(void *ctx_cookie);
+ void (*set_master)(void *ctx_cookie);
+ void * (*get_context)(struct pci_dev *dev, void *afu_cookie);
+ void * (*dev_context_init)(struct pci_dev *dev, void *afu_cookie);
+ int (*release_context)(void *ctx_cookie);
+ void (*perst_reloads_same_image)(void *afu_cookie, bool image);
+ ssize_t (*read_adapter_vpd)(struct pci_dev *dev, void *buf,
+ size_t count);
+ int (*allocate_afu_irqs)(void *ctx_cookie, int num);
+ void (*free_afu_irqs)(void *ctx_cookie);
+ void * (*create_afu)(struct pci_dev *dev);
+ struct file * (*get_fd)(void *ctx_cookie, struct file_operations *fops,
+ int *fd);
+ void * (*fops_get_context)(struct file *file);
+ int (*start_work)(void *ctx_cookie, u64 irqs);
+ int (*fd_mmap)(struct file *file, struct vm_area_struct *vm);
+ int (*fd_release)(struct inode *inode, struct file *file);
};
diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index 5d7ca01..d7fccea 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -232,8 +232,8 @@ struct hwq {
struct afu {
struct hwq hwqs[CXLFLASH_MAX_HWQS];
- int (*send_cmd)(struct afu *, struct afu_cmd *);
- int (*context_reset)(struct hwq *);
+ int (*send_cmd)(struct afu *afu, struct afu_cmd *cmd);
+ int (*context_reset)(struct hwq *hwq);
/* AFU HW */
struct cxlflash_afu_map __iomem *afu_map; /* entire MMIO map */
--
2.1.0
^ permalink raw reply related
* [PATCH v3 02/41] cxlflash: Avoid clobbering context control register value
From: Uma Krishnan @ 2018-03-26 16:30 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522081759-57431-1-git-send-email-ukrishn@linux.vnet.ibm.com>
From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>
The SISLite specification originally defined the context control
register with a single field of bits to represent the LISN and
also stipulated that the register reset value be 0. The cxlflash
driver took advantage of this when programming the LISN for the
master contexts via an unconditional write - no other bits were
preserved.
When unmap support was added, SISLite was updated to define bit
0 of the context control register as a way for the AFU to notify
the context owner that unmap operations were supported. Thus the
assumptions under which the register is setup changed and the
existing unconditional write is clobbering the unmap state for
master contexts. This is presently not an issue due to the order
in which the context control register is programmed in relation to
the unmap bit being queried but should be addressed to avoid a
future regression in the event this code is moved elsewhere.
To remedy this issue, preserve the bits when programming the LISN
field in the context control register. Since the LISN will now be
programmed using a read value, assert that the initial state of the
LISN field is as described in SISLite (0).
Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
drivers/scsi/cxlflash/main.c | 5 ++++-
drivers/scsi/cxlflash/sislite.h | 1 +
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 3d3e003..b83a55a 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -1303,7 +1303,10 @@ static void afu_err_intr_init(struct afu *afu)
for (i = 0; i < afu->num_hwqs; i++) {
hwq = get_hwq(afu, i);
- writeq_be(SISL_MSI_SYNC_ERROR, &hwq->host_map->ctx_ctrl);
+ reg = readq_be(&hwq->host_map->ctx_ctrl);
+ WARN_ON((reg & SISL_CTX_CTRL_LISN_MASK) != 0);
+ reg |= SISL_MSI_SYNC_ERROR;
+ writeq_be(reg, &hwq->host_map->ctx_ctrl);
writeq_be(SISL_ISTATUS_MASK, &hwq->host_map->intr_mask);
}
}
diff --git a/drivers/scsi/cxlflash/sislite.h b/drivers/scsi/cxlflash/sislite.h
index bedf1ce..d8940f1 100644
--- a/drivers/scsi/cxlflash/sislite.h
+++ b/drivers/scsi/cxlflash/sislite.h
@@ -284,6 +284,7 @@ struct sisl_host_map {
__be64 cmd_room;
__be64 ctx_ctrl; /* least significant byte or b56:63 is LISN# */
#define SISL_CTX_CTRL_UNMAP_SECTOR 0x8000000000000000ULL /* b0 */
+#define SISL_CTX_CTRL_LISN_MASK (0xFFULL)
__be64 mbox_w; /* restricted use */
__be64 sq_start; /* Submission Queue (R/W): write sequence and */
__be64 sq_end; /* inclusion semantics are the same as RRQ */
--
2.1.0
^ permalink raw reply related
* [PATCH v3 01/41] cxlflash: Preserve number of interrupts for master contexts
From: Uma Krishnan @ 2018-03-26 16:29 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
In-Reply-To: <1522081759-57431-1-git-send-email-ukrishn@linux.vnet.ibm.com>
The number of interrupts requested for user contexts are stored in the
context specific structures and utilized to manage the interrupts. For the
master contexts, this number is only used once and therefore not saved.
To prepare for future commits where the number of interrupts will be
required in more than one place, preserve the value in the master context
structure.
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
---
drivers/scsi/cxlflash/common.h | 1 +
drivers/scsi/cxlflash/main.c | 11 ++++++++---
2 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index 102fd26..5d7ca01 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -211,6 +211,7 @@ struct hwq {
struct sisl_ctrl_map __iomem *ctrl_map; /* MC control map */
ctx_hndl_t ctx_hndl; /* master's context handle */
u32 index; /* Index of this hwq */
+ int num_irqs; /* Numer of interrupts requested for context */
struct list_head pending_cmds; /* Commands pending completion */
atomic_t hsq_credits;
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index d8fe7ab8..3d3e003 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -1911,7 +1911,7 @@ static enum undo_level init_intr(struct cxlflash_cfg *cfg,
int rc = 0;
enum undo_level level = UNDO_NOOP;
bool is_primary_hwq = (hwq->index == PRIMARY_HWQ);
- int num_irqs = is_primary_hwq ? 3 : 2;
+ int num_irqs = hwq->num_irqs;
rc = cfg->ops->allocate_afu_irqs(ctx, num_irqs);
if (unlikely(rc)) {
@@ -1965,16 +1965,20 @@ static int init_mc(struct cxlflash_cfg *cfg, u32 index)
struct device *dev = &cfg->dev->dev;
struct hwq *hwq = get_hwq(cfg->afu, index);
int rc = 0;
+ int num_irqs;
enum undo_level level;
hwq->afu = cfg->afu;
hwq->index = index;
INIT_LIST_HEAD(&hwq->pending_cmds);
- if (index == PRIMARY_HWQ)
+ if (index == PRIMARY_HWQ) {
ctx = cfg->ops->get_context(cfg->dev, cfg->afu_cookie);
- else
+ num_irqs = 3;
+ } else {
ctx = cfg->ops->dev_context_init(cfg->dev, cfg->afu_cookie);
+ num_irqs = 2;
+ }
if (IS_ERR_OR_NULL(ctx)) {
rc = -ENOMEM;
goto err1;
@@ -1982,6 +1986,7 @@ static int init_mc(struct cxlflash_cfg *cfg, u32 index)
WARN_ON(hwq->ctx_cookie);
hwq->ctx_cookie = ctx;
+ hwq->num_irqs = num_irqs;
/* Set it up as a master with the CXL */
cfg->ops->set_master(ctx);
--
2.1.0
^ permalink raw reply related
* [PATCH v3 00/41] cxlflash: OCXL transport support and miscellaneous fixes
From: Uma Krishnan @ 2018-03-26 16:29 UTC (permalink / raw)
To: linux-scsi, James Bottomley, Martin K. Petersen, Matthew R. Ochs,
Manoj N. Kumar
Cc: linuxppc-dev, Andrew Donnellan, Frederic Barrat,
Christophe Lombard
This patch series adds OCXL support to the cxlflash driver. With this
support, new devices using the OCXL transport will be supported by the
cxlflash driver along with the existing CXL devices. An effort is made
to keep this transport specific function independent of the existing
core driver that communicates with the AFU.
The first three patches contain a minor fix and staging improvements.
This series is intended for 4.17 and is bisectable.
v3 Changes:
- Addressed comments by Frederic Barrat
- Resolved IDR initialization bug
- Properly identify functions that do not have an AFU defined
- Added 3 new patches to the end of the series which resolve corner cases
v2 Changes:
- Replaced OpenCXL with OCXL in the commit messages and comments
Matthew R. Ochs (1):
cxlflash: Avoid clobbering context control register value
Uma Krishnan (40):
cxlflash: Preserve number of interrupts for master contexts
cxlflash: Add argument identifier names
cxlflash: Introduce OCXL backend
cxlflash: Hardware AFU for OCXL
cxlflash: Read host function configuration
cxlflash: Setup function acTag range
cxlflash: Read host AFU configuration
cxlflash: Setup AFU acTag range
cxlflash: Setup AFU PASID
cxlflash: Adapter context support for OCXL
cxlflash: Use IDR to manage adapter contexts
cxlflash: Support adapter file descriptors for OCXL
cxlflash: Support adapter context discovery
cxlflash: Support image reload policy modification
cxlflash: MMIO map the AFU
cxlflash: Support starting an adapter context
cxlflash: Support process specific mappings
cxlflash: Support AFU state toggling
cxlflash: Support reading adapter VPD data
cxlflash: Setup function OCXL link
cxlflash: Setup OCXL transaction layer
cxlflash: Support process element lifecycle
cxlflash: Support AFU interrupt management
cxlflash: Support AFU interrupt mapping and registration
cxlflash: Support starting user contexts
cxlflash: Support adapter context polling
cxlflash: Support adapter context reading
cxlflash: Support adapter context mmap and release
cxlflash: Support file descriptor mapping
cxlflash: Introduce object handle fop
cxlflash: Setup LISNs for user contexts
cxlflash: Setup LISNs for master contexts
cxlflash: Update synchronous interrupt status bits
cxlflash: Introduce OCXL context state machine
cxlflash: Register for translation errors
cxlflash: Support AFU reset
cxlflash: Enable OCXL operations
cxlflash: Synchronize reset and remove ops
cxlflash: Remove commmands from pending list on timeout
cxlflash: Handle spurious interrupts
drivers/scsi/cxlflash/Kconfig | 2 +-
drivers/scsi/cxlflash/Makefile | 2 +-
drivers/scsi/cxlflash/backend.h | 50 +-
drivers/scsi/cxlflash/common.h | 11 +-
drivers/scsi/cxlflash/cxl_hw.c | 13 +
drivers/scsi/cxlflash/main.c | 86 ++-
drivers/scsi/cxlflash/main.h | 1 +
drivers/scsi/cxlflash/ocxl_hw.c | 1436 +++++++++++++++++++++++++++++++++++++
drivers/scsi/cxlflash/ocxl_hw.h | 77 ++
drivers/scsi/cxlflash/sislite.h | 41 +-
drivers/scsi/cxlflash/superpipe.c | 14 +
11 files changed, 1682 insertions(+), 51 deletions(-)
create mode 100644 drivers/scsi/cxlflash/ocxl_hw.c
create mode 100644 drivers/scsi/cxlflash/ocxl_hw.h
--
2.1.0
^ permalink raw reply
* Re: RFC on writel and writel_relaxed
From: Sinan Kaya @ 2018-03-26 16:00 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Oliver, linux-rdma@vger.kernel.org,
open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)
In-Reply-To: <CAK8P3a2tumcR7mcE3++cofhF-FHzgZg_1OF+GXCZuwGckMOF1Q@mail.gmail.com>
On 3/26/2018 9:43 AM, Arnd Bergmann wrote:
> On Wed, Mar 21, 2018 at 2:58 PM, Sinan Kaya <okaya@codeaurora.org> wrote:
>> On 3/21/2018 8:53 AM, Sinan Kaya wrote:
>>> BTW, I have no idea what compiler barrier does on PPC and if
>>>
>>> wrltel() == compiler barrier() + wrltel_relaxed()
>>>
>>> can be said.
>>
>> this should have been
>>
>> writel_relaxed() == compiler barrier() + __raw_writel()
>
> I don't think anyone clarified this so far, but there are additional differences
> between the two, writel_relaxed() assumes we are talking to a 32-bit
> little-endian
> MMIO register, while __raw_writel() is primarily used for writing into
> memory-type
> regions with no particular byte order. This means:
>
> - writel_relaxed() must perform a byte swap when running on big-endian kernels
> - when used with __packed MMIO pointers, __raw_writel() may turn into a series
> of byte writes, while writel_relaxed() must result in a single 32-bit access.
> - A set if consecutive writel_relaxed() on the same device is issued in program
> order, while __raw_writel() is not ordered. This typically requires
> only a compiler
> barrier, but may also need a CPU barrier (in addition to the
> barriers we use to
> serialize with spinlocks and DMA in writel() but not writel_relaxed()).
Thanks for the great summary. I didn't know that __raw_writel() could get converted
to byte writes.
>
> Arnd
>
--
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
^ permalink raw reply
* [RFC PATCH] powerpc/xmon: Use OPAL_DEBUG to debug srest in OPAL
From: Nicholas Piggin @ 2018-03-26 15:09 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Nicholas Piggin
xmon can be entered via sreset NMI (from a management sreset, or an
NMI IPI), which can interrupt OPAL. Add checks to xmon to see if pc
or sp are within OPAL memory, and if so, then use OPAL_DEBUG to
print the opal stack and return the Linux stack, which can then be
dumped by xmon
The OPAL side of this, with sample xmon output is here:
https://lists.ozlabs.org/pipermail/skiboot/2018-March/010856.html
This could be plumed into the oops printing code as well.
Thanks,
Nick
---
arch/powerpc/include/asm/opal.h | 4 ++++
arch/powerpc/platforms/powernv/opal-wrappers.S | 1 +
arch/powerpc/platforms/powernv/opal.c | 5 +++++
arch/powerpc/xmon/xmon.c | 27 ++++++++++++++++++++++++++
4 files changed, 37 insertions(+)
diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 12e70fb58700..afcc0c5ed5b0 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -27,6 +27,8 @@ extern struct kobject *opal_kobj;
/* /ibm,opal */
extern struct device_node *opal_node;
+bool in_opal_text_heap_stack(u64 address);
+
/* API functions */
int64_t opal_invalid_call(void);
int64_t opal_npu_destroy_context(uint64_t phb_id, uint64_t pid, uint64_t bdf);
@@ -289,6 +291,8 @@ int opal_sensor_group_clear(u32 group_hndl, int token);
s64 opal_signal_system_reset(s32 cpu);
+s64 opal_debug(u32 debug_type, u64 r1);
+
/* Internal functions */
extern int early_init_dt_scan_opal(unsigned long node, const char *uname,
int depth, void *data);
diff --git a/arch/powerpc/platforms/powernv/opal-wrappers.S b/arch/powerpc/platforms/powernv/opal-wrappers.S
index 1b2936ba6040..78b9ae003553 100644
--- a/arch/powerpc/platforms/powernv/opal-wrappers.S
+++ b/arch/powerpc/platforms/powernv/opal-wrappers.S
@@ -323,3 +323,4 @@ OPAL_CALL(opal_sensor_group_clear, OPAL_SENSOR_GROUP_CLEAR);
OPAL_CALL(opal_npu_spa_setup, OPAL_NPU_SPA_SETUP);
OPAL_CALL(opal_npu_spa_clear_cache, OPAL_NPU_SPA_CLEAR_CACHE);
OPAL_CALL(opal_npu_tl_set, OPAL_NPU_TL_SET);
+OPAL_CALL(opal_debug, 167);
diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
index c15182765ff5..0b7ff5fb18f8 100644
--- a/arch/powerpc/platforms/powernv/opal.c
+++ b/arch/powerpc/platforms/powernv/opal.c
@@ -64,6 +64,11 @@ static struct atomic_notifier_head opal_msg_notifier_head[OPAL_MSG_TYPE_MAX];
static uint32_t opal_heartbeat;
static struct task_struct *kopald_tsk;
+bool in_opal_text_heap_stack(u64 address)
+{
+ return (address >= opal.base && address < opal.base + opal.size);
+}
+
void opal_configure_cores(void)
{
u64 reinit_flags = 0;
diff --git a/arch/powerpc/xmon/xmon.c b/arch/powerpc/xmon/xmon.c
index 82e1a3ee6e0f..ade1adcc1ab8 100644
--- a/arch/powerpc/xmon/xmon.c
+++ b/arch/powerpc/xmon/xmon.c
@@ -452,6 +452,15 @@ static inline int unrecoverable_excp(struct pt_regs *regs)
#endif
}
+static bool in_opal(unsigned long addr)
+{
+ if (firmware_has_feature(FW_FEATURE_OPAL))
+ if (in_opal_text_heap_stack(addr))
+ return true;
+
+ return false;
+}
+
static int xmon_core(struct pt_regs *regs, int fromipi)
{
int cmd = 0;
@@ -510,6 +519,9 @@ static int xmon_core(struct pt_regs *regs, int fromipi)
xmon_fault_jmp[cpu] = recurse_jmp;
+ if (in_opal(regs->nip))
+ printf("xmon: cpu 0x%x stopped in OPAL!\n", cpu);
+
bp = NULL;
if ((regs->msr & (MSR_IR|MSR_PR|MSR_64BIT)) == (MSR_IR|MSR_64BIT))
bp = at_breakpoint(regs->nip);
@@ -1484,8 +1496,23 @@ static void xmon_show_stack(unsigned long sp, unsigned long lr,
unsigned long marker;
struct pt_regs regs;
+ if (in_opal(sp)) {
+ struct debug_struct {
+ unsigned long nip;
+ unsigned long r1;
+ unsigned long r1_caller;
+ } db;
+ printf("SP is in OPAL, calling OPAL to dump stack\n");
+ db.nip = cpu_to_be64(pc);
+ db.r1 = cpu_to_be64(sp);
+ opal_debug(1, (unsigned long)&db);
+ sp = be64_to_cpu(db.r1_caller);
+ }
+
while (max_to_print--) {
if (!is_kernel_addr(sp)) {
+ if (in_opal(pc) && in_opal(sp))
+ printf("SP (%lx) is in OPAL\n", sp);
if (sp != 0)
printf("SP (%lx) is in userspace\n", sp);
break;
--
2.16.1
^ permalink raw reply related
* [PATCH] powerpc/powernv/nvram: opal_nvram_write handle unknown OPAL errors
From: Nicholas Piggin @ 2018-03-26 15:02 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Nicholas Piggin
opal_nvram_write currently just assumes success if it encounters an
error other than OPAL_BUSY or OPAL_BUSY_EVENT. Have it return -EIO
on other errors instead.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/platforms/powernv/opal-nvram.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/opal-nvram.c b/arch/powerpc/platforms/powernv/opal-nvram.c
index 9db4398ded5d..13bf625dc3e8 100644
--- a/arch/powerpc/platforms/powernv/opal-nvram.c
+++ b/arch/powerpc/platforms/powernv/opal-nvram.c
@@ -59,6 +59,8 @@ static ssize_t opal_nvram_write(char *buf, size_t count, loff_t *index)
if (rc == OPAL_BUSY_EVENT)
opal_poll_events(NULL);
}
+ if (rc)
+ return -EIO;
*index += count;
return count;
}
--
2.16.1
^ permalink raw reply related
* [PATCH] powerpc/64s: sreset panic if there is no debugger or crash dump handlers
From: Nicholas Piggin @ 2018-03-26 15:01 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Nicholas Piggin
system_reset_exception does most of its own crash handling now,
invoking the debugger or crash dumps if they are registered. If not,
then it goes through to die() to print stack traces, and then is
supposed to panic (according to comments).
However after die() prints oopses, it does its own handling which
doesn't allow system_reset_exception to panic (e.g., it may just
kill the current process). This patch causes sreset exceptions to
return from die after it prints messages but before acting.
This also stops die from invoking the debugger on 0x100 crashes.
system_reset_exception similarly calls the debugger. It had been
thought this was harmless (because if the debugger was disabled,
neither call would fire, and if it was enabled the first call
would return). However in some cases like xmon 'X' command, the
debugger returns 0, which currently causes it to be entered
again (first in system_reset_exception, then in die), which is
confusing.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/kernel/traps.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c
index 1e48d157196a..b0ac57fa5056 100644
--- a/arch/powerpc/kernel/traps.c
+++ b/arch/powerpc/kernel/traps.c
@@ -208,6 +208,12 @@ static void oops_end(unsigned long flags, struct pt_regs *regs,
}
raw_local_irq_restore(flags);
+ /*
+ * system_reset_excption handles debugger, crash dump, panic, for 0x100
+ */
+ if (TRAP(regs) == 0x100)
+ return;
+
crash_fadump(regs, "die oops");
if (kexec_should_crash(current))
@@ -272,8 +278,13 @@ void die(const char *str, struct pt_regs *regs, long err)
{
unsigned long flags;
- if (debugger(regs))
- return;
+ /*
+ * system_reset_excption handles debugger, crash dump, panic, for 0x100
+ */
+ if (TRAP(regs) != 0x100) {
+ if (debugger(regs))
+ return;
+ }
flags = oops_begin(regs);
if (__die(str, regs, err))
--
2.16.1
^ permalink raw reply related
* [PATCH] powerpc/64s: return more carefully from sreset NMI
From: Nicholas Piggin @ 2018-03-26 15:01 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Nicholas Piggin
System Reset, being an NMI, must return more carefully than other
interrupts. It has traditionally returned via the nromal return
from exception path, but that has a number of problems.
- r13 does not get restored if returning to kernel. This is for
interrupts which may cause a context switch, which sreset will
never do. Interrupting OPAL (which uses a different r13) is one
place where this causes breakage.
- It may cause several other problems returning to kernel with
preempt or TIF_EMULATE_STACK_STORE if it hits at the wrong time.
It's safer just to have a simple restore and return, like machine
check which is the other NMI.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
arch/powerpc/kernel/exceptions-64s.S | 61 ++++++++++++++++++++++++++++++++++--
1 file changed, 58 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S
index 3ac87e53b3da..114ff7170562 100644
--- a/arch/powerpc/kernel/exceptions-64s.S
+++ b/arch/powerpc/kernel/exceptions-64s.S
@@ -139,6 +139,21 @@ EXC_COMMON_BEGIN(system_reset_idle_common)
b pnv_powersave_wakeup
#endif
+/*
+ * Set IRQS_ALL_DISABLED unconditionally so arch_irqs_disabled does
+ * the right thing. We do not want to reconcile because that goes
+ * through irq tracing which we don't want in NMI.
+ *
+ * Save PACAIRQHAPPENED because some code will do a hard disable
+ * (e.g., xmon). So we want to restore this back to where it was
+ * when we return. DAR is unused in the stack, so save it there.
+ */
+#define ADD_RECONCILE_NMI \
+ li r10,IRQS_ALL_DISABLED; \
+ stb r10,PACAIRQSOFTMASK(r13); \
+ lbz r10,PACAIRQHAPPENED(r13); \
+ std r10,_DAR(r1)
+
EXC_COMMON_BEGIN(system_reset_common)
/*
* Increment paca->in_nmi then enable MSR_RI. SLB or MCE will be able
@@ -157,16 +172,56 @@ EXC_COMMON_BEGIN(system_reset_common)
subi r1,r1,INT_FRAME_SIZE
EXCEPTION_COMMON_NORET_STACK(PACA_EXNMI, 0x100,
system_reset, system_reset_exception,
- ADD_NVGPRS;ADD_RECONCILE)
+ ADD_NVGPRS;ADD_RECONCILE_NMI)
+
+ /* This (and MCE) can be simplified with mtmsrd L=1 */
+ /* Clear MSR_RI before setting SRR0 and SRR1. */
+ li r0,MSR_RI
+ mfmsr r9
+ andc r9,r9,r0
+ mtmsrd r9,1
/*
- * The stack is no longer in use, decrement in_nmi.
+ * MSR_RI is clear, now we can decrement paca->in_nmi.
*/
lhz r10,PACA_IN_NMI(r13)
subi r10,r10,1
sth r10,PACA_IN_NMI(r13)
- b ret_from_except
+ /*
+ * Restore soft mask settings.
+ */
+ ld r10,_DAR(r1)
+ stb r10,PACAIRQHAPPENED(r13)
+ ld r10,SOFTE(r1)
+ stb r10,PACAIRQSOFTMASK(r13)
+
+ /*
+ * Keep below code in synch with MACHINE_CHECK_HANDLER_WINDUP.
+ * Should share common bits...
+ */
+
+ /* Move original SRR0 and SRR1 into the respective regs */
+ ld r9,_MSR(r1)
+ mtspr SPRN_SRR1,r9
+ ld r3,_NIP(r1)
+ mtspr SPRN_SRR0,r3
+ ld r9,_CTR(r1)
+ mtctr r9
+ ld r9,_XER(r1)
+ mtxer r9
+ ld r9,_LINK(r1)
+ mtlr r9
+ REST_GPR(0, r1)
+ REST_8GPRS(2, r1)
+ REST_GPR(10, r1)
+ ld r11,_CCR(r1)
+ mtcr r11
+ REST_GPR(11, r1)
+ REST_2GPRS(12, r1)
+ /* restore original r1. */
+ ld r1,GPR1(r1)
+ RFI_TO_USER_OR_KERNEL
#ifdef CONFIG_PPC_PSERIES
/*
--
2.16.1
^ permalink raw reply related
* [PATCH -next] powerpc: Fix error return code in ppc4xx_msi_probe()
From: Wei Yongjun @ 2018-03-26 14:43 UTC (permalink / raw)
To: Benjamin Herrenschmidt, Paul Mackerras, Michael Ellerman,
Rob Herring, Markus Elfring, Tyrel Datwyler
Cc: Wei Yongjun, linuxppc-dev, linux-kernel, kernel-janitors
Fix to return a negative error code from the error handling
case instead of 0, as done elsewhere in this function.
Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
---
arch/powerpc/platforms/4xx/msi.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/4xx/msi.c b/arch/powerpc/platforms/4xx/msi.c
index 4b859c8..0b71c52 100644
--- a/arch/powerpc/platforms/4xx/msi.c
+++ b/arch/powerpc/platforms/4xx/msi.c
@@ -241,7 +241,8 @@ static int ppc4xx_msi_probe(struct platform_device *dev)
if (!msi_irqs)
return -ENODEV;
- if (ppc4xx_setup_pcieh_hw(dev, res, msi))
+ err = ppc4xx_setup_pcieh_hw(dev, res, msi)
+ if (err)
goto error_out;
err = ppc4xx_msi_init_allocator(dev, msi);
^ permalink raw reply related
* Re: [PATCH v6 04/12] ima: Introduce is_ima_sig()
From: Mimi Zohar @ 2018-03-26 14:24 UTC (permalink / raw)
To: Thiago Jung Bauermann, linux-integrity
Cc: linux-security-module, keyrings, linux-crypto, linuxppc-dev,
linux-kernel, Dmitry Kasatkin, James Morris, Serge E. Hallyn,
David Howells, David Woodhouse, Jessica Yu, Herbert Xu,
David S. Miller, AKASHI, Takahiro
In-Reply-To: <20180316203837.10174-5-bauerman@linux.vnet.ibm.com>
On Fri, 2018-03-16 at 17:38 -0300, Thiago Jung Bauermann wrote:
> With the introduction of another IMA signature type (modsig), some places
> will need to check for both of them. It is cleaner to do that if there's a
> helper function to tell whether an xattr_value represents an IMA
> signature.
Initially the function name "is_ima_sig" is fine, since it reflects
the 'imasig' type. Having a more generic function name would be
better when adding 'modsig' support. As long as the function is
locally define, we can drop 'ima' from the name. Perhaps something
like has_signature or is_signed() would be preferable.
Mimi
>
> Suggested-by: Mimi Zohar <zohar@linux.vnet.ibm.com>
> Signed-off-by: Thiago Jung Bauermann <bauerman@linux.vnet.ibm.com>
> ---
> security/integrity/ima/ima.h | 5 +++++
> security/integrity/ima/ima_appraise.c | 7 +++----
> security/integrity/ima/ima_template_lib.c | 2 +-
> 3 files changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/security/integrity/ima/ima.h b/security/integrity/ima/ima.h
> index 35fe91aa1fc9..4bafa6a97967 100644
> --- a/security/integrity/ima/ima.h
> +++ b/security/integrity/ima/ima.h
> @@ -155,6 +155,11 @@ unsigned long ima_get_binary_runtime_size(void);
> int ima_init_template(void);
> void ima_init_template_list(void);
>
> +static inline bool is_ima_sig(const struct evm_ima_xattr_data *xattr_value)
> +{
> + return xattr_value && xattr_value->type == EVM_IMA_XATTR_DIGSIG;
> +}
> +
> /*
> * used to protect h_table and sha_table
> */
> diff --git a/security/integrity/ima/ima_appraise.c b/security/integrity/ima/ima_appraise.c
> index a6b2995b7d0b..01172eab297b 100644
> --- a/security/integrity/ima/ima_appraise.c
> +++ b/security/integrity/ima/ima_appraise.c
> @@ -325,15 +325,14 @@ int ima_appraise_measurement(enum ima_hooks func,
> } else if (status != INTEGRITY_PASS) {
> /* Fix mode, but don't replace file signatures. */
> if ((ima_appraise & IMA_APPRAISE_FIX) &&
> - (!xattr_value ||
> - xattr_value->type != EVM_IMA_XATTR_DIGSIG)) {
> + !is_ima_sig(xattr_value)) {
> if (!ima_fix_xattr(dentry, iint))
> status = INTEGRITY_PASS;
> }
>
> /* Permit new files with file signatures, but without data. */
> if (inode->i_size == 0 && iint->flags & IMA_NEW_FILE &&
> - xattr_value && xattr_value->type == EVM_IMA_XATTR_DIGSIG) {
> + is_ima_sig(xattr_value)) {
> status = INTEGRITY_PASS;
> }
>
> @@ -448,7 +447,7 @@ int ima_inode_setxattr(struct dentry *dentry, const char *xattr_name,
> if (!xattr_value_len || (xvalue->type >= IMA_XATTR_LAST))
> return -EINVAL;
> ima_reset_appraise_flags(d_backing_inode(dentry),
> - xvalue->type == EVM_IMA_XATTR_DIGSIG);
> + is_ima_sig(xvalue));
> result = 0;
> }
> return result;
> diff --git a/security/integrity/ima/ima_template_lib.c b/security/integrity/ima/ima_template_lib.c
> index 5afaa53decc5..afb52a90e532 100644
> --- a/security/integrity/ima/ima_template_lib.c
> +++ b/security/integrity/ima/ima_template_lib.c
> @@ -380,7 +380,7 @@ int ima_eventsig_init(struct ima_event_data *event_data,
> {
> struct evm_ima_xattr_data *xattr_value = event_data->xattr_value;
>
> - if ((!xattr_value) || (xattr_value->type != EVM_IMA_XATTR_DIGSIG))
> + if (!is_ima_sig(xattr_value))
> return 0;
>
> return ima_write_template_field_data(xattr_value, event_data->xattr_len,
>
^ permalink raw reply
* Re: RFC on writel and writel_relaxed
From: Arnd Bergmann @ 2018-03-26 13:43 UTC (permalink / raw)
To: Sinan Kaya
Cc: Oliver, linux-rdma@vger.kernel.org,
open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)
In-Reply-To: <2d2a744a-57b2-ca42-db40-4705df0375db@codeaurora.org>
On Wed, Mar 21, 2018 at 2:58 PM, Sinan Kaya <okaya@codeaurora.org> wrote:
> On 3/21/2018 8:53 AM, Sinan Kaya wrote:
>> BTW, I have no idea what compiler barrier does on PPC and if
>>
>> wrltel() == compiler barrier() + wrltel_relaxed()
>>
>> can be said.
>
> this should have been
>
> writel_relaxed() == compiler barrier() + __raw_writel()
I don't think anyone clarified this so far, but there are additional differences
between the two, writel_relaxed() assumes we are talking to a 32-bit
little-endian
MMIO register, while __raw_writel() is primarily used for writing into
memory-type
regions with no particular byte order. This means:
- writel_relaxed() must perform a byte swap when running on big-endian kernels
- when used with __packed MMIO pointers, __raw_writel() may turn into a series
of byte writes, while writel_relaxed() must result in a single 32-bit access.
- A set if consecutive writel_relaxed() on the same device is issued in program
order, while __raw_writel() is not ordered. This typically requires
only a compiler
barrier, but may also need a CPU barrier (in addition to the
barriers we use to
serialize with spinlocks and DMA in writel() but not writel_relaxed()).
Arnd
^ permalink raw reply
* Re: DMA Mapping Error in ppc64
From: Jared Bents @ 2018-03-26 13:10 UTC (permalink / raw)
To: Oliver; +Cc: linuxppc-dev
In-Reply-To: <CAOSf1CGb7HCuhLgyMo4EdiMkmMM0CPD-=QDQ+2S=wgZ43G8w5A@mail.gmail.com>
Hi Ben
On Sat, Mar 24, 2018 at 3:19 AM, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> On Fri, 2018-03-23 at 07:41 -0500, Jared Bents wrote:
>> Thank you for the advice. Looks like I get to try to rewrite the ath9k and ath10k drivers to use dma_alloc_coherent() instead of kmemdup() and dev_alloc_skb()
>
> Euh no... dev_alloc_skb() is the right thing to do for receive
> packets for a device driver.
>
> The arch should be able to map that for DMA, even if include
> bounce buffers via swiotlb.
>
> Cheers,
> Ben.
I have fixed the kmemdup usage to be dma_alloc_coherent() in the ath10k driver.
While dev_alloc_skb() is the right thing to do for receive packets,
the dma_map_single for all of those buffers fails. So it looks like I
have to add the ifdef conditional from
arch/powerpc/platforms/85xx/corenet_generic.c to struct sk_buff
*__netdev_alloc_skb() in net/core/skbuff.c
#if defined(CONFIG_FSL_PCI) && defined(CONFIG_ZONE_DMA32)
gfp_mask |= GFP_DMA32;
#endif
On Sun, Mar 25, 2018 at 6:27 PM, Oliver <oohall@gmail.com> wrote:
> On Fri, Mar 23, 2018 at 11:41 PM, Jared Bents
> <jared.bents@rockwellcollins.com> wrote:
>> Thank you for the advice. Looks like I get to try to rewrite the ath9k and
>> ath10k drivers to use dma_alloc_coherent() instead of kmemdup() and
>> dev_alloc_skb()
>
> I don't think you need to go that far. It looks like you might be able
> to fix the uses of kmemdup() and kzalloc() in
> ath10k_pci_hif_exchange_bmi_msg() and call it a day. Auditing the
> other uses of dma_map_single() to see if they're using kmalloc()
> memory might be a good idea too.
>
> Anyway this is probably something you're better off taking to the ath10k list.
>
> Thanks,
> Oliver
>
I'll take my update of kmemdup to dma_alloc_coherent() to the ath10k
mailing list. However, even after updating to use
dma_alloc_coherent() and adding the conditional to
__netdev_alloc_skb() for the rx skb's used in the driver, I am still
getting a transmit error. I'm struggling to track down where in the
kernel the skb being taken from a queue is coming from in
drivers/net/wireless/ath/ath10k/mac.c
I will ask ath10k about this as well. The skb being taken off the
queue below is later DMAed with dma_map_single and that fails but
since I haven't figured out where it comes from, I haven't been able
to try to fix it.
void ath10k_offchan_tx_work(struct work_struct *work)
{
>.......struct ath10k *ar = container_of(work, struct ath10k, offchan_tx_work);
[...]
>.......for (;;) {
>.......>.......skb = skb_dequeue(&ar->offchan_tx_queue);
Thank you for all the help,
Jared
>>
>> On Thu, Mar 22, 2018 at 8:19 PM, Oliver <oohall@gmail.com> wrote:
>>>
>>> On Fri, Mar 23, 2018 at 1:37 AM, Jared Bents
>>> <jared.bents@rockwellcollins.com> wrote:
>>> > Thank you for the response but unfortunately, it looks like I already
>>> > have that and it is being used. To verify, I commented that out and
>>> > got the failure "dma_direct_alloc_coherent: No suitable zone for pfn
>>> > 0xe0000". Below is the code flow for function
>>> > ath10k_pci_hif_exchange_bmi_msg which is showing the first dma mapping
>>> > error.
>>> >
>>> > ath10k_pci_hif_exchange_bmi_msg -> dma_map_single ->
>>> > dma_map_single_attrs -> swiotlb_map_page -> dma_capable (returns
>>> > false)
>>> >
>>> >
>>> > dma_capable is what reports the failure in that flow.
>>> >
>>> > static inline bool dma_capable(struct device *dev, dma_addr_t addr,
>>> > size_t size)
>>> > {
>>> > #ifdef CONFIG_SWIOTLB
>>> > struct dev_archdata *sd = &dev->archdata;
>>> >
>>> > if (sd->max_direct_dma_addr && addr + size > sd->max_direct_dma_addr)
>>> > return false;
>>> > #endif
>>> >
>>> > if (!dev->dma_mask)
>>> > return false;
>>> >
>>> > return addr + size - 1 <= *dev->dma_mask;
>>> > }
>>> > Getting the below values:
>>> > addr = 1ee376218
>>> > size = 4
>>> > sd->max_direct_dma_addr = e0000000 which is I believe DMA window size
>>> > (e0000000)
>>> >
>>> > when executed sd->max_direct_dma_addr(e0000000) && addr(1ee376218) +
>>> > size(4) becomes e0000004 which is > sd->max_direct_dma_addr (e0000000)
>>> >
>>> >
>>> > So even though limit_zone_pfn(ZONE_DMA32, 1UL << (31 - PAGE_SHIFT)) is
>>> > being used in arch/powerpc/platforms/85xx/corenet_generic.c,
>>>
>>> > kmemdup(req, req_len, GFP_KERNEL) is returning an address that when
>>> > sent to dma_map_single(), results in a bad map.
>>>
>>> You need to use (GFP_KERNEL | GFP_DMA32) to constrain the allocations
>>> to ZONE_DMA32. Without that the kmemdup() will allocate from any zone
>>> so you'll probably get an unmappable address.
>>>
>>> That said, the driver probably shouldn't be using kmemdup() here.
>>> DMA-API.txt pretty explicitly says that drivers should not assume that
>>> dma_map_single() will work with arbitrary memory. It should be using
>>> dma_alloc_coherent() or a dma pool here.
>>>
>>> > - Jared
>>> >
>>> > On Wed, Mar 21, 2018 at 11:54 PM, Oliver <oohall@gmail.com> wrote:
>>> >> On Thu, Mar 22, 2018 at 8:00 AM, Jared Bents
>>> >> <jared.bents@rockwellcollins.com> wrote:
>>> >>> Hi all,
>>> >>>
>>> >>> Apologies for the amount of information but we've been debugging this
>>> >>> for a while and I wanted to get what we are seeing captured as much as
>>> >>> possible. We are a T1042 processor and have a total 8GB DDR and our
>>> >>> kernel version is fsl-sdk-v2.0-1703 (linux v4.1.35) as that is the
>>> >>> latest version supplied by NXP.
>>> >>>
>>> >>> A while ago we ported from 32 bit to 64 bit. Everything continued to
>>> >>> work except the ath10k module we have. So as a first step, we checked
>>> >>> to see if an ath9k module also failed to work and it was also no
>>> >>> longer working. The ath10k is working fine on a 32 bit system but
>>> >>> it's not working on 64 bit system as we are getting dma mapping errors
>>> >>> when trying to initialize the wifi modules.
>>> >>>
>>> >>> pci_bus 0002:01: bus scan returning with max=01
>>> >>> pci_bus 0002:01: busn_res: [bus 01] end is updated to 01
>>> >>> pci_bus 0002:00: bus scan returning with max=01
>>> >>> ath10k_pci 0000:01:00.0: unable to get target info from device
>>> >>> ath10k_pci 0000:01:00.0: could not get target info (-5)
>>> >>> ath10k_pci 0000:01:00.0: could not probe fw (-5)
>>> >>> ath10k_pci 0001:01:00.0: Direct firmware load for
>>> >>> ath10k/cal-pci-0001:01:00.0.bin failed with error -2
>>> >>>
>>> >>>
>>> >>> First, we have tried the mainline kernel (v4.15) to see if that would
>>> >>> fix the issue, it did not. So I made a patch for the ath10k driver to
>>> >>> restrict to just GFP_DMA areas when allocating memory or creating
>>> >>> sk_buffs and have attached it. The ath10k wifi modules now initialize
>>> >>> correctly but when I try to connect them and send traffic, they get a
>>> >>> DMA mapping error from the sk_buff that it receives from elsewhere in
>>> >>> the kernel. So while the driver appears to be fixable with the patch,
>>> >>> the modules are still unusable due to data being sent to the driver
>>> >>> when ath10k_tx is called and it tries to dma map with the provided
>>> >>> skb. Also, according to the ath10k mailing list, GFP_DMA is not
>>> >>> supposed to be used in general. The error below is the same sort of
>>> >>> dma mapping error that is seen when initializing the modules without
>>> >>> the patch to OR with GFP_DMA.
>>> >>>
>>> >>> ath10k_pci 0001:01:00.0: failed to transmit packet, dropping: -5
>>> >>>
>>> >>>
>>> >>> We asked on the ath10k mailing list if anyone else is having this
>>> >>> problem and no one else seems to have the issue but they are using
>>> >>> different architectures (ARM or X86). As a result, it does not seem to
>>> >>> be a driver issue to us but something within the PowerPC arch. So we
>>> >>> dug a little deeper to try to find what addresses being mapped are
>>> >>> working and what address being mapped are not working.
>>> >>>
>>> >>> We found that when the virtual address of data pointer (a member of
>>> >>> sk_buff) is above ~3.7 GB RAM address range then return address from
>>> >>> dma_map_single API is failed to validate in dma_mapping_error
>>> >>> function.
>>> >>>
>>> >>> We also noticed that in a 64bit machine sometimes ping is working and
>>> >>> because of the virtual address is under ~3.7GAM RAM address range. So
>>> >>> if we set mem=2048M in the bootargs, the ath10k module works
>>> >>> perfectly, however this isn't a real solution since it cuts our
>>> >>> available RAM from 8GB to 2GB.
>>> >>
>>> >> I think there's a known issue with the freescale PCIe root complex
>>> >> where it can't DMA beyond the 4GB mark. There's a workaround in
>>> >> the form of limit_zone_pfn() which you can use to put the lower 4GB
>>> >> into
>>> >> ZONE_DMA32 and allocate from there rather than ZONE_NORMAL.
>>> >> For details of how to use it have a look at corenet_gen_setup_arch() in
>>> >> arch/powerpc/platforms/85xx/corenet_generic.c
>>> >>
>>> >> Hope that helps,
>>> >> Oliver
>>
>>
^ permalink raw reply
* Re: [PATCH v6 11/12] ima: Implement support for module-style appended signatures
From: Mimi Zohar @ 2018-03-26 12:56 UTC (permalink / raw)
To: Thiago Jung Bauermann, linux-integrity
Cc: linux-security-module, keyrings, linux-crypto, linuxppc-dev,
linux-kernel, Dmitry Kasatkin, James Morris, Serge E. Hallyn,
David Howells, David Woodhouse, Jessica Yu, Herbert Xu,
David S. Miller, AKASHI, Takahiro
In-Reply-To: <20180316203837.10174-12-bauerman@linux.vnet.ibm.com>
On Fri, 2018-03-16 at 17:38 -0300, Thiago Jung Bauermann wrote:
> This patch actually implements the appraise_type=imasig|modsig option,
> allowing IMA to read and verify modsig signatures.
>
> In case both are present in the same file, IMA will first check whether the
> key used by the xattr signature is present in the kernel keyring. If not,
> it will try the appended signature.
Yes, this sounds right.
>
> Signed-off-by: Thiago Jung Bauermann <bauerman@linux.vnet.ibm.com>
> ---
> security/integrity/ima/ima.h | 11 +++++++-
> security/integrity/ima/ima_appraise.c | 53 +++++++++++++++++++++++++++++++----
> security/integrity/ima/ima_main.c | 21 +++++++++++---
> 3 files changed, 74 insertions(+), 11 deletions(-)
>
> diff --git a/security/integrity/ima/ima.h b/security/integrity/ima/ima.h
> index 49aef56dc96d..c11ccb7c5bfb 100644
> --- a/security/integrity/ima/ima.h
> +++ b/security/integrity/ima/ima.h
> @@ -157,7 +157,8 @@ void ima_init_template_list(void);
>
> static inline bool is_ima_sig(const struct evm_ima_xattr_data *xattr_value)
> {
> - return xattr_value && xattr_value->type == EVM_IMA_XATTR_DIGSIG;
> + return xattr_value && (xattr_value->type == EVM_IMA_XATTR_DIGSIG ||
> + xattr_value->type == IMA_MODSIG);
> }
>
> /*
> @@ -253,6 +254,8 @@ enum integrity_status ima_get_cache_status(struct integrity_iint_cache *iint,
> enum ima_hooks func);
> enum hash_algo ima_get_hash_algo(struct evm_ima_xattr_data *xattr_value,
> int xattr_len);
> +bool ima_xattr_sig_known_key(const struct evm_ima_xattr_data *xattr_value,
> + int xattr_len);
> int ima_read_xattr(struct dentry *dentry,
> struct evm_ima_xattr_data **xattr_value);
>
> @@ -291,6 +294,12 @@ ima_get_hash_algo(struct evm_ima_xattr_data *xattr_value, int xattr_len)
> return ima_hash_algo;
> }
>
> +static inline bool ima_xattr_sig_known_key(const struct evm_ima_xattr_data
> + *xattr_value, int xattr_len)
> +{
> + return false;
> +}
> +
> static inline int ima_read_xattr(struct dentry *dentry,
> struct evm_ima_xattr_data **xattr_value)
> {
> diff --git a/security/integrity/ima/ima_appraise.c b/security/integrity/ima/ima_appraise.c
> index 01172eab297b..84e0fd5a19c8 100644
> --- a/security/integrity/ima/ima_appraise.c
> +++ b/security/integrity/ima/ima_appraise.c
> @@ -189,6 +189,22 @@ enum hash_algo ima_get_hash_algo(struct evm_ima_xattr_data *xattr_value,
> return ima_hash_algo;
> }
>
> +bool ima_xattr_sig_known_key(const struct evm_ima_xattr_data *xattr_value,
> + int xattr_len)
> +{
> + struct key *keyring;
> +
> + if (xattr_value->type != EVM_IMA_XATTR_DIGSIG)
> + return false;
> +
> + keyring = integrity_keyring_from_id(INTEGRITY_KEYRING_IMA);
> + if (IS_ERR(keyring))
> + return false;
> +
> + return asymmetric_sig_has_known_key(keyring, (const char *) xattr_value,
> + xattr_len);
> +}
> +
> int ima_read_xattr(struct dentry *dentry,
> struct evm_ima_xattr_data **xattr_value)
> {
> @@ -221,8 +237,12 @@ int ima_appraise_measurement(enum ima_hooks func,
> struct inode *inode = d_backing_inode(dentry);
> enum integrity_status status = INTEGRITY_UNKNOWN;
> int rc = xattr_len, hash_start = 0;
> + size_t xattr_contents_len;
> + void *xattr_contents;
>
> - if (!(inode->i_opflags & IOP_XATTR))
> + /* If not appraising a modsig, we need an xattr. */
> + if ((xattr_value == NULL || xattr_value->type != IMA_MODSIG) &&
> + !(inode->i_opflags & IOP_XATTR))
> return INTEGRITY_UNKNOWN;
>
> if (rc <= 0) {
> @@ -241,13 +261,29 @@ int ima_appraise_measurement(enum ima_hooks func,
> goto out;
> }
>
> - status = evm_verifyxattr(dentry, XATTR_NAME_IMA, xattr_value, rc, iint);
> + /*
> + * If it's a modsig, we don't have the xattr contents to pass to
> + * evm_verifyxattr().
> + */
> + if (xattr_value->type == IMA_MODSIG) {
> + xattr_contents = NULL;
> + xattr_contents_len = 0;
> + } else {
> + xattr_contents = xattr_value;
> + xattr_contents_len = xattr_len;
> + }
> +
> + status = evm_verifyxattr(dentry, XATTR_NAME_IMA, xattr_contents,
> + xattr_contents_len, iint);
> switch (status) {
> case INTEGRITY_PASS:
> case INTEGRITY_PASS_IMMUTABLE:
> case INTEGRITY_UNKNOWN:
> break;
> case INTEGRITY_NOXATTRS: /* No EVM protected xattrs. */
> + /* It's fine not to have xattrs when using a modsig. */
> + if (xattr_value->type == IMA_MODSIG)
> + break;
> case INTEGRITY_NOLABEL: /* No security.evm xattr. */
> cause = "missing-HMAC";
> goto out;
> @@ -288,11 +324,16 @@ int ima_appraise_measurement(enum ima_hooks func,
> status = INTEGRITY_PASS;
> break;
> case EVM_IMA_XATTR_DIGSIG:
> + case IMA_MODSIG:
> set_bit(IMA_DIGSIG, &iint->atomic_flags);
> - rc = integrity_digsig_verify(INTEGRITY_KEYRING_IMA,
> - (const char *)xattr_value, rc,
> - iint->ima_hash->digest,
> - iint->ima_hash->length);
> + if (xattr_value->type == EVM_IMA_XATTR_DIGSIG)
> + rc = integrity_digsig_verify(INTEGRITY_KEYRING_IMA,
> + (const char *)xattr_value,
> + rc, iint->ima_hash->digest,
> + iint->ima_hash->length);
> + else
> + rc = ima_modsig_verify(INTEGRITY_KEYRING_IMA,
> + xattr_value);
> if (rc == -EOPNOTSUPP) {
> status = INTEGRITY_UNKNOWN;
> } else if (rc) {
> diff --git a/security/integrity/ima/ima_main.c b/security/integrity/ima/ima_main.c
> index 5d122daf5c8a..1b11c10f09df 100644
> --- a/security/integrity/ima/ima_main.c
> +++ b/security/integrity/ima/ima_main.c
> @@ -183,7 +183,7 @@ static int process_measurement(struct file *file, const struct cred *cred,
> struct evm_ima_xattr_data *xattr_value = NULL;
> int xattr_len = 0;
> bool violation_check;
> - enum hash_algo hash_algo;
> + enum hash_algo hash_algo = HASH_ALGO__LAST;
>
> if (!ima_policy_flag || !S_ISREG(inode->i_mode))
> return 0;
> @@ -277,11 +277,24 @@ static int process_measurement(struct file *file, const struct cred *cred,
>
> template_desc = ima_template_desc_current();
> if ((action & IMA_APPRAISE_SUBMASK) ||
> - strcmp(template_desc->name, IMA_TEMPLATE_IMA_NAME) != 0)
> + strcmp(template_desc->name, IMA_TEMPLATE_IMA_NAME) != 0) {
> /* read 'security.ima' */
> xattr_len = ima_read_xattr(file_dentry(file), &xattr_value);
> + if (iint->flags & IMA_MODSIG_ALLOWED &&
> + (xattr_len <= 0 || !ima_xattr_sig_known_key(xattr_value,
> + xattr_len))) {
> + /*
> + * Even if we end up using a modsig, hash_algo should
> + * come from the xattr (or even the default hash algo).
> + */
> + hash_algo = ima_get_hash_algo(xattr_value, xattr_len);
> + ima_read_modsig(func, buf, size, &xattr_value,
> + &xattr_len);
> + }
> + }
>
> - hash_algo = ima_get_hash_algo(xattr_value, xattr_len);
> + if (hash_algo == HASH_ALGO__LAST)
> + hash_algo = ima_get_hash_algo(xattr_value, xattr_len);
Previous versions needed to calculate the file hash based on the
modsig hash algorithm. With the introduction of the digest signature
template field ('d-sig'), the file digest field ('d-ng') is always
calculated based on either the xattr hash algorithm, if one exists, or
the IMA default hash algorithm.
Mimi
>
> rc = ima_collect_measurement(iint, file, buf, size, hash_algo);
> if (rc != 0 && rc != -EBADF && rc != -EINVAL)
> @@ -309,7 +322,7 @@ static int process_measurement(struct file *file, const struct cred *cred,
> !(iint->flags & IMA_NEW_FILE))
> rc = -EACCES;
> mutex_unlock(&iint->mutex);
> - kfree(xattr_value);
> + ima_free_xattr_data(xattr_value);
> out:
> if (pathbuf)
> __putname(pathbuf);
>
^ permalink raw reply
* Re: [PATCH 2/2] smp: introduce kick_active_cpus_sync()
From: Paul E. McKenney @ 2018-03-26 12:45 UTC (permalink / raw)
To: Yury Norov
Cc: Chris Metcalf, Christopher Lameter, Russell King - ARM Linux,
Mark Rutland, Steven Rostedt, Mathieu Desnoyers, Catalin Marinas,
Will Deacon, Pekka Enberg, David Rientjes, Joonsoo Kim,
Andrew Morton, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, linux-arm-kernel, linuxppc-dev, kvm-ppc,
linux-mm, linux-kernel, luto
In-Reply-To: <20180325201154.icdcyl4nw2jootqq@yury-thinkpad>
On Sun, Mar 25, 2018 at 11:11:54PM +0300, Yury Norov wrote:
> On Sun, Mar 25, 2018 at 12:23:28PM -0700, Paul E. McKenney wrote:
> > On Sun, Mar 25, 2018 at 08:50:04PM +0300, Yury Norov wrote:
> > > kick_all_cpus_sync() forces all CPUs to sync caches by sending broadcast IPI.
> > > If CPU is in extended quiescent state (idle task or nohz_full userspace), this
> > > work may be done at the exit of this state. Delaying synchronization helps to
> > > save power if CPU is in idle state and decrease latency for real-time tasks.
> > >
> > > This patch introduces kick_active_cpus_sync() and uses it in mm/slab and arm64
> > > code to delay syncronization.
> > >
> > > For task isolation (https://lkml.org/lkml/2017/11/3/589), IPI to the CPU running
> > > isolated task would be fatal, as it breaks isolation. The approach with delaying
> > > of synchronization work helps to maintain isolated state.
> > >
> > > I've tested it with test from task isolation series on ThunderX2 for more than
> > > 10 hours (10k giga-ticks) without breaking isolation.
> > >
> > > Signed-off-by: Yury Norov <ynorov@caviumnetworks.com>
> > > ---
> > > arch/arm64/kernel/insn.c | 2 +-
> > > include/linux/smp.h | 2 ++
> > > kernel/smp.c | 24 ++++++++++++++++++++++++
> > > mm/slab.c | 2 +-
> > > 4 files changed, 28 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/arch/arm64/kernel/insn.c b/arch/arm64/kernel/insn.c
> > > index 2718a77da165..9d7c492e920e 100644
> > > --- a/arch/arm64/kernel/insn.c
> > > +++ b/arch/arm64/kernel/insn.c
> > > @@ -291,7 +291,7 @@ int __kprobes aarch64_insn_patch_text(void *addrs[], u32 insns[], int cnt)
> > > * synchronization.
> > > */
> > > ret = aarch64_insn_patch_text_nosync(addrs[0], insns[0]);
> > > - kick_all_cpus_sync();
> > > + kick_active_cpus_sync();
> > > return ret;
> > > }
> > > }
> > > diff --git a/include/linux/smp.h b/include/linux/smp.h
> > > index 9fb239e12b82..27215e22240d 100644
> > > --- a/include/linux/smp.h
> > > +++ b/include/linux/smp.h
> > > @@ -105,6 +105,7 @@ int smp_call_function_any(const struct cpumask *mask,
> > > smp_call_func_t func, void *info, int wait);
> > >
> > > void kick_all_cpus_sync(void);
> > > +void kick_active_cpus_sync(void);
> > > void wake_up_all_idle_cpus(void);
> > >
> > > /*
> > > @@ -161,6 +162,7 @@ smp_call_function_any(const struct cpumask *mask, smp_call_func_t func,
> > > }
> > >
> > > static inline void kick_all_cpus_sync(void) { }
> > > +static inline void kick_active_cpus_sync(void) { }
> > > static inline void wake_up_all_idle_cpus(void) { }
> > >
> > > #ifdef CONFIG_UP_LATE_INIT
> > > diff --git a/kernel/smp.c b/kernel/smp.c
> > > index 084c8b3a2681..0358d6673850 100644
> > > --- a/kernel/smp.c
> > > +++ b/kernel/smp.c
> > > @@ -724,6 +724,30 @@ void kick_all_cpus_sync(void)
> > > }
> > > EXPORT_SYMBOL_GPL(kick_all_cpus_sync);
> > >
> > > +/**
> > > + * kick_active_cpus_sync - Force CPUs that are not in extended
> > > + * quiescent state (idle or nohz_full userspace) sync by sending
> > > + * IPI. Extended quiescent state CPUs will sync at the exit of
> > > + * that state.
> > > + */
> > > +void kick_active_cpus_sync(void)
> > > +{
> > > + int cpu;
> > > + struct cpumask kernel_cpus;
> > > +
> > > + smp_mb();
> > > +
> > > + cpumask_clear(&kernel_cpus);
> > > + preempt_disable();
> > > + for_each_online_cpu(cpu) {
> > > + if (!rcu_eqs_special_set(cpu))
> >
> > If we get here, the CPU is not in a quiescent state, so we therefore
> > must IPI it, correct?
> >
> > But don't you also need to define rcu_eqs_special_exit() so that RCU
> > can invoke it when it next leaves its quiescent state? Or are you able
> > to ignore the CPU in that case? (If you are able to ignore the CPU in
> > that case, I could give you a lower-cost function to get your job done.)
> >
> > Thanx, Paul
>
> What's actually needed for synchronization is issuing memory barrier on target
> CPUs before we start executing kernel code.
>
> smp_mb() is implicitly called in smp_call_function*() path for it. In
> rcu_eqs_special_set() -> rcu_dynticks_eqs_exit() path, smp_mb__after_atomic()
> is called just before rcu_eqs_special_exit().
>
> So I think, rcu_eqs_special_exit() may be left untouched. Empty
> rcu_eqs_special_exit() in new RCU path corresponds empty do_nothing() in old
> IPI path.
>
> Or my understanding of smp_mb__after_atomic() is wrong? By default,
> smp_mb__after_atomic() is just alias to smp_mb(). But some
> architectures define it differently. x86, for example, aliases it to
> just barrier() with a comment: "Atomic operations are already
> serializing on x86".
>
> I was initially thinking that it's also fine to leave
> rcu_eqs_special_exit() empty in this case, but now I'm not sure...
>
> Anyway, answering to your question, we shouldn't ignore quiescent
> CPUs, and rcu_eqs_special_set() path is really needed as it issues
> memory barrier on them.
An alternative approach would be for me to make something like this
and export it:
bool rcu_cpu_in_eqs(int cpu)
{
struct rcu_dynticks *rdtp = &per_cpu(rcu_dynticks, cpu);
int snap;
smp_mb(); /* Obtain consistent snapshot, pairs with update. */
snap = READ_ONCE(&rdtp->dynticks);
smp_mb(); /* See above. */
return !(snap & RCU_DYNTICK_CTRL_CTR);
}
Then you could replace your use of rcu_cpu_in_eqs() above with
the new rcu_cpu_in_eqs(). This would avoid the RMW atomic, and, more
important, the unnecessary write to ->dynticks.
Or am I missing something?
Thanx, Paul
> Yury
>
> > > + cpumask_set_cpu(cpu, &kernel_cpus);
> > > + }
> > > + smp_call_function_many(&kernel_cpus, do_nothing, NULL, 1);
> > > + preempt_enable();
> > > +}
> > > +EXPORT_SYMBOL_GPL(kick_active_cpus_sync);
> > > +
> > > /**
> > > * wake_up_all_idle_cpus - break all cpus out of idle
> > > * wake_up_all_idle_cpus try to break all cpus which is in idle state even
> > > diff --git a/mm/slab.c b/mm/slab.c
> > > index 324446621b3e..678d5dbd6f46 100644
> > > --- a/mm/slab.c
> > > +++ b/mm/slab.c
> > > @@ -3856,7 +3856,7 @@ static int __do_tune_cpucache(struct kmem_cache *cachep, int limit,
> > > * cpus, so skip the IPIs.
> > > */
> > > if (prev)
> > > - kick_all_cpus_sync();
> > > + kick_active_cpus_sync();
> > >
> > > check_irq_on();
> > > cachep->batchcount = batchcount;
> > > --
> > > 2.14.1
> > >
>
^ permalink raw reply
* Re: RFC on writel and writel_relaxed
From: Sinan Kaya @ 2018-03-26 12:42 UTC (permalink / raw)
To: Will Deacon
Cc: Benjamin Herrenschmidt, Oliver, linuxppc dev list, linux-rdma,
Marc Zyngier
In-Reply-To: <495123579cbc031ca7f1a6b0b305db18@codeaurora.org>
On 3/26/2018 8:11 AM, okaya@codeaurora.org wrote:
> On 2018-03-26 07:44, Will Deacon wrote:
>> Hi Ben,
>>
>> I don't seem to have the beginning of this thread, so please bounce it over
>> if you'd like me to look at it!
>>
>
> https://www.spinics.net/lists/linux-rdma/msg62570.html
>
> https://www.spinics.net/lists/linux-rdma/index.html#62666
>
To add some more details on why we are looking at this now:
I posted several patches last week to remove duplicate barriers on ARM while
trying to make the code friendly with other architectures.
https://www.spinics.net/lists/netdev/msg491842.html
https://www.spinics.net/lists/linux-rdma/msg62434.html
https://www.spinics.net/lists/arm-kernel/msg642336.html
The conversation on this thread is interesting.
https://patchwork.kernel.org/patch/10288987/
1. I tried to replace wmb()+writel() with wmb()+writel_relaxed().
2. writel_relaxed() is equal to writel() at this moment for PPC.
3. Chelsio developers wanted to pull it into wmb()+__raw_writel() direction
to take advantage of the same optimization for PPC.
4. Dave informed us that behavior of __raw_write() is not identical on all
architectures.
5. We decided to go back to PPC and ask to implement writel_relaxed()
instead of coming up with writel_realy_relaxed() API.
>
>> On Fri, Mar 23, 2018 at 11:16:08AM +1100, Benjamin Herrenschmidt wrote:
>>> On Thu, 2018-03-22 at 12:51 -0500, Sinan Kaya wrote:
>>> > On 3/22/2018 8:52 AM, Benjamin Herrenschmidt wrote:
>>> > > > > No, it's not sufficient.
>>> > >
>>> > > Just to clarify ... barrier() is just a compiler barrier, it means the
>>> > > compiler will generate things in the order they are written. This isn't
>>> > > sufficient on archs with an OO memory model, where an actual memory
>>> > > barrier instruction needs to be emited.
>>> >
>>> > Surprisingly, ARM64 GCC compiler generates a write barrier as
>>> > opposed to preventing code reordering.
>>
>> In context, this looks like a misunderstanding somewhere. barrier() is a
>> compiler barrier for us just like everybody else and we use the generic
>> implementation with the empty asm + memory clobber.
>>
>
> True, I clarified it this weekend
>
> https://www.spinics.net/lists/linux-rdma/msg62788.html
>
>
>
--
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox