* [PATCH v3 1/8] powerpc/perf/hv-24x7: Fix inconsistent output values incase multiple hv-24x7 events run
From: Kajol Jain @ 2020-02-29 9:41 UTC (permalink / raw)
To: acme, linuxppc-dev, mpe, sukadev
Cc: mark.rutland, maddy, peterz, yao.jin, mingo, kan.liang, ak,
alexander.shishkin, anju, mamatha4, ravi.bangoria, kjain, jmario,
namhyung, tglx, mpetlan, gregkh, linux-kernel, linux-perf-users,
jolsa
In-Reply-To: <20200229094159.25573-1-kjain@linux.ibm.com>
Commit 2b206ee6b0df ("powerpc/perf/hv-24x7: Display change in counter
values")' added to print _change_ in the counter value rather then raw
value for 24x7 counters. Incase of transactions, the event count
is set to 0 at the beginning of the transaction. It also sets
the event's prev_count to the raw value at the time of initialization.
Because of setting event count to 0, we are seeing some weird behaviour,
whenever we run multiple 24x7 events at a time.
For example:
command#: ./perf stat -e "{hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=0/,
hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=1/}"
-C 0 -I 1000 sleep 100
1.000121704 120 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=0/
1.000121704 5 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=1/
2.000357733 8 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=0/
2.000357733 10 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=1/
3.000495215 18,446,744,073,709,551,616 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=0/
3.000495215 18,446,744,073,709,551,616 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=1/
4.000641884 56 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=0/
4.000641884 18,446,744,073,709,551,616 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=1/
5.000791887 18,446,744,073,709,551,616 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=0/
Getting these large values in case we do -I.
As we are setting event_count to 0, for interval case, overall event_count is not
coming in incremental order. As we may can get new delta lesser then previous count.
Because of which when we print intervals, we are getting negative value which create
these large values.
This patch removes part where we set event_count to 0 in function
'h_24x7_event_read'. There won't be much impact as we do set event->hw.prev_count
to the raw value at the time of initialization to print change value.
With this patch
In power9 platform
command#: ./perf stat -e "{hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=0/,
hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=1/}"
-C 0 -I 1000 sleep 100
1.000117685 93 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=0/
1.000117685 1 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=1/
2.000349331 98 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=0/
2.000349331 2 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=1/
3.000495900 131 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=0/
3.000495900 4 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=1/
4.000645920 204 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=0/
4.000645920 61 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=1/
4.284169997 22 hv_24x7/PM_MCS01_128B_RD_DISP_PORT01,chip=0/
Signed-off-by: Kajol Jain <kjain@linux.ibm.com>
Suggested-by: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
---
arch/powerpc/perf/hv-24x7.c | 10 ----------
1 file changed, 10 deletions(-)
diff --git a/arch/powerpc/perf/hv-24x7.c b/arch/powerpc/perf/hv-24x7.c
index 573e0b309c0c..48e8f4b17b91 100644
--- a/arch/powerpc/perf/hv-24x7.c
+++ b/arch/powerpc/perf/hv-24x7.c
@@ -1400,16 +1400,6 @@ static void h_24x7_event_read(struct perf_event *event)
h24x7hw = &get_cpu_var(hv_24x7_hw);
h24x7hw->events[i] = event;
put_cpu_var(h24x7hw);
- /*
- * Clear the event count so we can compute the _change_
- * in the 24x7 raw counter value at the end of the txn.
- *
- * Note that we could alternatively read the 24x7 value
- * now and save its value in event->hw.prev_count. But
- * that would require issuing a hcall, which would then
- * defeat the purpose of using the txn interface.
- */
- local64_set(&event->count, 0);
}
put_cpu_var(hv_24x7_reqb);
--
2.21.0
^ permalink raw reply related
* [PATCH v3 2/8] powerpc/hv-24x7: Add rtas call in hv-24x7 driver to get processor details
From: Kajol Jain @ 2020-02-29 9:41 UTC (permalink / raw)
To: acme, linuxppc-dev, mpe, sukadev
Cc: mark.rutland, maddy, peterz, yao.jin, mingo, kan.liang, ak,
alexander.shishkin, anju, mamatha4, ravi.bangoria, kjain, jmario,
namhyung, tglx, mpetlan, gregkh, linux-kernel, linux-perf-users,
jolsa
In-Reply-To: <20200229094159.25573-1-kjain@linux.ibm.com>
For hv_24x7 socket/chip level events, specific chip-id to which
the data requested should be added as part of pmu events.
But number of chips/socket in the system details are not exposed.
Patch implements read_sys_info_pseries() to get system
parameter values like number of sockets and chips per socket.
Rtas_call with token "PROCESSOR_MODULE_INFO"
is used to get these values.
Sub-sequent patch exports these values via sysfs.
Patch also make these parameters default to 1.
Signed-off-by: Kajol Jain <kjain@linux.ibm.com>
---
arch/powerpc/perf/hv-24x7.c | 72 ++++++++++++++++++++++++
arch/powerpc/platforms/pseries/pseries.h | 3 +
2 files changed, 75 insertions(+)
diff --git a/arch/powerpc/perf/hv-24x7.c b/arch/powerpc/perf/hv-24x7.c
index 48e8f4b17b91..9ae00f29bd21 100644
--- a/arch/powerpc/perf/hv-24x7.c
+++ b/arch/powerpc/perf/hv-24x7.c
@@ -20,6 +20,11 @@
#include <asm/io.h>
#include <linux/byteorder/generic.h>
+#ifdef CONFIG_PPC_RTAS
+#include <asm/rtas.h>
+#include <../../platforms/pseries/pseries.h>
+#endif
+
#include "hv-24x7.h"
#include "hv-24x7-catalog.h"
#include "hv-common.h"
@@ -57,6 +62,69 @@ static bool is_physical_domain(unsigned domain)
}
}
+#ifdef CONFIG_PPC_RTAS
+#define PROCESSOR_MODULE_INFO 43
+#define PROCESSOR_MAX_LENGTH (8 * 1024)
+
+static int strbe16toh(const char *buf, int offset)
+{
+ return (buf[offset] << 8) + buf[offset + 1];
+}
+
+static u32 physsockets; /* Physical sockets */
+static u32 physchips; /* Physical chips */
+
+/*
+ * Function read_sys_info_pseries() make a rtas_call which require
+ * data buffer of size 8K. As standard 'rtas_data_buf' is of size
+ * 4K, we are adding new local buffer 'rtas_local_data_buf'.
+ */
+char rtas_local_data_buf[PROCESSOR_MAX_LENGTH] __cacheline_aligned;
+
+/*
+ * read_sys_info_pseries()
+ * Retrieve the number of sockets and chips per socket details
+ * through the get-system-parameter rtas call.
+ */
+void read_sys_info_pseries(void)
+{
+ int call_status, len, ntypes;
+
+ /*
+ * Making system parameter: chips and sockets default to 1.
+ */
+ physsockets = 1;
+ physchips = 1;
+ memset(rtas_local_data_buf, 0, PROCESSOR_MAX_LENGTH);
+ spin_lock(&rtas_data_buf_lock);
+
+ call_status = rtas_call(rtas_token("ibm,get-system-parameter"), 3, 1,
+ NULL,
+ PROCESSOR_MODULE_INFO,
+ __pa(rtas_local_data_buf),
+ PROCESSOR_MAX_LENGTH);
+
+ spin_unlock(&rtas_data_buf_lock);
+
+ if (call_status != 0) {
+ pr_info("%s %s Error calling get-system-parameter (0x%x)\n",
+ __FILE__, __func__, call_status);
+ } else {
+ rtas_local_data_buf[PROCESSOR_MAX_LENGTH - 1] = '\0';
+ len = strbe16toh(rtas_local_data_buf, 0);
+ if (len < 6)
+ return;
+
+ ntypes = strbe16toh(rtas_local_data_buf, 2);
+
+ if (!ntypes)
+ return;
+ physsockets = strbe16toh(rtas_local_data_buf, 4);
+ physchips = strbe16toh(rtas_local_data_buf, 6);
+ }
+}
+#endif /* CONFIG_PPC_RTAS */
+
/* Domains for which more than one result element are returned for each event. */
static bool domain_needs_aggregation(unsigned int domain)
{
@@ -1605,6 +1673,10 @@ static int hv_24x7_init(void)
if (r)
return r;
+#ifdef CONFIG_PPC_RTAS
+ read_sys_info_pseries();
+#endif
+
return 0;
}
diff --git a/arch/powerpc/platforms/pseries/pseries.h b/arch/powerpc/platforms/pseries/pseries.h
index 13fa370a87e4..1727559ce304 100644
--- a/arch/powerpc/platforms/pseries/pseries.h
+++ b/arch/powerpc/platforms/pseries/pseries.h
@@ -19,6 +19,9 @@ extern void request_event_sources_irqs(struct device_node *np,
struct pt_regs;
extern int pSeries_system_reset_exception(struct pt_regs *regs);
+#ifdef CONFIG_PPC_RTAS
+extern void read_sys_info_pseries(void);
+#endif
extern int pSeries_machine_check_exception(struct pt_regs *regs);
extern long pseries_machine_check_realmode(struct pt_regs *regs);
--
2.21.0
^ permalink raw reply related
* [PATCH v3 3/8] powerpc/hv-24x7: Add sysfs files inside hv-24x7 device to show processor details
From: Kajol Jain @ 2020-02-29 9:41 UTC (permalink / raw)
To: acme, linuxppc-dev, mpe, sukadev
Cc: mark.rutland, maddy, peterz, yao.jin, mingo, kan.liang, ak,
alexander.shishkin, anju, mamatha4, ravi.bangoria, kjain, jmario,
namhyung, tglx, mpetlan, gregkh, linux-kernel, linux-perf-users,
jolsa
In-Reply-To: <20200229094159.25573-1-kjain@linux.ibm.com>
To expose the system dependent parameter like total number of
sockets and numbers of chips per socket, patch adds two sysfs files.
"sockets" and "chips" are added to /sys/devices/hv_24x7/interface/
of the "hv_24x7" pmu.
Signed-off-by: Kajol Jain <kjain@linux.ibm.com>
---
arch/powerpc/perf/hv-24x7.c | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/arch/powerpc/perf/hv-24x7.c b/arch/powerpc/perf/hv-24x7.c
index 9ae00f29bd21..a31bd5b88f7a 100644
--- a/arch/powerpc/perf/hv-24x7.c
+++ b/arch/powerpc/perf/hv-24x7.c
@@ -454,6 +454,20 @@ static ssize_t device_show_string(struct device *dev,
return sprintf(buf, "%s\n", (char *)d->var);
}
+#ifdef CONFIG_PPC_RTAS
+static ssize_t sockets_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ return sprintf(buf, "%d\n", physsockets);
+}
+
+static ssize_t chips_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ return sprintf(buf, "%d\n", physchips);
+}
+#endif
+
static struct attribute *device_str_attr_create_(char *name, char *str)
{
struct dev_ext_attribute *attr = kzalloc(sizeof(*attr), GFP_KERNEL);
@@ -1100,6 +1114,10 @@ PAGE_0_ATTR(catalog_len, "%lld\n",
(unsigned long long)be32_to_cpu(page_0->length) * 4096);
static BIN_ATTR_RO(catalog, 0/* real length varies */);
static DEVICE_ATTR_RO(domains);
+#ifdef CONFIG_PPC_RTAS
+static DEVICE_ATTR_RO(sockets);
+static DEVICE_ATTR_RO(chips);
+#endif
static struct bin_attribute *if_bin_attrs[] = {
&bin_attr_catalog,
@@ -1110,6 +1128,10 @@ static struct attribute *if_attrs[] = {
&dev_attr_catalog_len.attr,
&dev_attr_catalog_version.attr,
&dev_attr_domains.attr,
+#ifdef CONFIG_PPC_RTAS
+ &dev_attr_sockets.attr,
+ &dev_attr_chips.attr,
+#endif
NULL,
};
--
2.21.0
^ permalink raw reply related
* [PATCH v3 4/8] Documentation/ABI: Add ABI documentation for chips and sockets
From: Kajol Jain @ 2020-02-29 9:41 UTC (permalink / raw)
To: acme, linuxppc-dev, mpe, sukadev
Cc: mark.rutland, maddy, peterz, yao.jin, mingo, kan.liang, ak,
alexander.shishkin, anju, mamatha4, ravi.bangoria, kjain, jmario,
namhyung, tglx, mpetlan, gregkh, linux-kernel, linux-perf-users,
jolsa
In-Reply-To: <20200229094159.25573-1-kjain@linux.ibm.com>
Add documentation for the following sysfs files:
/sys/devices/hv_24x7/interface/chips,
/sys/devices/hv_24x7/interface/sockets
Signed-off-by: Kajol Jain <kjain@linux.ibm.com>
---
.../testing/sysfs-bus-event_source-devices-hv_24x7 | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7 b/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7
index ec27c6c9e737..eb16a3b87ea8 100644
--- a/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7
+++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7
@@ -22,6 +22,20 @@ Description:
Exposes the "version" field of the 24x7 catalog. This is also
extractable from the provided binary "catalog" sysfs entry.
+What: /sys/devices/hv_24x7/interface/sockets
+Date: February 2020
+Contact: Linux on PowerPC Developer List <linuxppc-dev@lists.ozlabs.org>
+Description: read only
+ This sysfs interface exposes the number of sockets present in the
+ system.
+
+What: /sys/devices/hv_24x7/interface/chips
+Date: February 2020
+Contact: Linux on PowerPC Developer List <linuxppc-dev@lists.ozlabs.org>
+Description: read only
+ This sysfs interface exposes the number of chips per socket
+ present in the system.
+
What: /sys/bus/event_source/devices/hv_24x7/event_descs/<event-name>
Date: February 2014
Contact: Linux on PowerPC Developer List <linuxppc-dev@lists.ozlabs.org>
--
2.21.0
^ permalink raw reply related
* [PATCH v3 5/8] powerpc/hv-24x7: Update post_mobility_fixup() to handle migration
From: Kajol Jain @ 2020-02-29 9:41 UTC (permalink / raw)
To: acme, linuxppc-dev, mpe, sukadev
Cc: mark.rutland, maddy, peterz, yao.jin, mingo, kan.liang, ak,
alexander.shishkin, anju, mamatha4, ravi.bangoria, kjain, jmario,
namhyung, tglx, mpetlan, gregkh, linux-kernel, linux-perf-users,
jolsa
In-Reply-To: <20200229094159.25573-1-kjain@linux.ibm.com>
Function 'read_sys_info_pseries()' is added to get system parameter
values like number of sockets and chips per socket.
and it gets these details via rtas_call with token
"PROCESSOR_MODULE_INFO".
Incase lpar migrate from one system to another, system
parameter details like chips per sockets or number of sockets might
change. So, it needs to be re-initialized otherwise, these values
corresponds to previous system values.
This patch adds a call to 'read_sys_info_pseries()' from
'post-mobility_fixup()' to re-init the physsockets and physchips values.
Signed-off-by: Kajol Jain <kjain@linux.ibm.com>
---
arch/powerpc/platforms/pseries/mobility.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/arch/powerpc/platforms/pseries/mobility.c b/arch/powerpc/platforms/pseries/mobility.c
index b571285f6c14..226accd6218b 100644
--- a/arch/powerpc/platforms/pseries/mobility.c
+++ b/arch/powerpc/platforms/pseries/mobility.c
@@ -371,6 +371,18 @@ void post_mobility_fixup(void)
/* Possibly switch to a new RFI flush type */
pseries_setup_rfi_flush();
+ /*
+ * Incase lpar migrate from one system to another, system
+ * parameter details like chips per sockets and number of sockets
+ * might change. So, it needs to be re-initialized otherwise these
+ * values corresponds to previous system.
+ * Here, adding a call to read_sys_info_pseries() declared in
+ * platforms/pseries/pseries.h to re-init the physsockets and
+ * physchips value.
+ */
+ if (IS_ENABLED(CONFIG_HV_PERF_CTRS) && IS_ENABLED(CONFIG_PPC_RTAS))
+ read_sys_info_pseries();
+
return;
}
--
2.21.0
^ permalink raw reply related
* [PATCH v3 6/8] perf/tools: Enhance JSON/metric infrastructure to handle "?"
From: Kajol Jain @ 2020-02-29 9:41 UTC (permalink / raw)
To: acme, linuxppc-dev, mpe, sukadev
Cc: mark.rutland, maddy, peterz, yao.jin, mingo, kan.liang, ak,
alexander.shishkin, anju, mamatha4, ravi.bangoria, kjain, jmario,
namhyung, tglx, mpetlan, gregkh, linux-kernel, linux-perf-users,
jolsa
In-Reply-To: <20200229094159.25573-1-kjain@linux.ibm.com>
Patch enhances current metric infrastructure to handle "?" in the metric
expression. The "?" can be use for parameters whose value not known while
creating metric events and which can be replace later at runtime to
the proper value. It also add flexibility to create multiple events out
of single metric event added in json file.
Patch adds function 'arch_get_runtimeparam' which is a arch specific
function, returns the count of metric events need to be created.
By default it return 1.
One loop is added in function 'metricgroup__add_metric', which create
multiple events at run time depend on return value of
'arch_get_runtimeparam' and merge that event in 'group_list'.
This infrastructure needed for hv_24x7 socket/chip level events.
"hv_24x7" chip level events needs specific chip-id to which the
data is requested. Function 'arch_get_runtimeparam' implemented
in header.c which extract number of sockets from sysfs file
"sockets" under "/sys/devices/hv_24x7/interface/".
Signed-off-by: Kajol Jain <kjain@linux.ibm.com>
---
| 47 +++++++++++
tools/perf/util/expr.h | 1 +
tools/perf/util/expr.l | 18 ++++-
tools/perf/util/expr.y | 2 +
tools/perf/util/metricgroup.c | 112 ++++++++++++++++----------
tools/perf/util/metricgroup.h | 1 +
tools/perf/util/stat-shadow.c | 6 ++
7 files changed, 144 insertions(+), 43 deletions(-)
--git a/tools/perf/arch/powerpc/util/header.c b/tools/perf/arch/powerpc/util/header.c
index 3b4cdfc5efd6..5c15589522fd 100644
--- a/tools/perf/arch/powerpc/util/header.c
+++ b/tools/perf/arch/powerpc/util/header.c
@@ -7,6 +7,11 @@
#include <string.h>
#include <linux/stringify.h>
#include "header.h"
+#include "metricgroup.h"
+#include "evlist.h"
+#include <dirent.h>
+#include "pmu.h"
+#include <api/fs/fs.h>
#define mfspr(rn) ({unsigned long rval; \
asm volatile("mfspr %0," __stringify(rn) \
@@ -16,6 +21,8 @@
#define PVR_VER(pvr) (((pvr) >> 16) & 0xFFFF) /* Version field */
#define PVR_REV(pvr) (((pvr) >> 0) & 0xFFFF) /* Revison field */
+#define SOCKETS_INFO_FILE_PATH "/devices/hv_24x7/interface/"
+
int
get_cpuid(char *buffer, size_t sz)
{
@@ -44,3 +51,43 @@ get_cpuid_str(struct perf_pmu *pmu __maybe_unused)
return bufp;
}
+
+int arch_get_runtimeparam(void)
+{
+ int count = 0;
+ DIR *dir;
+ char path[PATH_MAX];
+ const char *sysfs = sysfs__mountpoint();
+ char filename[] = "sockets";
+ FILE *file;
+ char buf[16], *num;
+ int data;
+
+ if (!sysfs)
+ goto out;
+
+ snprintf(path, PATH_MAX,
+ "%s" SOCKETS_INFO_FILE_PATH, sysfs);
+ dir = opendir(path);
+
+ if (!dir)
+ goto out;
+
+ strcat(path, filename);
+ file = fopen(path, "r");
+
+ if (!file)
+ goto out;
+
+ data = fread(buf, 1, sizeof(buf), file);
+
+ if (data == 0)
+ goto out;
+
+ count = strtol(buf, &num, 10);
+out:
+ if (!count)
+ count = 1;
+
+ return count;
+}
diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
index 28a0a2dfbd8c..7f7c902bff46 100644
--- a/tools/perf/util/expr.h
+++ b/tools/perf/util/expr.h
@@ -15,6 +15,7 @@ struct parse_ctx {
struct parse_id ids[MAX_PARSE_ID];
};
+int expr__runtimeparam;
void expr__ctx_init(struct parse_ctx *ctx);
void expr__add_id(struct parse_ctx *ctx, const char *id, double val);
int expr__parse(double *final_val, struct parse_ctx *ctx, const char *expr);
diff --git a/tools/perf/util/expr.l b/tools/perf/util/expr.l
index 1928f2a3dddc..2cc6fa4b1960 100644
--- a/tools/perf/util/expr.l
+++ b/tools/perf/util/expr.l
@@ -45,6 +45,20 @@ static char *normalize(char *str)
*dst++ = '/';
else if (*str == '\\')
*dst++ = *++str;
+ else if (*str == '?') {
+ int size = snprintf(NULL, 0, "%d", expr__runtimeparam);
+ char * paramval = (char *)malloc(size);
+ int i = 0;
+
+ if(!paramval)
+ *dst++ = '0';
+ else {
+ sprintf(paramval, "%d", expr__runtimeparam);
+ while(i < size)
+ *dst++ = paramval[i++];
+ free(paramval);
+ }
+ }
else
*dst++ = *str;
str++;
@@ -72,8 +86,8 @@ number [0-9]+
sch [-,=]
spec \\{sch}
-sym [0-9a-zA-Z_\.:@]+
-symbol {spec}*{sym}*{spec}*{sym}*
+sym [0-9a-zA-Z_\.:@?]+
+symbol {spec}*{sym}*{spec}*{sym}*{spec}*{sym}
%%
{
diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
index 4720cbe79357..0f3ef0f37bf4 100644
--- a/tools/perf/util/expr.y
+++ b/tools/perf/util/expr.y
@@ -38,6 +38,8 @@
%type <num> expr if_expr
%{
+int expr__runtimeparam;
+
static void expr_error(double *final_val __maybe_unused,
struct parse_ctx *ctx __maybe_unused,
void *scanner,
diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
index 02aee946b6c1..f629828cc0de 100644
--- a/tools/perf/util/metricgroup.c
+++ b/tools/perf/util/metricgroup.c
@@ -399,6 +399,11 @@ void metricgroup__print(bool metrics, bool metricgroups, char *filter,
strlist__delete(metriclist);
}
+int __weak arch_get_runtimeparam(void)
+{
+ return 1;
+}
+
static int metricgroup__add_metric(const char *metric, struct strbuf *events,
struct list_head *group_list)
{
@@ -419,52 +424,77 @@ static int metricgroup__add_metric(const char *metric, struct strbuf *events,
continue;
if (match_metric(pe->metric_group, metric) ||
match_metric(pe->metric_name, metric)) {
- const char **ids;
- int idnum;
- struct egroup *eg;
- bool no_group = false;
+ int k, count;
+
+ if (strstr(pe->metric_expr, "?"))
+ count = arch_get_runtimeparam();
+ else
+ count = 1;
+
+ /* This loop is added to create multiple
+ * events depend on count value and add
+ * those events to group_list.
+ */
+ for (k = 0; k < count; k++) {
+ const char **ids;
+ int idnum;
+ struct egroup *eg;
+ bool no_group = false;
+ char value[PATH_MAX];
+
+ pr_debug("metric expr %s for %s\n",
+ pe->metric_expr, pe->metric_name);
+ expr__runtimeparam = k;
+ if (expr__find_other(pe->metric_expr, NULL,
+ &ids, &idnum) < 0)
+ continue;
+ if (events->len > 0)
+ strbuf_addf(events, ",");
+ for (j = 0; j < idnum; j++) {
+ pr_debug("found event %s\n", ids[j]);
+ /*
+ * Duration time maps to a software
+ * event and can make groups not count.
+ * Always use it outside a group.
+ */
+ if (!strcmp(ids[j], "duration_time")) {
+ if (j > 0)
+ strbuf_addf(events,
+ "}:W,");
+ strbuf_addf(events,
+ "duration_time");
+ no_group = true;
+ continue;
+ }
+ strbuf_addf(events, "%s%s",
+ j == 0 || no_group ? "{" :
+ ",", ids[j]);
+ no_group = false;
+ }
+ if (!no_group)
+ strbuf_addf(events, "}:W");
- pr_debug("metric expr %s for %s\n", pe->metric_expr, pe->metric_name);
+ eg = malloc(sizeof(struct egroup));
+ if (!eg) {
+ ret = -ENOMEM;
+ break;
+ }
+ eg->ids = ids;
+ eg->idnum = idnum;
- if (expr__find_other(pe->metric_expr,
- NULL, &ids, &idnum) < 0)
- continue;
- if (events->len > 0)
- strbuf_addf(events, ",");
- for (j = 0; j < idnum; j++) {
- pr_debug("found event %s\n", ids[j]);
- /*
- * Duration time maps to a software event and can make
- * groups not count. Always use it outside a
- * group.
- */
- if (!strcmp(ids[j], "duration_time")) {
- if (j > 0)
- strbuf_addf(events, "}:W,");
- strbuf_addf(events, "duration_time");
- no_group = true;
- continue;
+ if (strstr(pe->metric_expr, "?")) {
+ sprintf(value, "%s%c%d",
+ pe->metric_name, '_', k);
+ } else {
+ strcpy(value, pe->metric_name);
}
- strbuf_addf(events, "%s%s",
- j == 0 || no_group ? "{" : ",",
- ids[j]);
- no_group = false;
- }
- if (!no_group)
- strbuf_addf(events, "}:W");
- eg = malloc(sizeof(struct egroup));
- if (!eg) {
- ret = -ENOMEM;
- break;
+ eg->metric_name = strdup(value);
+ eg->metric_expr = pe->metric_expr;
+ eg->metric_unit = pe->unit;
+ list_add_tail(&eg->nd, group_list);
+ ret = 0;
}
- eg->ids = ids;
- eg->idnum = idnum;
- eg->metric_name = pe->metric_name;
- eg->metric_expr = pe->metric_expr;
- eg->metric_unit = pe->unit;
- list_add_tail(&eg->nd, group_list);
- ret = 0;
}
}
return ret;
diff --git a/tools/perf/util/metricgroup.h b/tools/perf/util/metricgroup.h
index 475c7f912864..81224ba1270d 100644
--- a/tools/perf/util/metricgroup.h
+++ b/tools/perf/util/metricgroup.h
@@ -34,4 +34,5 @@ int metricgroup__parse_groups(const struct option *opt,
void metricgroup__print(bool metrics, bool groups, char *filter,
bool raw, bool details);
bool metricgroup__has_metric(const char *metric);
+int arch_get_runtimeparam(void);
#endif
diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
index 0fd713d3674f..c1179f7d2b23 100644
--- a/tools/perf/util/stat-shadow.c
+++ b/tools/perf/util/stat-shadow.c
@@ -777,6 +777,12 @@ static void generic_metric(struct perf_stat_config *config,
}
if (!metric_events[i]) {
+ if (strstr(metric_expr, "?")) {
+ char *tmp = strrchr(metric_name, '_');
+
+ tmp++;
+ expr__runtimeparam = strtol(tmp, &tmp, 10);
+ }
if (expr__parse(&ratio, &pctx, metric_expr) == 0) {
char *unit;
char metric_bf[64];
--
2.21.0
^ permalink raw reply related
* [PATCH v3 7/8] tools/perf: Enable Hz/hz prinitg for --metric-only option
From: Kajol Jain @ 2020-02-29 9:41 UTC (permalink / raw)
To: acme, linuxppc-dev, mpe, sukadev
Cc: mark.rutland, maddy, peterz, yao.jin, mingo, kan.liang, ak,
alexander.shishkin, anju, mamatha4, ravi.bangoria, kjain, jmario,
namhyung, tglx, mpetlan, gregkh, linux-kernel, linux-perf-users,
jolsa
In-Reply-To: <20200229094159.25573-1-kjain@linux.ibm.com>
Commit 54b5091606c18 ("perf stat: Implement --metric-only mode")
added function 'valid_only_metric()' which drops "Hz" or "hz",
if it is part of "ScaleUnit". This patch enable it since hv_24x7
supports couple of frequency events.
Signed-off-by: Kajol Jain <kjain@linux.ibm.com>
---
tools/perf/util/stat-display.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/tools/perf/util/stat-display.c b/tools/perf/util/stat-display.c
index bc31fccc0057..22dcdfbb9e10 100644
--- a/tools/perf/util/stat-display.c
+++ b/tools/perf/util/stat-display.c
@@ -236,8 +236,6 @@ static bool valid_only_metric(const char *unit)
if (!unit)
return false;
if (strstr(unit, "/sec") ||
- strstr(unit, "hz") ||
- strstr(unit, "Hz") ||
strstr(unit, "CPUs utilized"))
return false;
return true;
--
2.21.0
^ permalink raw reply related
* [PATCH v3 8/8] perf/tools/pmu-events/powerpc: Add hv_24x7 socket/chip level metric events
From: Kajol Jain @ 2020-02-29 9:41 UTC (permalink / raw)
To: acme, linuxppc-dev, mpe, sukadev
Cc: mark.rutland, maddy, peterz, yao.jin, mingo, kan.liang, ak,
alexander.shishkin, anju, mamatha4, ravi.bangoria, kjain, jmario,
namhyung, tglx, mpetlan, gregkh, linux-kernel, linux-perf-users,
jolsa
In-Reply-To: <20200229094159.25573-1-kjain@linux.ibm.com>
The hv_24×7 feature in IBM® POWER9™ processor-based servers provide the
facility to continuously collect large numbers of hardware performance
metrics efficiently and accurately.
This patch adds hv_24x7 metric file for different Socket/chip
resources.
Result:
power9 platform:
command:# ./perf stat --metric-only -M Memory_RD_BW_Chip -C 0 -I 1000
1.000096188 0.9 0.3
2.000285720 0.5 0.1
3.000424990 0.4 0.1
command:# ./perf stat --metric-only -M PowerBUS_Frequency -C 0 -I 1000
1.000097981 2.3 2.3
2.000291713 2.3 2.3
3.000421719 2.3 2.3
4.000550912 2.3 2.3
Signed-off-by: Kajol Jain <kjain@linux.ibm.com>
---
.../arch/powerpc/power9/nest_metrics.json | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
create mode 100644 tools/perf/pmu-events/arch/powerpc/power9/nest_metrics.json
diff --git a/tools/perf/pmu-events/arch/powerpc/power9/nest_metrics.json b/tools/perf/pmu-events/arch/powerpc/power9/nest_metrics.json
new file mode 100644
index 000000000000..ac38f5540ac6
--- /dev/null
+++ b/tools/perf/pmu-events/arch/powerpc/power9/nest_metrics.json
@@ -0,0 +1,19 @@
+[
+ {
+ "MetricExpr": "(hv_24x7@PM_MCS01_128B_RD_DISP_PORT01\\,chip\\=?@ + hv_24x7@PM_MCS01_128B_RD_DISP_PORT23\\,chip\\=?@ + hv_24x7@PM_MCS23_128B_RD_DISP_PORT01\\,chip\\=?@ + hv_24x7@PM_MCS23_128B_RD_DISP_PORT23\\,chip\\=?@)",
+ "MetricName": "Memory_RD_BW_Chip",
+ "MetricGroup": "Memory_BW",
+ "ScaleUnit": "1.6e-2MB"
+ },
+ {
+ "MetricExpr": "(hv_24x7@PM_MCS01_128B_WR_DISP_PORT01\\,chip\\=?@ + hv_24x7@PM_MCS01_128B_WR_DISP_PORT23\\,chip\\=?@ + hv_24x7@PM_MCS23_128B_WR_DISP_PORT01\\,chip\\=?@ + hv_24x7@PM_MCS23_128B_WR_DISP_PORT23\\,chip\\=?@ )",
+ "MetricName": "Memory_WR_BW_Chip",
+ "MetricGroup": "Memory_BW",
+ "ScaleUnit": "1.6e-2MB"
+ },
+ {
+ "MetricExpr": "(hv_24x7@PM_PB_CYC\\,chip\\=?@ )",
+ "MetricName": "PowerBUS_Frequency",
+ "ScaleUnit": "2.5e-7GHz"
+ }
+]
--
2.21.0
^ permalink raw reply related
* [PATCH V2 0/9] crypto/nx: Enable GZIP engine and provide userpace API
From: Haren Myneni @ 2020-02-29 11:04 UTC (permalink / raw)
To: herbert, mpe; +Cc: mikey, sukadev, linuxppc-dev, linux-crypto, npiggin
Power9 processor supports Virtual Accelerator Switchboard (VAS) which
allows kernel and userspace to send compression requests to Nest
Accelerator (NX) directly. The NX unit comprises of 2 842 compression
engines and 1 GZIP engine. Linux kernel already has 842 compression
support on kernel. This patch series adds GZIP compression support
from user space. The GZIP Compression engine implements the ZLIB and
GZIP compression algorithms. No plans of adding NX-GZIP compression
support in kernel right now.
Applications can send requests to NX directly with COPY/PASTE
instructions. But kernel has to establish channel / window on NX-GZIP
device for the userspace. So userspace access to the GZIP engine is
provided through /dev/crypto/nx-gzip device with several operations.
An application must open the this device to obtain a file descriptor (fd).
Using the fd, application should issue the VAS_TX_WIN_OPEN ioctl to
establish a connection to the engine. Once window is opened, should use
mmap() system call to map the hardware address of engine's request queue
into the application's virtual address space. Then user space forms the
request as co-processor Request Block (CRB) and paste this CRB on the
mapped HW address using COPY/PASTE instructions. Application can poll
on status flags (part of CRB) with timeout for request completion.
For VAS_TX_WIN_OPEN ioctl, if user space passes vas_id = -1 (struct
vas_tx_win_open_attr), kernel determins the VAS instance on the
corresponding chip based on the CPU on which the process is executing.
Otherwise, the specified VAS instance is used if application passes the
proper VAS instance (vas_id listed in /proc/device-tree/vas@*/ibm,vas_id).
Process can open multiple windows with different FDs or can send several
requests to NX on the same window at the same time.
A userspace library libnxz is available:
https://github.com/abalib/power-gzip
Applications that use inflate/deflate calls can link with libNXz and use
NX GZIP compression without any modification.
Tested the available 842 compression on power8 and power9 system to make
sure no regression and tested GZIP compression on power9 with tests
available in the above link.
Thanks to Bulent Abali for nxz library and tests development.
Changelog:
V2:
- Move user space API code to powerpc as suggested. Also this API
can be extended to any other coprocessor type that VAS can support
in future. Example: Fast thread wakeup feature from VAS
- Rebased to 5.6-rc3
Haren Myneni (9):
powerpc/vas: Initialize window attributes for GZIP coprocessor type
powerpc/vas: Define VAS_TX_WIN_OPEN ioctl API
powerpc/vas: Add VAS user space API
crypto/nx: Initialize coproc entry with kzalloc
crypto/nx: Rename nx-842-powernv file name to nx-common-powernv
crypto/NX: Make enable code generic to add new GZIP compression type
crypto/nx: Enable and setup GZIP compresstion type
crypto/nx: Remove 'pid' in vas_tx_win_attr struct
Documentation/powerpc: VAS API
Documentation/powerpc/index.rst | 1 +
Documentation/powerpc/vas-api.rst | 246 +++++
Documentation/userspace-api/ioctl/ioctl-number.rst | 1 +
arch/powerpc/include/asm/vas.h | 12 +-
arch/powerpc/include/uapi/asm/vas-api.h | 22 +
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/platforms/powernv/vas-api.c | 290 +++++
arch/powerpc/platforms/powernv/vas-window.c | 23 +-
arch/powerpc/platforms/powernv/vas.h | 2 +
drivers/crypto/nx/Makefile | 2 +-
drivers/crypto/nx/nx-842-powernv.c | 1062 ------------------
drivers/crypto/nx/nx-common-powernv.c | 1133 ++++++++++++++++++++
12 files changed, 1723 insertions(+), 1073 deletions(-)
create mode 100644 Documentation/powerpc/vas-api.rst
create mode 100644 arch/powerpc/include/uapi/asm/vas-api.h
create mode 100644 arch/powerpc/platforms/powernv/vas-api.c
delete mode 100644 drivers/crypto/nx/nx-842-powernv.c
create mode 100644 drivers/crypto/nx/nx-common-powernv.c
--
1.8.3.1
^ permalink raw reply
* [PATCH V2 1/9] powerpc/vas: Initialize window attributes for GZIP coprocessor type
From: Haren Myneni @ 2020-02-29 11:11 UTC (permalink / raw)
To: herbert; +Cc: mikey, npiggin, linux-crypto, sukadev, linuxppc-dev
In-Reply-To: <1582974266.18705.28.camel@hbabu-laptop>
Initialize send and receive window attributes for GZIP high and
normal priority types.
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
arch/powerpc/platforms/powernv/vas-window.c | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c b/arch/powerpc/platforms/powernv/vas-window.c
index 5f9c915..b6572af 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -817,7 +817,8 @@ void vas_init_rx_win_attr(struct vas_rx_win_attr *rxattr, enum vas_cop_type cop)
{
memset(rxattr, 0, sizeof(*rxattr));
- if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI) {
+ if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI ||
+ cop == VAS_COP_TYPE_GZIP || cop == VAS_COP_TYPE_GZIP_HIPRI) {
rxattr->pin_win = true;
rxattr->nx_win = true;
rxattr->fault_win = false;
@@ -892,7 +893,8 @@ void vas_init_tx_win_attr(struct vas_tx_win_attr *txattr, enum vas_cop_type cop)
{
memset(txattr, 0, sizeof(*txattr));
- if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI) {
+ if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI ||
+ cop == VAS_COP_TYPE_GZIP || cop == VAS_COP_TYPE_GZIP_HIPRI) {
txattr->rej_no_credit = false;
txattr->rx_wcred_mode = true;
txattr->tx_wcred_mode = true;
@@ -976,9 +978,14 @@ static bool tx_win_args_valid(enum vas_cop_type cop,
if (attr->wcreds_max > VAS_TX_WCREDS_MAX)
return false;
- if (attr->user_win &&
- (cop != VAS_COP_TYPE_FTW || attr->rsvd_txbuf_count))
- return false;
+ if (attr->user_win) {
+ if (attr->rsvd_txbuf_count)
+ return false;
+
+ if (cop != VAS_COP_TYPE_FTW && cop != VAS_COP_TYPE_GZIP &&
+ cop != VAS_COP_TYPE_GZIP_HIPRI)
+ return false;
+ }
return true;
}
--
1.8.3.1
^ permalink raw reply related
* [PATCH V2 2/9] powerpc/vas: Define VAS_TX_WIN_OPEN ioctl API
From: Haren Myneni @ 2020-02-29 11:11 UTC (permalink / raw)
To: herbert; +Cc: mikey, npiggin, linux-crypto, sukadev, linuxppc-dev
In-Reply-To: <1582974266.18705.28.camel@hbabu-laptop>
Define the VAS_TX_WIN_OPEN ioctl interface for NX GZIP access
from user space. This interface is used to open GZIP send window and
mmap region which can be used by userspace to send requests to NX
directly with copy/paste instructions.
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
Documentation/userspace-api/ioctl/ioctl-number.rst | 1 +
arch/powerpc/include/uapi/asm/vas-api.h | 22 ++++++++++++++++++++++
2 files changed, 23 insertions(+)
create mode 100644 arch/powerpc/include/uapi/asm/vas-api.h
diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
index 2e91370..deabc73 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -287,6 +287,7 @@ Code Seq# Include File Comments
'v' 00-1F linux/fs.h conflict!
'v' 00-0F linux/sonypi.h conflict!
'v' 00-0F media/v4l2-subdev.h conflict!
+'v' 20-27 arch/powerpc/include/uapi/asm/vas-api.h VAS API
'v' C0-FF linux/meye.h conflict!
'w' all CERN SCI driver
'y' 00-1F packet based user level communications
diff --git a/arch/powerpc/include/uapi/asm/vas-api.h b/arch/powerpc/include/uapi/asm/vas-api.h
new file mode 100644
index 0000000..fe95d67
--- /dev/null
+++ b/arch/powerpc/include/uapi/asm/vas-api.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
+/*
+ * Copyright 2019 IBM Corp.
+ */
+
+#ifndef _UAPI_MISC_VAS_H
+#define _UAPI_MISC_VAS_H
+
+#include <asm/ioctl.h>
+
+#define VAS_MAGIC 'v'
+#define VAS_TX_WIN_OPEN _IOW(VAS_MAGIC, 0x20, struct vas_tx_win_open_attr)
+
+struct vas_tx_win_open_attr {
+ __u32 version;
+ __s16 vas_id; /* specific instance of vas or -1 for default */
+ __u16 reserved1;
+ __u64 flags; /* Future use */
+ __u64 reserved2[6];
+};
+
+#endif /* _UAPI_MISC_VAS_H */
--
1.8.3.1
^ permalink raw reply related
* [PATCH V3 3/9] powerpc/vas: Add VAS user space API
From: Haren Myneni @ 2020-02-29 11:14 UTC (permalink / raw)
To: herbert; +Cc: mikey, npiggin, linux-crypto, sukadev, linuxppc-dev
In-Reply-To: <1582974266.18705.28.camel@hbabu-laptop>
On power9, userspace can send GZIP compression requests directly to NX
once kernel establishes NX channel / window with VAS. This patch provides
user space API which allows user space to establish channel using open
VAS_TX_WIN_OPEN ioctl, mmap and close operations.
Each window corresponds to file descriptor and application can open
multiple windows. After the window is opened, VAS_TX_WIN_OPEN icoctl to
open a window on specific VAS instance, mmap() system call to map
the hardware address of engine's request queue into the application's
virtual address space.
Then the application can then submit one or more requests to the the
engine by using the copy/paste instructions and pasting the CRBs to
the virtual address (aka paste_address) returned by mmap().
Only NX GZIP coprocessor type is supported right now and allow GZIP
engine access via /dev/crypto/nx-gzip device node.
Signed-off-by: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
arch/powerpc/include/asm/vas.h | 11 ++
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/platforms/powernv/vas-api.c | 290 ++++++++++++++++++++++++++++
arch/powerpc/platforms/powernv/vas-window.c | 6 +-
arch/powerpc/platforms/powernv/vas.h | 2 +
5 files changed, 307 insertions(+), 4 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/vas-api.c
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index f93e6b0..e064953 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -163,4 +163,15 @@ struct vas_window *vas_tx_win_open(int vasid, enum vas_cop_type cop,
*/
int vas_paste_crb(struct vas_window *win, int offset, bool re);
+/*
+ * Register / unregister coprocessor type to VAS API which will be exported
+ * to user space. Applications can use this API to open / close window
+ * which can be used to send / receive requests directly to cooprcessor.
+ *
+ * Only NX GZIP coprocessor type is supported now, but this API can be
+ * used for others in future.
+ */
+int vas_register_coproc_api(struct module *mod);
+void vas_unregister_coproc_api(void);
+
#endif /* __ASM_POWERPC_VAS_H */
diff --git a/arch/powerpc/platforms/powernv/Makefile b/arch/powerpc/platforms/powernv/Makefile
index 395789f..fe3f0fb 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -17,7 +17,7 @@ obj-$(CONFIG_MEMORY_FAILURE) += opal-memory-errors.o
obj-$(CONFIG_OPAL_PRD) += opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
-obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-fault.o
+obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-fault.o vas-api.o
obj-$(CONFIG_OCXL_BASE) += ocxl.o
obj-$(CONFIG_SCOM_DEBUGFS) += opal-xscom.o
obj-$(CONFIG_PPC_SECURE_BOOT) += opal-secvar.o
diff --git a/arch/powerpc/platforms/powernv/vas-api.c b/arch/powerpc/platforms/powernv/vas-api.c
new file mode 100644
index 0000000..d72755c
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas-api.c
@@ -0,0 +1,290 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * VAS user space API for its accelerators (Only NX-GZIP is supported now)
+ * Copyright (C) 2019 Haren Myneni, IBM Corp
+ */
+
+#include <linux/kernel.h>
+#include <linux/device.h>
+#include <linux/cdev.h>
+#include <linux/fs.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+#include <asm/vas.h>
+#include <uapi/asm/vas-api.h>
+#include "vas.h"
+
+/*
+ * The driver creates the device node that can be used as follows:
+ * For NX-GZIP
+ *
+ * fd = open("/dev/crypto/nx-gzip", O_RDWR);
+ * rc = ioctl(fd, VAS_TX_WIN_OPEN, &attr);
+ * paste_addr = mmap(NULL, PAGE_SIZE, prot, MAP_SHARED, fd, 0ULL).
+ * vas_copy(&crb, 0, 1);
+ * vas_paste(paste_addr, 0, 1);
+ * close(fd) or exit process to close window.
+ *
+ * where "vas_copy" and "vas_paste" are defined in copy-paste.h.
+ * copy/paste returns to the user space directly. So refer NX hardware
+ * documententation for excat copy/paste usage and completion / error
+ * conditions.
+ */
+
+static char *coproc_dev_name = "nx-gzip";
+static atomic_t coproc_instid = ATOMIC_INIT(0);
+
+/*
+ * Wrapper object for the nx-gzip device - there is just one instance of
+ * this node for the whole system.
+ */
+struct coproc_dev {
+ struct cdev cdev;
+ struct device *device;
+ char *name;
+ dev_t devt;
+ struct class *class;
+} coproc_device;
+
+/*
+ * One instance per open of a nx-gzip device. Each coproc_instance is
+ * associated with a VAS window after the caller issues
+ * VAS_GZIP_TX_WIN_OPEN ioctl.
+ */
+struct coproc_instance {
+ int id;
+ struct vas_window *txwin;
+};
+
+static char *coproc_devnode(struct device *dev, umode_t *mode)
+{
+ return kasprintf(GFP_KERNEL, "crypto/%s", dev_name(dev));
+}
+
+static int coproc_open(struct inode *inode, struct file *fp)
+{
+ struct coproc_instance *instance;
+
+ instance = kzalloc(sizeof(*instance), GFP_KERNEL);
+ if (!instance)
+ return -ENOMEM;
+
+ instance->id = atomic_inc_return(&coproc_instid);
+
+ fp->private_data = instance;
+ return 0;
+}
+
+static int coproc_ioc_tx_win_open(struct file *fp, unsigned long arg)
+{
+ int rc, vasid;
+ struct vas_tx_win_attr txattr;
+ struct vas_tx_win_open_attr uattr;
+ void __user *uptr = (void *)arg;
+ struct vas_window *txwin;
+ struct coproc_instance *nxti = fp->private_data;
+
+ if (!nxti)
+ return -EINVAL;
+
+ /*
+ * One window for file descriptor
+ */
+ if (nxti->txwin)
+ return -EEXIST;
+
+ rc = copy_from_user(&uattr, uptr, sizeof(uattr));
+ if (rc) {
+ pr_err("%s(): copy_from_user() returns %d\n", __func__, rc);
+ return -EFAULT;
+ }
+
+ if (uattr.version != 1) {
+ pr_err("Invalid version\n");
+ return -EINVAL;
+ }
+
+ vasid = uattr.vas_id;
+
+ memset(&txattr, 0, sizeof(struct vas_tx_win_attr));
+ vas_init_tx_win_attr(&txattr, VAS_COP_TYPE_GZIP);
+
+ txattr.lpid = mfspr(SPRN_LPID);
+ txattr.pidr = mfspr(SPRN_PID);
+ txattr.user_win = true;
+ txattr.rsvd_txbuf_count = false;
+ txattr.pswid = false;
+ /*
+ * txattr.wcreds_max is set to VAS_WCREDS_DEFAULT (1024) in
+ * vas-window.c, but can be changed specific to GZIP depends
+ * on user space need.
+ * If needed to set txattr.wcreds_max here.
+ */
+
+ pr_devel("Pid %d: Opening txwin, PIDR %ld\n", txattr.pidr,
+ mfspr(SPRN_PID));
+
+ txwin = vas_tx_win_open(vasid, VAS_COP_TYPE_GZIP, &txattr);
+ if (IS_ERR(txwin)) {
+ pr_err("%s() vas_tx_win_open() failed, %ld\n", __func__,
+ PTR_ERR(txwin));
+ return PTR_ERR(txwin);
+ }
+
+ nxti->txwin = txwin;
+
+ return 0;
+}
+
+static int coproc_release(struct inode *inode, struct file *fp)
+{
+ struct coproc_instance *instance;
+
+ instance = fp->private_data;
+
+ if (instance && instance->txwin) {
+ vas_win_close(instance->txwin);
+ instance->txwin = NULL;
+ }
+
+ /*
+ * We don't know here if user has other receive windows
+ * open, so we can't really call clear_thread_tidr().
+ * So, once the process calls set_thread_tidr(), the
+ * TIDR value sticks around until process exits, resulting
+ * in an extra copy in restore_sprs().
+ */
+
+ kfree(instance);
+ fp->private_data = NULL;
+ atomic_dec(&coproc_instid);
+
+ return 0;
+}
+
+static int coproc_mmap(struct file *fp, struct vm_area_struct *vma)
+{
+ int rc;
+ pgprot_t prot;
+ u64 paste_addr;
+ unsigned long pfn;
+ struct coproc_instance *instance = fp->private_data;
+
+ if ((vma->vm_end - vma->vm_start) > PAGE_SIZE) {
+ pr_debug("%s(): size 0x%zx, PAGE_SIZE 0x%zx\n", __func__,
+ (vma->vm_end - vma->vm_start), PAGE_SIZE);
+ return -EINVAL;
+ }
+
+ /* Ensure instance has an open send window */
+ if (!instance->txwin) {
+ pr_err("%s(): No send window open?\n", __func__);
+ return -EINVAL;
+ }
+
+ vas_win_paste_addr(instance->txwin, &paste_addr, NULL);
+ pfn = paste_addr >> PAGE_SHIFT;
+
+ /* flags, page_prot from cxl_mmap(), except we want cachable */
+ vma->vm_flags |= VM_IO | VM_PFNMAP;
+ vma->vm_page_prot = pgprot_cached(vma->vm_page_prot);
+
+ prot = __pgprot(pgprot_val(vma->vm_page_prot) | _PAGE_DIRTY);
+
+ rc = remap_pfn_range(vma, vma->vm_start, pfn + vma->vm_pgoff,
+ vma->vm_end - vma->vm_start, prot);
+
+ pr_devel("%s(): paste addr %llx at %lx, rc %d\n", __func__,
+ paste_addr, vma->vm_start, rc);
+
+ return rc;
+}
+
+static long coproc_ioctl(struct file *fp, unsigned int cmd, unsigned long arg)
+{
+ switch (cmd) {
+ case VAS_TX_WIN_OPEN:
+ return coproc_ioc_tx_win_open(fp, arg);
+ default:
+ return -EINVAL;
+ }
+}
+
+struct file_operations coproc_fops = {
+ .open = coproc_open,
+ .release = coproc_release,
+ .mmap = coproc_mmap,
+ .unlocked_ioctl = coproc_ioctl,
+};
+
+/*
+ * Supporting only nx-gzip coprocessor type now, but this API code
+ * extended to other coprocessor types later.
+ */
+int vas_register_coproc_api(struct module *mod)
+{
+ int rc = -EINVAL;
+ dev_t devno;
+
+ rc = alloc_chrdev_region(&coproc_device.devt, 1, 1, "nx-gzip");
+ if (rc) {
+ pr_err("Unable to allocate coproc major number: %i\n", rc);
+ return rc;
+ }
+
+ pr_devel("NX-GZIP device allocated, dev [%i,%i]\n",
+ MAJOR(coproc_device.devt), MINOR(coproc_device.devt));
+
+ coproc_device.class = class_create(mod, "nx-gzip");
+ if (IS_ERR(coproc_device.class)) {
+ rc = PTR_ERR(coproc_device.class);
+ pr_err("Unable to create NX-GZIP class %d\n", rc);
+ goto err_class;
+ }
+ coproc_device.class->devnode = coproc_devnode;
+
+ coproc_fops.owner = mod;
+ cdev_init(&coproc_device.cdev, &coproc_fops);
+
+ devno = MKDEV(MAJOR(coproc_device.devt), 0);
+ rc = cdev_add(&coproc_device.cdev, devno, 1);
+ if (rc) {
+ pr_err("cdev_add() failed %d\n", rc);
+ goto err_cdev;
+ }
+
+ coproc_device.device = device_create(coproc_device.class, NULL,
+ devno, NULL, coproc_dev_name, MINOR(devno));
+ if (IS_ERR(coproc_device.device)) {
+ rc = PTR_ERR(coproc_device.device);
+ pr_err("Unable to create coproc-%d %d\n", MINOR(devno), rc);
+ goto err;
+ }
+
+ pr_devel("%s: Added dev [%d,%d]\n", __func__, MAJOR(devno),
+ MINOR(devno));
+
+ return 0;
+
+err:
+ cdev_del(&coproc_device.cdev);
+err_cdev:
+ class_destroy(coproc_device.class);
+err_class:
+ unregister_chrdev_region(coproc_device.devt, 1);
+ return rc;
+}
+EXPORT_SYMBOL_GPL(vas_register_coproc_api);
+
+void vas_unregister_coproc_api(void)
+{
+ dev_t devno;
+
+ cdev_del(&coproc_device.cdev);
+ devno = MKDEV(MAJOR(coproc_device.devt), 0);
+ device_destroy(coproc_device.class, devno);
+
+ class_destroy(coproc_device.class);
+ unregister_chrdev_region(coproc_device.devt, 1);
+}
+EXPORT_SYMBOL_GPL(vas_unregister_coproc_api);
diff --git a/arch/powerpc/platforms/powernv/vas-window.c b/arch/powerpc/platforms/powernv/vas-window.c
index b6572af..d4a575e 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -26,7 +26,7 @@
* Compute the paste address region for the window @window using the
* ->paste_base_addr and ->paste_win_id_shift we got from device tree.
*/
-static void compute_paste_address(struct vas_window *window, u64 *addr, int *len)
+void vas_win_paste_addr(struct vas_window *window, u64 *addr, int *len)
{
int winid;
u64 base, shift;
@@ -80,7 +80,7 @@ static void *map_paste_region(struct vas_window *txwin)
goto free_name;
txwin->paste_addr_name = name;
- compute_paste_address(txwin, &start, &len);
+ vas_win_paste_addr(txwin, &start, &len);
if (!request_mem_region(start, len, name)) {
pr_devel("%s(): request_mem_region(0x%llx, %d) failed\n",
@@ -138,7 +138,7 @@ static void unmap_paste_region(struct vas_window *window)
u64 busaddr_start;
if (window->paste_kaddr) {
- compute_paste_address(window, &busaddr_start, &len);
+ vas_win_paste_addr(window, &busaddr_start, &len);
unmap_region(window->paste_kaddr, busaddr_start, len);
window->paste_kaddr = NULL;
kfree(window->paste_addr_name);
diff --git a/arch/powerpc/platforms/powernv/vas.h b/arch/powerpc/platforms/powernv/vas.h
index 495937a..96efb99 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -418,6 +418,8 @@ struct vas_winctx {
extern void vas_return_credit(struct vas_window *window, bool tx);
extern struct vas_window *vas_pswid_to_window(struct vas_instance *vinst,
uint32_t pswid);
+extern void vas_win_paste_addr(struct vas_window *window, u64 *addr,
+ int *len);
static inline int vas_window_pid(struct vas_window *window)
{
--
1.8.3.1
^ permalink raw reply related
* [PATCH V2 4/9] crypto/nx: Initialize coproc entry with kzalloc
From: Haren Myneni @ 2020-02-29 11:15 UTC (permalink / raw)
To: herbert; +Cc: mikey, npiggin, linux-crypto, sukadev, linuxppc-dev
In-Reply-To: <1582974266.18705.28.camel@hbabu-laptop>
coproc entry is initialized during NX probe on power9, but not on P8.
nx842_delete_coprocs() is used for both and frees receive window if it
is allocated. Getting crash for rmmod on P8 since coproc->vas.rxwin
is not initialized.
This patch replaces kmalloc with kzalloc in nx842_powernv_probe()
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
drivers/crypto/nx/nx-842-powernv.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/crypto/nx/nx-842-powernv.c b/drivers/crypto/nx/nx-842-powernv.c
index c037a24..8e63326 100644
--- a/drivers/crypto/nx/nx-842-powernv.c
+++ b/drivers/crypto/nx/nx-842-powernv.c
@@ -922,7 +922,7 @@ static int __init nx842_powernv_probe(struct device_node *dn)
return -EINVAL;
}
- coproc = kmalloc(sizeof(*coproc), GFP_KERNEL);
+ coproc = kzalloc(sizeof(*coproc), GFP_KERNEL);
if (!coproc)
return -ENOMEM;
--
1.8.3.1
^ permalink raw reply related
* [PATCH V2 5/9] crypto/nx: Rename nx-842-powernv file name to nx-common-powernv
From: Haren Myneni @ 2020-02-29 11:15 UTC (permalink / raw)
To: herbert; +Cc: mikey, npiggin, linux-crypto, sukadev, linuxppc-dev
In-Reply-To: <1582974266.18705.28.camel@hbabu-laptop>
Rename nx-842-powernv.c to nx-common-powernv.c to add code for setup
and enable new GZIP compression type. The actual functionality is not
changed in this patch.
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
drivers/crypto/nx/Makefile | 2 +-
drivers/crypto/nx/nx-842-powernv.c | 1062 ---------------------------------
drivers/crypto/nx/nx-common-powernv.c | 1062 +++++++++++++++++++++++++++++++++
3 files changed, 1063 insertions(+), 1063 deletions(-)
delete mode 100644 drivers/crypto/nx/nx-842-powernv.c
create mode 100644 drivers/crypto/nx/nx-common-powernv.c
diff --git a/drivers/crypto/nx/Makefile b/drivers/crypto/nx/Makefile
index 015155d..bc89a20 100644
--- a/drivers/crypto/nx/Makefile
+++ b/drivers/crypto/nx/Makefile
@@ -15,4 +15,4 @@ obj-$(CONFIG_CRYPTO_DEV_NX_COMPRESS_PSERIES) += nx-compress-pseries.o nx-compres
obj-$(CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV) += nx-compress-powernv.o nx-compress.o
nx-compress-objs := nx-842.o
nx-compress-pseries-objs := nx-842-pseries.o
-nx-compress-powernv-objs := nx-842-powernv.o
+nx-compress-powernv-objs := nx-common-powernv.o
diff --git a/drivers/crypto/nx/nx-842-powernv.c b/drivers/crypto/nx/nx-842-powernv.c
deleted file mode 100644
index 8e63326..0000000
--- a/drivers/crypto/nx/nx-842-powernv.c
+++ /dev/null
@@ -1,1062 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * Driver for IBM PowerNV 842 compression accelerator
- *
- * Copyright (C) 2015 Dan Streetman, IBM Corp
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include "nx-842.h"
-
-#include <linux/timer.h>
-
-#include <asm/prom.h>
-#include <asm/icswx.h>
-#include <asm/vas.h>
-#include <asm/reg.h>
-#include <asm/opal-api.h>
-#include <asm/opal.h>
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Dan Streetman <ddstreet@ieee.org>");
-MODULE_DESCRIPTION("842 H/W Compression driver for IBM PowerNV processors");
-MODULE_ALIAS_CRYPTO("842");
-MODULE_ALIAS_CRYPTO("842-nx");
-
-#define WORKMEM_ALIGN (CRB_ALIGN)
-#define CSB_WAIT_MAX (5000) /* ms */
-#define VAS_RETRIES (10)
-
-struct nx842_workmem {
- /* Below fields must be properly aligned */
- struct coprocessor_request_block crb; /* CRB_ALIGN align */
- struct data_descriptor_entry ddl_in[DDL_LEN_MAX]; /* DDE_ALIGN align */
- struct data_descriptor_entry ddl_out[DDL_LEN_MAX]; /* DDE_ALIGN align */
- /* Above fields must be properly aligned */
-
- ktime_t start;
-
- char padding[WORKMEM_ALIGN]; /* unused, to allow alignment */
-} __packed __aligned(WORKMEM_ALIGN);
-
-struct nx842_coproc {
- unsigned int chip_id;
- unsigned int ct;
- unsigned int ci; /* Coprocessor instance, used with icswx */
- struct {
- struct vas_window *rxwin;
- int id;
- } vas;
- struct list_head list;
-};
-
-/*
- * Send the request to NX engine on the chip for the corresponding CPU
- * where the process is executing. Use with VAS function.
- */
-static DEFINE_PER_CPU(struct vas_window *, cpu_txwin);
-
-/* no cpu hotplug on powernv, so this list never changes after init */
-static LIST_HEAD(nx842_coprocs);
-static unsigned int nx842_ct; /* used in icswx function */
-
-static int (*nx842_powernv_exec)(const unsigned char *in,
- unsigned int inlen, unsigned char *out,
- unsigned int *outlenp, void *workmem, int fc);
-
-/**
- * setup_indirect_dde - Setup an indirect DDE
- *
- * The DDE is setup with the the DDE count, byte count, and address of
- * first direct DDE in the list.
- */
-static void setup_indirect_dde(struct data_descriptor_entry *dde,
- struct data_descriptor_entry *ddl,
- unsigned int dde_count, unsigned int byte_count)
-{
- dde->flags = 0;
- dde->count = dde_count;
- dde->index = 0;
- dde->length = cpu_to_be32(byte_count);
- dde->address = cpu_to_be64(nx842_get_pa(ddl));
-}
-
-/**
- * setup_direct_dde - Setup single DDE from buffer
- *
- * The DDE is setup with the buffer and length. The buffer must be properly
- * aligned. The used length is returned.
- * Returns:
- * N Successfully set up DDE with N bytes
- */
-static unsigned int setup_direct_dde(struct data_descriptor_entry *dde,
- unsigned long pa, unsigned int len)
-{
- unsigned int l = min_t(unsigned int, len, LEN_ON_PAGE(pa));
-
- dde->flags = 0;
- dde->count = 0;
- dde->index = 0;
- dde->length = cpu_to_be32(l);
- dde->address = cpu_to_be64(pa);
-
- return l;
-}
-
-/**
- * setup_ddl - Setup DDL from buffer
- *
- * Returns:
- * 0 Successfully set up DDL
- */
-static int setup_ddl(struct data_descriptor_entry *dde,
- struct data_descriptor_entry *ddl,
- unsigned char *buf, unsigned int len,
- bool in)
-{
- unsigned long pa = nx842_get_pa(buf);
- int i, ret, total_len = len;
-
- if (!IS_ALIGNED(pa, DDE_BUFFER_ALIGN)) {
- pr_debug("%s buffer pa 0x%lx not 0x%x-byte aligned\n",
- in ? "input" : "output", pa, DDE_BUFFER_ALIGN);
- return -EINVAL;
- }
-
- /* only need to check last mult; since buffer must be
- * DDE_BUFFER_ALIGN aligned, and that is a multiple of
- * DDE_BUFFER_SIZE_MULT, and pre-last page DDE buffers
- * are guaranteed a multiple of DDE_BUFFER_SIZE_MULT.
- */
- if (len % DDE_BUFFER_LAST_MULT) {
- pr_debug("%s buffer len 0x%x not a multiple of 0x%x\n",
- in ? "input" : "output", len, DDE_BUFFER_LAST_MULT);
- if (in)
- return -EINVAL;
- len = round_down(len, DDE_BUFFER_LAST_MULT);
- }
-
- /* use a single direct DDE */
- if (len <= LEN_ON_PAGE(pa)) {
- ret = setup_direct_dde(dde, pa, len);
- WARN_ON(ret < len);
- return 0;
- }
-
- /* use the DDL */
- for (i = 0; i < DDL_LEN_MAX && len > 0; i++) {
- ret = setup_direct_dde(&ddl[i], pa, len);
- buf += ret;
- len -= ret;
- pa = nx842_get_pa(buf);
- }
-
- if (len > 0) {
- pr_debug("0x%x total %s bytes 0x%x too many for DDL.\n",
- total_len, in ? "input" : "output", len);
- if (in)
- return -EMSGSIZE;
- total_len -= len;
- }
- setup_indirect_dde(dde, ddl, i, total_len);
-
- return 0;
-}
-
-#define CSB_ERR(csb, msg, ...) \
- pr_err("ERROR: " msg " : %02x %02x %02x %02x %08x\n", \
- ##__VA_ARGS__, (csb)->flags, \
- (csb)->cs, (csb)->cc, (csb)->ce, \
- be32_to_cpu((csb)->count))
-
-#define CSB_ERR_ADDR(csb, msg, ...) \
- CSB_ERR(csb, msg " at %lx", ##__VA_ARGS__, \
- (unsigned long)be64_to_cpu((csb)->address))
-
-/**
- * wait_for_csb
- */
-static int wait_for_csb(struct nx842_workmem *wmem,
- struct coprocessor_status_block *csb)
-{
- ktime_t start = wmem->start, now = ktime_get();
- ktime_t timeout = ktime_add_ms(start, CSB_WAIT_MAX);
-
- while (!(READ_ONCE(csb->flags) & CSB_V)) {
- cpu_relax();
- now = ktime_get();
- if (ktime_after(now, timeout))
- break;
- }
-
- /* hw has updated csb and output buffer */
- barrier();
-
- /* check CSB flags */
- if (!(csb->flags & CSB_V)) {
- CSB_ERR(csb, "CSB still not valid after %ld us, giving up",
- (long)ktime_us_delta(now, start));
- return -ETIMEDOUT;
- }
- if (csb->flags & CSB_F) {
- CSB_ERR(csb, "Invalid CSB format");
- return -EPROTO;
- }
- if (csb->flags & CSB_CH) {
- CSB_ERR(csb, "Invalid CSB chaining state");
- return -EPROTO;
- }
-
- /* verify CSB completion sequence is 0 */
- if (csb->cs) {
- CSB_ERR(csb, "Invalid CSB completion sequence");
- return -EPROTO;
- }
-
- /* check CSB Completion Code */
- switch (csb->cc) {
- /* no error */
- case CSB_CC_SUCCESS:
- break;
- case CSB_CC_TPBC_GT_SPBC:
- /* not an error, but the compressed data is
- * larger than the uncompressed data :(
- */
- break;
-
- /* input data errors */
- case CSB_CC_OPERAND_OVERLAP:
- /* input and output buffers overlap */
- CSB_ERR(csb, "Operand Overlap error");
- return -EINVAL;
- case CSB_CC_INVALID_OPERAND:
- CSB_ERR(csb, "Invalid operand");
- return -EINVAL;
- case CSB_CC_NOSPC:
- /* output buffer too small */
- return -ENOSPC;
- case CSB_CC_ABORT:
- CSB_ERR(csb, "Function aborted");
- return -EINTR;
- case CSB_CC_CRC_MISMATCH:
- CSB_ERR(csb, "CRC mismatch");
- return -EINVAL;
- case CSB_CC_TEMPL_INVALID:
- CSB_ERR(csb, "Compressed data template invalid");
- return -EINVAL;
- case CSB_CC_TEMPL_OVERFLOW:
- CSB_ERR(csb, "Compressed data template shows data past end");
- return -EINVAL;
- case CSB_CC_EXCEED_BYTE_COUNT: /* P9 or later */
- /*
- * DDE byte count exceeds the limit specified in Maximum
- * byte count register.
- */
- CSB_ERR(csb, "DDE byte count exceeds the limit");
- return -EINVAL;
-
- /* these should not happen */
- case CSB_CC_INVALID_ALIGN:
- /* setup_ddl should have detected this */
- CSB_ERR_ADDR(csb, "Invalid alignment");
- return -EINVAL;
- case CSB_CC_DATA_LENGTH:
- /* setup_ddl should have detected this */
- CSB_ERR(csb, "Invalid data length");
- return -EINVAL;
- case CSB_CC_WR_TRANSLATION:
- case CSB_CC_TRANSLATION:
- case CSB_CC_TRANSLATION_DUP1:
- case CSB_CC_TRANSLATION_DUP2:
- case CSB_CC_TRANSLATION_DUP3:
- case CSB_CC_TRANSLATION_DUP4:
- case CSB_CC_TRANSLATION_DUP5:
- case CSB_CC_TRANSLATION_DUP6:
- /* should not happen, we use physical addrs */
- CSB_ERR_ADDR(csb, "Translation error");
- return -EPROTO;
- case CSB_CC_WR_PROTECTION:
- case CSB_CC_PROTECTION:
- case CSB_CC_PROTECTION_DUP1:
- case CSB_CC_PROTECTION_DUP2:
- case CSB_CC_PROTECTION_DUP3:
- case CSB_CC_PROTECTION_DUP4:
- case CSB_CC_PROTECTION_DUP5:
- case CSB_CC_PROTECTION_DUP6:
- /* should not happen, we use physical addrs */
- CSB_ERR_ADDR(csb, "Protection error");
- return -EPROTO;
- case CSB_CC_PRIVILEGE:
- /* shouldn't happen, we're in HYP mode */
- CSB_ERR(csb, "Insufficient Privilege error");
- return -EPROTO;
- case CSB_CC_EXCESSIVE_DDE:
- /* shouldn't happen, setup_ddl doesn't use many dde's */
- CSB_ERR(csb, "Too many DDEs in DDL");
- return -EINVAL;
- case CSB_CC_TRANSPORT:
- case CSB_CC_INVALID_CRB: /* P9 or later */
- /* shouldn't happen, we setup CRB correctly */
- CSB_ERR(csb, "Invalid CRB");
- return -EINVAL;
- case CSB_CC_INVALID_DDE: /* P9 or later */
- /*
- * shouldn't happen, setup_direct/indirect_dde creates
- * DDE right
- */
- CSB_ERR(csb, "Invalid DDE");
- return -EINVAL;
- case CSB_CC_SEGMENTED_DDL:
- /* shouldn't happen, setup_ddl creates DDL right */
- CSB_ERR(csb, "Segmented DDL error");
- return -EINVAL;
- case CSB_CC_DDE_OVERFLOW:
- /* shouldn't happen, setup_ddl creates DDL right */
- CSB_ERR(csb, "DDE overflow error");
- return -EINVAL;
- case CSB_CC_SESSION:
- /* should not happen with ICSWX */
- CSB_ERR(csb, "Session violation error");
- return -EPROTO;
- case CSB_CC_CHAIN:
- /* should not happen, we don't use chained CRBs */
- CSB_ERR(csb, "Chained CRB error");
- return -EPROTO;
- case CSB_CC_SEQUENCE:
- /* should not happen, we don't use chained CRBs */
- CSB_ERR(csb, "CRB sequence number error");
- return -EPROTO;
- case CSB_CC_UNKNOWN_CODE:
- CSB_ERR(csb, "Unknown subfunction code");
- return -EPROTO;
-
- /* hardware errors */
- case CSB_CC_RD_EXTERNAL:
- case CSB_CC_RD_EXTERNAL_DUP1:
- case CSB_CC_RD_EXTERNAL_DUP2:
- case CSB_CC_RD_EXTERNAL_DUP3:
- CSB_ERR_ADDR(csb, "Read error outside coprocessor");
- return -EPROTO;
- case CSB_CC_WR_EXTERNAL:
- CSB_ERR_ADDR(csb, "Write error outside coprocessor");
- return -EPROTO;
- case CSB_CC_INTERNAL:
- CSB_ERR(csb, "Internal error in coprocessor");
- return -EPROTO;
- case CSB_CC_PROVISION:
- CSB_ERR(csb, "Storage provision error");
- return -EPROTO;
- case CSB_CC_HW:
- CSB_ERR(csb, "Correctable hardware error");
- return -EPROTO;
- case CSB_CC_HW_EXPIRED_TIMER: /* P9 or later */
- CSB_ERR(csb, "Job did not finish within allowed time");
- return -EPROTO;
-
- default:
- CSB_ERR(csb, "Invalid CC %d", csb->cc);
- return -EPROTO;
- }
-
- /* check Completion Extension state */
- if (csb->ce & CSB_CE_TERMINATION) {
- CSB_ERR(csb, "CSB request was terminated");
- return -EPROTO;
- }
- if (csb->ce & CSB_CE_INCOMPLETE) {
- CSB_ERR(csb, "CSB request not complete");
- return -EPROTO;
- }
- if (!(csb->ce & CSB_CE_TPBC)) {
- CSB_ERR(csb, "TPBC not provided, unknown target length");
- return -EPROTO;
- }
-
- /* successful completion */
- pr_debug_ratelimited("Processed %u bytes in %lu us\n",
- be32_to_cpu(csb->count),
- (unsigned long)ktime_us_delta(now, start));
-
- return 0;
-}
-
-static int nx842_config_crb(const unsigned char *in, unsigned int inlen,
- unsigned char *out, unsigned int outlen,
- struct nx842_workmem *wmem)
-{
- struct coprocessor_request_block *crb;
- struct coprocessor_status_block *csb;
- u64 csb_addr;
- int ret;
-
- crb = &wmem->crb;
- csb = &crb->csb;
-
- /* Clear any previous values */
- memset(crb, 0, sizeof(*crb));
-
- /* set up DDLs */
- ret = setup_ddl(&crb->source, wmem->ddl_in,
- (unsigned char *)in, inlen, true);
- if (ret)
- return ret;
-
- ret = setup_ddl(&crb->target, wmem->ddl_out,
- out, outlen, false);
- if (ret)
- return ret;
-
- /* set up CRB's CSB addr */
- csb_addr = nx842_get_pa(csb) & CRB_CSB_ADDRESS;
- csb_addr |= CRB_CSB_AT; /* Addrs are phys */
- crb->csb_addr = cpu_to_be64(csb_addr);
-
- return 0;
-}
-
-/**
- * nx842_exec_icswx - compress/decompress data using the 842 algorithm
- *
- * (De)compression provided by the NX842 coprocessor on IBM PowerNV systems.
- * This compresses or decompresses the provided input buffer into the provided
- * output buffer.
- *
- * Upon return from this function @outlen contains the length of the
- * output data. If there is an error then @outlen will be 0 and an
- * error will be specified by the return code from this function.
- *
- * The @workmem buffer should only be used by one function call at a time.
- *
- * @in: input buffer pointer
- * @inlen: input buffer size
- * @out: output buffer pointer
- * @outlenp: output buffer size pointer
- * @workmem: working memory buffer pointer, size determined by
- * nx842_powernv_driver.workmem_size
- * @fc: function code, see CCW Function Codes in nx-842.h
- *
- * Returns:
- * 0 Success, output of length @outlenp stored in the buffer at @out
- * -ENODEV Hardware unavailable
- * -ENOSPC Output buffer is to small
- * -EMSGSIZE Input buffer too large
- * -EINVAL buffer constraints do not fix nx842_constraints
- * -EPROTO hardware error during operation
- * -ETIMEDOUT hardware did not complete operation in reasonable time
- * -EINTR operation was aborted
- */
-static int nx842_exec_icswx(const unsigned char *in, unsigned int inlen,
- unsigned char *out, unsigned int *outlenp,
- void *workmem, int fc)
-{
- struct coprocessor_request_block *crb;
- struct coprocessor_status_block *csb;
- struct nx842_workmem *wmem;
- int ret;
- u32 ccw;
- unsigned int outlen = *outlenp;
-
- wmem = PTR_ALIGN(workmem, WORKMEM_ALIGN);
-
- *outlenp = 0;
-
- /* shoudn't happen, we don't load without a coproc */
- if (!nx842_ct) {
- pr_err_ratelimited("coprocessor CT is 0");
- return -ENODEV;
- }
-
- ret = nx842_config_crb(in, inlen, out, outlen, wmem);
- if (ret)
- return ret;
-
- crb = &wmem->crb;
- csb = &crb->csb;
-
- /* set up CCW */
- ccw = 0;
- ccw = SET_FIELD(CCW_CT, ccw, nx842_ct);
- ccw = SET_FIELD(CCW_CI_842, ccw, 0); /* use 0 for hw auto-selection */
- ccw = SET_FIELD(CCW_FC_842, ccw, fc);
-
- wmem->start = ktime_get();
-
- /* do ICSWX */
- ret = icswx(cpu_to_be32(ccw), crb);
-
- pr_debug_ratelimited("icswx CR %x ccw %x crb->ccw %x\n", ret,
- (unsigned int)ccw,
- (unsigned int)be32_to_cpu(crb->ccw));
-
- /*
- * NX842 coprocessor sets 3rd bit in CR register with XER[S0].
- * XER[S0] is the integer summary overflow bit which is nothing
- * to do NX. Since this bit can be set with other return values,
- * mask this bit.
- */
- ret &= ~ICSWX_XERS0;
-
- switch (ret) {
- case ICSWX_INITIATED:
- ret = wait_for_csb(wmem, csb);
- break;
- case ICSWX_BUSY:
- pr_debug_ratelimited("842 Coprocessor busy\n");
- ret = -EBUSY;
- break;
- case ICSWX_REJECTED:
- pr_err_ratelimited("ICSWX rejected\n");
- ret = -EPROTO;
- break;
- }
-
- if (!ret)
- *outlenp = be32_to_cpu(csb->count);
-
- return ret;
-}
-
-/**
- * nx842_exec_vas - compress/decompress data using the 842 algorithm
- *
- * (De)compression provided by the NX842 coprocessor on IBM PowerNV systems.
- * This compresses or decompresses the provided input buffer into the provided
- * output buffer.
- *
- * Upon return from this function @outlen contains the length of the
- * output data. If there is an error then @outlen will be 0 and an
- * error will be specified by the return code from this function.
- *
- * The @workmem buffer should only be used by one function call at a time.
- *
- * @in: input buffer pointer
- * @inlen: input buffer size
- * @out: output buffer pointer
- * @outlenp: output buffer size pointer
- * @workmem: working memory buffer pointer, size determined by
- * nx842_powernv_driver.workmem_size
- * @fc: function code, see CCW Function Codes in nx-842.h
- *
- * Returns:
- * 0 Success, output of length @outlenp stored in the buffer
- * at @out
- * -ENODEV Hardware unavailable
- * -ENOSPC Output buffer is to small
- * -EMSGSIZE Input buffer too large
- * -EINVAL buffer constraints do not fix nx842_constraints
- * -EPROTO hardware error during operation
- * -ETIMEDOUT hardware did not complete operation in reasonable time
- * -EINTR operation was aborted
- */
-static int nx842_exec_vas(const unsigned char *in, unsigned int inlen,
- unsigned char *out, unsigned int *outlenp,
- void *workmem, int fc)
-{
- struct coprocessor_request_block *crb;
- struct coprocessor_status_block *csb;
- struct nx842_workmem *wmem;
- struct vas_window *txwin;
- int ret, i = 0;
- u32 ccw;
- unsigned int outlen = *outlenp;
-
- wmem = PTR_ALIGN(workmem, WORKMEM_ALIGN);
-
- *outlenp = 0;
-
- crb = &wmem->crb;
- csb = &crb->csb;
-
- ret = nx842_config_crb(in, inlen, out, outlen, wmem);
- if (ret)
- return ret;
-
- ccw = 0;
- ccw = SET_FIELD(CCW_FC_842, ccw, fc);
- crb->ccw = cpu_to_be32(ccw);
-
- do {
- wmem->start = ktime_get();
- preempt_disable();
- txwin = this_cpu_read(cpu_txwin);
-
- /*
- * VAS copy CRB into L2 cache. Refer <asm/vas.h>.
- * @crb and @offset.
- */
- vas_copy_crb(crb, 0);
-
- /*
- * VAS paste previously copied CRB to NX.
- * @txwin, @offset and @last (must be true).
- */
- ret = vas_paste_crb(txwin, 0, 1);
- preempt_enable();
- /*
- * Retry copy/paste function for VAS failures.
- */
- } while (ret && (i++ < VAS_RETRIES));
-
- if (ret) {
- pr_err_ratelimited("VAS copy/paste failed\n");
- return ret;
- }
-
- ret = wait_for_csb(wmem, csb);
- if (!ret)
- *outlenp = be32_to_cpu(csb->count);
-
- return ret;
-}
-
-/**
- * nx842_powernv_compress - Compress data using the 842 algorithm
- *
- * Compression provided by the NX842 coprocessor on IBM PowerNV systems.
- * The input buffer is compressed and the result is stored in the
- * provided output buffer.
- *
- * Upon return from this function @outlen contains the length of the
- * compressed data. If there is an error then @outlen will be 0 and an
- * error will be specified by the return code from this function.
- *
- * @in: input buffer pointer
- * @inlen: input buffer size
- * @out: output buffer pointer
- * @outlenp: output buffer size pointer
- * @workmem: working memory buffer pointer, size determined by
- * nx842_powernv_driver.workmem_size
- *
- * Returns: see @nx842_powernv_exec()
- */
-static int nx842_powernv_compress(const unsigned char *in, unsigned int inlen,
- unsigned char *out, unsigned int *outlenp,
- void *wmem)
-{
- return nx842_powernv_exec(in, inlen, out, outlenp,
- wmem, CCW_FC_842_COMP_CRC);
-}
-
-/**
- * nx842_powernv_decompress - Decompress data using the 842 algorithm
- *
- * Decompression provided by the NX842 coprocessor on IBM PowerNV systems.
- * The input buffer is decompressed and the result is stored in the
- * provided output buffer.
- *
- * Upon return from this function @outlen contains the length of the
- * decompressed data. If there is an error then @outlen will be 0 and an
- * error will be specified by the return code from this function.
- *
- * @in: input buffer pointer
- * @inlen: input buffer size
- * @out: output buffer pointer
- * @outlenp: output buffer size pointer
- * @workmem: working memory buffer pointer, size determined by
- * nx842_powernv_driver.workmem_size
- *
- * Returns: see @nx842_powernv_exec()
- */
-static int nx842_powernv_decompress(const unsigned char *in, unsigned int inlen,
- unsigned char *out, unsigned int *outlenp,
- void *wmem)
-{
- return nx842_powernv_exec(in, inlen, out, outlenp,
- wmem, CCW_FC_842_DECOMP_CRC);
-}
-
-static inline void nx842_add_coprocs_list(struct nx842_coproc *coproc,
- int chipid)
-{
- coproc->chip_id = chipid;
- INIT_LIST_HEAD(&coproc->list);
- list_add(&coproc->list, &nx842_coprocs);
-}
-
-static struct vas_window *nx842_alloc_txwin(struct nx842_coproc *coproc)
-{
- struct vas_window *txwin = NULL;
- struct vas_tx_win_attr txattr;
-
- /*
- * Kernel requests will be high priority. So open send
- * windows only for high priority RxFIFO entries.
- */
- vas_init_tx_win_attr(&txattr, coproc->ct);
- txattr.lpid = 0; /* lpid is 0 for kernel requests */
- txattr.pid = 0; /* pid is 0 for kernel requests */
-
- /*
- * Open a VAS send window which is used to send request to NX.
- */
- txwin = vas_tx_win_open(coproc->vas.id, coproc->ct, &txattr);
- if (IS_ERR(txwin))
- pr_err("ibm,nx-842: Can not open TX window: %ld\n",
- PTR_ERR(txwin));
-
- return txwin;
-}
-
-/*
- * Identify chip ID for each CPU, open send wndow for the corresponding NX
- * engine and save txwin in percpu cpu_txwin.
- * cpu_txwin is used in copy/paste operation for each compression /
- * decompression request.
- */
-static int nx842_open_percpu_txwins(void)
-{
- struct nx842_coproc *coproc, *n;
- unsigned int i, chip_id;
-
- for_each_possible_cpu(i) {
- struct vas_window *txwin = NULL;
-
- chip_id = cpu_to_chip_id(i);
-
- list_for_each_entry_safe(coproc, n, &nx842_coprocs, list) {
- /*
- * Kernel requests use only high priority FIFOs. So
- * open send windows for these FIFOs.
- */
-
- if (coproc->ct != VAS_COP_TYPE_842_HIPRI)
- continue;
-
- if (coproc->chip_id == chip_id) {
- txwin = nx842_alloc_txwin(coproc);
- if (IS_ERR(txwin))
- return PTR_ERR(txwin);
-
- per_cpu(cpu_txwin, i) = txwin;
- break;
- }
- }
-
- if (!per_cpu(cpu_txwin, i)) {
- /* shouldn't happen, Each chip will have NX engine */
- pr_err("NX engine is not available for CPU %d\n", i);
- return -EINVAL;
- }
- }
-
- return 0;
-}
-
-static int __init vas_cfg_coproc_info(struct device_node *dn, int chip_id,
- int vasid, int *ct)
-{
- struct vas_window *rxwin = NULL;
- struct vas_rx_win_attr rxattr;
- struct nx842_coproc *coproc;
- u32 lpid, pid, tid, fifo_size;
- u64 rx_fifo;
- const char *priority;
- int ret;
-
- ret = of_property_read_u64(dn, "rx-fifo-address", &rx_fifo);
- if (ret) {
- pr_err("Missing rx-fifo-address property\n");
- return ret;
- }
-
- ret = of_property_read_u32(dn, "rx-fifo-size", &fifo_size);
- if (ret) {
- pr_err("Missing rx-fifo-size property\n");
- return ret;
- }
-
- ret = of_property_read_u32(dn, "lpid", &lpid);
- if (ret) {
- pr_err("Missing lpid property\n");
- return ret;
- }
-
- ret = of_property_read_u32(dn, "pid", &pid);
- if (ret) {
- pr_err("Missing pid property\n");
- return ret;
- }
-
- ret = of_property_read_u32(dn, "tid", &tid);
- if (ret) {
- pr_err("Missing tid property\n");
- return ret;
- }
-
- ret = of_property_read_string(dn, "priority", &priority);
- if (ret) {
- pr_err("Missing priority property\n");
- return ret;
- }
-
- coproc = kzalloc(sizeof(*coproc), GFP_KERNEL);
- if (!coproc)
- return -ENOMEM;
-
- if (!strcmp(priority, "High"))
- coproc->ct = VAS_COP_TYPE_842_HIPRI;
- else if (!strcmp(priority, "Normal"))
- coproc->ct = VAS_COP_TYPE_842;
- else {
- pr_err("Invalid RxFIFO priority value\n");
- ret = -EINVAL;
- goto err_out;
- }
-
- vas_init_rx_win_attr(&rxattr, coproc->ct);
- rxattr.rx_fifo = (void *)rx_fifo;
- rxattr.rx_fifo_size = fifo_size;
- rxattr.lnotify_lpid = lpid;
- rxattr.lnotify_pid = pid;
- rxattr.lnotify_tid = tid;
- /*
- * Maximum RX window credits can not be more than #CRBs in
- * RxFIFO. Otherwise, can get checkstop if RxFIFO overruns.
- */
- rxattr.wcreds_max = fifo_size / CRB_SIZE;
-
- /*
- * Open a VAS receice window which is used to configure RxFIFO
- * for NX.
- */
- rxwin = vas_rx_win_open(vasid, coproc->ct, &rxattr);
- if (IS_ERR(rxwin)) {
- ret = PTR_ERR(rxwin);
- pr_err("setting RxFIFO with VAS failed: %d\n",
- ret);
- goto err_out;
- }
-
- coproc->vas.rxwin = rxwin;
- coproc->vas.id = vasid;
- nx842_add_coprocs_list(coproc, chip_id);
-
- /*
- * (lpid, pid, tid) combination has to be unique for each
- * coprocessor instance in the system. So to make it
- * unique, skiboot uses coprocessor type such as 842 or
- * GZIP for pid and provides this value to kernel in pid
- * device-tree property.
- */
- *ct = pid;
-
- return 0;
-
-err_out:
- kfree(coproc);
- return ret;
-}
-
-
-static int __init nx842_powernv_probe_vas(struct device_node *pn)
-{
- struct device_node *dn;
- int chip_id, vasid, ret = 0;
- int nx_fifo_found = 0;
- int uninitialized_var(ct);
-
- chip_id = of_get_ibm_chip_id(pn);
- if (chip_id < 0) {
- pr_err("ibm,chip-id missing\n");
- return -EINVAL;
- }
-
- vasid = chip_to_vas_id(chip_id);
- if (vasid < 0) {
- pr_err("Unable to map chip_id %d to vasid\n", chip_id);
- return -EINVAL;
- }
-
- for_each_child_of_node(pn, dn) {
- if (of_device_is_compatible(dn, "ibm,p9-nx-842")) {
- ret = vas_cfg_coproc_info(dn, chip_id, vasid, &ct);
- if (ret) {
- of_node_put(dn);
- return ret;
- }
- nx_fifo_found++;
- }
- }
-
- if (!nx_fifo_found) {
- pr_err("NX842 FIFO nodes are missing\n");
- return -EINVAL;
- }
-
- /*
- * Initialize NX instance for both high and normal priority FIFOs.
- */
- if (opal_check_token(OPAL_NX_COPROC_INIT)) {
- ret = opal_nx_coproc_init(chip_id, ct);
- if (ret) {
- pr_err("Failed to initialize NX for chip(%d): %d\n",
- chip_id, ret);
- ret = opal_error_code(ret);
- }
- } else
- pr_warn("Firmware doesn't support NX initialization\n");
-
- return ret;
-}
-
-static int __init nx842_powernv_probe(struct device_node *dn)
-{
- struct nx842_coproc *coproc;
- unsigned int ct, ci;
- int chip_id;
-
- chip_id = of_get_ibm_chip_id(dn);
- if (chip_id < 0) {
- pr_err("ibm,chip-id missing\n");
- return -EINVAL;
- }
-
- if (of_property_read_u32(dn, "ibm,842-coprocessor-type", &ct)) {
- pr_err("ibm,842-coprocessor-type missing\n");
- return -EINVAL;
- }
-
- if (of_property_read_u32(dn, "ibm,842-coprocessor-instance", &ci)) {
- pr_err("ibm,842-coprocessor-instance missing\n");
- return -EINVAL;
- }
-
- coproc = kzalloc(sizeof(*coproc), GFP_KERNEL);
- if (!coproc)
- return -ENOMEM;
-
- coproc->ct = ct;
- coproc->ci = ci;
- nx842_add_coprocs_list(coproc, chip_id);
-
- pr_info("coprocessor found on chip %d, CT %d CI %d\n", chip_id, ct, ci);
-
- if (!nx842_ct)
- nx842_ct = ct;
- else if (nx842_ct != ct)
- pr_err("NX842 chip %d, CT %d != first found CT %d\n",
- chip_id, ct, nx842_ct);
-
- return 0;
-}
-
-static void nx842_delete_coprocs(void)
-{
- struct nx842_coproc *coproc, *n;
- struct vas_window *txwin;
- int i;
-
- /*
- * close percpu txwins that are opened for the corresponding coproc.
- */
- for_each_possible_cpu(i) {
- txwin = per_cpu(cpu_txwin, i);
- if (txwin)
- vas_win_close(txwin);
-
- per_cpu(cpu_txwin, i) = 0;
- }
-
- list_for_each_entry_safe(coproc, n, &nx842_coprocs, list) {
- if (coproc->vas.rxwin)
- vas_win_close(coproc->vas.rxwin);
-
- list_del(&coproc->list);
- kfree(coproc);
- }
-}
-
-static struct nx842_constraints nx842_powernv_constraints = {
- .alignment = DDE_BUFFER_ALIGN,
- .multiple = DDE_BUFFER_LAST_MULT,
- .minimum = DDE_BUFFER_LAST_MULT,
- .maximum = (DDL_LEN_MAX - 1) * PAGE_SIZE,
-};
-
-static struct nx842_driver nx842_powernv_driver = {
- .name = KBUILD_MODNAME,
- .owner = THIS_MODULE,
- .workmem_size = sizeof(struct nx842_workmem),
- .constraints = &nx842_powernv_constraints,
- .compress = nx842_powernv_compress,
- .decompress = nx842_powernv_decompress,
-};
-
-static int nx842_powernv_crypto_init(struct crypto_tfm *tfm)
-{
- return nx842_crypto_init(tfm, &nx842_powernv_driver);
-}
-
-static struct crypto_alg nx842_powernv_alg = {
- .cra_name = "842",
- .cra_driver_name = "842-nx",
- .cra_priority = 300,
- .cra_flags = CRYPTO_ALG_TYPE_COMPRESS,
- .cra_ctxsize = sizeof(struct nx842_crypto_ctx),
- .cra_module = THIS_MODULE,
- .cra_init = nx842_powernv_crypto_init,
- .cra_exit = nx842_crypto_exit,
- .cra_u = { .compress = {
- .coa_compress = nx842_crypto_compress,
- .coa_decompress = nx842_crypto_decompress } }
-};
-
-static __init int nx842_powernv_init(void)
-{
- struct device_node *dn;
- int ret;
-
- /* verify workmem size/align restrictions */
- BUILD_BUG_ON(WORKMEM_ALIGN % CRB_ALIGN);
- BUILD_BUG_ON(CRB_ALIGN % DDE_ALIGN);
- BUILD_BUG_ON(CRB_SIZE % DDE_ALIGN);
- /* verify buffer size/align restrictions */
- BUILD_BUG_ON(PAGE_SIZE % DDE_BUFFER_ALIGN);
- BUILD_BUG_ON(DDE_BUFFER_ALIGN % DDE_BUFFER_SIZE_MULT);
- BUILD_BUG_ON(DDE_BUFFER_SIZE_MULT % DDE_BUFFER_LAST_MULT);
-
- for_each_compatible_node(dn, NULL, "ibm,power9-nx") {
- ret = nx842_powernv_probe_vas(dn);
- if (ret) {
- nx842_delete_coprocs();
- of_node_put(dn);
- return ret;
- }
- }
-
- if (list_empty(&nx842_coprocs)) {
- for_each_compatible_node(dn, NULL, "ibm,power-nx")
- nx842_powernv_probe(dn);
-
- if (!nx842_ct)
- return -ENODEV;
-
- nx842_powernv_exec = nx842_exec_icswx;
- } else {
- ret = nx842_open_percpu_txwins();
- if (ret) {
- nx842_delete_coprocs();
- return ret;
- }
-
- nx842_powernv_exec = nx842_exec_vas;
- }
-
- ret = crypto_register_alg(&nx842_powernv_alg);
- if (ret) {
- nx842_delete_coprocs();
- return ret;
- }
-
- return 0;
-}
-module_init(nx842_powernv_init);
-
-static void __exit nx842_powernv_exit(void)
-{
- crypto_unregister_alg(&nx842_powernv_alg);
-
- nx842_delete_coprocs();
-}
-module_exit(nx842_powernv_exit);
diff --git a/drivers/crypto/nx/nx-common-powernv.c b/drivers/crypto/nx/nx-common-powernv.c
new file mode 100644
index 0000000..f42881f
--- /dev/null
+++ b/drivers/crypto/nx/nx-common-powernv.c
@@ -0,0 +1,1062 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Driver for IBM PowerNV compression accelerator
+ *
+ * Copyright (C) 2015 Dan Streetman, IBM Corp
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include "nx-842.h"
+
+#include <linux/timer.h>
+
+#include <asm/prom.h>
+#include <asm/icswx.h>
+#include <asm/vas.h>
+#include <asm/reg.h>
+#include <asm/opal-api.h>
+#include <asm/opal.h>
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Dan Streetman <ddstreet@ieee.org>");
+MODULE_DESCRIPTION("H/W Compression driver for IBM PowerNV processors");
+MODULE_ALIAS_CRYPTO("842");
+MODULE_ALIAS_CRYPTO("842-nx");
+
+#define WORKMEM_ALIGN (CRB_ALIGN)
+#define CSB_WAIT_MAX (5000) /* ms */
+#define VAS_RETRIES (10)
+
+struct nx842_workmem {
+ /* Below fields must be properly aligned */
+ struct coprocessor_request_block crb; /* CRB_ALIGN align */
+ struct data_descriptor_entry ddl_in[DDL_LEN_MAX]; /* DDE_ALIGN align */
+ struct data_descriptor_entry ddl_out[DDL_LEN_MAX]; /* DDE_ALIGN align */
+ /* Above fields must be properly aligned */
+
+ ktime_t start;
+
+ char padding[WORKMEM_ALIGN]; /* unused, to allow alignment */
+} __packed __aligned(WORKMEM_ALIGN);
+
+struct nx842_coproc {
+ unsigned int chip_id;
+ unsigned int ct;
+ unsigned int ci; /* Coprocessor instance, used with icswx */
+ struct {
+ struct vas_window *rxwin;
+ int id;
+ } vas;
+ struct list_head list;
+};
+
+/*
+ * Send the request to NX engine on the chip for the corresponding CPU
+ * where the process is executing. Use with VAS function.
+ */
+static DEFINE_PER_CPU(struct vas_window *, cpu_txwin);
+
+/* no cpu hotplug on powernv, so this list never changes after init */
+static LIST_HEAD(nx842_coprocs);
+static unsigned int nx842_ct; /* used in icswx function */
+
+static int (*nx842_powernv_exec)(const unsigned char *in,
+ unsigned int inlen, unsigned char *out,
+ unsigned int *outlenp, void *workmem, int fc);
+
+/**
+ * setup_indirect_dde - Setup an indirect DDE
+ *
+ * The DDE is setup with the the DDE count, byte count, and address of
+ * first direct DDE in the list.
+ */
+static void setup_indirect_dde(struct data_descriptor_entry *dde,
+ struct data_descriptor_entry *ddl,
+ unsigned int dde_count, unsigned int byte_count)
+{
+ dde->flags = 0;
+ dde->count = dde_count;
+ dde->index = 0;
+ dde->length = cpu_to_be32(byte_count);
+ dde->address = cpu_to_be64(nx842_get_pa(ddl));
+}
+
+/**
+ * setup_direct_dde - Setup single DDE from buffer
+ *
+ * The DDE is setup with the buffer and length. The buffer must be properly
+ * aligned. The used length is returned.
+ * Returns:
+ * N Successfully set up DDE with N bytes
+ */
+static unsigned int setup_direct_dde(struct data_descriptor_entry *dde,
+ unsigned long pa, unsigned int len)
+{
+ unsigned int l = min_t(unsigned int, len, LEN_ON_PAGE(pa));
+
+ dde->flags = 0;
+ dde->count = 0;
+ dde->index = 0;
+ dde->length = cpu_to_be32(l);
+ dde->address = cpu_to_be64(pa);
+
+ return l;
+}
+
+/**
+ * setup_ddl - Setup DDL from buffer
+ *
+ * Returns:
+ * 0 Successfully set up DDL
+ */
+static int setup_ddl(struct data_descriptor_entry *dde,
+ struct data_descriptor_entry *ddl,
+ unsigned char *buf, unsigned int len,
+ bool in)
+{
+ unsigned long pa = nx842_get_pa(buf);
+ int i, ret, total_len = len;
+
+ if (!IS_ALIGNED(pa, DDE_BUFFER_ALIGN)) {
+ pr_debug("%s buffer pa 0x%lx not 0x%x-byte aligned\n",
+ in ? "input" : "output", pa, DDE_BUFFER_ALIGN);
+ return -EINVAL;
+ }
+
+ /* only need to check last mult; since buffer must be
+ * DDE_BUFFER_ALIGN aligned, and that is a multiple of
+ * DDE_BUFFER_SIZE_MULT, and pre-last page DDE buffers
+ * are guaranteed a multiple of DDE_BUFFER_SIZE_MULT.
+ */
+ if (len % DDE_BUFFER_LAST_MULT) {
+ pr_debug("%s buffer len 0x%x not a multiple of 0x%x\n",
+ in ? "input" : "output", len, DDE_BUFFER_LAST_MULT);
+ if (in)
+ return -EINVAL;
+ len = round_down(len, DDE_BUFFER_LAST_MULT);
+ }
+
+ /* use a single direct DDE */
+ if (len <= LEN_ON_PAGE(pa)) {
+ ret = setup_direct_dde(dde, pa, len);
+ WARN_ON(ret < len);
+ return 0;
+ }
+
+ /* use the DDL */
+ for (i = 0; i < DDL_LEN_MAX && len > 0; i++) {
+ ret = setup_direct_dde(&ddl[i], pa, len);
+ buf += ret;
+ len -= ret;
+ pa = nx842_get_pa(buf);
+ }
+
+ if (len > 0) {
+ pr_debug("0x%x total %s bytes 0x%x too many for DDL.\n",
+ total_len, in ? "input" : "output", len);
+ if (in)
+ return -EMSGSIZE;
+ total_len -= len;
+ }
+ setup_indirect_dde(dde, ddl, i, total_len);
+
+ return 0;
+}
+
+#define CSB_ERR(csb, msg, ...) \
+ pr_err("ERROR: " msg " : %02x %02x %02x %02x %08x\n", \
+ ##__VA_ARGS__, (csb)->flags, \
+ (csb)->cs, (csb)->cc, (csb)->ce, \
+ be32_to_cpu((csb)->count))
+
+#define CSB_ERR_ADDR(csb, msg, ...) \
+ CSB_ERR(csb, msg " at %lx", ##__VA_ARGS__, \
+ (unsigned long)be64_to_cpu((csb)->address))
+
+/**
+ * wait_for_csb
+ */
+static int wait_for_csb(struct nx842_workmem *wmem,
+ struct coprocessor_status_block *csb)
+{
+ ktime_t start = wmem->start, now = ktime_get();
+ ktime_t timeout = ktime_add_ms(start, CSB_WAIT_MAX);
+
+ while (!(READ_ONCE(csb->flags) & CSB_V)) {
+ cpu_relax();
+ now = ktime_get();
+ if (ktime_after(now, timeout))
+ break;
+ }
+
+ /* hw has updated csb and output buffer */
+ barrier();
+
+ /* check CSB flags */
+ if (!(csb->flags & CSB_V)) {
+ CSB_ERR(csb, "CSB still not valid after %ld us, giving up",
+ (long)ktime_us_delta(now, start));
+ return -ETIMEDOUT;
+ }
+ if (csb->flags & CSB_F) {
+ CSB_ERR(csb, "Invalid CSB format");
+ return -EPROTO;
+ }
+ if (csb->flags & CSB_CH) {
+ CSB_ERR(csb, "Invalid CSB chaining state");
+ return -EPROTO;
+ }
+
+ /* verify CSB completion sequence is 0 */
+ if (csb->cs) {
+ CSB_ERR(csb, "Invalid CSB completion sequence");
+ return -EPROTO;
+ }
+
+ /* check CSB Completion Code */
+ switch (csb->cc) {
+ /* no error */
+ case CSB_CC_SUCCESS:
+ break;
+ case CSB_CC_TPBC_GT_SPBC:
+ /* not an error, but the compressed data is
+ * larger than the uncompressed data :(
+ */
+ break;
+
+ /* input data errors */
+ case CSB_CC_OPERAND_OVERLAP:
+ /* input and output buffers overlap */
+ CSB_ERR(csb, "Operand Overlap error");
+ return -EINVAL;
+ case CSB_CC_INVALID_OPERAND:
+ CSB_ERR(csb, "Invalid operand");
+ return -EINVAL;
+ case CSB_CC_NOSPC:
+ /* output buffer too small */
+ return -ENOSPC;
+ case CSB_CC_ABORT:
+ CSB_ERR(csb, "Function aborted");
+ return -EINTR;
+ case CSB_CC_CRC_MISMATCH:
+ CSB_ERR(csb, "CRC mismatch");
+ return -EINVAL;
+ case CSB_CC_TEMPL_INVALID:
+ CSB_ERR(csb, "Compressed data template invalid");
+ return -EINVAL;
+ case CSB_CC_TEMPL_OVERFLOW:
+ CSB_ERR(csb, "Compressed data template shows data past end");
+ return -EINVAL;
+ case CSB_CC_EXCEED_BYTE_COUNT: /* P9 or later */
+ /*
+ * DDE byte count exceeds the limit specified in Maximum
+ * byte count register.
+ */
+ CSB_ERR(csb, "DDE byte count exceeds the limit");
+ return -EINVAL;
+
+ /* these should not happen */
+ case CSB_CC_INVALID_ALIGN:
+ /* setup_ddl should have detected this */
+ CSB_ERR_ADDR(csb, "Invalid alignment");
+ return -EINVAL;
+ case CSB_CC_DATA_LENGTH:
+ /* setup_ddl should have detected this */
+ CSB_ERR(csb, "Invalid data length");
+ return -EINVAL;
+ case CSB_CC_WR_TRANSLATION:
+ case CSB_CC_TRANSLATION:
+ case CSB_CC_TRANSLATION_DUP1:
+ case CSB_CC_TRANSLATION_DUP2:
+ case CSB_CC_TRANSLATION_DUP3:
+ case CSB_CC_TRANSLATION_DUP4:
+ case CSB_CC_TRANSLATION_DUP5:
+ case CSB_CC_TRANSLATION_DUP6:
+ /* should not happen, we use physical addrs */
+ CSB_ERR_ADDR(csb, "Translation error");
+ return -EPROTO;
+ case CSB_CC_WR_PROTECTION:
+ case CSB_CC_PROTECTION:
+ case CSB_CC_PROTECTION_DUP1:
+ case CSB_CC_PROTECTION_DUP2:
+ case CSB_CC_PROTECTION_DUP3:
+ case CSB_CC_PROTECTION_DUP4:
+ case CSB_CC_PROTECTION_DUP5:
+ case CSB_CC_PROTECTION_DUP6:
+ /* should not happen, we use physical addrs */
+ CSB_ERR_ADDR(csb, "Protection error");
+ return -EPROTO;
+ case CSB_CC_PRIVILEGE:
+ /* shouldn't happen, we're in HYP mode */
+ CSB_ERR(csb, "Insufficient Privilege error");
+ return -EPROTO;
+ case CSB_CC_EXCESSIVE_DDE:
+ /* shouldn't happen, setup_ddl doesn't use many dde's */
+ CSB_ERR(csb, "Too many DDEs in DDL");
+ return -EINVAL;
+ case CSB_CC_TRANSPORT:
+ case CSB_CC_INVALID_CRB: /* P9 or later */
+ /* shouldn't happen, we setup CRB correctly */
+ CSB_ERR(csb, "Invalid CRB");
+ return -EINVAL;
+ case CSB_CC_INVALID_DDE: /* P9 or later */
+ /*
+ * shouldn't happen, setup_direct/indirect_dde creates
+ * DDE right
+ */
+ CSB_ERR(csb, "Invalid DDE");
+ return -EINVAL;
+ case CSB_CC_SEGMENTED_DDL:
+ /* shouldn't happen, setup_ddl creates DDL right */
+ CSB_ERR(csb, "Segmented DDL error");
+ return -EINVAL;
+ case CSB_CC_DDE_OVERFLOW:
+ /* shouldn't happen, setup_ddl creates DDL right */
+ CSB_ERR(csb, "DDE overflow error");
+ return -EINVAL;
+ case CSB_CC_SESSION:
+ /* should not happen with ICSWX */
+ CSB_ERR(csb, "Session violation error");
+ return -EPROTO;
+ case CSB_CC_CHAIN:
+ /* should not happen, we don't use chained CRBs */
+ CSB_ERR(csb, "Chained CRB error");
+ return -EPROTO;
+ case CSB_CC_SEQUENCE:
+ /* should not happen, we don't use chained CRBs */
+ CSB_ERR(csb, "CRB sequence number error");
+ return -EPROTO;
+ case CSB_CC_UNKNOWN_CODE:
+ CSB_ERR(csb, "Unknown subfunction code");
+ return -EPROTO;
+
+ /* hardware errors */
+ case CSB_CC_RD_EXTERNAL:
+ case CSB_CC_RD_EXTERNAL_DUP1:
+ case CSB_CC_RD_EXTERNAL_DUP2:
+ case CSB_CC_RD_EXTERNAL_DUP3:
+ CSB_ERR_ADDR(csb, "Read error outside coprocessor");
+ return -EPROTO;
+ case CSB_CC_WR_EXTERNAL:
+ CSB_ERR_ADDR(csb, "Write error outside coprocessor");
+ return -EPROTO;
+ case CSB_CC_INTERNAL:
+ CSB_ERR(csb, "Internal error in coprocessor");
+ return -EPROTO;
+ case CSB_CC_PROVISION:
+ CSB_ERR(csb, "Storage provision error");
+ return -EPROTO;
+ case CSB_CC_HW:
+ CSB_ERR(csb, "Correctable hardware error");
+ return -EPROTO;
+ case CSB_CC_HW_EXPIRED_TIMER: /* P9 or later */
+ CSB_ERR(csb, "Job did not finish within allowed time");
+ return -EPROTO;
+
+ default:
+ CSB_ERR(csb, "Invalid CC %d", csb->cc);
+ return -EPROTO;
+ }
+
+ /* check Completion Extension state */
+ if (csb->ce & CSB_CE_TERMINATION) {
+ CSB_ERR(csb, "CSB request was terminated");
+ return -EPROTO;
+ }
+ if (csb->ce & CSB_CE_INCOMPLETE) {
+ CSB_ERR(csb, "CSB request not complete");
+ return -EPROTO;
+ }
+ if (!(csb->ce & CSB_CE_TPBC)) {
+ CSB_ERR(csb, "TPBC not provided, unknown target length");
+ return -EPROTO;
+ }
+
+ /* successful completion */
+ pr_debug_ratelimited("Processed %u bytes in %lu us\n",
+ be32_to_cpu(csb->count),
+ (unsigned long)ktime_us_delta(now, start));
+
+ return 0;
+}
+
+static int nx842_config_crb(const unsigned char *in, unsigned int inlen,
+ unsigned char *out, unsigned int outlen,
+ struct nx842_workmem *wmem)
+{
+ struct coprocessor_request_block *crb;
+ struct coprocessor_status_block *csb;
+ u64 csb_addr;
+ int ret;
+
+ crb = &wmem->crb;
+ csb = &crb->csb;
+
+ /* Clear any previous values */
+ memset(crb, 0, sizeof(*crb));
+
+ /* set up DDLs */
+ ret = setup_ddl(&crb->source, wmem->ddl_in,
+ (unsigned char *)in, inlen, true);
+ if (ret)
+ return ret;
+
+ ret = setup_ddl(&crb->target, wmem->ddl_out,
+ out, outlen, false);
+ if (ret)
+ return ret;
+
+ /* set up CRB's CSB addr */
+ csb_addr = nx842_get_pa(csb) & CRB_CSB_ADDRESS;
+ csb_addr |= CRB_CSB_AT; /* Addrs are phys */
+ crb->csb_addr = cpu_to_be64(csb_addr);
+
+ return 0;
+}
+
+/**
+ * nx842_exec_icswx - compress/decompress data using the 842 algorithm
+ *
+ * (De)compression provided by the NX842 coprocessor on IBM PowerNV systems.
+ * This compresses or decompresses the provided input buffer into the provided
+ * output buffer.
+ *
+ * Upon return from this function @outlen contains the length of the
+ * output data. If there is an error then @outlen will be 0 and an
+ * error will be specified by the return code from this function.
+ *
+ * The @workmem buffer should only be used by one function call at a time.
+ *
+ * @in: input buffer pointer
+ * @inlen: input buffer size
+ * @out: output buffer pointer
+ * @outlenp: output buffer size pointer
+ * @workmem: working memory buffer pointer, size determined by
+ * nx842_powernv_driver.workmem_size
+ * @fc: function code, see CCW Function Codes in nx-842.h
+ *
+ * Returns:
+ * 0 Success, output of length @outlenp stored in the buffer at @out
+ * -ENODEV Hardware unavailable
+ * -ENOSPC Output buffer is to small
+ * -EMSGSIZE Input buffer too large
+ * -EINVAL buffer constraints do not fix nx842_constraints
+ * -EPROTO hardware error during operation
+ * -ETIMEDOUT hardware did not complete operation in reasonable time
+ * -EINTR operation was aborted
+ */
+static int nx842_exec_icswx(const unsigned char *in, unsigned int inlen,
+ unsigned char *out, unsigned int *outlenp,
+ void *workmem, int fc)
+{
+ struct coprocessor_request_block *crb;
+ struct coprocessor_status_block *csb;
+ struct nx842_workmem *wmem;
+ int ret;
+ u32 ccw;
+ unsigned int outlen = *outlenp;
+
+ wmem = PTR_ALIGN(workmem, WORKMEM_ALIGN);
+
+ *outlenp = 0;
+
+ /* shoudn't happen, we don't load without a coproc */
+ if (!nx842_ct) {
+ pr_err_ratelimited("coprocessor CT is 0");
+ return -ENODEV;
+ }
+
+ ret = nx842_config_crb(in, inlen, out, outlen, wmem);
+ if (ret)
+ return ret;
+
+ crb = &wmem->crb;
+ csb = &crb->csb;
+
+ /* set up CCW */
+ ccw = 0;
+ ccw = SET_FIELD(CCW_CT, ccw, nx842_ct);
+ ccw = SET_FIELD(CCW_CI_842, ccw, 0); /* use 0 for hw auto-selection */
+ ccw = SET_FIELD(CCW_FC_842, ccw, fc);
+
+ wmem->start = ktime_get();
+
+ /* do ICSWX */
+ ret = icswx(cpu_to_be32(ccw), crb);
+
+ pr_debug_ratelimited("icswx CR %x ccw %x crb->ccw %x\n", ret,
+ (unsigned int)ccw,
+ (unsigned int)be32_to_cpu(crb->ccw));
+
+ /*
+ * NX842 coprocessor sets 3rd bit in CR register with XER[S0].
+ * XER[S0] is the integer summary overflow bit which is nothing
+ * to do NX. Since this bit can be set with other return values,
+ * mask this bit.
+ */
+ ret &= ~ICSWX_XERS0;
+
+ switch (ret) {
+ case ICSWX_INITIATED:
+ ret = wait_for_csb(wmem, csb);
+ break;
+ case ICSWX_BUSY:
+ pr_debug_ratelimited("842 Coprocessor busy\n");
+ ret = -EBUSY;
+ break;
+ case ICSWX_REJECTED:
+ pr_err_ratelimited("ICSWX rejected\n");
+ ret = -EPROTO;
+ break;
+ }
+
+ if (!ret)
+ *outlenp = be32_to_cpu(csb->count);
+
+ return ret;
+}
+
+/**
+ * nx842_exec_vas - compress/decompress data using the 842 algorithm
+ *
+ * (De)compression provided by the NX842 coprocessor on IBM PowerNV systems.
+ * This compresses or decompresses the provided input buffer into the provided
+ * output buffer.
+ *
+ * Upon return from this function @outlen contains the length of the
+ * output data. If there is an error then @outlen will be 0 and an
+ * error will be specified by the return code from this function.
+ *
+ * The @workmem buffer should only be used by one function call at a time.
+ *
+ * @in: input buffer pointer
+ * @inlen: input buffer size
+ * @out: output buffer pointer
+ * @outlenp: output buffer size pointer
+ * @workmem: working memory buffer pointer, size determined by
+ * nx842_powernv_driver.workmem_size
+ * @fc: function code, see CCW Function Codes in nx-842.h
+ *
+ * Returns:
+ * 0 Success, output of length @outlenp stored in the buffer
+ * at @out
+ * -ENODEV Hardware unavailable
+ * -ENOSPC Output buffer is to small
+ * -EMSGSIZE Input buffer too large
+ * -EINVAL buffer constraints do not fix nx842_constraints
+ * -EPROTO hardware error during operation
+ * -ETIMEDOUT hardware did not complete operation in reasonable time
+ * -EINTR operation was aborted
+ */
+static int nx842_exec_vas(const unsigned char *in, unsigned int inlen,
+ unsigned char *out, unsigned int *outlenp,
+ void *workmem, int fc)
+{
+ struct coprocessor_request_block *crb;
+ struct coprocessor_status_block *csb;
+ struct nx842_workmem *wmem;
+ struct vas_window *txwin;
+ int ret, i = 0;
+ u32 ccw;
+ unsigned int outlen = *outlenp;
+
+ wmem = PTR_ALIGN(workmem, WORKMEM_ALIGN);
+
+ *outlenp = 0;
+
+ crb = &wmem->crb;
+ csb = &crb->csb;
+
+ ret = nx842_config_crb(in, inlen, out, outlen, wmem);
+ if (ret)
+ return ret;
+
+ ccw = 0;
+ ccw = SET_FIELD(CCW_FC_842, ccw, fc);
+ crb->ccw = cpu_to_be32(ccw);
+
+ do {
+ wmem->start = ktime_get();
+ preempt_disable();
+ txwin = this_cpu_read(cpu_txwin);
+
+ /*
+ * VAS copy CRB into L2 cache. Refer <asm/vas.h>.
+ * @crb and @offset.
+ */
+ vas_copy_crb(crb, 0);
+
+ /*
+ * VAS paste previously copied CRB to NX.
+ * @txwin, @offset and @last (must be true).
+ */
+ ret = vas_paste_crb(txwin, 0, 1);
+ preempt_enable();
+ /*
+ * Retry copy/paste function for VAS failures.
+ */
+ } while (ret && (i++ < VAS_RETRIES));
+
+ if (ret) {
+ pr_err_ratelimited("VAS copy/paste failed\n");
+ return ret;
+ }
+
+ ret = wait_for_csb(wmem, csb);
+ if (!ret)
+ *outlenp = be32_to_cpu(csb->count);
+
+ return ret;
+}
+
+/**
+ * nx842_powernv_compress - Compress data using the 842 algorithm
+ *
+ * Compression provided by the NX842 coprocessor on IBM PowerNV systems.
+ * The input buffer is compressed and the result is stored in the
+ * provided output buffer.
+ *
+ * Upon return from this function @outlen contains the length of the
+ * compressed data. If there is an error then @outlen will be 0 and an
+ * error will be specified by the return code from this function.
+ *
+ * @in: input buffer pointer
+ * @inlen: input buffer size
+ * @out: output buffer pointer
+ * @outlenp: output buffer size pointer
+ * @workmem: working memory buffer pointer, size determined by
+ * nx842_powernv_driver.workmem_size
+ *
+ * Returns: see @nx842_powernv_exec()
+ */
+static int nx842_powernv_compress(const unsigned char *in, unsigned int inlen,
+ unsigned char *out, unsigned int *outlenp,
+ void *wmem)
+{
+ return nx842_powernv_exec(in, inlen, out, outlenp,
+ wmem, CCW_FC_842_COMP_CRC);
+}
+
+/**
+ * nx842_powernv_decompress - Decompress data using the 842 algorithm
+ *
+ * Decompression provided by the NX842 coprocessor on IBM PowerNV systems.
+ * The input buffer is decompressed and the result is stored in the
+ * provided output buffer.
+ *
+ * Upon return from this function @outlen contains the length of the
+ * decompressed data. If there is an error then @outlen will be 0 and an
+ * error will be specified by the return code from this function.
+ *
+ * @in: input buffer pointer
+ * @inlen: input buffer size
+ * @out: output buffer pointer
+ * @outlenp: output buffer size pointer
+ * @workmem: working memory buffer pointer, size determined by
+ * nx842_powernv_driver.workmem_size
+ *
+ * Returns: see @nx842_powernv_exec()
+ */
+static int nx842_powernv_decompress(const unsigned char *in, unsigned int inlen,
+ unsigned char *out, unsigned int *outlenp,
+ void *wmem)
+{
+ return nx842_powernv_exec(in, inlen, out, outlenp,
+ wmem, CCW_FC_842_DECOMP_CRC);
+}
+
+static inline void nx842_add_coprocs_list(struct nx842_coproc *coproc,
+ int chipid)
+{
+ coproc->chip_id = chipid;
+ INIT_LIST_HEAD(&coproc->list);
+ list_add(&coproc->list, &nx842_coprocs);
+}
+
+static struct vas_window *nx842_alloc_txwin(struct nx842_coproc *coproc)
+{
+ struct vas_window *txwin = NULL;
+ struct vas_tx_win_attr txattr;
+
+ /*
+ * Kernel requests will be high priority. So open send
+ * windows only for high priority RxFIFO entries.
+ */
+ vas_init_tx_win_attr(&txattr, coproc->ct);
+ txattr.lpid = 0; /* lpid is 0 for kernel requests */
+ txattr.pid = 0; /* pid is 0 for kernel requests */
+
+ /*
+ * Open a VAS send window which is used to send request to NX.
+ */
+ txwin = vas_tx_win_open(coproc->vas.id, coproc->ct, &txattr);
+ if (IS_ERR(txwin))
+ pr_err("ibm,nx-842: Can not open TX window: %ld\n",
+ PTR_ERR(txwin));
+
+ return txwin;
+}
+
+/*
+ * Identify chip ID for each CPU, open send wndow for the corresponding NX
+ * engine and save txwin in percpu cpu_txwin.
+ * cpu_txwin is used in copy/paste operation for each compression /
+ * decompression request.
+ */
+static int nx842_open_percpu_txwins(void)
+{
+ struct nx842_coproc *coproc, *n;
+ unsigned int i, chip_id;
+
+ for_each_possible_cpu(i) {
+ struct vas_window *txwin = NULL;
+
+ chip_id = cpu_to_chip_id(i);
+
+ list_for_each_entry_safe(coproc, n, &nx842_coprocs, list) {
+ /*
+ * Kernel requests use only high priority FIFOs. So
+ * open send windows for these FIFOs.
+ */
+
+ if (coproc->ct != VAS_COP_TYPE_842_HIPRI)
+ continue;
+
+ if (coproc->chip_id == chip_id) {
+ txwin = nx842_alloc_txwin(coproc);
+ if (IS_ERR(txwin))
+ return PTR_ERR(txwin);
+
+ per_cpu(cpu_txwin, i) = txwin;
+ break;
+ }
+ }
+
+ if (!per_cpu(cpu_txwin, i)) {
+ /* shouldn't happen, Each chip will have NX engine */
+ pr_err("NX engine is not available for CPU %d\n", i);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static int __init vas_cfg_coproc_info(struct device_node *dn, int chip_id,
+ int vasid, int *ct)
+{
+ struct vas_window *rxwin = NULL;
+ struct vas_rx_win_attr rxattr;
+ struct nx842_coproc *coproc;
+ u32 lpid, pid, tid, fifo_size;
+ u64 rx_fifo;
+ const char *priority;
+ int ret;
+
+ ret = of_property_read_u64(dn, "rx-fifo-address", &rx_fifo);
+ if (ret) {
+ pr_err("Missing rx-fifo-address property\n");
+ return ret;
+ }
+
+ ret = of_property_read_u32(dn, "rx-fifo-size", &fifo_size);
+ if (ret) {
+ pr_err("Missing rx-fifo-size property\n");
+ return ret;
+ }
+
+ ret = of_property_read_u32(dn, "lpid", &lpid);
+ if (ret) {
+ pr_err("Missing lpid property\n");
+ return ret;
+ }
+
+ ret = of_property_read_u32(dn, "pid", &pid);
+ if (ret) {
+ pr_err("Missing pid property\n");
+ return ret;
+ }
+
+ ret = of_property_read_u32(dn, "tid", &tid);
+ if (ret) {
+ pr_err("Missing tid property\n");
+ return ret;
+ }
+
+ ret = of_property_read_string(dn, "priority", &priority);
+ if (ret) {
+ pr_err("Missing priority property\n");
+ return ret;
+ }
+
+ coproc = kzalloc(sizeof(*coproc), GFP_KERNEL);
+ if (!coproc)
+ return -ENOMEM;
+
+ if (!strcmp(priority, "High"))
+ coproc->ct = VAS_COP_TYPE_842_HIPRI;
+ else if (!strcmp(priority, "Normal"))
+ coproc->ct = VAS_COP_TYPE_842;
+ else {
+ pr_err("Invalid RxFIFO priority value\n");
+ ret = -EINVAL;
+ goto err_out;
+ }
+
+ vas_init_rx_win_attr(&rxattr, coproc->ct);
+ rxattr.rx_fifo = (void *)rx_fifo;
+ rxattr.rx_fifo_size = fifo_size;
+ rxattr.lnotify_lpid = lpid;
+ rxattr.lnotify_pid = pid;
+ rxattr.lnotify_tid = tid;
+ /*
+ * Maximum RX window credits can not be more than #CRBs in
+ * RxFIFO. Otherwise, can get checkstop if RxFIFO overruns.
+ */
+ rxattr.wcreds_max = fifo_size / CRB_SIZE;
+
+ /*
+ * Open a VAS receice window which is used to configure RxFIFO
+ * for NX.
+ */
+ rxwin = vas_rx_win_open(vasid, coproc->ct, &rxattr);
+ if (IS_ERR(rxwin)) {
+ ret = PTR_ERR(rxwin);
+ pr_err("setting RxFIFO with VAS failed: %d\n",
+ ret);
+ goto err_out;
+ }
+
+ coproc->vas.rxwin = rxwin;
+ coproc->vas.id = vasid;
+ nx842_add_coprocs_list(coproc, chip_id);
+
+ /*
+ * (lpid, pid, tid) combination has to be unique for each
+ * coprocessor instance in the system. So to make it
+ * unique, skiboot uses coprocessor type such as 842 or
+ * GZIP for pid and provides this value to kernel in pid
+ * device-tree property.
+ */
+ *ct = pid;
+
+ return 0;
+
+err_out:
+ kfree(coproc);
+ return ret;
+}
+
+
+static int __init nx842_powernv_probe_vas(struct device_node *pn)
+{
+ struct device_node *dn;
+ int chip_id, vasid, ret = 0;
+ int nx_fifo_found = 0;
+ int uninitialized_var(ct);
+
+ chip_id = of_get_ibm_chip_id(pn);
+ if (chip_id < 0) {
+ pr_err("ibm,chip-id missing\n");
+ return -EINVAL;
+ }
+
+ vasid = chip_to_vas_id(chip_id);
+ if (vasid < 0) {
+ pr_err("Unable to map chip_id %d to vasid\n", chip_id);
+ return -EINVAL;
+ }
+
+ for_each_child_of_node(pn, dn) {
+ if (of_device_is_compatible(dn, "ibm,p9-nx-842")) {
+ ret = vas_cfg_coproc_info(dn, chip_id, vasid, &ct);
+ if (ret) {
+ of_node_put(dn);
+ return ret;
+ }
+ nx_fifo_found++;
+ }
+ }
+
+ if (!nx_fifo_found) {
+ pr_err("NX842 FIFO nodes are missing\n");
+ return -EINVAL;
+ }
+
+ /*
+ * Initialize NX instance for both high and normal priority FIFOs.
+ */
+ if (opal_check_token(OPAL_NX_COPROC_INIT)) {
+ ret = opal_nx_coproc_init(chip_id, ct);
+ if (ret) {
+ pr_err("Failed to initialize NX for chip(%d): %d\n",
+ chip_id, ret);
+ ret = opal_error_code(ret);
+ }
+ } else
+ pr_warn("Firmware doesn't support NX initialization\n");
+
+ return ret;
+}
+
+static int __init nx842_powernv_probe(struct device_node *dn)
+{
+ struct nx842_coproc *coproc;
+ unsigned int ct, ci;
+ int chip_id;
+
+ chip_id = of_get_ibm_chip_id(dn);
+ if (chip_id < 0) {
+ pr_err("ibm,chip-id missing\n");
+ return -EINVAL;
+ }
+
+ if (of_property_read_u32(dn, "ibm,842-coprocessor-type", &ct)) {
+ pr_err("ibm,842-coprocessor-type missing\n");
+ return -EINVAL;
+ }
+
+ if (of_property_read_u32(dn, "ibm,842-coprocessor-instance", &ci)) {
+ pr_err("ibm,842-coprocessor-instance missing\n");
+ return -EINVAL;
+ }
+
+ coproc = kzalloc(sizeof(*coproc), GFP_KERNEL);
+ if (!coproc)
+ return -ENOMEM;
+
+ coproc->ct = ct;
+ coproc->ci = ci;
+ nx842_add_coprocs_list(coproc, chip_id);
+
+ pr_info("coprocessor found on chip %d, CT %d CI %d\n", chip_id, ct, ci);
+
+ if (!nx842_ct)
+ nx842_ct = ct;
+ else if (nx842_ct != ct)
+ pr_err("NX842 chip %d, CT %d != first found CT %d\n",
+ chip_id, ct, nx842_ct);
+
+ return 0;
+}
+
+static void nx842_delete_coprocs(void)
+{
+ struct nx842_coproc *coproc, *n;
+ struct vas_window *txwin;
+ int i;
+
+ /*
+ * close percpu txwins that are opened for the corresponding coproc.
+ */
+ for_each_possible_cpu(i) {
+ txwin = per_cpu(cpu_txwin, i);
+ if (txwin)
+ vas_win_close(txwin);
+
+ per_cpu(cpu_txwin, i) = 0;
+ }
+
+ list_for_each_entry_safe(coproc, n, &nx842_coprocs, list) {
+ if (coproc->vas.rxwin)
+ vas_win_close(coproc->vas.rxwin);
+
+ list_del(&coproc->list);
+ kfree(coproc);
+ }
+}
+
+static struct nx842_constraints nx842_powernv_constraints = {
+ .alignment = DDE_BUFFER_ALIGN,
+ .multiple = DDE_BUFFER_LAST_MULT,
+ .minimum = DDE_BUFFER_LAST_MULT,
+ .maximum = (DDL_LEN_MAX - 1) * PAGE_SIZE,
+};
+
+static struct nx842_driver nx842_powernv_driver = {
+ .name = KBUILD_MODNAME,
+ .owner = THIS_MODULE,
+ .workmem_size = sizeof(struct nx842_workmem),
+ .constraints = &nx842_powernv_constraints,
+ .compress = nx842_powernv_compress,
+ .decompress = nx842_powernv_decompress,
+};
+
+static int nx842_powernv_crypto_init(struct crypto_tfm *tfm)
+{
+ return nx842_crypto_init(tfm, &nx842_powernv_driver);
+}
+
+static struct crypto_alg nx842_powernv_alg = {
+ .cra_name = "842",
+ .cra_driver_name = "842-nx",
+ .cra_priority = 300,
+ .cra_flags = CRYPTO_ALG_TYPE_COMPRESS,
+ .cra_ctxsize = sizeof(struct nx842_crypto_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_init = nx842_powernv_crypto_init,
+ .cra_exit = nx842_crypto_exit,
+ .cra_u = { .compress = {
+ .coa_compress = nx842_crypto_compress,
+ .coa_decompress = nx842_crypto_decompress } }
+};
+
+static __init int nx842_powernv_init(void)
+{
+ struct device_node *dn;
+ int ret;
+
+ /* verify workmem size/align restrictions */
+ BUILD_BUG_ON(WORKMEM_ALIGN % CRB_ALIGN);
+ BUILD_BUG_ON(CRB_ALIGN % DDE_ALIGN);
+ BUILD_BUG_ON(CRB_SIZE % DDE_ALIGN);
+ /* verify buffer size/align restrictions */
+ BUILD_BUG_ON(PAGE_SIZE % DDE_BUFFER_ALIGN);
+ BUILD_BUG_ON(DDE_BUFFER_ALIGN % DDE_BUFFER_SIZE_MULT);
+ BUILD_BUG_ON(DDE_BUFFER_SIZE_MULT % DDE_BUFFER_LAST_MULT);
+
+ for_each_compatible_node(dn, NULL, "ibm,power9-nx") {
+ ret = nx842_powernv_probe_vas(dn);
+ if (ret) {
+ nx842_delete_coprocs();
+ of_node_put(dn);
+ return ret;
+ }
+ }
+
+ if (list_empty(&nx842_coprocs)) {
+ for_each_compatible_node(dn, NULL, "ibm,power-nx")
+ nx842_powernv_probe(dn);
+
+ if (!nx842_ct)
+ return -ENODEV;
+
+ nx842_powernv_exec = nx842_exec_icswx;
+ } else {
+ ret = nx842_open_percpu_txwins();
+ if (ret) {
+ nx842_delete_coprocs();
+ return ret;
+ }
+
+ nx842_powernv_exec = nx842_exec_vas;
+ }
+
+ ret = crypto_register_alg(&nx842_powernv_alg);
+ if (ret) {
+ nx842_delete_coprocs();
+ return ret;
+ }
+
+ return 0;
+}
+module_init(nx842_powernv_init);
+
+static void __exit nx842_powernv_exit(void)
+{
+ crypto_unregister_alg(&nx842_powernv_alg);
+
+ nx842_delete_coprocs();
+}
+module_exit(nx842_powernv_exit);
--
1.8.3.1
^ permalink raw reply related
* [PATCH V2 6/9] crypto/nx: Make enable code generic to add new GZIP compression type
From: Haren Myneni @ 2020-02-29 11:16 UTC (permalink / raw)
To: herbert; +Cc: mikey, npiggin, linux-crypto, sukadev, linuxppc-dev
In-Reply-To: <1582974266.18705.28.camel@hbabu-laptop>
Make setup and enable code generic to support new GZIP compression type.
Changed nx842 reference to nx and moved some code to new functions. The
actual functionality is not changed.
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
drivers/crypto/nx/nx-common-powernv.c | 159 +++++++++++++++++++++-------------
1 file changed, 100 insertions(+), 59 deletions(-)
diff --git a/drivers/crypto/nx/nx-common-powernv.c b/drivers/crypto/nx/nx-common-powernv.c
index f42881f..1cd4f40 100644
--- a/drivers/crypto/nx/nx-common-powernv.c
+++ b/drivers/crypto/nx/nx-common-powernv.c
@@ -40,9 +40,9 @@ struct nx842_workmem {
char padding[WORKMEM_ALIGN]; /* unused, to allow alignment */
} __packed __aligned(WORKMEM_ALIGN);
-struct nx842_coproc {
+struct nx_coproc {
unsigned int chip_id;
- unsigned int ct;
+ unsigned int ct; /* Can be 842 or GZIP high/normal*/
unsigned int ci; /* Coprocessor instance, used with icswx */
struct {
struct vas_window *rxwin;
@@ -58,9 +58,15 @@ struct nx842_coproc {
static DEFINE_PER_CPU(struct vas_window *, cpu_txwin);
/* no cpu hotplug on powernv, so this list never changes after init */
-static LIST_HEAD(nx842_coprocs);
+static LIST_HEAD(nx_coprocs);
static unsigned int nx842_ct; /* used in icswx function */
+/*
+ * Using same values as in skiboot or coprocessor type representing
+ * in NX workbook.
+ */
+#define NX_CT_842 (3)
+
static int (*nx842_powernv_exec)(const unsigned char *in,
unsigned int inlen, unsigned char *out,
unsigned int *outlenp, void *workmem, int fc);
@@ -666,15 +672,15 @@ static int nx842_powernv_decompress(const unsigned char *in, unsigned int inlen,
wmem, CCW_FC_842_DECOMP_CRC);
}
-static inline void nx842_add_coprocs_list(struct nx842_coproc *coproc,
+static inline void nx_add_coprocs_list(struct nx_coproc *coproc,
int chipid)
{
coproc->chip_id = chipid;
INIT_LIST_HEAD(&coproc->list);
- list_add(&coproc->list, &nx842_coprocs);
+ list_add(&coproc->list, &nx_coprocs);
}
-static struct vas_window *nx842_alloc_txwin(struct nx842_coproc *coproc)
+static struct vas_window *nx_alloc_txwin(struct nx_coproc *coproc)
{
struct vas_window *txwin = NULL;
struct vas_tx_win_attr txattr;
@@ -704,9 +710,9 @@ static struct vas_window *nx842_alloc_txwin(struct nx842_coproc *coproc)
* cpu_txwin is used in copy/paste operation for each compression /
* decompression request.
*/
-static int nx842_open_percpu_txwins(void)
+static int nx_open_percpu_txwins(void)
{
- struct nx842_coproc *coproc, *n;
+ struct nx_coproc *coproc, *n;
unsigned int i, chip_id;
for_each_possible_cpu(i) {
@@ -714,17 +720,18 @@ static int nx842_open_percpu_txwins(void)
chip_id = cpu_to_chip_id(i);
- list_for_each_entry_safe(coproc, n, &nx842_coprocs, list) {
+ list_for_each_entry_safe(coproc, n, &nx_coprocs, list) {
/*
* Kernel requests use only high priority FIFOs. So
* open send windows for these FIFOs.
+ * GZIP is not supported in kernel right now.
*/
if (coproc->ct != VAS_COP_TYPE_842_HIPRI)
continue;
if (coproc->chip_id == chip_id) {
- txwin = nx842_alloc_txwin(coproc);
+ txwin = nx_alloc_txwin(coproc);
if (IS_ERR(txwin))
return PTR_ERR(txwin);
@@ -743,13 +750,28 @@ static int nx842_open_percpu_txwins(void)
return 0;
}
+static int __init nx_set_ct(struct nx_coproc *coproc, const char *priority,
+ int high, int normal)
+{
+ if (!strcmp(priority, "High"))
+ coproc->ct = high;
+ else if (!strcmp(priority, "Normal"))
+ coproc->ct = normal;
+ else {
+ pr_err("Invalid RxFIFO priority value\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int __init vas_cfg_coproc_info(struct device_node *dn, int chip_id,
- int vasid, int *ct)
+ int vasid, int type, int *ct)
{
struct vas_window *rxwin = NULL;
struct vas_rx_win_attr rxattr;
- struct nx842_coproc *coproc;
u32 lpid, pid, tid, fifo_size;
+ struct nx_coproc *coproc;
u64 rx_fifo;
const char *priority;
int ret;
@@ -794,15 +816,12 @@ static int __init vas_cfg_coproc_info(struct device_node *dn, int chip_id,
if (!coproc)
return -ENOMEM;
- if (!strcmp(priority, "High"))
- coproc->ct = VAS_COP_TYPE_842_HIPRI;
- else if (!strcmp(priority, "Normal"))
- coproc->ct = VAS_COP_TYPE_842;
- else {
- pr_err("Invalid RxFIFO priority value\n");
- ret = -EINVAL;
+ if (type == NX_CT_842)
+ ret = nx_set_ct(coproc, priority, VAS_COP_TYPE_842_HIPRI,
+ VAS_COP_TYPE_842);
+
+ if (ret)
goto err_out;
- }
vas_init_rx_win_attr(&rxattr, coproc->ct);
rxattr.rx_fifo = (void *)rx_fifo;
@@ -830,7 +849,7 @@ static int __init vas_cfg_coproc_info(struct device_node *dn, int chip_id,
coproc->vas.rxwin = rxwin;
coproc->vas.id = vasid;
- nx842_add_coprocs_list(coproc, chip_id);
+ nx_add_coprocs_list(coproc, chip_id);
/*
* (lpid, pid, tid) combination has to be unique for each
@@ -848,13 +867,43 @@ static int __init vas_cfg_coproc_info(struct device_node *dn, int chip_id,
return ret;
}
+static int __init nx_coproc_init(int chip_id, int ct_842)
+{
+ int ret = 0;
-static int __init nx842_powernv_probe_vas(struct device_node *pn)
+ if (opal_check_token(OPAL_NX_COPROC_INIT)) {
+ ret = opal_nx_coproc_init(chip_id, ct_842);
+ if (ret) {
+ ret = opal_error_code(ret);
+ pr_err("Failed to initialize NX for chip(%d): %d\n",
+ chip_id, ret);
+ }
+ } else
+ pr_warn("Firmware doesn't support NX initialization\n");
+
+ return ret;
+}
+
+static int __init find_nx_device_tree(struct device_node *dn, int chip_id,
+ int vasid, int type, char *devname,
+ int *ct)
+{
+ int ret = 0;
+
+ if (of_device_is_compatible(dn, devname)) {
+ ret = vas_cfg_coproc_info(dn, chip_id, vasid, type, ct);
+ if (ret)
+ of_node_put(dn);
+ }
+
+ return ret;
+}
+
+static int __init nx_powernv_probe_vas(struct device_node *pn)
{
- struct device_node *dn;
int chip_id, vasid, ret = 0;
- int nx_fifo_found = 0;
- int uninitialized_var(ct);
+ struct device_node *dn;
+ int ct_842 = 0;
chip_id = of_get_ibm_chip_id(pn);
if (chip_id < 0) {
@@ -869,17 +918,13 @@ static int __init nx842_powernv_probe_vas(struct device_node *pn)
}
for_each_child_of_node(pn, dn) {
- if (of_device_is_compatible(dn, "ibm,p9-nx-842")) {
- ret = vas_cfg_coproc_info(dn, chip_id, vasid, &ct);
- if (ret) {
- of_node_put(dn);
- return ret;
- }
- nx_fifo_found++;
- }
+ ret = find_nx_device_tree(dn, chip_id, vasid, NX_CT_842,
+ "ibm,p9-nx-842", &ct_842);
+ if (ret)
+ return ret;
}
- if (!nx_fifo_found) {
+ if (!ct_842) {
pr_err("NX842 FIFO nodes are missing\n");
return -EINVAL;
}
@@ -887,22 +932,14 @@ static int __init nx842_powernv_probe_vas(struct device_node *pn)
/*
* Initialize NX instance for both high and normal priority FIFOs.
*/
- if (opal_check_token(OPAL_NX_COPROC_INIT)) {
- ret = opal_nx_coproc_init(chip_id, ct);
- if (ret) {
- pr_err("Failed to initialize NX for chip(%d): %d\n",
- chip_id, ret);
- ret = opal_error_code(ret);
- }
- } else
- pr_warn("Firmware doesn't support NX initialization\n");
+ ret = nx_coproc_init(chip_id, ct_842);
return ret;
}
static int __init nx842_powernv_probe(struct device_node *dn)
{
- struct nx842_coproc *coproc;
+ struct nx_coproc *coproc;
unsigned int ct, ci;
int chip_id;
@@ -928,7 +965,7 @@ static int __init nx842_powernv_probe(struct device_node *dn)
coproc->ct = ct;
coproc->ci = ci;
- nx842_add_coprocs_list(coproc, chip_id);
+ nx_add_coprocs_list(coproc, chip_id);
pr_info("coprocessor found on chip %d, CT %d CI %d\n", chip_id, ct, ci);
@@ -941,9 +978,9 @@ static int __init nx842_powernv_probe(struct device_node *dn)
return 0;
}
-static void nx842_delete_coprocs(void)
+static void nx_delete_coprocs(void)
{
- struct nx842_coproc *coproc, *n;
+ struct nx_coproc *coproc, *n;
struct vas_window *txwin;
int i;
@@ -958,7 +995,7 @@ static void nx842_delete_coprocs(void)
per_cpu(cpu_txwin, i) = 0;
}
- list_for_each_entry_safe(coproc, n, &nx842_coprocs, list) {
+ list_for_each_entry_safe(coproc, n, &nx_coprocs, list) {
if (coproc->vas.rxwin)
vas_win_close(coproc->vas.rxwin);
@@ -1002,7 +1039,7 @@ static int nx842_powernv_crypto_init(struct crypto_tfm *tfm)
.coa_decompress = nx842_crypto_decompress } }
};
-static __init int nx842_powernv_init(void)
+static __init int nx_compress_powernv_init(void)
{
struct device_node *dn;
int ret;
@@ -1017,15 +1054,15 @@ static __init int nx842_powernv_init(void)
BUILD_BUG_ON(DDE_BUFFER_SIZE_MULT % DDE_BUFFER_LAST_MULT);
for_each_compatible_node(dn, NULL, "ibm,power9-nx") {
- ret = nx842_powernv_probe_vas(dn);
+ ret = nx_powernv_probe_vas(dn);
if (ret) {
- nx842_delete_coprocs();
+ nx_delete_coprocs();
of_node_put(dn);
return ret;
}
}
- if (list_empty(&nx842_coprocs)) {
+ if (list_empty(&nx_coprocs)) {
for_each_compatible_node(dn, NULL, "ibm,power-nx")
nx842_powernv_probe(dn);
@@ -1034,9 +1071,13 @@ static __init int nx842_powernv_init(void)
nx842_powernv_exec = nx842_exec_icswx;
} else {
- ret = nx842_open_percpu_txwins();
+ /*
+ * GZIP is not supported in kernel right now.
+ * So open tx windows only for 842.
+ */
+ ret = nx_open_percpu_txwins();
if (ret) {
- nx842_delete_coprocs();
+ nx_delete_coprocs();
return ret;
}
@@ -1045,18 +1086,18 @@ static __init int nx842_powernv_init(void)
ret = crypto_register_alg(&nx842_powernv_alg);
if (ret) {
- nx842_delete_coprocs();
+ nx_delete_coprocs();
return ret;
}
return 0;
}
-module_init(nx842_powernv_init);
+module_init(nx_compress_powernv_init);
-static void __exit nx842_powernv_exit(void)
+static void __exit nx_compress_powernv_exit(void)
{
crypto_unregister_alg(&nx842_powernv_alg);
- nx842_delete_coprocs();
+ nx_delete_coprocs();
}
-module_exit(nx842_powernv_exit);
+module_exit(nx_compress_powernv_exit);
--
1.8.3.1
^ permalink raw reply related
* [PATCH V2 7/9] crypto/nx: Enable and setup GZIP compression type
From: Haren Myneni @ 2020-02-29 11:17 UTC (permalink / raw)
To: herbert; +Cc: mikey, npiggin, linux-crypto, sukadev, linuxppc-dev
In-Reply-To: <1582974266.18705.28.camel@hbabu-laptop>
Changes to probe GZIP device-tree nodes, open RX windows and setup
GZIP compression type. No plans to provide GZIP usage in kernel right
now, but this patch enables GZIP for user space usage.
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
drivers/crypto/nx/nx-common-powernv.c | 43 ++++++++++++++++++++++++++++++-----
1 file changed, 37 insertions(+), 6 deletions(-)
diff --git a/drivers/crypto/nx/nx-common-powernv.c b/drivers/crypto/nx/nx-common-powernv.c
index 1cd4f40..6848aea1 100644
--- a/drivers/crypto/nx/nx-common-powernv.c
+++ b/drivers/crypto/nx/nx-common-powernv.c
@@ -65,6 +65,7 @@ struct nx_coproc {
* Using same values as in skiboot or coprocessor type representing
* in NX workbook.
*/
+#define NX_CT_GZIP (2) /* on P9 and later */
#define NX_CT_842 (3)
static int (*nx842_powernv_exec)(const unsigned char *in,
@@ -819,6 +820,9 @@ static int __init vas_cfg_coproc_info(struct device_node *dn, int chip_id,
if (type == NX_CT_842)
ret = nx_set_ct(coproc, priority, VAS_COP_TYPE_842_HIPRI,
VAS_COP_TYPE_842);
+ else if (type == NX_CT_GZIP)
+ ret = nx_set_ct(coproc, priority, VAS_COP_TYPE_GZIP_HIPRI,
+ VAS_COP_TYPE_GZIP);
if (ret)
goto err_out;
@@ -867,12 +871,16 @@ static int __init vas_cfg_coproc_info(struct device_node *dn, int chip_id,
return ret;
}
-static int __init nx_coproc_init(int chip_id, int ct_842)
+static int __init nx_coproc_init(int chip_id, int ct_842, int ct_gzip)
{
int ret = 0;
if (opal_check_token(OPAL_NX_COPROC_INIT)) {
ret = opal_nx_coproc_init(chip_id, ct_842);
+
+ if (!ret)
+ ret = opal_nx_coproc_init(chip_id, ct_gzip);
+
if (ret) {
ret = opal_error_code(ret);
pr_err("Failed to initialize NX for chip(%d): %d\n",
@@ -902,8 +910,8 @@ static int __init find_nx_device_tree(struct device_node *dn, int chip_id,
static int __init nx_powernv_probe_vas(struct device_node *pn)
{
int chip_id, vasid, ret = 0;
+ int ct_842 = 0, ct_gzip = 0;
struct device_node *dn;
- int ct_842 = 0;
chip_id = of_get_ibm_chip_id(pn);
if (chip_id < 0) {
@@ -920,19 +928,24 @@ static int __init nx_powernv_probe_vas(struct device_node *pn)
for_each_child_of_node(pn, dn) {
ret = find_nx_device_tree(dn, chip_id, vasid, NX_CT_842,
"ibm,p9-nx-842", &ct_842);
+
+ if (!ret)
+ ret = find_nx_device_tree(dn, chip_id, vasid,
+ NX_CT_GZIP, "ibm,p9-nx-gzip", &ct_gzip);
+
if (ret)
return ret;
}
- if (!ct_842) {
- pr_err("NX842 FIFO nodes are missing\n");
+ if (!ct_842 || !ct_gzip) {
+ pr_err("NX FIFO nodes are missing\n");
return -EINVAL;
}
/*
* Initialize NX instance for both high and normal priority FIFOs.
*/
- ret = nx_coproc_init(chip_id, ct_842);
+ ret = nx_coproc_init(chip_id, ct_842, ct_gzip);
return ret;
}
@@ -1072,10 +1085,19 @@ static __init int nx_compress_powernv_init(void)
nx842_powernv_exec = nx842_exec_icswx;
} else {
/*
+ * Register VAS user space API for NX GZIP so
+ * that user space can use GZIP engine.
+ * 842 compression is supported only in kernel.
+ */
+ ret = vas_register_coproc_api(THIS_MODULE);
+
+ /*
* GZIP is not supported in kernel right now.
* So open tx windows only for 842.
*/
- ret = nx_open_percpu_txwins();
+ if (!ret)
+ ret = nx_open_percpu_txwins();
+
if (ret) {
nx_delete_coprocs();
return ret;
@@ -1096,6 +1118,15 @@ static __init int nx_compress_powernv_init(void)
static void __exit nx_compress_powernv_exit(void)
{
+ /*
+ * GZIP engine is supported only in power9 or later and nx842_ct
+ * is used on power8 (icswx).
+ * VAS API for NX GZIP is registered during init for user space
+ * use. So delete this API use for GZIP engine.
+ */
+ if (!nx842_ct)
+ vas_unregister_coproc_api();
+
crypto_unregister_alg(&nx842_powernv_alg);
nx_delete_coprocs();
--
1.8.3.1
^ permalink raw reply related
* [PATCH V2 8/9] crypto/nx: Remove 'pid' in vas_tx_win_attr struct
From: Haren Myneni @ 2020-02-29 11:17 UTC (permalink / raw)
To: herbert; +Cc: mikey, npiggin, linux-crypto, sukadev, linuxppc-dev
In-Reply-To: <1582974266.18705.28.camel@hbabu-laptop>
When window is opened, pid reference is taken for user space
windows. Not needed for kernel windows. So remove 'pid' in
vas_tx_win_attr struct.
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
arch/powerpc/include/asm/vas.h | 1 -
drivers/crypto/nx/nx-common-powernv.c | 1 -
2 files changed, 2 deletions(-)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index e064953..994db6f 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -86,7 +86,6 @@ struct vas_tx_win_attr {
int wcreds_max;
int lpid;
int pidr; /* hardware PID (from SPRN_PID) */
- int pid; /* linux process id */
int pswid;
int rsvd_txbuf_count;
int tc_mode;
diff --git a/drivers/crypto/nx/nx-common-powernv.c b/drivers/crypto/nx/nx-common-powernv.c
index 6848aea1..c8b2e01 100644
--- a/drivers/crypto/nx/nx-common-powernv.c
+++ b/drivers/crypto/nx/nx-common-powernv.c
@@ -692,7 +692,6 @@ static struct vas_window *nx_alloc_txwin(struct nx_coproc *coproc)
*/
vas_init_tx_win_attr(&txattr, coproc->ct);
txattr.lpid = 0; /* lpid is 0 for kernel requests */
- txattr.pid = 0; /* pid is 0 for kernel requests */
/*
* Open a VAS send window which is used to send request to NX.
--
1.8.3.1
^ permalink raw reply related
* [PATCH V2 9/9] Documentation/powerpc: VAS API
From: Haren Myneni @ 2020-02-29 11:18 UTC (permalink / raw)
To: herbert; +Cc: mikey, npiggin, linux-crypto, sukadev, linuxppc-dev
In-Reply-To: <1582974266.18705.28.camel@hbabu-laptop>
Power9 introduced Virtual Accelerator Switchboard (VAS) which allows
userspace to communicate with Nest Accelerator (NX) directly. But
kernel has to establish channel to NX for userspace. This document
describes user space API that application can use to establish
communication channel.
Signed-off-by: Sukadev Bhattiprolu <sukadev@linux.ibm.com>
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
Documentation/powerpc/index.rst | 1 +
Documentation/powerpc/vas-api.rst | 246 ++++++++++++++++++++++++++++++++++++++
2 files changed, 247 insertions(+)
create mode 100644 Documentation/powerpc/vas-api.rst
diff --git a/Documentation/powerpc/index.rst b/Documentation/powerpc/index.rst
index 0d45f0f..afe2d5e 100644
--- a/Documentation/powerpc/index.rst
+++ b/Documentation/powerpc/index.rst
@@ -30,6 +30,7 @@ powerpc
syscall64-abi
transactional_memory
ultravisor
+ vas-api
.. only:: subproject and html
diff --git a/Documentation/powerpc/vas-api.rst b/Documentation/powerpc/vas-api.rst
new file mode 100644
index 0000000..13ce4e7
--- /dev/null
+++ b/Documentation/powerpc/vas-api.rst
@@ -0,0 +1,246 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. _VAS-API:
+
+===================================================
+Virtual Accelerator Switchboard (VAS) userspace API
+===================================================
+
+Introduction
+============
+
+Power9 processor introduced Virtual Accelerator Switchboard (VAS) which
+allows both userspace and kernel communicate to co-processor
+(hardware accelerator) referred to as the Nest Accelerator (NX). The NX
+unit comprises of one or more hardware engines or co-processor types
+such as 842 compression, GZIP compression and encryption. On power9,
+userspace applications will have access to only GZIP Compression engine
+which supports ZLIB and GZIP compression algorithms in the hardware.
+
+To communicate with NX, kernel has to establish a channel or window and
+then requests can be submitted directly without kernel involvement.
+Requests to the GZIP engine must be formatted as a co-processor Request
+Block (CRB) and these CRBs must be submitted to the NX using COPY/PASTE
+instructions to paste the CRB to hardware address that is associated with
+the engine's request queue.
+
+The GZIP engine provides two priority levels of requests: Normal and
+High. Only Normal requests are supported from userspace right now.
+
+This document explains userspace API that is used to interact with
+kernel to setup channel / window which can be used to send compression
+requests directly to NX accelerator.
+
+
+Overview
+========
+
+Application access to the GZIP engine is provided through
+/dev/crypto/nx-gzip device node implemented by the VAS/NX device driver.
+An application must open the /dev/crypto/nx-gzip device to obtain a file
+descriptor (fd). Then should issue VAS_TX_WIN_OPEN ioctl with this fd to
+establish connection to the engine. It means send window is opened on GZIP
+engine for this process. Once a connection is established, the application
+should use the mmap() system call to map the hardware address of engine's
+request queue into the application's virtual address space.
+
+The application can then submit one or more requests to the the engine by
+using copy/paste instructions and pasting the CRBs to the virtual address
+(aka paste_address) returned by mmap(). User space can close the
+established connection or send window by closing the file descriptior
+(close(fd)) or upon the process exit.
+
+Note that applications can send several requests with the same window or
+can establish multiple windows, but one window for each file descriptor.
+
+Following sections provide additional details and references about the
+individual steps.
+
+NX-GZIP Device Node
+===================
+
+There is one /dev/crypto/nx-gzip node in the system and it provides
+access to all GZIP engines in the system. The only valid operations on
+/dev/crypto/nx-gzip are:
+
+ * open() the device for read and write.
+ * issue VAS_TX_WIN_OPEN ioctl
+ * mmap() the engine's request queue into application's virtual
+ address space (i.e. get a paste_address for the co-processor
+ engine).
+ * close the device node.
+
+Other file operations on this device node are undefined.
+
+Note that the copy and paste operations go directly to the hardware and
+do not go through this device. Refer COPY/PASTE document for more
+details.
+
+Although a system may have several instances of the NX co-processor
+engines (typically, one per P9 chip) there is just one
+/dev/crypto/nx-gzip device node in the system. When the nx-gzip device
+node is opened, Kernel opens send window on a suitable instance of NX
+accelerator. It finds CPU on which the user process is executing and
+determine the NX instance for the corresponding chip on which this CPU
+belongs.
+
+Applications may chose a specific instance of the NX co-processor using
+the vas_id field in the VAS_TX_WIN_OPEN ioctl as detailed below.
+
+A userspace library libnxz is available here but still in development:
+ https://github.com/abalib/power-gzip
+
+Applications that use inflate / deflate calls can link with libnxz
+instead of libz and use NX GZIP compression without any modification.
+
+Open /dev/crypto/nx-gzip
+========================
+
+The nx-gzip device should be opened for read and write. No special
+privileges are needed to open the device. Each window coreesponds to one
+file descriptor. So if the userspace process needs multiple windows,
+several open calls have to be issued.
+
+See open(2) system call man pages for other details such as return values,
+error codes and restrictions.
+codes and restrictions.
+
+VAS_TX_WIN_OPEN ioctl
+=====================
+
+Applications should use the VAS_TX_WIN_OPEN ioctl as follows to establish
+a connection with NX co-processor engine:
+
+ ::
+ struct vas_tx_win_open_attr {
+ __u32 version;
+ __s16 vas_id; /* specific instance of vas or -1
+ for default */
+ __u16 reserved1;
+ __u64 flags; /* For future use */
+ __u64 reserved2[6];
+ };
+
+ version: The version field must be currently set to 1.
+ vas_id: If '-1' is passed, kernel will make a best-effort attempt
+ to assign an optimal instance of NX for the process. To
+ select the specific VAS instance, refer
+ "Discovery of available VAS engines" section below.
+
+ flags, reserved1 and reserved2[6] fields are for future extension
+ and must be set to 0.
+
+ The attributes attr for the VAS_TX_WIN_OPEN ioctl are defined as
+ follows:
+ #define VAS_MAGIC 'v'
+ #define VAS_TX_WIN_OPEN _IOW(VAS_MAGIC, 1,
+ struct vas_tx_win_open_attr)
+
+ struct vas_tx_win_open_attr attr;
+ rc = ioctl(fd, VAS_TX_WIN_OPEN, &attr);
+
+ The VAS_TX_WIN_OPEN ioctl returns 0 on success. On errors, it
+ returns -1 and sets the errno variable to indicate the error.
+
+ Error conditions:
+ EINVAL fd does not refer to a valid VAS device.
+ EINVAL Invalid vas ID
+ EINVAL version is not set with proper value
+ EEXIST Window is already opened for the given fd
+ ENOMEM Memory is not available to allocate window
+ ENOSPC System has too many active windows (connections)
+ opened
+ EINVAL reserved fields are not set to 0.
+
+ See the ioctl(2) man page for more details, error codes and
+ restrictions.
+
+mmap() NX-GZIP device
+=====================
+
+The mmap() system call for a NX-GZIP device fd returns a paste_address
+that the application can use to copy/paste its CRB to the hardware engines.
+ ::
+
+ paste_addr = mmap(addr, size, prot, flags, fd, offset);
+
+ Only restrictions on mmap for a NX-GZIP device fd are:
+ * size should be 4K page size
+ * offset parameter should be 0ULL
+
+ Refer to mmap(2) man page for additional details/restrictions.
+ In addition to the error conditions listed on the mmap(2) man
+ page, can also fail with one of the following error codes:
+
+ EINVAL fd is not associated with an open window
+ (i.e mmap() does not follow a successful call
+ to the VAS_TX_WIN_OPEN ioctl).
+ EINVAL offset field is not 0ULL.
+
+Discovery of available VAS engines
+==================================
+
+Each available VAS instance in the system will have a device tree node
+like /proc/device-tree/vas@* or /proc/device-tree/xscom@*/vas@*.
+Determine the chip or VAS instance and use the corresponding ibm,vas-id
+property value in this node to select specific VAS instance.
+
+Copy/Paste operations
+=====================
+
+Applications should use the copy and paste instructions defined in the RFC
+to copy/paste the CRB.
+
+CRB Specification and use NX
+============================
+
+Applications should format requests to the co-processor using the
+co-processor Request Block (CRBs). Refer NX workbook for the format of
+CRB and use NX from userspace such as sending requests and checking
+request status.
+
+Simple example
+==============
+
+ ::
+ int use_nx_gzip()
+ {
+ int rc, fd;
+ void *addr;
+ struct vas_setup_attr txattr;
+
+ fd = open("/dev/crypto/nx-gzip", O_RDWR);
+ if (fd < 0) {
+ fprintf(stderr, "open nx-gzip failed\n");
+ return -1;
+ }
+ memset(&txattr, 0, sizeof(txattr));
+ txattr.version = 1;
+ txattr.vas_id = -1
+ rc = ioctl(fd, VAS_TX_WIN_OPEN,
+ (unsigned long)&txattr);
+ if (rc < 0) {
+ fprintf(stderr, "ioctl() n %d, error %d\n",
+ rc, errno);
+ return rc;
+ }
+ addr = mmap(NULL, 4096, PROT_READ|PROT_WRITE,
+ MAP_SHARED, fd, 0ULL);
+ if (addr == MAP_FAILED) {
+ fprintf(stderr, "mmap() failed, errno %d\n",
+ errno);
+ return -errno;
+ }
+ do {
+ //Format CRB request with compression or
+ //uncompression
+ // Refer tests for vas_copy/vas_paste
+ vas_copy((&crb, 0, 1);
+ vas_paste(addr, 0, 1);
+ // Poll on csb.flags with timeout
+ // csb address is listed in CRB
+ } while (true)
+ close(fd) or window can be closed upon process exit
+ }
+
+ Refer https://github.com/abalib/power-gzip for tests or more
+ use cases.
--
1.8.3.1
^ permalink raw reply related
* Re: [PATCH 6/8] powerpc: Update MPC5XXX MAINTAINERS entry
From: Anatolij Gustschin @ 2020-02-29 18:08 UTC (permalink / raw)
To: Michael Ellerman; +Cc: linuxppc-dev, linux-kernel
In-Reply-To: <20200224233146.23734-6-mpe@ellerman.id.au>
On Tue, 25 Feb 2020 10:31:44 +1100
Michael Ellerman mpe@ellerman.id.au wrote:
>It's several years since the last commit from Anatolij, so mark
>MPC5XXX as "Odd Fixes" rather than "Maintained".
>
>Also the git link no longer works so remove it.
>
>Cc: Anatolij Gustschin <agust@denx.de>
>Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
Acked-by: Anatolij Gustschin <agust@denx.de>
^ permalink raw reply
* Re: [PATCH v3 2/7] mm/memory_hotplug: Rename mhp_restrictions to mhp_params
From: Dan Williams @ 2020-02-29 20:44 UTC (permalink / raw)
To: Logan Gunthorpe
Cc: linux-ia64, Linux-sh, Peter Zijlstra, Catalin Marinas,
Dave Hansen, platform-driver-x86, Linux MM, Will Deacon,
Christoph Hellwig, linux-s390, David Hildenbrand, Ingo Molnar,
Borislav Petkov, Andy Lutomirski, Thomas Gleixner, Michal Hocko,
Linux ARM, Eric Badger, Linux Kernel Mailing List, Andrew Morton,
linuxppc-dev
In-Reply-To: <20200221182503.28317-3-logang@deltatee.com>
On Fri, Feb 21, 2020 at 10:25 AM Logan Gunthorpe <logang@deltatee.com> wrote:
>
> The mhp_restrictions struct really doesn't specify anything resembling
> a restriction anymore so rename it to be mhp_params as it is a list
> of extended parameters.
>
> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Tests ok, and looks good to me:
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
^ permalink raw reply
* Re: [PATCH v3 4/7] x86/mm: Introduce _set_memory_prot()
From: Dan Williams @ 2020-02-29 22:33 UTC (permalink / raw)
To: Logan Gunthorpe
Cc: X86 ML, linux-ia64, Linux-sh, Peter Zijlstra, Catalin Marinas,
Dave Hansen, platform-driver-x86, Linux MM, H. Peter Anvin,
Will Deacon, Christoph Hellwig, linux-s390, David Hildenbrand,
Ingo Molnar, Borislav Petkov, Andy Lutomirski, Thomas Gleixner,
Michal Hocko, Linux ARM, Eric Badger, Linux Kernel Mailing List,
Andrew Morton, linuxppc-dev
In-Reply-To: <20200221182503.28317-5-logang@deltatee.com>
On Fri, Feb 21, 2020 at 10:25 AM Logan Gunthorpe <logang@deltatee.com> wrote:
>
> For use in the 32bit arch_add_memory() to set the pgprot type of the
> memory to add.
>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: x86@kernel.org
> Cc: Dave Hansen <dave.hansen@linux.intel.com>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> ---
> arch/x86/include/asm/set_memory.h | 1 +
> arch/x86/mm/pat/set_memory.c | 7 +++++++
> 2 files changed, 8 insertions(+)
>
> diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h
> index 64c3dce374e5..0aca959cf9a4 100644
> --- a/arch/x86/include/asm/set_memory.h
> +++ b/arch/x86/include/asm/set_memory.h
> @@ -34,6 +34,7 @@
> * The caller is required to take care of these.
> */
>
> +int _set_memory_prot(unsigned long addr, int numpages, pgprot_t prot);
I wonder if this should be separated from the naming convention of the
other routines because this is only an internal helper for code paths
where the prot was established by an upper layer. For example, I
expect that the kernel does not want new usages to make the mistake of
calling:
_set_memory_prot(..., pgprot_writecombine(pgprot))
...instead of
_set_memory_wc()
I'm thinking just a double underscore rename (__set_memory_prot) and a
kerneldoc comment for that pointing people to use the direct
_set_memory_<cachemode> helpers.
With that you can add:
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
^ permalink raw reply
* Re: [PATCH v3 3/7] x86/mm: Thread pgprot_t through init_memory_mapping()
From: Dan Williams @ 2020-02-29 22:37 UTC (permalink / raw)
To: Logan Gunthorpe
Cc: X86 ML, linux-ia64, Linux-sh, Peter Zijlstra, Catalin Marinas,
Dave Hansen, platform-driver-x86, Linux MM, H. Peter Anvin,
Will Deacon, Christoph Hellwig, linux-s390, David Hildenbrand,
Ingo Molnar, Borislav Petkov, Andy Lutomirski, Thomas Gleixner,
Michal Hocko, Linux ARM, Eric Badger, Linux Kernel Mailing List,
Andrew Morton, linuxppc-dev
In-Reply-To: <20200221182503.28317-4-logang@deltatee.com>
On Fri, Feb 21, 2020 at 10:25 AM Logan Gunthorpe <logang@deltatee.com> wrote:
>
> In prepartion to support a pgprot_t argument for arch_add_memory().
>
> It's required to move the prototype of init_memory_mapping() seeing
> the original location came before the definition of pgprot_t.
>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Borislav Petkov <bp@alien8.de>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: x86@kernel.org
> Cc: Dave Hansen <dave.hansen@linux.intel.com>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Looks good, checked for argument confusion, passes the nvdimm unit tests.
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
^ permalink raw reply
* Re: [PATCH v3 6/7] mm/memory_hotplug: Add pgprot_t to mhp_params
From: Dan Williams @ 2020-02-29 22:44 UTC (permalink / raw)
To: Logan Gunthorpe
Cc: Michal Hocko, linux-ia64, Linux-sh, Peter Zijlstra,
Catalin Marinas, Dave Hansen, platform-driver-x86, Linux MM,
Will Deacon, Christoph Hellwig, linux-s390, David Hildenbrand,
Ingo Molnar, Borislav Petkov, Andy Lutomirski, Thomas Gleixner,
Michal Hocko, Linux ARM, Eric Badger, Linux Kernel Mailing List,
Andrew Morton, linuxppc-dev
In-Reply-To: <20200221182503.28317-7-logang@deltatee.com>
On Fri, Feb 21, 2020 at 10:25 AM Logan Gunthorpe <logang@deltatee.com> wrote:
>
> devm_memremap_pages() is currently used by the PCI P2PDMA code to create
> struct page mappings for IO memory. At present, these mappings are created
> with PAGE_KERNEL which implies setting the PAT bits to be WB. However, on
> x86, an mtrr register will typically override this and force the cache
> type to be UC-. In the case firmware doesn't set this register it is
> effectively WB and will typically result in a machine check exception
> when it's accessed.
>
> Other arches are not currently likely to function correctly seeing they
> don't have any MTRR registers to fall back on.
>
> To solve this, provide a way to specify the pgprot value explicitly to
> arch_add_memory().
>
> Of the arches that support MEMORY_HOTPLUG: x86_64, and arm64 need a simple
> change to pass the pgprot_t down to their respective functions which set
> up the page tables. For x86_32, set the page tables explicitly using
> _set_memory_prot() (seeing they are already mapped). For ia64, s390 and
> sh, reject anything but PAGE_KERNEL settings -- this should be fine,
> for now, seeing these architectures don't support ZONE_DEVICE.
>
> A check in __add_pages() is also added to ensure the pgprot parameter was
> set for all arches.
>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> Acked-by: David Hildenbrand <david@redhat.com>
> Acked-by: Michal Hocko <mhocko@suse.com>
> ---
> arch/arm64/mm/mmu.c | 3 ++-
> arch/ia64/mm/init.c | 3 +++
> arch/powerpc/mm/mem.c | 3 ++-
> arch/s390/mm/init.c | 3 +++
> arch/sh/mm/init.c | 3 +++
> arch/x86/mm/init_32.c | 5 +++++
> arch/x86/mm/init_64.c | 2 +-
> include/linux/memory_hotplug.h | 2 ++
> mm/memory_hotplug.c | 5 ++++-
> mm/memremap.c | 6 +++---
> 10 files changed, 28 insertions(+), 7 deletions(-)
>
> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> index ee37bca8aba8..ea3fa844a8a2 100644
> --- a/arch/arm64/mm/mmu.c
> +++ b/arch/arm64/mm/mmu.c
> @@ -1058,7 +1058,8 @@ int arch_add_memory(int nid, u64 start, u64 size,
> flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS;
>
> __create_pgd_mapping(swapper_pg_dir, start, __phys_to_virt(start),
> - size, PAGE_KERNEL, __pgd_pgtable_alloc, flags);
> + size, params->pgprot, __pgd_pgtable_alloc,
> + flags);
>
> memblock_clear_nomap(start, size);
>
> diff --git a/arch/ia64/mm/init.c b/arch/ia64/mm/init.c
> index 97bbc23ea1e3..d637b4ea3147 100644
> --- a/arch/ia64/mm/init.c
> +++ b/arch/ia64/mm/init.c
> @@ -676,6 +676,9 @@ int arch_add_memory(int nid, u64 start, u64 size,
> unsigned long nr_pages = size >> PAGE_SHIFT;
> int ret;
>
> + if (WARN_ON_ONCE(params->pgprot.pgprot != PAGE_KERNEL.pgprot))
> + return -EINVAL;
> +
> ret = __add_pages(nid, start_pfn, nr_pages, params);
> if (ret)
> printk("%s: Problem encountered in __add_pages() as ret=%d\n",
> diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c
> index 19b1da5d7eca..832412bc7fad 100644
> --- a/arch/powerpc/mm/mem.c
> +++ b/arch/powerpc/mm/mem.c
> @@ -138,7 +138,8 @@ int __ref arch_add_memory(int nid, u64 start, u64 size,
> resize_hpt_for_hotplug(memblock_phys_mem_size());
>
> start = (unsigned long)__va(start);
> - rc = create_section_mapping(start, start + size, nid, PAGE_KERNEL);
> + rc = create_section_mapping(start, start + size, nid,
> + params->pgprot);
> if (rc) {
> pr_warn("Unable to create mapping for hot added memory 0x%llx..0x%llx: %d\n",
> start, start + size, rc);
> diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c
> index e9e4a7abd0cc..87b2d024e75a 100644
> --- a/arch/s390/mm/init.c
> +++ b/arch/s390/mm/init.c
> @@ -277,6 +277,9 @@ int arch_add_memory(int nid, u64 start, u64 size,
> if (WARN_ON_ONCE(params->altmap))
> return -EINVAL;
>
> + if (WARN_ON_ONCE(params->pgprot.pgprot != PAGE_KERNEL.pgprot))
> + return -EINVAL;
> +
> rc = vmem_add_mapping(start, size);
> if (rc)
> return rc;
> diff --git a/arch/sh/mm/init.c b/arch/sh/mm/init.c
> index e5114c053364..b9de2d4fa57e 100644
> --- a/arch/sh/mm/init.c
> +++ b/arch/sh/mm/init.c
> @@ -412,6 +412,9 @@ int arch_add_memory(int nid, u64 start, u64 size,
> unsigned long nr_pages = size >> PAGE_SHIFT;
> int ret;
>
> + if (WARN_ON_ONCE(params->pgprot.pgprot != PAGE_KERNEL.pgprot)
> + return -EINVAL;
> +
> /* We only have ZONE_NORMAL, so this is easy.. */
> ret = __add_pages(nid, start_pfn, nr_pages, params);
> if (unlikely(ret))
> diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c
> index e25a4218e6ff..96d8e4fb1cc8 100644
> --- a/arch/x86/mm/init_32.c
> +++ b/arch/x86/mm/init_32.c
> @@ -858,6 +858,11 @@ int arch_add_memory(int nid, u64 start, u64 size,
> {
> unsigned long start_pfn = start >> PAGE_SHIFT;
> unsigned long nr_pages = size >> PAGE_SHIFT;
> + int ret;
> +
> + ret = _set_memory_prot(start, nr_pages, params->pgprot);
Perhaps a comment since it's not immediately obvious where the
PAGE_KERNEL prot was established, and perhaps add a conditional to
skip this call in the param->pgprot == PAGE_KERNEL case?
Other than that looks good to me, but only an ack since I'm only
testing the x86 changes.
Acked-by: Dan Williams <dan.j.williams@intel.com>
^ permalink raw reply
* Re: [PATCH v3 7/7] mm/memremap: Set caching mode for PCI P2PDMA memory to WC
From: Dan Williams @ 2020-02-29 22:47 UTC (permalink / raw)
To: Logan Gunthorpe
Cc: linux-ia64, Linux-sh, Peter Zijlstra, Catalin Marinas,
Dave Hansen, platform-driver-x86, Linux MM, Will Deacon,
Christoph Hellwig, linux-s390, David Hildenbrand, Jason Gunthorpe,
Ingo Molnar, Borislav Petkov, Andy Lutomirski, Thomas Gleixner,
Michal Hocko, Linux ARM, Eric Badger, Linux Kernel Mailing List,
Andrew Morton, linuxppc-dev
In-Reply-To: <20200221182503.28317-8-logang@deltatee.com>
On Fri, Feb 21, 2020 at 10:25 AM Logan Gunthorpe <logang@deltatee.com> wrote:
>
> PCI BAR IO memory should never be mapped as WB, however prior to this
> the PAT bits were set WB and it was typically overridden by MTRR
> registers set by the firmware.
>
> Set PCI P2PDMA memory to be WC (writecombining) as the only current
> user (the NVMe CMB) was originally mapped WC before the P2PDMA code
> replaced the mapping with devm_memremap_pages().
Will the change to UC regress this existing use case?
>
> Future use-cases may need to generalize this by adding flags to
> select the caching type, as some P2PDMA cases will not want WC.
> However, those use-cases are not upstream yet and this can be changed
> when they arrive.
>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Jason Gunthorpe <jgg@ziepe.ca>
> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> ---
> mm/memremap.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/mm/memremap.c b/mm/memremap.c
> index 06742372a203..8d141c3e3364 100644
> --- a/mm/memremap.c
> +++ b/mm/memremap.c
> @@ -190,7 +190,10 @@ void *memremap_pages(struct dev_pagemap *pgmap, int nid)
> }
> break;
> case MEMORY_DEVICE_DEVDAX:
> + need_devmap_managed = false;
> + break;
> case MEMORY_DEVICE_PCI_P2PDMA:
> + params.pgprot = pgprot_writecombine(params.pgprot);
Approach looks good to me, modulo Jason's comment that this should be
UC. Upcoming DAX changes will want to pass this via pgmap, but as you
say this can wait for this changes to arrive.
After change to UC:
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
^ permalink raw reply
* RE: [RFC PATCH v1] powerpc/prom_init: disable XIVE in Secure VM.
From: Ram Pai @ 2020-02-29 22:51 UTC (permalink / raw)
To: Cédric Le Goater
Cc: aik, andmike, groug, kvm-ppc, clg, sukadev, linuxppc-dev,
bauerman, david
In-Reply-To: <1e28fb80-7bae-8d80-1a72-f616af030aab@kaod.org>
On Sat, Feb 29, 2020 at 09:27:54AM +0100, Cédric Le Goater wrote:
> On 2/29/20 8:54 AM, Ram Pai wrote:
> > XIVE is not correctly enabled for Secure VM in the KVM Hypervisor yet.
> >
> > Hence Secure VM, must always default to XICS interrupt controller.
>
> have you tried XIVE emulation 'kernel-irqchip=off' ?
yes and it hangs. I think that option, continues to enable some variant
of XIVE in the VM. There are some known deficiencies between KVM
and the ultravisor negotiation, resulting in a hang in the SVM.
>
> > If XIVE is requested through kernel command line option "xive=on",
> > override and turn it off.
>
> This is incorrect. It is negotiated through CAS depending on the FW
> capabilities and the KVM capabilities.
Yes I understand, qemu/KVM have predetermined a set of capabilties that
it can offer to the VM. The kernel within the VM has a list of
capabilties it needs to operate correctly. So both negotiate and
determine something mutually ammicable.
Here I am talking about the list of capabilities that the kernel is
trying to determine, it needs to operate correctly. "xive=on" is one of
those capabilities the kernel is told by the VM-adminstrator, to enable.
Unfortunately if the VM-administrtor blindly requests to enable it, the
kernel must override it, if it knows that will be switching the VM into
a SVM soon. No point negotiating a capability with Qemu; through CAS,
if it knows it cannot handle that capability.
>
> > If XIVE is the only supported platform interrupt controller; specified
> > through qemu option "ic-mode=xive", simply abort. Otherwise default to
> > XICS.
>
>
> I don't think it is a good approach to downgrade the guest kernel
> capabilities this way.
>
> PAPR has specified the CAS negotiation process for this purpose. It
> comes in two parts under KVM. First the KVM hypervisor advertises or
> not a capability to QEMU. The second is the CAS negotiation process
> between QEMU and the guest OS.
Unfortunately, this is not viable. At the time the hypervisor
advertises its capabilities to qemu, the hypervisor has no idea whether
that VM will switch into a SVM or not. The decision to switch into a
SVM is taken by the kernel running in the VM. This happens much later,
after the hypervisor has already conveyed its capabilties to the qemu, and
qemu has than instantiated the VM.
As a result, CAS in prom_init is the only place where this negotiation
can take place.
>
> The SVM specifications might not be complete yet and if some features
> are incompatible, I think we should modify the capabilities advertised
> by the hypervisor : no XIVE in case of SVM. QEMU will automatically
> use the fallback path and emulate the XIVE device, same as setting
> 'kernel-irqchip=off'.
As mentioned above, this would be an excellent approach, if the
Hypervisor was aware of the VM's intent to switch into a SVM. Neither
the hypervisor knows, nor the qemu. Only the kernel running within the
VM knows about it.
Do you still think, my approach is wrong?
RP
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox