* [Bug 205099] KASAN hit at raid6_pq: BUG: Unable to handle kernel data access at 0x00f0fd0d
From: bugzilla-daemon @ 2020-04-13 14:15 UTC (permalink / raw)
To: linuxppc-dev
In-Reply-To: <bug-205099-206035@https.bugzilla.kernel.org/>
https://bugzilla.kernel.org/show_bug.cgi?id=205099
Erhard F. (erhard_f@mailbox.org) changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #287623|0 |1
is obsolete| |
--- Comment #29 from Erhard F. (erhard_f@mailbox.org) ---
Created attachment 288413
--> https://bugzilla.kernel.org/attachment.cgi?id=288413&action=edit
kernel .config (5.7-rc1, OUTLINE KASAN, PowerMac G4 DP)
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply
* Re: [PATCH] papr/scm: Add bad memory ranges to nvdimm bad ranges
From: Mahesh J Salgaonkar @ 2020-04-13 14:24 UTC (permalink / raw)
To: Santosh Sivaraj; +Cc: Oliver, Ganesh Goudar, linuxppc-dev, Aneesh Kumar K.V
In-Reply-To: <20200401074741.1562361-1-santosh@fossix.org>
On 2020-04-01 13:17:41 Wed, Santosh Sivaraj wrote:
> Subscribe to the MCE notification and add the physical address which
> generated a memory error to nvdimm bad range.
>
> Signed-off-by: Santosh Sivaraj <santosh@fossix.org>
> ---
>
> This patch depends on "powerpc/mce: Add MCE notification chain" [1].
>
> Unlike the previous series[2], the patch adds badblock registration only for
> pseries scm driver. Handling badblocks for baremetal (powernv) PMEM will be done
> later and if possible get the badblock handling as a common code.
>
> [1] https://lore.kernel.org/linuxppc-dev/20200330071219.12284-1-ganeshgr@linux.ibm.com/
> [2] https://lore.kernel.org/linuxppc-dev/20190820023030.18232-1-santosh@fossix.org/
>
> arch/powerpc/platforms/pseries/papr_scm.c | 96 ++++++++++++++++++++++-
> 1 file changed, 95 insertions(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/platforms/pseries/papr_scm.c b/arch/powerpc/platforms/pseries/papr_scm.c
> index 0b4467e378e5..5012cbf4606e 100644
> --- a/arch/powerpc/platforms/pseries/papr_scm.c
> +++ b/arch/powerpc/platforms/pseries/papr_scm.c
[...]
> +static int handle_mce_ue(struct notifier_block *nb, unsigned long val,
> + void *data)
> +{
> + struct machine_check_event *evt = data;
> + struct papr_scm_priv *p;
> + u64 phys_addr;
> + bool found = false;
> +
> + if (evt->error_type != MCE_ERROR_TYPE_UE)
> + return NOTIFY_DONE;
> +
> + if (list_empty(&papr_nd_regions))
> + return NOTIFY_DONE;
Do you really need this check ?
> +
> + phys_addr = evt->u.ue_error.physical_address +
> + (evt->u.ue_error.effective_address & ~PAGE_MASK);
> +
> + if (!evt->u.ue_error.physical_address_provided ||
> + !is_zone_device_page(pfn_to_page(phys_addr >> PAGE_SHIFT)))
> + return NOTIFY_DONE;
> +
> + /* mce notifier is called from a process context, so mutex is safe */
> + mutex_lock(&papr_ndr_lock);
> + list_for_each_entry(p, &papr_nd_regions, region_list) {
> + struct resource res = p->res;
> +
> + if (phys_addr >= res.start && phys_addr <= res.end) {
> + found = true;
> + break;
> + }
> + }
> +
> + mutex_unlock(&papr_ndr_lock);
> +
> + if (!found)
> + return NOTIFY_DONE;
> +
> + papr_scm_add_badblock(p->region, p->bus, phys_addr);
> +
> + return NOTIFY_OK;
> +}
> +
> +static struct notifier_block mce_ue_nb = {
> + .notifier_call = handle_mce_ue
> +};
> +
[...]
> -module_platform_driver(papr_scm_driver);
> +static int __init papr_scm_init(void)
> +{
> + int ret;
> +
> + ret = platform_driver_register(&papr_scm_driver);
> + if (!ret)
> + mce_register_notifier(&mce_ue_nb);
> +
> + return ret;
> +}
> +module_init(papr_scm_init);
> +
> +static void __exit papr_scm_exit(void)
> +{
> + mce_unregister_notifier(&mce_ue_nb);
> + platform_driver_unregister(&papr_scm_driver);
> +}
> +module_exit(papr_scm_exit);
Rest Looks good to me.
Reviewed-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Thanks,
-Mahesh.
> +
> MODULE_DEVICE_TABLE(of, papr_scm_match);
> MODULE_LICENSE("GPL");
> MODULE_AUTHOR("IBM Corporation");
> --
> 2.25.1
>
--
Mahesh J Salgaonkar
^ permalink raw reply
* Re: [PATCH] papr/scm: Add bad memory ranges to nvdimm bad ranges
From: Santosh Sivaraj @ 2020-04-13 15:37 UTC (permalink / raw)
To: mahesh; +Cc: Oliver, Ganesh Goudar, linuxppc-dev, Aneesh Kumar K.V
In-Reply-To: <20200413142446.abccjt223vrhfgot@in.ibm.com>
Mahesh J Salgaonkar <mahesh@linux.ibm.com> writes:
> On 2020-04-01 13:17:41 Wed, Santosh Sivaraj wrote:
>> Subscribe to the MCE notification and add the physical address which
>> generated a memory error to nvdimm bad range.
>>
>> Signed-off-by: Santosh Sivaraj <santosh@fossix.org>
>> ---
>>
>> This patch depends on "powerpc/mce: Add MCE notification chain" [1].
>>
>> Unlike the previous series[2], the patch adds badblock registration only for
>> pseries scm driver. Handling badblocks for baremetal (powernv) PMEM will be done
>> later and if possible get the badblock handling as a common code.
>>
>> [1] https://lore.kernel.org/linuxppc-dev/20200330071219.12284-1-ganeshgr@linux.ibm.com/
>> [2] https://lore.kernel.org/linuxppc-dev/20190820023030.18232-1-santosh@fossix.org/
>>
>> arch/powerpc/platforms/pseries/papr_scm.c | 96 ++++++++++++++++++++++-
>> 1 file changed, 95 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/powerpc/platforms/pseries/papr_scm.c b/arch/powerpc/platforms/pseries/papr_scm.c
>> index 0b4467e378e5..5012cbf4606e 100644
>> --- a/arch/powerpc/platforms/pseries/papr_scm.c
>> +++ b/arch/powerpc/platforms/pseries/papr_scm.c
> [...]
>> +static int handle_mce_ue(struct notifier_block *nb, unsigned long val,
>> + void *data)
>> +{
>> + struct machine_check_event *evt = data;
>> + struct papr_scm_priv *p;
>> + u64 phys_addr;
>> + bool found = false;
>> +
>> + if (evt->error_type != MCE_ERROR_TYPE_UE)
>> + return NOTIFY_DONE;
>> +
>> + if (list_empty(&papr_nd_regions))
>> + return NOTIFY_DONE;
>
> Do you really need this check ?
Quite harmless I guess, atleast it saves a branch and mutex_lock/unlock.
>
>> +
>> + phys_addr = evt->u.ue_error.physical_address +
>> + (evt->u.ue_error.effective_address & ~PAGE_MASK);
>> +
>> + if (!evt->u.ue_error.physical_address_provided ||
>> + !is_zone_device_page(pfn_to_page(phys_addr >> PAGE_SHIFT)))
>> + return NOTIFY_DONE;
>> +
>> + /* mce notifier is called from a process context, so mutex is safe */
>> + mutex_lock(&papr_ndr_lock);
>> + list_for_each_entry(p, &papr_nd_regions, region_list) {
>> + struct resource res = p->res;
>> +
>> + if (phys_addr >= res.start && phys_addr <= res.end) {
>> + found = true;
>> + break;
>> + }
>> + }
>> +
>> + mutex_unlock(&papr_ndr_lock);
>> +
>> + if (!found)
>> + return NOTIFY_DONE;
>> +
>> + papr_scm_add_badblock(p->region, p->bus, phys_addr);
>> +
>> + return NOTIFY_OK;
>> +}
>> +
>> +static struct notifier_block mce_ue_nb = {
>> + .notifier_call = handle_mce_ue
>> +};
>> +
> [...]
>> -module_platform_driver(papr_scm_driver);
>> +static int __init papr_scm_init(void)
>> +{
>> + int ret;
>> +
>> + ret = platform_driver_register(&papr_scm_driver);
>> + if (!ret)
>> + mce_register_notifier(&mce_ue_nb);
>> +
>> + return ret;
>> +}
>> +module_init(papr_scm_init);
>> +
>> +static void __exit papr_scm_exit(void)
>> +{
>> + mce_unregister_notifier(&mce_ue_nb);
>> + platform_driver_unregister(&papr_scm_driver);
>> +}
>> +module_exit(papr_scm_exit);
>
> Rest Looks good to me.
>
> Reviewed-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>
Thanks for the review.
Santosh
>
> Thanks,
> -Mahesh.
>
>> +
>> MODULE_DEVICE_TABLE(of, papr_scm_match);
>> MODULE_LICENSE("GPL");
>> MODULE_AUTHOR("IBM Corporation");
>> --
>> 2.25.1
>>
>
> --
> Mahesh J Salgaonkar
^ permalink raw reply
* [PATCH V3 0/5] selftests/powerpc: Add NX-GZIP engine testcase
From: Raphael Moreira Zinsly @ 2020-04-13 15:59 UTC (permalink / raw)
To: linuxppc-dev, linux-crypto, dja; +Cc: abali, haren, herbert, rzinsly
This patch series are intended to test the POWER9 Nest
Accelerator (NX) GZIP engine that is being introduced by
https://lists.ozlabs.org/pipermail/linuxppc-dev/2020-March/205659.html
More information about how to access the NX can be found in that patch, also a
complete userspace library and more documentation can be found at:
https://github.com/libnxz/power-gzip
Changes in V3:
- Defined a macro and increased the number of retries for page faults
to work in system with less memory, mentioning the issue on README.
- Returned to use volatile on the touch pages routine and a few structs
on inc/nxu.h as they are handled by hardware and some compilers could
optmize it wrongly.
- Moved common functions to gzip_vas.c.
^ permalink raw reply
* [PATCH V3 1/5] selftests/powerpc: Add header files for GZIP engine test
From: Raphael Moreira Zinsly @ 2020-04-13 15:59 UTC (permalink / raw)
To: linuxppc-dev, linux-crypto, dja; +Cc: abali, haren, herbert, rzinsly
In-Reply-To: <20200413155916.16900-1-rzinsly@linux.ibm.com>
Add files to access the powerpc NX-GZIP engine in user space.
Signed-off-by: Bulent Abali <abali@us.ibm.com>
Signed-off-by: Raphael Moreira Zinsly <rzinsly@linux.ibm.com>
---
.../selftests/powerpc/nx-gzip/inc/crb.h | 159 ++++++++++++++++++
.../selftests/powerpc/nx-gzip/inc/nx-gzip.h | 27 +++
.../powerpc/nx-gzip/inc/nx-helpers.h | 54 ++++++
.../selftests/powerpc/nx-gzip/inc/nx.h | 38 +++++
4 files changed, 278 insertions(+)
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/inc/crb.h
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/inc/nx-gzip.h
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/inc/nx-helpers.h
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/inc/nx.h
diff --git a/tools/testing/selftests/powerpc/nx-gzip/inc/crb.h b/tools/testing/selftests/powerpc/nx-gzip/inc/crb.h
new file mode 100644
index 000000000000..9056e3dc1831
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/inc/crb.h
@@ -0,0 +1,159 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef __CRB_H
+#define __CRB_H
+#include <linux/types.h>
+#include "nx.h"
+
+typedef unsigned char u8;
+typedef unsigned int u32;
+typedef unsigned long long u64;
+
+/* CCW 842 CI/FC masks
+ * NX P8 workbook, section 4.3.1, figure 4-6
+ * "CI/FC Boundary by NX CT type"
+ */
+#define CCW_CI_842 (0x00003ff8)
+#define CCW_FC_842 (0x00000007)
+
+/* Chapter 6.5.8 Coprocessor-Completion Block (CCB) */
+
+#define CCB_VALUE (0x3fffffffffffffff)
+#define CCB_ADDRESS (0xfffffffffffffff8)
+#define CCB_CM (0x0000000000000007)
+#define CCB_CM0 (0x0000000000000004)
+#define CCB_CM12 (0x0000000000000003)
+
+#define CCB_CM0_ALL_COMPLETIONS (0x0)
+#define CCB_CM0_LAST_IN_CHAIN (0x4)
+#define CCB_CM12_STORE (0x0)
+#define CCB_CM12_INTERRUPT (0x1)
+
+#define CCB_SIZE (0x10)
+#define CCB_ALIGN CCB_SIZE
+
+struct coprocessor_completion_block {
+ __be64 value;
+ __be64 address;
+} __aligned(CCB_ALIGN);
+
+
+/* Chapter 6.5.7 Coprocessor-Status Block (CSB) */
+
+#define CSB_V (0x80)
+#define CSB_F (0x04)
+#define CSB_CH (0x03)
+#define CSB_CE_INCOMPLETE (0x80)
+#define CSB_CE_TERMINATION (0x40)
+#define CSB_CE_TPBC (0x20)
+
+#define CSB_CC_SUCCESS (0)
+#define CSB_CC_INVALID_ALIGN (1)
+#define CSB_CC_OPERAND_OVERLAP (2)
+#define CSB_CC_DATA_LENGTH (3)
+#define CSB_CC_TRANSLATION (5)
+#define CSB_CC_PROTECTION (6)
+#define CSB_CC_RD_EXTERNAL (7)
+#define CSB_CC_INVALID_OPERAND (8)
+#define CSB_CC_PRIVILEGE (9)
+#define CSB_CC_INTERNAL (10)
+#define CSB_CC_WR_EXTERNAL (12)
+#define CSB_CC_NOSPC (13)
+#define CSB_CC_EXCESSIVE_DDE (14)
+#define CSB_CC_WR_TRANSLATION (15)
+#define CSB_CC_WR_PROTECTION (16)
+#define CSB_CC_UNKNOWN_CODE (17)
+#define CSB_CC_ABORT (18)
+#define CSB_CC_TRANSPORT (20)
+#define CSB_CC_SEGMENTED_DDL (31)
+#define CSB_CC_PROGRESS_POINT (32)
+#define CSB_CC_DDE_OVERFLOW (33)
+#define CSB_CC_SESSION (34)
+#define CSB_CC_PROVISION (36)
+#define CSB_CC_CHAIN (37)
+#define CSB_CC_SEQUENCE (38)
+#define CSB_CC_HW (39)
+
+#define CSB_SIZE (0x10)
+#define CSB_ALIGN CSB_SIZE
+
+struct coprocessor_status_block {
+ u8 flags;
+ u8 cs;
+ u8 cc;
+ u8 ce;
+ __be32 count;
+ __be64 address;
+} __aligned(CSB_ALIGN);
+
+
+/* Chapter 6.5.10 Data-Descriptor List (DDL)
+ * each list contains one or more Data-Descriptor Entries (DDE)
+ */
+
+#define DDE_P (0x8000)
+
+#define DDE_SIZE (0x10)
+#define DDE_ALIGN DDE_SIZE
+
+struct data_descriptor_entry {
+ __be16 flags;
+ u8 count;
+ u8 index;
+ __be32 length;
+ __be64 address;
+} __aligned(DDE_ALIGN);
+
+
+/* Chapter 6.5.2 Coprocessor-Request Block (CRB) */
+
+#define CRB_SIZE (0x80)
+#define CRB_ALIGN (0x100) /* Errata: requires 256 alignment */
+
+
+/* Coprocessor Status Block field
+ * ADDRESS address of CSB
+ * C CCB is valid
+ * AT 0 = addrs are virtual, 1 = addrs are phys
+ * M enable perf monitor
+ */
+#define CRB_CSB_ADDRESS (0xfffffffffffffff0)
+#define CRB_CSB_C (0x0000000000000008)
+#define CRB_CSB_AT (0x0000000000000002)
+#define CRB_CSB_M (0x0000000000000001)
+
+struct coprocessor_request_block {
+ __be32 ccw;
+ __be32 flags;
+ __be64 csb_addr;
+
+ struct data_descriptor_entry source;
+ struct data_descriptor_entry target;
+
+ struct coprocessor_completion_block ccb;
+
+ u8 reserved[48];
+
+ struct coprocessor_status_block csb;
+} __aligned(CRB_ALIGN);
+
+#define crb_csb_addr(c) __be64_to_cpu(c->csb_addr)
+#define crb_nx_fault_addr(c) __be64_to_cpu(c->stamp.nx.fault_storage_addr)
+#define crb_nx_flags(c) c->stamp.nx.flags
+#define crb_nx_fault_status(c) c->stamp.nx.fault_status
+#define crb_nx_pswid(c) c->stamp.nx.pswid
+
+
+/* RFC02167 Initiate Coprocessor Instructions document
+ * Chapter 8.2.1.1.1 RS
+ * Chapter 8.2.3 Coprocessor Directive
+ * Chapter 8.2.4 Execution
+ *
+ * The CCW must be converted to BE before passing to icswx()
+ */
+
+#define CCW_PS (0xff000000)
+#define CCW_CT (0x00ff0000)
+#define CCW_CD (0x0000ffff)
+#define CCW_CL (0x0000c000)
+
+#endif
diff --git a/tools/testing/selftests/powerpc/nx-gzip/inc/nx-gzip.h b/tools/testing/selftests/powerpc/nx-gzip/inc/nx-gzip.h
new file mode 100644
index 000000000000..75482c45574d
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/inc/nx-gzip.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright 2020 IBM Corp.
+ *
+ */
+
+#ifndef _UAPI_MISC_VAS_H
+#define _UAPI_MISC_VAS_H
+
+#include <asm/ioctl.h>
+
+#define VAS_FLAGS_PIN_WINDOW 0x1
+#define VAS_FLAGS_HIGH_PRI 0x2
+
+#define VAS_FTW_SETUP _IOW('v', 1, struct vas_gzip_setup_attr)
+#define VAS_842_TX_WIN_OPEN _IOW('v', 2, struct vas_gzip_setup_attr)
+#define VAS_GZIP_TX_WIN_OPEN _IOW('v', 0x20, struct vas_gzip_setup_attr)
+
+struct vas_gzip_setup_attr {
+ int32_t version;
+ int16_t vas_id;
+ int16_t reserved1;
+ int64_t flags;
+ int64_t reserved2[6];
+};
+
+#endif /* _UAPI_MISC_VAS_H */
diff --git a/tools/testing/selftests/powerpc/nx-gzip/inc/nx-helpers.h b/tools/testing/selftests/powerpc/nx-gzip/inc/nx-helpers.h
new file mode 100644
index 000000000000..e0d68914c941
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/inc/nx-helpers.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#include <sys/time.h>
+#include <asm/byteorder.h>
+#include <stdint.h>
+#include <stdbool.h>
+#include "crb.h"
+
+#define cpu_to_be32 __cpu_to_be32
+#define cpu_to_be64 __cpu_to_be64
+#define be32_to_cpu __be32_to_cpu
+#define be64_to_cpu __be64_to_cpu
+
+/*
+ * Several helpers/macros below were copied from the tree
+ * (kernel.h, nx-842.h, nx-ftw.h, asm-compat.h etc)
+ */
+
+/* from kernel.h */
+#define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0)
+#define __round_mask(x, y) ((__typeof__(x))((y)-1))
+#define round_up(x, y) ((((x)-1) | __round_mask(x, y))+1)
+#define round_down(x, y) ((x) & ~__round_mask(x, y))
+
+#define min_t(t, x, y) ((x) < (y) ? (x) : (y))
+/*
+ * Get/Set bit fields. (from nx-842.h)
+ */
+#define GET_FIELD(m, v) (((v) & (m)) >> MASK_LSH(m))
+#define MASK_LSH(m) (__builtin_ffsl(m) - 1)
+#define SET_FIELD(m, v, val) \
+ (((v) & ~(m)) | ((((typeof(v))(val)) << MASK_LSH(m)) & (m)))
+
+/* From asm-compat.h */
+#define __stringify_in_c(...) #__VA_ARGS__
+#define stringify_in_c(...) __stringify_in_c(__VA_ARGS__) " "
+
+#define pr_debug
+#define pr_debug_ratelimited printf
+#define pr_err printf
+#define pr_err_ratelimited printf
+
+#define WARN_ON_ONCE(x) do {if (x) \
+ printf("WARNING: %s:%d\n", __func__, __LINE__)\
+ } while (0)
+
+extern void dump_buffer(char *msg, char *buf, int len);
+extern void *alloc_aligned_mem(int len, int align, char *msg);
+extern void get_payload(char *buf, int len);
+extern void time_add(struct timeval *in, int seconds, struct timeval *out);
+
+extern bool time_after(struct timeval *a, struct timeval *b);
+extern long time_delta(struct timeval *a, struct timeval *b);
+extern void dump_dde(struct data_descriptor_entry *dde, char *msg);
+extern void copy_paste_crb_data(struct coprocessor_request_block *crb);
diff --git a/tools/testing/selftests/powerpc/nx-gzip/inc/nx.h b/tools/testing/selftests/powerpc/nx-gzip/inc/nx.h
new file mode 100644
index 000000000000..1ae8348b59d6
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/inc/nx.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright 2020 IBM Corp.
+ *
+ */
+#ifndef _NX_H
+#define _NX_H
+
+#include <stdbool.h>
+
+#define NX_FUNC_COMP_842 1
+#define NX_FUNC_COMP_GZIP 2
+
+#ifndef __aligned
+#define __aligned(x) __attribute__((aligned(x)))
+#endif
+
+struct nx842_func_args {
+ bool use_crc;
+ bool decompress; /* true decompress; false compress */
+ bool move_data;
+ int timeout; /* seconds */
+};
+
+struct nxbuf_t {
+ int len;
+ char *buf;
+};
+
+/* @function should be EFT (aka 842), GZIP etc */
+extern void *nx_function_begin(int function, int pri);
+
+extern int nx_function(void *handle, struct nxbuf_t *in, struct nxbuf_t *out,
+ void *arg);
+
+extern int nx_function_end(void *handle);
+
+#endif /* _NX_H */
--
2.21.0
^ permalink raw reply related
* [PATCH V3 2/5] selftests/powerpc: Add header files for NX compresion/decompression
From: Raphael Moreira Zinsly @ 2020-04-13 15:59 UTC (permalink / raw)
To: linuxppc-dev, linux-crypto, dja; +Cc: abali, haren, herbert, rzinsly
In-Reply-To: <20200413155916.16900-1-rzinsly@linux.ibm.com>
Add files to be able to compress and decompress files using the
powerpc NX-GZIP engine.
Signed-off-by: Bulent Abali <abali@us.ibm.com>
Signed-off-by: Raphael Moreira Zinsly <rzinsly@linux.ibm.com>
---
.../powerpc/nx-gzip/inc/copy-paste.h | 54 ++
.../selftests/powerpc/nx-gzip/inc/nx_dbg.h | 95 +++
.../selftests/powerpc/nx-gzip/inc/nxu.h | 650 ++++++++++++++++++
3 files changed, 799 insertions(+)
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/inc/copy-paste.h
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/inc/nx_dbg.h
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/inc/nxu.h
diff --git a/tools/testing/selftests/powerpc/nx-gzip/inc/copy-paste.h b/tools/testing/selftests/powerpc/nx-gzip/inc/copy-paste.h
new file mode 100644
index 000000000000..107139b6c7df
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/inc/copy-paste.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "nx-helpers.h"
+
+/*
+ * Macros taken from arch/powerpc/include/asm/ppc-opcode.h and other
+ * header files.
+ */
+#define ___PPC_RA(a) (((a) & 0x1f) << 16)
+#define ___PPC_RB(b) (((b) & 0x1f) << 11)
+
+#define PPC_INST_COPY 0x7c20060c
+#define PPC_INST_PASTE 0x7c20070d
+
+#define PPC_COPY(a, b) stringify_in_c(.long PPC_INST_COPY | \
+ ___PPC_RA(a) | ___PPC_RB(b))
+#define PPC_PASTE(a, b) stringify_in_c(.long PPC_INST_PASTE | \
+ ___PPC_RA(a) | ___PPC_RB(b))
+#define CR0_SHIFT 28
+#define CR0_MASK 0xF
+/*
+ * Copy/paste instructions:
+ *
+ * copy RA,RB
+ * Copy contents of address (RA) + effective_address(RB)
+ * to internal copy-buffer.
+ *
+ * paste RA,RB
+ * Paste contents of internal copy-buffer to the address
+ * (RA) + effective_address(RB)
+ */
+static inline int vas_copy(void *crb, int offset)
+{
+ asm volatile(PPC_COPY(%0, %1)";"
+ :
+ : "b" (offset), "b" (crb)
+ : "memory");
+
+ return 0;
+}
+
+static inline int vas_paste(void *paste_address, int offset)
+{
+ u32 cr;
+
+ cr = 0;
+ asm volatile(PPC_PASTE(%1, %2)";"
+ "mfocrf %0, 0x80;"
+ : "=r" (cr)
+ : "b" (offset), "b" (paste_address)
+ : "memory", "cr0");
+
+ return (cr >> CR0_SHIFT) & CR0_MASK;
+}
diff --git a/tools/testing/selftests/powerpc/nx-gzip/inc/nx_dbg.h b/tools/testing/selftests/powerpc/nx-gzip/inc/nx_dbg.h
new file mode 100644
index 000000000000..f2c0eee2317e
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/inc/nx_dbg.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Copyright 2020 IBM Corporation
+ *
+ */
+
+#ifndef _NXU_DBG_H_
+#define _NXU_DBG_H_
+
+#include <sys/file.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <time.h>
+#include <pthread.h>
+
+extern FILE * nx_gzip_log;
+extern int nx_gzip_trace;
+extern unsigned int nx_gzip_inflate_impl;
+extern unsigned int nx_gzip_deflate_impl;
+extern unsigned int nx_gzip_inflate_flags;
+extern unsigned int nx_gzip_deflate_flags;
+
+extern int nx_dbg;
+pthread_mutex_t mutex_log;
+
+#define nx_gzip_trace_enabled() (nx_gzip_trace & 0x1)
+#define nx_gzip_hw_trace_enabled() (nx_gzip_trace & 0x2)
+#define nx_gzip_sw_trace_enabled() (nx_gzip_trace & 0x4)
+#define nx_gzip_gather_statistics() (nx_gzip_trace & 0x8)
+#define nx_gzip_per_stream_stat() (nx_gzip_trace & 0x10)
+
+#define prt(fmt, ...) do { \
+ pthread_mutex_lock(&mutex_log); \
+ flock(nx_gzip_log->_fileno, LOCK_EX); \
+ time_t t; struct tm *m; time(&t); m = localtime(&t); \
+ fprintf(nx_gzip_log, "[%04d/%02d/%02d %02d:%02d:%02d] " \
+ "pid %d: " fmt, \
+ (int)m->tm_year + 1900, (int)m->tm_mon+1, (int)m->tm_mday, \
+ (int)m->tm_hour, (int)m->tm_min, (int)m->tm_sec, \
+ (int)getpid(), ## __VA_ARGS__); \
+ fflush(nx_gzip_log); \
+ flock(nx_gzip_log->_fileno, LOCK_UN); \
+ pthread_mutex_unlock(&mutex_log); \
+} while (0)
+
+/* Use in case of an error */
+#define prt_err(fmt, ...) do { if (nx_dbg >= 0) { \
+ prt("%s:%u: Error: "fmt, \
+ __FILE__, __LINE__, ## __VA_ARGS__); \
+}} while (0)
+
+/* Use in case of an warning */
+#define prt_warn(fmt, ...) do { if (nx_dbg >= 1) { \
+ prt("%s:%u: Warning: "fmt, \
+ __FILE__, __LINE__, ## __VA_ARGS__); \
+}} while (0)
+
+/* Informational printouts */
+#define prt_info(fmt, ...) do { if (nx_dbg >= 2) { \
+ prt("Info: "fmt, ## __VA_ARGS__); \
+}} while (0)
+
+/* Trace zlib wrapper code */
+#define prt_trace(fmt, ...) do { if (nx_gzip_trace_enabled()) { \
+ prt("### "fmt, ## __VA_ARGS__); \
+}} while (0)
+
+/* Trace statistics */
+#define prt_stat(fmt, ...) do { if (nx_gzip_gather_statistics()) { \
+ prt("### "fmt, ## __VA_ARGS__); \
+}} while (0)
+
+/* Trace zlib hardware implementation */
+#define hw_trace(fmt, ...) do { \
+ if (nx_gzip_hw_trace_enabled()) \
+ fprintf(nx_gzip_log, "hhh " fmt, ## __VA_ARGS__); \
+ } while (0)
+
+/* Trace zlib software implementation */
+#define sw_trace(fmt, ...) do { \
+ if (nx_gzip_sw_trace_enabled()) \
+ fprintf(nx_gzip_log, "sss " fmt, ## __VA_ARGS__); \
+ } while (0)
+
+
+/**
+ * str_to_num - Convert string into number and copy with endings like
+ * KiB for kilobyte
+ * MiB for megabyte
+ * GiB for gigabyte
+ */
+uint64_t str_to_num(char *str);
+void nx_lib_debug(int onoff);
+
+#endif /* _NXU_DBG_H_ */
diff --git a/tools/testing/selftests/powerpc/nx-gzip/inc/nxu.h b/tools/testing/selftests/powerpc/nx-gzip/inc/nxu.h
new file mode 100644
index 000000000000..a9f07367b3e6
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/inc/nxu.h
@@ -0,0 +1,650 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later
+ *
+ * Hardware interface of the NX-GZIP compression accelerator
+ *
+ * Copyright (C) IBM Corporation, 2020
+ *
+ * Author: Bulent Abali <abali@us.ibm.com>
+ *
+ */
+
+#ifndef _NXU_H
+#define _NXU_H
+
+#include <stdint.h>
+#include <endian.h>
+#include "nx.h"
+
+/* deflate */
+#define LLSZ 286
+#define DSZ 30
+
+/* nx */
+#define DHTSZ 18
+#define DHT_MAXSZ 288
+#define MAX_DDE_COUNT 256
+
+/* util */
+#ifdef NXDBG
+#define NXPRT(X) X
+#else
+#define NXPRT(X)
+#endif
+
+#ifdef NXTIMER
+#include <sys/platform/ppc.h>
+#define NX_CLK(X) X
+#define nx_get_time() __ppc_get_timebase()
+#define nx_get_freq() __ppc_get_timebase_freq()
+#else
+#define NX_CLK(X)
+#define nx_get_time() (-1)
+#define nx_get_freq() (-1)
+#endif
+
+#define NX_MAX_FAULTS 500
+
+/*
+ * Definitions of acronyms used here. See
+ * P9 NX Gzip Accelerator User's Manual for details:
+ * https://github.com/libnxz/power-gzip/blob/develop/doc/power_nx_gzip_um.pdf
+ *
+ * adler/crc: 32 bit checksums appended to stream tail
+ * ce: completion extension
+ * cpb: coprocessor parameter block (metadata)
+ * crb: coprocessor request block (command)
+ * csb: coprocessor status block (status)
+ * dht: dynamic huffman table
+ * dde: data descriptor element (address, length)
+ * ddl: list of ddes
+ * dh/fh: dynamic and fixed huffman types
+ * fc: coprocessor function code
+ * histlen: history/dictionary length
+ * history: sliding window of up to 32KB of data
+ * lzcount: Deflate LZ symbol counts
+ * rembytecnt: remaining byte count
+ * sfbt: source final block type; last block's type during decomp
+ * spbc: source processed byte count
+ * subc: source unprocessed bit count
+ * tebc: target ending bit count; valid bits in the last byte
+ * tpbc: target processed byte count
+ * vas: virtual accelerator switch; the user mode interface
+ */
+
+union nx_qw_t {
+ uint32_t word[4];
+ uint64_t dword[2];
+} __aligned(16);
+
+/*
+ * Note: NX registers with fewer than 32 bits are declared by
+ * convention as uint32_t variables in unions. If *_offset and *_mask
+ * are defined for a variable, then use get_ put_ macros to
+ * conveniently access the register fields for endian conversions.
+ */
+
+struct nx_dde_t {
+ /* Data Descriptor Element, Section 6.4 */
+ union {
+ uint32_t dde_count;
+ /* When dde_count == 0 ddead is a pointer to a data buffer;
+ * ddebc is the buffer length bytes.
+ * When dde_count > 0 dde is an indirect dde; ddead is a
+ * pointer to a contiguous list of direct ddes; ddebc is the
+ * total length of all data pointed to by the list of direct
+ * ddes. Note that only one level of indirection is permitted.
+ * See Section 6.4 of the user manual for additional details.
+ */
+ };
+ uint32_t ddebc; /* dde byte count */
+ uint64_t ddead; /* dde address */
+} __aligned(16);
+
+struct nx_csb_t {
+ /* Coprocessor Status Block, Section 6.6 */
+ union {
+ uint32_t csb_v;
+ /* Valid bit. v must be set to 0 by the program
+ * before submitting the coprocessor command.
+ * Software can poll for the v bit
+ */
+
+ uint32_t csb_f;
+ /* 16B CSB size. Written to 0 by DMA when it writes the CPB */
+
+ uint32_t csb_cs;
+ /* cs completion sequence; unused */
+
+ uint32_t csb_cc;
+ /* cc completion code; cc != 0 exception occurred */
+
+ uint32_t csb_ce;
+ /* ce completion extension */
+
+ };
+ uint32_t tpbc;
+ /* target processed byte count TPBC */
+
+ uint64_t fsaddr;
+ /* Section 6.12.1 CSB NonZero error summary. FSA Failing storage
+ * address. Address where error occurred. When available, written
+ * to A field of CSB
+ */
+} __aligned(16);
+
+struct nx_ccb_t {
+ /* Coprocessor Completion Block, Section 6.7 */
+
+ uint32_t reserved[3];
+ union {
+ /* When crb.c==0 (no ccb defined) it is reserved;
+ * When crb.c==1 (ccb defined) it is cm
+ */
+
+ uint32_t ccb_cm;
+ /* Signal interrupt of crb.c==1 and cm==1 */
+
+ uint32_t word;
+ /* generic access to the 32bit word */
+ };
+} __aligned(16);
+
+struct vas_stamped_crb_t {
+ /*
+ * CRB operand of the paste coprocessor instruction is stamped
+ * in quadword 4 with the information shown here as its written
+ * in to the receive FIFO of the coprocessor
+ */
+
+ union {
+ uint32_t vas_buf_num;
+ /* Verification only vas buffer number which correlates to
+ * the low order bits of the atag in the paste command
+ */
+
+ uint32_t send_wc_id;
+ /* Pointer to Send Window Context that provides for NX address
+ * translation information, such as MSR and LPCR bits, job
+ * completion interrupt RA, PSWID, and job utilization counter.
+ */
+
+ };
+ union {
+ uint32_t recv_wc_id;
+ /* Pointer to Receive Window Context. NX uses this to return
+ * credits to a Receive FIFO as entries are dequeued.
+ */
+
+ };
+ uint32_t reserved2;
+ union {
+ uint32_t vas_invalid;
+ /* Invalid bit. If this bit is 1 the CRB is discarded by
+ * NX upon fetching from the receive FIFO. If this bit is 0
+ * the CRB is processed normally. The bit is stamped to 0
+ * by VAS and may be written to 1 by hypervisor while
+ * the CRB is in the receive FIFO (in memory).
+ */
+
+ };
+};
+
+struct nx_stamped_fault_crb_t {
+ /*
+ * A CRB that has a translation fault is stamped by NX in quadword 4
+ * and pasted to the Fault Send Window in VAS.
+ */
+ uint64_t fsa;
+ union {
+ uint32_t nxsf_t;
+ uint32_t nxsf_fs;
+ };
+ uint32_t pswid;
+};
+
+union stamped_crb_t {
+ struct vas_stamped_crb_t vas;
+ struct nx_stamped_fault_crb_t nx;
+};
+
+struct nx_gzip_cpb_t {
+ /*
+ * Coprocessor Parameter Block In/Out are used to pass metadata
+ * to/from accelerator. Tables 6.5 and 6.6 of the user manual.
+ */
+
+ /* CPBInput */
+
+ struct {
+ union {
+ union nx_qw_t qw0;
+ struct {
+ uint32_t in_adler; /* bits 0:31 */
+ uint32_t in_crc; /* bits 32:63 */
+ union {
+ uint32_t in_histlen; /* bits 64:75 */
+ uint32_t in_subc; /* bits 93:95 */
+ };
+ union {
+ /* bits 108:111 */
+ uint32_t in_sfbt;
+ /* bits 112:127 */
+ uint32_t in_rembytecnt;
+ /* bits 116:127 */
+ uint32_t in_dhtlen;
+ };
+ };
+ };
+ union {
+ union nx_qw_t in_dht[DHTSZ]; /* qw[1:18] */
+ char in_dht_char[DHT_MAXSZ]; /* byte access */
+ };
+ union nx_qw_t reserved[5]; /* qw[19:23] */
+ };
+
+ /* CPBOutput */
+
+ volatile struct {
+ union {
+ union nx_qw_t qw24;
+ struct {
+ uint32_t out_adler; /* bits 0:31 qw[24] */
+ uint32_t out_crc; /* bits 32:63 qw[24] */
+ union {
+ /* bits 77:79 qw[24] */
+ uint32_t out_tebc;
+ /* bits 80:95 qw[24] */
+ uint32_t out_subc;
+ };
+ union {
+ /* bits 108:111 qw[24] */
+ uint32_t out_sfbt;
+ /* bits 112:127 qw[24] */
+ uint32_t out_rembytecnt;
+ /* bits 116:127 qw[24] */
+ uint32_t out_dhtlen;
+ };
+ };
+ };
+ union {
+ union nx_qw_t qw25[79]; /* qw[25:103] */
+ /* qw[25] compress no lzcounts or wrap */
+ uint32_t out_spbc_comp_wrap;
+ uint32_t out_spbc_wrap; /* qw[25] wrap */
+ /* qw[25] compress no lzcounts */
+ uint32_t out_spbc_comp;
+ /* 286 LL and 30 D symbol counts */
+ uint32_t out_lzcount[LLSZ+DSZ];
+ struct {
+ union nx_qw_t out_dht[DHTSZ]; /* qw[25:42] */
+ /* qw[43] decompress */
+ uint32_t out_spbc_decomp;
+ };
+ };
+ /* qw[104] compress with lzcounts */
+ uint32_t out_spbc_comp_with_count;
+ };
+} __aligned(128);
+
+struct nx_gzip_crb_t {
+ union { /* byte[0:3] */
+ uint32_t gzip_fc; /* bits[24-31] */
+ };
+ uint32_t reserved1; /* byte[4:7] */
+ union {
+ uint64_t csb_address; /* byte[8:15] */
+ struct {
+ uint32_t reserved2;
+ union {
+ uint32_t crb_c;
+ /* c==0 no ccb defined */
+
+ uint32_t crb_at;
+ /* at==0 address type is ignored;
+ * all addrs effective assumed.
+ */
+
+ };
+ };
+ };
+ struct nx_dde_t source_dde; /* byte[16:31] */
+ struct nx_dde_t target_dde; /* byte[32:47] */
+ volatile struct nx_ccb_t ccb; /* byte[48:63] */
+ volatile union {
+ /* byte[64:239] shift csb by 128 bytes out of the crb; csb was
+ * in crb earlier; JReilly says csb written with partial inject
+ */
+ union nx_qw_t reserved64[11];
+ union stamped_crb_t stamp; /* byte[64:79] */
+ };
+ volatile struct nx_csb_t csb;
+} __aligned(128);
+
+struct nx_gzip_crb_cpb_t {
+ struct nx_gzip_crb_t crb;
+ struct nx_gzip_cpb_t cpb;
+} __aligned(2048);
+
+
+/*
+ * NX hardware convention has the msb bit on the left numbered 0.
+ * The defines below has *_offset defined as the right most bit
+ * position of a field. x of size_mask(x) is the field width in bits.
+ */
+
+#define size_mask(x) ((1U<<(x))-1)
+
+/*
+ * Offsets and Widths within the containing 32 bits of the various NX
+ * gzip hardware registers. Use the getnn/putnn macros to access
+ * these regs
+ */
+
+#define dde_count_mask size_mask(8)
+#define dde_count_offset 23
+
+/* CSB */
+
+#define csb_v_mask size_mask(1)
+#define csb_v_offset 0
+#define csb_f_mask size_mask(1)
+#define csb_f_offset 6
+#define csb_cs_mask size_mask(8)
+#define csb_cs_offset 15
+#define csb_cc_mask size_mask(8)
+#define csb_cc_offset 23
+#define csb_ce_mask size_mask(8)
+#define csb_ce_offset 31
+
+/* CCB */
+
+#define ccb_cm_mask size_mask(3)
+#define ccb_cm_offset 31
+
+/* VAS stamped CRB fields */
+
+#define vas_buf_num_mask size_mask(6)
+#define vas_buf_num_offset 5
+#define send_wc_id_mask size_mask(16)
+#define send_wc_id_offset 31
+#define recv_wc_id_mask size_mask(16)
+#define recv_wc_id_offset 31
+#define vas_invalid_mask size_mask(1)
+#define vas_invalid_offset 31
+
+/* NX stamped fault CRB fields */
+
+#define nxsf_t_mask size_mask(1)
+#define nxsf_t_offset 23
+#define nxsf_fs_mask size_mask(8)
+#define nxsf_fs_offset 31
+
+/* CPB input */
+
+#define in_histlen_mask size_mask(12)
+#define in_histlen_offset 11
+#define in_dhtlen_mask size_mask(12)
+#define in_dhtlen_offset 31
+#define in_subc_mask size_mask(3)
+#define in_subc_offset 31
+#define in_sfbt_mask size_mask(4)
+#define in_sfbt_offset 15
+#define in_rembytecnt_mask size_mask(16)
+#define in_rembytecnt_offset 31
+
+/* CPB output */
+
+#define out_tebc_mask size_mask(3)
+#define out_tebc_offset 15
+#define out_subc_mask size_mask(16)
+#define out_subc_offset 31
+#define out_sfbt_mask size_mask(4)
+#define out_sfbt_offset 15
+#define out_rembytecnt_mask size_mask(16)
+#define out_rembytecnt_offset 31
+#define out_dhtlen_mask size_mask(12)
+#define out_dhtlen_offset 31
+
+/* CRB */
+
+#define gzip_fc_mask size_mask(8)
+#define gzip_fc_offset 31
+#define crb_c_mask size_mask(1)
+#define crb_c_offset 28
+#define crb_at_mask size_mask(1)
+#define crb_at_offset 30
+#define csb_address_mask ~(15UL) /* mask off bottom 4b */
+
+/*
+ * Access macros for the registers. Do not access registers directly
+ * because of the endian conversion. P9 processor may run either as
+ * Little or Big endian. However the NX coprocessor regs are always
+ * big endian.
+ * Use the 32 and 64b macros to access respective
+ * register sizes.
+ * Use nn forms for the register fields shorter than 32 bits.
+ */
+
+#define getnn(ST, REG) ((be32toh(ST.REG) >> (31-REG##_offset)) \
+ & REG##_mask)
+#define getpnn(ST, REG) ((be32toh((ST)->REG) >> (31-REG##_offset)) \
+ & REG##_mask)
+#define get32(ST, REG) (be32toh(ST.REG))
+#define getp32(ST, REG) (be32toh((ST)->REG))
+#define get64(ST, REG) (be64toh(ST.REG))
+#define getp64(ST, REG) (be64toh((ST)->REG))
+
+#define unget32(ST, REG) (get32(ST, REG) & ~((REG##_mask) \
+ << (31-REG##_offset)))
+/* get 32bits less the REG field */
+
+#define ungetp32(ST, REG) (getp32(ST, REG) & ~((REG##_mask) \
+ << (31-REG##_offset)))
+/* get 32bits less the REG field */
+
+#define clear_regs(ST) memset((void *)(&(ST)), 0, sizeof(ST))
+#define clear_dde(ST) do { ST.dde_count = ST.ddebc = 0; ST.ddead = 0; \
+ } while (0)
+#define clearp_dde(ST) do { (ST)->dde_count = (ST)->ddebc = 0; \
+ (ST)->ddead = 0; \
+ } while (0)
+#define clear_struct(ST) memset((void *)(&(ST)), 0, sizeof(ST))
+#define putnn(ST, REG, X) (ST.REG = htobe32(unget32(ST, REG) | (((X) \
+ & REG##_mask) << (31-REG##_offset))))
+#define putpnn(ST, REG, X) ((ST)->REG = htobe32(ungetp32(ST, REG) \
+ | (((X) & REG##_mask) << (31-REG##_offset))))
+
+#define put32(ST, REG, X) (ST.REG = htobe32(X))
+#define putp32(ST, REG, X) ((ST)->REG = htobe32(X))
+#define put64(ST, REG, X) (ST.REG = htobe64(X))
+#define putp64(ST, REG, X) ((ST)->REG = htobe64(X))
+
+/*
+ * Completion extension ce(0) ce(1) ce(2). Bits ce(3-7)
+ * unused. Section 6.6 Figure 6.7.
+ */
+
+#define get_csb_ce(ST) ((uint32_t)getnn(ST, csb_ce))
+#define get_csb_ce_ms3b(ST) (get_csb_ce(ST) >> 5)
+#define put_csb_ce_ms3b(ST, X) putnn(ST, csb_ce, ((uint32_t)(X) << 5))
+
+#define CSB_CE_PARTIAL 0x4
+#define CSB_CE_TERMINATE 0x2
+#define CSB_CE_TPBC_VALID 0x1
+
+#define csb_ce_termination(X) (!!((X) & CSB_CE_TERMINATE))
+/* termination, output buffers may be modified, SPBC/TPBC invalid Fig.6-7 */
+
+#define csb_ce_check_completion(X) (!csb_ce_termination(X))
+/* if not terminated then check full or partial completion */
+
+#define csb_ce_partial_completion(X) (!!((X) & CSB_CE_PARTIAL))
+#define csb_ce_full_completion(X) (!csb_ce_partial_completion(X))
+#define csb_ce_tpbc_valid(X) (!!((X) & CSB_CE_TPBC_VALID))
+/* TPBC indicates successfully stored data count */
+
+#define csb_ce_default_err(X) csb_ce_termination(X)
+/* most error CEs have CE(0)=0 and CE(1)=1 */
+
+#define csb_ce_cc3_partial(X) csb_ce_partial_completion(X)
+/* some CC=3 are partially completed, Table 6-8 */
+
+#define csb_ce_cc64(X) ((X)&(CSB_CE_PARTIAL \
+ | CSB_CE_TERMINATE) == 0)
+/* Compression: when TPBC>SPBC then CC=64 Table 6-8; target didn't
+ * compress smaller than source.
+ */
+
+/* Decompress SFBT combinations Tables 5-3, 6-4, 6-6 */
+
+#define SFBT_BFINAL 0x1
+#define SFBT_LIT 0x4
+#define SFBT_FHT 0x5
+#define SFBT_DHT 0x6
+#define SFBT_HDR 0x7
+
+/*
+ * NX gzip function codes. Table 6.2.
+ * Bits 0:4 are the FC. Bit 5 is used by the DMA controller to
+ * select one of the two Byte Count Limits.
+ */
+
+#define GZIP_FC_LIMIT_MASK 0x01
+#define GZIP_FC_COMPRESS_FHT 0x00
+#define GZIP_FC_COMPRESS_DHT 0x02
+#define GZIP_FC_COMPRESS_FHT_COUNT 0x04
+#define GZIP_FC_COMPRESS_DHT_COUNT 0x06
+#define GZIP_FC_COMPRESS_RESUME_FHT 0x08
+#define GZIP_FC_COMPRESS_RESUME_DHT 0x0a
+#define GZIP_FC_COMPRESS_RESUME_FHT_COUNT 0x0c
+#define GZIP_FC_COMPRESS_RESUME_DHT_COUNT 0x0e
+#define GZIP_FC_DECOMPRESS 0x10
+#define GZIP_FC_DECOMPRESS_SINGLE_BLK_N_SUSPEND 0x12
+#define GZIP_FC_DECOMPRESS_RESUME 0x14
+#define GZIP_FC_DECOMPRESS_RESUME_SINGLE_BLK_N_SUSPEND 0x16
+#define GZIP_FC_WRAP 0x1e
+
+#define fc_is_compress(fc) (((fc) & 0x10) == 0)
+#define fc_has_count(fc) (fc_is_compress(fc) && (((fc) & 0x4) != 0))
+
+/* CSB.CC Error codes */
+
+#define ERR_NX_OK 0
+#define ERR_NX_ALIGNMENT 1
+#define ERR_NX_OPOVERLAP 2
+#define ERR_NX_DATA_LENGTH 3
+#define ERR_NX_TRANSLATION 5
+#define ERR_NX_PROTECTION 6
+#define ERR_NX_EXTERNAL_UE7 7
+#define ERR_NX_INVALID_OP 8
+#define ERR_NX_PRIVILEGE 9
+#define ERR_NX_INTERNAL_UE 10
+#define ERR_NX_EXTERN_UE_WR 12
+#define ERR_NX_TARGET_SPACE 13
+#define ERR_NX_EXCESSIVE_DDE 14
+#define ERR_NX_TRANSL_WR 15
+#define ERR_NX_PROTECT_WR 16
+#define ERR_NX_SUBFUNCTION 17
+#define ERR_NX_FUNC_ABORT 18
+#define ERR_NX_BYTE_MAX 19
+#define ERR_NX_CORRUPT_CRB 20
+#define ERR_NX_INVALID_CRB 21
+#define ERR_NX_INVALID_DDE 30
+#define ERR_NX_SEGMENTED_DDL 31
+#define ERR_NX_DDE_OVERFLOW 33
+#define ERR_NX_TPBC_GT_SPBC 64
+#define ERR_NX_MISSING_CODE 66
+#define ERR_NX_INVALID_DIST 67
+#define ERR_NX_INVALID_DHT 68
+#define ERR_NX_EXTERNAL_UE90 90
+#define ERR_NX_WDOG_TIMER 224
+#define ERR_NX_AT_FAULT 250
+#define ERR_NX_INTR_SERVER 252
+#define ERR_NX_UE253 253
+#define ERR_NX_NO_HW 254
+#define ERR_NX_HUNG_OP 255
+#define ERR_NX_END 256
+
+/* initial values for non-resume operations */
+#define INIT_CRC 0 /* crc32(0L, Z_NULL, 0) */
+#define INIT_ADLER 1 /* adler32(0L, Z_NULL, 0) adler is initialized to 1 */
+
+/* prototypes */
+int nxu_submit_job(struct nx_gzip_crb_cpb_t *c, void *handle);
+
+extern void nxu_sigsegv_handler(int sig, siginfo_t *info, void *ctx);
+extern int nxu_touch_pages(void *buf, long buf_len, long page_len, int wr);
+
+/* caller supplies a print buffer 4*sizeof(crb) */
+
+char *nx_crb_str(struct nx_gzip_crb_t *crb, char *prbuf);
+char *nx_cpb_str(struct nx_gzip_cpb_t *cpb, char *prbuf);
+char *nx_prt_hex(void *cp, int sz, char *prbuf);
+char *nx_lzcount_str(struct nx_gzip_cpb_t *cpb, char *prbuf);
+char *nx_strerror(int e);
+
+#ifdef NX_SIM
+#include <stdio.h>
+int nx_sim_init(void *ctx);
+int nx_sim_end(void *ctx);
+int nxu_run_sim_job(struct nx_gzip_crb_cpb_t *c, void *ctx);
+#endif /* NX_SIM */
+
+/* Deflate stream manipulation */
+
+#define set_final_bit(x) (x |= (unsigned char)1)
+#define clr_final_bit(x) (x &= ~(unsigned char)1)
+
+#define append_empty_fh_blk(p, b) do { *(p) = (2 | (1&(b))); *((p)+1) = 0; \
+ } while (0)
+/* append 10 bits 0000001b 00...... ;
+ * assumes appending starts on a byte boundary; b is the final bit.
+ */
+
+
+#ifdef NX_842
+
+/* 842 Engine */
+
+struct nx_eft_crb_t {
+ union { /* byte[0:3] */
+ uint32_t eft_fc; /* bits[29-31] */
+ };
+ uint32_t reserved1; /* byte[4:7] */
+ union {
+ uint64_t csb_address; /* byte[8:15] */
+ struct {
+ uint32_t reserved2;
+ union {
+ uint32_t crb_c;
+ /* c==0 no ccb defined */
+
+ uint32_t crb_at;
+ /* at==0 address type is ignored;
+ * all addrs effective assumed.
+ */
+
+ };
+ };
+ };
+ struct nx_dde_t source_dde; /* byte[16:31] */
+ struct nx_dde_t target_dde; /* byte[32:47] */
+ struct nx_ccb_t ccb; /* byte[48:63] */
+ union {
+ union nx_qw_t reserved64[3]; /* byte[64:96] */
+ };
+ struct nx_csb_t csb;
+} __aligned(128);
+
+/* 842 CRB */
+
+#define EFT_FC_MASK size_mask(3)
+#define EFT_FC_OFFSET 31
+#define EFT_FC_COMPRESS 0x0
+#define EFT_FC_COMPRESS_WITH_CRC 0x1
+#define EFT_FC_DECOMPRESS 0x2
+#define EFT_FC_DECOMPRESS_WITH_CRC 0x3
+#define EFT_FC_BLK_DATA_MOVE 0x4
+#endif /* NX_842 */
+
+#endif /* _NXU_H */
--
2.21.0
^ permalink raw reply related
* [PATCH V3 3/5] selftests/powerpc: Add NX-GZIP engine compress testcase
From: Raphael Moreira Zinsly @ 2020-04-13 15:59 UTC (permalink / raw)
To: linuxppc-dev, linux-crypto, dja; +Cc: abali, haren, herbert, rzinsly
In-Reply-To: <20200413155916.16900-1-rzinsly@linux.ibm.com>
Add a compression testcase for the powerpc NX-GZIP engine.
Signed-off-by: Bulent Abali <abali@us.ibm.com>
Signed-off-by: Raphael Moreira Zinsly <rzinsly@linux.ibm.com>
---
.../selftests/powerpc/nx-gzip/Makefile | 21 +
.../selftests/powerpc/nx-gzip/gzfht_test.c | 432 ++++++++++++++++++
.../selftests/powerpc/nx-gzip/gzip_vas.c | 316 +++++++++++++
3 files changed, 769 insertions(+)
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/Makefile
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/gzfht_test.c
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/gzip_vas.c
diff --git a/tools/testing/selftests/powerpc/nx-gzip/Makefile b/tools/testing/selftests/powerpc/nx-gzip/Makefile
new file mode 100644
index 000000000000..ab903f63bbbd
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/Makefile
@@ -0,0 +1,21 @@
+CC = gcc
+CFLAGS = -O3
+INC = ./inc
+SRC = gzfht_test.c
+OBJ = $(SRC:.c=.o)
+TESTS = gzfht_test
+EXTRA_SOURCES = gzip_vas.c
+
+all: $(TESTS)
+
+$(OBJ): %.o: %.c
+ $(CC) $(CFLAGS) -I$(INC) -c $<
+
+$(TESTS): $(OBJ)
+ $(CC) $(CFLAGS) -I$(INC) -o $@ $@.o $(EXTRA_SOURCES)
+
+run_tests: $(TESTS)
+ ./gzfht_test gzip_vas.c
+
+clean:
+ rm -f $(TESTS) *.o *~ *.gz
diff --git a/tools/testing/selftests/powerpc/nx-gzip/gzfht_test.c b/tools/testing/selftests/powerpc/nx-gzip/gzfht_test.c
new file mode 100644
index 000000000000..e60f743e2c6b
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/gzfht_test.c
@@ -0,0 +1,432 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+
+/* P9 gzip sample code for demonstrating the P9 NX hardware interface.
+ * Not intended for productive uses or for performance or compression
+ * ratio measurements. For simplicity of demonstration, this sample
+ * code compresses in to fixed Huffman blocks only (Deflate btype=1)
+ * and has very simple memory management. Dynamic Huffman blocks
+ * (Deflate btype=2) are more involved as detailed in the user guide.
+ * Note also that /dev/crypto/gzip, VAS and skiboot support are
+ * required.
+ *
+ * Copyright 2020 IBM Corp.
+ *
+ * https://github.com/libnxz/power-gzip for zlib api and other utils
+ *
+ * Author: Bulent Abali <abali@us.ibm.com>
+ *
+ * Definitions of acronyms used here. See
+ * P9 NX Gzip Accelerator User's Manual for details:
+ * https://github.com/libnxz/power-gzip/blob/develop/doc/power_nx_gzip_um.pdf
+ *
+ * adler/crc: 32 bit checksums appended to stream tail
+ * ce: completion extension
+ * cpb: coprocessor parameter block (metadata)
+ * crb: coprocessor request block (command)
+ * csb: coprocessor status block (status)
+ * dht: dynamic huffman table
+ * dde: data descriptor element (address, length)
+ * ddl: list of ddes
+ * dh/fh: dynamic and fixed huffman types
+ * fc: coprocessor function code
+ * histlen: history/dictionary length
+ * history: sliding window of up to 32KB of data
+ * lzcount: Deflate LZ symbol counts
+ * rembytecnt: remaining byte count
+ * sfbt: source final block type; last block's type during decomp
+ * spbc: source processed byte count
+ * subc: source unprocessed bit count
+ * tebc: target ending bit count; valid bits in the last byte
+ * tpbc: target processed byte count
+ * vas: virtual accelerator switch; the user mode interface
+ */
+
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdint.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/time.h>
+#include <sys/fcntl.h>
+#include <sys/mman.h>
+#include <endian.h>
+#include <bits/endian.h>
+#include <sys/ioctl.h>
+#include <assert.h>
+#include <errno.h>
+#include <signal.h>
+#include "nxu.h"
+#include "nx.h"
+
+int nx_dbg;
+FILE *nx_gzip_log;
+
+#define NX_MIN(X, Y) (((X) < (Y)) ? (X) : (Y))
+#define FNAME_MAX 1024
+#define FEXT ".nx.gz"
+
+/*
+ * LZ counts returned in the user supplied nx_gzip_crb_cpb_t structure.
+ */
+static int compress_fht_sample(char *src, uint32_t srclen, char *dst,
+ uint32_t dstlen, int with_count,
+ struct nx_gzip_crb_cpb_t *cmdp, void *handle)
+{
+ int cc;
+ uint32_t fc;
+
+ assert(!!cmdp);
+
+ put32(cmdp->crb, gzip_fc, 0); /* clear */
+ fc = (with_count) ? GZIP_FC_COMPRESS_RESUME_FHT_COUNT :
+ GZIP_FC_COMPRESS_RESUME_FHT;
+ putnn(cmdp->crb, gzip_fc, fc);
+ putnn(cmdp->cpb, in_histlen, 0); /* resuming with no history */
+ memset((void *) &cmdp->crb.csb, 0, sizeof(cmdp->crb.csb));
+
+ /* Section 6.6 programming notes; spbc may be in two different
+ * places depending on FC.
+ */
+ if (!with_count)
+ put32(cmdp->cpb, out_spbc_comp, 0);
+ else
+ put32(cmdp->cpb, out_spbc_comp_with_count, 0);
+
+ /* Figure 6-3 6-4; CSB location */
+ put64(cmdp->crb, csb_address, 0);
+ put64(cmdp->crb, csb_address,
+ (uint64_t) &cmdp->crb.csb & csb_address_mask);
+
+ /* Source direct dde (scatter-gather list) */
+ clear_dde(cmdp->crb.source_dde);
+ putnn(cmdp->crb.source_dde, dde_count, 0);
+ put32(cmdp->crb.source_dde, ddebc, srclen);
+ put64(cmdp->crb.source_dde, ddead, (uint64_t) src);
+
+ /* Target direct dde (scatter-gather list) */
+ clear_dde(cmdp->crb.target_dde);
+ putnn(cmdp->crb.target_dde, dde_count, 0);
+ put32(cmdp->crb.target_dde, ddebc, dstlen);
+ put64(cmdp->crb.target_dde, ddead, (uint64_t) dst);
+
+ /* Submit the crb, the job descriptor, to the accelerator */
+ return nxu_submit_job(cmdp, handle);
+}
+
+/*
+ * Prepares a blank no filename no timestamp gzip header and returns
+ * the number of bytes written to buf.
+ * Gzip specification at https://tools.ietf.org/html/rfc1952
+ */
+int gzip_header_blank(char *buf)
+{
+ int i = 0;
+
+ buf[i++] = 0x1f; /* ID1 */
+ buf[i++] = 0x8b; /* ID2 */
+ buf[i++] = 0x08; /* CM */
+ buf[i++] = 0x00; /* FLG */
+ buf[i++] = 0x00; /* MTIME */
+ buf[i++] = 0x00; /* MTIME */
+ buf[i++] = 0x00; /* MTIME */
+ buf[i++] = 0x00; /* MTIME */
+ buf[i++] = 0x04; /* XFL 4=fastest */
+ buf[i++] = 0x03; /* OS UNIX */
+
+ return i;
+}
+
+/* Caller must free the allocated buffer return nonzero on error. */
+int read_alloc_input_file(char *fname, char **buf, size_t *bufsize)
+{
+ struct stat statbuf;
+ FILE *fp;
+ char *p;
+ size_t num_bytes;
+
+ if (stat(fname, &statbuf)) {
+ perror(fname);
+ return(-1);
+ }
+ fp = fopen(fname, "r");
+ if (fp == NULL) {
+ perror(fname);
+ return(-1);
+ }
+ assert(NULL != (p = (char *) malloc(statbuf.st_size)));
+ num_bytes = fread(p, 1, statbuf.st_size, fp);
+ if (ferror(fp) || (num_bytes != statbuf.st_size)) {
+ perror(fname);
+ return(-1);
+ }
+ *buf = p;
+ *bufsize = num_bytes;
+ return 0;
+}
+
+/* Returns nonzero on error */
+int write_output_file(char *fname, char *buf, size_t bufsize)
+{
+ FILE *fp;
+ size_t num_bytes;
+
+ fp = fopen(fname, "w");
+ if (fp == NULL) {
+ perror(fname);
+ return(-1);
+ }
+ num_bytes = fwrite(buf, 1, bufsize, fp);
+ if (ferror(fp) || (num_bytes != bufsize)) {
+ perror(fname);
+ return(-1);
+ }
+ fclose(fp);
+ return 0;
+}
+
+/*
+ * Z_SYNC_FLUSH as described in zlib.h.
+ * Returns number of appended bytes
+ */
+int append_sync_flush(char *buf, int tebc, int final)
+{
+ uint64_t flush;
+ int shift = (tebc & 0x7);
+
+ if (tebc > 0) {
+ /* Last byte is partially full */
+ buf = buf - 1;
+ *buf = *buf & (unsigned char) ((1<<tebc)-1);
+ } else
+ *buf = 0;
+ flush = ((0x1ULL & final) << shift) | *buf;
+ shift = shift + 3; /* BFINAL and BTYPE written */
+ shift = (shift <= 8) ? 8 : 16;
+ flush |= (0xFFFF0000ULL) << shift; /* Zero length block */
+ shift = shift + 32;
+ while (shift > 0) {
+ *buf++ = (unsigned char) (flush & 0xffULL);
+ flush = flush >> 8;
+ shift = shift - 8;
+ }
+ return(((tebc > 5) || (tebc == 0)) ? 5 : 4);
+}
+
+/*
+ * Final deflate block bit. This call assumes the block
+ * beginning is byte aligned.
+ */
+static void set_bfinal(void *buf, int bfinal)
+{
+ char *b = buf;
+
+ if (bfinal)
+ *b = *b | (unsigned char) 0x01;
+ else
+ *b = *b & (unsigned char) 0xfe;
+}
+
+int compress_file(int argc, char **argv, void *handle)
+{
+ char *inbuf, *outbuf, *srcbuf, *dstbuf;
+ char outname[FNAME_MAX];
+ uint32_t srclen, dstlen;
+ uint32_t flushlen, chunk;
+ size_t inlen, outlen, dsttotlen, srctotlen;
+ uint32_t crc, spbc, tpbc, tebc;
+ int lzcounts = 0;
+ int cc;
+ int num_hdr_bytes;
+ struct nx_gzip_crb_cpb_t *cmdp;
+ uint32_t pagelen = 65536;
+ int fault_tries = NX_MAX_FAULTS;
+
+ cmdp = (void *)(uintptr_t)
+ aligned_alloc(sizeof(struct nx_gzip_crb_cpb_t),
+ sizeof(struct nx_gzip_crb_cpb_t));
+
+ if (argc != 2) {
+ fprintf(stderr, "usage: %s <fname>\n", argv[0]);
+ exit(-1);
+ }
+ if (read_alloc_input_file(argv[1], &inbuf, &inlen))
+ exit(-1);
+ fprintf(stderr, "file %s read, %ld bytes\n", argv[1], inlen);
+
+ /* Generous output buffer for header/trailer */
+ outlen = 2 * inlen + 1024;
+
+ assert(NULL != (outbuf = (char *)malloc(outlen)));
+ nxu_touch_pages(outbuf, outlen, pagelen, 1);
+
+ /* Compress piecemeal in smallish chunks */
+ chunk = 1<<22;
+
+ /* Write the gzip header to the stream */
+ num_hdr_bytes = gzip_header_blank(outbuf);
+ dstbuf = outbuf + num_hdr_bytes;
+ outlen = outlen - num_hdr_bytes;
+ dsttotlen = num_hdr_bytes;
+
+ srcbuf = inbuf;
+ srctotlen = 0;
+
+ /* Init the CRB, the coprocessor request block */
+ memset(&cmdp->crb, 0, sizeof(cmdp->crb));
+
+ /* Initial gzip crc32 */
+ put32(cmdp->cpb, in_crc, 0);
+
+ while (inlen > 0) {
+
+ /* Submit chunk size source data per job */
+ srclen = NX_MIN(chunk, inlen);
+ /* Supply large target in case data expands */
+ dstlen = NX_MIN(2*srclen, outlen);
+
+ /* Page faults are handled by the user code */
+
+ /* Fault-in pages; an improved code wouldn't touch so
+ * many pages but would try to estimate the
+ * compression ratio and adjust both the src and dst
+ * touch amounts.
+ */
+ nxu_touch_pages(cmdp, sizeof(struct nx_gzip_crb_cpb_t), pagelen,
+ 1);
+ nxu_touch_pages(srcbuf, srclen, pagelen, 0);
+ nxu_touch_pages(dstbuf, dstlen, pagelen, 1);
+
+ cc = compress_fht_sample(
+ srcbuf, srclen,
+ dstbuf, dstlen,
+ lzcounts, cmdp, handle);
+
+ if (cc != ERR_NX_OK && cc != ERR_NX_TPBC_GT_SPBC &&
+ cc != ERR_NX_TRANSLATION) {
+ fprintf(stderr, "nx error: cc= %d\n", cc);
+ exit(-1);
+ }
+
+ /* Page faults are handled by the user code */
+ if (cc == ERR_NX_TRANSLATION) {
+ NXPRT(fprintf(stderr, "page fault: cc= %d, ", cc));
+ NXPRT(fprintf(stderr, "try= %d, fsa= %08llx\n",
+ fault_tries,
+ (unsigned long long) cmdp->crb.csb.fsaddr));
+ fault_tries--;
+ if (fault_tries > 0) {
+ continue;
+ } else {
+ fprintf(stderr, "error: cannot progress; ");
+ fprintf(stderr, "too many faults\n");
+ exit(-1);
+ };
+ }
+
+ fault_tries = NX_MAX_FAULTS; /* Reset for the next chunk */
+
+ inlen = inlen - srclen;
+ srcbuf = srcbuf + srclen;
+ srctotlen = srctotlen + srclen;
+
+ /* Two possible locations for spbc depending on the function
+ * code.
+ */
+ spbc = (!lzcounts) ? get32(cmdp->cpb, out_spbc_comp) :
+ get32(cmdp->cpb, out_spbc_comp_with_count);
+ assert(spbc == srclen);
+
+ /* Target byte count */
+ tpbc = get32(cmdp->crb.csb, tpbc);
+ /* Target ending bit count */
+ tebc = getnn(cmdp->cpb, out_tebc);
+ NXPRT(fprintf(stderr, "compressed chunk %d ", spbc));
+ NXPRT(fprintf(stderr, "to %d bytes, tebc= %d\n", tpbc, tebc));
+
+ if (inlen > 0) { /* More chunks to go */
+ set_bfinal(dstbuf, 0);
+ dstbuf = dstbuf + tpbc;
+ dsttotlen = dsttotlen + tpbc;
+ outlen = outlen - tpbc;
+ /* Round up to the next byte with a flush
+ * block; do not set the BFINAqL bit.
+ */
+ flushlen = append_sync_flush(dstbuf, tebc, 0);
+ dsttotlen = dsttotlen + flushlen;
+ outlen = outlen - flushlen;
+ dstbuf = dstbuf + flushlen;
+ NXPRT(fprintf(stderr, "added sync_flush %d bytes\n",
+ flushlen));
+ } else { /* Done */
+ /* Set the BFINAL bit of the last block per Deflate
+ * specification.
+ */
+ set_bfinal(dstbuf, 1);
+ dstbuf = dstbuf + tpbc;
+ dsttotlen = dsttotlen + tpbc;
+ outlen = outlen - tpbc;
+ }
+
+ /* Resuming crc32 for the next chunk */
+ crc = get32(cmdp->cpb, out_crc);
+ put32(cmdp->cpb, in_crc, crc);
+ crc = be32toh(crc);
+ }
+
+ /* Append crc32 and ISIZE to the end */
+ memcpy(dstbuf, &crc, 4);
+ memcpy(dstbuf+4, &srctotlen, 4);
+ dsttotlen = dsttotlen + 8;
+ outlen = outlen - 8;
+
+ assert(FNAME_MAX > (strlen(argv[1]) + strlen(FEXT)));
+ strcpy(outname, argv[1]);
+ strcat(outname, FEXT);
+ if (write_output_file(outname, outbuf, dsttotlen)) {
+ fprintf(stderr, "write error: %s\n", outname);
+ exit(-1);
+ }
+
+ fprintf(stderr, "compressed %ld to %ld bytes total, ", srctotlen,
+ dsttotlen);
+ fprintf(stderr, "crc32 checksum = %08x\n", crc);
+
+ if (inbuf != NULL)
+ free(inbuf);
+
+ if (outbuf != NULL)
+ free(outbuf);
+
+ return 0;
+}
+
+int main(int argc, char **argv)
+{
+ int rc;
+ struct sigaction act;
+ void *handle;
+
+ nx_dbg = 0;
+ nx_gzip_log = NULL;
+ act.sa_handler = 0;
+ act.sa_sigaction = nxu_sigsegv_handler;
+ act.sa_flags = SA_SIGINFO;
+ act.sa_restorer = 0;
+ sigemptyset(&act.sa_mask);
+ sigaction(SIGSEGV, &act, NULL);
+
+ handle = nx_function_begin(NX_FUNC_COMP_GZIP, 0);
+ if (!handle) {
+ fprintf(stderr, "Unable to init NX, errno %d\n", errno);
+ exit(-1);
+ }
+
+ rc = compress_file(argc, argv, handle);
+
+ nx_function_end(handle);
+
+ return rc;
+}
diff --git a/tools/testing/selftests/powerpc/nx-gzip/gzip_vas.c b/tools/testing/selftests/powerpc/nx-gzip/gzip_vas.c
new file mode 100644
index 000000000000..e96b073ca2e5
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/gzip_vas.c
@@ -0,0 +1,316 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+
+/* Copyright 2020 IBM Corp.
+ *
+ * Author: Bulent Abali <abali@us.ibm.com>
+ *
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdint.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/time.h>
+#include <sys/fcntl.h>
+#include <sys/mman.h>
+#include <endian.h>
+#include <bits/endian.h>
+#include <sys/ioctl.h>
+#include <assert.h>
+#include <errno.h>
+#include <signal.h>
+#include "nx-gzip.h"
+#include "nx.h"
+#include "copy-paste.h"
+#include "nxu.h"
+#include "nx_dbg.h"
+#include <sys/platform/ppc.h>
+
+#define barrier()
+#define hwsync() ({ asm volatile("sync" ::: "memory"); })
+
+#ifndef NX_NO_CPU_PRI
+#define cpu_pri_default() ({ asm volatile ("or 2, 2, 2"); })
+#define cpu_pri_low() ({ asm volatile ("or 31, 31, 31"); })
+#else
+#define cpu_pri_default()
+#define cpu_pri_low()
+#endif
+
+void *nx_fault_storage_address;
+
+struct nx_handle {
+ int fd;
+ int function;
+ void *paste_addr;
+};
+
+static int open_device_nodes(char *devname, int pri, struct nx_handle *handle)
+{
+ int rc, fd;
+ void *addr;
+ struct vas_gzip_setup_attr txattr;
+
+ fd = open(devname, O_RDWR);
+ if (fd < 0) {
+ fprintf(stderr, " open device name %s\n", devname);
+ return -errno;
+ }
+
+ memset(&txattr, 0, sizeof(txattr));
+ txattr.version = 1;
+ txattr.vas_id = pri;
+ rc = ioctl(fd, VAS_GZIP_TX_WIN_OPEN, (unsigned long)&txattr);
+ if (rc < 0) {
+ fprintf(stderr, "ioctl() n %d, error %d\n", rc, errno);
+ rc = -errno;
+ goto out;
+ }
+
+ addr = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0ULL);
+ if (addr == MAP_FAILED) {
+ fprintf(stderr, "mmap() failed, errno %d\n", errno);
+ rc = -errno;
+ goto out;
+ }
+ handle->fd = fd;
+ handle->paste_addr = (void *)((char *)addr + 0x400);
+
+ rc = 0;
+out:
+ close(fd);
+ return rc;
+}
+
+void *nx_function_begin(int function, int pri)
+{
+ int rc;
+ char *devname = "/dev/crypto/nx-gzip";
+ struct nx_handle *nxhandle;
+
+ if (function != NX_FUNC_COMP_GZIP) {
+ errno = EINVAL;
+ fprintf(stderr, " NX_FUNC_COMP_GZIP not found\n");
+ return NULL;
+ }
+
+
+ nxhandle = malloc(sizeof(*nxhandle));
+ if (!nxhandle) {
+ errno = ENOMEM;
+ fprintf(stderr, " No memory\n");
+ return NULL;
+ }
+
+ nxhandle->function = function;
+ rc = open_device_nodes(devname, pri, nxhandle);
+ if (rc < 0) {
+ errno = -rc;
+ fprintf(stderr, " open_device_nodes failed\n");
+ return NULL;
+ }
+
+ return nxhandle;
+}
+
+int nx_function_end(void *handle)
+{
+ int rc = 0;
+ struct nx_handle *nxhandle = handle;
+
+ rc = munmap(nxhandle->paste_addr - 0x400, 4096);
+ if (rc < 0) {
+ fprintf(stderr, "munmap() failed, errno %d\n", errno);
+ return rc;
+ }
+ close(nxhandle->fd);
+ free(nxhandle);
+
+ return rc;
+}
+
+static int nx_wait_for_csb(struct nx_gzip_crb_cpb_t *cmdp)
+{
+ long poll = 0;
+ uint64_t t;
+
+ /* Save power and let other threads use the h/w. top may show
+ * 100% but only because OS doesn't know we slowed the this
+ * h/w thread while polling. We're letting other threads have
+ * higher throughput on the core.
+ */
+ cpu_pri_low();
+
+#define CSB_MAX_POLL 200000000UL
+#define USLEEP_TH 300000UL
+
+ t = __ppc_get_timebase();
+
+ while (getnn(cmdp->crb.csb, csb_v) == 0) {
+ ++poll;
+ hwsync();
+
+ cpu_pri_low();
+
+ /* usleep(0) takes around 29000 ticks ~60 us.
+ * 300000 is spinning for about 600 us then
+ * start sleeping.
+ */
+ if ((__ppc_get_timebase() - t) > USLEEP_TH) {
+ cpu_pri_default();
+ usleep(1);
+ }
+
+ if (poll > CSB_MAX_POLL)
+ break;
+
+ /* Fault address from signal handler */
+ if (nx_fault_storage_address) {
+ cpu_pri_default();
+ return -EAGAIN;
+ }
+
+ }
+
+ cpu_pri_default();
+
+ /* hw has updated csb and output buffer */
+ hwsync();
+
+ /* Check CSB flags. */
+ if (getnn(cmdp->crb.csb, csb_v) == 0) {
+ fprintf(stderr, "CSB still not valid after %d polls.\n",
+ (int) poll);
+ prt_err("CSB still not valid after %d polls, giving up.\n",
+ (int) poll);
+ return -ETIMEDOUT;
+ }
+
+ return 0;
+}
+
+static int nxu_run_job(struct nx_gzip_crb_cpb_t *cmdp, void *handle)
+{
+ int i, ret, retries;
+ struct nx_handle *nxhandle = handle;
+
+ assert(handle != NULL);
+ i = 0;
+ retries = 5000;
+ while (i++ < retries) {
+ hwsync();
+ vas_copy(&cmdp->crb, 0);
+ ret = vas_paste(nxhandle->paste_addr, 0);
+ hwsync();
+
+ NXPRT(fprintf(stderr, "Paste attempt %d/%d returns 0x%x\n",
+ i, retries, ret));
+
+ if ((ret == 2) || (ret == 3)) {
+
+ ret = nx_wait_for_csb(cmdp);
+ if (!ret) {
+ goto out;
+ } else if (ret == -EAGAIN) {
+ long x;
+
+ prt_err("Touching address %p, 0x%lx\n",
+ nx_fault_storage_address,
+ *(long *) nx_fault_storage_address);
+ x = *(long *) nx_fault_storage_address;
+ *(long *) nx_fault_storage_address = x;
+ nx_fault_storage_address = 0;
+ continue;
+ } else {
+ prt_err("wait_for_csb() returns %d\n", ret);
+ break;
+ }
+ } else {
+ if (i < 10) {
+ /* spin for few ticks */
+#define SPIN_TH 500UL
+ uint64_t fail_spin;
+
+ fail_spin = __ppc_get_timebase();
+ while ((__ppc_get_timebase() - fail_spin) <
+ SPIN_TH)
+ ;
+ } else {
+ /* sleep */
+ unsigned int pr = 0;
+
+ if (pr++ % 100 == 0) {
+ prt_err("Paste attempt %d/", i);
+ prt_err("%d, failed pid= %d\n", retries,
+ getpid());
+ }
+ usleep(1);
+ }
+ continue;
+ }
+ }
+
+out:
+ cpu_pri_default();
+
+ return ret;
+}
+
+int nxu_submit_job(struct nx_gzip_crb_cpb_t *cmdp, void *handle)
+{
+ int cc;
+
+ cc = nxu_run_job(cmdp, handle);
+
+ if (!cc)
+ cc = getnn(cmdp->crb.csb, csb_cc); /* CC Table 6-8 */
+
+ return cc;
+}
+
+
+void nxu_sigsegv_handler(int sig, siginfo_t *info, void *ctx)
+{
+ fprintf(stderr, "%d: Got signal %d si_code %d, si_addr %p\n", getpid(),
+ sig, info->si_code, info->si_addr);
+
+ nx_fault_storage_address = info->si_addr;
+}
+
+/*
+ * Fault in pages prior to NX job submission. wr=1 may be required to
+ * touch writeable pages. System zero pages do not fault-in the page as
+ * intended. Typically set wr=1 for NX target pages and set wr=0 for NX
+ * source pages.
+ */
+int nxu_touch_pages(void *buf, long buf_len, long page_len, int wr)
+{
+ char *begin = buf;
+ char *end = (char *) buf + buf_len - 1;
+ volatile char t;
+
+ assert(buf_len >= 0 && !!buf);
+
+ NXPRT(fprintf(stderr, "touch %p %p len 0x%lx wr=%d\n", buf,
+ (buf + buf_len), buf_len, wr));
+
+ if (buf_len <= 0 || buf == NULL)
+ return -1;
+
+ do {
+ t = *begin;
+ if (wr)
+ *begin = t;
+ begin = begin + page_len;
+ } while (begin < end);
+
+ /* When buf_sz is small or buf tail is in another page */
+ t = *end;
+ if (wr)
+ *end = t;
+
+ return 0;
+}
+
--
2.21.0
^ permalink raw reply related
* [PATCH V3 4/5] selftests/powerpc: Add NX-GZIP engine decompress testcase
From: Raphael Moreira Zinsly @ 2020-04-13 15:59 UTC (permalink / raw)
To: linuxppc-dev, linux-crypto, dja; +Cc: abali, haren, herbert, rzinsly
In-Reply-To: <20200413155916.16900-1-rzinsly@linux.ibm.com>
Include a decompression testcase for the powerpc NX-GZIP
engine.
Signed-off-by: Bulent Abali <abali@us.ibm.com>
Signed-off-by: Raphael Moreira Zinsly <rzinsly@linux.ibm.com>
---
.../selftests/powerpc/nx-gzip/Makefile | 7 +-
.../selftests/powerpc/nx-gzip/gunz_test.c | 1026 +++++++++++++++++
2 files changed, 1030 insertions(+), 3 deletions(-)
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/gunz_test.c
diff --git a/tools/testing/selftests/powerpc/nx-gzip/Makefile b/tools/testing/selftests/powerpc/nx-gzip/Makefile
index ab903f63bbbd..82abc19a49a0 100644
--- a/tools/testing/selftests/powerpc/nx-gzip/Makefile
+++ b/tools/testing/selftests/powerpc/nx-gzip/Makefile
@@ -1,9 +1,9 @@
CC = gcc
CFLAGS = -O3
INC = ./inc
-SRC = gzfht_test.c
+SRC = gzfht_test.c gunz_test.c
OBJ = $(SRC:.c=.o)
-TESTS = gzfht_test
+TESTS = gzfht_test gunz_test
EXTRA_SOURCES = gzip_vas.c
all: $(TESTS)
@@ -16,6 +16,7 @@ $(TESTS): $(OBJ)
run_tests: $(TESTS)
./gzfht_test gzip_vas.c
+ ./gunz_test gzip_vas.c.nx.gz
clean:
- rm -f $(TESTS) *.o *~ *.gz
+ rm -f $(TESTS) *.o *~ *.gz *.gunzip
diff --git a/tools/testing/selftests/powerpc/nx-gzip/gunz_test.c b/tools/testing/selftests/powerpc/nx-gzip/gunz_test.c
new file mode 100644
index 000000000000..94cb79616225
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/gunz_test.c
@@ -0,0 +1,1026 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+
+/* P9 gunzip sample code for demonstrating the P9 NX hardware
+ * interface. Not intended for productive uses or for performance or
+ * compression ratio measurements. Note also that /dev/crypto/gzip,
+ * VAS and skiboot support are required
+ *
+ * Copyright 2020 IBM Corp.
+ *
+ * Author: Bulent Abali <abali@us.ibm.com>
+ *
+ * https://github.com/libnxz/power-gzip for zlib api and other utils
+ * Definitions of acronyms used here. See
+ * P9 NX Gzip Accelerator User's Manual for details:
+ * https://github.com/libnxz/power-gzip/blob/develop/doc/power_nx_gzip_um.pdf
+ *
+ * adler/crc: 32 bit checksums appended to stream tail
+ * ce: completion extension
+ * cpb: coprocessor parameter block (metadata)
+ * crb: coprocessor request block (command)
+ * csb: coprocessor status block (status)
+ * dht: dynamic huffman table
+ * dde: data descriptor element (address, length)
+ * ddl: list of ddes
+ * dh/fh: dynamic and fixed huffman types
+ * fc: coprocessor function code
+ * histlen: history/dictionary length
+ * history: sliding window of up to 32KB of data
+ * lzcount: Deflate LZ symbol counts
+ * rembytecnt: remaining byte count
+ * sfbt: source final block type; last block's type during decomp
+ * spbc: source processed byte count
+ * subc: source unprocessed bit count
+ * tebc: target ending bit count; valid bits in the last byte
+ * tpbc: target processed byte count
+ * vas: virtual accelerator switch; the user mode interface
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdint.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/time.h>
+#include <sys/fcntl.h>
+#include <sys/mman.h>
+#include <endian.h>
+#include <bits/endian.h>
+#include <sys/ioctl.h>
+#include <assert.h>
+#include <errno.h>
+#include <signal.h>
+#include "nxu.h"
+#include "nx.h"
+#include "crb.h"
+
+int nx_dbg;
+FILE *nx_gzip_log;
+
+#define NX_MIN(X, Y) (((X) < (Y))?(X):(Y))
+#define NX_MAX(X, Y) (((X) > (Y))?(X):(Y))
+
+#define GETINPC(X) fgetc(X)
+#define FNAME_MAX 1024
+
+/* fifo queue management */
+#define fifo_used_bytes(used) (used)
+#define fifo_free_bytes(used, len) ((len)-(used))
+/* amount of free bytes in the first and last parts */
+#define fifo_free_first_bytes(cur, used, len) ((((cur)+(used)) <= (len)) \
+ ? (len)-((cur)+(used)) : 0)
+#define fifo_free_last_bytes(cur, used, len) ((((cur)+(used)) <= (len)) \
+ ? (cur) : (len)-(used))
+/* amount of used bytes in the first and last parts */
+#define fifo_used_first_bytes(cur, used, len) ((((cur)+(used)) <= (len)) \
+ ? (used) : (len)-(cur))
+#define fifo_used_last_bytes(cur, used, len) ((((cur)+(used)) <= (len)) \
+ ? 0 : ((used)+(cur))-(len))
+/* first and last free parts start here */
+#define fifo_free_first_offset(cur, used) ((cur)+(used))
+#define fifo_free_last_offset(cur, used, len) \
+ fifo_used_last_bytes(cur, used, len)
+/* first and last used parts start here */
+#define fifo_used_first_offset(cur) (cur)
+#define fifo_used_last_offset(cur) (0)
+
+const int fifo_in_len = 1<<24;
+const int fifo_out_len = 1<<24;
+const int page_sz = 1<<16;
+const int line_sz = 1<<7;
+const int window_max = 1<<15;
+
+/*
+ * Adds an (address, len) pair to the list of ddes (ddl) and updates
+ * the base dde. ddl[0] is the only dde in a direct dde which
+ * contains a single (addr,len) pair. For more pairs, ddl[0] becomes
+ * the indirect (base) dde that points to a list of direct ddes.
+ * See Section 6.4 of the NX-gzip user manual for DDE description.
+ * Addr=NULL, len=0 clears the ddl[0]. Returns the total number of
+ * bytes in ddl. Caller is responsible for allocting the array of
+ * nx_dde_t *ddl. If N addresses are required in the scatter-gather
+ * list, the ddl array must have N+1 entries minimum.
+ */
+static inline uint32_t nx_append_dde(struct nx_dde_t *ddl, void *addr,
+ uint32_t len)
+{
+ uint32_t ddecnt;
+ uint32_t bytes;
+
+ if (addr == NULL && len == 0) {
+ clearp_dde(ddl);
+ return 0;
+ }
+
+ NXPRT(fprintf(stderr, "%d: %s addr %p len %x\n", __LINE__, addr,
+ __func__, len));
+
+ /* Number of ddes in the dde list ; == 0 when it is a direct dde */
+ ddecnt = getpnn(ddl, dde_count);
+ bytes = getp32(ddl, ddebc);
+
+ if (ddecnt == 0 && bytes == 0) {
+ /* First dde is unused; make it a direct dde */
+ bytes = len;
+ putp32(ddl, ddebc, bytes);
+ putp64(ddl, ddead, (uint64_t) addr);
+ } else if (ddecnt == 0) {
+ /* Converting direct to indirect dde
+ * ddl[0] becomes head dde of ddl
+ * copy direct to indirect first.
+ */
+ ddl[1] = ddl[0];
+
+ /* Add the new dde next */
+ clear_dde(ddl[2]);
+ put32(ddl[2], ddebc, len);
+ put64(ddl[2], ddead, (uint64_t) addr);
+
+ /* Ddl head points to 2 direct ddes */
+ ddecnt = 2;
+ putpnn(ddl, dde_count, ddecnt);
+ bytes = bytes + len;
+ putp32(ddl, ddebc, bytes);
+ /* Pointer to the first direct dde */
+ putp64(ddl, ddead, (uint64_t) &ddl[1]);
+ } else {
+ /* Append a dde to an existing indirect ddl */
+ ++ddecnt;
+ clear_dde(ddl[ddecnt]);
+ put64(ddl[ddecnt], ddead, (uint64_t) addr);
+ put32(ddl[ddecnt], ddebc, len);
+
+ putpnn(ddl, dde_count, ddecnt);
+ bytes = bytes + len;
+ putp32(ddl, ddebc, bytes); /* byte sum of all dde */
+ }
+ return bytes;
+}
+
+/*
+ * Touch specified number of pages represented in number bytes
+ * beginning from the first buffer in a dde list.
+ * Do not touch the pages past buf_sz-th byte's page.
+ *
+ * Set buf_sz = 0 to touch all pages described by the ddep.
+ */
+static int nx_touch_pages_dde(struct nx_dde_t *ddep, long buf_sz, long page_sz,
+ int wr)
+{
+ uint32_t indirect_count;
+ uint32_t buf_len;
+ long total;
+ uint64_t buf_addr;
+ struct nx_dde_t *dde_list;
+ int i;
+
+ assert(!!ddep);
+
+ indirect_count = getpnn(ddep, dde_count);
+
+ NXPRT(fprintf(stderr, "%s dde_count %d request len ", __func__,
+ indirect_count));
+ NXPRT(fprintf(stderr, "0x%lx\n", buf_sz));
+
+ if (indirect_count == 0) {
+ /* Direct dde */
+ buf_len = getp32(ddep, ddebc);
+ buf_addr = getp64(ddep, ddead);
+
+ NXPRT(fprintf(stderr, "touch direct ddebc 0x%x ddead %p\n",
+ buf_len, (void *)buf_addr));
+
+ if (buf_sz == 0)
+ nxu_touch_pages((void *)buf_addr, buf_len, page_sz, wr);
+ else
+ nxu_touch_pages((void *)buf_addr, NX_MIN(buf_len,
+ buf_sz), page_sz, wr);
+
+ return ERR_NX_OK;
+ }
+
+ /* Indirect dde */
+ if (indirect_count > MAX_DDE_COUNT)
+ return ERR_NX_EXCESSIVE_DDE;
+
+ /* First address of the list */
+ dde_list = (struct nx_dde_t *) getp64(ddep, ddead);
+
+ if (buf_sz == 0)
+ buf_sz = getp32(ddep, ddebc);
+
+ total = 0;
+ for (i = 0; i < indirect_count; i++) {
+ buf_len = get32(dde_list[i], ddebc);
+ buf_addr = get64(dde_list[i], ddead);
+ total += buf_len;
+
+ NXPRT(fprintf(stderr, "touch loop len 0x%x ddead %p total ",
+ buf_len, (void *)buf_addr));
+ NXPRT(fprintf(stderr, "0x%lx\n", total));
+
+ /* Touching fewer pages than encoded in the ddebc */
+ if (total > buf_sz) {
+ buf_len = NX_MIN(buf_len, total - buf_sz);
+ nxu_touch_pages((void *)buf_addr, buf_len, page_sz, wr);
+ NXPRT(fprintf(stderr, "touch loop break len 0x%x ",
+ buf_len));
+ NXPRT(fprintf(stderr, "ddead %p\n", (void *)buf_addr));
+ break;
+ }
+ nxu_touch_pages((void *)buf_addr, buf_len, page_sz, wr);
+ }
+ return ERR_NX_OK;
+}
+
+/*
+ * Src and dst buffers are supplied in scatter gather lists.
+ * NX function code and other parameters supplied in cmdp.
+ */
+static int nx_submit_job(struct nx_dde_t *src, struct nx_dde_t *dst,
+ struct nx_gzip_crb_cpb_t *cmdp, void *handle)
+{
+ int cc;
+ uint64_t csbaddr;
+
+ memset((void *)&cmdp->crb.csb, 0, sizeof(cmdp->crb.csb));
+
+ cmdp->crb.source_dde = *src;
+ cmdp->crb.target_dde = *dst;
+
+ /* Status, output byte count in tpbc */
+ csbaddr = ((uint64_t) &cmdp->crb.csb) & csb_address_mask;
+ put64(cmdp->crb, csb_address, csbaddr);
+
+ /* NX reports input bytes in spbc; cleared */
+ cmdp->cpb.out_spbc_comp_wrap = 0;
+ cmdp->cpb.out_spbc_comp_with_count = 0;
+ cmdp->cpb.out_spbc_decomp = 0;
+
+ /* Clear output */
+ put32(cmdp->cpb, out_crc, INIT_CRC);
+ put32(cmdp->cpb, out_adler, INIT_ADLER);
+
+ /* Submit the crb, the job descriptor, to the accelerator. */
+ return nxu_submit_job(cmdp, handle);
+}
+
+int decompress_file(int argc, char **argv, void *devhandle)
+{
+ FILE *inpf;
+ FILE *outf;
+
+ int c, expect, i, cc, rc = 0;
+ char gzfname[FNAME_MAX];
+
+ /* Queuing, file ops, byte counting */
+ char *fifo_in, *fifo_out;
+ int used_in, cur_in, used_out, cur_out, read_sz, n;
+ int first_free, last_free, first_used, last_used;
+ int first_offset, last_offset;
+ int write_sz, free_space, source_sz;
+ int source_sz_estimate, target_sz_estimate;
+ uint64_t last_comp_ratio; /* 1000 max */
+ uint64_t total_out;
+ int is_final, is_eof;
+
+ /* nx hardware */
+ int sfbt, subc, spbc, tpbc, nx_ce, fc, resuming = 0;
+ int history_len = 0;
+ struct nx_gzip_crb_cpb_t cmd, *cmdp;
+ struct nx_dde_t *ddl_in;
+ struct nx_dde_t dde_in[6] __aligned(128);
+ struct nx_dde_t *ddl_out;
+ struct nx_dde_t dde_out[6] __aligned(128);
+ int pgfault_retries;
+
+ /* when using mmap'ed files */
+ off_t input_file_offset;
+
+ if (argc > 2) {
+ fprintf(stderr, "usage: %s <fname> or stdin\n", argv[0]);
+ fprintf(stderr, " writes to stdout or <fname>.nx.gunzip\n");
+ return -1;
+ }
+
+ if (argc == 1) {
+ inpf = stdin;
+ outf = stdout;
+ } else if (argc == 2) {
+ char w[1024];
+ char *wp;
+
+ inpf = fopen(argv[1], "r");
+ if (inpf == NULL) {
+ perror(argv[1]);
+ return -1;
+ }
+
+ /* Make a new file name to write to. Ignoring '.gz' */
+ wp = (NULL != (wp = strrchr(argv[1], '/'))) ? (wp+1) : argv[1];
+ strcpy(w, wp);
+ strcat(w, ".nx.gunzip");
+
+ outf = fopen(w, "w");
+ if (outf == NULL) {
+ perror(w);
+ return -1;
+ }
+ }
+
+ /* Decode the gzip header */
+ c = GETINPC(inpf); expect = 0x1f; /* ID1 */
+ if (c != expect)
+ goto err1;
+
+ c = GETINPC(inpf); expect = 0x8b; /* ID2 */
+ if (c != expect)
+ goto err1;
+
+ c = GETINPC(inpf); expect = 0x08; /* CM */
+ if (c != expect)
+ goto err1;
+
+ int flg = GETINPC(inpf); /* FLG */
+
+ if (flg & 0xE0 || flg & 0x4 || flg == EOF)
+ goto err2;
+
+ fprintf(stderr, "gzHeader FLG %x\n", flg);
+
+ /* Read 6 bytes; ignoring the MTIME, XFL, OS fields in this
+ * sample code.
+ */
+ for (i = 0; i < 6; i++) {
+ char tmp[10];
+
+ tmp[i] = GETINPC(inpf);
+ if (tmp[i] == EOF)
+ goto err3;
+ fprintf(stderr, "%02x ", tmp[i]);
+ if (i == 5)
+ fprintf(stderr, "\n");
+ }
+ fprintf(stderr, "gzHeader MTIME, XFL, OS ignored\n");
+
+ /* FNAME */
+ if (flg & 0x8) {
+ int k = 0;
+
+ do {
+ c = GETINPC(inpf);
+ if (c == EOF || k >= FNAME_MAX)
+ goto err3;
+ gzfname[k++] = c;
+ } while (c);
+ fprintf(stderr, "gzHeader FNAME: %s\n", gzfname);
+ }
+
+ /* FHCRC */
+ if (flg & 0x2) {
+ c = GETINPC(inpf);
+ if (c == EOF)
+ goto err3;
+ c = GETINPC(inpf);
+ if (c == EOF)
+ goto err3;
+ fprintf(stderr, "gzHeader FHCRC: ignored\n");
+ }
+
+ used_in = cur_in = used_out = cur_out = 0;
+ is_final = is_eof = 0;
+
+ /* Allocate one page larger to prevent page faults due to NX
+ * overfetching.
+ * Either do this (char*)(uintptr_t)aligned_alloc or use
+ * -std=c11 flag to make the int-to-pointer warning go away.
+ */
+ assert((fifo_in = (char *)(uintptr_t)aligned_alloc(line_sz,
+ fifo_in_len + page_sz)) != NULL);
+ assert((fifo_out = (char *)(uintptr_t)aligned_alloc(line_sz,
+ fifo_out_len + page_sz + line_sz)) != NULL);
+ /* Leave unused space due to history rounding rules */
+ fifo_out = fifo_out + line_sz;
+ nxu_touch_pages(fifo_out, fifo_out_len, page_sz, 1);
+
+ ddl_in = &dde_in[0];
+ ddl_out = &dde_out[0];
+ cmdp = &cmd;
+ memset(&cmdp->crb, 0, sizeof(cmdp->crb));
+
+read_state:
+
+ /* Read from .gz file */
+
+ NXPRT(fprintf(stderr, "read_state:\n"));
+
+ if (is_eof != 0)
+ goto write_state;
+
+ /* We read in to fifo_in in two steps: first: read in to from
+ * cur_in to the end of the buffer. last: if free space wrapped
+ * around, read from fifo_in offset 0 to offset cur_in.
+ */
+
+ /* Reset fifo head to reduce unnecessary wrap arounds */
+ cur_in = (used_in == 0) ? 0 : cur_in;
+
+ /* Free space total is reduced by a gap */
+ free_space = NX_MAX(0, fifo_free_bytes(used_in, fifo_in_len)
+ - line_sz);
+
+ /* Free space may wrap around as first and last */
+ first_free = fifo_free_first_bytes(cur_in, used_in, fifo_in_len);
+ last_free = fifo_free_last_bytes(cur_in, used_in, fifo_in_len);
+
+ /* Start offsets of the free memory */
+ first_offset = fifo_free_first_offset(cur_in, used_in);
+ last_offset = fifo_free_last_offset(cur_in, used_in, fifo_in_len);
+
+ /* Reduce read_sz because of the line_sz gap */
+ read_sz = NX_MIN(free_space, first_free);
+ n = 0;
+ if (read_sz > 0) {
+ /* Read in to offset cur_in + used_in */
+ n = fread(fifo_in + first_offset, 1, read_sz, inpf);
+ used_in = used_in + n;
+ free_space = free_space - n;
+ assert(n <= read_sz);
+ if (n != read_sz) {
+ /* Either EOF or error; exit the read loop */
+ is_eof = 1;
+ goto write_state;
+ }
+ }
+
+ /* If free space wrapped around */
+ if (last_free > 0) {
+ /* Reduce read_sz because of the line_sz gap */
+ read_sz = NX_MIN(free_space, last_free);
+ n = 0;
+ if (read_sz > 0) {
+ n = fread(fifo_in + last_offset, 1, read_sz, inpf);
+ used_in = used_in + n; /* Increase used space */
+ free_space = free_space - n; /* Decrease free space */
+ assert(n <= read_sz);
+ if (n != read_sz) {
+ /* Either EOF or error; exit the read loop */
+ is_eof = 1;
+ goto write_state;
+ }
+ }
+ }
+
+ /* At this point we have used_in bytes in fifo_in with the
+ * data head starting at cur_in and possibly wrapping around.
+ */
+
+write_state:
+
+ /* Write decompressed data to output file */
+
+ NXPRT(fprintf(stderr, "write_state:\n"));
+
+ if (used_out == 0)
+ goto decomp_state;
+
+ /* If fifo_out has data waiting, write it out to the file to
+ * make free target space for the accelerator used bytes in
+ * the first and last parts of fifo_out.
+ */
+
+ first_used = fifo_used_first_bytes(cur_out, used_out, fifo_out_len);
+ last_used = fifo_used_last_bytes(cur_out, used_out, fifo_out_len);
+
+ write_sz = first_used;
+
+ n = 0;
+ if (write_sz > 0) {
+ n = fwrite(fifo_out + cur_out, 1, write_sz, outf);
+ used_out = used_out - n;
+ /* Move head of the fifo */
+ cur_out = (cur_out + n) % fifo_out_len;
+ assert(n <= write_sz);
+ if (n != write_sz) {
+ fprintf(stderr, "error: write\n");
+ rc = -1;
+ goto err5;
+ }
+ }
+
+ if (last_used > 0) { /* If more data available in the last part */
+ write_sz = last_used; /* Keep it here for later */
+ n = 0;
+ if (write_sz > 0) {
+ n = fwrite(fifo_out, 1, write_sz, outf);
+ used_out = used_out - n;
+ cur_out = (cur_out + n) % fifo_out_len;
+ assert(n <= write_sz);
+ if (n != write_sz) {
+ fprintf(stderr, "error: write\n");
+ rc = -1;
+ goto err5;
+ }
+ }
+ }
+
+decomp_state:
+
+ /* NX decompresses input data */
+
+ NXPRT(fprintf(stderr, "decomp_state:\n"));
+
+ if (is_final)
+ goto finish_state;
+
+ /* Address/len lists */
+ clearp_dde(ddl_in);
+ clearp_dde(ddl_out);
+
+ /* FC, CRC, HistLen, Table 6-6 */
+ if (resuming) {
+ /* Resuming a partially decompressed input.
+ * The key to resume is supplying the 32KB
+ * dictionary (history) to NX, which is basically
+ * the last 32KB of output produced.
+ */
+ fc = GZIP_FC_DECOMPRESS_RESUME;
+
+ cmdp->cpb.in_crc = cmdp->cpb.out_crc;
+ cmdp->cpb.in_adler = cmdp->cpb.out_adler;
+
+ /* Round up the history size to quadword. Section 2.10 */
+ history_len = (history_len + 15) / 16;
+ putnn(cmdp->cpb, in_histlen, history_len);
+ history_len = history_len * 16; /* bytes */
+
+ if (history_len > 0) {
+ /* Chain in the history buffer to the DDE list */
+ if (cur_out >= history_len) {
+ nx_append_dde(ddl_in, fifo_out
+ + (cur_out - history_len),
+ history_len);
+ } else {
+ nx_append_dde(ddl_in, fifo_out
+ + ((fifo_out_len + cur_out)
+ - history_len),
+ history_len - cur_out);
+ /* Up to 32KB history wraps around fifo_out */
+ nx_append_dde(ddl_in, fifo_out, cur_out);
+ }
+
+ }
+ } else {
+ /* First decompress job */
+ fc = GZIP_FC_DECOMPRESS;
+
+ history_len = 0;
+ /* Writing 0 clears out subc as well */
+ cmdp->cpb.in_histlen = 0;
+ total_out = 0;
+
+ put32(cmdp->cpb, in_crc, INIT_CRC);
+ put32(cmdp->cpb, in_adler, INIT_ADLER);
+ put32(cmdp->cpb, out_crc, INIT_CRC);
+ put32(cmdp->cpb, out_adler, INIT_ADLER);
+
+ /* Assuming 10% compression ratio initially; use the
+ * most recently measured compression ratio as a
+ * heuristic to estimate the input and output
+ * sizes. If we give too much input, the target buffer
+ * overflows and NX cycles are wasted, and then we
+ * must retry with smaller input size. 1000 is 100%.
+ */
+ last_comp_ratio = 100UL;
+ }
+ cmdp->crb.gzip_fc = 0;
+ putnn(cmdp->crb, gzip_fc, fc);
+
+ /*
+ * NX source buffers
+ */
+ first_used = fifo_used_first_bytes(cur_in, used_in, fifo_in_len);
+ last_used = fifo_used_last_bytes(cur_in, used_in, fifo_in_len);
+
+ if (first_used > 0)
+ nx_append_dde(ddl_in, fifo_in + cur_in, first_used);
+
+ if (last_used > 0)
+ nx_append_dde(ddl_in, fifo_in, last_used);
+
+ /*
+ * NX target buffers
+ */
+ first_free = fifo_free_first_bytes(cur_out, used_out, fifo_out_len);
+ last_free = fifo_free_last_bytes(cur_out, used_out, fifo_out_len);
+
+ /* Reduce output free space amount not to overwrite the history */
+ int target_max = NX_MAX(0, fifo_free_bytes(used_out, fifo_out_len)
+ - (1<<16));
+
+ NXPRT(fprintf(stderr, "target_max %d (0x%x)\n", target_max,
+ target_max));
+
+ first_free = NX_MIN(target_max, first_free);
+ if (first_free > 0) {
+ first_offset = fifo_free_first_offset(cur_out, used_out);
+ nx_append_dde(ddl_out, fifo_out + first_offset, first_free);
+ }
+
+ if (last_free > 0) {
+ last_free = NX_MIN(target_max - first_free, last_free);
+ if (last_free > 0) {
+ last_offset = fifo_free_last_offset(cur_out, used_out,
+ fifo_out_len);
+ nx_append_dde(ddl_out, fifo_out + last_offset,
+ last_free);
+ }
+ }
+
+ /* Target buffer size is used to limit the source data size
+ * based on previous measurements of compression ratio.
+ */
+
+ /* source_sz includes history */
+ source_sz = getp32(ddl_in, ddebc);
+ assert(source_sz > history_len);
+ source_sz = source_sz - history_len;
+
+ /* Estimating how much source is needed to 3/4 fill a
+ * target_max size target buffer. If we overshoot, then NX
+ * must repeat the job with smaller input and we waste
+ * bandwidth. If we undershoot then we use more NX calls than
+ * necessary.
+ */
+
+ source_sz_estimate = ((uint64_t)target_max * last_comp_ratio * 3UL)
+ / 4000;
+
+ if (source_sz_estimate < source_sz) {
+ /* Target might be small, therefore limiting the
+ * source data.
+ */
+ source_sz = source_sz_estimate;
+ target_sz_estimate = target_max;
+ } else {
+ /* Source file might be small, therefore limiting target
+ * touch pages to a smaller value to save processor cycles.
+ */
+ target_sz_estimate = ((uint64_t)source_sz * 1000UL)
+ / (last_comp_ratio + 1);
+ target_sz_estimate = NX_MIN(2 * target_sz_estimate,
+ target_max);
+ }
+
+ source_sz = source_sz + history_len;
+
+ /* Some NX condition codes require submitting the NX job again.
+ * Kernel doesn't handle NX page faults. Expects user code to
+ * touch pages.
+ */
+ pgfault_retries = NX_MAX_FAULTS;
+
+restart_nx:
+
+ putp32(ddl_in, ddebc, source_sz);
+
+ /* Fault in pages */
+ nxu_touch_pages(cmdp, sizeof(struct nx_gzip_crb_cpb_t), page_sz, 1);
+ nx_touch_pages_dde(ddl_in, 0, page_sz, 0);
+ nx_touch_pages_dde(ddl_out, target_sz_estimate, page_sz, 1);
+
+ /* Send job to NX */
+ cc = nx_submit_job(ddl_in, ddl_out, cmdp, devhandle);
+
+ switch (cc) {
+
+ case ERR_NX_TRANSLATION:
+
+ /* We touched the pages ahead of time. In the most common case
+ * we shouldn't be here. But may be some pages were paged out.
+ * Kernel should have placed the faulting address to fsaddr.
+ */
+ NXPRT(fprintf(stderr, "ERR_NX_TRANSLATION %p\n",
+ (void *)cmdp->crb.csb.fsaddr));
+
+ if (pgfault_retries == NX_MAX_FAULTS) {
+ /* Try once with exact number of pages */
+ --pgfault_retries;
+ goto restart_nx;
+ } else if (pgfault_retries > 0) {
+ /* If still faulting try fewer input pages
+ * assuming memory outage
+ */
+ if (source_sz > page_sz)
+ source_sz = NX_MAX(source_sz / 2, page_sz);
+ --pgfault_retries;
+ goto restart_nx;
+ } else {
+ fprintf(stderr, "cannot make progress; too many ");
+ fprintf(stderr, "page fault retries cc= %d\n", cc);
+ rc = -1;
+ goto err5;
+ }
+
+ case ERR_NX_DATA_LENGTH:
+
+ NXPRT(fprintf(stderr, "ERR_NX_DATA_LENGTH; "));
+ NXPRT(fprintf(stderr, "stream may have trailing data\n"));
+
+ /* Not an error in the most common case; it just says
+ * there is trailing data that we must examine.
+ *
+ * CC=3 CE(1)=0 CE(0)=1 indicates partial completion
+ * Fig.6-7 and Table 6-8.
+ */
+ nx_ce = get_csb_ce_ms3b(cmdp->crb.csb);
+
+ if (!csb_ce_termination(nx_ce) &&
+ csb_ce_partial_completion(nx_ce)) {
+ /* Check CPB for more information
+ * spbc and tpbc are valid
+ */
+ sfbt = getnn(cmdp->cpb, out_sfbt); /* Table 6-4 */
+ subc = getnn(cmdp->cpb, out_subc); /* Table 6-4 */
+ spbc = get32(cmdp->cpb, out_spbc_decomp);
+ tpbc = get32(cmdp->crb.csb, tpbc);
+ assert(target_max >= tpbc);
+
+ goto ok_cc3; /* not an error */
+ } else {
+ /* History length error when CE(1)=1 CE(0)=0. */
+ rc = -1;
+ fprintf(stderr, "history length error cc= %d\n", cc);
+ goto err5;
+ }
+
+ case ERR_NX_TARGET_SPACE:
+
+ /* Target buffer not large enough; retry smaller input
+ * data; give at least 1 byte. SPBC/TPBC are not valid.
+ */
+ assert(source_sz > history_len);
+ source_sz = ((source_sz - history_len + 2) / 2) + history_len;
+ NXPRT(fprintf(stderr, "ERR_NX_TARGET_SPACE; retry with "));
+ NXPRT(fprintf(stderr, "smaller input data src %d hist %d\n",
+ source_sz, history_len));
+ goto restart_nx;
+
+ case ERR_NX_OK:
+
+ /* This should not happen for gzip formatted data;
+ * we need trailing crc and isize
+ */
+ fprintf(stderr, "ERR_NX_OK\n");
+ spbc = get32(cmdp->cpb, out_spbc_decomp);
+ tpbc = get32(cmdp->crb.csb, tpbc);
+ assert(target_max >= tpbc);
+ assert(spbc >= history_len);
+ source_sz = spbc - history_len;
+ goto offsets_state;
+
+ default:
+ fprintf(stderr, "error: cc= %d\n", cc);
+ rc = -1;
+ goto err5;
+ }
+
+ok_cc3:
+
+ NXPRT(fprintf(stderr, "cc3: sfbt: %x\n", sfbt));
+
+ assert(spbc > history_len);
+ source_sz = spbc - history_len;
+
+ /* Table 6-4: Source Final Block Type (SFBT) describes the
+ * last processed deflate block and clues the software how to
+ * resume the next job. SUBC indicates how many input bits NX
+ * consumed but did not process. SPBC indicates how many
+ * bytes of source were given to the accelerator including
+ * history bytes.
+ */
+
+ switch (sfbt) {
+ int dhtlen;
+
+ case 0x0: /* Deflate final EOB received */
+
+ /* Calculating the checksum start position. */
+
+ source_sz = source_sz - subc / 8;
+ is_final = 1;
+ break;
+
+ /* Resume decompression cases are below. Basically
+ * indicates where NX has suspended and how to resume
+ * the input stream.
+ */
+
+ case 0x8: /* Within a literal block; use rembytecount */
+ case 0x9: /* Within a literal block; use rembytecount; bfinal=1 */
+
+ /* Supply the partially processed source byte again */
+ source_sz = source_sz - ((subc + 7) / 8);
+
+ /* SUBC LS 3bits: number of bits in the first source byte need
+ * to be processed.
+ * 000 means all 8 bits; Table 6-3
+ * Clear subc, histlen, sfbt, rembytecnt, dhtlen
+ */
+ cmdp->cpb.in_subc = 0;
+ cmdp->cpb.in_sfbt = 0;
+ putnn(cmdp->cpb, in_subc, subc % 8);
+ putnn(cmdp->cpb, in_sfbt, sfbt);
+ putnn(cmdp->cpb, in_rembytecnt, getnn(cmdp->cpb,
+ out_rembytecnt));
+ break;
+
+ case 0xA: /* Within a FH block; */
+ case 0xB: /* Within a FH block; bfinal=1 */
+
+ source_sz = source_sz - ((subc + 7) / 8);
+
+ /* Clear subc, histlen, sfbt, rembytecnt, dhtlen */
+ cmdp->cpb.in_subc = 0;
+ cmdp->cpb.in_sfbt = 0;
+ putnn(cmdp->cpb, in_subc, subc % 8);
+ putnn(cmdp->cpb, in_sfbt, sfbt);
+ break;
+
+ case 0xC: /* Within a DH block; */
+ case 0xD: /* Within a DH block; bfinal=1 */
+
+ source_sz = source_sz - ((subc + 7) / 8);
+
+ /* Clear subc, histlen, sfbt, rembytecnt, dhtlen */
+ cmdp->cpb.in_subc = 0;
+ cmdp->cpb.in_sfbt = 0;
+ putnn(cmdp->cpb, in_subc, subc % 8);
+ putnn(cmdp->cpb, in_sfbt, sfbt);
+
+ dhtlen = getnn(cmdp->cpb, out_dhtlen);
+ putnn(cmdp->cpb, in_dhtlen, dhtlen);
+ assert(dhtlen >= 42);
+
+ /* Round up to a qword */
+ dhtlen = (dhtlen + 127) / 128;
+
+ while (dhtlen > 0) { /* Copy dht from cpb.out to cpb.in */
+ --dhtlen;
+ cmdp->cpb.in_dht[dhtlen] = cmdp->cpb.out_dht[dhtlen];
+ }
+ break;
+
+ case 0xE: /* Within a block header; bfinal=0; */
+ /* Also given if source data exactly ends (SUBC=0) with
+ * EOB code with BFINAL=0. Means the next byte will
+ * contain a block header.
+ */
+ case 0xF: /* within a block header with BFINAL=1. */
+
+ source_sz = source_sz - ((subc + 7) / 8);
+
+ /* Clear subc, histlen, sfbt, rembytecnt, dhtlen */
+ cmdp->cpb.in_subc = 0;
+ cmdp->cpb.in_sfbt = 0;
+ putnn(cmdp->cpb, in_subc, subc % 8);
+ putnn(cmdp->cpb, in_sfbt, sfbt);
+
+ /* Engine did not process any data */
+ if (is_eof && (source_sz == 0))
+ is_final = 1;
+ }
+
+offsets_state:
+
+ /* Adjust the source and target buffer offsets and lengths */
+
+ NXPRT(fprintf(stderr, "offsets_state:\n"));
+
+ /* Delete input data from fifo_in */
+ used_in = used_in - source_sz;
+ cur_in = (cur_in + source_sz) % fifo_in_len;
+ input_file_offset = input_file_offset + source_sz;
+
+ /* Add output data to fifo_out */
+ used_out = used_out + tpbc;
+
+ assert(used_out <= fifo_out_len);
+
+ total_out = total_out + tpbc;
+
+ /* Deflate history is 32KB max. No need to supply more
+ * than 32KB on a resume.
+ */
+ history_len = (total_out > window_max) ? window_max : total_out;
+
+ /* To estimate expected expansion in the next NX job; 500 means 50%.
+ * Deflate best case is around 1 to 1000.
+ */
+ last_comp_ratio = (1000UL * ((uint64_t)source_sz + 1))
+ / ((uint64_t)tpbc + 1);
+ last_comp_ratio = NX_MAX(NX_MIN(1000UL, last_comp_ratio), 1);
+ NXPRT(fprintf(stderr, "comp_ratio %ld source_sz %d spbc %d tpbc %d\n",
+ last_comp_ratio, source_sz, spbc, tpbc));
+
+ resuming = 1;
+
+finish_state:
+
+ NXPRT(fprintf(stderr, "finish_state:\n"));
+
+ if (is_final) {
+ if (used_out)
+ goto write_state; /* More data to write out */
+ else if (used_in < 8) {
+ /* Need at least 8 more bytes containing gzip crc
+ * and isize.
+ */
+ rc = -1;
+ goto err4;
+ } else {
+ /* Compare checksums and exit */
+ int i;
+ unsigned char tail[8];
+ uint32_t cksum, isize;
+
+ for (i = 0; i < 8; i++)
+ tail[i] = fifo_in[(cur_in + i) % fifo_in_len];
+ fprintf(stderr, "computed checksum %08x isize %08x\n",
+ cmdp->cpb.out_crc, (uint32_t) (total_out
+ % (1ULL<<32)));
+ cksum = ((uint32_t) tail[0] | (uint32_t) tail[1]<<8
+ | (uint32_t) tail[2]<<16
+ | (uint32_t) tail[3]<<24);
+ isize = ((uint32_t) tail[4] | (uint32_t) tail[5]<<8
+ | (uint32_t) tail[6]<<16
+ | (uint32_t) tail[7]<<24);
+ fprintf(stderr, "stored checksum %08x isize %08x\n",
+ cksum, isize);
+
+ if (cksum == cmdp->cpb.out_crc && isize == (uint32_t)
+ (total_out % (1ULL<<32))) {
+ rc = 0; goto ok1;
+ } else {
+ rc = -1; goto err4;
+ }
+ }
+ } else
+ goto read_state;
+
+ return -1;
+
+err1:
+ fprintf(stderr, "error: not a gzip file, expect %x, read %x\n",
+ expect, c);
+ return -1;
+
+err2:
+ fprintf(stderr, "error: the FLG byte is wrong or not being handled\n");
+ return -1;
+
+err3:
+ fprintf(stderr, "error: gzip header\n");
+ return -1;
+
+err4:
+ fprintf(stderr, "error: checksum missing or mismatch\n");
+
+err5:
+ok1:
+ fprintf(stderr, "decomp is complete: fclose\n");
+ fclose(outf);
+
+ return rc;
+}
+
+
+int main(int argc, char **argv)
+{
+ int rc;
+ struct sigaction act;
+ void *handle;
+
+ nx_dbg = 0;
+ nx_gzip_log = NULL;
+ act.sa_handler = 0;
+ act.sa_sigaction = nxu_sigsegv_handler;
+ act.sa_flags = SA_SIGINFO;
+ act.sa_restorer = 0;
+ sigemptyset(&act.sa_mask);
+ sigaction(SIGSEGV, &act, NULL);
+
+ handle = nx_function_begin(NX_FUNC_COMP_GZIP, 0);
+ if (!handle) {
+ fprintf(stderr, "Unable to init NX, errno %d\n", errno);
+ exit(-1);
+ }
+
+ rc = decompress_file(argc, argv, handle);
+
+ nx_function_end(handle);
+
+ return rc;
+}
--
2.21.0
^ permalink raw reply related
* [PATCH V3 5/5] selftests/powerpc: Add README for GZIP engine tests
From: Raphael Moreira Zinsly @ 2020-04-13 15:59 UTC (permalink / raw)
To: linuxppc-dev, linux-crypto, dja; +Cc: abali, haren, herbert, rzinsly
In-Reply-To: <20200413155916.16900-1-rzinsly@linux.ibm.com>
Include a README file with the instructions to use the
testcases at selftests/powerpc/nx-gzip.
Signed-off-by: Bulent Abali <abali@us.ibm.com>
Signed-off-by: Raphael Moreira Zinsly <rzinsly@linux.ibm.com>
---
.../powerpc/nx-gzip/99-nx-gzip.rules | 1 +
.../testing/selftests/powerpc/nx-gzip/README | 46 +++++++++++++++++++
2 files changed, 47 insertions(+)
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/99-nx-gzip.rules
create mode 100644 tools/testing/selftests/powerpc/nx-gzip/README
diff --git a/tools/testing/selftests/powerpc/nx-gzip/99-nx-gzip.rules b/tools/testing/selftests/powerpc/nx-gzip/99-nx-gzip.rules
new file mode 100644
index 000000000000..5a7118495cb3
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/99-nx-gzip.rules
@@ -0,0 +1 @@
+SUBSYSTEM=="nxgzip", KERNEL=="nx-gzip", MODE="0666"
diff --git a/tools/testing/selftests/powerpc/nx-gzip/README b/tools/testing/selftests/powerpc/nx-gzip/README
new file mode 100644
index 000000000000..9a491daaef4d
--- /dev/null
+++ b/tools/testing/selftests/powerpc/nx-gzip/README
@@ -0,0 +1,46 @@
+Test the nx-gzip function:
+=========================
+
+Verify that following device exists:
+ /dev/crypto/nx-gzip
+If you get a permission error run as sudo or set the device permissions:
+ sudo chmod go+rw /dev/crypto/nx-gzip
+However, chmod may not survive across boots. You may create a udev file such
+as:
+ /etc/udev/rules.d/99-nx-gzip.rules
+
+
+Then make and run:
+$ make
+gcc -O3 -I./inc -o gzfht_test gzfht_test.c gzip_vas.c
+gcc -O3 -I./inc -o gunz_test gunz_test.c gzip_vas.c
+
+
+Compress any file using Fixed Huffman mode. Output will have a .nx.gz suffix:
+$ ./gzfht_test gzip_vas.c
+file gzip_vas.c read, 6418 bytes
+compressed 6418 to 3131 bytes total, crc32 checksum = 96b9717d
+
+
+Uncompress the previous output. Output will have a .nx.gunzip suffix:
+./gunz_test gzip_vas.c.nx.gz
+gzHeader FLG 0
+00 00 00 00 04 03
+gzHeader MTIME, XFL, OS ignored
+computed checksum 96b9717d isize 00001912
+stored checksum 96b9717d isize 00001912
+decomp is complete: fclose
+
+
+Compare two files:
+$ sha1sum gzip_vas.c.nx.gz.nx.gunzip gzip_vas.c
+4c0b494f657c0c89a7d9f87dd3da8597be9a887a gzip_vas.c.nx.gz.nx.gunzip
+4c0b494f657c0c89a7d9f87dd3da8597be9a887a gzip_vas.c
+
+
+Note that the code here are intended for testing the nx-gzip hardware function.
+They are not intended for demonstrating performance or compression ratio.
+By being simplistic these selftests expect to allocate the entire set of source
+and target pages in the memory so it needs enough memory to work.
+For more information and source code consider using:
+https://github.com/libnxz/power-gzip
--
2.21.0
^ permalink raw reply related
* Re: [PATCH vdsotest] Use vdso wrapper for gettimeofday()
From: Nathan Lynch @ 2020-04-13 16:05 UTC (permalink / raw)
To: Christophe Leroy; +Cc: Thomas Gleixner, linuxppc-dev, linux-kernel
In-Reply-To: <1c2c6da1-e10f-d2a8-fc5e-ecf405879df5@c-s.fr>
Christophe Leroy <christophe.leroy@c-s.fr> writes:
> Hi Nathan,
>
> Le 16/01/2020 à 17:56, Nathan Lynch a écrit :
>> Hi Christophe,
>>
>> Christophe Leroy <christophe.leroy@c-s.fr> writes:
>>> To properly handle errors returned by gettimeofday(), the
>>> DO_VDSO_CALL() macro has to be used, otherwise vdsotest
>>> misinterpret VDSO function return on error.
>>>
>>> This has gone unnoticed until now because the powerpc VDSO
>>> gettimeofday() always succeed, but while porting powerpc to
>>> generic C VDSO, the following has been encountered:
>>
>> Thanks for this, I'll review it soon.
>>
>> Can you point me to patches for the powerpc generic vdso work?
>>
>
> I have not seen any update on the vdsotest repository, have you been
> able to have a look at the patch ?
Thanks for the reminder. I've applied this now.
^ permalink raw reply
* [PATCH 23/26] perf expr: Add expr_ prefix for parse_ctx and parse_id
From: Arnaldo Carvalho de Melo @ 2020-04-13 16:52 UTC (permalink / raw)
To: Ingo Molnar, Thomas Gleixner
Cc: Mark Rutland, Madhavan Srinivasan, Peter Zijlstra, Jin Yao,
Kan Liang, Andi Kleen, Clark Williams, Anju T Sudhakar,
Mamatha Inamdar, Sukadev Bhattiprolu, Ravi Bangoria, Kajol Jain,
Arnaldo Carvalho de Melo, Joe Mario, Namhyung Kim, Michael Petlan,
Greg Kroah-Hartman, linux-kernel, Alexander Shishkin,
linux-perf-users, Jiri Olsa, linuxppc-dev
In-Reply-To: <20200413165203.1816-1-acme@kernel.org>
From: Jiri Olsa <jolsa@kernel.org>
Adding expr_ prefix for parse_ctx and parse_id, to straighten out the
expr* namespace.
There's no functional change.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Anju T Sudhakar <anju@linux.vnet.ibm.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Jin Yao <yao.jin@linux.intel.com>
Cc: Joe Mario <jmario@redhat.com>
Cc: Kajol Jain <kjain@linux.ibm.com>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
Cc: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@ozlabs.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ravi Bangoria <ravi.bangoria@linux.ibm.com>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linuxppc-dev@lists.ozlabs.org
Link: http://lore.kernel.org/lkml/20200401203340.31402-2-kjain@linux.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/tests/expr.c | 4 ++--
tools/perf/util/expr.c | 10 +++++-----
tools/perf/util/expr.h | 12 ++++++------
tools/perf/util/expr.y | 6 +++---
tools/perf/util/stat-shadow.c | 2 +-
5 files changed, 17 insertions(+), 17 deletions(-)
diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c
index 28313e59d6f6..ea10fc4412c4 100644
--- a/tools/perf/tests/expr.c
+++ b/tools/perf/tests/expr.c
@@ -6,7 +6,7 @@
#include <string.h>
#include <linux/zalloc.h>
-static int test(struct parse_ctx *ctx, const char *e, double val2)
+static int test(struct expr_parse_ctx *ctx, const char *e, double val2)
{
double val;
@@ -22,7 +22,7 @@ int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused)
const char **other;
double val;
int i, ret;
- struct parse_ctx ctx;
+ struct expr_parse_ctx ctx;
int num_other;
expr__ctx_init(&ctx);
diff --git a/tools/perf/util/expr.c b/tools/perf/util/expr.c
index fd192ddf93c1..c8ccc548a585 100644
--- a/tools/perf/util/expr.c
+++ b/tools/perf/util/expr.c
@@ -11,7 +11,7 @@ extern int expr_debug;
#endif
/* Caller must make sure id is allocated */
-void expr__add_id(struct parse_ctx *ctx, const char *name, double val)
+void expr__add_id(struct expr_parse_ctx *ctx, const char *name, double val)
{
int idx;
@@ -21,13 +21,13 @@ void expr__add_id(struct parse_ctx *ctx, const char *name, double val)
ctx->ids[idx].val = val;
}
-void expr__ctx_init(struct parse_ctx *ctx)
+void expr__ctx_init(struct expr_parse_ctx *ctx)
{
ctx->num_ids = 0;
}
static int
-__expr__parse(double *val, struct parse_ctx *ctx, const char *expr,
+__expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,
int start)
{
YY_BUFFER_STATE buffer;
@@ -52,7 +52,7 @@ __expr__parse(double *val, struct parse_ctx *ctx, const char *expr,
return ret;
}
-int expr__parse(double *final_val, struct parse_ctx *ctx, const char *expr)
+int expr__parse(double *final_val, struct expr_parse_ctx *ctx, const char *expr)
{
return __expr__parse(final_val, ctx, expr, EXPR_PARSE) ? -1 : 0;
}
@@ -75,7 +75,7 @@ int expr__find_other(const char *expr, const char *one, const char ***other,
int *num_other)
{
int err, i = 0, j = 0;
- struct parse_ctx ctx;
+ struct expr_parse_ctx ctx;
expr__ctx_init(&ctx);
err = __expr__parse(NULL, &ctx, expr, EXPR_OTHER);
diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
index 9377538f4097..b9e53f2b5844 100644
--- a/tools/perf/util/expr.h
+++ b/tools/perf/util/expr.h
@@ -5,19 +5,19 @@
#define EXPR_MAX_OTHER 20
#define MAX_PARSE_ID EXPR_MAX_OTHER
-struct parse_id {
+struct expr_parse_id {
const char *name;
double val;
};
-struct parse_ctx {
+struct expr_parse_ctx {
int num_ids;
- struct parse_id ids[MAX_PARSE_ID];
+ struct expr_parse_id ids[MAX_PARSE_ID];
};
-void expr__ctx_init(struct parse_ctx *ctx);
-void expr__add_id(struct parse_ctx *ctx, const char *id, double val);
-int expr__parse(double *final_val, struct parse_ctx *ctx, const char *expr);
+void expr__ctx_init(struct expr_parse_ctx *ctx);
+void expr__add_id(struct expr_parse_ctx *ctx, const char *id, double val);
+int expr__parse(double *final_val, struct expr_parse_ctx *ctx, const char *expr);
int expr__find_other(const char *expr, const char *one, const char ***other,
int *num_other);
diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
index 4720cbe79357..cd17486c1c5d 100644
--- a/tools/perf/util/expr.y
+++ b/tools/perf/util/expr.y
@@ -15,7 +15,7 @@
%define api.pure full
%parse-param { double *final_val }
-%parse-param { struct parse_ctx *ctx }
+%parse-param { struct expr_parse_ctx *ctx }
%parse-param {void *scanner}
%lex-param {void* scanner}
@@ -39,14 +39,14 @@
%{
static void expr_error(double *final_val __maybe_unused,
- struct parse_ctx *ctx __maybe_unused,
+ struct expr_parse_ctx *ctx __maybe_unused,
void *scanner,
const char *s)
{
pr_debug("%s\n", s);
}
-static int lookup_id(struct parse_ctx *ctx, char *id, double *val)
+static int lookup_id(struct expr_parse_ctx *ctx, char *id, double *val)
{
int i;
diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
index 03ecb8cd0eec..1ad5c5be7e97 100644
--- a/tools/perf/util/stat-shadow.c
+++ b/tools/perf/util/stat-shadow.c
@@ -729,7 +729,7 @@ static void generic_metric(struct perf_stat_config *config,
struct runtime_stat *st)
{
print_metric_t print_metric = out->print_metric;
- struct parse_ctx pctx;
+ struct expr_parse_ctx pctx;
double ratio, scale;
int i;
void *ctxp = out->ctx;
--
2.21.1
^ permalink raw reply related
* [PATCH 24/26] perf expr: Add expr_scanner_ctx object
From: Arnaldo Carvalho de Melo @ 2020-04-13 16:52 UTC (permalink / raw)
To: Ingo Molnar, Thomas Gleixner
Cc: Mark Rutland, Madhavan Srinivasan, Peter Zijlstra, Jin Yao,
Kan Liang, Andi Kleen, Clark Williams, Anju T Sudhakar,
Mamatha Inamdar, Sukadev Bhattiprolu, Ravi Bangoria, Kajol Jain,
Arnaldo Carvalho de Melo, Joe Mario, Namhyung Kim, Michael Petlan,
Greg Kroah-Hartman, linux-kernel, Alexander Shishkin,
linux-perf-users, Jiri Olsa, linuxppc-dev
In-Reply-To: <20200413165203.1816-1-acme@kernel.org>
From: Jiri Olsa <jolsa@kernel.org>
Add the expr_scanner_ctx object to hold user data for the expr scanner.
Currently it holds only start_token, Kajol Jain will use it to hold 24x7
runtime param.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Anju T Sudhakar <anju@linux.vnet.ibm.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Jin Yao <yao.jin@linux.intel.com>
Cc: Joe Mario <jmario@redhat.com>
Cc: Kajol Jain <kjain@linux.ibm.com>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
Cc: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@ozlabs.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ravi Bangoria <ravi.bangoria@linux.ibm.com>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linuxppc-dev@lists.ozlabs.org
Link: http://lore.kernel.org/lkml/20200401203340.31402-3-kjain@linux.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/util/expr.c | 6 ++++--
tools/perf/util/expr.h | 4 ++++
tools/perf/util/expr.l | 10 +++++-----
3 files changed, 13 insertions(+), 7 deletions(-)
diff --git a/tools/perf/util/expr.c b/tools/perf/util/expr.c
index c8ccc548a585..c3382d58cf40 100644
--- a/tools/perf/util/expr.c
+++ b/tools/perf/util/expr.c
@@ -3,7 +3,6 @@
#include <assert.h>
#include "expr.h"
#include "expr-bison.h"
-#define YY_EXTRA_TYPE int
#include "expr-flex.h"
#ifdef PARSER_DEBUG
@@ -30,11 +29,14 @@ static int
__expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,
int start)
{
+ struct expr_scanner_ctx scanner_ctx = {
+ .start_token = start,
+ };
YY_BUFFER_STATE buffer;
void *scanner;
int ret;
- ret = expr_lex_init_extra(start, &scanner);
+ ret = expr_lex_init_extra(&scanner_ctx, &scanner);
if (ret)
return ret;
diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
index b9e53f2b5844..0938ad166ece 100644
--- a/tools/perf/util/expr.h
+++ b/tools/perf/util/expr.h
@@ -15,6 +15,10 @@ struct expr_parse_ctx {
struct expr_parse_id ids[MAX_PARSE_ID];
};
+struct expr_scanner_ctx {
+ int start_token;
+};
+
void expr__ctx_init(struct expr_parse_ctx *ctx);
void expr__add_id(struct expr_parse_ctx *ctx, const char *id, double val);
int expr__parse(double *final_val, struct expr_parse_ctx *ctx, const char *expr);
diff --git a/tools/perf/util/expr.l b/tools/perf/util/expr.l
index eaad29243c23..2582c2464938 100644
--- a/tools/perf/util/expr.l
+++ b/tools/perf/util/expr.l
@@ -76,13 +76,13 @@ sym [0-9a-zA-Z_\.:@]+
symbol {spec}*{sym}*{spec}*{sym}*
%%
- {
- int start_token;
+ struct expr_scanner_ctx *sctx = expr_get_extra(yyscanner);
- start_token = expr_get_extra(yyscanner);
+ {
+ int start_token = sctx->start_token;
- if (start_token) {
- expr_set_extra(NULL, yyscanner);
+ if (sctx->start_token) {
+ sctx->start_token = 0;
return start_token;
}
}
--
2.21.1
^ permalink raw reply related
* [PATCH 25/26] perf metrictroup: Split the metricgroup__add_metric function
From: Arnaldo Carvalho de Melo @ 2020-04-13 16:52 UTC (permalink / raw)
To: Ingo Molnar, Thomas Gleixner
Cc: Mark Rutland, Madhavan Srinivasan, Peter Zijlstra, Jin Yao,
Jiri Olsa, Kan Liang, Andi Kleen, Clark Williams, Anju T Sudhakar,
Mamatha Inamdar, Sukadev Bhattiprolu, Ravi Bangoria, Kajol Jain,
Arnaldo Carvalho de Melo, Joe Mario, Namhyung Kim, Michael Petlan,
Greg Kroah-Hartman, linux-kernel, Alexander Shishkin,
linux-perf-users, Jiri Olsa, linuxppc-dev
In-Reply-To: <20200413165203.1816-1-acme@kernel.org>
From: Kajol Jain <kjain@linux.ibm.com>
This patch refactors metricgroup__add_metric function where some part of
it move to function metricgroup__add_metric_param. No logic change.
Signed-off-by: Kajol Jain <kjain@linux.ibm.com>
Acked-by: Jiri Olsa <jolsa@redhat.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Anju T Sudhakar <anju@linux.vnet.ibm.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Jin Yao <yao.jin@linux.intel.com>
Cc: Joe Mario <jmario@redhat.com>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
Cc: Mamatha Inamdar <mamatha4@linux.vnet.ibm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@ozlabs.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ravi Bangoria <ravi.bangoria@linux.ibm.com>
Cc: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linuxppc-dev@lists.ozlabs.org
Link: http://lore.kernel.org/lkml/20200401203340.31402-4-kjain@linux.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
tools/perf/util/metricgroup.c | 60 ++++++++++++++++++++---------------
1 file changed, 35 insertions(+), 25 deletions(-)
diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
index 926449a7cdbf..7ad81c8177ea 100644
--- a/tools/perf/util/metricgroup.c
+++ b/tools/perf/util/metricgroup.c
@@ -485,6 +485,39 @@ static bool metricgroup__has_constraint(struct pmu_event *pe)
return false;
}
+static int __metricgroup__add_metric(struct strbuf *events,
+ struct list_head *group_list, struct pmu_event *pe)
+{
+
+ const char **ids;
+ int idnum;
+ struct egroup *eg;
+
+ if (expr__find_other(pe->metric_expr, NULL, &ids, &idnum) < 0)
+ return -EINVAL;
+
+ if (events->len > 0)
+ strbuf_addf(events, ",");
+
+ if (metricgroup__has_constraint(pe))
+ metricgroup__add_metric_non_group(events, ids, idnum);
+ else
+ metricgroup__add_metric_weak_group(events, ids, idnum);
+
+ eg = malloc(sizeof(*eg));
+ if (!eg)
+ return -ENOMEM;
+
+ eg->ids = ids;
+ eg->idnum = idnum;
+ eg->metric_name = pe->metric_name;
+ eg->metric_expr = pe->metric_expr;
+ eg->metric_unit = pe->unit;
+ list_add_tail(&eg->nd, group_list);
+
+ return 0;
+}
+
static int metricgroup__add_metric(const char *metric, struct strbuf *events,
struct list_head *group_list)
{
@@ -504,35 +537,12 @@ static int metricgroup__add_metric(const char *metric, struct strbuf *events,
continue;
if (match_metric(pe->metric_group, metric) ||
match_metric(pe->metric_name, metric)) {
- const char **ids;
- int idnum;
- struct egroup *eg;
pr_debug("metric expr %s for %s\n", pe->metric_expr, pe->metric_name);
- if (expr__find_other(pe->metric_expr,
- NULL, &ids, &idnum) < 0)
- continue;
- if (events->len > 0)
- strbuf_addf(events, ",");
-
- if (metricgroup__has_constraint(pe))
- metricgroup__add_metric_non_group(events, ids, idnum);
- else
- metricgroup__add_metric_weak_group(events, ids, idnum);
-
- eg = malloc(sizeof(struct egroup));
- if (!eg) {
- ret = -ENOMEM;
+ ret = __metricgroup__add_metric(events, group_list, pe);
+ if (ret == -ENOMEM)
break;
- }
- eg->ids = ids;
- eg->idnum = idnum;
- eg->metric_name = pe->metric_name;
- eg->metric_expr = pe->metric_expr;
- eg->metric_unit = pe->unit;
- list_add_tail(&eg->nd, group_list);
- ret = 0;
}
}
return ret;
--
2.21.1
^ permalink raw reply related
* Re: [PATCH v2 1/4] hugetlbfs: add arch_hugetlb_valid_size
From: Mike Kravetz @ 2020-04-13 17:04 UTC (permalink / raw)
To: Peter Xu
Cc: linux-doc, Catalin Marinas, Dave Hansen, Heiko Carstens, linux-mm,
Paul Mackerras, sparclinux, linux-riscv, Will Deacon,
Mina Almasry, linux-s390, Jonathan Corbet, Christian Borntraeger,
Ingo Molnar, Longpeng, Albert Ou, Vasily Gorbik, Paul Walmsley,
Thomas Gleixner, linux-arm-kernel, linux-kernel, Palmer Dabbelt,
Andrew Morton, linuxppc-dev, David S.Miller
In-Reply-To: <20200410191613.GD3172@xz-x1>
On 4/10/20 12:16 PM, Peter Xu wrote:
> On Wed, Apr 01, 2020 at 11:38:16AM -0700, Mike Kravetz wrote:
>> diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h
>> index 2eb6c234d594..81606223494f 100644
>> --- a/arch/arm64/include/asm/hugetlb.h
>> +++ b/arch/arm64/include/asm/hugetlb.h
>> @@ -59,6 +59,8 @@ extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr,
>> extern void set_huge_swap_pte_at(struct mm_struct *mm, unsigned long addr,
>> pte_t *ptep, pte_t pte, unsigned long sz);
>> #define set_huge_swap_pte_at set_huge_swap_pte_at
>> +bool __init arch_hugetlb_valid_size(unsigned long size);
>> +#define arch_hugetlb_valid_size arch_hugetlb_valid_size
>
> Sorry for chimming in late.
Thank you for taking a look!
> Since we're working on removing arch-dependent codes after all.. I'm
> thinking whether we can define arch_hugetlb_valid_size() once in the
> common header (e.g. linux/hugetlb.h), then in mm/hugetlb.c:
>
> bool __init __attribute((weak)) arch_hugetlb_valid_size(unsigned long size)
> {
> return size == HPAGE_SIZE;
> }
>
> We can simply redefine arch_hugetlb_valid_size() in arch specific C
> files where we want to override the default. Would that be slightly
> cleaner?
I think both the #define X X and weak attribute methods are acceptable.
I went with the #define method only because it was most familiar to me.
Using the weak attribute method does appear to be cleaner. I'll code it up.
Anyone else have a preference?
--
Mike Kravetz
^ permalink raw reply
* Re: [PATCH v2 4/4] hugetlbfs: clean up command line processing
From: Mike Kravetz @ 2020-04-13 17:59 UTC (permalink / raw)
To: Peter Xu
Cc: linux-doc, Catalin Marinas, Dave Hansen, Heiko Carstens, linux-mm,
Paul Mackerras, sparclinux, linux-riscv, Will Deacon,
Mina Almasry, linux-s390, Jonathan Corbet, Christian Borntraeger,
Ingo Molnar, Longpeng, Albert Ou, Vasily Gorbik, Paul Walmsley,
Thomas Gleixner, linux-arm-kernel, linux-kernel, Palmer Dabbelt,
Andrew Morton, linuxppc-dev, David S.Miller
In-Reply-To: <20200410203730.GG3172@xz-x1>
On 4/10/20 1:37 PM, Peter Xu wrote:
> On Wed, Apr 01, 2020 at 11:38:19AM -0700, Mike Kravetz wrote:
>> With all hugetlb page processing done in a single file clean up code.
>> - Make code match desired semantics
>> - Update documentation with semantics
>> - Make all warnings and errors messages start with 'HugeTLB:'.
>> - Consistently name command line parsing routines.
>> - Check for hugepages_supported() before processing parameters.
>> - Add comments to code
>> - Describe some of the subtle interactions
>> - Describe semantics of command line arguments
>>
>> Signed-off-by: Mike Kravetz <mike.kravetz@oracle.com>
>> ---
>> .../admin-guide/kernel-parameters.txt | 35 ++++---
>> Documentation/admin-guide/mm/hugetlbpage.rst | 44 +++++++++
>> mm/hugetlb.c | 96 +++++++++++++++----
>> 3 files changed, 142 insertions(+), 33 deletions(-)
>>
>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
>> index 1bd5454b5e5f..de653cfe1726 100644
>> --- a/Documentation/admin-guide/kernel-parameters.txt
>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>> @@ -832,12 +832,15 @@
>> See also Documentation/networking/decnet.txt.
>>
>> default_hugepagesz=
>> - [same as hugepagesz=] The size of the default
>> - HugeTLB page size. This is the size represented by
>> - the legacy /proc/ hugepages APIs, used for SHM, and
>> - default size when mounting hugetlbfs filesystems.
>> - Defaults to the default architecture's huge page size
>> - if not specified.
>> + [HW] The size of the default HugeTLB page size. This
>
> Could I ask what's "HW"? Sorry this is not a comment at all but
> really a pure question I wanted to ask... :)
kernel-parameters.rst includes kernel-parameters.txt and included the meaning
for these codes.
HW Appropriate hardware is enabled.
Previously, it listed an obsolete list of architectures.
>> + is the size represented by the legacy /proc/ hugepages
>> + APIs. In addition, this is the default hugetlb size
>> + used for shmget(), mmap() and mounting hugetlbfs
>> + filesystems. If not specified, defaults to the
>> + architecture's default huge page size. Huge page
>> + sizes are architecture dependent. See also
>> + Documentation/admin-guide/mm/hugetlbpage.rst.
>> + Format: size[KMG]
>>
>> deferred_probe_timeout=
>> [KNL] Debugging option to set a timeout in seconds for
>> @@ -1480,13 +1483,19 @@
>> If enabled, boot-time allocation of gigantic hugepages
>> is skipped.
>>
>> - hugepages= [HW,X86-32,IA-64] HugeTLB pages to allocate at boot.
>> - hugepagesz= [HW,IA-64,PPC,X86-64] The size of the HugeTLB pages.
>> - On x86-64 and powerpc, this option can be specified
>> - multiple times interleaved with hugepages= to reserve
>> - huge pages of different sizes. Valid pages sizes on
>> - x86-64 are 2M (when the CPU supports "pse") and 1G
>> - (when the CPU supports the "pdpe1gb" cpuinfo flag).
>> + hugepages= [HW] Number of HugeTLB pages to allocate at boot.
>> + If this follows hugepagesz (below), it specifies
>> + the number of pages of hugepagesz to be allocated.
>
> "... Otherwise it specifies the number of pages to allocate for the
> default huge page size." ?
Yes, best to be specific. I suspect this is the most common way this
parameter is used.
>
>> + Format: <integer>
>
> How about add a new line here?
Sure
>> + hugepagesz=
>> + [HW] The size of the HugeTLB pages. This is used in
>> + conjunction with hugepages (above) to allocate huge
>> + pages of a specific size at boot. The pair
>> + hugepagesz=X hugepages=Y can be specified once for
>> + each supported huge page size. Huge page sizes are
>> + architecture dependent. See also
>> + Documentation/admin-guide/mm/hugetlbpage.rst.
>> + Format: size[KMG]
>>
>> hung_task_panic=
>> [KNL] Should the hung task detector generate panics.
>> diff --git a/Documentation/admin-guide/mm/hugetlbpage.rst b/Documentation/admin-guide/mm/hugetlbpage.rst
>> index 1cc0bc78d10e..de340c586995 100644
>> --- a/Documentation/admin-guide/mm/hugetlbpage.rst
>> +++ b/Documentation/admin-guide/mm/hugetlbpage.rst
>> @@ -100,6 +100,50 @@ with a huge page size selection parameter "hugepagesz=<size>". <size> must
>> be specified in bytes with optional scale suffix [kKmMgG]. The default huge
>> page size may be selected with the "default_hugepagesz=<size>" boot parameter.
>>
>> +Hugetlb boot command line parameter semantics
>> +hugepagesz - Specify a huge page size. Used in conjunction with hugepages
>> + parameter to preallocate a number of huge pages of the specified
>> + size. Hence, hugepagesz and hugepages are typically specified in
>> + pairs such as:
>> + hugepagesz=2M hugepages=512
>> + hugepagesz can only be specified once on the command line for a
>> + specific huge page size. Valid huge page sizes are architecture
>> + dependent.
>> +hugepages - Specify the number of huge pages to preallocate. This typically
>> + follows a valid hugepagesz parameter. However, if hugepages is the
>> + first or only hugetlb command line parameter it specifies the number
>> + of huge pages of default size to allocate. The number of huge pages
>> + of default size specified in this manner can be overwritten by a
>> + hugepagesz,hugepages parameter pair for the default size.
>> + For example, on an architecture with 2M default huge page size:
>> + hugepages=256 hugepagesz=2M hugepages=512
>> + will result in 512 2M huge pages being allocated. If a hugepages
>> + parameter is preceded by an invalid hugepagesz parameter, it will
>> + be ignored.
>> +default_hugepagesz - Specify the default huge page size. This parameter can
>> + only be specified once on the command line. No other hugetlb command
>> + line parameter is associated with default_hugepagesz. Therefore, it
>> + can appear anywhere on the command line. If hugepages= is the first
>> + hugetlb command line parameter, the specified number of huge pages
>> + will apply to the default huge page size specified with
>> + default_hugepagesz. For example,
>> + hugepages=512 default_hugepagesz=2M
>
> No strong opinion, but considering to the special case of gigantic
> huge page mentioned below, I'm thinking maybe it's easier to just ask
> the user to always use "hugepagesz=X hugepages=Y" pair when people
> want to reserve huge pages.
We can ask people to do this. However, I do not think we can force it at
this time. Why? Mostly because I have seen many instances where people
only specify 'hugepages=X' on the command line to preallocate X huge pages
of default size. So, forcing 'hugepagesz=X hugepages=Y' would break those
users.
> For example, some user might start to use this after this series
> legally:
>
> default_hugepagesz=2M hugepages=1024
Well, that 'works' today. You get that silly error message:
HugeTLB: unsupported default_hugepagesz 2097152. Reverting to 2097152
But, it does preallocate 1024 huge pages of size 2M. Because people
have noticed the silly error message, I suspect this usage,
default_hugepagesz=X hugepages=Y
is in use today and we need to support it.
> Then the user thinks, hmm, maybe it's good to use 1G pages, by just
> changing some numbers:
>
> default_hugepagesz=1G hugepages=2
>
> Then if it stops working it could really confuse the user.
>
> (Besides, it could be an extra maintainaince burden for linux itself)
I did not think about/look into the different behavior for gigantic pages
until updating the documentation. This is not 'new' behavior introduced
by this patch series. It comes about because we do not definitively set
the default huge page size until after command line processing
(in hugetlb_init). And, we must preallocate gigantic huge pages during
command line processing because that is when the bootmem allocater is
available.
I really did not want to change the code to handle this (gigantic page)
situation. There is no indication it is a problem today. However, the
more I think about it the more I think the code should be changed. I'll
work on code modifications to support this.
>> + will result in 512 2M huge pages being allocated. However, specifying
>> + the number of default huge pages in this manner will not apply to
>> + gigantic huge pages. For example,
>> + hugepages=10 default_hugepagesz=1G
>> + or
>> + default_hugepagesz=1G hugepages=10
>> + will NOT result in the allocation of 10 1G huge pages. In order to
>> + preallocate gigantic huge pages, there must be hugepagesz, hugepages
>> + parameter pair. For example,
>> + hugepagesz=1G hugepages=10 default_hugepagesz=1G
>> + or
>> + default_hugepagesz=1G hugepagesz=1G hugepages=10
>> + will result 10 1G huge pages being allocated and the default huge
>> + page size will be set to 1G. Valid default huge page size is
>> + architecture dependent.
>> +
>> When multiple huge page sizes are supported, ``/proc/sys/vm/nr_hugepages``
>> indicates the current number of pre-allocated huge pages of the default size.
>> Thus, one can use the following command to dynamically allocate/deallocate
>> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
>> index 72a4343509d5..74ef53f7c5a7 100644
>> --- a/mm/hugetlb.c
>> +++ b/mm/hugetlb.c
>> @@ -3054,7 +3054,7 @@ static void __init hugetlb_sysfs_init(void)
>> err = hugetlb_sysfs_add_hstate(h, hugepages_kobj,
>> hstate_kobjs, &hstate_attr_group);
>> if (err)
>> - pr_err("Hugetlb: Unable to add hstate %s", h->name);
>> + pr_err("HugeTLB: Unable to add hstate %s", h->name);
>> }
>> }
>>
>> @@ -3158,7 +3158,7 @@ static void hugetlb_register_node(struct node *node)
>> nhs->hstate_kobjs,
>> &per_node_hstate_attr_group);
>> if (err) {
>> - pr_err("Hugetlb: Unable to add hstate %s for node %d\n",
>> + pr_err("HugeTLB: Unable to add hstate %s for node %d\n",
>> h->name, node->dev.id);
>> hugetlb_unregister_node(node);
>> break;
>> @@ -3209,19 +3209,35 @@ static int __init hugetlb_init(void)
>> if (!hugepages_supported())
>> return 0;
>>
>> - if (!size_to_hstate(default_hstate_size)) {
>> - if (default_hstate_size != 0) {
>> - pr_err("HugeTLB: unsupported default_hugepagesz %lu. Reverting to %lu\n",
>> - default_hstate_size, HPAGE_SIZE);
>> - }
>> -
>> + /*
>> + * Make sure HPAGE_SIZE (HUGETLB_PAGE_ORDER) hstate exists. Some
>> + * architectures depend on setup being done here.
>> + *
>> + * If a valid default huge page size was specified on the command line,
>> + * add associated hstate if necessary. If not, set default_hstate_size
>> + * to default size. default_hstate_idx is used at runtime to identify
>> + * the default huge page size/hstate.
>> + */
>> + hugetlb_add_hstate(HUGETLB_PAGE_ORDER);
>> + if (default_hstate_size)
>> + hugetlb_add_hstate(ilog2(default_hstate_size) - PAGE_SHIFT);
>> + else
>> default_hstate_size = HPAGE_SIZE;
>> - hugetlb_add_hstate(HUGETLB_PAGE_ORDER);
>> - }
>> default_hstate_idx = hstate_index(size_to_hstate(default_hstate_size));
>> +
>> + /*
>> + * default_hstate_max_huge_pages != 0 indicates a count (hugepages=)
>> + * specified before a size (hugepagesz=). Use this count for the
>> + * default huge page size, unless a specific value was specified for
>> + * this size in a hugepagesz/hugepages pair.
>> + */
>> if (default_hstate_max_huge_pages) {
>
> Since we're refactoring this - Could default_hstate_max_huge_pages be
> dropped directly (in hugepages= we can create the default hstate, then
> we set max_huge_pages of the default hstate there)? Or did I miss
> anything important?
I do not think that works for 'hugepages=X default_hugepagesz=Y' processing?
It seems like there will need to be more work done on default_hugepagesz
processing.
>> if (!default_hstate.max_huge_pages)
>> - default_hstate.max_huge_pages = default_hstate_max_huge_pages;
>> + default_hstate.max_huge_pages =
>> + default_hstate_max_huge_pages;
>> + else
>> + pr_warn("HugeTLB: First hugepages=%lu ignored\n",
>> + default_hstate_max_huge_pages);
>> }
>>
>> hugetlb_init_hstates();
>> @@ -3274,20 +3290,31 @@ void __init hugetlb_add_hstate(unsigned int order)
>> parsed_hstate = h;
>> }
>>
>> -static int __init hugetlb_nrpages_setup(char *s)
>> +/*
>> + * hugepages command line processing
>> + * hugepages normally follows a valid hugepagsz specification. If not, ignore
>> + * the hugepages value. hugepages can also be the first huge page command line
>> + * option in which case it specifies the number of huge pages for the default
>> + * size.
>> + */
>> +static int __init hugepages_setup(char *s)
>> {
>> unsigned long *mhp;
>> static unsigned long *last_mhp;
>>
>> + if (!hugepages_supported()) {
>> + pr_warn("HugeTLB: huge pages not supported, ignoring hugepages = %s\n", s);
>> + return 0;
>> + }
>> +
>> if (!parsed_valid_hugepagesz) {
>> - pr_warn("hugepages = %s preceded by "
>> - "an unsupported hugepagesz, ignoring\n", s);
>> + pr_warn("HugeTLB: hugepages = %s preceded by an unsupported hugepagesz, ignoring\n", s);
>
> s/preceded/is preceded/?
Thanks
>
>> parsed_valid_hugepagesz = true;
>> - return 1;
>> + return 0;
>> }
>> /*
>> - * !hugetlb_max_hstate means we haven't parsed a hugepagesz= parameter yet,
>> - * so this hugepages= parameter goes to the "default hstate".
>> + * !hugetlb_max_hstate means we haven't parsed a hugepagesz= parameter
>> + * yet, so this hugepages= parameter goes to the "default hstate".
>> */
>> else if (!hugetlb_max_hstate)
>> mhp = &default_hstate_max_huge_pages;
>> @@ -3295,8 +3322,8 @@ static int __init hugetlb_nrpages_setup(char *s)
>> mhp = &parsed_hstate->max_huge_pages;
>>
>> if (mhp == last_mhp) {
>> - pr_warn("hugepages= specified twice without interleaving hugepagesz=, ignoring\n");
>> - return 1;
>> + pr_warn("HugeTLB: hugepages= specified twice without interleaving hugepagesz=, ignoring hugepages=%s\n", s);
>> + return 0;
>> }
>>
>> if (sscanf(s, "%lu", mhp) <= 0)
>> @@ -3314,12 +3341,24 @@ static int __init hugetlb_nrpages_setup(char *s)
>>
>> return 1;
>> }
>> -__setup("hugepages=", hugetlb_nrpages_setup);
>> +__setup("hugepages=", hugepages_setup);
>>
>> +/*
>> + * hugepagesz command line processing
>> + * A specific huge page size can only be specified once with hugepagesz.
>> + * hugepagesz is followed by hugepages on the command line. The global
>> + * variable 'parsed_valid_hugepagesz' is used to determine if prior
>> + * hugepagesz argument was valid.
>> + */
>> static int __init hugepagesz_setup(char *s)
>> {
>> unsigned long size;
>>
>> + if (!hugepages_supported()) {
>> + pr_warn("HugeTLB: huge pages not supported, ignoring hugepagesz = %s\n", s);
>> + return 0;
>> + }
>> +
>> size = (unsigned long)memparse(s, NULL);
>>
>> if (!arch_hugetlb_valid_size(size)) {
>> @@ -3329,19 +3368,31 @@ static int __init hugepagesz_setup(char *s)
>> }
>>
>> if (size_to_hstate(size)) {
>> + parsed_valid_hugepagesz = false;
>> pr_warn("HugeTLB: hugepagesz %s specified twice, ignoring\n", s);
>> return 0;
>> }
>>
>> + parsed_valid_hugepagesz = true;
>> hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT);
>> return 1;
>> }
>> __setup("hugepagesz=", hugepagesz_setup);
>>
>> +/*
>> + * default_hugepagesz command line input
>> + * Only one instance of default_hugepagesz allowed on command line. Do not
>> + * add hstate here as that will confuse hugepagesz/hugepages processing.
>> + */
>> static int __init default_hugepagesz_setup(char *s)
>> {
>> unsigned long size;
>>
>> + if (!hugepages_supported()) {
>> + pr_warn("HugeTLB: huge pages not supported, ignoring default_hugepagesz = %s\n", s);
>> + return 0;
>> + }
>> +
>> size = (unsigned long)memparse(s, NULL);
>>
>> if (!arch_hugetlb_valid_size(size)) {
>> @@ -3349,6 +3400,11 @@ static int __init default_hugepagesz_setup(char *s)
>> return 0;
>> }
>>
>> + if (default_hstate_size) {
>> + pr_err("HugeTLB: default_hugepagesz previously specified, ignoring %s\n", s);
>> + return 0;
>> + }
>
> Nitpick: ideally this can be moved before memparse().
>
> Thanks,
Thanks you.
Unless someone thinks otherwise, I believe more work is needed to handle
preallocation of gigantic huge page sizes in instances like:
hugepages=128 default_hugepagesz=1G
or
default_hugepagesz=1G hugepages=128
I'll work on making such changes.
--
Mike Kravetz
^ permalink raw reply
* [PATCH] powerpc/wii: Fix declaration made after definition
From: Nathan Chancellor @ 2020-04-13 19:06 UTC (permalink / raw)
To: Michael Ellerman
Cc: kbuild test robot, linux-kernel, clang-built-linux,
Paul Mackerras, Nathan Chancellor, linuxppc-dev
A 0day randconfig uncovered an error with clang, trimmed for brevity:
arch/powerpc/platforms/embedded6xx/wii.c:195:7: error: attribute
declaration must precede definition [-Werror,-Wignored-attributes]
if (!machine_is(wii))
^
The macro machine_is declares mach_##name but define_machine actually
defines mach_##name, hence the warning.
To fix this, move define_machine after the is_machine usage.
Fixes: 5a7ee3198dfa ("powerpc: wii: platform support")
Reported-by: kbuild test robot <lkp@intel.com>
Link: https://github.com/ClangBuiltLinux/linux/issues/989
Signed-off-by: Nathan Chancellor <natechancellor@gmail.com>
---
arch/powerpc/platforms/embedded6xx/wii.c | 25 ++++++++++++------------
1 file changed, 12 insertions(+), 13 deletions(-)
diff --git a/arch/powerpc/platforms/embedded6xx/wii.c b/arch/powerpc/platforms/embedded6xx/wii.c
index 67e48b0a164e..a802ef957d63 100644
--- a/arch/powerpc/platforms/embedded6xx/wii.c
+++ b/arch/powerpc/platforms/embedded6xx/wii.c
@@ -172,19 +172,6 @@ static void wii_shutdown(void)
flipper_quiesce();
}
-define_machine(wii) {
- .name = "wii",
- .probe = wii_probe,
- .setup_arch = wii_setup_arch,
- .restart = wii_restart,
- .halt = wii_halt,
- .init_IRQ = wii_pic_probe,
- .get_irq = flipper_pic_get_irq,
- .calibrate_decr = generic_calibrate_decr,
- .progress = udbg_progress,
- .machine_shutdown = wii_shutdown,
-};
-
static const struct of_device_id wii_of_bus[] = {
{ .compatible = "nintendo,hollywood", },
{ },
@@ -200,3 +187,15 @@ static int __init wii_device_probe(void)
}
device_initcall(wii_device_probe);
+define_machine(wii) {
+ .name = "wii",
+ .probe = wii_probe,
+ .setup_arch = wii_setup_arch,
+ .restart = wii_restart,
+ .halt = wii_halt,
+ .init_IRQ = wii_pic_probe,
+ .get_irq = flipper_pic_get_irq,
+ .calibrate_decr = generic_calibrate_decr,
+ .progress = udbg_progress,
+ .machine_shutdown = wii_shutdown,
+};
base-commit: 8f3d9f354286745c751374f5f1fcafee6b3f3136
--
2.26.0
^ permalink raw reply related
* [PATCH] lib/mpi: Fix building for powerpc with clang
From: Nathan Chancellor @ 2020-04-13 19:50 UTC (permalink / raw)
To: Herbert Xu, Michael Ellerman
Cc: kbuild test robot, linux-kernel, clang-built-linux,
Paul Mackerras, linux-crypto, Nathan Chancellor, linuxppc-dev
0day reports over and over on an powerpc randconfig with clang:
lib/mpi/generic_mpih-mul1.c:37:13: error: invalid use of a cast in a
inline asm context requiring an l-value: remove the cast or build with
-fheinous-gnu-extensions
Remove the superfluous casts, which have been done previously for x86
and arm32 in commit dea632cadd12 ("lib/mpi: fix build with clang") and
commit 7b7c1df2883d ("lib/mpi/longlong.h: fix building with 32-bit
x86").
Reported-by: kbuild test robot <lkp@intel.com>
Link: https://github.com/ClangBuiltLinux/linux/issues/991
Signed-off-by: Nathan Chancellor <natechancellor@gmail.com>
---
Herbet seems to take lib/mpi patches but there does not seem to be a
formal maintainer so Michael could take it since it is just a powerpc
thing.
lib/mpi/longlong.h | 34 +++++++++++++++++-----------------
1 file changed, 17 insertions(+), 17 deletions(-)
diff --git a/lib/mpi/longlong.h b/lib/mpi/longlong.h
index 2dceaca27489..891e1c3549c4 100644
--- a/lib/mpi/longlong.h
+++ b/lib/mpi/longlong.h
@@ -722,22 +722,22 @@ do { \
do { \
if (__builtin_constant_p(bh) && (bh) == 0) \
__asm__ ("{a%I4|add%I4c} %1,%3,%4\n\t{aze|addze} %0,%2" \
- : "=r" ((USItype)(sh)), \
- "=&r" ((USItype)(sl)) \
+ : "=r" (sh), \
+ "=&r" (sl) \
: "%r" ((USItype)(ah)), \
"%r" ((USItype)(al)), \
"rI" ((USItype)(bl))); \
else if (__builtin_constant_p(bh) && (bh) == ~(USItype) 0) \
__asm__ ("{a%I4|add%I4c} %1,%3,%4\n\t{ame|addme} %0,%2" \
- : "=r" ((USItype)(sh)), \
- "=&r" ((USItype)(sl)) \
+ : "=r" (sh), \
+ "=&r" (sl) \
: "%r" ((USItype)(ah)), \
"%r" ((USItype)(al)), \
"rI" ((USItype)(bl))); \
else \
__asm__ ("{a%I5|add%I5c} %1,%4,%5\n\t{ae|adde} %0,%2,%3" \
- : "=r" ((USItype)(sh)), \
- "=&r" ((USItype)(sl)) \
+ : "=r" (sh), \
+ "=&r" (sl) \
: "%r" ((USItype)(ah)), \
"r" ((USItype)(bh)), \
"%r" ((USItype)(al)), \
@@ -747,36 +747,36 @@ do { \
do { \
if (__builtin_constant_p(ah) && (ah) == 0) \
__asm__ ("{sf%I3|subf%I3c} %1,%4,%3\n\t{sfze|subfze} %0,%2" \
- : "=r" ((USItype)(sh)), \
- "=&r" ((USItype)(sl)) \
+ : "=r" (sh), \
+ "=&r" (sl) \
: "r" ((USItype)(bh)), \
"rI" ((USItype)(al)), \
"r" ((USItype)(bl))); \
else if (__builtin_constant_p(ah) && (ah) == ~(USItype) 0) \
__asm__ ("{sf%I3|subf%I3c} %1,%4,%3\n\t{sfme|subfme} %0,%2" \
- : "=r" ((USItype)(sh)), \
- "=&r" ((USItype)(sl)) \
+ : "=r" (sh), \
+ "=&r" (sl) \
: "r" ((USItype)(bh)), \
"rI" ((USItype)(al)), \
"r" ((USItype)(bl))); \
else if (__builtin_constant_p(bh) && (bh) == 0) \
__asm__ ("{sf%I3|subf%I3c} %1,%4,%3\n\t{ame|addme} %0,%2" \
- : "=r" ((USItype)(sh)), \
- "=&r" ((USItype)(sl)) \
+ : "=r" (sh), \
+ "=&r" (sl) \
: "r" ((USItype)(ah)), \
"rI" ((USItype)(al)), \
"r" ((USItype)(bl))); \
else if (__builtin_constant_p(bh) && (bh) == ~(USItype) 0) \
__asm__ ("{sf%I3|subf%I3c} %1,%4,%3\n\t{aze|addze} %0,%2" \
- : "=r" ((USItype)(sh)), \
- "=&r" ((USItype)(sl)) \
+ : "=r" (sh), \
+ "=&r" (sl) \
: "r" ((USItype)(ah)), \
"rI" ((USItype)(al)), \
"r" ((USItype)(bl))); \
else \
__asm__ ("{sf%I4|subf%I4c} %1,%5,%4\n\t{sfe|subfe} %0,%3,%2" \
- : "=r" ((USItype)(sh)), \
- "=&r" ((USItype)(sl)) \
+ : "=r" (sh), \
+ "=&r" (sl) \
: "r" ((USItype)(ah)), \
"r" ((USItype)(bh)), \
"rI" ((USItype)(al)), \
@@ -787,7 +787,7 @@ do { \
do { \
USItype __m0 = (m0), __m1 = (m1); \
__asm__ ("mulhwu %0,%1,%2" \
- : "=r" ((USItype) ph) \
+ : "=r" (ph) \
: "%r" (__m0), \
"r" (__m1)); \
(pl) = __m0 * __m1; \
base-commit: 8f3d9f354286745c751374f5f1fcafee6b3f3136
--
2.26.0
^ permalink raw reply related
* Re: [PATCH 25/28] mm: remove vmalloc_user_node_flags
From: Johannes Weiner @ 2020-04-13 20:03 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: linux-hyperv, David Airlie, dri-devel, linux-mm, K. Y. Srinivasan,
Sumit Semwal, linux-arch, linux-s390, Wei Liu, Stephen Hemminger,
x86, Christoph Hellwig, Peter Zijlstra, Laura Abbott, Nitin Gupta,
Daniel Vetter, Haiyang Zhang, linaro-mm-sig, bpf,
linux-arm-kernel, Robin Murphy, open list, Minchan Kim, iommu,
Sakari Ailus, Andrew Morton, linuxppc-dev
In-Reply-To: <CAEf4BzZOC2tLrqt_Km=WQb=9xiya2e31i6K3oJuzgYQt6wp1LQ@mail.gmail.com>
On Thu, Apr 09, 2020 at 03:25:03PM -0700, Andrii Nakryiko wrote:
> cc Johannes who suggested this API call originally
I forgot why we did it this way - probably just cruft begetting more
cruft. Either way, Christoph's cleanup makes this look a lot better.
> On Wed, Apr 8, 2020 at 5:03 AM Christoph Hellwig <hch@lst.de> wrote:
> >
> > Open code it in __bpf_map_area_alloc, which is the only caller. Also
> > clean up __bpf_map_area_alloc to have a single vmalloc call with
> > slightly different flags instead of the current two different calls.
> >
> > For this to compile for the nommu case add a __vmalloc_node_range stub
> > to nommu.c.
> >
> > Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
^ permalink raw reply
* -Wincompatible-pointer-types in arch/powerpc/platforms/embedded6xx/mvme5100.c
From: Nathan Chancellor @ 2020-04-13 20:10 UTC (permalink / raw)
To: linuxppc-dev, clang-built-linux
Hi all,
0day reported a build error in arch/powerpc/platforms/embedded6xx/mvme5100.c
when building with clang [1]. This is not a clang specific issue since
it also happens with gcc:
$ curl -LSs https://lore.kernel.org/lkml/202004131704.6MH1jcq3%25lkp@intel.com/2-a.bin | gzip -d > .config
$ make -j$(nproc) -s ARCH=powerpc CROSS_COMPILE=powerpc-linux- olddefconfig arch/powerpc/platforms/embedded6xx/mvme5100.o
arch/powerpc/platforms/embedded6xx/mvme5100.c: In function 'mvme5100_add_bridge':
arch/powerpc/platforms/embedded6xx/mvme5100.c:135:58: error: passing argument 5 of 'early_read_config_dword' from incompatible pointer type [-Werror=incompatible-pointer-types]
135 | early_read_config_dword(hose, 0, 0, PCI_BASE_ADDRESS_1, &pci_membase);
| ^~~~~~~~~~~~
| |
| phys_addr_t * {aka long long unsigned int *}
In file included from arch/powerpc/platforms/embedded6xx/mvme5100.c:18:
./arch/powerpc/include/asm/pci-bridge.h:139:32: note: expected 'u32 *' {aka 'unsigned int *'} but argument is of type 'phys_addr_t *' {aka 'long long unsigned int *'}
139 | int dev_fn, int where, u32 *val);
| ~~~~~^~~
In file included from ./include/linux/printk.h:7,
from ./include/linux/kernel.h:15,
from ./include/linux/list.h:9,
from ./include/linux/rculist.h:10,
from ./include/linux/pid.h:5,
from ./include/linux/sched.h:14,
from ./include/linux/ratelimit.h:6,
from ./include/linux/dev_printk.h:16,
from ./include/linux/device.h:15,
from ./include/linux/of_platform.h:9,
from arch/powerpc/platforms/embedded6xx/mvme5100.c:15:
./include/linux/kern_levels.h:5:18: error: format '%x' expects argument of type 'unsigned int', but argument 2 has type 'phys_addr_t' {aka 'long long unsigned int'} [-Werror=format=]
5 | #define KERN_SOH "\001" /* ASCII Start Of Header */
| ^~~~~~
./include/linux/kern_levels.h:14:19: note: in expansion of macro 'KERN_SOH'
14 | #define KERN_INFO KERN_SOH "6" /* informational */
| ^~~~~~~~
./include/linux/printk.h:305:9: note: in expansion of macro 'KERN_INFO'
305 | printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__)
| ^~~~~~~~~
arch/powerpc/platforms/embedded6xx/mvme5100.c:142:2: note: in expansion of macro 'pr_info'
142 | pr_info("mvme5100_pic_init: pci_membase: %x\n", pci_membase);
| ^~~~~~~
arch/powerpc/platforms/embedded6xx/mvme5100.c:142:44: note: format string is defined here
142 | pr_info("mvme5100_pic_init: pci_membase: %x\n", pci_membase);
| ~^
| |
| unsigned int
| %llx
cc1: all warnings being treated as errors
make[4]: *** [scripts/Makefile.build:267: arch/powerpc/platforms/embedded6xx/mvme5100.o] Error 1
make[3]: *** [scripts/Makefile.build:488: arch/powerpc/platforms/embedded6xx] Error 2
make[2]: *** [scripts/Makefile.build:488: arch/powerpc/platforms] Error 2
make[1]: *** [Makefile:1722: arch/powerpc] Error 2
make: *** [Makefile:328: __build_one_by_one] Error 2
I am not sure how exactly this should be fixed. Should this driver just
not be selectable when CONFIG_PHYS_ADDR_T_64BIT is selected or is there
something else that I am missing?
[1]: https://lore.kernel.org/lkml/202004131704.6MH1jcq3%25lkp@intel.com/
Cheers,
Nathan
^ permalink raw reply
* [PATCH 0/2] mm, treewide: Rename kzfree() to kfree_sensitive()
From: Waiman Long @ 2020-04-13 21:15 UTC (permalink / raw)
To: Andrew Morton, David Howells, Jarkko Sakkinen, James Morris,
Serge E. Hallyn, Linus Torvalds, Joe Perches, Matthew Wilcox,
David Rientjes
Cc: samba-technical, virtualization, linux-mm, linux-sctp,
target-devel, linux-stm32, devel, linux-s390, linux-scsi, x86,
kasan-dev, cocci, linux-wpan, intel-wired-lan, Waiman Long,
linux-crypto, linux-pm, ecryptfs, linux-nfs, linux-fscrypt,
linux-mediatek, linux-amlogic, linux-arm-kernel, linux-cifs,
netdev, linux-wireless, linux-kernel, linux-bluetooth,
linux-security-module, keyrings, tipc-discussion, wireguard,
linux-ppp, linux-integrity, linuxppc-dev, linux-btrfs
This patchset makes a global rename of the kzfree() to kfree_sensitive()
to highlight the fact buffer clearing is only needed if the data objects
contain sensitive information like encrpytion key. The fact that kzfree()
uses memset() to do the clearing isn't totally safe either as compiler
may compile out the clearing in their optimizer. Instead, the new
kfree_sensitive() uses memzero_explicit() which won't get compiled out.
Waiman Long (2):
mm, treewide: Rename kzfree() to kfree_sensitive()
crypto: Remove unnecessary memzero_explicit()
arch/s390/crypto/prng.c | 4 +--
arch/x86/power/hibernate.c | 2 +-
crypto/adiantum.c | 2 +-
crypto/ahash.c | 4 +--
crypto/api.c | 2 +-
crypto/asymmetric_keys/verify_pefile.c | 4 +--
crypto/deflate.c | 2 +-
crypto/drbg.c | 10 +++---
crypto/ecc.c | 8 ++---
crypto/ecdh.c | 2 +-
crypto/gcm.c | 2 +-
crypto/gf128mul.c | 4 +--
crypto/jitterentropy-kcapi.c | 2 +-
crypto/rng.c | 2 +-
crypto/rsa-pkcs1pad.c | 6 ++--
crypto/seqiv.c | 2 +-
crypto/shash.c | 2 +-
crypto/skcipher.c | 2 +-
crypto/testmgr.c | 6 ++--
crypto/zstd.c | 2 +-
.../allwinner/sun8i-ce/sun8i-ce-cipher.c | 17 +++-------
.../allwinner/sun8i-ss/sun8i-ss-cipher.c | 18 +++-------
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 14 +++-----
drivers/crypto/atmel-ecc.c | 2 +-
drivers/crypto/caam/caampkc.c | 28 +++++++--------
drivers/crypto/cavium/cpt/cptvf_main.c | 6 ++--
drivers/crypto/cavium/cpt/cptvf_reqmanager.c | 12 +++----
drivers/crypto/cavium/nitrox/nitrox_lib.c | 4 +--
drivers/crypto/cavium/zip/zip_crypto.c | 6 ++--
drivers/crypto/ccp/ccp-crypto-rsa.c | 6 ++--
drivers/crypto/ccree/cc_aead.c | 4 +--
drivers/crypto/ccree/cc_buffer_mgr.c | 4 +--
drivers/crypto/ccree/cc_cipher.c | 6 ++--
drivers/crypto/ccree/cc_hash.c | 8 ++---
drivers/crypto/ccree/cc_request_mgr.c | 2 +-
drivers/crypto/inside-secure/safexcel_hash.c | 3 +-
drivers/crypto/marvell/cesa/hash.c | 2 +-
.../crypto/marvell/octeontx/otx_cptvf_main.c | 6 ++--
.../marvell/octeontx/otx_cptvf_reqmgr.h | 2 +-
drivers/crypto/mediatek/mtk-aes.c | 2 +-
drivers/crypto/nx/nx.c | 4 +--
drivers/crypto/virtio/virtio_crypto_algs.c | 12 +++----
drivers/crypto/virtio/virtio_crypto_core.c | 2 +-
drivers/md/dm-crypt.c | 34 +++++++++----------
drivers/md/dm-integrity.c | 6 ++--
drivers/misc/ibmvmc.c | 6 ++--
.../hisilicon/hns3/hns3pf/hclge_mbx.c | 2 +-
.../net/ethernet/intel/ixgbe/ixgbe_ipsec.c | 6 ++--
drivers/net/ppp/ppp_mppe.c | 6 ++--
drivers/net/wireguard/noise.c | 4 +--
drivers/net/wireguard/peer.c | 2 +-
drivers/net/wireless/intel/iwlwifi/pcie/rx.c | 2 +-
.../net/wireless/intel/iwlwifi/pcie/tx-gen2.c | 6 ++--
drivers/net/wireless/intel/iwlwifi/pcie/tx.c | 6 ++--
drivers/net/wireless/intersil/orinoco/wext.c | 4 +--
drivers/s390/crypto/ap_bus.h | 4 +--
drivers/staging/ks7010/ks_hostif.c | 2 +-
drivers/staging/rtl8723bs/core/rtw_security.c | 2 +-
drivers/staging/wlan-ng/p80211netdev.c | 2 +-
drivers/target/iscsi/iscsi_target_auth.c | 2 +-
fs/btrfs/ioctl.c | 2 +-
fs/cifs/cifsencrypt.c | 2 +-
fs/cifs/connect.c | 10 +++---
fs/cifs/dfs_cache.c | 2 +-
fs/cifs/misc.c | 8 ++---
fs/crypto/keyring.c | 6 ++--
fs/crypto/keysetup_v1.c | 4 +--
fs/ecryptfs/keystore.c | 4 +--
fs/ecryptfs/messaging.c | 2 +-
include/crypto/aead.h | 2 +-
include/crypto/akcipher.h | 2 +-
include/crypto/gf128mul.h | 2 +-
include/crypto/hash.h | 2 +-
include/crypto/internal/acompress.h | 2 +-
include/crypto/kpp.h | 2 +-
include/crypto/skcipher.h | 2 +-
include/linux/slab.h | 2 +-
lib/mpi/mpiutil.c | 6 ++--
lib/test_kasan.c | 6 ++--
mm/slab_common.c | 10 +++---
net/atm/mpoa_caches.c | 4 +--
net/bluetooth/ecdh_helper.c | 6 ++--
net/bluetooth/smp.c | 24 ++++++-------
net/core/sock.c | 2 +-
net/ipv4/tcp_fastopen.c | 2 +-
net/mac80211/aead_api.c | 4 +--
net/mac80211/aes_gmac.c | 2 +-
net/mac80211/key.c | 2 +-
net/mac802154/llsec.c | 20 +++++------
net/sctp/auth.c | 2 +-
net/sctp/socket.c | 2 +-
net/sunrpc/auth_gss/gss_krb5_crypto.c | 4 +--
net/sunrpc/auth_gss/gss_krb5_keys.c | 6 ++--
net/sunrpc/auth_gss/gss_krb5_mech.c | 2 +-
net/tipc/crypto.c | 10 +++---
net/wireless/core.c | 2 +-
net/wireless/ibss.c | 4 +--
net/wireless/lib80211_crypt_tkip.c | 2 +-
net/wireless/lib80211_crypt_wep.c | 2 +-
net/wireless/nl80211.c | 24 ++++++-------
net/wireless/sme.c | 6 ++--
net/wireless/util.c | 2 +-
net/wireless/wext-sme.c | 2 +-
scripts/coccinelle/free/devm_free.cocci | 4 +--
scripts/coccinelle/free/ifnullfree.cocci | 4 +--
scripts/coccinelle/free/kfree.cocci | 6 ++--
scripts/coccinelle/free/kfreeaddr.cocci | 2 +-
security/apparmor/domain.c | 4 +--
security/apparmor/include/file.h | 2 +-
security/apparmor/policy.c | 24 ++++++-------
security/apparmor/policy_ns.c | 6 ++--
security/apparmor/policy_unpack.c | 14 ++++----
security/keys/big_key.c | 6 ++--
security/keys/dh.c | 14 ++++----
security/keys/encrypted-keys/encrypted.c | 14 ++++----
security/keys/trusted-keys/trusted_tpm1.c | 34 +++++++++----------
security/keys/user_defined.c | 6 ++--
117 files changed, 332 insertions(+), 358 deletions(-)
--
2.18.1
^ permalink raw reply
* [PATCH 1/2] mm, treewide: Rename kzfree() to kfree_sensitive()
From: Waiman Long @ 2020-04-13 21:15 UTC (permalink / raw)
To: Andrew Morton, David Howells, Jarkko Sakkinen, James Morris,
Serge E. Hallyn, Linus Torvalds, Joe Perches, Matthew Wilcox,
David Rientjes
Cc: samba-technical, virtualization, linux-mm, linux-sctp,
target-devel, linux-stm32, devel, linux-s390, linux-scsi, x86,
kasan-dev, cocci, linux-wpan, intel-wired-lan, Waiman Long,
linux-crypto, linux-pm, ecryptfs, linux-nfs, linux-fscrypt,
linux-mediatek, linux-amlogic, linux-arm-kernel, linux-cifs,
netdev, linux-wireless, linux-kernel, linux-bluetooth,
linux-security-module, keyrings, tipc-discussion, wireguard,
linux-ppp, linux-integrity, linuxppc-dev, linux-btrfs
In-Reply-To: <20200413211550.8307-1-longman@redhat.com>
As said by Linus:
A symmetric naming is only helpful if it implies symmetries in use.
Otherwise it's actively misleading.
In "kzalloc()", the z is meaningful and an important part of what the
caller wants.
In "kzfree()", the z is actively detrimental, because maybe in the
future we really _might_ want to use that "memfill(0xdeadbeef)" or
something. The "zero" part of the interface isn't even _relevant_.
The main reason that kzfree() exists is to clear sensitive information
that should not be leaked to other future users of the same memory
objects.
Rename kzfree() to kfree_sensitive() to follow the example of the
recently added kvfree_sensitive() and make the intention of the API
more explicit. In addition, memzero_explicit() is used to clear the
memory to make sure that it won't get optimized away by the compiler.
The renaming is done by using the command sequence:
git grep -w --name-only kzfree |\
xargs sed -i 's/\bkzfree\b/kfree_sensitive/'
followed by some editing of the kfree_sensitive() kerneldoc and the
use of memzero_explicit() instead of memset().
Suggested-by: Joe Perches <joe@perches.com>
Signed-off-by: Waiman Long <longman@redhat.com>
---
arch/s390/crypto/prng.c | 4 +--
arch/x86/power/hibernate.c | 2 +-
crypto/adiantum.c | 2 +-
crypto/ahash.c | 4 +--
crypto/api.c | 2 +-
crypto/asymmetric_keys/verify_pefile.c | 4 +--
crypto/deflate.c | 2 +-
crypto/drbg.c | 10 +++---
crypto/ecc.c | 8 ++---
crypto/ecdh.c | 2 +-
crypto/gcm.c | 2 +-
crypto/gf128mul.c | 4 +--
crypto/jitterentropy-kcapi.c | 2 +-
crypto/rng.c | 2 +-
crypto/rsa-pkcs1pad.c | 6 ++--
crypto/seqiv.c | 2 +-
crypto/shash.c | 2 +-
crypto/skcipher.c | 2 +-
crypto/testmgr.c | 6 ++--
crypto/zstd.c | 2 +-
.../allwinner/sun8i-ce/sun8i-ce-cipher.c | 2 +-
.../allwinner/sun8i-ss/sun8i-ss-cipher.c | 2 +-
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 4 +--
drivers/crypto/atmel-ecc.c | 2 +-
drivers/crypto/caam/caampkc.c | 28 +++++++--------
drivers/crypto/cavium/cpt/cptvf_main.c | 6 ++--
drivers/crypto/cavium/cpt/cptvf_reqmanager.c | 12 +++----
drivers/crypto/cavium/nitrox/nitrox_lib.c | 4 +--
drivers/crypto/cavium/zip/zip_crypto.c | 6 ++--
drivers/crypto/ccp/ccp-crypto-rsa.c | 6 ++--
drivers/crypto/ccree/cc_aead.c | 4 +--
drivers/crypto/ccree/cc_buffer_mgr.c | 4 +--
drivers/crypto/ccree/cc_cipher.c | 6 ++--
drivers/crypto/ccree/cc_hash.c | 8 ++---
drivers/crypto/ccree/cc_request_mgr.c | 2 +-
drivers/crypto/marvell/cesa/hash.c | 2 +-
.../crypto/marvell/octeontx/otx_cptvf_main.c | 6 ++--
.../marvell/octeontx/otx_cptvf_reqmgr.h | 2 +-
drivers/crypto/mediatek/mtk-aes.c | 2 +-
drivers/crypto/nx/nx.c | 4 +--
drivers/crypto/virtio/virtio_crypto_algs.c | 12 +++----
drivers/crypto/virtio/virtio_crypto_core.c | 2 +-
drivers/md/dm-crypt.c | 34 +++++++++----------
drivers/md/dm-integrity.c | 6 ++--
drivers/misc/ibmvmc.c | 6 ++--
.../hisilicon/hns3/hns3pf/hclge_mbx.c | 2 +-
.../net/ethernet/intel/ixgbe/ixgbe_ipsec.c | 6 ++--
drivers/net/ppp/ppp_mppe.c | 6 ++--
drivers/net/wireguard/noise.c | 4 +--
drivers/net/wireguard/peer.c | 2 +-
drivers/net/wireless/intel/iwlwifi/pcie/rx.c | 2 +-
.../net/wireless/intel/iwlwifi/pcie/tx-gen2.c | 6 ++--
drivers/net/wireless/intel/iwlwifi/pcie/tx.c | 6 ++--
drivers/net/wireless/intersil/orinoco/wext.c | 4 +--
drivers/s390/crypto/ap_bus.h | 4 +--
drivers/staging/ks7010/ks_hostif.c | 2 +-
drivers/staging/rtl8723bs/core/rtw_security.c | 2 +-
drivers/staging/wlan-ng/p80211netdev.c | 2 +-
drivers/target/iscsi/iscsi_target_auth.c | 2 +-
fs/btrfs/ioctl.c | 2 +-
fs/cifs/cifsencrypt.c | 2 +-
fs/cifs/connect.c | 10 +++---
fs/cifs/dfs_cache.c | 2 +-
fs/cifs/misc.c | 8 ++---
fs/crypto/keyring.c | 6 ++--
fs/crypto/keysetup_v1.c | 4 +--
fs/ecryptfs/keystore.c | 4 +--
fs/ecryptfs/messaging.c | 2 +-
include/crypto/aead.h | 2 +-
include/crypto/akcipher.h | 2 +-
include/crypto/gf128mul.h | 2 +-
include/crypto/hash.h | 2 +-
include/crypto/internal/acompress.h | 2 +-
include/crypto/kpp.h | 2 +-
include/crypto/skcipher.h | 2 +-
include/linux/slab.h | 2 +-
lib/mpi/mpiutil.c | 6 ++--
lib/test_kasan.c | 6 ++--
mm/slab_common.c | 10 +++---
net/atm/mpoa_caches.c | 4 +--
net/bluetooth/ecdh_helper.c | 6 ++--
net/bluetooth/smp.c | 24 ++++++-------
net/core/sock.c | 2 +-
net/ipv4/tcp_fastopen.c | 2 +-
net/mac80211/aead_api.c | 4 +--
net/mac80211/aes_gmac.c | 2 +-
net/mac80211/key.c | 2 +-
net/mac802154/llsec.c | 20 +++++------
net/sctp/auth.c | 2 +-
net/sctp/socket.c | 2 +-
net/sunrpc/auth_gss/gss_krb5_crypto.c | 4 +--
net/sunrpc/auth_gss/gss_krb5_keys.c | 6 ++--
net/sunrpc/auth_gss/gss_krb5_mech.c | 2 +-
net/tipc/crypto.c | 10 +++---
net/wireless/core.c | 2 +-
net/wireless/ibss.c | 4 +--
net/wireless/lib80211_crypt_tkip.c | 2 +-
net/wireless/lib80211_crypt_wep.c | 2 +-
net/wireless/nl80211.c | 24 ++++++-------
net/wireless/sme.c | 6 ++--
net/wireless/util.c | 2 +-
net/wireless/wext-sme.c | 2 +-
scripts/coccinelle/free/devm_free.cocci | 4 +--
scripts/coccinelle/free/ifnullfree.cocci | 4 +--
scripts/coccinelle/free/kfree.cocci | 6 ++--
scripts/coccinelle/free/kfreeaddr.cocci | 2 +-
security/apparmor/domain.c | 4 +--
security/apparmor/include/file.h | 2 +-
security/apparmor/policy.c | 24 ++++++-------
security/apparmor/policy_ns.c | 6 ++--
security/apparmor/policy_unpack.c | 14 ++++----
security/keys/big_key.c | 6 ++--
security/keys/dh.c | 14 ++++----
security/keys/encrypted-keys/encrypted.c | 14 ++++----
security/keys/trusted-keys/trusted_tpm1.c | 34 +++++++++----------
security/keys/user_defined.c | 6 ++--
116 files changed, 323 insertions(+), 323 deletions(-)
diff --git a/arch/s390/crypto/prng.c b/arch/s390/crypto/prng.c
index d977643fa627..04caac037b7a 100644
--- a/arch/s390/crypto/prng.c
+++ b/arch/s390/crypto/prng.c
@@ -249,7 +249,7 @@ static void prng_tdes_deinstantiate(void)
{
pr_debug("The prng module stopped "
"after running in triple DES mode\n");
- kzfree(prng_data);
+ kfree_sensitive(prng_data);
}
@@ -442,7 +442,7 @@ static int __init prng_sha512_instantiate(void)
static void prng_sha512_deinstantiate(void)
{
pr_debug("The prng module stopped after running in SHA-512 mode\n");
- kzfree(prng_data);
+ kfree_sensitive(prng_data);
}
diff --git a/arch/x86/power/hibernate.c b/arch/x86/power/hibernate.c
index fc413717a45f..45e8b775b88e 100644
--- a/arch/x86/power/hibernate.c
+++ b/arch/x86/power/hibernate.c
@@ -98,7 +98,7 @@ static int get_e820_md5(struct e820_table *table, void *buf)
if (crypto_shash_digest(desc, (u8 *)table, size, buf))
ret = -EINVAL;
- kzfree(desc);
+ kfree_sensitive(desc);
free_tfm:
crypto_free_shash(tfm);
diff --git a/crypto/adiantum.c b/crypto/adiantum.c
index cf2b9f4103dd..b7824e214961 100644
--- a/crypto/adiantum.c
+++ b/crypto/adiantum.c
@@ -177,7 +177,7 @@ static int adiantum_setkey(struct crypto_skcipher *tfm, const u8 *key,
keyp += NHPOLY1305_KEY_SIZE;
WARN_ON(keyp != &data->derived_keys[ARRAY_SIZE(data->derived_keys)]);
out:
- kzfree(data);
+ kfree_sensitive(data);
return err;
}
diff --git a/crypto/ahash.c b/crypto/ahash.c
index 68a0f0cb75c4..d9d65d1cc669 100644
--- a/crypto/ahash.c
+++ b/crypto/ahash.c
@@ -183,7 +183,7 @@ static int ahash_setkey_unaligned(struct crypto_ahash *tfm, const u8 *key,
alignbuffer = (u8 *)ALIGN((unsigned long)buffer, alignmask + 1);
memcpy(alignbuffer, key, keylen);
ret = tfm->setkey(tfm, alignbuffer, keylen);
- kzfree(buffer);
+ kfree_sensitive(buffer);
return ret;
}
@@ -302,7 +302,7 @@ static void ahash_restore_req(struct ahash_request *req, int err)
req->priv = NULL;
/* Free the req->priv.priv from the ADJUSTED request. */
- kzfree(priv);
+ kfree_sensitive(priv);
}
static void ahash_notify_einprogress(struct ahash_request *req)
diff --git a/crypto/api.c b/crypto/api.c
index 7d71a9b10e5f..5fa4fac4bd02 100644
--- a/crypto/api.c
+++ b/crypto/api.c
@@ -564,7 +564,7 @@ void crypto_destroy_tfm(void *mem, struct crypto_tfm *tfm)
alg->cra_exit(tfm);
crypto_exit_ops(tfm);
crypto_mod_put(alg);
- kzfree(mem);
+ kfree_sensitive(mem);
}
EXPORT_SYMBOL_GPL(crypto_destroy_tfm);
diff --git a/crypto/asymmetric_keys/verify_pefile.c b/crypto/asymmetric_keys/verify_pefile.c
index cc9dbcecaaca..7553ab18db89 100644
--- a/crypto/asymmetric_keys/verify_pefile.c
+++ b/crypto/asymmetric_keys/verify_pefile.c
@@ -376,7 +376,7 @@ static int pefile_digest_pe(const void *pebuf, unsigned int pelen,
}
error:
- kzfree(desc);
+ kfree_sensitive(desc);
error_no_desc:
crypto_free_shash(tfm);
kleave(" = %d", ret);
@@ -447,6 +447,6 @@ int verify_pefile_signature(const void *pebuf, unsigned pelen,
ret = pefile_digest_pe(pebuf, pelen, &ctx);
error:
- kzfree(ctx.digest);
+ kfree_sensitive(ctx.digest);
return ret;
}
diff --git a/crypto/deflate.c b/crypto/deflate.c
index 4c0e6c9d942a..b2a46f6dc961 100644
--- a/crypto/deflate.c
+++ b/crypto/deflate.c
@@ -163,7 +163,7 @@ static void __deflate_exit(void *ctx)
static void deflate_free_ctx(struct crypto_scomp *tfm, void *ctx)
{
__deflate_exit(ctx);
- kzfree(ctx);
+ kfree_sensitive(ctx);
}
static void deflate_exit(struct crypto_tfm *tfm)
diff --git a/crypto/drbg.c b/crypto/drbg.c
index b6929eb5f565..2e4b2189636e 100644
--- a/crypto/drbg.c
+++ b/crypto/drbg.c
@@ -1206,19 +1206,19 @@ static inline void drbg_dealloc_state(struct drbg_state *drbg)
{
if (!drbg)
return;
- kzfree(drbg->Vbuf);
+ kfree_sensitive(drbg->Vbuf);
drbg->Vbuf = NULL;
drbg->V = NULL;
- kzfree(drbg->Cbuf);
+ kfree_sensitive(drbg->Cbuf);
drbg->Cbuf = NULL;
drbg->C = NULL;
- kzfree(drbg->scratchpadbuf);
+ kfree_sensitive(drbg->scratchpadbuf);
drbg->scratchpadbuf = NULL;
drbg->reseed_ctr = 0;
drbg->d_ops = NULL;
drbg->core = NULL;
if (IS_ENABLED(CONFIG_CRYPTO_FIPS)) {
- kzfree(drbg->prev);
+ kfree_sensitive(drbg->prev);
drbg->prev = NULL;
drbg->fips_primed = false;
}
@@ -1685,7 +1685,7 @@ static int drbg_fini_hash_kernel(struct drbg_state *drbg)
struct sdesc *sdesc = (struct sdesc *)drbg->priv_data;
if (sdesc) {
crypto_free_shash(sdesc->shash.tfm);
- kzfree(sdesc);
+ kfree_sensitive(sdesc);
}
drbg->priv_data = NULL;
return 0;
diff --git a/crypto/ecc.c b/crypto/ecc.c
index 02d35be7702b..37540332c1f3 100644
--- a/crypto/ecc.c
+++ b/crypto/ecc.c
@@ -67,7 +67,7 @@ static u64 *ecc_alloc_digits_space(unsigned int ndigits)
static void ecc_free_digits_space(u64 *space)
{
- kzfree(space);
+ kfree_sensitive(space);
}
static struct ecc_point *ecc_alloc_point(unsigned int ndigits)
@@ -101,9 +101,9 @@ static void ecc_free_point(struct ecc_point *p)
if (!p)
return;
- kzfree(p->x);
- kzfree(p->y);
- kzfree(p);
+ kfree_sensitive(p->x);
+ kfree_sensitive(p->y);
+ kfree_sensitive(p);
}
static void vli_clear(u64 *vli, unsigned int ndigits)
diff --git a/crypto/ecdh.c b/crypto/ecdh.c
index bd599053a8c4..b0232d6ab4ce 100644
--- a/crypto/ecdh.c
+++ b/crypto/ecdh.c
@@ -124,7 +124,7 @@ static int ecdh_compute_value(struct kpp_request *req)
/* fall through */
free_all:
- kzfree(shared_secret);
+ kfree_sensitive(shared_secret);
free_pubkey:
kfree(public_key);
return ret;
diff --git a/crypto/gcm.c b/crypto/gcm.c
index 0103d28c541e..5c2fbb08be56 100644
--- a/crypto/gcm.c
+++ b/crypto/gcm.c
@@ -139,7 +139,7 @@ static int crypto_gcm_setkey(struct crypto_aead *aead, const u8 *key,
CRYPTO_TFM_REQ_MASK);
err = crypto_ahash_setkey(ghash, (u8 *)&data->hash, sizeof(be128));
out:
- kzfree(data);
+ kfree_sensitive(data);
return err;
}
diff --git a/crypto/gf128mul.c b/crypto/gf128mul.c
index a4b1c026aaee..a69ae3e6c16c 100644
--- a/crypto/gf128mul.c
+++ b/crypto/gf128mul.c
@@ -304,8 +304,8 @@ void gf128mul_free_64k(struct gf128mul_64k *t)
int i;
for (i = 0; i < 16; i++)
- kzfree(t->t[i]);
- kzfree(t);
+ kfree_sensitive(t->t[i]);
+ kfree_sensitive(t);
}
EXPORT_SYMBOL(gf128mul_free_64k);
diff --git a/crypto/jitterentropy-kcapi.c b/crypto/jitterentropy-kcapi.c
index a5ce8f96790f..bc5f969360ae 100644
--- a/crypto/jitterentropy-kcapi.c
+++ b/crypto/jitterentropy-kcapi.c
@@ -57,7 +57,7 @@ void *jent_zalloc(unsigned int len)
void jent_zfree(void *ptr)
{
- kzfree(ptr);
+ kfree_sensitive(ptr);
}
int jent_fips_enabled(void)
diff --git a/crypto/rng.c b/crypto/rng.c
index 1490d210f1a1..a888d84b524a 100644
--- a/crypto/rng.c
+++ b/crypto/rng.c
@@ -53,7 +53,7 @@ int crypto_rng_reset(struct crypto_rng *tfm, const u8 *seed, unsigned int slen)
err = crypto_rng_alg(tfm)->seed(tfm, seed, slen);
crypto_stats_rng_seed(alg, err);
out:
- kzfree(buf);
+ kfree_sensitive(buf);
return err;
}
EXPORT_SYMBOL_GPL(crypto_rng_reset);
diff --git a/crypto/rsa-pkcs1pad.c b/crypto/rsa-pkcs1pad.c
index d31031de51bc..6c992eb5c72f 100644
--- a/crypto/rsa-pkcs1pad.c
+++ b/crypto/rsa-pkcs1pad.c
@@ -199,7 +199,7 @@ static int pkcs1pad_encrypt_sign_complete(struct akcipher_request *req, int err)
sg_copy_from_buffer(req->dst,
sg_nents_for_len(req->dst, ctx->key_size),
out_buf, ctx->key_size);
- kzfree(out_buf);
+ kfree_sensitive(out_buf);
out:
req->dst_len = ctx->key_size;
@@ -322,7 +322,7 @@ static int pkcs1pad_decrypt_complete(struct akcipher_request *req, int err)
out_buf + pos, req->dst_len);
done:
- kzfree(req_ctx->out_buf);
+ kfree_sensitive(req_ctx->out_buf);
return err;
}
@@ -500,7 +500,7 @@ static int pkcs1pad_verify_complete(struct akcipher_request *req, int err)
req->dst_len) != 0)
err = -EKEYREJECTED;
done:
- kzfree(req_ctx->out_buf);
+ kfree_sensitive(req_ctx->out_buf);
return err;
}
diff --git a/crypto/seqiv.c b/crypto/seqiv.c
index f124b9b54e15..27b2387bc972 100644
--- a/crypto/seqiv.c
+++ b/crypto/seqiv.c
@@ -33,7 +33,7 @@ static void seqiv_aead_encrypt_complete2(struct aead_request *req, int err)
memcpy(req->iv, subreq->iv, crypto_aead_ivsize(geniv));
out:
- kzfree(subreq->iv);
+ kfree_sensitive(subreq->iv);
}
static void seqiv_aead_encrypt_complete(struct crypto_async_request *base,
diff --git a/crypto/shash.c b/crypto/shash.c
index c075b26c2a1d..fcd8d8c5408b 100644
--- a/crypto/shash.c
+++ b/crypto/shash.c
@@ -44,7 +44,7 @@ static int shash_setkey_unaligned(struct crypto_shash *tfm, const u8 *key,
alignbuffer = (u8 *)ALIGN((unsigned long)buffer, alignmask + 1);
memcpy(alignbuffer, key, keylen);
err = shash->setkey(tfm, alignbuffer, keylen);
- kzfree(buffer);
+ kfree_sensitive(buffer);
return err;
}
diff --git a/crypto/skcipher.c b/crypto/skcipher.c
index 7221def7b9a7..1c4a0d2132c3 100644
--- a/crypto/skcipher.c
+++ b/crypto/skcipher.c
@@ -592,7 +592,7 @@ static int skcipher_setkey_unaligned(struct crypto_skcipher *tfm,
alignbuffer = (u8 *)ALIGN((unsigned long)buffer, alignmask + 1);
memcpy(alignbuffer, key, keylen);
ret = cipher->setkey(tfm, alignbuffer, keylen);
- kzfree(buffer);
+ kfree_sensitive(buffer);
return ret;
}
diff --git a/crypto/testmgr.c b/crypto/testmgr.c
index 6863f911fcee..23c27fc96394 100644
--- a/crypto/testmgr.c
+++ b/crypto/testmgr.c
@@ -1744,7 +1744,7 @@ static int test_hash_vs_generic_impl(const char *driver,
kfree(vec.plaintext);
kfree(vec.digest);
crypto_free_shash(generic_tfm);
- kzfree(generic_desc);
+ kfree_sensitive(generic_desc);
return err;
}
#else /* !CONFIG_CRYPTO_MANAGER_EXTRA_TESTS */
@@ -3665,7 +3665,7 @@ static int drbg_cavs_test(const struct drbg_testvec *test, int pr,
if (IS_ERR(drng)) {
printk(KERN_ERR "alg: drbg: could not allocate DRNG handle for "
"%s\n", driver);
- kzfree(buf);
+ kfree_sensitive(buf);
return -ENOMEM;
}
@@ -3712,7 +3712,7 @@ static int drbg_cavs_test(const struct drbg_testvec *test, int pr,
outbuf:
crypto_free_rng(drng);
- kzfree(buf);
+ kfree_sensitive(buf);
return ret;
}
diff --git a/crypto/zstd.c b/crypto/zstd.c
index 5a3ff258d8f7..1a3309f066f7 100644
--- a/crypto/zstd.c
+++ b/crypto/zstd.c
@@ -137,7 +137,7 @@ static void __zstd_exit(void *ctx)
static void zstd_free_ctx(struct crypto_scomp *tfm, void *ctx)
{
__zstd_exit(ctx);
- kzfree(ctx);
+ kfree_sensitive(ctx);
}
static void zstd_exit(struct crypto_tfm *tfm)
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
index a5fd8975f3d3..aa4e8fdc2b32 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
@@ -257,7 +257,7 @@ static int sun8i_ce_cipher(struct skcipher_request *areq)
offset = areq->cryptlen - ivsize;
if (rctx->op_dir & CE_DECRYPTION) {
memcpy(areq->iv, backup_iv, ivsize);
- kzfree(backup_iv);
+ kfree_sensitive(backup_iv);
} else {
scatterwalk_map_and_copy(areq->iv, areq->dst, offset,
ivsize, 0);
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
index 84d52fc3a2da..5246ef4f5430 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
@@ -250,7 +250,7 @@ static int sun8i_ss_cipher(struct skcipher_request *areq)
if (rctx->op_dir & SS_DECRYPTION) {
memcpy(areq->iv, backup_iv, ivsize);
memzero_explicit(backup_iv, ivsize);
- kzfree(backup_iv);
+ kfree_sensitive(backup_iv);
} else {
scatterwalk_map_and_copy(areq->iv, areq->dst, offset,
ivsize, 0);
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 9819dd50fbad..fd1269900d67 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -254,8 +254,8 @@ static int meson_cipher(struct skcipher_request *areq)
}
}
theend:
- kzfree(bkeyiv);
- kzfree(backup_iv);
+ kfree_sensitive(bkeyiv);
+ kfree_sensitive(backup_iv);
return err;
}
diff --git a/drivers/crypto/atmel-ecc.c b/drivers/crypto/atmel-ecc.c
index ff02cc05affb..9bd8e5167be3 100644
--- a/drivers/crypto/atmel-ecc.c
+++ b/drivers/crypto/atmel-ecc.c
@@ -69,7 +69,7 @@ static void atmel_ecdh_done(struct atmel_i2c_work_data *work_data, void *areq,
/* fall through */
free_work_data:
- kzfree(work_data);
+ kfree_sensitive(work_data);
kpp_request_complete(req, status);
}
diff --git a/drivers/crypto/caam/caampkc.c b/drivers/crypto/caam/caampkc.c
index 4fcae37a2e33..203bcd66f8b0 100644
--- a/drivers/crypto/caam/caampkc.c
+++ b/drivers/crypto/caam/caampkc.c
@@ -850,14 +850,14 @@ static int caam_rsa_dec(struct akcipher_request *req)
static void caam_rsa_free_key(struct caam_rsa_key *key)
{
- kzfree(key->d);
- kzfree(key->p);
- kzfree(key->q);
- kzfree(key->dp);
- kzfree(key->dq);
- kzfree(key->qinv);
- kzfree(key->tmp1);
- kzfree(key->tmp2);
+ kfree_sensitive(key->d);
+ kfree_sensitive(key->p);
+ kfree_sensitive(key->q);
+ kfree_sensitive(key->dp);
+ kfree_sensitive(key->dq);
+ kfree_sensitive(key->qinv);
+ kfree_sensitive(key->tmp1);
+ kfree_sensitive(key->tmp2);
kfree(key->e);
kfree(key->n);
memset(key, 0, sizeof(*key));
@@ -1014,17 +1014,17 @@ static void caam_rsa_set_priv_key_form(struct caam_rsa_ctx *ctx,
return;
free_dq:
- kzfree(rsa_key->dq);
+ kfree_sensitive(rsa_key->dq);
free_dp:
- kzfree(rsa_key->dp);
+ kfree_sensitive(rsa_key->dp);
free_tmp2:
- kzfree(rsa_key->tmp2);
+ kfree_sensitive(rsa_key->tmp2);
free_tmp1:
- kzfree(rsa_key->tmp1);
+ kfree_sensitive(rsa_key->tmp1);
free_q:
- kzfree(rsa_key->q);
+ kfree_sensitive(rsa_key->q);
free_p:
- kzfree(rsa_key->p);
+ kfree_sensitive(rsa_key->p);
}
static int caam_rsa_set_priv_key(struct crypto_akcipher *tfm, const void *key,
diff --git a/drivers/crypto/cavium/cpt/cptvf_main.c b/drivers/crypto/cavium/cpt/cptvf_main.c
index 0f72e9abdefe..a15245992cf9 100644
--- a/drivers/crypto/cavium/cpt/cptvf_main.c
+++ b/drivers/crypto/cavium/cpt/cptvf_main.c
@@ -74,7 +74,7 @@ static void cleanup_worker_threads(struct cpt_vf *cptvf)
for (i = 0; i < cptvf->nr_queues; i++)
tasklet_kill(&cwqe_info->vq_wqe[i].twork);
- kzfree(cwqe_info);
+ kfree_sensitive(cwqe_info);
cptvf->wqe_info = NULL;
}
@@ -88,7 +88,7 @@ static void free_pending_queues(struct pending_qinfo *pqinfo)
continue;
/* free single queue */
- kzfree((queue->head));
+ kfree_sensitive((queue->head));
queue->front = 0;
queue->rear = 0;
@@ -189,7 +189,7 @@ static void free_command_queues(struct cpt_vf *cptvf,
chunk->head = NULL;
chunk->dma_addr = 0;
hlist_del(&chunk->nextchunk);
- kzfree(chunk);
+ kfree_sensitive(chunk);
}
queue->nchunks = 0;
diff --git a/drivers/crypto/cavium/cpt/cptvf_reqmanager.c b/drivers/crypto/cavium/cpt/cptvf_reqmanager.c
index 7a24019356b5..472dbc2d7c5c 100644
--- a/drivers/crypto/cavium/cpt/cptvf_reqmanager.c
+++ b/drivers/crypto/cavium/cpt/cptvf_reqmanager.c
@@ -305,12 +305,12 @@ static void do_request_cleanup(struct cpt_vf *cptvf,
}
}
- kzfree(info->scatter_components);
- kzfree(info->gather_components);
- kzfree(info->out_buffer);
- kzfree(info->in_buffer);
- kzfree((void *)info->completion_addr);
- kzfree(info);
+ kfree_sensitive(info->scatter_components);
+ kfree_sensitive(info->gather_components);
+ kfree_sensitive(info->out_buffer);
+ kfree_sensitive(info->in_buffer);
+ kfree_sensitive((void *)info->completion_addr);
+ kfree_sensitive(info);
}
static void do_post_process(struct cpt_vf *cptvf, struct cpt_info_buffer *info)
diff --git a/drivers/crypto/cavium/nitrox/nitrox_lib.c b/drivers/crypto/cavium/nitrox/nitrox_lib.c
index 5cbc64b851b9..a5cdc2b48bd6 100644
--- a/drivers/crypto/cavium/nitrox/nitrox_lib.c
+++ b/drivers/crypto/cavium/nitrox/nitrox_lib.c
@@ -90,7 +90,7 @@ static void nitrox_free_aqm_queues(struct nitrox_device *ndev)
for (i = 0; i < ndev->nr_queues; i++) {
nitrox_cmdq_cleanup(ndev->aqmq[i]);
- kzfree(ndev->aqmq[i]);
+ kfree_sensitive(ndev->aqmq[i]);
ndev->aqmq[i] = NULL;
}
}
@@ -122,7 +122,7 @@ static int nitrox_alloc_aqm_queues(struct nitrox_device *ndev)
err = nitrox_cmdq_init(cmdq, AQM_Q_ALIGN_BYTES);
if (err) {
- kzfree(cmdq);
+ kfree_sensitive(cmdq);
goto aqmq_fail;
}
ndev->aqmq[i] = cmdq;
diff --git a/drivers/crypto/cavium/zip/zip_crypto.c b/drivers/crypto/cavium/zip/zip_crypto.c
index 4985bc812b0e..7df71fcebe8f 100644
--- a/drivers/crypto/cavium/zip/zip_crypto.c
+++ b/drivers/crypto/cavium/zip/zip_crypto.c
@@ -260,7 +260,7 @@ void *zip_alloc_scomp_ctx_deflate(struct crypto_scomp *tfm)
ret = zip_ctx_init(zip_ctx, 0);
if (ret) {
- kzfree(zip_ctx);
+ kfree_sensitive(zip_ctx);
return ERR_PTR(ret);
}
@@ -279,7 +279,7 @@ void *zip_alloc_scomp_ctx_lzs(struct crypto_scomp *tfm)
ret = zip_ctx_init(zip_ctx, 1);
if (ret) {
- kzfree(zip_ctx);
+ kfree_sensitive(zip_ctx);
return ERR_PTR(ret);
}
@@ -291,7 +291,7 @@ void zip_free_scomp_ctx(struct crypto_scomp *tfm, void *ctx)
struct zip_kernel_ctx *zip_ctx = ctx;
zip_ctx_exit(zip_ctx);
- kzfree(zip_ctx);
+ kfree_sensitive(zip_ctx);
}
int zip_scomp_compress(struct crypto_scomp *tfm,
diff --git a/drivers/crypto/ccp/ccp-crypto-rsa.c b/drivers/crypto/ccp/ccp-crypto-rsa.c
index 649c91d60401..1223ac70aea2 100644
--- a/drivers/crypto/ccp/ccp-crypto-rsa.c
+++ b/drivers/crypto/ccp/ccp-crypto-rsa.c
@@ -112,13 +112,13 @@ static int ccp_check_key_length(unsigned int len)
static void ccp_rsa_free_key_bufs(struct ccp_ctx *ctx)
{
/* Clean up old key data */
- kzfree(ctx->u.rsa.e_buf);
+ kfree_sensitive(ctx->u.rsa.e_buf);
ctx->u.rsa.e_buf = NULL;
ctx->u.rsa.e_len = 0;
- kzfree(ctx->u.rsa.n_buf);
+ kfree_sensitive(ctx->u.rsa.n_buf);
ctx->u.rsa.n_buf = NULL;
ctx->u.rsa.n_len = 0;
- kzfree(ctx->u.rsa.d_buf);
+ kfree_sensitive(ctx->u.rsa.d_buf);
ctx->u.rsa.d_buf = NULL;
ctx->u.rsa.d_len = 0;
}
diff --git a/drivers/crypto/ccree/cc_aead.c b/drivers/crypto/ccree/cc_aead.c
index 1cf51edbc4b9..35794c7271fb 100644
--- a/drivers/crypto/ccree/cc_aead.c
+++ b/drivers/crypto/ccree/cc_aead.c
@@ -448,7 +448,7 @@ static int cc_get_plain_hmac_key(struct crypto_aead *tfm, const u8 *authkey,
if (dma_mapping_error(dev, key_dma_addr)) {
dev_err(dev, "Mapping key va=0x%p len=%u for DMA failed\n",
key, keylen);
- kzfree(key);
+ kfree_sensitive(key);
return -ENOMEM;
}
if (keylen > blocksize) {
@@ -533,7 +533,7 @@ static int cc_get_plain_hmac_key(struct crypto_aead *tfm, const u8 *authkey,
if (key_dma_addr)
dma_unmap_single(dev, key_dma_addr, keylen, DMA_TO_DEVICE);
- kzfree(key);
+ kfree_sensitive(key);
return rc;
}
diff --git a/drivers/crypto/ccree/cc_buffer_mgr.c b/drivers/crypto/ccree/cc_buffer_mgr.c
index b2bd093e7013..a5e041d9d2cf 100644
--- a/drivers/crypto/ccree/cc_buffer_mgr.c
+++ b/drivers/crypto/ccree/cc_buffer_mgr.c
@@ -488,7 +488,7 @@ void cc_unmap_aead_request(struct device *dev, struct aead_request *req)
if (areq_ctx->gen_ctx.iv_dma_addr) {
dma_unmap_single(dev, areq_ctx->gen_ctx.iv_dma_addr,
hw_iv_size, DMA_BIDIRECTIONAL);
- kzfree(areq_ctx->gen_ctx.iv);
+ kfree_sensitive(areq_ctx->gen_ctx.iv);
}
/* Release pool */
@@ -559,7 +559,7 @@ static int cc_aead_chain_iv(struct cc_drvdata *drvdata,
if (dma_mapping_error(dev, areq_ctx->gen_ctx.iv_dma_addr)) {
dev_err(dev, "Mapping iv %u B at va=%pK for DMA failed\n",
hw_iv_size, req->iv);
- kzfree(areq_ctx->gen_ctx.iv);
+ kfree_sensitive(areq_ctx->gen_ctx.iv);
areq_ctx->gen_ctx.iv = NULL;
rc = -ENOMEM;
goto chain_iv_exit;
diff --git a/drivers/crypto/ccree/cc_cipher.c b/drivers/crypto/ccree/cc_cipher.c
index a84335328f37..f380a4b5e32b 100644
--- a/drivers/crypto/ccree/cc_cipher.c
+++ b/drivers/crypto/ccree/cc_cipher.c
@@ -229,7 +229,7 @@ static void cc_cipher_exit(struct crypto_tfm *tfm)
&ctx_p->user.key_dma_addr);
/* Free key buffer in context */
- kzfree(ctx_p->user.key);
+ kfree_sensitive(ctx_p->user.key);
dev_dbg(dev, "Free key buffer in context. key=@%p\n", ctx_p->user.key);
}
@@ -828,7 +828,7 @@ static void cc_cipher_complete(struct device *dev, void *cc_req, int err)
/* Not a BACKLOG notification */
cc_unmap_cipher_request(dev, req_ctx, ivsize, src, dst);
memcpy(req->iv, req_ctx->iv, ivsize);
- kzfree(req_ctx->iv);
+ kfree_sensitive(req_ctx->iv);
}
skcipher_request_complete(req, err);
@@ -930,7 +930,7 @@ static int cc_cipher_process(struct skcipher_request *req,
exit_process:
if (rc != -EINPROGRESS && rc != -EBUSY) {
- kzfree(req_ctx->iv);
+ kfree_sensitive(req_ctx->iv);
}
return rc;
diff --git a/drivers/crypto/ccree/cc_hash.c b/drivers/crypto/ccree/cc_hash.c
index d5310783af15..683c9a430e11 100644
--- a/drivers/crypto/ccree/cc_hash.c
+++ b/drivers/crypto/ccree/cc_hash.c
@@ -764,7 +764,7 @@ static int cc_hash_setkey(struct crypto_ahash *ahash, const u8 *key,
if (dma_mapping_error(dev, ctx->key_params.key_dma_addr)) {
dev_err(dev, "Mapping key va=0x%p len=%u for DMA failed\n",
ctx->key_params.key, keylen);
- kzfree(ctx->key_params.key);
+ kfree_sensitive(ctx->key_params.key);
return -ENOMEM;
}
dev_dbg(dev, "mapping key-buffer: key_dma_addr=%pad keylen=%u\n",
@@ -913,7 +913,7 @@ static int cc_hash_setkey(struct crypto_ahash *ahash, const u8 *key,
&ctx->key_params.key_dma_addr, ctx->key_params.keylen);
}
- kzfree(ctx->key_params.key);
+ kfree_sensitive(ctx->key_params.key);
return rc;
}
@@ -950,7 +950,7 @@ static int cc_xcbc_setkey(struct crypto_ahash *ahash,
if (dma_mapping_error(dev, ctx->key_params.key_dma_addr)) {
dev_err(dev, "Mapping key va=0x%p len=%u for DMA failed\n",
key, keylen);
- kzfree(ctx->key_params.key);
+ kfree_sensitive(ctx->key_params.key);
return -ENOMEM;
}
dev_dbg(dev, "mapping key-buffer: key_dma_addr=%pad keylen=%u\n",
@@ -999,7 +999,7 @@ static int cc_xcbc_setkey(struct crypto_ahash *ahash,
dev_dbg(dev, "Unmapped key-buffer: key_dma_addr=%pad keylen=%u\n",
&ctx->key_params.key_dma_addr, ctx->key_params.keylen);
- kzfree(ctx->key_params.key);
+ kfree_sensitive(ctx->key_params.key);
return rc;
}
diff --git a/drivers/crypto/ccree/cc_request_mgr.c b/drivers/crypto/ccree/cc_request_mgr.c
index 1d7649ecf44e..33fb27745d52 100644
--- a/drivers/crypto/ccree/cc_request_mgr.c
+++ b/drivers/crypto/ccree/cc_request_mgr.c
@@ -107,7 +107,7 @@ void cc_req_mgr_fini(struct cc_drvdata *drvdata)
/* Kill tasklet */
tasklet_kill(&req_mgr_h->comptask);
#endif
- kzfree(req_mgr_h);
+ kfree_sensitive(req_mgr_h);
drvdata->request_mgr_handle = NULL;
}
diff --git a/drivers/crypto/marvell/cesa/hash.c b/drivers/crypto/marvell/cesa/hash.c
index b971284332b6..2fdd3d55ed08 100644
--- a/drivers/crypto/marvell/cesa/hash.c
+++ b/drivers/crypto/marvell/cesa/hash.c
@@ -1154,7 +1154,7 @@ static int mv_cesa_ahmac_pad_init(struct ahash_request *req,
}
/* Set the memory region to 0 to avoid any leak. */
- kzfree(keydup);
+ kfree_sensitive(keydup);
if (ret)
return ret;
diff --git a/drivers/crypto/marvell/octeontx/otx_cptvf_main.c b/drivers/crypto/marvell/octeontx/otx_cptvf_main.c
index a91860b5dc77..ffd33121bb55 100644
--- a/drivers/crypto/marvell/octeontx/otx_cptvf_main.c
+++ b/drivers/crypto/marvell/octeontx/otx_cptvf_main.c
@@ -68,7 +68,7 @@ static void cleanup_worker_threads(struct otx_cptvf *cptvf)
for (i = 0; i < cptvf->num_queues; i++)
tasklet_kill(&cwqe_info->vq_wqe[i].twork);
- kzfree(cwqe_info);
+ kfree_sensitive(cwqe_info);
cptvf->wqe_info = NULL;
}
@@ -82,7 +82,7 @@ static void free_pending_queues(struct otx_cpt_pending_qinfo *pqinfo)
continue;
/* free single queue */
- kzfree((queue->head));
+ kfree_sensitive((queue->head));
queue->front = 0;
queue->rear = 0;
queue->qlen = 0;
@@ -176,7 +176,7 @@ static void free_command_queues(struct otx_cptvf *cptvf,
chunk->head = NULL;
chunk->dma_addr = 0;
list_del(&chunk->nextchunk);
- kzfree(chunk);
+ kfree_sensitive(chunk);
}
queue->num_chunks = 0;
queue->idx = 0;
diff --git a/drivers/crypto/marvell/octeontx/otx_cptvf_reqmgr.h b/drivers/crypto/marvell/octeontx/otx_cptvf_reqmgr.h
index a4c9ff730b13..cfaaf8e2f9c2 100644
--- a/drivers/crypto/marvell/octeontx/otx_cptvf_reqmgr.h
+++ b/drivers/crypto/marvell/octeontx/otx_cptvf_reqmgr.h
@@ -215,7 +215,7 @@ static inline void do_request_cleanup(struct pci_dev *pdev,
DMA_BIDIRECTIONAL);
}
}
- kzfree(info);
+ kfree_sensitive(info);
}
struct otx_cptvf_wqe;
diff --git a/drivers/crypto/mediatek/mtk-aes.c b/drivers/crypto/mediatek/mtk-aes.c
index 78d660d963e2..5c71f85da7e2 100644
--- a/drivers/crypto/mediatek/mtk-aes.c
+++ b/drivers/crypto/mediatek/mtk-aes.c
@@ -1057,7 +1057,7 @@ static int mtk_aes_gcm_setkey(struct crypto_aead *aead, const u8 *key,
mtk_aes_write_state_be(ctx->key + ctx->keylen, data->hash,
AES_BLOCK_SIZE);
out:
- kzfree(data);
+ kfree_sensitive(data);
return err;
}
diff --git a/drivers/crypto/nx/nx.c b/drivers/crypto/nx/nx.c
index f03c238f5a31..40882d6d52c1 100644
--- a/drivers/crypto/nx/nx.c
+++ b/drivers/crypto/nx/nx.c
@@ -746,7 +746,7 @@ void nx_crypto_ctx_exit(struct crypto_tfm *tfm)
{
struct nx_crypto_ctx *nx_ctx = crypto_tfm_ctx(tfm);
- kzfree(nx_ctx->kmem);
+ kfree_sensitive(nx_ctx->kmem);
nx_ctx->csbcpb = NULL;
nx_ctx->csbcpb_aead = NULL;
nx_ctx->in_sg = NULL;
@@ -762,7 +762,7 @@ void nx_crypto_ctx_aead_exit(struct crypto_aead *tfm)
{
struct nx_crypto_ctx *nx_ctx = crypto_aead_ctx(tfm);
- kzfree(nx_ctx->kmem);
+ kfree_sensitive(nx_ctx->kmem);
}
static int nx_probe(struct vio_dev *viodev, const struct vio_device_id *id)
diff --git a/drivers/crypto/virtio/virtio_crypto_algs.c b/drivers/crypto/virtio/virtio_crypto_algs.c
index fd045e64972a..bc575d63dee6 100644
--- a/drivers/crypto/virtio/virtio_crypto_algs.c
+++ b/drivers/crypto/virtio/virtio_crypto_algs.c
@@ -167,7 +167,7 @@ static int virtio_crypto_alg_skcipher_init_session(
num_in, vcrypto, GFP_ATOMIC);
if (err < 0) {
spin_unlock(&vcrypto->ctrl_lock);
- kzfree(cipher_key);
+ kfree_sensitive(cipher_key);
return err;
}
virtqueue_kick(vcrypto->ctrl_vq);
@@ -184,7 +184,7 @@ static int virtio_crypto_alg_skcipher_init_session(
spin_unlock(&vcrypto->ctrl_lock);
pr_err("virtio_crypto: Create session failed status: %u\n",
le32_to_cpu(vcrypto->input.status));
- kzfree(cipher_key);
+ kfree_sensitive(cipher_key);
return -EINVAL;
}
@@ -197,7 +197,7 @@ static int virtio_crypto_alg_skcipher_init_session(
spin_unlock(&vcrypto->ctrl_lock);
- kzfree(cipher_key);
+ kfree_sensitive(cipher_key);
return 0;
}
@@ -466,9 +466,9 @@ __virtio_crypto_skcipher_do_req(struct virtio_crypto_sym_request *vc_sym_req,
return 0;
free_iv:
- kzfree(iv);
+ kfree_sensitive(iv);
free:
- kzfree(req_data);
+ kfree_sensitive(req_data);
kfree(sgs);
return err;
}
@@ -579,7 +579,7 @@ static void virtio_crypto_skcipher_finalize_req(
AES_BLOCK_SIZE, 0);
crypto_finalize_skcipher_request(vc_sym_req->base.dataq->engine,
req, err);
- kzfree(vc_sym_req->iv);
+ kfree_sensitive(vc_sym_req->iv);
virtcrypto_clear_request(&vc_sym_req->base);
}
diff --git a/drivers/crypto/virtio/virtio_crypto_core.c b/drivers/crypto/virtio/virtio_crypto_core.c
index c8a962c62663..ba8a19c72391 100644
--- a/drivers/crypto/virtio/virtio_crypto_core.c
+++ b/drivers/crypto/virtio/virtio_crypto_core.c
@@ -17,7 +17,7 @@ void
virtcrypto_clear_request(struct virtio_crypto_request *vc_req)
{
if (vc_req) {
- kzfree(vc_req->req_data);
+ kfree_sensitive(vc_req->req_data);
kfree(vc_req->sgs);
}
}
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 3df90daba89e..2423eade2ae6 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -405,7 +405,7 @@ static void crypt_iv_lmk_dtr(struct crypt_config *cc)
crypto_free_shash(lmk->hash_tfm);
lmk->hash_tfm = NULL;
- kzfree(lmk->seed);
+ kfree_sensitive(lmk->seed);
lmk->seed = NULL;
}
@@ -556,9 +556,9 @@ static void crypt_iv_tcw_dtr(struct crypt_config *cc)
{
struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
- kzfree(tcw->iv_seed);
+ kfree_sensitive(tcw->iv_seed);
tcw->iv_seed = NULL;
- kzfree(tcw->whitening);
+ kfree_sensitive(tcw->whitening);
tcw->whitening = NULL;
if (tcw->crc32_tfm && !IS_ERR(tcw->crc32_tfm))
@@ -992,8 +992,8 @@ static int crypt_iv_elephant(struct crypt_config *cc, struct dm_crypt_request *d
kunmap_atomic(data);
out:
- kzfree(ks);
- kzfree(es);
+ kfree_sensitive(ks);
+ kfree_sensitive(es);
skcipher_request_free(req);
return r;
}
@@ -2247,7 +2247,7 @@ static int crypt_set_keyring_key(struct crypt_config *cc, const char *key_string
key = request_key(key_string[0] == 'l' ? &key_type_logon : &key_type_user,
key_desc + 1, NULL);
if (IS_ERR(key)) {
- kzfree(new_key_string);
+ kfree_sensitive(new_key_string);
return PTR_ERR(key);
}
@@ -2257,14 +2257,14 @@ static int crypt_set_keyring_key(struct crypt_config *cc, const char *key_string
if (!ukp) {
up_read(&key->sem);
key_put(key);
- kzfree(new_key_string);
+ kfree_sensitive(new_key_string);
return -EKEYREVOKED;
}
if (cc->key_size != ukp->datalen) {
up_read(&key->sem);
key_put(key);
- kzfree(new_key_string);
+ kfree_sensitive(new_key_string);
return -EINVAL;
}
@@ -2280,10 +2280,10 @@ static int crypt_set_keyring_key(struct crypt_config *cc, const char *key_string
if (!ret) {
set_bit(DM_CRYPT_KEY_VALID, &cc->flags);
- kzfree(cc->key_string);
+ kfree_sensitive(cc->key_string);
cc->key_string = new_key_string;
} else
- kzfree(new_key_string);
+ kfree_sensitive(new_key_string);
return ret;
}
@@ -2344,7 +2344,7 @@ static int crypt_set_key(struct crypt_config *cc, char *key)
clear_bit(DM_CRYPT_KEY_VALID, &cc->flags);
/* wipe references to any kernel keyring key */
- kzfree(cc->key_string);
+ kfree_sensitive(cc->key_string);
cc->key_string = NULL;
/* Decode key from its hex representation. */
@@ -2376,7 +2376,7 @@ static int crypt_wipe_key(struct crypt_config *cc)
return r;
}
- kzfree(cc->key_string);
+ kfree_sensitive(cc->key_string);
cc->key_string = NULL;
r = crypt_setkey(cc);
memset(&cc->key, 0, cc->key_size * sizeof(u8));
@@ -2455,15 +2455,15 @@ static void crypt_dtr(struct dm_target *ti)
if (cc->dev)
dm_put_device(ti, cc->dev);
- kzfree(cc->cipher_string);
- kzfree(cc->key_string);
- kzfree(cc->cipher_auth);
- kzfree(cc->authenc_key);
+ kfree_sensitive(cc->cipher_string);
+ kfree_sensitive(cc->key_string);
+ kfree_sensitive(cc->cipher_auth);
+ kfree_sensitive(cc->authenc_key);
mutex_destroy(&cc->bio_alloc_lock);
/* Must zero key material before freeing */
- kzfree(cc);
+ kfree_sensitive(cc);
spin_lock(&dm_crypt_clients_lock);
WARN_ON(!dm_crypt_clients_n);
diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c
index 4094c47eca7f..d08ef3c0d148 100644
--- a/drivers/md/dm-integrity.c
+++ b/drivers/md/dm-integrity.c
@@ -3409,8 +3409,8 @@ static struct scatterlist **dm_integrity_alloc_journal_scatterlist(struct dm_int
static void free_alg(struct alg_spec *a)
{
- kzfree(a->alg_string);
- kzfree(a->key);
+ kfree_sensitive(a->alg_string);
+ kfree_sensitive(a->key);
memset(a, 0, sizeof *a);
}
@@ -4341,7 +4341,7 @@ static void dm_integrity_dtr(struct dm_target *ti)
for (i = 0; i < ic->journal_sections; i++) {
struct skcipher_request *req = ic->sk_requests[i];
if (req) {
- kzfree(req->iv);
+ kfree_sensitive(req->iv);
skcipher_request_free(req);
}
}
diff --git a/drivers/misc/ibmvmc.c b/drivers/misc/ibmvmc.c
index 2ed23c99f59f..beda69075a97 100644
--- a/drivers/misc/ibmvmc.c
+++ b/drivers/misc/ibmvmc.c
@@ -286,7 +286,7 @@ static void *alloc_dma_buffer(struct vio_dev *vdev, size_t size,
if (dma_mapping_error(&vdev->dev, *dma_handle)) {
*dma_handle = 0;
- kzfree(buffer);
+ kfree_sensitive(buffer);
return NULL;
}
@@ -310,7 +310,7 @@ static void free_dma_buffer(struct vio_dev *vdev, size_t size, void *vaddr,
dma_unmap_single(&vdev->dev, dma_handle, size, DMA_BIDIRECTIONAL);
/* deallocate memory */
- kzfree(vaddr);
+ kfree_sensitive(vaddr);
}
/**
@@ -883,7 +883,7 @@ static int ibmvmc_close(struct inode *inode, struct file *file)
spin_unlock_irqrestore(&hmc->lock, flags);
}
- kzfree(session);
+ kfree_sensitive(session);
return rc;
}
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c
index 7f24fcb4f96a..965ada2e9d24 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c
@@ -132,7 +132,7 @@ static void hclge_free_vector_ring_chain(struct hnae3_ring_chain_node *head)
while (chain) {
chain_tmp = chain->next;
- kzfree(chain);
+ kfree_sensitive(chain);
chain = chain_tmp;
}
}
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c
index 113f6087c7c9..e567f4ab8a79 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c
@@ -960,9 +960,9 @@ int ixgbe_ipsec_vf_add_sa(struct ixgbe_adapter *adapter, u32 *msgbuf, u32 vf)
return 0;
err_aead:
- kzfree(xs->aead);
+ kfree_sensitive(xs->aead);
err_xs:
- kzfree(xs);
+ kfree_sensitive(xs);
err_out:
msgbuf[1] = err;
return err;
@@ -1047,7 +1047,7 @@ int ixgbe_ipsec_vf_del_sa(struct ixgbe_adapter *adapter, u32 *msgbuf, u32 vf)
ixgbe_ipsec_del_sa(xs);
/* remove the xs that was made-up in the add request */
- kzfree(xs);
+ kfree_sensitive(xs);
return 0;
}
diff --git a/drivers/net/ppp/ppp_mppe.c b/drivers/net/ppp/ppp_mppe.c
index de3b57d09d0c..208f6e24f37c 100644
--- a/drivers/net/ppp/ppp_mppe.c
+++ b/drivers/net/ppp/ppp_mppe.c
@@ -222,7 +222,7 @@ static void *mppe_alloc(unsigned char *options, int optlen)
kfree(state->sha1_digest);
if (state->sha1) {
crypto_free_shash(state->sha1->tfm);
- kzfree(state->sha1);
+ kfree_sensitive(state->sha1);
}
kfree(state);
out:
@@ -238,8 +238,8 @@ static void mppe_free(void *arg)
if (state) {
kfree(state->sha1_digest);
crypto_free_shash(state->sha1->tfm);
- kzfree(state->sha1);
- kzfree(state);
+ kfree_sensitive(state->sha1);
+ kfree_sensitive(state);
}
}
diff --git a/drivers/net/wireguard/noise.c b/drivers/net/wireguard/noise.c
index 708dc61c974f..596eb3be2d9e 100644
--- a/drivers/net/wireguard/noise.c
+++ b/drivers/net/wireguard/noise.c
@@ -113,7 +113,7 @@ static struct noise_keypair *keypair_create(struct wg_peer *peer)
static void keypair_free_rcu(struct rcu_head *rcu)
{
- kzfree(container_of(rcu, struct noise_keypair, rcu));
+ kfree_sensitive(container_of(rcu, struct noise_keypair, rcu));
}
static void keypair_free_kref(struct kref *kref)
@@ -825,7 +825,7 @@ bool wg_noise_handshake_begin_session(struct noise_handshake *handshake,
handshake->entry.peer->device->index_hashtable,
&handshake->entry, &new_keypair->entry);
} else {
- kzfree(new_keypair);
+ kfree_sensitive(new_keypair);
}
rcu_read_unlock_bh();
diff --git a/drivers/net/wireguard/peer.c b/drivers/net/wireguard/peer.c
index 1d634bd3038f..b3b6370e6b95 100644
--- a/drivers/net/wireguard/peer.c
+++ b/drivers/net/wireguard/peer.c
@@ -203,7 +203,7 @@ static void rcu_release(struct rcu_head *rcu)
/* The final zeroing takes care of clearing any remaining handshake key
* material and other potentially sensitive information.
*/
- kzfree(peer);
+ kfree_sensitive(peer);
}
static void kref_release(struct kref *refcount)
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/rx.c b/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
index 8c29071cb415..82b449c249a4 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/rx.c
@@ -1369,7 +1369,7 @@ static void iwl_pcie_rx_handle_rb(struct iwl_trans *trans,
&rxcb, rxq->id);
if (reclaim) {
- kzfree(txq->entries[cmd_index].free_buf);
+ kfree_sensitive(txq->entries[cmd_index].free_buf);
txq->entries[cmd_index].free_buf = NULL;
}
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c b/drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c
index 86fc00167817..022f7a069295 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c
@@ -1024,7 +1024,7 @@ static int iwl_pcie_gen2_enqueue_hcmd(struct iwl_trans *trans,
BUILD_BUG_ON(IWL_TFH_NUM_TBS > sizeof(out_meta->tbs) * BITS_PER_BYTE);
out_meta->flags = cmd->flags;
if (WARN_ON_ONCE(txq->entries[idx].free_buf))
- kzfree(txq->entries[idx].free_buf);
+ kfree_sensitive(txq->entries[idx].free_buf);
txq->entries[idx].free_buf = dup_buf;
trace_iwlwifi_dev_hcmd(trans->dev, cmd, cmd_size, &out_cmd->hdr_wide);
@@ -1254,8 +1254,8 @@ static void iwl_pcie_gen2_txq_free(struct iwl_trans *trans, int txq_id)
/* De-alloc array of command/tx buffers */
if (txq_id == trans_pcie->cmd_queue)
for (i = 0; i < txq->n_window; i++) {
- kzfree(txq->entries[i].cmd);
- kzfree(txq->entries[i].free_buf);
+ kfree_sensitive(txq->entries[i].cmd);
+ kfree_sensitive(txq->entries[i].free_buf);
}
del_timer_sync(&txq->stuck_timer);
diff --git a/drivers/net/wireless/intel/iwlwifi/pcie/tx.c b/drivers/net/wireless/intel/iwlwifi/pcie/tx.c
index 4582d418ba4d..aa8277df2f17 100644
--- a/drivers/net/wireless/intel/iwlwifi/pcie/tx.c
+++ b/drivers/net/wireless/intel/iwlwifi/pcie/tx.c
@@ -724,8 +724,8 @@ static void iwl_pcie_txq_free(struct iwl_trans *trans, int txq_id)
/* De-alloc array of command/tx buffers */
if (txq_id == trans_pcie->cmd_queue)
for (i = 0; i < txq->n_window; i++) {
- kzfree(txq->entries[i].cmd);
- kzfree(txq->entries[i].free_buf);
+ kfree_sensitive(txq->entries[i].cmd);
+ kfree_sensitive(txq->entries[i].free_buf);
}
/* De-alloc circular buffer of TFDs */
@@ -1768,7 +1768,7 @@ static int iwl_pcie_enqueue_hcmd(struct iwl_trans *trans,
BUILD_BUG_ON(IWL_TFH_NUM_TBS > sizeof(out_meta->tbs) * BITS_PER_BYTE);
out_meta->flags = cmd->flags;
if (WARN_ON_ONCE(txq->entries[idx].free_buf))
- kzfree(txq->entries[idx].free_buf);
+ kfree_sensitive(txq->entries[idx].free_buf);
txq->entries[idx].free_buf = dup_buf;
trace_iwlwifi_dev_hcmd(trans->dev, cmd, cmd_size, &out_cmd->hdr_wide);
diff --git a/drivers/net/wireless/intersil/orinoco/wext.c b/drivers/net/wireless/intersil/orinoco/wext.c
index 1d4dae422106..7b6c4ae8ddb3 100644
--- a/drivers/net/wireless/intersil/orinoco/wext.c
+++ b/drivers/net/wireless/intersil/orinoco/wext.c
@@ -31,8 +31,8 @@ static int orinoco_set_key(struct orinoco_private *priv, int index,
enum orinoco_alg alg, const u8 *key, int key_len,
const u8 *seq, int seq_len)
{
- kzfree(priv->keys[index].key);
- kzfree(priv->keys[index].seq);
+ kfree_sensitive(priv->keys[index].key);
+ kfree_sensitive(priv->keys[index].seq);
if (key_len) {
priv->keys[index].key = kzalloc(key_len, GFP_ATOMIC);
diff --git a/drivers/s390/crypto/ap_bus.h b/drivers/s390/crypto/ap_bus.h
index 8e8e37b6c0ee..99125e29b2c6 100644
--- a/drivers/s390/crypto/ap_bus.h
+++ b/drivers/s390/crypto/ap_bus.h
@@ -219,8 +219,8 @@ static inline void ap_init_message(struct ap_message *ap_msg)
*/
static inline void ap_release_message(struct ap_message *ap_msg)
{
- kzfree(ap_msg->message);
- kzfree(ap_msg->private);
+ kfree_sensitive(ap_msg->message);
+ kfree_sensitive(ap_msg->private);
}
#define for_each_ap_card(_ac) \
diff --git a/drivers/staging/ks7010/ks_hostif.c b/drivers/staging/ks7010/ks_hostif.c
index 2666f9e30c15..d70b671b06aa 100644
--- a/drivers/staging/ks7010/ks_hostif.c
+++ b/drivers/staging/ks7010/ks_hostif.c
@@ -246,7 +246,7 @@ michael_mic(u8 *key, u8 *data, unsigned int len, u8 priority, u8 *result)
ret = crypto_shash_finup(desc, data + 12, len - 12, result);
err_free_desc:
- kzfree(desc);
+ kfree_sensitive(desc);
err_free_tfm:
crypto_free_shash(tfm);
diff --git a/drivers/staging/rtl8723bs/core/rtw_security.c b/drivers/staging/rtl8723bs/core/rtw_security.c
index 5ebf691bd743..9d7eb027a213 100644
--- a/drivers/staging/rtl8723bs/core/rtw_security.c
+++ b/drivers/staging/rtl8723bs/core/rtw_security.c
@@ -2251,7 +2251,7 @@ static void gf_mulx(u8 *pad)
static void aes_encrypt_deinit(void *ctx)
{
- kzfree(ctx);
+ kfree_sensitive(ctx);
}
diff --git a/drivers/staging/wlan-ng/p80211netdev.c b/drivers/staging/wlan-ng/p80211netdev.c
index b809c0015c0c..7b091c5a2984 100644
--- a/drivers/staging/wlan-ng/p80211netdev.c
+++ b/drivers/staging/wlan-ng/p80211netdev.c
@@ -429,7 +429,7 @@ static netdev_tx_t p80211knetdev_hard_start_xmit(struct sk_buff *skb,
failed:
/* Free up the WEP buffer if it's not the same as the skb */
if ((p80211_wep.data) && (p80211_wep.data != skb->data))
- kzfree(p80211_wep.data);
+ kfree_sensitive(p80211_wep.data);
/* we always free the skb here, never in a lower level. */
if (!result)
diff --git a/drivers/target/iscsi/iscsi_target_auth.c b/drivers/target/iscsi/iscsi_target_auth.c
index 0e54627d9aa8..62d912b79c61 100644
--- a/drivers/target/iscsi/iscsi_target_auth.c
+++ b/drivers/target/iscsi/iscsi_target_auth.c
@@ -484,7 +484,7 @@ static int chap_server_compute_hash(
pr_debug("[server] Sending CHAP_R=0x%s\n", response);
auth_ret = 0;
out:
- kzfree(desc);
+ kfree_sensitive(desc);
if (tfm)
crypto_free_shash(tfm);
kfree(initiatorchg);
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 40b729dce91c..eab3f8510426 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -2691,7 +2691,7 @@ static int btrfs_ioctl_get_subvol_info(struct file *file, void __user *argp)
btrfs_put_root(root);
out_free:
btrfs_free_path(path);
- kzfree(subvol_info);
+ kfree_sensitive(subvol_info);
return ret;
}
diff --git a/fs/cifs/cifsencrypt.c b/fs/cifs/cifsencrypt.c
index 97b7497c13ef..c6baca36b74c 100644
--- a/fs/cifs/cifsencrypt.c
+++ b/fs/cifs/cifsencrypt.c
@@ -797,7 +797,7 @@ calc_seckey(struct cifs_ses *ses)
ses->auth_key.len = CIFS_SESS_KEY_SIZE;
memzero_explicit(sec_key, CIFS_SESS_KEY_SIZE);
- kzfree(ctx_arc4);
+ kfree_sensitive(ctx_arc4);
return 0;
}
diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c
index 95b3ab0ca8c0..cdeac9d66e26 100644
--- a/fs/cifs/connect.c
+++ b/fs/cifs/connect.c
@@ -2177,7 +2177,7 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
tmp_end++;
if (!(tmp_end < end && tmp_end[1] == delim)) {
/* No it is not. Set the password to NULL */
- kzfree(vol->password);
+ kfree_sensitive(vol->password);
vol->password = NULL;
break;
}
@@ -2215,7 +2215,7 @@ cifs_parse_mount_options(const char *mountdata, const char *devname,
options = end;
}
- kzfree(vol->password);
+ kfree_sensitive(vol->password);
/* Now build new password string */
temp_len = strlen(value);
vol->password = kzalloc(temp_len+1, GFP_KERNEL);
@@ -3200,7 +3200,7 @@ cifs_set_cifscreds(struct smb_vol *vol, struct cifs_ses *ses)
rc = -ENOMEM;
kfree(vol->username);
vol->username = NULL;
- kzfree(vol->password);
+ kfree_sensitive(vol->password);
vol->password = NULL;
goto out_key_put;
}
@@ -4212,7 +4212,7 @@ void
cifs_cleanup_volume_info_contents(struct smb_vol *volume_info)
{
kfree(volume_info->username);
- kzfree(volume_info->password);
+ kfree_sensitive(volume_info->password);
kfree(volume_info->UNC);
kfree(volume_info->domainname);
kfree(volume_info->iocharset);
@@ -5337,7 +5337,7 @@ cifs_construct_tcon(struct cifs_sb_info *cifs_sb, kuid_t fsuid)
out:
kfree(vol_info->username);
- kzfree(vol_info->password);
+ kfree_sensitive(vol_info->password);
kfree(vol_info);
return tcon;
diff --git a/fs/cifs/dfs_cache.c b/fs/cifs/dfs_cache.c
index a67f88bf7ae1..a0223b47dc1b 100644
--- a/fs/cifs/dfs_cache.c
+++ b/fs/cifs/dfs_cache.c
@@ -1131,7 +1131,7 @@ static int dup_vol(struct smb_vol *vol, struct smb_vol *new)
err_free_unc:
kfree(new->UNC);
err_free_password:
- kzfree(new->password);
+ kfree_sensitive(new->password);
err_free_username:
kfree(new->username);
kfree(new);
diff --git a/fs/cifs/misc.c b/fs/cifs/misc.c
index a456febd4109..503810e9cb76 100644
--- a/fs/cifs/misc.c
+++ b/fs/cifs/misc.c
@@ -100,12 +100,12 @@ sesInfoFree(struct cifs_ses *buf_to_free)
kfree(buf_to_free->serverOS);
kfree(buf_to_free->serverDomain);
kfree(buf_to_free->serverNOS);
- kzfree(buf_to_free->password);
+ kfree_sensitive(buf_to_free->password);
kfree(buf_to_free->user_name);
kfree(buf_to_free->domainName);
- kzfree(buf_to_free->auth_key.response);
+ kfree_sensitive(buf_to_free->auth_key.response);
kfree(buf_to_free->iface_list);
- kzfree(buf_to_free);
+ kfree_sensitive(buf_to_free);
}
struct cifs_tcon *
@@ -145,7 +145,7 @@ tconInfoFree(struct cifs_tcon *buf_to_free)
}
atomic_dec(&tconInfoAllocCount);
kfree(buf_to_free->nativeFileSystem);
- kzfree(buf_to_free->password);
+ kfree_sensitive(buf_to_free->password);
kfree(buf_to_free->crfid.fid);
#ifdef CONFIG_CIFS_DFS_UPCALL
kfree(buf_to_free->dfs_path);
diff --git a/fs/crypto/keyring.c b/fs/crypto/keyring.c
index ab41b25d4fa1..41460c6331e4 100644
--- a/fs/crypto/keyring.c
+++ b/fs/crypto/keyring.c
@@ -49,7 +49,7 @@ static void free_master_key(struct fscrypt_master_key *mk)
}
key_put(mk->mk_users);
- kzfree(mk);
+ kfree_sensitive(mk);
}
static inline bool valid_key_spec(const struct fscrypt_key_specifier *spec)
@@ -491,7 +491,7 @@ static int fscrypt_provisioning_key_preparse(struct key_preparsed_payload *prep)
static void fscrypt_provisioning_key_free_preparse(
struct key_preparsed_payload *prep)
{
- kzfree(prep->payload.data[0]);
+ kfree_sensitive(prep->payload.data[0]);
}
static void fscrypt_provisioning_key_describe(const struct key *key,
@@ -508,7 +508,7 @@ static void fscrypt_provisioning_key_describe(const struct key *key,
static void fscrypt_provisioning_key_destroy(struct key *key)
{
- kzfree(key->payload.data[0]);
+ kfree_sensitive(key->payload.data[0]);
}
static struct key_type key_type_fscrypt_provisioning = {
diff --git a/fs/crypto/keysetup_v1.c b/fs/crypto/keysetup_v1.c
index 801b48c0cd7f..c8a930f8faf2 100644
--- a/fs/crypto/keysetup_v1.c
+++ b/fs/crypto/keysetup_v1.c
@@ -155,7 +155,7 @@ static void free_direct_key(struct fscrypt_direct_key *dk)
{
if (dk) {
crypto_free_skcipher(dk->dk_ctfm);
- kzfree(dk);
+ kfree_sensitive(dk);
}
}
@@ -285,7 +285,7 @@ static int setup_v1_file_key_derived(struct fscrypt_info *ci,
err = fscrypt_set_per_file_enc_key(ci, derived_key);
out:
- kzfree(derived_key);
+ kfree_sensitive(derived_key);
return err;
}
diff --git a/fs/ecryptfs/keystore.c b/fs/ecryptfs/keystore.c
index af3eb02bbca1..f6a17d259db7 100644
--- a/fs/ecryptfs/keystore.c
+++ b/fs/ecryptfs/keystore.c
@@ -838,7 +838,7 @@ ecryptfs_write_tag_70_packet(char *dest, size_t *remaining_bytes,
out_release_free_unlock:
crypto_free_shash(s->hash_tfm);
out_free_unlock:
- kzfree(s->block_aligned_filename);
+ kfree_sensitive(s->block_aligned_filename);
out_unlock:
mutex_unlock(s->tfm_mutex);
out:
@@ -847,7 +847,7 @@ ecryptfs_write_tag_70_packet(char *dest, size_t *remaining_bytes,
key_put(auth_tok_key);
}
skcipher_request_free(s->skcipher_req);
- kzfree(s->hash_desc);
+ kfree_sensitive(s->hash_desc);
kfree(s);
return rc;
}
diff --git a/fs/ecryptfs/messaging.c b/fs/ecryptfs/messaging.c
index 8646ba76def3..c0dfd9647627 100644
--- a/fs/ecryptfs/messaging.c
+++ b/fs/ecryptfs/messaging.c
@@ -175,7 +175,7 @@ int ecryptfs_exorcise_daemon(struct ecryptfs_daemon *daemon)
}
hlist_del(&daemon->euid_chain);
mutex_unlock(&daemon->mux);
- kzfree(daemon);
+ kfree_sensitive(daemon);
out:
return rc;
}
diff --git a/include/crypto/aead.h b/include/crypto/aead.h
index 62c68550aab6..c32a6f5664e9 100644
--- a/include/crypto/aead.h
+++ b/include/crypto/aead.h
@@ -425,7 +425,7 @@ static inline struct aead_request *aead_request_alloc(struct crypto_aead *tfm,
*/
static inline void aead_request_free(struct aead_request *req)
{
- kzfree(req);
+ kfree_sensitive(req);
}
/**
diff --git a/include/crypto/akcipher.h b/include/crypto/akcipher.h
index 6924b091adec..1d3aa252caba 100644
--- a/include/crypto/akcipher.h
+++ b/include/crypto/akcipher.h
@@ -207,7 +207,7 @@ static inline struct akcipher_request *akcipher_request_alloc(
*/
static inline void akcipher_request_free(struct akcipher_request *req)
{
- kzfree(req);
+ kfree_sensitive(req);
}
/**
diff --git a/include/crypto/gf128mul.h b/include/crypto/gf128mul.h
index fa0a63d298dc..81330c6446f6 100644
--- a/include/crypto/gf128mul.h
+++ b/include/crypto/gf128mul.h
@@ -230,7 +230,7 @@ void gf128mul_4k_bbe(be128 *a, const struct gf128mul_4k *t);
void gf128mul_x8_ble(le128 *r, const le128 *x);
static inline void gf128mul_free_4k(struct gf128mul_4k *t)
{
- kzfree(t);
+ kfree_sensitive(t);
}
diff --git a/include/crypto/hash.h b/include/crypto/hash.h
index cee446c59497..cef16b7d153e 100644
--- a/include/crypto/hash.h
+++ b/include/crypto/hash.h
@@ -606,7 +606,7 @@ static inline struct ahash_request *ahash_request_alloc(
*/
static inline void ahash_request_free(struct ahash_request *req)
{
- kzfree(req);
+ kfree_sensitive(req);
}
static inline void ahash_request_zero(struct ahash_request *req)
diff --git a/include/crypto/internal/acompress.h b/include/crypto/internal/acompress.h
index cf478681b53e..cfc47e18820f 100644
--- a/include/crypto/internal/acompress.h
+++ b/include/crypto/internal/acompress.h
@@ -46,7 +46,7 @@ static inline struct acomp_req *__acomp_request_alloc(struct crypto_acomp *tfm)
static inline void __acomp_request_free(struct acomp_req *req)
{
- kzfree(req);
+ kfree_sensitive(req);
}
/**
diff --git a/include/crypto/kpp.h b/include/crypto/kpp.h
index cd9a9b500624..88b591215d5c 100644
--- a/include/crypto/kpp.h
+++ b/include/crypto/kpp.h
@@ -187,7 +187,7 @@ static inline struct kpp_request *kpp_request_alloc(struct crypto_kpp *tfm,
*/
static inline void kpp_request_free(struct kpp_request *req)
{
- kzfree(req);
+ kfree_sensitive(req);
}
/**
diff --git a/include/crypto/skcipher.h b/include/crypto/skcipher.h
index 141e7690f9c3..1013c9cbae69 100644
--- a/include/crypto/skcipher.h
+++ b/include/crypto/skcipher.h
@@ -508,7 +508,7 @@ static inline struct skcipher_request *skcipher_request_alloc(
*/
static inline void skcipher_request_free(struct skcipher_request *req)
{
- kzfree(req);
+ kfree_sensitive(req);
}
static inline void skcipher_request_zero(struct skcipher_request *req)
diff --git a/include/linux/slab.h b/include/linux/slab.h
index 6d454886bcaf..7f2018943997 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -186,7 +186,7 @@ void memcg_deactivate_kmem_caches(struct mem_cgroup *, struct mem_cgroup *);
*/
void * __must_check krealloc(const void *, size_t, gfp_t);
void kfree(const void *);
-void kzfree(const void *);
+void kfree_sensitive(const void *);
size_t __ksize(const void *);
size_t ksize(const void *);
diff --git a/lib/mpi/mpiutil.c b/lib/mpi/mpiutil.c
index 20ed0f766787..4cd2b335cb7f 100644
--- a/lib/mpi/mpiutil.c
+++ b/lib/mpi/mpiutil.c
@@ -69,7 +69,7 @@ void mpi_free_limb_space(mpi_ptr_t a)
if (!a)
return;
- kzfree(a);
+ kfree_sensitive(a);
}
void mpi_assign_limb_space(MPI a, mpi_ptr_t ap, unsigned nlimbs)
@@ -95,7 +95,7 @@ int mpi_resize(MPI a, unsigned nlimbs)
if (!p)
return -ENOMEM;
memcpy(p, a->d, a->alloced * sizeof(mpi_limb_t));
- kzfree(a->d);
+ kfree_sensitive(a->d);
a->d = p;
} else {
a->d = kcalloc(nlimbs, sizeof(mpi_limb_t), GFP_KERNEL);
@@ -112,7 +112,7 @@ void mpi_free(MPI a)
return;
if (a->flags & 4)
- kzfree(a->d);
+ kfree_sensitive(a->d);
else
mpi_free_limb_space(a->d);
diff --git a/lib/test_kasan.c b/lib/test_kasan.c
index e3087d90e00d..a1c82aa68a33 100644
--- a/lib/test_kasan.c
+++ b/lib/test_kasan.c
@@ -757,15 +757,15 @@ static noinline void __init kmalloc_double_kzfree(void)
char *ptr;
size_t size = 16;
- pr_info("double-free (kzfree)\n");
+ pr_info("double-free (kfree_sensitive)\n");
ptr = kmalloc(size, GFP_KERNEL);
if (!ptr) {
pr_err("Allocation failed\n");
return;
}
- kzfree(ptr);
- kzfree(ptr);
+ kfree_sensitive(ptr);
+ kfree_sensitive(ptr);
}
#ifdef CONFIG_KASAN_VMALLOC
diff --git a/mm/slab_common.c b/mm/slab_common.c
index 23c7500eea7d..c08bc7eb20bd 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -1707,17 +1707,17 @@ void *krealloc(const void *p, size_t new_size, gfp_t flags)
EXPORT_SYMBOL(krealloc);
/**
- * kzfree - like kfree but zero memory
+ * kfree_sensitive - Clear sensitive information in memory before freeing
* @p: object to free memory of
*
* The memory of the object @p points to is zeroed before freed.
- * If @p is %NULL, kzfree() does nothing.
+ * If @p is %NULL, kfree_sensitive() does nothing.
*
* Note: this function zeroes the whole allocated buffer which can be a good
* deal bigger than the requested buffer size passed to kmalloc(). So be
* careful when using this function in performance sensitive code.
*/
-void kzfree(const void *p)
+void kfree_sensitive(const void *p)
{
size_t ks;
void *mem = (void *)p;
@@ -1725,10 +1725,10 @@ void kzfree(const void *p)
if (unlikely(ZERO_OR_NULL_PTR(mem)))
return;
ks = ksize(mem);
- memset(mem, 0, ks);
+ memzero_explicit(mem, ks);
kfree(mem);
}
-EXPORT_SYMBOL(kzfree);
+EXPORT_SYMBOL(kfree_sensitive);
/**
* ksize - get the actual amount of memory allocated for a given object
diff --git a/net/atm/mpoa_caches.c b/net/atm/mpoa_caches.c
index 3286f9d527d3..f7a2f0e41105 100644
--- a/net/atm/mpoa_caches.c
+++ b/net/atm/mpoa_caches.c
@@ -180,7 +180,7 @@ static int cache_hit(in_cache_entry *entry, struct mpoa_client *mpc)
static void in_cache_put(in_cache_entry *entry)
{
if (refcount_dec_and_test(&entry->use)) {
- kzfree(entry);
+ kfree_sensitive(entry);
}
}
@@ -415,7 +415,7 @@ static eg_cache_entry *eg_cache_get_by_src_ip(__be32 ipaddr,
static void eg_cache_put(eg_cache_entry *entry)
{
if (refcount_dec_and_test(&entry->use)) {
- kzfree(entry);
+ kfree_sensitive(entry);
}
}
diff --git a/net/bluetooth/ecdh_helper.c b/net/bluetooth/ecdh_helper.c
index 2155ce802877..3226fe02e875 100644
--- a/net/bluetooth/ecdh_helper.c
+++ b/net/bluetooth/ecdh_helper.c
@@ -104,7 +104,7 @@ int compute_ecdh_secret(struct crypto_kpp *tfm, const u8 public_key[64],
free_all:
kpp_request_free(req);
free_tmp:
- kzfree(tmp);
+ kfree_sensitive(tmp);
return err;
}
@@ -151,9 +151,9 @@ int set_ecdh_privkey(struct crypto_kpp *tfm, const u8 private_key[32])
err = crypto_kpp_set_secret(tfm, buf, buf_len);
/* fall through */
free_all:
- kzfree(buf);
+ kfree_sensitive(buf);
free_tmp:
- kzfree(tmp);
+ kfree_sensitive(tmp);
return err;
}
diff --git a/net/bluetooth/smp.c b/net/bluetooth/smp.c
index 1476a91ce935..445ec3d09b01 100644
--- a/net/bluetooth/smp.c
+++ b/net/bluetooth/smp.c
@@ -753,9 +753,9 @@ static void smp_chan_destroy(struct l2cap_conn *conn)
complete = test_bit(SMP_FLAG_COMPLETE, &smp->flags);
mgmt_smp_complete(hcon, complete);
- kzfree(smp->csrk);
- kzfree(smp->slave_csrk);
- kzfree(smp->link_key);
+ kfree_sensitive(smp->csrk);
+ kfree_sensitive(smp->slave_csrk);
+ kfree_sensitive(smp->link_key);
crypto_free_shash(smp->tfm_cmac);
crypto_free_kpp(smp->tfm_ecdh);
@@ -789,7 +789,7 @@ static void smp_chan_destroy(struct l2cap_conn *conn)
}
chan->data = NULL;
- kzfree(smp);
+ kfree_sensitive(smp);
hci_conn_drop(hcon);
}
@@ -1149,7 +1149,7 @@ static void sc_generate_link_key(struct smp_chan *smp)
const u8 salt[16] = { 0x31, 0x70, 0x6d, 0x74 };
if (smp_h7(smp->tfm_cmac, smp->tk, salt, smp->link_key)) {
- kzfree(smp->link_key);
+ kfree_sensitive(smp->link_key);
smp->link_key = NULL;
return;
}
@@ -1158,14 +1158,14 @@ static void sc_generate_link_key(struct smp_chan *smp)
const u8 tmp1[4] = { 0x31, 0x70, 0x6d, 0x74 };
if (smp_h6(smp->tfm_cmac, smp->tk, tmp1, smp->link_key)) {
- kzfree(smp->link_key);
+ kfree_sensitive(smp->link_key);
smp->link_key = NULL;
return;
}
}
if (smp_h6(smp->tfm_cmac, smp->link_key, lebr, smp->link_key)) {
- kzfree(smp->link_key);
+ kfree_sensitive(smp->link_key);
smp->link_key = NULL;
return;
}
@@ -1400,7 +1400,7 @@ static struct smp_chan *smp_chan_create(struct l2cap_conn *conn)
free_shash:
crypto_free_shash(smp->tfm_cmac);
zfree_smp:
- kzfree(smp);
+ kfree_sensitive(smp);
return NULL;
}
@@ -3263,7 +3263,7 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid)
tfm_cmac = crypto_alloc_shash("cmac(aes)", 0, 0);
if (IS_ERR(tfm_cmac)) {
BT_ERR("Unable to create CMAC crypto context");
- kzfree(smp);
+ kfree_sensitive(smp);
return ERR_CAST(tfm_cmac);
}
@@ -3271,7 +3271,7 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid)
if (IS_ERR(tfm_ecdh)) {
BT_ERR("Unable to create ECDH crypto context");
crypto_free_shash(tfm_cmac);
- kzfree(smp);
+ kfree_sensitive(smp);
return ERR_CAST(tfm_ecdh);
}
@@ -3285,7 +3285,7 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid)
if (smp) {
crypto_free_shash(smp->tfm_cmac);
crypto_free_kpp(smp->tfm_ecdh);
- kzfree(smp);
+ kfree_sensitive(smp);
}
return ERR_PTR(-ENOMEM);
}
@@ -3332,7 +3332,7 @@ static void smp_del_chan(struct l2cap_chan *chan)
chan->data = NULL;
crypto_free_shash(smp->tfm_cmac);
crypto_free_kpp(smp->tfm_ecdh);
- kzfree(smp);
+ kfree_sensitive(smp);
}
l2cap_chan_put(chan);
diff --git a/net/core/sock.c b/net/core/sock.c
index ce1d8dce9b7a..e20a672afee5 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -2182,7 +2182,7 @@ static inline void __sock_kfree_s(struct sock *sk, void *mem, int size,
if (WARN_ON_ONCE(!mem))
return;
if (nullify)
- kzfree(mem);
+ kfree_sensitive(mem);
else
kfree(mem);
atomic_sub(size, &sk->sk_omem_alloc);
diff --git a/net/ipv4/tcp_fastopen.c b/net/ipv4/tcp_fastopen.c
index 19ad9586c720..c1a54f3d58f5 100644
--- a/net/ipv4/tcp_fastopen.c
+++ b/net/ipv4/tcp_fastopen.c
@@ -38,7 +38,7 @@ static void tcp_fastopen_ctx_free(struct rcu_head *head)
struct tcp_fastopen_context *ctx =
container_of(head, struct tcp_fastopen_context, rcu);
- kzfree(ctx);
+ kfree_sensitive(ctx);
}
void tcp_fastopen_destroy_cipher(struct sock *sk)
diff --git a/net/mac80211/aead_api.c b/net/mac80211/aead_api.c
index c5fe95e49c68..d7b3d905d535 100644
--- a/net/mac80211/aead_api.c
+++ b/net/mac80211/aead_api.c
@@ -41,7 +41,7 @@ int aead_encrypt(struct crypto_aead *tfm, u8 *b_0, u8 *aad, size_t aad_len,
aead_request_set_ad(aead_req, sg[0].length);
crypto_aead_encrypt(aead_req);
- kzfree(aead_req);
+ kfree_sensitive(aead_req);
return 0;
}
@@ -76,7 +76,7 @@ int aead_decrypt(struct crypto_aead *tfm, u8 *b_0, u8 *aad, size_t aad_len,
aead_request_set_ad(aead_req, sg[0].length);
err = crypto_aead_decrypt(aead_req);
- kzfree(aead_req);
+ kfree_sensitive(aead_req);
return err;
}
diff --git a/net/mac80211/aes_gmac.c b/net/mac80211/aes_gmac.c
index 16ba09cb5def..6f3b3a0cc10a 100644
--- a/net/mac80211/aes_gmac.c
+++ b/net/mac80211/aes_gmac.c
@@ -60,7 +60,7 @@ int ieee80211_aes_gmac(struct crypto_aead *tfm, const u8 *aad, u8 *nonce,
aead_request_set_ad(aead_req, GMAC_AAD_LEN + data_len);
crypto_aead_encrypt(aead_req);
- kzfree(aead_req);
+ kfree_sensitive(aead_req);
return 0;
}
diff --git a/net/mac80211/key.c b/net/mac80211/key.c
index 8f403c1bb908..6bb765721862 100644
--- a/net/mac80211/key.c
+++ b/net/mac80211/key.c
@@ -732,7 +732,7 @@ static void ieee80211_key_free_common(struct ieee80211_key *key)
ieee80211_aes_gcm_key_free(key->u.gcmp.tfm);
break;
}
- kzfree(key);
+ kfree_sensitive(key);
}
static void __ieee80211_key_destroy(struct ieee80211_key *key,
diff --git a/net/mac802154/llsec.c b/net/mac802154/llsec.c
index c079ee69d3d0..585d33144c33 100644
--- a/net/mac802154/llsec.c
+++ b/net/mac802154/llsec.c
@@ -49,7 +49,7 @@ void mac802154_llsec_destroy(struct mac802154_llsec *sec)
msl = container_of(sl, struct mac802154_llsec_seclevel, level);
list_del(&sl->list);
- kzfree(msl);
+ kfree_sensitive(msl);
}
list_for_each_entry_safe(dev, dn, &sec->table.devices, list) {
@@ -66,7 +66,7 @@ void mac802154_llsec_destroy(struct mac802154_llsec *sec)
mkey = container_of(key->key, struct mac802154_llsec_key, key);
list_del(&key->list);
llsec_key_put(mkey);
- kzfree(key);
+ kfree_sensitive(key);
}
}
@@ -155,7 +155,7 @@ llsec_key_alloc(const struct ieee802154_llsec_key *template)
if (key->tfm[i])
crypto_free_aead(key->tfm[i]);
- kzfree(key);
+ kfree_sensitive(key);
return NULL;
}
@@ -170,7 +170,7 @@ static void llsec_key_release(struct kref *ref)
crypto_free_aead(key->tfm[i]);
crypto_free_sync_skcipher(key->tfm0);
- kzfree(key);
+ kfree_sensitive(key);
}
static struct mac802154_llsec_key*
@@ -261,7 +261,7 @@ int mac802154_llsec_key_add(struct mac802154_llsec *sec,
return 0;
fail:
- kzfree(new);
+ kfree_sensitive(new);
return -ENOMEM;
}
@@ -341,10 +341,10 @@ static void llsec_dev_free(struct mac802154_llsec_device *dev)
devkey);
list_del(&pos->list);
- kzfree(devkey);
+ kfree_sensitive(devkey);
}
- kzfree(dev);
+ kfree_sensitive(dev);
}
int mac802154_llsec_dev_add(struct mac802154_llsec *sec,
@@ -682,7 +682,7 @@ llsec_do_encrypt_auth(struct sk_buff *skb, const struct mac802154_llsec *sec,
rc = crypto_aead_encrypt(req);
- kzfree(req);
+ kfree_sensitive(req);
return rc;
}
@@ -886,7 +886,7 @@ llsec_do_decrypt_auth(struct sk_buff *skb, const struct mac802154_llsec *sec,
rc = crypto_aead_decrypt(req);
- kzfree(req);
+ kfree_sensitive(req);
skb_trim(skb, skb->len - authlen);
return rc;
@@ -926,7 +926,7 @@ llsec_update_devkey_record(struct mac802154_llsec_device *dev,
if (!devkey)
list_add_rcu(&next->devkey.list, &dev->dev.keys);
else
- kzfree(next);
+ kfree_sensitive(next);
spin_unlock_bh(&dev->lock);
}
diff --git a/net/sctp/auth.c b/net/sctp/auth.c
index 4278764d82b8..3a22e1747712 100644
--- a/net/sctp/auth.c
+++ b/net/sctp/auth.c
@@ -49,7 +49,7 @@ void sctp_auth_key_put(struct sctp_auth_bytes *key)
return;
if (refcount_dec_and_test(&key->refcnt)) {
- kzfree(key);
+ kfree_sensitive(key);
SCTP_DBG_OBJCNT_DEC(keys);
}
}
diff --git a/net/sctp/socket.c b/net/sctp/socket.c
index 827a9903ee28..3392d18a7e44 100644
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
@@ -3748,7 +3748,7 @@ static int sctp_setsockopt_auth_key(struct sock *sk,
}
out:
- kzfree(authkey);
+ kfree_sensitive(authkey);
return ret;
}
diff --git a/net/sunrpc/auth_gss/gss_krb5_crypto.c b/net/sunrpc/auth_gss/gss_krb5_crypto.c
index 6f2d30d7b766..19bb244d2444 100644
--- a/net/sunrpc/auth_gss/gss_krb5_crypto.c
+++ b/net/sunrpc/auth_gss/gss_krb5_crypto.c
@@ -1003,7 +1003,7 @@ krb5_rc4_setup_seq_key(struct krb5_ctx *kctx,
err = 0;
out_err:
- kzfree(desc);
+ kfree_sensitive(desc);
crypto_free_shash(hmac);
dprintk("%s: returning %d\n", __func__, err);
return err;
@@ -1079,7 +1079,7 @@ krb5_rc4_setup_enc_key(struct krb5_ctx *kctx,
err = 0;
out_err:
- kzfree(desc);
+ kfree_sensitive(desc);
crypto_free_shash(hmac);
dprintk("%s: returning %d\n", __func__, err);
return err;
diff --git a/net/sunrpc/auth_gss/gss_krb5_keys.c b/net/sunrpc/auth_gss/gss_krb5_keys.c
index 3b7f721c023b..726c076950c0 100644
--- a/net/sunrpc/auth_gss/gss_krb5_keys.c
+++ b/net/sunrpc/auth_gss/gss_krb5_keys.c
@@ -228,11 +228,11 @@ u32 krb5_derive_key(const struct gss_krb5_enctype *gk5e,
ret = 0;
err_free_raw:
- kzfree(rawkey);
+ kfree_sensitive(rawkey);
err_free_out:
- kzfree(outblockdata);
+ kfree_sensitive(outblockdata);
err_free_in:
- kzfree(inblockdata);
+ kfree_sensitive(inblockdata);
err_free_cipher:
crypto_free_sync_skcipher(cipher);
err_return:
diff --git a/net/sunrpc/auth_gss/gss_krb5_mech.c b/net/sunrpc/auth_gss/gss_krb5_mech.c
index 75b3c2e9e8f8..a84a5b289484 100644
--- a/net/sunrpc/auth_gss/gss_krb5_mech.c
+++ b/net/sunrpc/auth_gss/gss_krb5_mech.c
@@ -443,7 +443,7 @@ context_derive_keys_rc4(struct krb5_ctx *ctx)
desc->tfm = hmac;
err = crypto_shash_digest(desc, sigkeyconstant, slen, ctx->cksum);
- kzfree(desc);
+ kfree_sensitive(desc);
if (err)
goto out_err_free_hmac;
/*
diff --git a/net/tipc/crypto.c b/net/tipc/crypto.c
index c8c47fc72653..001bcb0f2480 100644
--- a/net/tipc/crypto.c
+++ b/net/tipc/crypto.c
@@ -441,7 +441,7 @@ static int tipc_aead_init(struct tipc_aead **aead, struct tipc_aead_key *ukey,
/* Allocate per-cpu TFM entry pointer */
tmp->tfm_entry = alloc_percpu(struct tipc_tfm *);
if (!tmp->tfm_entry) {
- kzfree(tmp);
+ kfree_sensitive(tmp);
return -ENOMEM;
}
@@ -491,7 +491,7 @@ static int tipc_aead_init(struct tipc_aead **aead, struct tipc_aead_key *ukey,
/* Not any TFM is allocated? */
if (!tfm_cnt) {
free_percpu(tmp->tfm_entry);
- kzfree(tmp);
+ kfree_sensitive(tmp);
return err;
}
@@ -545,7 +545,7 @@ static int tipc_aead_clone(struct tipc_aead **dst, struct tipc_aead *src)
aead->tfm_entry = alloc_percpu_gfp(struct tipc_tfm *, GFP_ATOMIC);
if (unlikely(!aead->tfm_entry)) {
- kzfree(aead);
+ kfree_sensitive(aead);
return -ENOMEM;
}
@@ -1352,7 +1352,7 @@ int tipc_crypto_start(struct tipc_crypto **crypto, struct net *net,
/* Allocate statistic structure */
c->stats = alloc_percpu_gfp(struct tipc_crypto_stats, GFP_ATOMIC);
if (!c->stats) {
- kzfree(c);
+ kfree_sensitive(c);
return -ENOMEM;
}
@@ -1408,7 +1408,7 @@ void tipc_crypto_stop(struct tipc_crypto **crypto)
free_percpu(c->stats);
*crypto = NULL;
- kzfree(c);
+ kfree_sensitive(c);
}
void tipc_crypto_timeout(struct tipc_crypto *rx)
diff --git a/net/wireless/core.c b/net/wireless/core.c
index 341402b4f178..cd2c982c5317 100644
--- a/net/wireless/core.c
+++ b/net/wireless/core.c
@@ -1107,7 +1107,7 @@ static void __cfg80211_unregister_wdev(struct wireless_dev *wdev, bool sync)
}
#ifdef CONFIG_CFG80211_WEXT
- kzfree(wdev->wext.keys);
+ kfree_sensitive(wdev->wext.keys);
wdev->wext.keys = NULL;
#endif
/* only initialized if we have a netdev */
diff --git a/net/wireless/ibss.c b/net/wireless/ibss.c
index ae8fe66a9bb8..a0621bb76d8e 100644
--- a/net/wireless/ibss.c
+++ b/net/wireless/ibss.c
@@ -127,7 +127,7 @@ int __cfg80211_join_ibss(struct cfg80211_registered_device *rdev,
return -EINVAL;
if (WARN_ON(wdev->connect_keys))
- kzfree(wdev->connect_keys);
+ kfree_sensitive(wdev->connect_keys);
wdev->connect_keys = connkeys;
wdev->ibss_fixed = params->channel_fixed;
@@ -161,7 +161,7 @@ static void __cfg80211_clear_ibss(struct net_device *dev, bool nowext)
ASSERT_WDEV_LOCK(wdev);
- kzfree(wdev->connect_keys);
+ kfree_sensitive(wdev->connect_keys);
wdev->connect_keys = NULL;
rdev_set_qos_map(rdev, dev, NULL);
diff --git a/net/wireless/lib80211_crypt_tkip.c b/net/wireless/lib80211_crypt_tkip.c
index f5e842ba7673..1b4d6c87a5c5 100644
--- a/net/wireless/lib80211_crypt_tkip.c
+++ b/net/wireless/lib80211_crypt_tkip.c
@@ -131,7 +131,7 @@ static void lib80211_tkip_deinit(void *priv)
crypto_free_shash(_priv->tx_tfm_michael);
crypto_free_shash(_priv->rx_tfm_michael);
}
- kzfree(priv);
+ kfree_sensitive(priv);
}
static inline u16 RotR1(u16 val)
diff --git a/net/wireless/lib80211_crypt_wep.c b/net/wireless/lib80211_crypt_wep.c
index dafc6f3571db..6ab9957b8f96 100644
--- a/net/wireless/lib80211_crypt_wep.c
+++ b/net/wireless/lib80211_crypt_wep.c
@@ -56,7 +56,7 @@ static void *lib80211_wep_init(int keyidx)
static void lib80211_wep_deinit(void *priv)
{
- kzfree(priv);
+ kfree_sensitive(priv);
}
/* Add WEP IV/key info to a frame that has at least 4 bytes of headroom */
diff --git a/net/wireless/nl80211.c b/net/wireless/nl80211.c
index 5fa402144cda..c1b37c798907 100644
--- a/net/wireless/nl80211.c
+++ b/net/wireless/nl80211.c
@@ -9783,7 +9783,7 @@ static int nl80211_join_ibss(struct sk_buff *skb, struct genl_info *info)
if ((ibss.chandef.width != NL80211_CHAN_WIDTH_20_NOHT) &&
no_ht) {
- kzfree(connkeys);
+ kfree_sensitive(connkeys);
return -EINVAL;
}
}
@@ -9795,7 +9795,7 @@ static int nl80211_join_ibss(struct sk_buff *skb, struct genl_info *info)
int r = validate_pae_over_nl80211(rdev, info);
if (r < 0) {
- kzfree(connkeys);
+ kfree_sensitive(connkeys);
return r;
}
@@ -9808,7 +9808,7 @@ static int nl80211_join_ibss(struct sk_buff *skb, struct genl_info *info)
wdev_lock(dev->ieee80211_ptr);
err = __cfg80211_join_ibss(rdev, dev, &ibss, connkeys);
if (err)
- kzfree(connkeys);
+ kfree_sensitive(connkeys);
else if (info->attrs[NL80211_ATTR_SOCKET_OWNER])
dev->ieee80211_ptr->conn_owner_nlportid = info->snd_portid;
wdev_unlock(dev->ieee80211_ptr);
@@ -10228,7 +10228,7 @@ static int nl80211_connect(struct sk_buff *skb, struct genl_info *info)
if (info->attrs[NL80211_ATTR_HT_CAPABILITY]) {
if (!info->attrs[NL80211_ATTR_HT_CAPABILITY_MASK]) {
- kzfree(connkeys);
+ kfree_sensitive(connkeys);
return -EINVAL;
}
memcpy(&connect.ht_capa,
@@ -10246,7 +10246,7 @@ static int nl80211_connect(struct sk_buff *skb, struct genl_info *info)
if (info->attrs[NL80211_ATTR_VHT_CAPABILITY]) {
if (!info->attrs[NL80211_ATTR_VHT_CAPABILITY_MASK]) {
- kzfree(connkeys);
+ kfree_sensitive(connkeys);
return -EINVAL;
}
memcpy(&connect.vht_capa,
@@ -10260,7 +10260,7 @@ static int nl80211_connect(struct sk_buff *skb, struct genl_info *info)
(rdev->wiphy.features & NL80211_FEATURE_QUIET)) &&
!wiphy_ext_feature_isset(&rdev->wiphy,
NL80211_EXT_FEATURE_RRM)) {
- kzfree(connkeys);
+ kfree_sensitive(connkeys);
return -EINVAL;
}
connect.flags |= ASSOC_REQ_USE_RRM;
@@ -10268,21 +10268,21 @@ static int nl80211_connect(struct sk_buff *skb, struct genl_info *info)
connect.pbss = nla_get_flag(info->attrs[NL80211_ATTR_PBSS]);
if (connect.pbss && !rdev->wiphy.bands[NL80211_BAND_60GHZ]) {
- kzfree(connkeys);
+ kfree_sensitive(connkeys);
return -EOPNOTSUPP;
}
if (info->attrs[NL80211_ATTR_BSS_SELECT]) {
/* bss selection makes no sense if bssid is set */
if (connect.bssid) {
- kzfree(connkeys);
+ kfree_sensitive(connkeys);
return -EINVAL;
}
err = parse_bss_select(info->attrs[NL80211_ATTR_BSS_SELECT],
wiphy, &connect.bss_select);
if (err) {
- kzfree(connkeys);
+ kfree_sensitive(connkeys);
return err;
}
}
@@ -10312,13 +10312,13 @@ static int nl80211_connect(struct sk_buff *skb, struct genl_info *info)
info->attrs[NL80211_ATTR_FILS_ERP_REALM] ||
info->attrs[NL80211_ATTR_FILS_ERP_NEXT_SEQ_NUM] ||
info->attrs[NL80211_ATTR_FILS_ERP_RRK]) {
- kzfree(connkeys);
+ kfree_sensitive(connkeys);
return -EINVAL;
}
if (nla_get_flag(info->attrs[NL80211_ATTR_EXTERNAL_AUTH_SUPPORT])) {
if (!info->attrs[NL80211_ATTR_SOCKET_OWNER]) {
- kzfree(connkeys);
+ kfree_sensitive(connkeys);
GENL_SET_ERR_MSG(info,
"external auth requires connection ownership");
return -EINVAL;
@@ -10331,7 +10331,7 @@ static int nl80211_connect(struct sk_buff *skb, struct genl_info *info)
err = cfg80211_connect(rdev, dev, &connect, connkeys,
connect.prev_bssid);
if (err)
- kzfree(connkeys);
+ kfree_sensitive(connkeys);
if (!err && info->attrs[NL80211_ATTR_SOCKET_OWNER]) {
dev->ieee80211_ptr->conn_owner_nlportid = info->snd_portid;
diff --git a/net/wireless/sme.c b/net/wireless/sme.c
index ac3e60aa1fc8..8e60d5a6236c 100644
--- a/net/wireless/sme.c
+++ b/net/wireless/sme.c
@@ -741,7 +741,7 @@ void __cfg80211_connect_result(struct net_device *dev,
}
if (cr->status != WLAN_STATUS_SUCCESS) {
- kzfree(wdev->connect_keys);
+ kfree_sensitive(wdev->connect_keys);
wdev->connect_keys = NULL;
wdev->ssid_len = 0;
wdev->conn_owner_nlportid = 0;
@@ -1096,7 +1096,7 @@ void __cfg80211_disconnected(struct net_device *dev, const u8 *ie,
wdev->current_bss = NULL;
wdev->ssid_len = 0;
wdev->conn_owner_nlportid = 0;
- kzfree(wdev->connect_keys);
+ kfree_sensitive(wdev->connect_keys);
wdev->connect_keys = NULL;
nl80211_send_disconnected(rdev, dev, reason, ie, ie_len, from_ap);
@@ -1276,7 +1276,7 @@ int cfg80211_disconnect(struct cfg80211_registered_device *rdev,
ASSERT_WDEV_LOCK(wdev);
- kzfree(wdev->connect_keys);
+ kfree_sensitive(wdev->connect_keys);
wdev->connect_keys = NULL;
wdev->conn_owner_nlportid = 0;
diff --git a/net/wireless/util.c b/net/wireless/util.c
index 6590efbbcbb9..ea172bc244c2 100644
--- a/net/wireless/util.c
+++ b/net/wireless/util.c
@@ -853,7 +853,7 @@ void cfg80211_upload_connect_keys(struct wireless_dev *wdev)
}
}
- kzfree(wdev->connect_keys);
+ kfree_sensitive(wdev->connect_keys);
wdev->connect_keys = NULL;
}
diff --git a/net/wireless/wext-sme.c b/net/wireless/wext-sme.c
index 73fd0eae08ca..73df23570d43 100644
--- a/net/wireless/wext-sme.c
+++ b/net/wireless/wext-sme.c
@@ -57,7 +57,7 @@ int cfg80211_mgd_wext_connect(struct cfg80211_registered_device *rdev,
err = cfg80211_connect(rdev, wdev->netdev,
&wdev->wext.connect, ck, prev_bssid);
if (err)
- kzfree(ck);
+ kfree_sensitive(ck);
return err;
}
diff --git a/scripts/coccinelle/free/devm_free.cocci b/scripts/coccinelle/free/devm_free.cocci
index 3357bf4dbd7c..da80050b91ff 100644
--- a/scripts/coccinelle/free/devm_free.cocci
+++ b/scripts/coccinelle/free/devm_free.cocci
@@ -89,7 +89,7 @@ position p;
(
kfree@p(x)
|
- kzfree@p(x)
+ kfree_sensitive@p(x)
|
krealloc@p(x, ...)
|
@@ -112,7 +112,7 @@ position p != safe.p;
(
* kfree@p(x)
|
-* kzfree@p(x)
+* kfree_sensitive@p(x)
|
* krealloc@p(x, ...)
|
diff --git a/scripts/coccinelle/free/ifnullfree.cocci b/scripts/coccinelle/free/ifnullfree.cocci
index b3290c4ee239..2045391e36a0 100644
--- a/scripts/coccinelle/free/ifnullfree.cocci
+++ b/scripts/coccinelle/free/ifnullfree.cocci
@@ -21,7 +21,7 @@ expression E;
(
kfree(E);
|
- kzfree(E);
+ kfree_sensitive(E);
|
debugfs_remove(E);
|
@@ -42,7 +42,7 @@ position p;
@@
* if (E != NULL)
-* \(kfree@p\|kzfree@p\|debugfs_remove@p\|debugfs_remove_recursive@p\|
+* \(kfree@p\|kfree_sensitive@p\|debugfs_remove@p\|debugfs_remove_recursive@p\|
* usb_free_urb@p\|kmem_cache_destroy@p\|mempool_destroy@p\|
* dma_pool_destroy@p\)(E);
diff --git a/scripts/coccinelle/free/kfree.cocci b/scripts/coccinelle/free/kfree.cocci
index e9d50e718e46..168568386034 100644
--- a/scripts/coccinelle/free/kfree.cocci
+++ b/scripts/coccinelle/free/kfree.cocci
@@ -24,7 +24,7 @@ position p1;
(
* kfree@p1(E)
|
-* kzfree@p1(E)
+* kfree_sensitive@p1(E)
)
@print expression@
@@ -68,7 +68,7 @@ while (1) { ...
(
* kfree@ok(E)
|
-* kzfree@ok(E)
+* kfree_sensitive@ok(E)
)
... when != break;
when != goto l;
@@ -86,7 +86,7 @@ position free.p1!=loop.ok,p2!={print.p,sz.p};
(
* kfree@p1(E,...)
|
-* kzfree@p1(E,...)
+* kfree_sensitive@p1(E,...)
)
...
(
diff --git a/scripts/coccinelle/free/kfreeaddr.cocci b/scripts/coccinelle/free/kfreeaddr.cocci
index cfaf308328d8..142af6337a04 100644
--- a/scripts/coccinelle/free/kfreeaddr.cocci
+++ b/scripts/coccinelle/free/kfreeaddr.cocci
@@ -20,7 +20,7 @@ position p;
(
* kfree@p(&e->f)
|
-* kzfree@p(&e->f)
+* kfree_sensitive@p(&e->f)
)
@script:python depends on org@
diff --git a/security/apparmor/domain.c b/security/apparmor/domain.c
index 6ceb74e0f789..9e9d4411b96e 100644
--- a/security/apparmor/domain.c
+++ b/security/apparmor/domain.c
@@ -40,8 +40,8 @@ void aa_free_domain_entries(struct aa_domain *domain)
return;
for (i = 0; i < domain->size; i++)
- kzfree(domain->table[i]);
- kzfree(domain->table);
+ kfree_sensitive(domain->table[i]);
+ kfree_sensitive(domain->table);
domain->table = NULL;
}
}
diff --git a/security/apparmor/include/file.h b/security/apparmor/include/file.h
index aff26fc71407..d4f8948517d9 100644
--- a/security/apparmor/include/file.h
+++ b/security/apparmor/include/file.h
@@ -72,7 +72,7 @@ static inline void aa_free_file_ctx(struct aa_file_ctx *ctx)
{
if (ctx) {
aa_put_label(rcu_access_pointer(ctx->label));
- kzfree(ctx);
+ kfree_sensitive(ctx);
}
}
diff --git a/security/apparmor/policy.c b/security/apparmor/policy.c
index 269f2f53c0b1..4ab00ab060f1 100644
--- a/security/apparmor/policy.c
+++ b/security/apparmor/policy.c
@@ -187,9 +187,9 @@ static void aa_free_data(void *ptr, void *arg)
{
struct aa_data *data = ptr;
- kzfree(data->data);
- kzfree(data->key);
- kzfree(data);
+ kfree_sensitive(data->data);
+ kfree_sensitive(data->key);
+ kfree_sensitive(data);
}
/**
@@ -217,19 +217,19 @@ void aa_free_profile(struct aa_profile *profile)
aa_put_profile(rcu_access_pointer(profile->parent));
aa_put_ns(profile->ns);
- kzfree(profile->rename);
+ kfree_sensitive(profile->rename);
aa_free_file_rules(&profile->file);
aa_free_cap_rules(&profile->caps);
aa_free_rlimit_rules(&profile->rlimits);
for (i = 0; i < profile->xattr_count; i++)
- kzfree(profile->xattrs[i]);
- kzfree(profile->xattrs);
+ kfree_sensitive(profile->xattrs[i]);
+ kfree_sensitive(profile->xattrs);
for (i = 0; i < profile->secmark_count; i++)
- kzfree(profile->secmark[i].label);
- kzfree(profile->secmark);
- kzfree(profile->dirname);
+ kfree_sensitive(profile->secmark[i].label);
+ kfree_sensitive(profile->secmark);
+ kfree_sensitive(profile->dirname);
aa_put_dfa(profile->xmatch);
aa_put_dfa(profile->policy.dfa);
@@ -237,13 +237,13 @@ void aa_free_profile(struct aa_profile *profile)
rht = profile->data;
profile->data = NULL;
rhashtable_free_and_destroy(rht, aa_free_data, NULL);
- kzfree(rht);
+ kfree_sensitive(rht);
}
- kzfree(profile->hash);
+ kfree_sensitive(profile->hash);
aa_put_loaddata(profile->rawdata);
- kzfree(profile);
+ kfree_sensitive(profile);
}
/**
diff --git a/security/apparmor/policy_ns.c b/security/apparmor/policy_ns.c
index d7ef540027a5..70921d95fb40 100644
--- a/security/apparmor/policy_ns.c
+++ b/security/apparmor/policy_ns.c
@@ -121,9 +121,9 @@ static struct aa_ns *alloc_ns(const char *prefix, const char *name)
return ns;
fail_unconfined:
- kzfree(ns->base.hname);
+ kfree_sensitive(ns->base.hname);
fail_ns:
- kzfree(ns);
+ kfree_sensitive(ns);
return NULL;
}
@@ -145,7 +145,7 @@ void aa_free_ns(struct aa_ns *ns)
ns->unconfined->ns = NULL;
aa_free_profile(ns->unconfined);
- kzfree(ns);
+ kfree_sensitive(ns);
}
/**
diff --git a/security/apparmor/policy_unpack.c b/security/apparmor/policy_unpack.c
index 2d743c004bc4..5cfbdc6ad106 100644
--- a/security/apparmor/policy_unpack.c
+++ b/security/apparmor/policy_unpack.c
@@ -163,10 +163,10 @@ static void do_loaddata_free(struct work_struct *work)
aa_put_ns(ns);
}
- kzfree(d->hash);
- kzfree(d->name);
+ kfree_sensitive(d->hash);
+ kfree_sensitive(d->name);
kvfree(d->data);
- kzfree(d);
+ kfree_sensitive(d);
}
void aa_loaddata_kref(struct kref *kref)
@@ -890,7 +890,7 @@ static struct aa_profile *unpack_profile(struct aa_ext *e, char **ns_name)
while (unpack_strdup(e, &key, NULL)) {
data = kzalloc(sizeof(*data), GFP_KERNEL);
if (!data) {
- kzfree(key);
+ kfree_sensitive(key);
goto fail;
}
@@ -898,8 +898,8 @@ static struct aa_profile *unpack_profile(struct aa_ext *e, char **ns_name)
data->size = unpack_blob(e, &data->data, NULL);
data->data = kvmemdup(data->data, data->size);
if (data->size && !data->data) {
- kzfree(data->key);
- kzfree(data);
+ kfree_sensitive(data->key);
+ kfree_sensitive(data);
goto fail;
}
@@ -1033,7 +1033,7 @@ void aa_load_ent_free(struct aa_load_ent *ent)
aa_put_profile(ent->old);
aa_put_profile(ent->new);
kfree(ent->ns_name);
- kzfree(ent);
+ kfree_sensitive(ent);
}
}
diff --git a/security/keys/big_key.c b/security/keys/big_key.c
index 82008f900930..a50094ee405a 100644
--- a/security/keys/big_key.c
+++ b/security/keys/big_key.c
@@ -281,7 +281,7 @@ int big_key_preparse(struct key_preparsed_payload *prep)
err_fput:
fput(file);
err_enckey:
- kzfree(enckey);
+ kfree_sensitive(enckey);
error:
big_key_free_buffer(buf);
return ret;
@@ -297,7 +297,7 @@ void big_key_free_preparse(struct key_preparsed_payload *prep)
path_put(path);
}
- kzfree(prep->payload.data[big_key_data]);
+ kfree_sensitive(prep->payload.data[big_key_data]);
}
/*
@@ -329,7 +329,7 @@ void big_key_destroy(struct key *key)
path->mnt = NULL;
path->dentry = NULL;
}
- kzfree(key->payload.data[big_key_data]);
+ kfree_sensitive(key->payload.data[big_key_data]);
key->payload.data[big_key_data] = NULL;
}
diff --git a/security/keys/dh.c b/security/keys/dh.c
index c4c629bb1c03..1abfa70ed6e1 100644
--- a/security/keys/dh.c
+++ b/security/keys/dh.c
@@ -58,9 +58,9 @@ static ssize_t dh_data_from_key(key_serial_t keyid, void **data)
static void dh_free_data(struct dh *dh)
{
- kzfree(dh->key);
- kzfree(dh->p);
- kzfree(dh->g);
+ kfree_sensitive(dh->key);
+ kfree_sensitive(dh->p);
+ kfree_sensitive(dh->g);
}
struct dh_completion {
@@ -126,7 +126,7 @@ static void kdf_dealloc(struct kdf_sdesc *sdesc)
if (sdesc->shash.tfm)
crypto_free_shash(sdesc->shash.tfm);
- kzfree(sdesc);
+ kfree_sensitive(sdesc);
}
/*
@@ -220,7 +220,7 @@ static int keyctl_dh_compute_kdf(struct kdf_sdesc *sdesc,
ret = -EFAULT;
err:
- kzfree(outbuf);
+ kfree_sensitive(outbuf);
return ret;
}
@@ -395,11 +395,11 @@ long __keyctl_dh_compute(struct keyctl_dh_params __user *params,
out6:
kpp_request_free(req);
out5:
- kzfree(outbuf);
+ kfree_sensitive(outbuf);
out4:
crypto_free_kpp(tfm);
out3:
- kzfree(secret);
+ kfree_sensitive(secret);
out2:
dh_free_data(&dh_inputs);
out1:
diff --git a/security/keys/encrypted-keys/encrypted.c b/security/keys/encrypted-keys/encrypted.c
index f6797ba44bf7..4b4c1d8c1494 100644
--- a/security/keys/encrypted-keys/encrypted.c
+++ b/security/keys/encrypted-keys/encrypted.c
@@ -382,7 +382,7 @@ static int get_derived_key(u8 *derived_key, enum derived_key_type key_type,
memcpy(derived_buf + strlen(derived_buf) + 1, master_key,
master_keylen);
ret = calc_hash(hash_tfm, derived_key, derived_buf, derived_buf_len);
- kzfree(derived_buf);
+ kfree_sensitive(derived_buf);
return ret;
}
@@ -824,13 +824,13 @@ static int encrypted_instantiate(struct key *key,
ret = encrypted_init(epayload, key->description, format, master_desc,
decrypted_datalen, hex_encoded_iv);
if (ret < 0) {
- kzfree(epayload);
+ kfree_sensitive(epayload);
goto out;
}
rcu_assign_keypointer(key, epayload);
out:
- kzfree(datablob);
+ kfree_sensitive(datablob);
return ret;
}
@@ -839,7 +839,7 @@ static void encrypted_rcu_free(struct rcu_head *rcu)
struct encrypted_key_payload *epayload;
epayload = container_of(rcu, struct encrypted_key_payload, rcu);
- kzfree(epayload);
+ kfree_sensitive(epayload);
}
/*
@@ -897,7 +897,7 @@ static int encrypted_update(struct key *key, struct key_preparsed_payload *prep)
rcu_assign_keypointer(key, new_epayload);
call_rcu(&epayload->rcu, encrypted_rcu_free);
out:
- kzfree(buf);
+ kfree_sensitive(buf);
return ret;
}
@@ -958,7 +958,7 @@ static long encrypted_read(const struct key *key, char *buffer,
memzero_explicit(derived_key, sizeof(derived_key));
memcpy(buffer, ascii_buf, asciiblob_len);
- kzfree(ascii_buf);
+ kfree_sensitive(ascii_buf);
return asciiblob_len;
out:
@@ -973,7 +973,7 @@ static long encrypted_read(const struct key *key, char *buffer,
*/
static void encrypted_destroy(struct key *key)
{
- kzfree(key->payload.data[0]);
+ kfree_sensitive(key->payload.data[0]);
}
struct key_type key_type_encrypted = {
diff --git a/security/keys/trusted-keys/trusted_tpm1.c b/security/keys/trusted-keys/trusted_tpm1.c
index 8001ab07e63b..b9fe02e5f84f 100644
--- a/security/keys/trusted-keys/trusted_tpm1.c
+++ b/security/keys/trusted-keys/trusted_tpm1.c
@@ -68,7 +68,7 @@ static int TSS_sha1(const unsigned char *data, unsigned int datalen,
}
ret = crypto_shash_digest(&sdesc->shash, data, datalen, digest);
- kzfree(sdesc);
+ kfree_sensitive(sdesc);
return ret;
}
@@ -112,7 +112,7 @@ static int TSS_rawhmac(unsigned char *digest, const unsigned char *key,
if (!ret)
ret = crypto_shash_final(&sdesc->shash, digest);
out:
- kzfree(sdesc);
+ kfree_sensitive(sdesc);
return ret;
}
@@ -166,7 +166,7 @@ int TSS_authhmac(unsigned char *digest, const unsigned char *key,
paramdigest, TPM_NONCE_SIZE, h1,
TPM_NONCE_SIZE, h2, 1, &c, 0, 0);
out:
- kzfree(sdesc);
+ kfree_sensitive(sdesc);
return ret;
}
EXPORT_SYMBOL_GPL(TSS_authhmac);
@@ -251,7 +251,7 @@ int TSS_checkhmac1(unsigned char *buffer,
if (memcmp(testhmac, authdata, SHA1_DIGEST_SIZE))
ret = -EINVAL;
out:
- kzfree(sdesc);
+ kfree_sensitive(sdesc);
return ret;
}
EXPORT_SYMBOL_GPL(TSS_checkhmac1);
@@ -353,7 +353,7 @@ static int TSS_checkhmac2(unsigned char *buffer,
if (memcmp(testhmac2, authdata2, SHA1_DIGEST_SIZE))
ret = -EINVAL;
out:
- kzfree(sdesc);
+ kfree_sensitive(sdesc);
return ret;
}
@@ -563,7 +563,7 @@ static int tpm_seal(struct tpm_buf *tb, uint16_t keytype,
*bloblen = storedsize;
}
out:
- kzfree(td);
+ kfree_sensitive(td);
return ret;
}
@@ -1031,12 +1031,12 @@ static int trusted_instantiate(struct key *key,
if (!ret && options->pcrlock)
ret = pcrlock(options->pcrlock);
out:
- kzfree(datablob);
- kzfree(options);
+ kfree_sensitive(datablob);
+ kfree_sensitive(options);
if (!ret)
rcu_assign_keypointer(key, payload);
else
- kzfree(payload);
+ kfree_sensitive(payload);
return ret;
}
@@ -1045,7 +1045,7 @@ static void trusted_rcu_free(struct rcu_head *rcu)
struct trusted_key_payload *p;
p = container_of(rcu, struct trusted_key_payload, rcu);
- kzfree(p);
+ kfree_sensitive(p);
}
/*
@@ -1087,13 +1087,13 @@ static int trusted_update(struct key *key, struct key_preparsed_payload *prep)
ret = datablob_parse(datablob, new_p, new_o);
if (ret != Opt_update) {
ret = -EINVAL;
- kzfree(new_p);
+ kfree_sensitive(new_p);
goto out;
}
if (!new_o->keyhandle) {
ret = -EINVAL;
- kzfree(new_p);
+ kfree_sensitive(new_p);
goto out;
}
@@ -1107,22 +1107,22 @@ static int trusted_update(struct key *key, struct key_preparsed_payload *prep)
ret = key_seal(new_p, new_o);
if (ret < 0) {
pr_info("trusted_key: key_seal failed (%d)\n", ret);
- kzfree(new_p);
+ kfree_sensitive(new_p);
goto out;
}
if (new_o->pcrlock) {
ret = pcrlock(new_o->pcrlock);
if (ret < 0) {
pr_info("trusted_key: pcrlock failed (%d)\n", ret);
- kzfree(new_p);
+ kfree_sensitive(new_p);
goto out;
}
}
rcu_assign_keypointer(key, new_p);
call_rcu(&p->rcu, trusted_rcu_free);
out:
- kzfree(datablob);
- kzfree(new_o);
+ kfree_sensitive(datablob);
+ kfree_sensitive(new_o);
return ret;
}
@@ -1154,7 +1154,7 @@ static long trusted_read(const struct key *key, char *buffer,
*/
static void trusted_destroy(struct key *key)
{
- kzfree(key->payload.data[0]);
+ kfree_sensitive(key->payload.data[0]);
}
struct key_type key_type_trusted = {
diff --git a/security/keys/user_defined.c b/security/keys/user_defined.c
index 07d4287e9084..749e2a4dcb13 100644
--- a/security/keys/user_defined.c
+++ b/security/keys/user_defined.c
@@ -82,7 +82,7 @@ EXPORT_SYMBOL_GPL(user_preparse);
*/
void user_free_preparse(struct key_preparsed_payload *prep)
{
- kzfree(prep->payload.data[0]);
+ kfree_sensitive(prep->payload.data[0]);
}
EXPORT_SYMBOL_GPL(user_free_preparse);
@@ -91,7 +91,7 @@ static void user_free_payload_rcu(struct rcu_head *head)
struct user_key_payload *payload;
payload = container_of(head, struct user_key_payload, rcu);
- kzfree(payload);
+ kfree_sensitive(payload);
}
/*
@@ -147,7 +147,7 @@ void user_destroy(struct key *key)
{
struct user_key_payload *upayload = key->payload.data[0];
- kzfree(upayload);
+ kfree_sensitive(upayload);
}
EXPORT_SYMBOL_GPL(user_destroy);
--
2.18.1
^ permalink raw reply related
* [PATCH 2/2] crypto: Remove unnecessary memzero_explicit()
From: Waiman Long @ 2020-04-13 21:15 UTC (permalink / raw)
To: Andrew Morton, David Howells, Jarkko Sakkinen, James Morris,
Serge E. Hallyn, Linus Torvalds, Joe Perches, Matthew Wilcox,
David Rientjes
Cc: samba-technical, virtualization, linux-mm, linux-sctp,
target-devel, linux-stm32, devel, linux-s390, linux-scsi, x86,
kasan-dev, cocci, linux-wpan, intel-wired-lan, Waiman Long,
linux-crypto, linux-pm, ecryptfs, linux-nfs, linux-fscrypt,
linux-mediatek, linux-amlogic, linux-arm-kernel, linux-cifs,
netdev, linux-wireless, linux-kernel, linux-bluetooth,
linux-security-module, keyrings, tipc-discussion, wireguard,
linux-ppp, linux-integrity, linuxppc-dev, linux-btrfs
In-Reply-To: <20200413211550.8307-1-longman@redhat.com>
Since kfree_sensitive() will do an implicit memzero_explicit(), there
is no need to call memzero_explicit() before it. Eliminate those
memzero_explicit() and simplify the call sites.
Signed-off-by: Waiman Long <longman@redhat.com>
---
.../crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c | 15 +++------------
.../crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c | 16 +++-------------
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 10 ++--------
drivers/crypto/inside-secure/safexcel_hash.c | 3 +--
4 files changed, 9 insertions(+), 35 deletions(-)
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
index aa4e8fdc2b32..46c10c7ca6d0 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
@@ -366,10 +366,7 @@ void sun8i_ce_cipher_exit(struct crypto_tfm *tfm)
{
struct sun8i_cipher_tfm_ctx *op = crypto_tfm_ctx(tfm);
- if (op->key) {
- memzero_explicit(op->key, op->keylen);
- kfree(op->key);
- }
+ kfree_sensitive(op->key);
crypto_free_sync_skcipher(op->fallback_tfm);
pm_runtime_put_sync_suspend(op->ce->dev);
}
@@ -391,10 +388,7 @@ int sun8i_ce_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
dev_dbg(ce->dev, "ERROR: Invalid keylen %u\n", keylen);
return -EINVAL;
}
- if (op->key) {
- memzero_explicit(op->key, op->keylen);
- kfree(op->key);
- }
+ kfree_sensitive(op->key);
op->keylen = keylen;
op->key = kmemdup(key, keylen, GFP_KERNEL | GFP_DMA);
if (!op->key)
@@ -416,10 +410,7 @@ int sun8i_ce_des3_setkey(struct crypto_skcipher *tfm, const u8 *key,
if (err)
return err;
- if (op->key) {
- memzero_explicit(op->key, op->keylen);
- kfree(op->key);
- }
+ free_sensitive(op->key, op->keylen);
op->keylen = keylen;
op->key = kmemdup(key, keylen, GFP_KERNEL | GFP_DMA);
if (!op->key)
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
index 5246ef4f5430..7e09a923cbaf 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
@@ -249,7 +249,6 @@ static int sun8i_ss_cipher(struct skcipher_request *areq)
offset = areq->cryptlen - ivsize;
if (rctx->op_dir & SS_DECRYPTION) {
memcpy(areq->iv, backup_iv, ivsize);
- memzero_explicit(backup_iv, ivsize);
kfree_sensitive(backup_iv);
} else {
scatterwalk_map_and_copy(areq->iv, areq->dst, offset,
@@ -367,10 +366,7 @@ void sun8i_ss_cipher_exit(struct crypto_tfm *tfm)
{
struct sun8i_cipher_tfm_ctx *op = crypto_tfm_ctx(tfm);
- if (op->key) {
- memzero_explicit(op->key, op->keylen);
- kfree(op->key);
- }
+ kfree_sensitive(op->key);
crypto_free_sync_skcipher(op->fallback_tfm);
pm_runtime_put_sync(op->ss->dev);
}
@@ -392,10 +388,7 @@ int sun8i_ss_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
dev_dbg(ss->dev, "ERROR: Invalid keylen %u\n", keylen);
return -EINVAL;
}
- if (op->key) {
- memzero_explicit(op->key, op->keylen);
- kfree(op->key);
- }
+ kfree_sensitive(op->key);
op->keylen = keylen;
op->key = kmemdup(key, keylen, GFP_KERNEL | GFP_DMA);
if (!op->key)
@@ -418,10 +411,7 @@ int sun8i_ss_des3_setkey(struct crypto_skcipher *tfm, const u8 *key,
return -EINVAL;
}
- if (op->key) {
- memzero_explicit(op->key, op->keylen);
- kfree(op->key);
- }
+ kfree_sensitive(op->key);
op->keylen = keylen;
op->key = kmemdup(key, keylen, GFP_KERNEL | GFP_DMA);
if (!op->key)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index fd1269900d67..f424397fbba4 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -341,10 +341,7 @@ void meson_cipher_exit(struct crypto_tfm *tfm)
{
struct meson_cipher_tfm_ctx *op = crypto_tfm_ctx(tfm);
- if (op->key) {
- memzero_explicit(op->key, op->keylen);
- kfree(op->key);
- }
+ kfree_sensitive(op->key)
crypto_free_sync_skcipher(op->fallback_tfm);
}
@@ -368,10 +365,7 @@ int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
dev_dbg(mc->dev, "ERROR: Invalid keylen %u\n", keylen);
return -EINVAL;
}
- if (op->key) {
- memzero_explicit(op->key, op->keylen);
- kfree(op->key);
- }
+ kfree_sensitive(op->key);
op->keylen = keylen;
op->key = kmemdup(key, keylen, GFP_KERNEL | GFP_DMA);
if (!op->key)
diff --git a/drivers/crypto/inside-secure/safexcel_hash.c b/drivers/crypto/inside-secure/safexcel_hash.c
index 43962bc709c6..4a2d162914de 100644
--- a/drivers/crypto/inside-secure/safexcel_hash.c
+++ b/drivers/crypto/inside-secure/safexcel_hash.c
@@ -1081,8 +1081,7 @@ static int safexcel_hmac_init_pad(struct ahash_request *areq,
}
/* Avoid leaking */
- memzero_explicit(keydup, keylen);
- kfree(keydup);
+ kfree_sensitive(keydup);
if (ret)
return ret;
--
2.18.1
^ permalink raw reply related
* Re: [PATCH 2/2] crypto: Remove unnecessary memzero_explicit()
From: Joe Perches @ 2020-04-13 21:31 UTC (permalink / raw)
To: Waiman Long, Andrew Morton, David Howells, Jarkko Sakkinen,
James Morris, Serge E. Hallyn, Linus Torvalds, Matthew Wilcox,
David Rientjes
Cc: samba-technical, virtualization, linux-mm, linux-sctp,
target-devel, linux-stm32, devel, linux-s390, linux-scsi, x86,
kasan-dev, cocci, linux-wpan, intel-wired-lan, linux-crypto,
linux-pm, ecryptfs, linux-nfs, linux-fscrypt, linux-mediatek,
linux-amlogic, linux-arm-kernel, linux-cifs, netdev,
linux-wireless, linux-kernel, linux-bluetooth,
linux-security-module, keyrings, tipc-discussion, wireguard,
linux-ppp, linux-integrity, linuxppc-dev, linux-btrfs
In-Reply-To: <20200413211550.8307-3-longman@redhat.com>
On Mon, 2020-04-13 at 17:15 -0400, Waiman Long wrote:
> Since kfree_sensitive() will do an implicit memzero_explicit(), there
> is no need to call memzero_explicit() before it. Eliminate those
> memzero_explicit() and simplify the call sites.
2 bits of trivia:
> diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
[]
> @@ -391,10 +388,7 @@ int sun8i_ce_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
> dev_dbg(ce->dev, "ERROR: Invalid keylen %u\n", keylen);
> return -EINVAL;
> }
> - if (op->key) {
> - memzero_explicit(op->key, op->keylen);
> - kfree(op->key);
> - }
> + kfree_sensitive(op->key);
> op->keylen = keylen;
> op->key = kmemdup(key, keylen, GFP_KERNEL | GFP_DMA);
> if (!op->key)
It might be a defect to set op->keylen before the kmemdup succeeds.
> @@ -416,10 +410,7 @@ int sun8i_ce_des3_setkey(struct crypto_skcipher *tfm, const u8 *key,
> if (err)
> return err;
>
> - if (op->key) {
> - memzero_explicit(op->key, op->keylen);
> - kfree(op->key);
> - }
> + free_sensitive(op->key, op->keylen);
Why not kfree_sensitive(op->key) ?
^ permalink raw reply
* [PATCH] kvm_host: unify VM_STAT and VCPU_STAT definitions in a single place
From: Emanuele Giuseppe Esposito @ 2020-04-13 14:03 UTC (permalink / raw)
To: kvm
Cc: Wanpeng Li, David Hildenbrand, linux-mips, kvmarm,
Emanuele Giuseppe Esposito, linux-s390, Janosch Frank,
Marc Zyngier, Joerg Roedel, Christian Borntraeger, Julien Thierry,
Suzuki K Poulose, kvm-ppc, linux-arm-kernel, Jim Mattson,
Cornelia Huck, linux-kernel, Sean Christopherson, James Morse,
Paolo Bonzini, Vitaly Kuznetsov, linuxppc-dev
The macros VM_STAT and VCPU_STAT are redundantly implemented in multiple
files, each used by a different architecure to initialize the debugfs
entries for statistics. Since they all have the same purpose, they can be
unified in a single common definition in include/linux/kvm_host.h
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
---
arch/arm64/kvm/guest.c | 23 +++++++--------
arch/mips/kvm/mips.c | 61 +++++++++++++++++++--------------------
arch/powerpc/kvm/book3s.c | 3 --
arch/powerpc/kvm/booke.c | 3 --
arch/s390/kvm/kvm-s390.c | 3 --
arch/x86/kvm/x86.c | 3 --
include/linux/kvm_host.h | 3 ++
7 files changed, 43 insertions(+), 56 deletions(-)
diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index 23ebe51410f0..3e3aee8b37c0 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -29,20 +29,17 @@
#include "trace.h"
-#define VM_STAT(x) { #x, offsetof(struct kvm, stat.x), KVM_STAT_VM }
-#define VCPU_STAT(x) { #x, offsetof(struct kvm_vcpu, stat.x), KVM_STAT_VCPU }
-
struct kvm_stats_debugfs_item debugfs_entries[] = {
- VCPU_STAT(halt_successful_poll),
- VCPU_STAT(halt_attempted_poll),
- VCPU_STAT(halt_poll_invalid),
- VCPU_STAT(halt_wakeup),
- VCPU_STAT(hvc_exit_stat),
- VCPU_STAT(wfe_exit_stat),
- VCPU_STAT(wfi_exit_stat),
- VCPU_STAT(mmio_exit_user),
- VCPU_STAT(mmio_exit_kernel),
- VCPU_STAT(exits),
+ { "halt_successful_poll", VCPU_STAT(halt_successful_poll) },
+ { "halt_attempted_poll", VCPU_STAT(halt_attempted_poll) },
+ { "halt_poll_invalid", VCPU_STAT(halt_poll_invalid) },
+ { "halt_wakeup", VCPU_STAT(halt_wakeup) },
+ { "hvc_exit_stat", VCPU_STAT(hvc_exit_stat) },
+ { "wfe_exit_stat", VCPU_STAT(wfe_exit_stat) },
+ { "wfi_exit_stat", VCPU_STAT(wfi_exit_stat) },
+ { "mmio_exit_user", VCPU_STAT(mmio_exit_user) },
+ { "mmio_exit_kernel", VCPU_STAT(mmio_exit_kernel) },
+ { "exits", VCPU_STAT(exits) },
{ NULL }
};
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index 8f05dd0a0f4e..f14b93d02f02 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -39,40 +39,39 @@
#define VECTORSPACING 0x100 /* for EI/VI mode */
#endif
-#define VCPU_STAT(x) offsetof(struct kvm_vcpu, stat.x)
struct kvm_stats_debugfs_item debugfs_entries[] = {
- { "wait", VCPU_STAT(wait_exits), KVM_STAT_VCPU },
- { "cache", VCPU_STAT(cache_exits), KVM_STAT_VCPU },
- { "signal", VCPU_STAT(signal_exits), KVM_STAT_VCPU },
- { "interrupt", VCPU_STAT(int_exits), KVM_STAT_VCPU },
- { "cop_unusable", VCPU_STAT(cop_unusable_exits), KVM_STAT_VCPU },
- { "tlbmod", VCPU_STAT(tlbmod_exits), KVM_STAT_VCPU },
- { "tlbmiss_ld", VCPU_STAT(tlbmiss_ld_exits), KVM_STAT_VCPU },
- { "tlbmiss_st", VCPU_STAT(tlbmiss_st_exits), KVM_STAT_VCPU },
- { "addrerr_st", VCPU_STAT(addrerr_st_exits), KVM_STAT_VCPU },
- { "addrerr_ld", VCPU_STAT(addrerr_ld_exits), KVM_STAT_VCPU },
- { "syscall", VCPU_STAT(syscall_exits), KVM_STAT_VCPU },
- { "resvd_inst", VCPU_STAT(resvd_inst_exits), KVM_STAT_VCPU },
- { "break_inst", VCPU_STAT(break_inst_exits), KVM_STAT_VCPU },
- { "trap_inst", VCPU_STAT(trap_inst_exits), KVM_STAT_VCPU },
- { "msa_fpe", VCPU_STAT(msa_fpe_exits), KVM_STAT_VCPU },
- { "fpe", VCPU_STAT(fpe_exits), KVM_STAT_VCPU },
- { "msa_disabled", VCPU_STAT(msa_disabled_exits), KVM_STAT_VCPU },
- { "flush_dcache", VCPU_STAT(flush_dcache_exits), KVM_STAT_VCPU },
+ { "wait", VCPU_STAT(wait_exits) },
+ { "cache", VCPU_STAT(cache_exits) },
+ { "signal", VCPU_STAT(signal_exits) },
+ { "interrupt", VCPU_STAT(int_exits) },
+ { "cop_unusable", VCPU_STAT(cop_unusable_exits) },
+ { "tlbmod", VCPU_STAT(tlbmod_exits) },
+ { "tlbmiss_ld", VCPU_STAT(tlbmiss_ld_exits) },
+ { "tlbmiss_st", VCPU_STAT(tlbmiss_st_exits) },
+ { "addrerr_st", VCPU_STAT(addrerr_st_exits) },
+ { "addrerr_ld", VCPU_STAT(addrerr_ld_exits) },
+ { "syscall", VCPU_STAT(syscall_exits) },
+ { "resvd_inst", VCPU_STAT(resvd_inst_exits) },
+ { "break_inst", VCPU_STAT(break_inst_exits) },
+ { "trap_inst", VCPU_STAT(trap_inst_exits) },
+ { "msa_fpe", VCPU_STAT(msa_fpe_exits) },
+ { "fpe", VCPU_STAT(fpe_exits) },
+ { "msa_disabled", VCPU_STAT(msa_disabled_exits) },
+ { "flush_dcache", VCPU_STAT(flush_dcache_exits) },
#ifdef CONFIG_KVM_MIPS_VZ
- { "vz_gpsi", VCPU_STAT(vz_gpsi_exits), KVM_STAT_VCPU },
- { "vz_gsfc", VCPU_STAT(vz_gsfc_exits), KVM_STAT_VCPU },
- { "vz_hc", VCPU_STAT(vz_hc_exits), KVM_STAT_VCPU },
- { "vz_grr", VCPU_STAT(vz_grr_exits), KVM_STAT_VCPU },
- { "vz_gva", VCPU_STAT(vz_gva_exits), KVM_STAT_VCPU },
- { "vz_ghfc", VCPU_STAT(vz_ghfc_exits), KVM_STAT_VCPU },
- { "vz_gpa", VCPU_STAT(vz_gpa_exits), KVM_STAT_VCPU },
- { "vz_resvd", VCPU_STAT(vz_resvd_exits), KVM_STAT_VCPU },
+ { "vz_gpsi", VCPU_STAT(vz_gpsi_exits) },
+ { "vz_gsfc", VCPU_STAT(vz_gsfc_exits) },
+ { "vz_hc", VCPU_STAT(vz_hc_exits) },
+ { "vz_grr", VCPU_STAT(vz_grr_exits) },
+ { "vz_gva", VCPU_STAT(vz_gva_exits) },
+ { "vz_ghfc", VCPU_STAT(vz_ghfc_exits) },
+ { "vz_gpa", VCPU_STAT(vz_gpa_exits) },
+ { "vz_resvd", VCPU_STAT(vz_resvd_exits) },
#endif
- { "halt_successful_poll", VCPU_STAT(halt_successful_poll), KVM_STAT_VCPU },
- { "halt_attempted_poll", VCPU_STAT(halt_attempted_poll), KVM_STAT_VCPU },
- { "halt_poll_invalid", VCPU_STAT(halt_poll_invalid), KVM_STAT_VCPU },
- { "halt_wakeup", VCPU_STAT(halt_wakeup), KVM_STAT_VCPU },
+ { "halt_successful_poll", VCPU_STAT(halt_successful_poll) },
+ { "halt_attempted_poll", VCPU_STAT(halt_attempted_poll) },
+ { "halt_poll_invalid", VCPU_STAT(halt_poll_invalid) },
+ { "halt_wakeup", VCPU_STAT(halt_wakeup) },
{NULL}
};
diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c
index 5690a1f9b976..55cb728ba06e 100644
--- a/arch/powerpc/kvm/book3s.c
+++ b/arch/powerpc/kvm/book3s.c
@@ -36,9 +36,6 @@
#include "book3s.h"
#include "trace.h"
-#define VM_STAT(x, ...) offsetof(struct kvm, stat.x), KVM_STAT_VM, ## __VA_ARGS__
-#define VCPU_STAT(x, ...) offsetof(struct kvm_vcpu, stat.x), KVM_STAT_VCPU, ## __VA_ARGS__
-
/* #define EXIT_DEBUG */
struct kvm_stats_debugfs_item debugfs_entries[] = {
diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c
index 6c18ea88fd25..fb8fa7060804 100644
--- a/arch/powerpc/kvm/booke.c
+++ b/arch/powerpc/kvm/booke.c
@@ -35,9 +35,6 @@
unsigned long kvmppc_booke_handlers;
-#define VM_STAT(x) offsetof(struct kvm, stat.x), KVM_STAT_VM
-#define VCPU_STAT(x) offsetof(struct kvm_vcpu, stat.x), KVM_STAT_VCPU
-
struct kvm_stats_debugfs_item debugfs_entries[] = {
{ "mmio", VCPU_STAT(mmio_exits) },
{ "sig", VCPU_STAT(signal_exits) },
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 19a81024fe16..1a7bf8759750 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -57,9 +57,6 @@
#define VCPU_IRQS_MAX_BUF (sizeof(struct kvm_s390_irq) * \
(KVM_MAX_VCPUS + LOCAL_IRQS))
-#define VCPU_STAT(x) offsetof(struct kvm_vcpu, stat.x), KVM_STAT_VCPU
-#define VM_STAT(x) offsetof(struct kvm, stat.x), KVM_STAT_VM
-
struct kvm_stats_debugfs_item debugfs_entries[] = {
{ "userspace_handled", VCPU_STAT(exit_userspace) },
{ "exit_null", VCPU_STAT(exit_null) },
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index b8124b562dea..fb035d304004 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -97,9 +97,6 @@ static u64 __read_mostly efer_reserved_bits = ~((u64)EFER_SCE);
static u64 __read_mostly cr4_reserved_bits = CR4_RESERVED_BITS;
-#define VM_STAT(x, ...) offsetof(struct kvm, stat.x), KVM_STAT_VM, ## __VA_ARGS__
-#define VCPU_STAT(x, ...) offsetof(struct kvm_vcpu, stat.x), KVM_STAT_VCPU, ## __VA_ARGS__
-
#define KVM_X2APIC_API_VALID_FLAGS (KVM_X2APIC_API_USE_32BIT_IDS | \
KVM_X2APIC_API_DISABLE_BROADCAST_QUIRK)
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 6d58beb65454..e02d38c7fff1 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -1130,6 +1130,9 @@ struct kvm_stats_debugfs_item {
#define KVM_DBGFS_GET_MODE(dbgfs_item) \
((dbgfs_item)->mode ? (dbgfs_item)->mode : 0644)
+#define VM_STAT(x, ...) offsetof(struct kvm, stat.x), KVM_STAT_VM, ## __VA_ARGS__
+#define VCPU_STAT(x, ...) offsetof(struct kvm_vcpu, stat.x), KVM_STAT_VCPU, ## __VA_ARGS__
+
extern struct kvm_stats_debugfs_item debugfs_entries[];
extern struct dentry *kvm_debugfs_dir;
--
2.17.1
^ permalink raw reply related
* Re: [PATCH] kvm_host: unify VM_STAT and VCPU_STAT definitions in a single place
From: Philippe Mathieu-Daudé @ 2020-04-13 21:34 UTC (permalink / raw)
To: Emanuele Giuseppe Esposito, kvm
Cc: Wanpeng Li, David Hildenbrand, linux-mips, kvmarm, linux-s390,
Janosch Frank, Marc Zyngier, Joerg Roedel, Christian Borntraeger,
Julien Thierry, Suzuki K Poulose, kvm-ppc, linux-arm-kernel,
Jim Mattson, Cornelia Huck, linux-kernel, Sean Christopherson,
James Morse, Paolo Bonzini, Vitaly Kuznetsov, linuxppc-dev
In-Reply-To: <20200413140332.22896-1-eesposit@redhat.com>
Hi Emanuele,
On 4/13/20 4:03 PM, Emanuele Giuseppe Esposito wrote:
> The macros VM_STAT and VCPU_STAT are redundantly implemented in multiple
> files, each used by a different architecure to initialize the debugfs
> entries for statistics. Since they all have the same purpose, they can be
> unified in a single common definition in include/linux/kvm_host.h
>
> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> ---
> arch/arm64/kvm/guest.c | 23 +++++++--------
> arch/mips/kvm/mips.c | 61 +++++++++++++++++++--------------------
> arch/powerpc/kvm/book3s.c | 3 --
> arch/powerpc/kvm/booke.c | 3 --
> arch/s390/kvm/kvm-s390.c | 3 --
> arch/x86/kvm/x86.c | 3 --
> include/linux/kvm_host.h | 3 ++
> 7 files changed, 43 insertions(+), 56 deletions(-)
>
> diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
> index 23ebe51410f0..3e3aee8b37c0 100644
> --- a/arch/arm64/kvm/guest.c
> +++ b/arch/arm64/kvm/guest.c
> @@ -29,20 +29,17 @@
>
> #include "trace.h"
>
> -#define VM_STAT(x) { #x, offsetof(struct kvm, stat.x), KVM_STAT_VM }
> -#define VCPU_STAT(x) { #x, offsetof(struct kvm_vcpu, stat.x), KVM_STAT_VCPU }
> -
> struct kvm_stats_debugfs_item debugfs_entries[] = {
> - VCPU_STAT(halt_successful_poll),
> - VCPU_STAT(halt_attempted_poll),
> - VCPU_STAT(halt_poll_invalid),
> - VCPU_STAT(halt_wakeup),
> - VCPU_STAT(hvc_exit_stat),
> - VCPU_STAT(wfe_exit_stat),
> - VCPU_STAT(wfi_exit_stat),
> - VCPU_STAT(mmio_exit_user),
> - VCPU_STAT(mmio_exit_kernel),
> - VCPU_STAT(exits),
> + { "halt_successful_poll", VCPU_STAT(halt_successful_poll) },
> + { "halt_attempted_poll", VCPU_STAT(halt_attempted_poll) },
> + { "halt_poll_invalid", VCPU_STAT(halt_poll_invalid) },
> + { "halt_wakeup", VCPU_STAT(halt_wakeup) },
> + { "hvc_exit_stat", VCPU_STAT(hvc_exit_stat) },
> + { "wfe_exit_stat", VCPU_STAT(wfe_exit_stat) },
> + { "wfi_exit_stat", VCPU_STAT(wfi_exit_stat) },
> + { "mmio_exit_user", VCPU_STAT(mmio_exit_user) },
> + { "mmio_exit_kernel", VCPU_STAT(mmio_exit_kernel) },
> + { "exits", VCPU_STAT(exits) },
> { NULL }
> };
>
[...]
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index 6d58beb65454..e02d38c7fff1 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -1130,6 +1130,9 @@ struct kvm_stats_debugfs_item {
> #define KVM_DBGFS_GET_MODE(dbgfs_item) \
> ((dbgfs_item)->mode ? (dbgfs_item)->mode : 0644)
>
> +#define VM_STAT(x, ...) offsetof(struct kvm, stat.x), KVM_STAT_VM, ## __VA_ARGS__
> +#define VCPU_STAT(x, ...) offsetof(struct kvm_vcpu, stat.x), KVM_STAT_VCPU, ## __VA_ARGS__
I find this macro expanding into multiple fields odd... Maybe a matter
of taste. Sugggestion, have the macro define the full structure, as in
the arm64 arch:
#define VM_STAT(n, x, ...) { n, offsetof(struct kvm, stat.x),
KVM_STAT_VM, ## __VA_ARGS__ }
Ditto for VCPU_STAT().
> +
> extern struct kvm_stats_debugfs_item debugfs_entries[];
> extern struct dentry *kvm_debugfs_dir;
>
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox