* [PATCH v28 01/12] Linux Random Number Generator
From: Stephan Müller @ 2020-01-15 10:31 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Greg Kroah-Hartman, linux-crypto-u79uwXL29TY76Z2rM5mHXA, LKML,
linux-api-u79uwXL29TY76Z2rM5mHXA, Eric W. Biederman,
Alexander E. Patrakov, Ahmed S. Darwish, Theodore Y. Ts'o,
Willy Tarreau, Matthew Garrett, Vito Caputo, Andreas Dilger,
Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange, "Peter, Matthias" <matth>
In-Reply-To: <5951792.lmNsirYsPE-jJGQKZiSfeo1haGO/jJMPxvVK+yQ3ZXh@public.gmane.org>
In an effort to provide a flexible implementation for a random number
generator that also delivers entropy during early boot time, allows
replacement of the deterministic random number generation mechanism,
implement the various components in separate code for easier
maintenance, and provide compliance to SP800-90[A|B|C], introduce
the Linux Random Number Generator (LRNG) framework.
The general design is as follows. Additional implementation details
are given in [1]. The LRNG consists of the following components:
1. The LRNG implements a DRNG. The DRNG always generates the
requested amount of output. When using the SP800-90A terminology
it operates without prediction resistance. The secondary DRNG
maintains a counter of how many bytes were generated since last
re-seed and a timer of the elapsed time since last re-seed. If either
the counter or the timer reaches a threshold, the secondary DRNG is
seeded from the entropy pool.
In case the Linux kernel detects a NUMA system, one secondary DRNG
instance per NUMA node is maintained.
2. The DRNG is seeded by concatenating the data from the
following sources:
(a) the output of the entropy pool,
(b) the Jitter RNG if available and enabled, and
(c) the CPU-based noise source such as Intel RDRAND if available and
enabled.
The entropy estimate of the data of all noise sources are added to
form the entropy estimate of the data used to seed the DRNG with.
The LRNG ensures, however, that the DRNG after seeding is at
maximum the security strength of the DRNG.
The LRNG is designed such that none of these noise sources can dominate
the other noise sources to provide seed data to the DRNG during due to
the following:
(a) During boot time, the amount of received interrupts are the trigger
points to (re)seed the DRNG.
(b) At runtime, the available entropy from the slow noise source is
concatenated with a pre-defined amount of data from the fast noise
sources. In addition, each DRNG reseed operation triggers external
noise source providers to deliver one block of data.
3. The entropy pool accumulates entropy obtained from certain events,
which will henceforth be collectively called "slow noise sources".
The entropy pool collects noise data from slow noise sources. Any data
received by the LRNG from the slow noise sources is inserted into the
entropy pool using an LFSR with a primitive and irreducible polynomial.
The following sources of entropy are used:
(a) When an interrupt occurs, the high-resolution time stamp is mixed
into the LFSR. This time stamp is credited with heuristically implied
entropy.
(b) HID event data like the key stroke or the mouse coordinates are
mixed into the LFSR. This data is not credited with entropy by the LRNG.
(c) Device drivers may provide data that is mixed into the LFSR. This
data is not credited with entropy by the LRNG.
(d) After the entropy pool is ``read'' by the DRNG, the data
used to seed the DRNG is mixed back into the entropy pool to
stir the pool. This data is not credited with entropy by the LRNG.
Any data provided from user space by either writing to /dev/random,
/dev/urandom or the IOCTL of RNDADDENTROPY on both device files
are always injected into the entropy pool.
In addition, when a hardware random number generator covered by the
Linux kernel HW generator framework wants to deliver random numbers,
it is injected into the entropy pool as well. HW generator noise source
is handled separately from the other noise source due to the fact that
the HW generator framework may decide by itself when to deliver data
whereas the other noise sources always requested for data driven by the
LRNG operation. Similarly any user space provided data is inserted into
the entropy pool.
When the DRNG requires data from the entropy pool, the entire
entropy pool is processed with an SP800-90A section 10.3.1 compliant
hash_df function to generate random numbers.
To speed up the interrupt handling code of the LRNG, the time stamp
collected for an interrupt event is truncated to the 8 least
significant bits. 64 truncated time stamps are concatenated and then
jointly inserted into the LFSR. During boot time, until the fully seeded
stage is reached, each time stamp with its 32 least significant bits is
inserted into the LFSR at the time of arrival.
The LRNG allows the DRNG mechanism to be changed at runtime. Per default,
a ChaCha20-based DRNG is used. The ChaCha20-DRNG implemented for the
LRNG is also provided as a stand-alone user space deterministic random
number generator. The LRNG also offers an SP800-90A DRBG based on the
Linux kernel crypto API DRBG implementation.
The processing of entropic data from the noise source before injecting
them into the DRNG is performed with the following mathematical
operations:
1. LFSR: The 8 least significant bits of the time stamp data received
from the interrupts are processed with an LFSR. That LFSR is implemented
identically to the LSFR used in the existing /dev/random implementation
except that it is capable of processing an entire word and that a
different polynomial is used. The reason for the different polynomial
is performance in a performance sensitive code section, the interrupt
handler. The chosen polynomials have 4 taps. Also, this LFSR-approach
is used in the OpenBSD /dev/random equivalent.
2. Concatenation: The temporary seed buffer used to seed the DRNG is
a concatenation of parts of the entropy pool data, and the CPU noise
source output.
The DRNG always tries to seed itself with 256 bits of entropy, except
during boot. In any case, if the noise sources cannot deliver that
amount, the available entropy is used and the DRNG keeps track on how
much entropy it was seeded with. The entropy implied by the LRNG
available in the entropy pool may be too conservative. To ensure
that during boot time all available entropy from the entropy pool is
transferred to the DRNG, the hash_df function always generates 256
data bits during boot to seed the DRNG. During boot, the DRNG is
seeded as follows:
1. The DRNG is reseeded from the entropy pool and potentially the fast
noise sources if the entropy pool has collected at least 32 bits of
entropy from the interrupt noise source. The goal of this step is to
ensure that the DRNG receives some initial entropy as early as
possible. In addition it receives the entropy available from
the fast noise sources.
2. The DRNG is reseeded from the entropy pool and potentially the fast
noise sources if all noise sources collectively can provide at least
128 bits of entropy.
3. The DRNG is reseeded from the entropy pool and potentially the fast
noise sources if all noise sources collectivel can provide at least 256
bits.
At the time of the reseeding steps, the DRNG requests as much entropy as
is available in order to skip certain steps and reach the seeding level
of 256 bits. This may imply that one or more of the aforementioned steps
are skipped.
In all listed steps, the DRNG is (re)seeded with a number of random
bytes from the entropy pool that is at most the amount of entropy
present in the entropy pool. This means that when the entropy pool
contains 128 or 256 bits of entropy, the DRNG is seeded with that
amount of entropy as well.
Before the DRNG is seeded with 256 bits of entropy in step 3,
requests of random data from /dev/random are not processed.
The hash_df operation providing random data from the entropy pool will
always require that all entropy sources collectively can deliver at
least 129 entropy bits as configured with (128 bits of entropy for
seeding plus one bit of entropy that is lost with the post
processing as defined in SP800-90B).
The DRNG operates as deterministic random number generator with the
following properties:
* The maximum number of random bytes that can be generated with one
DRNG generate operation is limited to 4096 bytes. When longer random
numbers are requested, multiple DRNG generate operations are performed.
The ChaCha20 DRNG as well as the SP800-90A DRBGs implement an update of
their state after completing a generate request for backtracking
resistance.
* The secondary DRNG is reseeded with whatever entropy is available –
in the worst case where no additional entropy can be provided by the
noise sources, the DRNG is not re-seeded and continues its operation
to try to reseed again after again the expiry of one of these thresholds:
- If the last reseeding of the secondary DRNG is more than 600 seconds
ago, or
- 2^20 DRNG generate operations are performed, whatever comes first, or
- the secondary DRNG is forced to reseed before the next generation of
random numbers if data has been injected into the LRNG by writing data
into /dev/random or /dev/urandom.
The chosen values prevent high-volume requests from user space to cause
frequent reseeding operations which drag down the performance of the
DRNG.
With the automatic reseeding after 600 seconds, the LRNG is triggered
to reseed itself before the first request after a suspend that put the
hardware to sleep for longer than 600 seconds.
The LRNG uses the following runtime memory using the currently
smallest configuration:
* 576 bytes (512 bytes for the entropy pool and 64 for the entropy pool
meta data) for the entropy pool management
* 64 bytes per CPU for the time stamp array
To support smaller devices including IoT environments, this patch
allows reducing the runtime memory footprint of the LRNG at compile
time by selecting smaller entropy pool sizes.
The entropy pool has support for sizes of 256, 128 and 64 bytes supported
by primitive and irreducible polynomials.
The time stamp array is reduced to one atomic_t variable per CPU, i.e.
4 bytes when CONFIG_BASE_SMALL is selected during kernel
configuration. This implies that after the receipt of 4 interrupts on
one CPU, the data is injected into the LFSR. Depending on the behavior
of the CPU caches, this may imply that the average interrupt handler
execution time increases a bit, since instead of injecting 8 atomic_t
values at one given time into the LFSR, only one is processed which
may incur cache misses.
When selecting the compilation of a kernel for a small environment,
prevent the allocation of a buffer up to 4096 bytes to serve user space
requests. In this case, the stack variable of 64 bytes is used to serve
all user space requests.
The LRNG has the following properties:
* internal noise source: interrupts timing with fast boot time seeding
* high performance of interrupt handling code: The LRNG impact on the
interrupt handling has been reduced to a minimum. On one example
system, the LRNG interrupt handling code executes within an average
of 65 cycles whereas the existing /dev/random on the same device
takes about 97 cycles when measuring the execution time of
add_interrupt_randomness().
* lockless LFSR to collect raw entropy supporing concurrency-free
use of massive parallel systems
* use of standalone ChaCha20 based RNG with the option to use a
different DRNG selectable at compile time
* "atomic" seeding of secondary DRBG to ensure full entropy transport
* instantiate one DRNG per NUMA node
* support for runtime switchable output DRNGs
* use of only well-defined entropy-preserving operations to collect,
compress and forward entropy: concatenation, LFSR, SP800-90A hash_df
function
* compile-time selectable entropy pool size: the choice also
uses the applicable LFSR polynomial to maintain the entropy pool
size
* support of small systems by allowing the reduction of the
runtime memory needs
Further details including the rationale for the design choices and
properties of the LRNG together with testing is provided at [1].
In addition, the documentation explains the conducted regression
tests to verify that the LRNG is API and ABI compatible with the
existing /dev/random implementation.
[1] https://www.chronox.de/lrng.html
CC: "Eric W. Biederman" <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
CC: "Alexander E. Patrakov" <patrakov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
CC: "Ahmed S. Darwish" <darwish.07-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
CC: "Theodore Y. Ts'o" <tytso-3s7WtUTddSA@public.gmane.org>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59-1xO5oi07KQx4cg9Nei1l7Q@public.gmane.org>
CC: Vito Caputo <vcaputo-IiWei5kqaphBDgjK7y7TUQ@public.gmane.org>
CC: Andreas Dilger <adilger.kernel-m1MBpc4rdrD3fQ9qLvQP4Q@public.gmane.org>
CC: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
CC: Ray Strode <rstrode-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
CC: William Jon McCann <mccann-4GNroTWusrE@public.gmane.org>
CC: zhangjs <zachary-+jQ8zvd9CV5SzTKKI/wJ2A@public.gmane.org>
CC: Andy Lutomirski <luto-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
CC: Florian Weimer <fweimer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
CC: Lennart Poettering <mzxreary-uLTowLwuiw4b1SvskN2V4Q@public.gmane.org>
CC: Nicolai Stange <nstange-l3A5Bk7waGM@public.gmane.org>
Mathematical aspects Reviewed-by: "Peter, Matthias" <matthias.peter-dgh0Zkf8HWc@public.gmane.orgd.de>
Reviewed-by: Marcelo Henrique Cerri <marcelo.cerri-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
Reviewed-by: Roman Drahtmueller <draht-UTA6Eu5RHcbyVfKhlJ+g5A@public.gmane.org>
Tested-by: Roman Drahtmüller <draht-UTA6Eu5RHcbyVfKhlJ+g5A@public.gmane.org>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
Tested-by: Neil Horman <nhorman-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
---
MAINTAINERS | 7 +
drivers/char/Kconfig | 2 +
drivers/char/Makefile | 9 +-
drivers/char/lrng/Kconfig | 67 +++
drivers/char/lrng/Makefile | 9 +
drivers/char/lrng/lrng_archrandom.c | 94 ++++
drivers/char/lrng/lrng_aux.c | 148 +++++++
drivers/char/lrng/lrng_chacha20.c | 265 ++++++++++++
drivers/char/lrng/lrng_chacha20.h | 25 ++
drivers/char/lrng/lrng_drng.c | 400 +++++++++++++++++
drivers/char/lrng/lrng_interfaces.c | 638 ++++++++++++++++++++++++++++
drivers/char/lrng/lrng_internal.h | 296 +++++++++++++
drivers/char/lrng/lrng_lfsr.h | 152 +++++++
drivers/char/lrng/lrng_pool.c | 588 +++++++++++++++++++++++++
drivers/char/lrng/lrng_sw_noise.c | 102 +++++
drivers/char/lrng/lrng_sw_noise.h | 57 +++
include/linux/lrng.h | 63 +++
17 files changed, 2921 insertions(+), 1 deletion(-)
create mode 100644 drivers/char/lrng/Kconfig
create mode 100644 drivers/char/lrng/Makefile
create mode 100644 drivers/char/lrng/lrng_archrandom.c
create mode 100644 drivers/char/lrng/lrng_aux.c
create mode 100644 drivers/char/lrng/lrng_chacha20.c
create mode 100644 drivers/char/lrng/lrng_chacha20.h
create mode 100644 drivers/char/lrng/lrng_drng.c
create mode 100644 drivers/char/lrng/lrng_interfaces.c
create mode 100644 drivers/char/lrng/lrng_internal.h
create mode 100644 drivers/char/lrng/lrng_lfsr.h
create mode 100644 drivers/char/lrng/lrng_pool.c
create mode 100644 drivers/char/lrng/lrng_sw_noise.c
create mode 100644 drivers/char/lrng/lrng_sw_noise.h
create mode 100644 include/linux/lrng.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 8982c6e013b3..f9beac15783c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9606,6 +9606,13 @@ F: Documentation/core-api/atomic_ops.rst
F: Documentation/core-api/refcount-vs-atomic.rst
F: Documentation/memory-barriers.txt
+LINUX RANDOM NUMBER GENERATOR (LRNG) DRIVER
+M: Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+S: Maintained
+W: https://www.chronox.de/lrng.html
+F: drivers/char/lrng/*
+F: include/linux/lrng.h
+
LIS3LV02D ACCELEROMETER DRIVER
M: Eric Piel <eric.piel-VkQ1JFuSMpfAbQlEx87xDw@public.gmane.org>
S: Maintained
diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig
index 26956c006987..1e758735f633 100644
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -535,6 +535,8 @@ config ADI
and SSM (Silicon Secured Memory). Intended consumers of this
driver include crash and makedumpfile.
+source "drivers/char/lrng/Kconfig"
+
endmenu
config RANDOM_TRUST_CPU
diff --git a/drivers/char/Makefile b/drivers/char/Makefile
index 7c5ea6f9df14..46ede09fd6d3 100644
--- a/drivers/char/Makefile
+++ b/drivers/char/Makefile
@@ -3,7 +3,14 @@
# Makefile for the kernel character device drivers.
#
-obj-y += mem.o random.o
+obj-y += mem.o
+
+ifeq ($(CONFIG_LRNG),y)
+ obj-y += lrng/
+else
+ obj-y += random.o
+endif
+
obj-$(CONFIG_TTY_PRINTK) += ttyprintk.o
obj-y += misc.o
obj-$(CONFIG_ATARI_DSP56K) += dsp56k.o
diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
new file mode 100644
index 000000000000..56f13efd3592
--- /dev/null
+++ b/drivers/char/lrng/Kconfig
@@ -0,0 +1,67 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Linux Random Number Generator configuration
+#
+
+menuconfig LRNG
+ bool "Linux Random Number Generator"
+ help
+ The Linux Random Number Generator (LRNG) is the replacement
+ of the existing /dev/random provided with drivers/char/random.c.
+ It generates entropy from different noise sources and
+ delivers significant entropy during boot.
+
+if LRNG
+
+choice
+ prompt "LRNG Entropy Pool Size"
+ default LRNG_POOL_SIZE_4096
+ help
+ Select the size of the LRNG entropy pool. The size of the
+ entropy pool is relevant for the amount of entropy that
+ the LRNG can maintain as a maximum. The larger the size
+ of the entropy pool is the more entropy can be maintained
+ but the less often older entropic values are overwritten
+ with new entropy.
+
+ config LRNG_POOL_SIZE_512
+ bool "512 bits"
+
+ config LRNG_POOL_SIZE_1024
+ bool "1024 bits"
+
+ config LRNG_POOL_SIZE_2048
+ bool "2048 bits"
+
+ config LRNG_POOL_SIZE_4096
+ bool "4096 bits (default)"
+
+ config LRNG_POOL_SIZE_8192
+ bool "8192 bits"
+
+ config LRNG_POOL_SIZE_16384
+ bool "16384 bits"
+
+ config LRNG_POOL_SIZE_32768
+ bool "32768 bits"
+
+ config LRNG_POOL_SIZE_65536
+ bool "65536 bits"
+
+ config LRNG_POOL_SIZE_131072
+ bool "131072 bits"
+endchoice
+
+config LRNG_POOL_SIZE
+ int
+ default 0 if LRNG_POOL_SIZE_512
+ default 1 if LRNG_POOL_SIZE_1024
+ default 2 if LRNG_POOL_SIZE_2048
+ default 3 if LRNG_POOL_SIZE_4096
+ default 4 if LRNG_POOL_SIZE_8192
+ default 5 if LRNG_POOL_SIZE_16384
+ default 6 if LRNG_POOL_SIZE_32768
+ default 7 if LRNG_POOL_SIZE_65536
+ default 8 if LRNG_POOL_SIZE_131072
+
+endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
new file mode 100644
index 000000000000..1d2a0211973d
--- /dev/null
+++ b/drivers/char/lrng/Makefile
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the Linux Random Number Generator.
+#
+
+obj-y += lrng_pool.o lrng_aux.o \
+ lrng_sw_noise.o lrng_archrandom.o \
+ lrng_drng.o lrng_chacha20.o \
+ lrng_interfaces.o \
diff --git a/drivers/char/lrng/lrng_archrandom.c b/drivers/char/lrng/lrng_archrandom.c
new file mode 100644
index 000000000000..eeba708d025f
--- /dev/null
+++ b/drivers/char/lrng/lrng_archrandom.c
@@ -0,0 +1,94 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG Fast Noise Source: CPU-based noise source
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/random.h>
+
+#include "lrng_internal.h"
+
+/*
+ * Estimated entropy of data is a 32th of LRNG_DRNG_SECURITY_STRENGTH_BITS.
+ * As we have no ability to review the implementation of those noise sources,
+ * it is prudent to have a conservative estimate here.
+ */
+#define LRNG_ARCHRANDOM_DEFAULT_STRENGTH (LRNG_DRNG_SECURITY_STRENGTH_BITS>>5)
+#define LRNG_ARCHRANDOM_TRUST_CPU_STRENGTH LRNG_DRNG_SECURITY_STRENGTH_BITS
+#ifdef CONFIG_RANDOM_TRUST_CPU
+static u32 archrandom = LRNG_ARCHRANDOM_TRUST_CPU_STRENGTH;
+#else
+static u32 archrandom = LRNG_ARCHRANDOM_DEFAULT_STRENGTH;
+#endif
+module_param(archrandom, uint, 0644);
+MODULE_PARM_DESC(archrandom, "Entropy in bits of 256 data bits from CPU noise "
+ "source (e.g. RDRAND)");
+
+static int __init lrng_parse_trust_cpu(char *arg)
+{
+ int ret;
+ bool trust_cpu = false;
+
+ ret = kstrtobool(arg, &trust_cpu);
+ if (ret)
+ return ret;
+
+ if (trust_cpu)
+ archrandom = LRNG_ARCHRANDOM_TRUST_CPU_STRENGTH;
+ else
+ archrandom = LRNG_ARCHRANDOM_DEFAULT_STRENGTH;
+
+ return 0;
+}
+early_param("random.trust_cpu", lrng_parse_trust_cpu);
+
+/**
+ * lrng_get_arch() - Get CPU noise source entropy
+ *
+ * @outbuf: buffer to store entropy of size LRNG_DRNG_SECURITY_STRENGTH_BYTES
+ *
+ * Return:
+ * * > 0 on success where value provides the added entropy in bits
+ * * 0 if no fast source was available
+ */
+u32 lrng_get_arch(u8 *outbuf)
+{
+ u32 i, ent_bits = archrandom;
+
+ /* operate on full blocks */
+ BUILD_BUG_ON(LRNG_DRNG_SECURITY_STRENGTH_BYTES % sizeof(unsigned long));
+ /* ensure we have aligned buffers */
+ BUILD_BUG_ON(LRNG_KCAPI_ALIGN % sizeof(unsigned long));
+
+ if (!ent_bits)
+ return 0;
+
+ for (i = 0; i < LRNG_DRNG_SECURITY_STRENGTH_BYTES;
+ i += sizeof(unsigned long)) {
+ if (!arch_get_random_seed_long((unsigned long *)(outbuf + i)) &&
+ !arch_get_random_long((unsigned long *)(outbuf + i))) {
+ archrandom = 0;
+ return 0;
+ }
+ }
+
+ /* Obtain entropy statement -- cap entropy to buffer size in bits */
+ ent_bits = min_t(u32, ent_bits, LRNG_DRNG_SECURITY_STRENGTH_BITS);
+ pr_debug("obtained %u bits of entropy from CPU RNG noise source\n",
+ ent_bits);
+ return ent_bits;
+}
+
+u32 lrng_slow_noise_req_entropy(u32 required_entropy_bits)
+{
+ u32 arch_ent_bits = min_t(u32, archrandom,
+ LRNG_DRNG_SECURITY_STRENGTH_BITS);
+ u32 fast_noise_entropy = arch_ent_bits + lrng_jent_entropylevel();
+
+ if (fast_noise_entropy > required_entropy_bits)
+ return 0;
+ return (required_entropy_bits - fast_noise_entropy);
+}
diff --git a/drivers/char/lrng/lrng_aux.c b/drivers/char/lrng/lrng_aux.c
new file mode 100644
index 000000000000..b66c7795d067
--- /dev/null
+++ b/drivers/char/lrng/lrng_aux.c
@@ -0,0 +1,148 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG auxiliary interfaces
+ *
+ * Copyright (C) 2019 Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ * Copyright (C) 2017 Jason A. Donenfeld <Jason-OnJsPKxuuEcAvxtiuMwx3w@public.gmane.org>. All
+ * Rights Reserved.
+ * Copyright (C) 2016 Jason Cooper <jason-NLaQJdtUoK4Be96aLqz0jA@public.gmane.org>
+ */
+
+#include <linux/mm.h>
+#include <linux/random.h>
+
+#include "lrng_internal.h"
+
+struct batched_entropy {
+ union {
+ u64 entropy_u64[LRNG_DRNG_BLOCKSIZE / sizeof(u64)];
+ u32 entropy_u32[LRNG_DRNG_BLOCKSIZE / sizeof(u32)];
+ };
+ unsigned int position;
+ spinlock_t batch_lock;
+};
+
+/*
+ * Get a random word for internal kernel use only. The quality of the random
+ * number is either as good as RDRAND or as good as /dev/urandom, with the
+ * goal of being quite fast and not depleting entropy.
+ */
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u64) = {
+ .batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u64.lock),
+};
+
+u64 get_random_u64(void)
+{
+ u64 ret;
+ unsigned long flags;
+ struct batched_entropy *batch;
+
+#if BITS_PER_LONG == 64
+ if (arch_get_random_long((unsigned long *)&ret))
+ return ret;
+#else
+ if (arch_get_random_long((unsigned long *)&ret) &&
+ arch_get_random_long((unsigned long *)&ret + 1))
+ return ret;
+#endif
+
+ lrng_debug_report_seedlevel("get_random_u64");
+
+ batch = raw_cpu_ptr(&batched_entropy_u64);
+ spin_lock_irqsave(&batch->batch_lock, flags);
+ if (batch->position % ARRAY_SIZE(batch->entropy_u64) == 0) {
+ lrng_drng_get_atomic((u8 *)batch->entropy_u64,
+ LRNG_DRNG_BLOCKSIZE);
+ batch->position = 0;
+ }
+ ret = batch->entropy_u64[batch->position++];
+ spin_unlock_irqrestore(&batch->batch_lock, flags);
+ return ret;
+}
+EXPORT_SYMBOL(get_random_u64);
+
+static DEFINE_PER_CPU(struct batched_entropy, batched_entropy_u32) = {
+ .batch_lock = __SPIN_LOCK_UNLOCKED(batched_entropy_u32.lock),
+};
+
+u32 get_random_u32(void)
+{
+ u32 ret;
+ unsigned long flags;
+ struct batched_entropy *batch;
+
+ if (arch_get_random_int(&ret))
+ return ret;
+
+ lrng_debug_report_seedlevel("get_random_u32");
+
+ batch = raw_cpu_ptr(&batched_entropy_u32);
+ spin_lock_irqsave(&batch->batch_lock, flags);
+ if (batch->position % ARRAY_SIZE(batch->entropy_u32) == 0) {
+ lrng_drng_get_atomic((u8 *)batch->entropy_u32,
+ LRNG_DRNG_BLOCKSIZE);
+ batch->position = 0;
+ }
+ ret = batch->entropy_u32[batch->position++];
+ spin_unlock_irqrestore(&batch->batch_lock, flags);
+ return ret;
+}
+EXPORT_SYMBOL(get_random_u32);
+
+/*
+ * It's important to invalidate all potential batched entropy that might
+ * be stored before the crng is initialized, which we can do lazily by
+ * simply resetting the counter to zero so that it's re-extracted on the
+ * next usage.
+ */
+void invalidate_batched_entropy(void)
+{
+ int cpu;
+ unsigned long flags;
+
+ for_each_possible_cpu(cpu) {
+ struct batched_entropy *batched_entropy;
+
+ batched_entropy = per_cpu_ptr(&batched_entropy_u32, cpu);
+ spin_lock_irqsave(&batched_entropy->batch_lock, flags);
+ batched_entropy->position = 0;
+ spin_unlock(&batched_entropy->batch_lock);
+
+ batched_entropy = per_cpu_ptr(&batched_entropy_u64, cpu);
+ spin_lock(&batched_entropy->batch_lock);
+ batched_entropy->position = 0;
+ spin_unlock_irqrestore(&batched_entropy->batch_lock, flags);
+ }
+}
+
+/**
+ * randomize_page - Generate a random, page aligned address
+ * @start: The smallest acceptable address the caller will take.
+ * @range: The size of the area, starting at @start, within which the
+ * random address must fall.
+ *
+ * If @start + @range would overflow, @range is capped.
+ *
+ * NOTE: Historical use of randomize_range, which this replaces, presumed that
+ * @start was already page aligned. We now align it regardless.
+ *
+ * Return: A page aligned address within [start, start + range). On error,
+ * @start is returned.
+ */
+unsigned long randomize_page(unsigned long start, unsigned long range)
+{
+ if (!PAGE_ALIGNED(start)) {
+ range -= PAGE_ALIGN(start) - start;
+ start = PAGE_ALIGN(start);
+ }
+
+ if (start > ULONG_MAX - range)
+ range = ULONG_MAX - start;
+
+ range >>= PAGE_SHIFT;
+
+ if (range == 0)
+ return start;
+
+ return start + (get_random_long() % range << PAGE_SHIFT);
+}
diff --git a/drivers/char/lrng/lrng_chacha20.c b/drivers/char/lrng/lrng_chacha20.c
new file mode 100644
index 000000000000..6d149ff355c6
--- /dev/null
+++ b/drivers/char/lrng/lrng_chacha20.c
@@ -0,0 +1,265 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Backend for the LRNG providing the cryptographic primitives using
+ * ChaCha20 cipher implementations.
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <crypto/chacha.h>
+#include <linux/cryptohash.h>
+#include <linux/lrng.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+
+#include "lrng_chacha20.h"
+#include "lrng_internal.h"
+
+/******************************* ChaCha20 DRNG *******************************/
+
+#define CHACHA_BLOCK_WORDS (CHACHA_BLOCK_SIZE / sizeof(u32))
+
+struct chacha20_state {
+ struct chacha20_block block;
+};
+
+/*
+ * Have a static memory blocks for the ChaCha20 DRNG instance to avoid calling
+ * kmalloc too early in the boot cycle. For subsequent allocation requests,
+ * such as per-NUMA-node DRNG instances, kmalloc will be used.
+ */
+struct chacha20_state chacha20;
+
+/**
+ * Update of the ChaCha20 state by either using an unused buffer part or by
+ * generating one ChaCha20 block which is half of the state of the ChaCha20.
+ * The block is XORed into the key part of the state. This shall ensure
+ * backtracking resistance as well as a proper mix of the ChaCha20 state once
+ * the key is injected.
+ */
+static void lrng_chacha20_update(struct chacha20_state *chacha20_state,
+ u32 *buf, u32 used_words)
+{
+ struct chacha20_block *chacha20 = &chacha20_state->block;
+ u32 i, tmp[CHACHA_BLOCK_WORDS];
+
+ BUILD_BUG_ON(sizeof(struct chacha20_block) != CHACHA_BLOCK_SIZE);
+ BUILD_BUG_ON(CHACHA_BLOCK_SIZE != 2 * CHACHA_KEY_SIZE);
+
+ if (used_words > CHACHA_KEY_SIZE_WORDS) {
+ chacha20_block(&chacha20->constants[0], (u8 *)tmp);
+ for (i = 0; i < CHACHA_KEY_SIZE_WORDS; i++)
+ chacha20->key.u[i] ^= tmp[i];
+ memzero_explicit(tmp, sizeof(tmp));
+ } else {
+ for (i = 0; i < CHACHA_KEY_SIZE_WORDS; i++)
+ chacha20->key.u[i] ^= buf[i + used_words];
+ }
+
+ /* Deterministic increment of nonce as required in RFC 7539 chapter 4 */
+ chacha20->nonce[0]++;
+ if (chacha20->nonce[0] == 0)
+ chacha20->nonce[1]++;
+ if (chacha20->nonce[1] == 0)
+ chacha20->nonce[2]++;
+
+ /* Leave counter untouched as it is start value is undefined in RFC */
+}
+
+/*
+ * Seed the ChaCha20 DRNG by injecting the input data into the key part of
+ * the ChaCha20 state. If the input data is longer than the ChaCha20 key size,
+ * perform a ChaCha20 operation after processing of key size input data.
+ * This operation shall spread out the entropy into the ChaCha20 state before
+ * new entropy is injected into the key part.
+ */
+static int lrng_cc20_drng_seed_helper(void *drng, const u8 *inbuf, u32 inbuflen)
+{
+ struct chacha20_state *chacha20_state = (struct chacha20_state *)drng;
+ struct chacha20_block *chacha20 = &chacha20_state->block;
+
+ while (inbuflen) {
+ u32 i, todo = min_t(u32, inbuflen, CHACHA_KEY_SIZE);
+
+ for (i = 0; i < todo; i++)
+ chacha20->key.b[i] ^= inbuf[i];
+
+ /* Break potential dependencies between the inbuf key blocks */
+ lrng_chacha20_update(chacha20_state, NULL,
+ CHACHA_BLOCK_WORDS);
+ inbuf += todo;
+ inbuflen -= todo;
+ }
+
+ return 0;
+}
+
+/*
+ * Chacha20 DRNG generation of random numbers: the stream output of ChaCha20
+ * is the random number. After the completion of the generation of the
+ * stream, the entire ChaCha20 state is updated.
+ *
+ * Note, as the ChaCha20 implements a 32 bit counter, we must ensure
+ * that this function is only invoked for at most 2^32 - 1 ChaCha20 blocks
+ * before a reseed or an update happens. This is ensured by the variable
+ * outbuflen which is a 32 bit integer defining the number of bytes to be
+ * generated by the ChaCha20 DRNG. At the end of this function, an update
+ * operation is invoked which implies that the 32 bit counter will never be
+ * overflown in this implementation.
+ */
+static int lrng_cc20_drng_generate_helper(void *drng, u8 *outbuf, u32 outbuflen)
+{
+ struct chacha20_state *chacha20_state = (struct chacha20_state *)drng;
+ struct chacha20_block *chacha20 = &chacha20_state->block;
+ u32 aligned_buf[CHACHA_BLOCK_WORDS], ret = outbuflen,
+ used = CHACHA_BLOCK_WORDS;
+ int zeroize_buf = 0;
+
+ while (outbuflen >= CHACHA_BLOCK_SIZE) {
+ chacha20_block(&chacha20->constants[0], outbuf);
+ outbuf += CHACHA_BLOCK_SIZE;
+ outbuflen -= CHACHA_BLOCK_SIZE;
+ }
+
+ if (outbuflen) {
+ chacha20_block(&chacha20->constants[0], (u8 *)aligned_buf);
+ memcpy(outbuf, aligned_buf, outbuflen);
+ used = ((outbuflen + sizeof(aligned_buf[0]) - 1) /
+ sizeof(aligned_buf[0]));
+ zeroize_buf = 1;
+ }
+
+ lrng_chacha20_update(chacha20_state, aligned_buf, used);
+
+ if (zeroize_buf)
+ memzero_explicit(aligned_buf, sizeof(aligned_buf));
+
+ return ret;
+}
+
+void lrng_cc20_init_state(struct chacha20_state *state)
+{
+ struct chacha20_block *chacha20 = &state->block;
+ unsigned long v;
+ u32 i;
+
+ lrng_cc20_init_rfc7539(chacha20);
+
+ for (i = 0; i < CHACHA_KEY_SIZE_WORDS; i++) {
+ chacha20->key.u[i] ^= jiffies;
+ chacha20->key.u[i] ^= random_get_entropy();
+ if (arch_get_random_seed_long(&v) || arch_get_random_long(&v))
+ chacha20->key.u[i] ^= v;
+ }
+
+ for (i = 0; i < 3; i++) {
+ chacha20->nonce[i] ^= jiffies;
+ chacha20->nonce[i] ^= random_get_entropy();
+ if (arch_get_random_seed_long(&v) || arch_get_random_long(&v))
+ chacha20->nonce[i] ^= v;
+ }
+
+ pr_info("ChaCha20 core initialized\n");
+}
+
+/*
+ * Allocation of the DRNG state
+ */
+static void *lrng_cc20_drng_alloc(u32 sec_strength)
+{
+ struct chacha20_state *state = NULL;
+
+ if (sec_strength > CHACHA_KEY_SIZE) {
+ pr_err("Security strength of ChaCha20 DRNG (%u bits) lower "
+ "than requested by LRNG (%u bits)\n",
+ CHACHA_KEY_SIZE * 8, sec_strength * 8);
+ return ERR_PTR(-EINVAL);
+ }
+ if (sec_strength < CHACHA_KEY_SIZE)
+ pr_warn("Security strength of ChaCha20 DRNG (%u bits) higher "
+ "than requested by LRNG (%u bits)\n",
+ CHACHA_KEY_SIZE * 8, sec_strength * 8);
+
+ state = kmalloc(sizeof(struct chacha20_state), GFP_KERNEL);
+ if (!state)
+ return ERR_PTR(-ENOMEM);
+ pr_debug("memory for ChaCha20 core allocated\n");
+
+ lrng_cc20_init_state(state);
+
+ return state;
+}
+
+static void lrng_cc20_drng_dealloc(void *drng)
+{
+ struct chacha20_state *chacha20_state = (struct chacha20_state *)drng;
+
+ if (drng == &chacha20) {
+ memzero_explicit(chacha20_state, sizeof(*chacha20_state));
+ pr_debug("static ChaCha20 core zeroized\n");
+ return;
+ }
+
+ pr_debug("ChaCha20 core zeroized and freed\n");
+ kzfree(chacha20_state);
+}
+
+/******************************* Hash Operation *******************************/
+
+static void *lrng_cc20_hash_alloc(const u8 *key, u32 keylen)
+{
+ pr_info("Hash SHA-1 allocated\n");
+ return NULL;
+}
+
+static void lrng_cc20_hash_dealloc(void *hash)
+{
+}
+
+static u32 lrng_cc20_hash_digestsize(void *hash)
+{
+ return (SHA_DIGEST_WORDS * sizeof(u32));
+}
+
+static int lrng_cc20_hash_buffer(void *hash, const u8 *inbuf, u32 inbuflen,
+ u8 *digest)
+{
+ u32 i;
+ u32 workspace[SHA_WORKSPACE_WORDS];
+
+ WARN_ON(inbuflen % (SHA_WORKSPACE_WORDS * sizeof(u32)));
+
+ sha_init((u32 *)digest);
+ for (i = 0; i < inbuflen; i += (SHA_WORKSPACE_WORDS * sizeof(u32)))
+ sha_transform((u32 *)digest, (inbuf + i), workspace);
+ memzero_explicit(workspace, sizeof(workspace));
+
+ return 0;
+}
+
+static const char *lrng_cc20_drng_name(void)
+{
+ const char *cc20_drng_name = "ChaCha20 DRNG";
+ return cc20_drng_name;
+}
+
+static const char *lrng_cc20_hash_name(void)
+{
+ const char *cc20_hash_name = "SHA-1";
+ return cc20_hash_name;
+}
+
+const struct lrng_crypto_cb lrng_cc20_crypto_cb = {
+ .lrng_drng_name = lrng_cc20_drng_name,
+ .lrng_hash_name = lrng_cc20_hash_name,
+ .lrng_drng_alloc = lrng_cc20_drng_alloc,
+ .lrng_drng_dealloc = lrng_cc20_drng_dealloc,
+ .lrng_drng_seed_helper = lrng_cc20_drng_seed_helper,
+ .lrng_drng_generate_helper = lrng_cc20_drng_generate_helper,
+ .lrng_hash_alloc = lrng_cc20_hash_alloc,
+ .lrng_hash_dealloc = lrng_cc20_hash_dealloc,
+ .lrng_hash_digestsize = lrng_cc20_hash_digestsize,
+ .lrng_hash_buffer = lrng_cc20_hash_buffer,
+};
diff --git a/drivers/char/lrng/lrng_chacha20.h b/drivers/char/lrng/lrng_chacha20.h
new file mode 100644
index 000000000000..0387ad59cfbc
--- /dev/null
+++ b/drivers/char/lrng/lrng_chacha20.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * LRNG ChaCha20 definitions
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+#include <crypto/chacha.h>
+
+/* State according to RFC 7539 section 2.3 */
+struct chacha20_block {
+ u32 constants[4];
+ union {
+#define CHACHA_KEY_SIZE_WORDS (CHACHA_KEY_SIZE / sizeof(u32))
+ u32 u[CHACHA_KEY_SIZE_WORDS];
+ u8 b[CHACHA_KEY_SIZE];
+ } key;
+ u32 counter;
+ u32 nonce[3];
+};
+
+static inline void lrng_cc20_init_rfc7539(struct chacha20_block *chacha20)
+{
+ memcpy(&chacha20->constants[0], "expand 32-byte k", 16);
+}
diff --git a/drivers/char/lrng/lrng_drng.c b/drivers/char/lrng/lrng_drng.c
new file mode 100644
index 000000000000..3f7fde556d66
--- /dev/null
+++ b/drivers/char/lrng/lrng_drng.c
@@ -0,0 +1,400 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG DRNG processing
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/lrng.h>
+
+#include "lrng_internal.h"
+
+/*
+ * Maximum number of seconds between DRNG reseed intervals of the DRNG. Note,
+ * this is enforced with the next request of random numbers from the
+ * DRNG. Setting this value to zero implies a reseeding attempt before every
+ * generated random number.
+ */
+int lrng_drng_reseed_max_time = 600;
+
+static atomic_t lrng_avail = ATOMIC_INIT(0);
+
+DEFINE_MUTEX(lrng_crypto_cb_update);
+
+/* DRNG for /dev/urandom, getrandom(2), get_random_bytes */
+static struct lrng_drng lrng_drng_init = {
+ .drng = &chacha20,
+ .crypto_cb = &lrng_cc20_crypto_cb,
+ .lock = __MUTEX_INITIALIZER(lrng_drng_init.lock),
+ .spin_lock = __SPIN_LOCK_UNLOCKED(lrng_drng_init.spin_lock)
+};
+
+/*
+ * DRNG for get_random_bytes when called in atomic context. This
+ * DRNG will always use the ChaCha20 DRNG. It will never benefit from a
+ * DRNG switch like the "regular" DRNG. If there was no DRNG switch, the atomic
+ * DRNG is identical to the "regular" DRNG.
+ *
+ * The reason for having this is due to the fact that DRNGs other than
+ * the ChaCha20 DRNG may sleep.
+ */
+static struct lrng_drng lrng_drng_atomic = {
+ .drng = &chacha20,
+ .crypto_cb = &lrng_cc20_crypto_cb,
+ .spin_lock = __SPIN_LOCK_UNLOCKED(lrng_drng_atomic.spin_lock)
+};
+
+/********************************** Helper ************************************/
+
+bool lrng_get_available(void)
+{
+ return likely(atomic_read(&lrng_avail));
+}
+
+void lrng_set_available(void)
+{
+ atomic_set(&lrng_avail, 1);
+}
+
+struct lrng_drng *lrng_drng_init_instance(void)
+{
+ return &lrng_drng_init;
+}
+
+struct lrng_drng *lrng_drng_atomic_instance(void)
+{
+ return &lrng_drng_atomic;
+}
+
+void lrng_drng_reset(struct lrng_drng *drng)
+{
+ atomic_set(&drng->requests, LRNG_DRNG_RESEED_THRESH);
+ drng->last_seeded = jiffies;
+ drng->fully_seeded = false;
+ drng->force_reseed = true;
+ pr_debug("reset DRNG\n");
+}
+
+/************************* Random Number Generation ***************************/
+
+/* Inject a data buffer into the DRNG */
+static void lrng_drng_inject(struct lrng_drng *drng,
+ const u8 *inbuf, u32 inbuflen)
+{
+ const char *drng_type = unlikely(drng == &lrng_drng_atomic) ?
+ "atomic" : "regular";
+ unsigned long flags = 0;
+
+ BUILD_BUG_ON(LRNG_DRNG_RESEED_THRESH > INT_MAX);
+ pr_debug("seeding %s DRNG with %u bytes\n", drng_type, inbuflen);
+ lrng_drng_lock(drng, &flags);
+ if (drng->crypto_cb->lrng_drng_seed_helper(drng->drng,
+ inbuf, inbuflen) < 0) {
+ pr_warn("seeding of %s DRNG failed\n", drng_type);
+ atomic_set(&drng->requests, 1);
+ } else {
+ pr_debug("%s DRNG stats since last seeding: %lu secs; "
+ "generate calls: %d\n", drng_type,
+ (time_after(jiffies, drng->last_seeded) ?
+ (jiffies - drng->last_seeded) : 0) / HZ,
+ (LRNG_DRNG_RESEED_THRESH -
+ atomic_read(&drng->requests)));
+ drng->last_seeded = jiffies;
+ atomic_set(&drng->requests, LRNG_DRNG_RESEED_THRESH);
+ drng->force_reseed = false;
+
+ if (drng->drng == lrng_drng_atomic.drng) {
+ lrng_drng_atomic.last_seeded = jiffies;
+ atomic_set(&lrng_drng_atomic.requests,
+ LRNG_DRNG_RESEED_THRESH);
+ lrng_drng_atomic.force_reseed = false;
+ }
+ }
+ lrng_drng_unlock(drng, &flags);
+}
+
+/*
+ * Perform the seeding of the DRNG with data from noise source
+ */
+static inline int _lrng_drng_seed(struct lrng_drng *drng)
+{
+ struct entropy_buf seedbuf __aligned(LRNG_KCAPI_ALIGN);
+ unsigned long flags = 0;
+ u32 total_entropy_bits;
+ int ret;
+
+ lrng_drng_lock(drng, &flags);
+ total_entropy_bits = lrng_fill_seed_buffer(drng->crypto_cb, drng->hash,
+ &seedbuf, 0);
+ lrng_drng_unlock(drng, &flags);
+
+ /* Allow the seeding operation to be called again */
+ lrng_pool_unlock();
+ lrng_init_ops(total_entropy_bits);
+ ret = total_entropy_bits >> 3;
+
+ lrng_drng_inject(drng, (u8 *)&seedbuf, sizeof(seedbuf));
+ memzero_explicit(&seedbuf, sizeof(seedbuf));
+
+ return ret;
+}
+
+static int lrng_drng_get(struct lrng_drng *drng, u8 *outbuf, u32 outbuflen);
+static void lrng_drng_seed(struct lrng_drng *drng)
+{
+ int ret = _lrng_drng_seed(drng);
+
+ if (ret >= LRNG_DRNG_SECURITY_STRENGTH_BYTES)
+ drng->fully_seeded = true;
+
+ BUILD_BUG_ON(LRNG_MIN_SEED_ENTROPY_BITS >
+ LRNG_DRNG_SECURITY_STRENGTH_BITS);
+
+ /*
+ * Reseed atomic DRNG from current DRNG,
+ *
+ * We can obtain random numbers from DRNG as the lock type
+ * chosen by lrng_drng_get is usable with the current caller.
+ */
+ if ((drng->drng != lrng_drng_atomic.drng) &&
+ (lrng_drng_atomic.force_reseed ||
+ atomic_read(&lrng_drng_atomic.requests) <= 0 ||
+ time_after(jiffies, lrng_drng_atomic.last_seeded +
+ lrng_drng_reseed_max_time * HZ))) {
+ u8 seedbuf[LRNG_DRNG_SECURITY_STRENGTH_BYTES]
+ __aligned(LRNG_KCAPI_ALIGN);
+
+ ret = lrng_drng_get(drng, seedbuf, sizeof(seedbuf));
+
+ if (ret < 0) {
+ pr_warn("Error generating random numbers for atomic "
+ "DRNG: %d\n", ret);
+ } else {
+ lrng_drng_inject(&lrng_drng_atomic, seedbuf, ret);
+ }
+ memzero_explicit(&seedbuf, sizeof(seedbuf));
+ }
+}
+
+static inline void _lrng_drng_seed_work(struct lrng_drng *drng, u32 node)
+{
+ pr_debug("reseed triggered by interrupt noise source for DRNG on NUMA "
+ "node %d\n", node);
+ lrng_drng_seed(drng);
+ if (drng->fully_seeded) {
+ /* Prevent reseed storm */
+ drng->last_seeded += node * 100 * HZ;
+ /* Prevent draining of pool on idle systems */
+ lrng_drng_reseed_max_time += 100;
+ }
+}
+
+/*
+ * DRNG reseed trigger: Kernel thread handler triggered by the schedule_work()
+ */
+void lrng_drng_seed_work(struct work_struct *dummy)
+{
+ struct lrng_drng **lrng_drng = lrng_drng_instances();
+ u32 node;
+
+ if (lrng_drng) {
+ for_each_online_node(node) {
+ struct lrng_drng *drng = lrng_drng[node];
+
+ if (drng && !drng->fully_seeded) {
+ _lrng_drng_seed_work(drng, node);
+ goto out;
+ }
+ }
+ lrng_pool_all_numa_nodes_seeded();
+ } else {
+ if (!lrng_drng_init.fully_seeded)
+ _lrng_drng_seed_work(&lrng_drng_init, 0);
+ }
+
+out:
+ /* Allow the seeding operation to be called again */
+ lrng_pool_unlock();
+}
+
+/* Force all DRNGs to reseed before next generation */
+void lrng_drng_force_reseed(void)
+{
+ struct lrng_drng **lrng_drng = lrng_drng_instances();
+ u32 node;
+
+ if (!lrng_drng) {
+ lrng_drng_init.force_reseed = true;
+ pr_debug("force reseed of initial DRNG\n");
+ return;
+ }
+ for_each_online_node(node) {
+ struct lrng_drng *drng = lrng_drng[node];
+
+ if (!drng)
+ continue;
+
+ drng->force_reseed = true;
+ pr_debug("force reseed of DRNG on node %u\n", node);
+ }
+ lrng_drng_atomic.force_reseed = true;
+}
+
+/**
+ * lrng_drng_get() - Get random data out of the DRNG which is reseeded
+ * frequently.
+ *
+ * @outbuf: buffer for storing random data
+ * @outbuflen: length of outbuf
+ *
+ * Return:
+ * * < 0 in error case (DRNG generation or update failed)
+ * * >=0 returning the returned number of bytes
+ */
+static int lrng_drng_get(struct lrng_drng *drng, u8 *outbuf, u32 outbuflen)
+{
+ unsigned long flags = 0;
+ u32 processed = 0;
+
+ if (!outbuf || !outbuflen)
+ return 0;
+
+ outbuflen = min_t(size_t, outbuflen, INT_MAX);
+
+ lrng_drngs_init_cc20();
+
+ while (outbuflen) {
+ u32 todo = min_t(u32, outbuflen, LRNG_DRNG_MAX_REQSIZE);
+ int ret;
+
+ /* All but the atomic DRNG are seeded during generation */
+ if (atomic_dec_and_test(&drng->requests) ||
+ drng->force_reseed ||
+ time_after(jiffies, drng->last_seeded +
+ lrng_drng_reseed_max_time * HZ)) {
+ if (likely(drng != &lrng_drng_atomic)) {
+ if (lrng_pool_trylock())
+ atomic_set(&drng->requests, 1);
+ else
+ lrng_drng_seed(drng);
+ }
+ }
+
+ lrng_drng_lock(drng, &flags);
+ ret = drng->crypto_cb->lrng_drng_generate_helper(
+ drng->drng, outbuf + processed, todo);
+ lrng_drng_unlock(drng, &flags);
+ if (ret <= 0) {
+ pr_warn("getting random data from DRNG failed (%d)\n",
+ ret);
+ return -EFAULT;
+ }
+ processed += ret;
+ outbuflen -= ret;
+ }
+
+ return processed;
+}
+
+int lrng_drng_get_atomic(u8 *outbuf, u32 outbuflen)
+{
+ return lrng_drng_get(&lrng_drng_atomic, outbuf, outbuflen);
+}
+
+int lrng_drng_get_sleep(u8 *outbuf, u32 outbuflen)
+{
+ struct lrng_drng **lrng_drng = lrng_drng_instances();
+ struct lrng_drng *drng = &lrng_drng_init;
+ int node = numa_node_id();
+
+ might_sleep();
+
+ if (lrng_drng && lrng_drng[node] && lrng_drng[node]->fully_seeded)
+ drng = lrng_drng[node];
+
+ return lrng_drng_get(drng, outbuf, outbuflen);
+}
+
+/* Initialize the default DRNG during boot */
+void lrng_drngs_init_cc20(void)
+{
+ unsigned long flags = 0;
+
+ if (lrng_get_available())
+ return;
+
+ lrng_drng_lock(&lrng_drng_init, &flags);
+ if (lrng_get_available()) {
+ lrng_drng_unlock(&lrng_drng_init, &flags);
+ return;
+ }
+
+ lrng_drng_reset(&lrng_drng_init);
+ lrng_cc20_init_state(&chacha20);
+ lrng_state_init_seed_work();
+ lrng_drng_unlock(&lrng_drng_init, &flags);
+
+ lrng_drng_lock(&lrng_drng_atomic, &flags);
+ lrng_drng_reset(&lrng_drng_atomic);
+ /*
+ * We do not initialize the state of the atomic DRNG as it is identical
+ * to the DRNG at this point.
+ */
+ lrng_drng_unlock(&lrng_drng_atomic, &flags);
+
+ lrng_set_available();
+}
+
+/* Reset LRNG such that all existing entropy is gone */
+static void _lrng_reset(struct work_struct *work)
+{
+ struct lrng_drng **lrng_drng = lrng_drng_instances();
+ unsigned long flags = 0;
+
+ if (!lrng_drng) {
+ lrng_drng_lock(&lrng_drng_init, &flags);
+ lrng_drng_reset(&lrng_drng_init);
+ lrng_drng_unlock(&lrng_drng_init, &flags);
+ } else {
+ u32 node;
+
+ for_each_online_node(node) {
+ struct lrng_drng *drng = lrng_drng[node];
+
+ if (!drng)
+ continue;
+ lrng_drng_lock(drng, &flags);
+ lrng_drng_reset(drng);
+ lrng_drng_unlock(drng, &flags);
+ }
+ }
+ lrng_set_entropy_thresh(LRNG_INIT_ENTROPY_BITS +
+ LRNG_CONDITIONING_ENTROPY_LOSS);
+
+ lrng_reset_state();
+}
+
+static DECLARE_WORK(lrng_reset_work, _lrng_reset);
+
+void lrng_reset(void)
+{
+ schedule_work(&lrng_reset_work);
+}
+
+/***************************** Initialize LRNG *******************************/
+
+static int __init lrng_init(void)
+{
+ lrng_drngs_init_cc20();
+
+ lrng_drngs_numa_alloc();
+ return 0;
+}
+
+late_initcall(lrng_init);
+
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_AUTHOR("Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>");
+MODULE_DESCRIPTION("Linux Random Number Generator");
diff --git a/drivers/char/lrng/lrng_interfaces.c b/drivers/char/lrng/lrng_interfaces.c
new file mode 100644
index 000000000000..d6ddc72fc9b7
--- /dev/null
+++ b/drivers/char/lrng/lrng_interfaces.c
@@ -0,0 +1,638 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG User and kernel space interfaces
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/freezer.h>
+#include <linux/fs.h>
+#include <linux/genhd.h>
+#include <linux/kthread.h>
+#include <linux/poll.h>
+#include <linux/preempt.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+#include <linux/syscalls.h>
+#include <linux/timex.h>
+
+#include "lrng_internal.h"
+
+/*
+ * If the entropy count falls under this number of bits, then we
+ * should wake up processes which are selecting or polling on write
+ * access to /dev/random.
+ */
+u32 lrng_write_wakeup_bits = LRNG_WRITE_WAKEUP_ENTROPY;
+
+static LIST_HEAD(lrng_ready_list);
+static DEFINE_SPINLOCK(lrng_ready_list_lock);
+
+static DECLARE_WAIT_QUEUE_HEAD(lrng_write_wait);
+static DECLARE_WAIT_QUEUE_HEAD(lrng_init_wait);
+static struct fasync_struct *fasync;
+
+struct ctl_table random_table[];
+/********************************** Helper ***********************************/
+
+/* Is the DRNG seed level too low? */
+static inline bool lrng_need_entropy(void)
+{
+ return (lrng_avail_entropy() < lrng_write_wakeup_bits);
+}
+
+void lrng_writer_wakeup(void)
+{
+ if (lrng_need_entropy() && wq_has_sleeper(&lrng_write_wait)) {
+ wake_up_interruptible(&lrng_write_wait);
+ kill_fasync(&fasync, SIGIO, POLL_OUT);
+ }
+}
+
+void lrng_init_wakeup(void)
+{
+ wake_up_all(&lrng_init_wait);
+ kill_fasync(&fasync, SIGIO, POLL_IN);
+}
+
+/**
+ * lrng_process_ready_list() - Ping all kernel internal callers waiting until
+ * the DRNG is at least minimally seeded to inform that the DRNG reached that
+ * seed level.
+ *
+ * When the SP800-90B testing is enabled, the ping only happens if the SP800-90B
+ * startup health tests are completed. This implies that kernel internal
+ * callers always have an SP800-90B compliant noise source when being
+ * pinged.
+ */
+void lrng_process_ready_list(void)
+{
+ unsigned long flags;
+ struct random_ready_callback *rdy, *tmp;
+
+ if (!lrng_sp80090b_startup_complete())
+ return;
+
+ spin_lock_irqsave(&lrng_ready_list_lock, flags);
+ list_for_each_entry_safe(rdy, tmp, &lrng_ready_list, list) {
+ struct module *owner = rdy->owner;
+
+ list_del_init(&rdy->list);
+ rdy->func(rdy);
+ module_put(owner);
+ }
+ spin_unlock_irqrestore(&lrng_ready_list_lock, flags);
+}
+
+void lrng_debug_report_seedlevel(const char *name)
+{
+#ifdef CONFIG_WARN_ALL_UNSEEDED_RANDOM
+ static void *previous = NULL;
+ void *caller = (void *) _RET_IP_;
+
+ if (READ_ONCE(previous) == caller)
+ return;
+
+ if (!lrng_state_min_seeded())
+ pr_notice("%pS %s called without reaching mimimally seeded "
+ "level (available entropy %u)\n", caller, name,
+ lrng_avail_entropy());
+
+ WRITE_ONCE(previous, caller);
+#endif
+}
+
+/************************ LRNG kernel input interfaces ************************/
+
+/**
+ * add_hwgenerator_randomness() - Interface for in-kernel drivers of true
+ * hardware RNGs.
+ *
+ * Those devices may produce endless random bits and will be throttled
+ * when our pool is full.
+ *
+ * @buffer: buffer holding the entropic data from HW noise sources to be used to
+ * insert into entropy pool.
+ * @count: length of buffer
+ * @entropy_bits: amount of entropy in buffer (value is in bits)
+ */
+void add_hwgenerator_randomness(const char *buffer, size_t count,
+ size_t entropy_bits)
+{
+ /*
+ * Suspend writing if we are fully loaded with entropy.
+ * We'll be woken up again once below lrng_write_wakeup_thresh,
+ * or when the calling thread is about to terminate.
+ */
+ wait_event_interruptible(lrng_write_wait,
+ lrng_need_entropy() ||
+ lrng_state_exseed_allow(lrng_noise_source_hw) ||
+ kthread_should_stop());
+ lrng_state_exseed_set(lrng_noise_source_hw, false);
+ lrng_pool_lfsr_nonaligned(buffer, count);
+ lrng_pool_add_entropy(entropy_bits);
+}
+EXPORT_SYMBOL_GPL(add_hwgenerator_randomness);
+
+/**
+ * add_bootloader_randomness() - Handle random seed passed by bootloader.
+ *
+ * If the seed is trustworthy, it would be regarded as hardware RNGs. Otherwise
+ * it would be regarded as device data.
+ * The decision is controlled by CONFIG_RANDOM_TRUST_BOOTLOADER.
+ *
+ * @buf: buffer holding the entropic data from HW noise sources to be used to
+ * insert into entropy pool.
+ * @size: length of buffer
+ */
+void add_bootloader_randomness(const void *buf, unsigned int size)
+{
+ if (IS_ENABLED(CONFIG_RANDOM_TRUST_BOOTLOADER))
+ add_hwgenerator_randomness(buf, size, size * 8);
+ else
+ add_device_randomness(buf, size);
+}
+EXPORT_SYMBOL_GPL(add_bootloader_randomness);
+
+/*
+ * Callback for HID layer -- use the HID event values to stir the entropy pool
+ */
+void add_input_randomness(unsigned int type, unsigned int code,
+ unsigned int value)
+{
+ static unsigned char last_value;
+
+ /* ignore autorepeat and the like */
+ if (value == last_value)
+ return;
+
+ last_value = value;
+
+ lrng_pool_lfsr_u32((type << 4) ^ code ^ (code >> 4) ^ value);
+}
+EXPORT_SYMBOL_GPL(add_input_randomness);
+
+/**
+ * add_device_randomness() - Add device- or boot-specific data to the entropy
+ * pool to help initialize it.
+ *
+ * None of this adds any entropy; it is meant to avoid the problem of
+ * the entropy pool having similar initial state across largely
+ * identical devices.
+ *
+ * @buf: buffer holding the entropic data from HW noise sources to be used to
+ * insert into entropy pool.
+ * @size: length of buffer
+ */
+void add_device_randomness(const void *buf, unsigned int size)
+{
+ lrng_pool_lfsr_nonaligned((u8 *)buf, size);
+ lrng_pool_lfsr_u32(random_get_entropy());
+ lrng_pool_lfsr_u32(jiffies);
+}
+EXPORT_SYMBOL(add_device_randomness);
+
+#ifdef CONFIG_BLOCK
+void rand_initialize_disk(struct gendisk *disk) { }
+void add_disk_randomness(struct gendisk *disk) { }
+EXPORT_SYMBOL(add_disk_randomness);
+#endif
+
+/**
+ * del_random_ready_callback() - Delete a previously registered readiness
+ * callback function.
+ *
+ * @rdy: callback definition that was registered initially
+ */
+void del_random_ready_callback(struct random_ready_callback *rdy)
+{
+ unsigned long flags;
+ struct module *owner = NULL;
+
+ spin_lock_irqsave(&lrng_ready_list_lock, flags);
+ if (!list_empty(&rdy->list)) {
+ list_del_init(&rdy->list);
+ owner = rdy->owner;
+ }
+ spin_unlock_irqrestore(&lrng_ready_list_lock, flags);
+
+ module_put(owner);
+}
+EXPORT_SYMBOL(del_random_ready_callback);
+
+/**
+ * add_random_ready_callback() - Add a callback function that will be invoked
+ * when the DRNG is mimimally seeded.
+ *
+ * @rdy: callback definition to be invoked when the LRNG is seeded
+ *
+ * Return:
+ * * 0 if callback is successfully added
+ * * -EALREADY if pool is already initialised (callback not called)
+ * * -ENOENT if module for callback is not alive
+ */
+int add_random_ready_callback(struct random_ready_callback *rdy)
+{
+ struct module *owner;
+ unsigned long flags;
+ int err = -EALREADY;
+
+ if (likely(lrng_state_min_seeded()))
+ return err;
+
+ owner = rdy->owner;
+ if (!try_module_get(owner))
+ return -ENOENT;
+
+ spin_lock_irqsave(&lrng_ready_list_lock, flags);
+ if (lrng_state_min_seeded())
+ goto out;
+
+ owner = NULL;
+
+ list_add(&rdy->list, &lrng_ready_list);
+ err = 0;
+
+out:
+ spin_unlock_irqrestore(&lrng_ready_list_lock, flags);
+
+ module_put(owner);
+
+ return err;
+}
+EXPORT_SYMBOL(add_random_ready_callback);
+
+/*********************** LRNG kernel output interfaces ************************/
+
+/**
+ * get_random_bytes() - Provider of cryptographic strong random numbers for
+ * kernel-internal usage.
+ *
+ * This function is appropriate for all in-kernel use cases. However,
+ * it will always use the ChaCha20 DRNG.
+ *
+ * @buf: buffer to store the random bytes
+ * @nbytes: size of the buffer
+ */
+void get_random_bytes(void *buf, int nbytes)
+{
+ lrng_drng_get_atomic((u8 *)buf, (u32)nbytes);
+ lrng_debug_report_seedlevel("get_random_bytes");
+}
+EXPORT_SYMBOL(get_random_bytes);
+
+/**
+ * get_random_bytes_full() - Provider of cryptographic strong random numbers
+ * for kernel-internal usage.
+ *
+ * This function is appropriate only for non-atomic use cases as this
+ * function may sleep. Though, it provides access to the full functionality
+ * of LRNG including the switchable DRNG support, that may support other
+ * DRNGs such as the SP800-90A DRBG.
+ *
+ * @buf: buffer to store the random bytes
+ * @nbytes: size of the buffer
+ */
+void get_random_bytes_full(void *buf, int nbytes)
+{
+ lrng_drng_get_sleep((u8 *)buf, (u32)nbytes);
+ lrng_debug_report_seedlevel("get_random_bytes_full");
+}
+EXPORT_SYMBOL(get_random_bytes_full);
+
+/**
+ * wait_for_random_bytes() - Wait for the LRNG to be seeded and thus
+ * guaranteed to supply cryptographically secure random numbers.
+ *
+ * This applies to: the /dev/urandom device, the get_random_bytes function,
+ * and the get_random_{u32,u64,int,long} family of functions. Using any of
+ * these functions without first calling this function forfeits the guarantee
+ * of security.
+ *
+ * Return:
+ * * 0 if the LRNG has been seeded.
+ * * -ERESTARTSYS if the function was interrupted by a signal.
+ */
+int wait_for_random_bytes(void)
+{
+ if (likely(lrng_state_min_seeded()))
+ return 0;
+ return wait_event_interruptible(lrng_init_wait,
+ lrng_state_min_seeded());
+}
+EXPORT_SYMBOL(wait_for_random_bytes);
+
+/**
+ * get_random_bytes_arch() - This function will use the architecture-specific
+ * hardware random number generator if it is available.
+ *
+ * The arch-specific hw RNG will almost certainly be faster than what we can
+ * do in software, but it is impossible to verify that it is implemented
+ * securely (as opposed, to, say, the AES encryption of a sequence number using
+ * a key known by the NSA). So it's useful if we need the speed, but only if
+ * we're willing to trust the hardware manufacturer not to have put in a back
+ * door.
+ *
+ * @buf: buffer allocated by caller to store the random data in
+ * @nbytes: length of outbuf
+ *
+ * Return: number of bytes filled in.
+ */
+int __must_check get_random_bytes_arch(void *buf, int nbytes)
+{
+ u8 *p = buf;
+
+ while (nbytes) {
+ unsigned long v;
+ int chunk = min_t(int, nbytes, sizeof(unsigned long));
+
+ if (!arch_get_random_long(&v))
+ break;
+
+ memcpy(p, &v, chunk);
+ p += chunk;
+ nbytes -= chunk;
+ }
+
+ if (nbytes)
+ lrng_drng_get_atomic((u8 *)p, (u32)nbytes);
+
+ return nbytes;
+}
+EXPORT_SYMBOL(get_random_bytes_arch);
+
+/************************ LRNG user output interfaces *************************/
+
+static ssize_t lrng_read_common(char __user *buf, size_t nbytes)
+{
+ ssize_t ret = 0;
+ u8 tmpbuf[LRNG_DRNG_BLOCKSIZE] __aligned(LRNG_KCAPI_ALIGN);
+ u8 *tmp_large = NULL, *tmp = tmpbuf;
+ u32 tmplen = sizeof(tmpbuf);
+
+ if (nbytes == 0)
+ return 0;
+
+ /*
+ * Satisfy large read requests -- as the common case are smaller
+ * request sizes, such as 16 or 32 bytes, avoid a kmalloc overhead for
+ * those by using the stack variable of tmpbuf.
+ */
+ if (!IS_ENABLED(CONFIG_BASE_SMALL) && (nbytes > sizeof(tmpbuf))) {
+ tmplen = min_t(u32, nbytes, LRNG_DRNG_MAX_REQSIZE);
+ tmp_large = kmalloc(tmplen + LRNG_KCAPI_ALIGN, GFP_KERNEL);
+ if (!tmp_large)
+ tmplen = sizeof(tmpbuf);
+ else
+ tmp = PTR_ALIGN(tmp_large, LRNG_KCAPI_ALIGN);
+ }
+
+ while (nbytes) {
+ u32 todo = min_t(u32, nbytes, tmplen);
+ int rc = 0;
+
+ /* Reschedule if we received a large request. */
+ if ((tmp_large) && need_resched()) {
+ if (signal_pending(current)) {
+ if (ret == 0)
+ ret = -ERESTARTSYS;
+ break;
+ }
+ schedule();
+ }
+
+ rc = lrng_drng_get_sleep(tmp, todo);
+ if (rc <= 0) {
+ if (rc < 0)
+ ret = rc;
+ break;
+ }
+ if (copy_to_user(buf, tmp, rc)) {
+ ret = -EFAULT;
+ break;
+ }
+
+ nbytes -= rc;
+ buf += rc;
+ ret += rc;
+ }
+
+ /* Wipe data just returned from memory */
+ if (tmp_large)
+ kzfree(tmp_large);
+ else
+ memzero_explicit(tmpbuf, sizeof(tmpbuf));
+
+ return ret;
+}
+
+static ssize_t
+lrng_read_common_block(int nonblock, char __user *buf, size_t nbytes)
+{
+ if (nbytes == 0)
+ return 0;
+
+ if (unlikely(!lrng_state_operational())) {
+ int ret;
+
+ if (nonblock)
+ return -EAGAIN;
+
+ ret = wait_event_interruptible(lrng_init_wait,
+ lrng_state_operational());
+ if (unlikely(ret))
+ return ret;
+ }
+
+ return lrng_read_common(buf, nbytes);
+}
+
+static ssize_t lrng_drng_read_block(struct file *file, char __user *buf,
+ size_t nbytes, loff_t *ppos)
+{
+ return lrng_read_common_block(file->f_flags & O_NONBLOCK, buf, nbytes);
+}
+
+static unsigned int lrng_random_poll(struct file *file, poll_table *wait)
+{
+ __poll_t mask;
+
+ poll_wait(file, &lrng_init_wait, wait);
+ poll_wait(file, &lrng_write_wait, wait);
+ mask = 0;
+ if (lrng_state_operational())
+ mask |= EPOLLIN | EPOLLRDNORM;
+ if (lrng_need_entropy() ||
+ lrng_state_exseed_allow(lrng_noise_source_user))
+ mask |= EPOLLOUT | EPOLLWRNORM;
+ return mask;
+}
+
+static ssize_t lrng_drng_write_common(const char __user *buffer, size_t count,
+ u32 entropy_bits)
+{
+ ssize_t ret = 0;
+ u8 buf[64] __aligned(LRNG_KCAPI_ALIGN);
+ const char __user *p = buffer;
+ u32 orig_entropy_bits = entropy_bits;
+
+ if (!lrng_get_available())
+ return -EAGAIN;
+
+ count = min_t(size_t, count, INT_MAX);
+ while (count > 0) {
+ size_t bytes = min_t(size_t, count, sizeof(buf));
+ u32 ent = min_t(u32, bytes<<3, entropy_bits);
+
+ if (copy_from_user(&buf, p, bytes))
+ return -EFAULT;
+ /* Inject data into entropy pool */
+ lrng_pool_lfsr(buf, bytes);
+ lrng_pool_add_entropy(ent);
+
+ count -= bytes;
+ p += bytes;
+ ret += bytes;
+ entropy_bits -= ent;
+
+ cond_resched();
+ }
+
+ /* Force reseed of DRNG during next data request. */
+ if (!orig_entropy_bits)
+ lrng_drng_force_reseed();
+
+ return ret;
+}
+
+static ssize_t lrng_drng_read(struct file *file, char __user *buf,
+ size_t nbytes, loff_t *ppos)
+{
+ if (!lrng_state_min_seeded())
+ pr_notice_ratelimited("%s - use of insufficiently seeded DRNG "
+ "(%zu bytes read)\n", current->comm,
+ nbytes);
+ else if (!lrng_state_operational())
+ pr_debug_ratelimited("%s - use of not fully seeded DRNG (%zu "
+ "bytes read)\n", current->comm, nbytes);
+
+ return lrng_read_common(buf, nbytes);
+}
+
+static ssize_t lrng_drng_write(struct file *file, const char __user *buffer,
+ size_t count, loff_t *ppos)
+{
+ return lrng_drng_write_common(buffer, count, 0);
+}
+
+static long lrng_ioctl(struct file *f, unsigned int cmd, unsigned long arg)
+{
+ int size, ent_count_bits;
+ int __user *p = (int __user *)arg;
+
+ switch (cmd) {
+ case RNDGETENTCNT:
+ ent_count_bits = lrng_avail_entropy();
+ if (put_user(ent_count_bits, p))
+ return -EFAULT;
+ return 0;
+ case RNDADDTOENTCNT:
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+ if (get_user(ent_count_bits, p))
+ return -EFAULT;
+ ent_count_bits = (int)lrng_avail_entropy() + ent_count_bits;
+ if (ent_count_bits < 0)
+ ent_count_bits = 0;
+ if (ent_count_bits > LRNG_POOL_SIZE_BITS)
+ ent_count_bits = LRNG_POOL_SIZE_BITS;
+ lrng_pool_set_entropy(ent_count_bits);
+ return 0;
+ case RNDADDENTROPY:
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+ if (get_user(ent_count_bits, p++))
+ return -EFAULT;
+ if (ent_count_bits < 0)
+ return -EINVAL;
+ if (get_user(size, p++))
+ return -EFAULT;
+ if (size < 0)
+ return -EINVAL;
+ lrng_state_exseed_set(lrng_noise_source_user, false);
+ /* there cannot be more entropy than data */
+ ent_count_bits = min(ent_count_bits, size<<3);
+ return lrng_drng_write_common((const char __user *)p, size,
+ ent_count_bits);
+ case RNDZAPENTCNT:
+ case RNDCLEARPOOL:
+ /* Clear the entropy pool counter. */
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+ lrng_pool_set_entropy(0);
+ return 0;
+ case RNDRESEEDCRNG:
+ /*
+ * We leave the capability check here since it is present
+ * in the upstream's RNG implementation. Yet, user space
+ * can trigger a reseed as easy as writing into /dev/random
+ * or /dev/urandom where no privilege is needed.
+ */
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+ /* Force a reseed of all DRNGs */
+ lrng_drng_force_reseed();
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int lrng_fasync(int fd, struct file *filp, int on)
+{
+ return fasync_helper(fd, filp, on, &fasync);
+}
+
+const struct file_operations random_fops = {
+ .read = lrng_drng_read_block,
+ .write = lrng_drng_write,
+ .poll = lrng_random_poll,
+ .unlocked_ioctl = lrng_ioctl,
+ .compat_ioctl = compat_ptr_ioctl,
+ .fasync = lrng_fasync,
+ .llseek = noop_llseek,
+};
+
+const struct file_operations urandom_fops = {
+ .read = lrng_drng_read,
+ .write = lrng_drng_write,
+ .unlocked_ioctl = lrng_ioctl,
+ .compat_ioctl = compat_ptr_ioctl,
+ .fasync = lrng_fasync,
+ .llseek = noop_llseek,
+};
+
+SYSCALL_DEFINE3(getrandom, char __user *, buf, size_t, count,
+ unsigned int, flags)
+{
+ if (flags & ~(GRND_NONBLOCK|GRND_RANDOM|GRND_INSECURE))
+ return -EINVAL;
+
+ /*
+ * Requesting insecure and blocking randomness at the same time makes
+ * no sense.
+ */
+ if ((flags &
+ (GRND_INSECURE|GRND_RANDOM)) == (GRND_INSECURE|GRND_RANDOM))
+ return -EINVAL;
+
+ if (count > INT_MAX)
+ count = INT_MAX;
+
+ if (flags & GRND_INSECURE)
+ return lrng_drng_read(NULL, buf, count, NULL);
+
+ return lrng_read_common_block(flags & GRND_NONBLOCK, buf, count);
+}
diff --git a/drivers/char/lrng/lrng_internal.h b/drivers/char/lrng/lrng_internal.h
new file mode 100644
index 000000000000..edf38121ec65
--- /dev/null
+++ b/drivers/char/lrng/lrng_internal.h
@@ -0,0 +1,296 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * Copyright (C) 2018 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+#ifndef _LRNG_INTERNAL_H
+#define _LRNG_INTERNAL_H
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/spinlock.h>
+
+/*************************** General LRNG parameter ***************************/
+
+/* Entropy pool parameter
+ *
+ * The LRNG_POOL_SIZE cannot be smaller than 64 bytes as the SHA-1 operation
+ * in lrng_chacha20.c requires multiples of 64 bytes
+ */
+#define LRNG_POOL_SIZE (16 << CONFIG_LRNG_POOL_SIZE)
+#define LRNG_POOL_WORD_BYTES (4) /* (sizeof(atomic_t)) */
+#define LRNG_POOL_SIZE_BYTES (LRNG_POOL_SIZE * LRNG_POOL_WORD_BYTES)
+#define LRNG_POOL_SIZE_BITS (LRNG_POOL_SIZE_BYTES * 8)
+#define LRNG_POOL_WORD_BITS (LRNG_POOL_WORD_BYTES * 8)
+
+/* Security strength of LRNG -- this must match DRNG security strength */
+#define LRNG_DRNG_SECURITY_STRENGTH_BYTES 32
+#define LRNG_DRNG_SECURITY_STRENGTH_BITS (LRNG_DRNG_SECURITY_STRENGTH_BYTES * 8)
+#define LRNG_DRNG_BLOCKSIZE 64 /* Maximum of DRNG block sizes */
+
+/*
+ * SP800-90A defines a maximum request size of 1<<16 bytes. The given value is
+ * considered a safer margin.
+ *
+ * This value is allowed to be changed.
+ */
+#define LRNG_DRNG_MAX_REQSIZE (1<<12)
+
+/*
+ * SP800-90A defines a maximum number of requests between reseeds of 2^48.
+ * The given value is considered a much safer margin, balancing requests for
+ * frequent reseeds with the need to conserve entropy. This value MUST NOT be
+ * larger than INT_MAX because it is used in an atomic_t.
+ *
+ * This value is allowed to be changed.
+ */
+#define LRNG_DRNG_RESEED_THRESH (1<<20)
+
+/*
+ * Number of interrupts to be recorded to assume that DRNG security strength
+ * bits of entropy are received.
+ * Note: a value below the DRNG security strength should not be defined as this
+ * may imply the DRNG can never be fully seeded in case other noise
+ * sources are unavailable.
+ *
+ * This value is allowed to be changed.
+ */
+#define LRNG_IRQ_ENTROPY_BITS LRNG_DRNG_SECURITY_STRENGTH_BITS
+
+/*
+ * Min required seed entropy is 128 bits covering the minimum entropy
+ * requirement of SP800-131A and the German BSI's TR02102.
+ *
+ * This value is allowed to be changed.
+ */
+#define LRNG_FULL_SEED_ENTROPY_BITS LRNG_DRNG_SECURITY_STRENGTH_BITS
+#define LRNG_MIN_SEED_ENTROPY_BITS 128
+#define LRNG_INIT_ENTROPY_BITS 32
+
+/*
+ * Amount of entropy that is lost with the conditioning functions of LFSR and
+ * hash_df as shown with the entropy analysis compliant to SP800-90B.
+ */
+#define LRNG_CONDITIONING_ENTROPY_LOSS 1
+
+/*
+ * Wakeup value
+ *
+ * This value is allowed to be changed.
+ */
+#if (LRNG_POOL_SIZE_BITS <= (LRNG_DRNG_SECURITY_STRENGTH_BITS * 2))
+# define LRNG_WRITE_WAKEUP_ENTROPY (LRNG_DRNG_SECURITY_STRENGTH_BITS + \
+ LRNG_CONDITIONING_ENTROPY_LOSS)
+#else
+# define LRNG_WRITE_WAKEUP_ENTROPY (LRNG_DRNG_SECURITY_STRENGTH_BITS * 2)
+#endif
+
+/*
+ * Oversampling factor of IRQ events to obtain
+ * LRNG_DRNG_SECURITY_STRENGTH_BYTES. This factor is used when a
+ * high-resolution time stamp is not available. In this case, jiffies and
+ * register contents are used to fill the entropy pool. These noise sources
+ * are much less entropic than the high-resolution timer. The entropy content
+ * is the entropy content assumed with LRNG_IRQ_ENTROPY_BITS divided by
+ * LRNG_IRQ_OVERSAMPLING_FACTOR.
+ *
+ * This value is allowed to be changed.
+ */
+#define LRNG_IRQ_OVERSAMPLING_FACTOR 10
+
+/*
+ * Alignmask which should cover all cipher implementations
+ * WARNING: If this is changed to a value larger than 8, manual
+ * alignment is necessary as older versions of GCC may not be capable
+ * of aligning stack variables at boundaries greater than 8.
+ * In this case, PTR_ALIGN must be used.
+ */
+#define LRNG_KCAPI_ALIGN 8
+
+/************************ Default DRNG implementation *************************/
+
+extern struct chacha20_state chacha20;
+extern const struct lrng_crypto_cb lrng_cc20_crypto_cb;
+void lrng_cc20_init_state(struct chacha20_state *state);
+
+/********************************** /proc *************************************/
+
+static inline void lrng_pool_inc_numa_node(void) { }
+
+/****************************** LRNG interfaces *******************************/
+
+extern u32 lrng_write_wakeup_bits;
+extern int lrng_drng_reseed_max_time;
+
+void lrng_writer_wakeup(void);
+void lrng_init_wakeup(void);
+void lrng_debug_report_seedlevel(const char *name);
+void lrng_process_ready_list(void);
+
+/************************** Entropy pool management ***************************/
+
+enum lrng_external_noise_source {
+ lrng_noise_source_hw,
+ lrng_noise_source_user
+};
+
+bool lrng_state_exseed_allow(enum lrng_external_noise_source source);
+void lrng_state_exseed_set(enum lrng_external_noise_source source, bool type);
+void lrng_state_init_seed_work(void);
+u32 lrng_avail_entropy(void);
+void lrng_set_entropy_thresh(u32 new);
+int lrng_pool_trylock(void);
+void lrng_pool_unlock(void);
+void lrng_reset_state(void);
+void lrng_pool_all_numa_nodes_seeded(void);
+bool lrng_state_min_seeded(void);
+bool lrng_state_fully_seeded(void);
+bool lrng_state_operational(void);
+bool lrng_pool_highres_timer(void);
+void lrng_pool_set_entropy(u32 entropy_bits);
+void lrng_pool_lfsr(const u8 *buf, u32 buflen);
+void lrng_pool_lfsr_nonaligned(const u8 *buf, u32 buflen);
+void lrng_pool_lfsr_u32(u32 value);
+void lrng_pool_add_irq(u32 irq_num);
+void lrng_pool_add_entropy(u32 entropy_bits);
+
+struct entropy_buf {
+ u8 a[LRNG_DRNG_SECURITY_STRENGTH_BYTES];
+ u8 b[LRNG_DRNG_SECURITY_STRENGTH_BYTES];
+ u8 c[LRNG_DRNG_SECURITY_STRENGTH_BYTES];
+ u32 now;
+};
+
+int lrng_fill_seed_buffer(const struct lrng_crypto_cb *crypto_cb, void *hash,
+ struct entropy_buf *entropy_buf, u32 entropy_retain);
+void lrng_init_ops(u32 seed_bits);
+
+/************************** Jitter RNG Noise Source ***************************/
+
+#ifdef CONFIG_LRNG_JENT
+u32 lrng_get_jent(u8 *outbuf, unsigned int outbuflen);
+u32 lrng_jent_entropylevel(void);
+#else /* CONFIG_CRYPTO_JITTERENTROPY */
+static inline u32 lrng_get_jent(u8 *outbuf, unsigned int outbuflen) {return 0; }
+static inline u32 lrng_jent_entropylevel(void) { return 0; }
+#endif /* CONFIG_CRYPTO_JITTERENTROPY */
+
+/*************************** CPU-based Noise Source ***************************/
+
+u32 lrng_get_arch(u8 *outbuf);
+u32 lrng_slow_noise_req_entropy(u32 required_entropy_bits);
+
+/****************************** DRNG processing *******************************/
+
+/* Secondary DRNG state handle */
+struct lrng_drng {
+ void *drng; /* DRNG handle */
+ void *hash; /* Hash handle */
+ const struct lrng_crypto_cb *crypto_cb; /* Crypto callbacks */
+ atomic_t requests; /* Number of DRNG requests */
+ unsigned long last_seeded; /* Last time it was seeded */
+ bool fully_seeded; /* Is DRNG fully seeded? */
+ bool force_reseed; /* Force a reseed */
+ struct mutex lock;
+ spinlock_t spin_lock;
+};
+
+extern struct mutex lrng_crypto_cb_update;
+
+struct lrng_drng *lrng_drng_init_instance(void);
+struct lrng_drng *lrng_drng_atomic_instance(void);
+
+static __always_inline bool lrng_drng_is_atomic(struct lrng_drng *drng)
+{
+ return (drng->drng == lrng_drng_atomic_instance()->drng);
+}
+
+/* Lock the DRNG */
+static __always_inline void lrng_drng_lock(struct lrng_drng *drng,
+ unsigned long *flags)
+{
+ /* Use spin lock in case the atomic DRNG context is used */
+ if (lrng_drng_is_atomic(drng)) {
+ spin_lock_irqsave(&drng->spin_lock, *flags);
+
+ /*
+ * In case a lock transition happened while we were spinning,
+ * catch this case and use the new lock type.
+ */
+ if (!lrng_drng_is_atomic(drng)) {
+ spin_unlock_irqrestore(&drng->spin_lock, *flags);
+ mutex_lock(&drng->lock);
+ }
+ } else {
+ mutex_lock(&drng->lock);
+ }
+}
+
+/* Unlock the DRNG */
+static __always_inline void lrng_drng_unlock(struct lrng_drng *drng,
+ unsigned long *flags)
+{
+ if (lrng_drng_is_atomic(drng))
+ spin_unlock_irqrestore(&drng->spin_lock, *flags);
+ else
+ mutex_unlock(&drng->lock);
+}
+
+bool lrng_get_available(void);
+void lrng_set_available(void);
+void lrng_drngs_init_cc20(void);
+void lrng_drng_reset(struct lrng_drng *drng);
+int lrng_drng_get_atomic(u8 *outbuf, u32 outbuflen);
+int lrng_drng_get_sleep(u8 *outbuf, u32 outbuflen);
+void lrng_drng_force_reseed(void);
+void lrng_drng_seed_work(struct work_struct *dummy);
+
+static inline struct lrng_drng **lrng_drng_instances(void) { return NULL; }
+static inline void lrng_drngs_numa_alloc(void) { return; }
+
+/************************** Health Test linking code **************************/
+
+enum lrng_health_res {
+ lrng_health_pass, /* Health test passes on time stamp */
+ lrng_health_fail_use, /* Time stamp unhealthy, but mix in */
+ lrng_health_fail_drop /* Time stamp unhealthy, drop it */
+};
+
+#ifdef CONFIG_LRNG_HEALTH_TESTS
+bool lrng_sp80090b_startup_complete(void);
+bool lrng_sp80090b_compliant(void);
+
+enum lrng_health_res lrng_health_test(u32 now_time);
+void lrng_health_disable(void);
+
+void lrng_reset(void);
+#else /* CONFIG_LRNG_HEALTH_TESTS */
+static inline bool lrng_sp80090b_startup_complete(void) { return true; }
+static inline bool lrng_sp80090b_compliant(void) { return false; }
+
+static inline enum lrng_health_res
+lrng_health_test(u32 now_time) { return lrng_health_pass; }
+static inline void lrng_health_disable(void) { }
+#endif /* CONFIG_LRNG_HEALTH_TESTS */
+
+/****************************** Helper code ***********************************/
+
+static inline u32 atomic_read_u32(atomic_t *v)
+{
+ return (u32)atomic_read(v);
+}
+
+/*************************** Auxiliary functions ******************************/
+
+void invalidate_batched_entropy(void);
+
+/***************************** Testing code ***********************************/
+
+#ifdef CONFIG_LRNG_TESTING
+bool lrng_raw_entropy_store(u32 value);
+#else /* CONFIG_LRNG_TESTING */
+static inline bool lrng_raw_entropy_store(u32 value) { return false; }
+#endif /* CONFIG_LRNG_TESTING */
+
+#endif /* _LRNG_INTERNAL_H */
diff --git a/drivers/char/lrng/lrng_lfsr.h b/drivers/char/lrng/lrng_lfsr.h
new file mode 100644
index 000000000000..f0e38a0aadfd
--- /dev/null
+++ b/drivers/char/lrng/lrng_lfsr.h
@@ -0,0 +1,152 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * LRNG Linear Feedback Shift Register
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+/* Status information about IRQ noise source */
+struct lrng_irq_info {
+ atomic_t num_events; /* Number of healthy IRQs since last read */
+ atomic_t num_events_thresh; /* Reseed threshold */
+ atomic_t reseed_in_progress; /* Flag for on executing reseed */
+ bool irq_highres_timer; /* Is high-resolution timer available? */
+ u32 irq_entropy_bits; /* LRNG_IRQ_ENTROPY_BITS? */
+};
+
+/*
+ * This is the entropy pool used by the slow noise source. Its size should
+ * be at least as large as LRNG_DRNG_SECURITY_STRENGTH_BITS.
+ *
+ * The pool array is aligned to 8 bytes to comfort the kernel crypto API cipher
+ * implementations of the hash functions used to read the pool: for some
+ * accelerated implementations, we need an alignment to avoid a realignment
+ * which involves memcpy(). The alignment to 8 bytes should satisfy all crypto
+ * implementations.
+ *
+ * LRNG_POOL_SIZE is allowed to be changed only if the taps of the polynomial
+ * used for the LFSR are changed as well. The size must be in powers of 2 due
+ * to the mask handling in lrng_pool_lfsr_u32 which uses AND instead of modulo.
+ */
+struct lrng_pool {
+ union {
+ struct {
+ /*
+ * hash_df implementation: counter, requested_bits and
+ * pool form a linear buffer that is used in the
+ * hash_df function specified in SP800-90A section
+ * 10.3.1
+ */
+ unsigned char counter;
+ __be32 requested_bits;
+
+ /* Pool */
+ atomic_t pool[LRNG_POOL_SIZE];
+ /* Ptr into pool for next IRQ word injection */
+ atomic_t pool_ptr;
+ /* rotate for LFSR */
+ atomic_t input_rotate;
+ /* All NUMA DRNGs seeded? */
+ bool all_online_numa_node_seeded;
+ /* IRQ noise source status info */
+ struct lrng_irq_info irq_info;
+ /* Serialize read of entropy pool */
+ spinlock_t lock;
+ };
+ /*
+ * Static SHA-1 implementation in lrng_cc20_hash_buffer
+ * processes data 64-byte-wise. Hence, ensure proper size
+ * of LRNG entropy pool data structure.
+ */
+ u8 hash_input_buf[LRNG_POOL_SIZE_BYTES + 64];
+ };
+};
+
+/*
+ * Implement a (modified) twisted Generalized Feedback Shift Register. (See M.
+ * Matsumoto & Y. Kurita, 1992. Twisted GFSR generators. ACM Transactions on
+ * Modeling and Computer Simulation 2(3):179-194. Also see M. Matsumoto & Y.
+ * Kurita, 1994. Twisted GFSR generators II. ACM Transactions on Modeling and
+ * Computer Simulation 4:254-266).
+ */
+static u32 const lrng_twist_table[8] = {
+ 0x00000000, 0x3b6e20c8, 0x76dc4190, 0x4db26158,
+ 0xedb88320, 0xd6d6a3e8, 0x9b64c2b0, 0xa00ae278 };
+
+/*
+ * The polynomials for the LFSR are taken from the document "Table of Linear
+ * Feedback Shift Registers" by Roy Ward, Tim Molteno, October 26, 2007.
+ * The first polynomial is from "Primitive Binary Polynomials" by Wayne
+ * Stahnke (1973) and is primitive as well as irreducible.
+ *
+ * Note, the tap values are smaller by one compared to the documentation because
+ * they are used as an index into an array where the index starts by zero.
+ *
+ * All polynomials were also checked to be primitive and irreducible with magma
+ * which ensures that the key property of the LFSR providing a compression
+ * function for entropy is guaranteed.
+ */
+static u32 const lrng_lfsr_polynomial[][4] = {
+ { 15, 13, 12, 10 }, /* 16 words */
+ { 31, 29, 25, 24 }, /* 32 words */
+ { 63, 62, 60, 59 }, /* 64 words */
+ { 127, 28, 26, 1 }, /* 128 words by Stahnke */
+ { 255, 253, 250, 245 }, /* 256 words */
+ { 511, 509, 506, 503 }, /* 512 words */
+ { 1023, 1014, 1001, 1000 }, /* 1024 words */
+ { 2047, 2034, 2033, 2028 }, /* 2048 words */
+ { 4095, 4094, 4080, 4068 }, /* 4096 words */
+};
+
+static inline void _lrng_pool_lfsr_u32(struct lrng_pool *pool, u32 value)
+{
+ /*
+ * Process the LFSR by altering not adjacent words but rather
+ * more spaced apart words. Using a prime number ensures that all words
+ * are processed evenly. As some the LFSR polynomials taps are close
+ * together, processing adjacent words with the LSFR taps may be
+ * inappropriate as the data just mixed-in at these taps may be not
+ * independent from the current data to be mixed in.
+ */
+ u32 ptr = (u32)atomic_add_return_relaxed(67, &pool->pool_ptr) &
+ (LRNG_POOL_SIZE - 1);
+ /*
+ * Add 7 bits of rotation to the pool. At the beginning of the
+ * pool, add an extra 7 bits rotation, so that successive passes
+ * spread the input bits across the pool evenly.
+ *
+ * Note, there is a race between getting ptr and calculating
+ * input_rotate when ptr is is obtained on two or more CPUs at the
+ * same time. This race is irrelevant as it may only come into effect
+ * if 3 or more CPUs race at the same time which is very unlikely. If
+ * the race happens, it applies to one event only. As this rolling
+ * supports the LFSR without being strictly needed, we accept this
+ * race.
+ */
+ u32 input_rotate = (u32)atomic_add_return_relaxed((ptr ? 7 : 14),
+ &pool->input_rotate) & 31;
+ u32 word = rol32(value, input_rotate);
+
+ BUILD_BUG_ON(LRNG_POOL_WORD_BYTES != sizeof(pool->pool[0]));
+ BUILD_BUG_ON(LRNG_POOL_SIZE - 1 !=
+ lrng_lfsr_polynomial[CONFIG_LRNG_POOL_SIZE][0]);
+ word ^= atomic_read_u32(&pool->pool[ptr]);
+ word ^= atomic_read_u32(&pool->pool[
+ (ptr + lrng_lfsr_polynomial[CONFIG_LRNG_POOL_SIZE][0]) &
+ (LRNG_POOL_SIZE - 1)]);
+ word ^= atomic_read_u32(&pool->pool[
+ (ptr + lrng_lfsr_polynomial[CONFIG_LRNG_POOL_SIZE][1]) &
+ (LRNG_POOL_SIZE - 1)]);
+ word ^= atomic_read_u32(&pool->pool[
+ (ptr + lrng_lfsr_polynomial[CONFIG_LRNG_POOL_SIZE][2]) &
+ (LRNG_POOL_SIZE - 1)]);
+ word ^= atomic_read_u32(&pool->pool[
+ (ptr + lrng_lfsr_polynomial[CONFIG_LRNG_POOL_SIZE][3]) &
+ (LRNG_POOL_SIZE - 1)]);
+
+ word = (word >> 3) ^ lrng_twist_table[word & 7];
+ atomic_set(&pool->pool[ptr], word);
+}
+
+u32 __lrng_pool_hash_df(const struct lrng_crypto_cb *crypto_cb, void *hash,
+ struct lrng_pool *pool, u8 *outbuf, u32 requested_bits);
diff --git a/drivers/char/lrng/lrng_pool.c b/drivers/char/lrng/lrng_pool.c
new file mode 100644
index 000000000000..d1c76532a759
--- /dev/null
+++ b/drivers/char/lrng/lrng_pool.c
@@ -0,0 +1,588 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG Entropy pool management
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <asm/irq_regs.h>
+#include <linux/lrng.h>
+#include <linux/percpu.h>
+#include <linux/random.h>
+#include <linux/utsname.h>
+#include <linux/workqueue.h>
+
+#include "lrng_internal.h"
+#include "lrng_lfsr.h"
+
+struct lrng_state {
+ bool lrng_operational; /* Is DRNG operational? */
+ bool lrng_fully_seeded; /* Is DRNG fully seeded? */
+ bool lrng_min_seeded; /* Is DRNG minimally seeded? */
+
+ /*
+ * To ensure that external entropy providers cannot dominate the
+ * internal noise sources but yet cannot be dominated by internal
+ * noise sources, the following booleans are intended to allow
+ * external to provide seed once when a DRNG reseed occurs. This
+ * triggering of external noise source is performed even when the
+ * entropy pool has sufficient entropy.
+ */
+ bool lrng_seed_hw; /* Allow HW to provide seed */
+ bool lrng_seed_user; /* Allow user space to provide seed */
+
+ struct work_struct lrng_seed_work; /* (re)seed work queue */
+};
+
+static struct lrng_pool lrng_pool __aligned(LRNG_KCAPI_ALIGN) = {
+ .irq_info = {
+ .irq_entropy_bits = LRNG_IRQ_ENTROPY_BITS,
+ .num_events_thresh = ATOMIC_INIT(LRNG_INIT_ENTROPY_BITS +
+ LRNG_CONDITIONING_ENTROPY_LOSS),
+ /* Sample IRQ pointer data at least during boot */
+ .irq_highres_timer = false },
+ .lock = __SPIN_LOCK_UNLOCKED(lrng_pool.lock)
+};
+
+static struct lrng_state lrng_state = { false, false, false, true, true };
+
+/********************************** Helper ***********************************/
+
+/* External entropy provider is allowed to provide seed data */
+bool lrng_state_exseed_allow(enum lrng_external_noise_source source)
+{
+ if (source == lrng_noise_source_hw)
+ return lrng_state.lrng_seed_hw;
+ return lrng_state.lrng_seed_user;
+}
+
+/* Enable / disable external entropy provider to furnish seed */
+void lrng_state_exseed_set(enum lrng_external_noise_source source, bool type)
+{
+ if (source == lrng_noise_source_hw)
+ lrng_state.lrng_seed_hw = type;
+ else
+ lrng_state.lrng_seed_user = type;
+}
+
+static inline void lrng_state_exseed_allow_all(void)
+{
+ lrng_state_exseed_set(lrng_noise_source_hw, true);
+ lrng_state_exseed_set(lrng_noise_source_user, true);
+}
+
+void lrng_state_init_seed_work(void)
+{
+ INIT_WORK(&lrng_state.lrng_seed_work, lrng_drng_seed_work);
+}
+
+static inline u32 lrng_entropy_to_data(u32 entropy_bits)
+{
+ return ((entropy_bits * lrng_pool.irq_info.irq_entropy_bits) /
+ LRNG_DRNG_SECURITY_STRENGTH_BITS);
+}
+
+static inline u32 lrng_data_to_entropy(u32 irqnum)
+{
+ return ((irqnum * LRNG_DRNG_SECURITY_STRENGTH_BITS) /
+ lrng_pool.irq_info.irq_entropy_bits);
+}
+
+u32 lrng_avail_entropy(void)
+{
+ return min_t(u32, LRNG_POOL_SIZE_BITS, lrng_data_to_entropy(
+ atomic_read_u32(&lrng_pool.irq_info.num_events)));
+}
+
+void lrng_set_entropy_thresh(u32 new)
+{
+ atomic_set(&lrng_pool.irq_info.num_events_thresh,
+ lrng_entropy_to_data(new));
+}
+
+/*
+ * Reading of the LRNG pool is only allowed by one caller. The reading is
+ * only performed to (re)seed DRNGs. Thus, if this "lock" is already taken,
+ * the reseeding operation is in progress. The caller is not intended to wait
+ * but continue with its other operation.
+ */
+int lrng_pool_trylock(void)
+{
+ return atomic_cmpxchg(&lrng_pool.irq_info.reseed_in_progress, 0, 1);
+}
+
+void lrng_pool_unlock(void)
+{
+ atomic_set(&lrng_pool.irq_info.reseed_in_progress, 0);
+}
+
+void lrng_reset_state(void)
+{
+ struct lrng_irq_info *irq_info = &lrng_pool.irq_info;
+
+ atomic_set(&irq_info->num_events, 0);
+ lrng_state.lrng_operational = false;
+ lrng_state.lrng_fully_seeded = false;
+ lrng_state.lrng_min_seeded = false;
+ lrng_pool.all_online_numa_node_seeded = false;
+ pr_debug("reset LRNG\n");
+}
+
+void lrng_pool_all_numa_nodes_seeded(void)
+{
+ lrng_pool.all_online_numa_node_seeded = true;
+}
+
+bool lrng_state_min_seeded(void)
+{
+ return lrng_state.lrng_min_seeded;
+}
+
+bool lrng_state_fully_seeded(void)
+{
+ return lrng_state.lrng_fully_seeded;
+}
+
+bool lrng_state_operational(void)
+{
+ return lrng_state.lrng_operational;
+}
+
+bool lrng_pool_highres_timer(void)
+{
+ return lrng_pool.irq_info.irq_highres_timer;
+}
+
+void lrng_pool_set_entropy(u32 entropy_bits)
+{
+ atomic_set(&lrng_pool.irq_info.num_events,
+ lrng_entropy_to_data(entropy_bits));
+}
+
+static void lrng_pool_configure(bool highres_timer, u32 irq_entropy_bits)
+{
+ struct lrng_irq_info *irq_info = &lrng_pool.irq_info;
+
+ irq_info->irq_highres_timer = highres_timer;
+ if (irq_info->irq_entropy_bits != irq_entropy_bits) {
+ irq_info->irq_entropy_bits = irq_entropy_bits;
+ /* Reset the threshold based on new oversampling factor. */
+ lrng_set_entropy_thresh(atomic_read_u32(
+ &irq_info->num_events_thresh));
+ }
+}
+
+static int __init lrng_init_time_source(void)
+{
+ if (random_get_entropy() || random_get_entropy()) {
+ /*
+ * As the highres timer is identified here, previous interrupts
+ * obtained during boot time are treated like a lowres-timer
+ * would have been present.
+ */
+ lrng_pool_configure(true, LRNG_IRQ_ENTROPY_BITS);
+ } else {
+ lrng_health_disable();
+ lrng_pool_configure(false, LRNG_IRQ_ENTROPY_BITS *
+ LRNG_IRQ_OVERSAMPLING_FACTOR);
+ pr_warn("operating without high-resolution timer and applying "
+ "IRQ oversampling factor %u\n",
+ LRNG_IRQ_OVERSAMPLING_FACTOR);
+ }
+
+ return 0;
+}
+
+core_initcall(lrng_init_time_source);
+
+/* invoke function with buffer aligned to 4 bytes */
+void lrng_pool_lfsr(const u8 *buf, u32 buflen)
+{
+ u32 *p_buf = (u32 *)buf;
+
+ for (; buflen >= 4; buflen -= 4)
+ lrng_pool_lfsr_u32(*p_buf++);
+
+ buf = (u8 *)p_buf;
+ while (buflen--)
+ lrng_pool_lfsr_u32(*buf++);
+}
+
+void lrng_pool_lfsr_nonaligned(const u8 *buf, u32 buflen)
+{
+ while (buflen) {
+ if (!((unsigned long)buf & (sizeof(u32) - 1))) {
+ lrng_pool_lfsr(buf, buflen);
+ return;
+ }
+
+ lrng_pool_lfsr_u32(*buf++);
+ buflen--;
+ }
+}
+
+/**************************** Interrupt processing ****************************/
+
+/*
+ * Hot code path - inject data into entropy pool using LFSR
+ */
+void lrng_pool_lfsr_u32(u32 value)
+{
+ _lrng_pool_lfsr_u32(&lrng_pool, value);
+}
+
+/*
+ * Hot code path - mix data into entropy pool
+ */
+void lrng_pool_add_irq(u32 irq_num)
+{
+ struct lrng_irq_info *irq_info = &lrng_pool.irq_info;
+
+ atomic_add(irq_num, &irq_info->num_events);
+
+ /*
+ * Once all DRNGs are fully seeded, the interrupt noise
+ * sources will not trigger any reseeding any more.
+ */
+ if (likely(lrng_pool.all_online_numa_node_seeded))
+ return;
+
+ /* Only try to reseed if the DRNG is alive. */
+ if (!lrng_get_available())
+ return;
+
+ /* Only trigger the DRNG reseed if we have collected enough IRQs. */
+ if (atomic_read_u32(&lrng_pool.irq_info.num_events) <
+ atomic_read_u32(&lrng_pool.irq_info.num_events_thresh))
+ return;
+
+ /* Ensure that the seeding only occurs once at any given time. */
+ if (lrng_pool_trylock())
+ return;
+
+ /* Seed the DRNG with IRQ noise. */
+ schedule_work(&lrng_state.lrng_seed_work);
+}
+
+void lrng_pool_add_entropy(u32 entropy_bits)
+{
+ lrng_pool_add_irq(lrng_entropy_to_data(entropy_bits));
+}
+
+/*
+ * Generate a hashed output of pool using the SP800-90A section 10.3.1 hash_df
+ * function
+ */
+u32 __lrng_pool_hash_df(const struct lrng_crypto_cb *crypto_cb, void *hash,
+ struct lrng_pool *pool, u8 *outbuf, u32 requested_bits)
+{
+ u32 digestsize, requested_bytes = requested_bits >> 3,
+ generated_bytes = 0;
+ u8 digest[64] __aligned(LRNG_KCAPI_ALIGN);
+
+ digestsize = crypto_cb->lrng_hash_digestsize(hash);
+ if (digestsize > sizeof(digest)) {
+ pr_err("Digest buffer too small\n");
+ return 0;
+ }
+
+ pool->counter = 1;
+ pool->requested_bits = cpu_to_be32(requested_bytes << 3);
+
+ while (requested_bytes) {
+ u32 tocopy = min_t(u32, requested_bytes, digestsize);
+
+ /* The counter must not wrap */
+ if (pool->counter == 0)
+ goto out;
+
+ if (crypto_cb->lrng_hash_buffer(hash, (u8 *)pool,
+ LRNG_POOL_SIZE_BYTES + 64,
+ digest))
+ goto out;
+
+ /* Copy the data out to the caller */
+ memcpy(outbuf + generated_bytes, digest, tocopy);
+ requested_bytes -= tocopy;
+ generated_bytes += tocopy;
+ pool->counter++;
+ }
+
+out:
+ /* Mix read data back into pool for backtracking resistance */
+ if (generated_bytes)
+ lrng_pool_lfsr(outbuf, generated_bytes);
+ memzero_explicit(digest, digestsize);
+ return (generated_bytes<<3);
+}
+
+static inline u32 lrng_pool_hash_df(const struct lrng_crypto_cb *crypto_cb,
+ void *hash, u8 *outbuf, u32 requested_bits)
+{
+ return __lrng_pool_hash_df(crypto_cb, hash, &lrng_pool, outbuf,
+ requested_bits);
+}
+
+/**
+ * lrng_get_pool() - Read the entropy pool out for use.
+ *
+ * This function handles the translation from the number of received interrupts
+ * into an entropy statement. The conversion depends on LRNG_IRQ_ENTROPY_BITS
+ * which defines how many interrupts must be received to obtain 256 bits of
+ * entropy. With this value, the function lrng_data_to_entropy converts a given
+ * data size (received interrupts, requested amount of data, etc.) into an
+ * entropy statement. lrng_entropy_to_data does the reverse.
+ *
+ * Both functions are agnostic about the type of data: when the number of
+ * interrupts is processed by these functions, the resulting entropy value is in
+ * bits as we assume the entropy of interrupts is measured in bits. When data is
+ * processed, the entropy value is in bytes as the data is measured in bytes.
+ *
+ * @outbuf: buffer to store data in with size LRNG_DRNG_SECURITY_STRENGTH_BYTES
+ * @requested_entropy_bits: requested bits of entropy -- the function will
+ * return at least this amount of entropy if available
+ * @entropy_retain: amount of entropy in bits that should be left in the pool
+ *
+ * Return: estimated entropy from the IRQs that was obtained
+ */
+static u32 lrng_get_pool(const struct lrng_crypto_cb *crypto_cb, void *hash,
+ u8 *outbuf, u32 requested_entropy_bits,
+ u32 entropy_retain)
+{
+ struct lrng_pool *pool = &lrng_pool;
+ struct lrng_state *state = &lrng_state;
+ struct lrng_irq_info *irq_info = &pool->irq_info;
+ unsigned long flags;
+
+ u32 irq_num_events_used, irq_num_events, avail_entropy_bits;
+
+ /* This get_pool operation must only be called once at a given time! */
+ spin_lock_irqsave(&pool->lock, flags);
+
+ /* How many unused interrupts are in entropy pool? */
+ irq_num_events = atomic_read_u32(&irq_info->num_events);
+ /* Convert available interrupts into entropy statement */
+ avail_entropy_bits = lrng_data_to_entropy(irq_num_events);
+
+ /* Cap available entropy to pool size */
+ avail_entropy_bits =
+ min_t(u32, avail_entropy_bits, LRNG_POOL_SIZE_BITS);
+
+ /* How much entropy we need to and can we use? */
+ if (unlikely(!state->lrng_fully_seeded)) {
+ /*
+ * During boot time, we read 256 bits data with
+ * avail_entropy_bits entropy. In case our conservative
+ * entropy estimate underestimates the available entropy
+ * we can transport as much available entropy as
+ * possible. The entropy pool does not operate compliant to
+ * the German AIS 21/31 NTG.1 yet.
+ */
+ requested_entropy_bits =
+ LRNG_DRNG_SECURITY_STRENGTH_BITS;
+ } else {
+ /* Provide all entropy above retaining level */
+ if (avail_entropy_bits < entropy_retain) {
+ requested_entropy_bits = 0;
+ goto out;
+ }
+ avail_entropy_bits -= entropy_retain;
+ requested_entropy_bits = min_t(u32, avail_entropy_bits,
+ requested_entropy_bits);
+ }
+
+ /* Hash is a compression function: we generate entropy amount of data */
+ requested_entropy_bits = round_down(requested_entropy_bits, 8);
+
+ requested_entropy_bits = lrng_pool_hash_df(crypto_cb, hash, outbuf,
+ requested_entropy_bits);
+
+ /* Boot time: After getting the full buffer adjust the entropy value. */
+ requested_entropy_bits = min_t(u32, avail_entropy_bits,
+ requested_entropy_bits);
+
+out:
+ /* Convert used entropy into interrupt number for subtraction */
+ irq_num_events_used = lrng_entropy_to_data(requested_entropy_bits);
+
+ /*
+ * The hash_df operation entropy assessment shows that the output
+ * entropy is one bit smaller than the input entropy. Therefore we
+ * account for this one bit of entropy here: if we have sufficient
+ * entropy in the LFSR, we say we used one bit of entropy more.
+ * Otherwise we reduce the amount of entropy we say we generated with
+ * the hash_df.
+ */
+ if (irq_num_events_used) {
+ if ((irq_num_events_used + LRNG_CONDITIONING_ENTROPY_LOSS) <=
+ lrng_entropy_to_data(avail_entropy_bits)) {
+ irq_num_events_used += LRNG_CONDITIONING_ENTROPY_LOSS;
+ } else {
+ if (unlikely(requested_entropy_bits <
+ LRNG_CONDITIONING_ENTROPY_LOSS))
+ requested_entropy_bits = 0;
+ else
+ requested_entropy_bits -=
+ LRNG_CONDITIONING_ENTROPY_LOSS;
+ }
+ }
+
+ /*
+ * New events might have arrived in the meanwhile and we don't
+ * want to throw them away unconditionally. On the other hand,
+ * these new events might have been mixed in before
+ * lrng_hash_df_pool() had been able to draw any entropy
+ * from the pool and thus, the pool capacity might have been
+ * exceeded at some point. Note that in theory, some events
+ * might get lost inbetween the atomic_read() and
+ * atomic_set() below. But that's fine, because it's no real
+ * concern while code preventing this would come at the cost of
+ * additional complexity. Likewise, some events which arrived
+ * after full or partial completion of the __lrng_hash_df_pool()
+ * above might get unnecessarily thrown away by the min()
+ * operation below; the same argument applies there.
+ */
+ irq_num_events = atomic_read_u32(&irq_info->num_events);
+ irq_num_events = min_t(u32, irq_num_events,
+ lrng_entropy_to_data(LRNG_POOL_SIZE_BITS));
+ irq_num_events -= irq_num_events_used;
+ atomic_set(&irq_info->num_events, irq_num_events);
+
+ spin_unlock_irqrestore(&pool->lock, flags);
+
+ /* Obtain entropy statement in bits from the used entropy */
+ pr_debug("obtained %u bits of entropy from %u newly collected "
+ "interrupts - not using %u interrupts\n",
+ requested_entropy_bits, irq_num_events_used,
+ irq_num_events);
+
+ return requested_entropy_bits;
+}
+
+/* Fill the seed buffer with data from the noise sources */
+int lrng_fill_seed_buffer(const struct lrng_crypto_cb *crypto_cb, void *hash,
+ struct entropy_buf *entropy_buf, u32 entropy_retain)
+{
+ struct lrng_state *state = &lrng_state;
+ u32 total_entropy_bits = 0;
+
+ /* Require at least 128 bits of entropy for any reseed. */
+ if (state->lrng_fully_seeded &&
+ (lrng_avail_entropy() <
+ lrng_slow_noise_req_entropy(LRNG_MIN_SEED_ENTROPY_BITS +
+ LRNG_CONDITIONING_ENTROPY_LOSS) +
+ entropy_retain))
+ goto wakeup;
+
+ /*
+ * Concatenate the output of the noise sources. This would be the
+ * spot to add an entropy extractor logic if desired. Note, this
+ * has the ability to collect entropy equal or larger than the DRNG
+ * strength.
+ */
+ total_entropy_bits = lrng_get_pool(crypto_cb, hash, entropy_buf->a,
+ LRNG_DRNG_SECURITY_STRENGTH_BITS,
+ entropy_retain);
+ total_entropy_bits += lrng_get_arch(entropy_buf->b);
+ total_entropy_bits += lrng_get_jent(entropy_buf->c,
+ LRNG_DRNG_SECURITY_STRENGTH_BYTES);
+
+ /* also reseed the DRNG with the current time stamp */
+ entropy_buf->now = random_get_entropy();
+
+ /* allow external entropy provider to provide seed */
+ lrng_state_exseed_allow_all();
+
+wakeup:
+ /*
+ * Shall we wake up user space writers? This location covers
+ * ensures that the user space provider does not dominate the internal
+ * noise sources since in case the first call of this function finds
+ * sufficient entropy in the entropy pool, it will not trigger the
+ * wakeup. This implies that when the next /dev/urandom read happens,
+ * the entropy pool is drained.
+ */
+ lrng_writer_wakeup();
+
+ return total_entropy_bits;
+}
+
+/**
+ * lrng_init_ops() - Set seed stages of LRNG
+ *
+ * Set the slow noise source reseed trigger threshold. The initial threshold
+ * is set to the minimum data size that can be read from the pool: a word. Upon
+ * reaching this value, the next seed threshold of 128 bits is set followed
+ * by 256 bits.
+ *
+ * @entropy_bits: size of entropy currently injected into DRNG
+ */
+void lrng_init_ops(u32 seed_bits)
+{
+ struct lrng_state *state = &lrng_state;
+
+ if (state->lrng_operational)
+ return;
+
+ /* DRNG is seeded with full security strength */
+ if (state->lrng_fully_seeded) {
+ state->lrng_operational = lrng_sp80090b_startup_complete();
+ lrng_process_ready_list();
+ lrng_init_wakeup();
+ } else if (seed_bits >= LRNG_FULL_SEED_ENTROPY_BITS) {
+ invalidate_batched_entropy();
+ state->lrng_fully_seeded = true;
+ state->lrng_operational = lrng_sp80090b_startup_complete();
+ state->lrng_min_seeded = true;
+ pr_info("LRNG fully seeded with %u bits of entropy\n",
+ seed_bits);
+ lrng_set_entropy_thresh(LRNG_FULL_SEED_ENTROPY_BITS +
+ LRNG_CONDITIONING_ENTROPY_LOSS);
+ lrng_process_ready_list();
+ lrng_init_wakeup();
+
+ } else if (!state->lrng_min_seeded) {
+
+ /* DRNG is seeded with at least 128 bits of entropy */
+ if (seed_bits >= LRNG_MIN_SEED_ENTROPY_BITS) {
+ invalidate_batched_entropy();
+ state->lrng_min_seeded = true;
+ pr_info("LRNG minimally seeded with %u bits of "
+ "entropy\n", seed_bits);
+ lrng_set_entropy_thresh(
+ lrng_slow_noise_req_entropy(
+ LRNG_FULL_SEED_ENTROPY_BITS +
+ LRNG_CONDITIONING_ENTROPY_LOSS));
+ lrng_process_ready_list();
+ lrng_init_wakeup();
+
+ /* DRNG is seeded with at least LRNG_INIT_ENTROPY_BITS bits */
+ } else if (seed_bits >= LRNG_INIT_ENTROPY_BITS) {
+ pr_info("LRNG initial entropy level %u bits of "
+ "entropy\n", seed_bits);
+ lrng_set_entropy_thresh(
+ lrng_slow_noise_req_entropy(
+ LRNG_MIN_SEED_ENTROPY_BITS +
+ LRNG_CONDITIONING_ENTROPY_LOSS));
+ }
+ }
+}
+
+int __init rand_initialize(void)
+{
+ ktime_t now_time = ktime_get_real();
+ unsigned int i, rand;
+
+ lrng_pool_lfsr_u32(now_time);
+ for (i = 0; i < LRNG_POOL_SIZE; i++) {
+ if (!arch_get_random_seed_int(&rand) &&
+ !arch_get_random_int(&rand))
+ rand = random_get_entropy();
+ lrng_pool_lfsr_u32(rand);
+ }
+ lrng_pool_lfsr_nonaligned((u8 *)utsname(), sizeof(*(utsname())));
+
+ return 0;
+}
diff --git a/drivers/char/lrng/lrng_sw_noise.c b/drivers/char/lrng/lrng_sw_noise.c
new file mode 100644
index 000000000000..c21db23875fc
--- /dev/null
+++ b/drivers/char/lrng/lrng_sw_noise.c
@@ -0,0 +1,102 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG Slow Noise Source: Interrupt data collection
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+#include <asm/irq_regs.h>
+#include <asm/ptrace.h>
+#include <linux/random.h>
+
+#include "lrng_internal.h"
+#include "lrng_sw_noise.h"
+
+/* Holder of time stamps before mixing them into the entropy pool */
+static DEFINE_PER_CPU(u32 [LRNG_TIME_ARRAY_SIZE], lrng_time);
+static DEFINE_PER_CPU(u32, lrng_time_ptr) = 0;
+static DEFINE_PER_CPU(u8, lrng_time_irqs) = 0;
+
+/*
+ * Batching up of entropy in per-CPU array before injecting into entropy pool.
+ */
+static inline void lrng_time_process(void)
+{
+ u32 i, ptr, now_time = random_get_entropy() &
+ (likely(lrng_state_fully_seeded()) ?
+ LRNG_TIME_SLOTSIZE_MASK : (u32)-1);
+ enum lrng_health_res health_test;
+
+ /* Ensure sufficient space in lrng_time_irqs */
+ BUILD_BUG_ON(LRNG_TIME_NUM_VALUES >= (1 << (sizeof(u8) << 3)));
+ BUILD_BUG_ON(LRNG_TIME_ARRAY_MEMBER_BITS % LRNG_TIME_SLOTSIZE_BITS);
+ /* Ensure consistency of values */
+ BUILD_BUG_ON(LRNG_TIME_ARRAY_MEMBER_BITS != sizeof(lrng_time[0]) << 3);
+
+ if (lrng_raw_entropy_store(now_time))
+ return;
+
+ health_test = lrng_health_test(now_time);
+ if (health_test > lrng_health_fail_use)
+ return;
+
+ /* During boot time, we mix the full time stamp directly into LFSR */
+ if (unlikely(!lrng_state_fully_seeded())) {
+ lrng_pool_lfsr_u32(now_time);
+ if (health_test == lrng_health_pass)
+ lrng_pool_add_irq(1);
+ return;
+ }
+
+ ptr = this_cpu_inc_return(lrng_time_ptr) & LRNG_TIME_WORD_MASK;
+ this_cpu_or(lrng_time[lrng_time_idx2array(ptr)],
+ lrng_time_slot_val(now_time & LRNG_TIME_SLOTSIZE_MASK,
+ lrng_time_idx2slot(ptr)));
+
+ /* Interrupt delivers entropy if health test passes */
+ if (health_test == lrng_health_pass)
+ this_cpu_inc(lrng_time_irqs);
+
+ /* Only mix the buffer of time stamps into LFSR when wrapping */
+ if (ptr < LRNG_TIME_WORD_MASK)
+ return;
+
+ for (i = 0; i < LRNG_TIME_ARRAY_SIZE; i++) {
+ lrng_pool_lfsr_u32(this_cpu_read(lrng_time[i]));
+ this_cpu_write(lrng_time[i], 0);
+ }
+ lrng_pool_add_irq(this_cpu_read(lrng_time_irqs));
+ this_cpu_write(lrng_time_irqs, 0);
+}
+
+/*
+ * Hot code path - Callback for interrupt handler
+ */
+void add_interrupt_randomness(int irq, int irq_flags)
+{
+ lrng_time_process();
+
+ if (!lrng_pool_highres_timer()) {
+ struct pt_regs *regs = get_irq_regs();
+ static atomic_t reg_idx = ATOMIC_INIT(0);
+ u64 ip;
+
+ lrng_pool_lfsr_u32(jiffies);
+ lrng_pool_lfsr_u32(irq);
+ lrng_pool_lfsr_u32(irq_flags);
+
+ if (regs) {
+ u32 *ptr = (u32 *)regs;
+ int reg_ptr = atomic_add_return_relaxed(1, ®_idx);
+ size_t n = (sizeof(struct pt_regs) / sizeof(u32));
+
+ ip = instruction_pointer(regs);
+ lrng_pool_lfsr_u32(*(ptr + (reg_ptr % n)));
+ } else
+ ip = _RET_IP_;
+
+ lrng_pool_lfsr_u32(ip >> 32);
+ lrng_pool_lfsr_u32(ip);
+ }
+}
+EXPORT_SYMBOL(add_interrupt_randomness);
diff --git a/drivers/char/lrng/lrng_sw_noise.h b/drivers/char/lrng/lrng_sw_noise.h
new file mode 100644
index 000000000000..107c6f9153bc
--- /dev/null
+++ b/drivers/char/lrng/lrng_sw_noise.h
@@ -0,0 +1,57 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * LRNG Slow Noise Source: Time stamp array handling
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+/*
+ * To limit the impact on the interrupt handling, the LRNG concatenates
+ * entropic LSB parts of the time stamps in a per-CPU array and only
+ * injects them into the entropy pool when the array is full.
+ */
+
+/* Store multiple integers in one u32 */
+#define LRNG_TIME_SLOTSIZE_BITS (8)
+#define LRNG_TIME_SLOTSIZE_MASK ((1 << LRNG_TIME_SLOTSIZE_BITS) - 1)
+#define LRNG_TIME_ARRAY_MEMBER_BITS (4 << 3)
+#define LRNG_TIME_SLOTS_PER_UINT (LRNG_TIME_ARRAY_MEMBER_BITS / \
+ LRNG_TIME_SLOTSIZE_BITS)
+
+/*
+ * Number of time values to store in the array - in small environments
+ * only one atomic_t variable per CPU is used.
+ */
+#define LRNG_TIME_NUM_VALUES (CONFIG_BASE_SMALL ? \
+ LRNG_TIME_SLOTS_PER_UINT : 64)
+/* Mask of LSB of time stamp to store */
+#define LRNG_TIME_WORD_MASK (LRNG_TIME_NUM_VALUES - 1)
+
+#define LRNG_TIME_SLOTS_MASK (LRNG_TIME_SLOTS_PER_UINT - 1)
+#define LRNG_TIME_ARRAY_SIZE (LRNG_TIME_NUM_VALUES / \
+ LRNG_TIME_SLOTS_PER_UINT)
+
+/* Starting bit index of slot */
+static inline unsigned int lrng_time_slot2bitindex(unsigned int slot)
+{
+ return (LRNG_TIME_SLOTSIZE_BITS * slot);
+}
+
+/* Convert index into the array index */
+static inline unsigned int lrng_time_idx2array(unsigned int idx)
+{
+ return idx / LRNG_TIME_SLOTS_PER_UINT;
+}
+
+/* Convert index into the slot of a given array index */
+static inline unsigned int lrng_time_idx2slot(unsigned int idx)
+{
+ return idx & LRNG_TIME_SLOTS_MASK;
+}
+
+/* Convert value into slot value */
+static inline unsigned int lrng_time_slot_val(unsigned int val,
+ unsigned int slot)
+{
+ return val << lrng_time_slot2bitindex(slot);
+}
diff --git a/include/linux/lrng.h b/include/linux/lrng.h
new file mode 100644
index 000000000000..2c3d2ed32a91
--- /dev/null
+++ b/include/linux/lrng.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/*
+ * Copyright (C) 2018 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+#ifndef _LRNG_H
+#define _LRNG_H
+
+#include <linux/errno.h>
+#include <linux/types.h>
+
+/**
+ * struct lrng_crypto_cb - cryptographic callback functions
+ * @lrng_drng_name Name of DRNG
+ * @lrng_hash_name Name of Hash used for reading entropy pool
+ * @lrng_drng_alloc: Allocate DRNG -- the provided integer should be
+ * used for sanity checks.
+ * return: allocated data structure or PTR_ERR on
+ * error
+ * @lrng_drng_dealloc: Deallocate DRNG
+ * @lrng_drng_seed_helper: Seed the DRNG with data of arbitrary length
+ * drng: is pointer to data structure allocated
+ * with lrng_drng_alloc
+ * return: >= 0 on success, < 0 on error
+ * @lrng_drng_generate_helper: Generate random numbers from the DRNG with
+ * arbitrary length
+ * @lrng_hash_alloc: Allocate the hash for reading the entropy pool
+ * return: allocated data structure (NULL is
+ * success too) or ERR_PTR on error
+ * @lrng_hash_dealloc: Deallocate Hash
+ * @lrng_hash_digestsize: Return the digestsize for the used hash to read
+ * out entropy pool
+ * hash: is pointer to data structure allocated
+ * with lrng_hash_alloc
+ * return: size of digest of hash in bytes
+ * @lrng_hash_buffer: Generate hash
+ * hash: is pointer to data structure allocated
+ * with lrng_hash_alloc
+ * return: 0 on success, < 0 on error
+ */
+struct lrng_crypto_cb {
+ const char *(*lrng_drng_name)(void);
+ const char *(*lrng_hash_name)(void);
+ void *(*lrng_drng_alloc)(u32 sec_strength);
+ void (*lrng_drng_dealloc)(void *drng);
+ int (*lrng_drng_seed_helper)(void *drng, const u8 *inbuf, u32 inbuflen);
+ int (*lrng_drng_generate_helper)(void *drng, u8 *outbuf, u32 outbuflen);
+ void *(*lrng_hash_alloc)(const u8 *key, u32 keylen);
+ void (*lrng_hash_dealloc)(void *hash);
+ u32 (*lrng_hash_digestsize)(void *hash);
+ int (*lrng_hash_buffer)(void *hash, const u8 *inbuf, u32 inbuflen,
+ u8 *digest);
+};
+
+/* Register cryptographic backend */
+#ifdef CONFIG_LRNG_DRNG_SWITCH
+int lrng_set_drng_cb(const struct lrng_crypto_cb *cb);
+#else /* CONFIG_LRNG_DRNG_SWITCH */
+static inline int
+lrng_set_drng_cb(const struct lrng_crypto_cb *cb) { return -EOPNOTSUPP; }
+#endif /* CONFIG_LRNG_DRNG_SWITCH */
+
+#endif /* _LRNG_H */
--
2.24.1
^ permalink raw reply related
* [PATCH v28 02/12] LRNG - allocate one DRNG instance per NUMA node
From: Stephan Müller @ 2020-01-15 10:32 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Greg Kroah-Hartman, linux-crypto-u79uwXL29TY76Z2rM5mHXA, LKML,
linux-api-u79uwXL29TY76Z2rM5mHXA, Eric W. Biederman,
Alexander E. Patrakov, Ahmed S. Darwish, Theodore Y. Ts'o,
Willy Tarreau, Matthew Garrett, Vito Caputo, Andreas Dilger,
Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange, "Peter, Matthias" <matth>
In-Reply-To: <5951792.lmNsirYsPE-jJGQKZiSfeo1haGO/jJMPxvVK+yQ3ZXh@public.gmane.org>
In order to improve NUMA-locality when serving getrandom(2) requests,
allocate one DRNG instance per node.
The DRNG instance that is present right from the start of the kernel is
reused as the first per-NUMA-node DRNG. For all remaining online NUMA
nodes a new DRNG instance is allocated.
During boot time, the multiple DRNG instances are seeded sequentially.
With this, the first DRNG instance (referenced as the initial DRNG
in the code) is completely seeded with 256 bits of entropy before the
next DRNG instance is completely seeded.
When random numbers are requested, the NUMA-node-local DRNG is checked
whether it has been already fully seeded. If this is not the case, the
initial DRNG is used to serve the request.
CC: "Eric W. Biederman" <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
CC: "Alexander E. Patrakov" <patrakov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
CC: "Ahmed S. Darwish" <darwish.07-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
CC: "Theodore Y. Ts'o" <tytso-3s7WtUTddSA@public.gmane.org>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59-1xO5oi07KQx4cg9Nei1l7Q@public.gmane.org>
CC: Vito Caputo <vcaputo-IiWei5kqaphBDgjK7y7TUQ@public.gmane.org>
CC: Andreas Dilger <adilger.kernel-m1MBpc4rdrD3fQ9qLvQP4Q@public.gmane.org>
CC: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
CC: Ray Strode <rstrode-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
CC: William Jon McCann <mccann-4GNroTWusrE@public.gmane.org>
CC: zhangjs <zachary-+jQ8zvd9CV5SzTKKI/wJ2A@public.gmane.org>
CC: Andy Lutomirski <luto-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
CC: Florian Weimer <fweimer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
CC: Lennart Poettering <mzxreary-uLTowLwuiw4b1SvskN2V4Q@public.gmane.org>
CC: Nicolai Stange <nstange-l3A5Bk7waGM@public.gmane.org>
Reviewed-by: Marcelo Henrique Cerri <marcelo.cerri-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
Reviewed-by: Roman Drahtmueller <draht-UTA6Eu5RHcbyVfKhlJ+g5A@public.gmane.org>
Tested-by: Roman Drahtmüller <draht-UTA6Eu5RHcbyVfKhlJ+g5A@public.gmane.org>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
Tested-by: Neil Horman <nhorman-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
---
drivers/char/lrng/Makefile | 2 +
drivers/char/lrng/lrng_internal.h | 5 ++
drivers/char/lrng/lrng_numa.c | 101 ++++++++++++++++++++++++++++++
3 files changed, 108 insertions(+)
create mode 100644 drivers/char/lrng/lrng_numa.c
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 1d2a0211973d..0a32f22c2c1a 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -7,3 +7,5 @@ obj-y += lrng_pool.o lrng_aux.o \
lrng_sw_noise.o lrng_archrandom.o \
lrng_drng.o lrng_chacha20.o \
lrng_interfaces.o \
+
+obj-$(CONFIG_NUMA) += lrng_numa.o
diff --git a/drivers/char/lrng/lrng_internal.h b/drivers/char/lrng/lrng_internal.h
index edf38121ec65..ecb0a7ad5e7e 100644
--- a/drivers/char/lrng/lrng_internal.h
+++ b/drivers/char/lrng/lrng_internal.h
@@ -246,8 +246,13 @@ int lrng_drng_get_sleep(u8 *outbuf, u32 outbuflen);
void lrng_drng_force_reseed(void);
void lrng_drng_seed_work(struct work_struct *dummy);
+#ifdef CONFIG_NUMA
+struct lrng_drng **lrng_drng_instances(void);
+void lrng_drngs_numa_alloc(void);
+#else /* CONFIG_NUMA */
static inline struct lrng_drng **lrng_drng_instances(void) { return NULL; }
static inline void lrng_drngs_numa_alloc(void) { return; }
+#endif /* CONFIG_NUMA */
/************************** Health Test linking code **************************/
diff --git a/drivers/char/lrng/lrng_numa.c b/drivers/char/lrng/lrng_numa.c
new file mode 100644
index 000000000000..947c5b3ed517
--- /dev/null
+++ b/drivers/char/lrng/lrng_numa.c
@@ -0,0 +1,101 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG NUMA support
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/lrng.h>
+#include <linux/slab.h>
+
+#include "lrng_internal.h"
+
+static struct lrng_drng **lrng_drng __read_mostly = NULL;
+
+struct lrng_drng **lrng_drng_instances(void)
+{
+ return lrng_drng;
+}
+
+/* Allocate the data structures for the per-NUMA node DRNGs */
+static void _lrng_drngs_numa_alloc(struct work_struct *work)
+{
+ struct lrng_drng **drngs;
+ struct lrng_drng *lrng_drng_init = lrng_drng_init_instance();
+ u32 node;
+ bool init_drng_used = false;
+
+ mutex_lock(&lrng_crypto_cb_update);
+
+ /* per-NUMA-node DRNGs are already present */
+ if (lrng_drng)
+ goto unlock;
+
+ drngs = kcalloc(nr_node_ids, sizeof(void *), GFP_KERNEL|__GFP_NOFAIL);
+ for_each_online_node(node) {
+ struct lrng_drng *drng;
+
+ if (!init_drng_used) {
+ drngs[node] = lrng_drng_init;
+ init_drng_used = true;
+ continue;
+ }
+
+ drng = kmalloc_node(sizeof(struct lrng_drng),
+ GFP_KERNEL|__GFP_NOFAIL, node);
+ memset(drng, 0, sizeof(lrng_drng));
+
+ drng->crypto_cb = lrng_drng_init->crypto_cb;
+ drng->drng = drng->crypto_cb->lrng_drng_alloc(
+ LRNG_DRNG_SECURITY_STRENGTH_BYTES);
+ if (IS_ERR(drng->drng)) {
+ kfree(drng);
+ goto err;
+ }
+
+ mutex_init(&drng->lock);
+ spin_lock_init(&drng->spin_lock);
+
+ /*
+ * No reseeding of NUMA DRNGs from previous DRNGs as this
+ * would complicate the code. Let it simply reseed.
+ */
+ lrng_drng_reset(drng);
+ drngs[node] = drng;
+
+ lrng_pool_inc_numa_node();
+ pr_info("DRNG for NUMA node %d allocated\n", node);
+ }
+
+ /* Ensure that all NUMA nodes receive changed memory here. */
+ mb();
+
+ if (!cmpxchg(&lrng_drng, NULL, drngs))
+ goto unlock;
+
+err:
+ for_each_online_node(node) {
+ struct lrng_drng *drng = drngs[node];
+
+ if (drng == lrng_drng_init)
+ continue;
+
+ if (drng) {
+ drng->crypto_cb->lrng_drng_dealloc(drng->drng);
+ kfree(drng);
+ }
+ }
+ kfree(drngs);
+
+unlock:
+ mutex_unlock(&lrng_crypto_cb_update);
+}
+
+static DECLARE_WORK(lrng_drngs_numa_alloc_work, _lrng_drngs_numa_alloc);
+
+void lrng_drngs_numa_alloc(void)
+{
+ schedule_work(&lrng_drngs_numa_alloc_work);
+}
--
2.24.1
^ permalink raw reply related
* [PATCH v28 03/12] LRNG - sysctls and /proc interface
From: Stephan Müller @ 2020-01-15 10:32 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Greg Kroah-Hartman, linux-crypto, LKML, linux-api,
Eric W. Biederman, Alexander E. Patrakov, Ahmed S. Darwish,
Theodore Y. Ts'o, Willy Tarreau, Matthew Garrett, Vito Caputo,
Andreas Dilger, Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange, "Peter, Matthias" <matth>
In-Reply-To: <5951792.lmNsirYsPE@positron.chronox.de>
The LRNG sysctl interface provides the same controls as the existing
/dev/random implementation. These sysctls behave identically and are
implemented identically. The goal is to allow a possible merge of the
existing /dev/random implementation with this implementation which
implies that this patch tries have a very close similarity. Yet, all
sysctls are documented at [1].
In addition, it provides the file lrng_type which provides details about
the LRNG:
- the name of the DRNG that produces the random numbers for /dev/random,
/dev/urandom, getrandom(2)
- the hash used to produce random numbers from the entropy pool
- the number of secondary DRNG instances
- indicator whether the LRNG operates SP800-90B compliant
- indicator whether a high-resolution timer is identified - only with a
high-resolution timer the interrupt noise source will deliver sufficient
entropy
- indicator whether the LRNG has been minimally seeded (i.e. is the
secondary DRNG seeded with at least 128 bits of of entropy)
- indicator whether the LRNG has been fully seeded (i.e. is the
secondary DRNG seeded with at least 256 bits of entropy)
[1] https://www.chronox.de/lrng.html
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
drivers/char/lrng/Makefile | 1 +
drivers/char/lrng/lrng_interfaces.c | 1 -
drivers/char/lrng/lrng_internal.h | 4 +
drivers/char/lrng/lrng_proc.c | 163 ++++++++++++++++++++++++++++
4 files changed, 168 insertions(+), 1 deletion(-)
create mode 100644 drivers/char/lrng/lrng_proc.c
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 0a32f22c2c1a..e69c176f0161 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -9,3 +9,4 @@ obj-y += lrng_pool.o lrng_aux.o \
lrng_interfaces.o \
obj-$(CONFIG_NUMA) += lrng_numa.o
+obj-$(CONFIG_SYSCTL) += lrng_proc.o
diff --git a/drivers/char/lrng/lrng_interfaces.c b/drivers/char/lrng/lrng_interfaces.c
index d6ddc72fc9b7..8e96c3ade76e 100644
--- a/drivers/char/lrng/lrng_interfaces.c
+++ b/drivers/char/lrng/lrng_interfaces.c
@@ -34,7 +34,6 @@ static DECLARE_WAIT_QUEUE_HEAD(lrng_write_wait);
static DECLARE_WAIT_QUEUE_HEAD(lrng_init_wait);
static struct fasync_struct *fasync;
-struct ctl_table random_table[];
/********************************** Helper ***********************************/
/* Is the DRNG seed level too low? */
diff --git a/drivers/char/lrng/lrng_internal.h b/drivers/char/lrng/lrng_internal.h
index ecb0a7ad5e7e..f65fc6647e8a 100644
--- a/drivers/char/lrng/lrng_internal.h
+++ b/drivers/char/lrng/lrng_internal.h
@@ -116,7 +116,11 @@ void lrng_cc20_init_state(struct chacha20_state *state);
/********************************** /proc *************************************/
+#ifdef CONFIG_SYSCTL
+void lrng_pool_inc_numa_node(void);
+#else
static inline void lrng_pool_inc_numa_node(void) { }
+#endif
/****************************** LRNG interfaces *******************************/
diff --git a/drivers/char/lrng/lrng_proc.c b/drivers/char/lrng/lrng_proc.c
new file mode 100644
index 000000000000..f4ee01f61925
--- /dev/null
+++ b/drivers/char/lrng/lrng_proc.c
@@ -0,0 +1,163 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG proc and sysctl interfaces
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#include <linux/lrng.h>
+#include <linux/proc_fs.h>
+#include <linux/seq_file.h>
+#include <linux/sysctl.h>
+#include <linux/uuid.h>
+
+#include "lrng_internal.h"
+
+/*
+ * This function is used to return both the bootid UUID, and random
+ * UUID. The difference is in whether table->data is NULL; if it is,
+ * then a new UUID is generated and returned to the user.
+ *
+ * If the user accesses this via the proc interface, the UUID will be
+ * returned as an ASCII string in the standard UUID format; if via the
+ * sysctl system call, as 16 bytes of binary data.
+ */
+static int lrng_proc_do_uuid(struct ctl_table *table, int write,
+ void __user *buffer, size_t *lenp, loff_t *ppos)
+{
+ struct ctl_table fake_table;
+ unsigned char buf[64], tmp_uuid[16], *uuid;
+
+ uuid = table->data;
+ if (!uuid) {
+ uuid = tmp_uuid;
+ generate_random_uuid(uuid);
+ } else {
+ static DEFINE_SPINLOCK(bootid_spinlock);
+
+ spin_lock(&bootid_spinlock);
+ if (!uuid[8])
+ generate_random_uuid(uuid);
+ spin_unlock(&bootid_spinlock);
+ }
+
+ sprintf(buf, "%pU", uuid);
+
+ fake_table.data = buf;
+ fake_table.maxlen = sizeof(buf);
+
+ return proc_dostring(&fake_table, write, buffer, lenp, ppos);
+}
+
+static int lrng_proc_do_entropy(struct ctl_table *table, int write,
+ void __user *buffer, size_t *lenp, loff_t *ppos)
+{
+ struct ctl_table fake_table;
+ int entropy_count;
+
+ entropy_count = lrng_avail_entropy();
+
+ fake_table.data = &entropy_count;
+ fake_table.maxlen = sizeof(entropy_count);
+
+ return proc_dointvec(&fake_table, write, buffer, lenp, ppos);
+}
+
+static int lrng_sysctl_poolsize = LRNG_POOL_SIZE_BITS;
+static int lrng_min_write_thresh;
+static int lrng_max_write_thresh = LRNG_POOL_SIZE_BITS;
+static char lrng_sysctl_bootid[16];
+static int lrng_drng_reseed_max_min;
+
+struct ctl_table random_table[] = {
+ {
+ .procname = "poolsize",
+ .data = &lrng_sysctl_poolsize,
+ .maxlen = sizeof(int),
+ .mode = 0444,
+ .proc_handler = proc_dointvec,
+ },
+ {
+ .procname = "entropy_avail",
+ .maxlen = sizeof(int),
+ .mode = 0444,
+ .proc_handler = lrng_proc_do_entropy,
+ },
+ {
+ .procname = "write_wakeup_threshold",
+ .data = &lrng_write_wakeup_bits,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &lrng_min_write_thresh,
+ .extra2 = &lrng_max_write_thresh,
+ },
+ {
+ .procname = "boot_id",
+ .data = &lrng_sysctl_bootid,
+ .maxlen = 16,
+ .mode = 0444,
+ .proc_handler = lrng_proc_do_uuid,
+ },
+ {
+ .procname = "uuid",
+ .maxlen = 16,
+ .mode = 0444,
+ .proc_handler = lrng_proc_do_uuid,
+ },
+ {
+ .procname = "urandom_min_reseed_secs",
+ .data = &lrng_drng_reseed_max_time,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec,
+ .extra1 = &lrng_drng_reseed_max_min,
+ },
+ { }
+};
+
+/* Number of online DRNGs */
+static u32 numa_drngs = 1;
+
+void lrng_pool_inc_numa_node(void)
+{
+ numa_drngs++;
+}
+
+static int lrng_proc_type_show(struct seq_file *m, void *v)
+{
+ struct lrng_drng *lrng_drng_init = lrng_drng_init_instance();
+ unsigned long flags = 0;
+ unsigned char buf[300];
+
+ lrng_drng_lock(lrng_drng_init, &flags);
+ snprintf(buf, sizeof(buf),
+ "DRNG name: %s\n"
+ "Hash for reading entropy pool: %s\n"
+ "DRNG security strength: %d bits\n"
+ "number of DRNG instances: %u\n"
+ "SP800-90B compliance: %s\n"
+ "High-resolution timer: %s\n"
+ "LRNG minimally seeded: %s\n"
+ "LRNG fully seeded: %s\n",
+ lrng_drng_init->crypto_cb->lrng_drng_name(),
+ lrng_drng_init->crypto_cb->lrng_hash_name(),
+ LRNG_DRNG_SECURITY_STRENGTH_BITS, numa_drngs,
+ lrng_sp80090b_compliant() ? "true" : "false",
+ lrng_pool_highres_timer() ? "true" : "false",
+ lrng_state_min_seeded() ? "true" : "false",
+ lrng_state_fully_seeded() ? "true" : "false");
+ lrng_drng_unlock(lrng_drng_init, &flags);
+
+ seq_write(m, buf, strlen(buf));
+
+ return 0;
+}
+
+static int __init lrng_proc_type_init(void)
+{
+ proc_create_single("lrng_type", 0444, NULL, &lrng_proc_type_show);
+ return 0;
+}
+
+module_init(lrng_proc_type_init);
--
2.24.1
^ permalink raw reply related
* [PATCH v28 04/12] LRNG - add switchable DRNG support
From: Stephan Müller @ 2020-01-15 10:32 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Greg Kroah-Hartman, linux-crypto-u79uwXL29TY76Z2rM5mHXA, LKML,
linux-api-u79uwXL29TY76Z2rM5mHXA, Eric W. Biederman,
Alexander E. Patrakov, Ahmed S. Darwish, Theodore Y. Ts'o,
Willy Tarreau, Matthew Garrett, Vito Caputo, Andreas Dilger,
Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange, "Peter, Matthias" <matth>
In-Reply-To: <5951792.lmNsirYsPE-jJGQKZiSfeo1haGO/jJMPxvVK+yQ3ZXh@public.gmane.org>
The DRNG switch support allows replacing the DRNG mechanism of the
LRNG. The switching support rests on the interface definition of
include/linux/lrng.h. A new DRNG is implemented by filling in the
interface defined in this header file.
In addition to the DRNG, the extension also has to provide a hash
implementation that is used to hash the entropy pool for random number
extraction.
Note: It is permissible to implement a DRNG whose operations may sleep.
However, the hash function must not sleep.
The switchable DRNG support allows replacing the DRNG at runtime.
However, only one DRNG extension is allowed to be loaded at any given
time. Before replacing it with another DRNG implementation, the possibly
existing DRNG extension must be unloaded.
The switchable DRNG extension activates the new DRNG during load time.
It is expected, however, that such a DRNG switch would be done only once
by an administrator to load the intended DRNG implementation.
It is permissible to compile DRNG extensions either as kernel modules or
statically. The initialization of the DRNG extension should be performed
with a late_initcall to ensure the extension is available when user
space starts but after all other initialization completed.
The initialization is performed by registering the function call data
structure with the lrng_set_drng_cb function. In order to unload the
DRNG extension, lrng_set_drng_cb must be invoked with the NULL
parameter.
The DRNG extension should always provide a security strength that is at
least as strong as LRNG_DRNG_SECURITY_STRENGTH_BITS.
CC: "Eric W. Biederman" <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
CC: "Alexander E. Patrakov" <patrakov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
CC: "Ahmed S. Darwish" <darwish.07-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
CC: "Theodore Y. Ts'o" <tytso-3s7WtUTddSA@public.gmane.org>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59-1xO5oi07KQx4cg9Nei1l7Q@public.gmane.org>
CC: Vito Caputo <vcaputo-IiWei5kqaphBDgjK7y7TUQ@public.gmane.org>
CC: Andreas Dilger <adilger.kernel-m1MBpc4rdrD3fQ9qLvQP4Q@public.gmane.org>
CC: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
CC: Ray Strode <rstrode-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
CC: William Jon McCann <mccann-4GNroTWusrE@public.gmane.org>
CC: zhangjs <zachary-+jQ8zvd9CV5SzTKKI/wJ2A@public.gmane.org>
CC: Andy Lutomirski <luto-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
CC: Florian Weimer <fweimer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
CC: Lennart Poettering <mzxreary-uLTowLwuiw4b1SvskN2V4Q@public.gmane.org>
CC: Nicolai Stange <nstange-l3A5Bk7waGM@public.gmane.org>
Reviewed-by: Marcelo Henrique Cerri <marcelo.cerri-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
Reviewed-by: Roman Drahtmueller <draht-UTA6Eu5RHcbyVfKhlJ+g5A@public.gmane.org>
Tested-by: Roman Drahtmüller <draht-UTA6Eu5RHcbyVfKhlJ+g5A@public.gmane.org>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
Tested-by: Neil Horman <nhorman-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
---
drivers/char/lrng/Kconfig | 7 ++
drivers/char/lrng/Makefile | 1 +
drivers/char/lrng/lrng_switch.c | 183 ++++++++++++++++++++++++++++++++
3 files changed, 191 insertions(+)
create mode 100644 drivers/char/lrng/lrng_switch.c
diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index 56f13efd3592..cb701bb0b8b6 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -64,4 +64,11 @@ config LRNG_POOL_SIZE
default 7 if LRNG_POOL_SIZE_65536
default 8 if LRNG_POOL_SIZE_131072
+menuconfig LRNG_DRNG_SWITCH
+ bool "Support DRNG runtime switching"
+ help
+ The Linux RNG per default uses a ChaCha20 DRNG that is
+ accessible via the external interfaces. With this configuration
+ option other DRNGs can be selected and loaded at runtime.
+
endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index e69c176f0161..31cfe87c999e 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -10,3 +10,4 @@ obj-y += lrng_pool.o lrng_aux.o \
obj-$(CONFIG_NUMA) += lrng_numa.o
obj-$(CONFIG_SYSCTL) += lrng_proc.o
+obj-$(CONFIG_LRNG_DRNG_SWITCH) += lrng_switch.o
diff --git a/drivers/char/lrng/lrng_switch.c b/drivers/char/lrng/lrng_switch.c
new file mode 100644
index 000000000000..48c9a364aecf
--- /dev/null
+++ b/drivers/char/lrng/lrng_switch.c
@@ -0,0 +1,183 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG DRNG switching support
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/lrng.h>
+
+#include "lrng_internal.h"
+
+static int lrng_drng_switch(struct lrng_drng *drng_store,
+ const struct lrng_crypto_cb *cb, int node)
+{
+ const struct lrng_crypto_cb *old_cb;
+ unsigned long flags = 0;
+ int ret;
+ u8 seed[LRNG_DRNG_SECURITY_STRENGTH_BYTES];
+ void *new_drng = cb->lrng_drng_alloc(LRNG_DRNG_SECURITY_STRENGTH_BYTES);
+ void *old_drng, *new_hash, *old_hash;
+ bool sl = false, reset_drng = !lrng_get_available();
+
+ if (IS_ERR(new_drng)) {
+ pr_warn("could not allocate new DRNG for NUMA node %d (%ld)\n",
+ node, PTR_ERR(new_drng));
+ return PTR_ERR(new_drng);
+ }
+
+ new_hash = cb->lrng_hash_alloc(seed, sizeof(seed));
+ if (IS_ERR(new_hash)) {
+ pr_warn("could not allocate new LRNG pool hash (%ld)\n",
+ PTR_ERR(new_hash));
+ cb->lrng_drng_dealloc(new_drng);
+ return PTR_ERR(new_hash);
+ }
+
+ lrng_drng_lock(drng_store, &flags);
+
+ /*
+ * Pull from existing DRNG to seed new DRNG regardless of seed status
+ * of old DRNG -- the entropy state for the DRNG is left unchanged which
+ * implies that als the new DRNG is reseeded when deemed necessary. This
+ * seeding of the new DRNG shall only ensure that the new DRNG has the
+ * same entropy as the old DRNG.
+ */
+ ret = drng_store->crypto_cb->lrng_drng_generate_helper(
+ drng_store->drng, seed, sizeof(seed));
+ lrng_drng_unlock(drng_store, &flags);
+
+ if (ret < 0) {
+ reset_drng = true;
+ pr_warn("getting random data from DRNG failed for NUMA node %d "
+ "(%d)\n", node, ret);
+ } else {
+ /* seed new DRNG with data */
+ ret = cb->lrng_drng_seed_helper(new_drng, seed, ret);
+ if (ret < 0) {
+ reset_drng = true;
+ pr_warn("seeding of new DRNG failed for NUMA node %d "
+ "(%d)\n", node, ret);
+ } else {
+ pr_debug("seeded new DRNG of NUMA node %d instance "
+ "from old DRNG instance\n", node);
+ }
+ }
+
+ mutex_lock(&drng_store->lock);
+ /*
+ * If we switch the DRNG from the initial ChaCha20 DRNG to something
+ * else, there is a lock transition from spin lock to mutex (see
+ * lrng_drng_is_atomic and how the lock is taken in lrng_drng_lock).
+ * Thus, we need to take both locks during the transition phase.
+ */
+ if (lrng_drng_is_atomic(drng_store)) {
+ spin_lock_irqsave(&drng_store->spin_lock, flags);
+ sl = true;
+ }
+
+ if (reset_drng)
+ lrng_drng_reset(drng_store);
+
+ old_drng = drng_store->drng;
+ old_cb = drng_store->crypto_cb;
+ drng_store->drng = new_drng;
+ drng_store->crypto_cb = cb;
+
+ old_hash = drng_store->hash;
+ drng_store->hash = new_hash;
+ pr_info("Entropy pool read-hash allocated for DRNG for NUMA node %d\n",
+ node);
+
+ if (sl)
+ spin_unlock_irqrestore(&drng_store->spin_lock, flags);
+ mutex_unlock(&drng_store->lock);
+
+ /* ChaCha20 serves as atomic instance left untouched. */
+ if (old_drng != &chacha20) {
+ old_cb->lrng_drng_dealloc(old_drng);
+ old_cb->lrng_hash_dealloc(old_hash);
+ }
+
+ pr_info("DRNG of NUMA node %d switched\n", node);
+
+ return 0;
+}
+
+/*
+ * Switch the existing DRNG instances with new using the new crypto callbacks.
+ * The caller must hold the lrng_crypto_cb_update lock.
+ */
+static int lrng_drngs_switch(const struct lrng_crypto_cb *cb)
+{
+ struct lrng_drng **lrng_drng = lrng_drng_instances();
+ struct lrng_drng *lrng_drng_init = lrng_drng_init_instance();
+ int ret = 0;
+
+ /* Update DRNG */
+ if (lrng_drng) {
+ u32 node;
+
+ for_each_online_node(node) {
+ if (lrng_drng[node])
+ ret = lrng_drng_switch(lrng_drng[node], cb,
+ node);
+ }
+ } else {
+ ret = lrng_drng_switch(lrng_drng_init, cb, 0);
+ }
+
+ if (!ret)
+ lrng_set_available();
+
+ return 0;
+}
+
+/**
+ * lrng_set_drng_cb - Register new cryptographic callback functions for DRNG
+ * The registering implies that all old DRNG states are replaced with new
+ * DRNG states.
+ *
+ * @cb: Callback functions to be registered -- if NULL, use the default
+ * callbacks pointing to the ChaCha20 DRNG.
+ *
+ * Return:
+ * * 0 on success
+ * * < 0 on error
+ */
+int lrng_set_drng_cb(const struct lrng_crypto_cb *cb)
+{
+ struct lrng_drng *lrng_drng_init = lrng_drng_init_instance();
+ int ret;
+
+ if (!cb)
+ cb = &lrng_cc20_crypto_cb;
+
+ mutex_lock(&lrng_crypto_cb_update);
+
+ /*
+ * If a callback other than the default is set, allow it only to be
+ * set back to the default callback. This ensures that multiple
+ * different callbacks can be registered at the same time. If a
+ * callback different from the current callback and the default
+ * callback shall be set, the current callback must be deregistered
+ * (e.g. the kernel module providing it must be unloaded) and the new
+ * implementation can be registered.
+ */
+ if ((cb != &lrng_cc20_crypto_cb) &&
+ (lrng_drng_init->crypto_cb != &lrng_cc20_crypto_cb)) {
+ pr_warn("disallow setting new cipher callbacks, unload the old "
+ "callbacks first!\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ret = lrng_drngs_switch(cb);
+
+out:
+ mutex_unlock(&lrng_crypto_cb_update);
+ return ret;
+}
+EXPORT_SYMBOL(lrng_set_drng_cb);
--
2.24.1
^ permalink raw reply related
* [PATCH v28 05/12] crypto: DRBG - externalize DRBG functions for LRNG
From: Stephan Müller @ 2020-01-15 10:33 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Greg Kroah-Hartman, linux-crypto, LKML, linux-api,
Eric W. Biederman, Alexander E. Patrakov, Ahmed S. Darwish,
Theodore Y. Ts'o, Willy Tarreau, Matthew Garrett, Vito Caputo,
Andreas Dilger, Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange, "Peter, Matthias" <matth>
In-Reply-To: <5951792.lmNsirYsPE@positron.chronox.de>
This patch allows several DRBG functions to be called by the LRNG kernel
code paths outside the drbg.c file.
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
crypto/drbg.c | 16 ++++++++++------
include/crypto/drbg.h | 7 +++++++
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/crypto/drbg.c b/crypto/drbg.c
index b6929eb5f565..5561dd0f09e1 100644
--- a/crypto/drbg.c
+++ b/crypto/drbg.c
@@ -113,7 +113,7 @@
* the SHA256 / AES 256 over other ciphers. Thus, the favored
* DRBGs are the latest entries in this array.
*/
-static const struct drbg_core drbg_cores[] = {
+const struct drbg_core drbg_cores[] = {
#ifdef CONFIG_CRYPTO_DRBG_CTR
{
.flags = DRBG_CTR | DRBG_STRENGTH128,
@@ -190,6 +190,7 @@ static const struct drbg_core drbg_cores[] = {
},
#endif /* CONFIG_CRYPTO_DRBG_HMAC */
};
+EXPORT_SYMBOL(drbg_cores);
static int drbg_uninstantiate(struct drbg_state *drbg);
@@ -205,7 +206,7 @@ static int drbg_uninstantiate(struct drbg_state *drbg);
* Return: normalized strength in *bytes* value or 32 as default
* to counter programming errors
*/
-static inline unsigned short drbg_sec_strength(drbg_flag_t flags)
+unsigned short drbg_sec_strength(drbg_flag_t flags)
{
switch (flags & DRBG_STRENGTH_MASK) {
case DRBG_STRENGTH128:
@@ -218,6 +219,7 @@ static inline unsigned short drbg_sec_strength(drbg_flag_t flags)
return 32;
}
}
+EXPORT_SYMBOL(drbg_sec_strength);
/*
* FIPS 140-2 continuous self test for the noise source
@@ -1202,7 +1204,7 @@ static int drbg_seed(struct drbg_state *drbg, struct drbg_string *pers,
}
/* Free all substructures in a DRBG state without the DRBG state structure */
-static inline void drbg_dealloc_state(struct drbg_state *drbg)
+void drbg_dealloc_state(struct drbg_state *drbg)
{
if (!drbg)
return;
@@ -1223,12 +1225,13 @@ static inline void drbg_dealloc_state(struct drbg_state *drbg)
drbg->fips_primed = false;
}
}
+EXPORT_SYMBOL(drbg_dealloc_state);
/*
* Allocate all sub-structures for a DRBG state.
* The DRBG state structure must already be allocated.
*/
-static inline int drbg_alloc_state(struct drbg_state *drbg)
+int drbg_alloc_state(struct drbg_state *drbg)
{
int ret = -ENOMEM;
unsigned int sb_size = 0;
@@ -1307,6 +1310,7 @@ static inline int drbg_alloc_state(struct drbg_state *drbg)
drbg_dealloc_state(drbg);
return ret;
}
+EXPORT_SYMBOL(drbg_alloc_state);
/*************************************************************************
* DRBG interface functions
@@ -1874,8 +1878,7 @@ static int drbg_kcapi_sym_ctr(struct drbg_state *drbg,
*
* return: flags
*/
-static inline void drbg_convert_tfm_core(const char *cra_driver_name,
- int *coreref, bool *pr)
+void drbg_convert_tfm_core(const char *cra_driver_name, int *coreref, bool *pr)
{
int i = 0;
size_t start = 0;
@@ -1902,6 +1905,7 @@ static inline void drbg_convert_tfm_core(const char *cra_driver_name,
}
}
}
+EXPORT_SYMBOL(drbg_convert_tfm_core);
static int drbg_kcapi_init(struct crypto_tfm *tfm)
{
diff --git a/include/crypto/drbg.h b/include/crypto/drbg.h
index 8c9af21efce1..c63b20375235 100644
--- a/include/crypto/drbg.h
+++ b/include/crypto/drbg.h
@@ -282,4 +282,11 @@ enum drbg_prefixes {
DRBG_PREFIX3
};
+extern int drbg_alloc_state(struct drbg_state *drbg);
+extern void drbg_dealloc_state(struct drbg_state *drbg);
+extern void drbg_convert_tfm_core(const char *cra_driver_name, int *coreref,
+ bool *pr);
+extern const struct drbg_core drbg_cores[];
+extern unsigned short drbg_sec_strength(drbg_flag_t flags);
+
#endif /* _DRBG_H */
--
2.24.1
^ permalink raw reply related
* [PATCH v28 06/12] LRNG - add SP800-90A DRBG extension
From: Stephan Müller @ 2020-01-15 10:33 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Greg Kroah-Hartman, linux-crypto, LKML, linux-api,
Eric W. Biederman, Alexander E. Patrakov, Ahmed S. Darwish,
Theodore Y. Ts'o, Willy Tarreau, Matthew Garrett, Vito Caputo,
Andreas Dilger, Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange, "Peter, Matthias" <matth>
In-Reply-To: <5951792.lmNsirYsPE@positron.chronox.de>
Using the LRNG switchable DRNG support, the SP800-90A DRBG extension is
implemented.
The DRBG uses the kernel crypto API DRBG implementation. In addition, it
uses the kernel crypto API SHASH support to provide the hashing
operation.
The DRBG supports the choice of either a CTR DRBG using AES-256, HMAC
DRBG with SHA-512 core or Hash DRBG with SHA-512 core. The used core can
be selected with the module parameter lrng_drbg_type. The default is the
CTR DRBG.
When compiling the DRBG extension statically, the DRBG is loaded at
late_initcall stage which implies that with the start of user space, the
user space interfaces of getrandom(2), /dev/random and /dev/urandom
provide random data produced by an SP800-90A DRBG.
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
drivers/char/lrng/Kconfig | 11 ++
drivers/char/lrng/Makefile | 1 +
drivers/char/lrng/lrng_drbg.c | 260 ++++++++++++++++++++++++++++++++++
3 files changed, 272 insertions(+)
create mode 100644 drivers/char/lrng/lrng_drbg.c
diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index cb701bb0b8b6..15fb623d9d1f 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -71,4 +71,15 @@ menuconfig LRNG_DRNG_SWITCH
accessible via the external interfaces. With this configuration
option other DRNGs can be selected and loaded at runtime.
+if LRNG_DRNG_SWITCH
+config LRNG_DRBG
+ tristate "SP800-90A support for the LRNG"
+ select CRYPTO_DRBG_MENU
+ select CRYPTO_CMAC if CRYPTO_DRBG_CTR
+ help
+ Enable the SP800-90A DRBG support for the LRNG. Once the
+ module is loaded, output from /dev/random, /dev/urandom,
+ getrandom(2), or get_random_bytes is provided by a DRBG.
+endif # LRNG_DRNG_SWITCH
+
endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 31cfe87c999e..0d320fcb7b9e 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -11,3 +11,4 @@ obj-y += lrng_pool.o lrng_aux.o \
obj-$(CONFIG_NUMA) += lrng_numa.o
obj-$(CONFIG_SYSCTL) += lrng_proc.o
obj-$(CONFIG_LRNG_DRNG_SWITCH) += lrng_switch.o
+obj-$(CONFIG_LRNG_DRBG) += lrng_drbg.o
diff --git a/drivers/char/lrng/lrng_drbg.c b/drivers/char/lrng/lrng_drbg.c
new file mode 100644
index 000000000000..8bf2badb1fe0
--- /dev/null
+++ b/drivers/char/lrng/lrng_drbg.c
@@ -0,0 +1,260 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Backend for the LRNG providing the cryptographic primitives using the
+ * kernel crypto API and its DRBG.
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <crypto/drbg.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/lrng.h>
+
+/*
+ * Define a DRBG plus a hash / MAC used to extract data from the entropy pool.
+ * For LRNG_HASH_NAME you can use a hash or a MAC (HMAC or CMAC) of your choice
+ * (Note, you should use the suggested selections below -- using SHA-1 or MD5
+ * is not wise). The idea is that the used cipher primitive can be selected to
+ * be the same as used for the DRBG. I.e. the LRNG only uses one cipher
+ * primitive using the same cipher implementation with the options offered in
+ * the following. This means, if the CTR DRBG is selected and AES-NI is present,
+ * both the CTR DRBG and the selected cmac(aes) use AES-NI.
+ *
+ * The security strengths of the DRBGs are all 256 bits according to
+ * SP800-57 section 5.6.1.
+ *
+ * This definition is allowed to be changed.
+ */
+#ifdef CONFIG_CRYPTO_DRBG_CTR
+static unsigned int lrng_drbg_type = 0;
+#elif defined CONFIG_CRYPTO_DRBG_HMAC
+static unsigned int lrng_drbg_type = 1;
+#elif defined CONFIG_CRYPTO_DRBG_HASH
+static unsigned int lrng_drbg_type = 2;
+#else
+#error "Unknown DRBG in use"
+#endif
+
+/* The parameter must be r/o in sysfs as otherwise races appear. */
+module_param(lrng_drbg_type, uint, 0444);
+MODULE_PARM_DESC(lrng_drbg_type, "DRBG type used for LRNG (0->CTR_DRBG, "
+ "1->HMAC_DRBG, 2->Hash_DRBG)");
+
+struct lrng_drbg {
+ const char *hash_name;
+ const char *drbg_core;
+};
+
+static const struct lrng_drbg lrng_drbg_types[] = {
+ { /* CTR_DRBG with AES-256 using derivation function */
+ .hash_name = "cmac(aes)",
+ .drbg_core = "drbg_nopr_ctr_aes256",
+ }, { /* HMAC_DRBG with SHA-512 */
+ .hash_name = "sha512",
+ .drbg_core = "drbg_nopr_hmac_sha512",
+ }, { /* Hash_DRBG with SHA-512 using derivation function */
+ .hash_name = "sha512",
+ .drbg_core = "drbg_nopr_sha512"
+ }
+};
+
+struct lrng_hash_info {
+ struct shash_desc shash;
+ char ctx[];
+};
+
+static int lrng_drbg_drng_seed_helper(void *drng, const u8 *inbuf, u32 inbuflen)
+{
+ struct drbg_state *drbg = (struct drbg_state *)drng;
+ LIST_HEAD(seedlist);
+ struct drbg_string data;
+ int ret;
+
+ drbg_string_fill(&data, inbuf, inbuflen);
+ list_add_tail(&data.list, &seedlist);
+ ret = drbg->d_ops->update(drbg, &seedlist, drbg->seeded);
+
+ if (ret >= 0)
+ drbg->seeded = true;
+
+ return ret;
+}
+
+static int lrng_drbg_drng_generate_helper(void *drng, u8 *outbuf, u32 outbuflen)
+{
+ struct drbg_state *drbg = (struct drbg_state *)drng;
+
+ return drbg->d_ops->generate(drbg, outbuf, outbuflen, NULL);
+}
+
+static void *lrng_drbg_drng_alloc(u32 sec_strength)
+{
+ struct drbg_state *drbg;
+ int coreref = -1;
+ bool pr = false;
+ int ret;
+
+ drbg_convert_tfm_core(lrng_drbg_types[lrng_drbg_type].drbg_core,
+ &coreref, &pr);
+ if (coreref < 0)
+ return ERR_PTR(-EFAULT);
+
+ drbg = kzalloc(sizeof(struct drbg_state), GFP_KERNEL);
+ if (!drbg)
+ return ERR_PTR(-ENOMEM);
+
+ drbg->core = &drbg_cores[coreref];
+ drbg->seeded = false;
+ ret = drbg_alloc_state(drbg);
+ if (ret)
+ goto err;
+
+ if (sec_strength > drbg_sec_strength(drbg->core->flags)) {
+ pr_err("Security strength of DRBG (%u bits) lower than "
+ "requested by LRNG (%u bits)\n",
+ drbg_sec_strength(drbg->core->flags) * 8,
+ sec_strength * 8);
+ goto dealloc;
+ }
+
+ if (sec_strength < drbg_sec_strength(drbg->core->flags))
+ pr_warn("Security strength of DRBG (%u bits) higher than "
+ "requested by LRNG (%u bits)\n",
+ drbg_sec_strength(drbg->core->flags) * 8,
+ sec_strength * 8);
+
+ pr_info("DRBG with %s core allocated\n", drbg->core->backend_cra_name);
+
+ return drbg;
+
+dealloc:
+ if (drbg->d_ops)
+ drbg->d_ops->crypto_fini(drbg);
+ drbg_dealloc_state(drbg);
+err:
+ kfree(drbg);
+ return ERR_PTR(-EINVAL);
+}
+
+static void lrng_drbg_drng_dealloc(void *drng)
+{
+ struct drbg_state *drbg = (struct drbg_state *)drng;
+
+ drbg_dealloc_state(drbg);
+ kzfree(drbg);
+ pr_info("DRBG deallocated\n");
+}
+
+static void *lrng_drbg_hash_alloc(const u8 *key, u32 keylen)
+{
+ struct lrng_hash_info *lrng_hash;
+ struct crypto_shash *tfm;
+ int size, ret;
+
+ tfm = crypto_alloc_shash(lrng_drbg_types[lrng_drbg_type].hash_name,
+ 0, 0);
+ if (IS_ERR(tfm)) {
+ pr_err("could not allocate hash %s\n",
+ lrng_drbg_types[lrng_drbg_type].hash_name);
+ return ERR_CAST(tfm);
+ }
+
+ size = sizeof(struct lrng_hash_info) + crypto_shash_descsize(tfm);
+ lrng_hash = kmalloc(size, GFP_KERNEL);
+ if (!lrng_hash) {
+ crypto_free_shash(tfm);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ lrng_hash->shash.tfm = tfm;
+
+ /* If the used hash is no MAC, ignore the ENOSYS return code */
+ ret = crypto_shash_setkey(tfm, key, keylen);
+ if (ret && ret != -ENOSYS) {
+ pr_err("could not set the key for MAC\n");
+ crypto_free_shash(tfm);
+ kfree(lrng_hash);
+ return ERR_PTR(ret);
+ }
+
+ pr_info("Hash %s allocated\n",
+ lrng_drbg_types[lrng_drbg_type].hash_name);
+
+ return lrng_hash;
+}
+
+static void lrng_drbg_hash_dealloc(void *hash)
+{
+ struct lrng_hash_info *lrng_hash = (struct lrng_hash_info *)hash;
+ struct shash_desc *shash = &lrng_hash->shash;
+ struct crypto_shash *tfm = shash->tfm;
+
+ crypto_free_shash(tfm);
+ kfree(lrng_hash);
+ pr_info("Hash deallocated\n");
+}
+
+static u32 lrng_drbg_hash_digestsize(void *hash)
+{
+ struct lrng_hash_info *lrng_hash = (struct lrng_hash_info *)hash;
+ struct shash_desc *shash = &lrng_hash->shash;
+
+ return crypto_shash_digestsize(shash->tfm);
+}
+
+static int lrng_drbg_hash_buffer(void *hash, const u8 *inbuf, u32 inbuflen,
+ u8 *digest)
+{
+ struct lrng_hash_info *lrng_hash = (struct lrng_hash_info *)hash;
+ struct shash_desc *shash = &lrng_hash->shash;
+
+ return crypto_shash_digest(shash, inbuf, inbuflen, digest);
+}
+
+static const char *lrng_drbg_name(void)
+{
+ return lrng_drbg_types[lrng_drbg_type].drbg_core;
+}
+
+static const char *lrng_hash_name(void)
+{
+ return lrng_drbg_types[lrng_drbg_type].hash_name;
+}
+
+const static struct lrng_crypto_cb lrng_drbg_crypto_cb = {
+ .lrng_drng_name = lrng_drbg_name,
+ .lrng_hash_name = lrng_hash_name,
+ .lrng_drng_alloc = lrng_drbg_drng_alloc,
+ .lrng_drng_dealloc = lrng_drbg_drng_dealloc,
+ .lrng_drng_seed_helper = lrng_drbg_drng_seed_helper,
+ .lrng_drng_generate_helper = lrng_drbg_drng_generate_helper,
+ .lrng_hash_alloc = lrng_drbg_hash_alloc,
+ .lrng_hash_dealloc = lrng_drbg_hash_dealloc,
+ .lrng_hash_digestsize = lrng_drbg_hash_digestsize,
+ .lrng_hash_buffer = lrng_drbg_hash_buffer,
+};
+
+static int __init lrng_drbg_init(void)
+{
+ if (lrng_drbg_type >= ARRAY_SIZE(lrng_drbg_types)) {
+ pr_err("lrng_drbg_type parameter too large (given %u - max: %lu)",
+ lrng_drbg_type,
+ (unsigned long)ARRAY_SIZE(lrng_drbg_types) - 1);
+ return -EAGAIN;
+ }
+ return lrng_set_drng_cb(&lrng_drbg_crypto_cb);
+}
+
+static void __exit lrng_drbg_exit(void)
+{
+ lrng_set_drng_cb(NULL);
+}
+
+late_initcall(lrng_drbg_init);
+module_exit(lrng_drbg_exit);
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_AUTHOR("Stephan Mueller <smueller@chronox.de>");
+MODULE_DESCRIPTION("Linux Random Number Generator - SP800-90A DRBG backend");
--
2.24.1
^ permalink raw reply related
* [PATCH v28 07/12] LRNG - add kernel crypto API PRNG extension
From: Stephan Müller @ 2020-01-15 10:34 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Greg Kroah-Hartman, linux-crypto, LKML, linux-api,
Eric W. Biederman, Alexander E. Patrakov, Ahmed S. Darwish,
Theodore Y. Ts'o, Willy Tarreau, Matthew Garrett, Vito Caputo,
Andreas Dilger, Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange, "Peter, Matthias" <matth>
In-Reply-To: <5951792.lmNsirYsPE@positron.chronox.de>
Add runtime-pluggable support for all PRNGs that are accessible via
the kernel crypto API, including hardware PRNGs. The PRNG is selected
with the module parameter drng_name where the name must be one that the
kernel crypto API can resolve into an RNG.
This allows using of the kernel crypto API PRNG implementations that
provide an interface to hardware PRNGs. Using this extension,
the LRNG uses the hardware PRNGs to generate random numbers. An
example is the S390 CPACF support providing such a PRNG.
The hash is provided by a kernel crypto API SHASH whose digest size
complies with the seedsize of the PRNG.
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
drivers/char/lrng/Kconfig | 10 +
drivers/char/lrng/Makefile | 1 +
drivers/char/lrng/lrng_kcapi.c | 327 +++++++++++++++++++++++++++++++++
3 files changed, 338 insertions(+)
create mode 100644 drivers/char/lrng/lrng_kcapi.c
diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index 15fb623d9d1f..0d070a3897dd 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -80,6 +80,16 @@ config LRNG_DRBG
Enable the SP800-90A DRBG support for the LRNG. Once the
module is loaded, output from /dev/random, /dev/urandom,
getrandom(2), or get_random_bytes is provided by a DRBG.
+
+config LRNG_KCAPI
+ tristate "Kernel Crypto API support for the LRNG"
+ select CRYPTO_RNG
+ help
+ Enable the support for generic pseudo-random number
+ generators offered by the kernel crypto API with the
+ LRNG. Once the module is loaded, output from /dev/random,
+ /dev/urandom, getrandom(2), or get_random_bytes is
+ provided by the selected kernel crypto API RNG.
endif # LRNG_DRNG_SWITCH
endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 0d320fcb7b9e..94b2dfb2dfdb 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -12,3 +12,4 @@ obj-$(CONFIG_NUMA) += lrng_numa.o
obj-$(CONFIG_SYSCTL) += lrng_proc.o
obj-$(CONFIG_LRNG_DRNG_SWITCH) += lrng_switch.o
obj-$(CONFIG_LRNG_DRBG) += lrng_drbg.o
+obj-$(CONFIG_LRNG_KCAPI) += lrng_kcapi.o
diff --git a/drivers/char/lrng/lrng_kcapi.c b/drivers/char/lrng/lrng_kcapi.c
new file mode 100644
index 000000000000..115af147bd5c
--- /dev/null
+++ b/drivers/char/lrng/lrng_kcapi.c
@@ -0,0 +1,327 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Backend for the LRNG providing the cryptographic primitives using the
+ * kernel crypto API.
+ *
+ * Copyright (C) 2018 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <crypto/hash.h>
+#include <crypto/rng.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/lrng.h>
+
+static char *drng_name = NULL;
+module_param(drng_name, charp, 0444);
+MODULE_PARM_DESC(drng_name, "Kernel crypto API name of DRNG");
+
+static char *pool_hash = "sha512";
+module_param(pool_hash, charp, 0444);
+MODULE_PARM_DESC(pool_hash,
+ "Kernel crypto API name of hash or keyed message digest to read the entropy pool");
+
+static char *seed_hash = NULL;
+module_param(seed_hash, charp, 0444);
+MODULE_PARM_DESC(seed_hash,
+ "Kernel crypto API name of hash with output size equal to seedsize of DRNG to bring seed string to the size required by the DRNG");
+
+struct lrng_hash_info {
+ struct shash_desc shash;
+ char ctx[];
+};
+
+struct lrng_drng_info {
+ struct crypto_rng *kcapi_rng;
+ struct lrng_hash_info *lrng_hash;
+};
+
+static struct lrng_hash_info *_lrng_kcapi_hash_alloc(const char *name)
+{
+ struct lrng_hash_info *lrng_hash;
+ struct crypto_shash *tfm;
+ int size;
+
+ if (!name) {
+ pr_err("Hash name missing\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ tfm = crypto_alloc_shash(name, 0, 0);
+ if (IS_ERR(tfm)) {
+ pr_err("could not allocate hash %s\n", name);
+ return ERR_CAST(tfm);
+ }
+
+ size = sizeof(struct lrng_hash_info) + crypto_shash_descsize(tfm);
+ lrng_hash = kmalloc(size, GFP_KERNEL);
+ if (!lrng_hash) {
+ crypto_free_shash(tfm);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ lrng_hash->shash.tfm = tfm;
+
+ return lrng_hash;
+}
+
+static inline u32 _lrng_kcapi_hash_digestsize(struct lrng_hash_info *lrng_hash)
+{
+ struct shash_desc *shash = &lrng_hash->shash;
+ struct crypto_shash *tfm = shash->tfm;
+
+ return crypto_shash_digestsize(tfm);
+}
+
+static inline void _lrng_kcapi_hash_free(struct lrng_hash_info *lrng_hash)
+{
+ struct shash_desc *shash = &lrng_hash->shash;
+ struct crypto_shash *tfm = shash->tfm;
+
+ crypto_free_shash(tfm);
+ kfree(lrng_hash);
+}
+
+static void *lrng_kcapi_hash_alloc(const u8 *key, u32 keylen)
+{
+ struct lrng_hash_info *lrng_hash;
+ int ret;
+
+ lrng_hash = _lrng_kcapi_hash_alloc(pool_hash);
+ if (IS_ERR(lrng_hash))
+ return ERR_CAST(lrng_hash);
+
+ /* If the used hash is no MAC, ignore the ENOSYS return code */
+ ret = crypto_shash_setkey(lrng_hash->shash.tfm, key, keylen);
+ if (ret && ret != -ENOSYS) {
+ pr_err("could not set the key for MAC\n");
+ _lrng_kcapi_hash_free(lrng_hash);
+ return ERR_PTR(ret);
+ }
+
+ pr_info("Hash %s allocated\n", pool_hash);
+
+ return lrng_hash;
+}
+
+static void lrng_kcapi_hash_dealloc(void *hash)
+{
+ struct lrng_hash_info *lrng_hash = (struct lrng_hash_info *)hash;
+
+ _lrng_kcapi_hash_free(lrng_hash);
+ pr_info("Hash %s deallocated\n", pool_hash);
+}
+
+static u32 lrng_kcapi_hash_digestsize(void *hash)
+{
+ struct lrng_hash_info *lrng_hash = (struct lrng_hash_info *)hash;
+
+ return _lrng_kcapi_hash_digestsize(lrng_hash);
+}
+
+static int lrng_kcapi_hash_buffer(void *hash, const u8 *inbuf, u32 inbuflen,
+ u8 *digest)
+{
+ struct lrng_hash_info *lrng_hash = (struct lrng_hash_info *)hash;
+ struct shash_desc *shash = &lrng_hash->shash;
+
+ return crypto_shash_digest(shash, inbuf, inbuflen, digest);
+}
+
+static int lrng_kcapi_drng_seed_helper(void *drng, const u8 *inbuf,
+ u32 inbuflen)
+{
+ struct lrng_drng_info *lrng_drng_info = (struct lrng_drng_info *)drng;
+ struct crypto_rng *kcapi_rng = lrng_drng_info->kcapi_rng;
+ struct lrng_hash_info *lrng_hash = lrng_drng_info->lrng_hash;
+
+ if (lrng_hash) {
+ struct shash_desc *shash = &lrng_hash->shash;
+ u32 digestsize = _lrng_kcapi_hash_digestsize(lrng_hash);
+ u8 digest[64] __aligned(8);
+ int ret;
+
+ BUG_ON(digestsize > sizeof(digest));
+
+ ret = crypto_shash_digest(shash, inbuf, inbuflen, digest);
+ if (ret)
+ return ret;
+
+ ret = crypto_rng_reset(kcapi_rng, digest, digestsize);
+ if (ret)
+ return ret;
+
+ memzero_explicit(digest, digestsize);
+
+ return 0;
+ } else {
+ return crypto_rng_reset(kcapi_rng, inbuf, inbuflen);
+ }
+}
+
+static int lrng_kcapi_drng_generate_helper(void *drng, u8 *outbuf,
+ u32 outbuflen)
+{
+ struct lrng_drng_info *lrng_drng_info = (struct lrng_drng_info *)drng;
+ struct crypto_rng *kcapi_rng = lrng_drng_info->kcapi_rng;
+ int ret = crypto_rng_get_bytes(kcapi_rng, outbuf, outbuflen);
+
+ if (ret < 0)
+ return ret;
+
+ return outbuflen;
+}
+
+static void *lrng_kcapi_drng_alloc(u32 sec_strength)
+{
+ struct lrng_drng_info *lrng_drng_info;
+ struct crypto_rng *kcapi_rng;
+ int seedsize;
+ void *ret = ERR_PTR(-ENOMEM);
+
+ if (!drng_name) {
+ pr_err("DRNG name missing\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ if (!memcmp(drng_name, "drbg", 4)) {
+ pr_err("SP800-90A DRBG cannot be allocated using lrng_kcapi "
+ "backend, use lrng_drbg backend instead\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ if (!memcmp(drng_name, "stdrng", 6)) {
+ pr_err("stdrng cannot be allocated using lrng_kcapi backend, "
+ "it is too unspecific and potentially may allocate the "
+ "DRBG\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ lrng_drng_info = kmalloc(sizeof(*lrng_drng_info), GFP_KERNEL);
+ if (!lrng_drng_info)
+ return ERR_PTR(-ENOMEM);
+
+ kcapi_rng = crypto_alloc_rng(drng_name, 0, 0);
+ if (IS_ERR(kcapi_rng)) {
+ pr_err("DRNG %s cannot be allocated\n", drng_name);
+ ret = ERR_CAST(kcapi_rng);
+ goto free;
+ }
+ lrng_drng_info->kcapi_rng = kcapi_rng;
+
+ seedsize = crypto_rng_seedsize(kcapi_rng);
+
+ if (sec_strength > seedsize)
+ pr_info("Seedsize DRNG (%u bits) lower than "
+ "security strength of LRNG noise source (%u bits)\n",
+ crypto_rng_seedsize(kcapi_rng) * 8,
+ sec_strength * 8);
+
+ if (seedsize) {
+ struct lrng_hash_info *lrng_hash;
+
+ if (!seed_hash) {
+ switch (seedsize) {
+ case 32:
+ seed_hash = "sha256";
+ break;
+ case 48:
+ seed_hash = "sha384";
+ break;
+ case 64:
+ seed_hash = "sha512";
+ break;
+ default:
+ pr_err("Seed size %d cannot be processed\n",
+ seedsize);
+ goto dealloc;
+ break;
+ }
+ }
+
+ lrng_hash = _lrng_kcapi_hash_alloc(seed_hash);
+ if (IS_ERR(lrng_hash)) {
+ ret = ERR_CAST(lrng_hash);
+ goto dealloc;
+ }
+
+ if (seedsize != _lrng_kcapi_hash_digestsize(lrng_hash)) {
+ pr_err("Seed hash output size not equal to DRNG seed "
+ "size\n");
+ _lrng_kcapi_hash_free(lrng_hash);
+ ret = ERR_PTR(-EINVAL);
+ goto dealloc;
+ }
+
+ lrng_drng_info->lrng_hash = lrng_hash;
+
+ pr_info("Seed hash %s allocated\n", seed_hash);
+ } else {
+ lrng_drng_info->lrng_hash = NULL;
+ }
+
+ pr_info("Kernel crypto API DRNG %s allocated\n", drng_name);
+
+ return lrng_drng_info;
+
+dealloc:
+ crypto_free_rng(kcapi_rng);
+free:
+ kfree(lrng_drng_info);
+ return ret;
+}
+
+static void lrng_kcapi_drng_dealloc(void *drng)
+{
+ struct lrng_drng_info *lrng_drng_info = (struct lrng_drng_info *)drng;
+ struct crypto_rng *kcapi_rng = lrng_drng_info->kcapi_rng;
+ struct lrng_hash_info *lrng_hash = lrng_drng_info->lrng_hash;
+
+ crypto_free_rng(kcapi_rng);
+ if (lrng_hash) {
+ _lrng_kcapi_hash_free(lrng_hash);
+ pr_info("Seed hash %s deallocated\n", seed_hash);
+ }
+ kfree(lrng_drng_info);
+ pr_info("DRNG %s deallocated\n", drng_name);
+}
+
+static const char *lrng_kcapi_drng_name(void)
+{
+ return drng_name;
+}
+
+static const char *lrng_kcapi_pool_hash(void)
+{
+ return pool_hash;
+}
+
+const static struct lrng_crypto_cb lrng_kcapi_crypto_cb = {
+ .lrng_drng_name = lrng_kcapi_drng_name,
+ .lrng_hash_name = lrng_kcapi_pool_hash,
+ .lrng_drng_alloc = lrng_kcapi_drng_alloc,
+ .lrng_drng_dealloc = lrng_kcapi_drng_dealloc,
+ .lrng_drng_seed_helper = lrng_kcapi_drng_seed_helper,
+ .lrng_drng_generate_helper = lrng_kcapi_drng_generate_helper,
+ .lrng_hash_alloc = lrng_kcapi_hash_alloc,
+ .lrng_hash_dealloc = lrng_kcapi_hash_dealloc,
+ .lrng_hash_digestsize = lrng_kcapi_hash_digestsize,
+ .lrng_hash_buffer = lrng_kcapi_hash_buffer,
+};
+
+static int __init lrng_kcapi_init(void)
+{
+ return lrng_set_drng_cb(&lrng_kcapi_crypto_cb);
+}
+static void __exit lrng_kcapi_exit(void)
+{
+ lrng_set_drng_cb(NULL);
+}
+
+late_initcall(lrng_kcapi_init);
+module_exit(lrng_kcapi_exit);
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_AUTHOR("Stephan Mueller <smueller@chronox.de>");
+MODULE_DESCRIPTION("Linux Random Number Generator - kernel crypto API DRNG backend");
--
2.24.1
^ permalink raw reply related
* [PATCH v28 08/12] crypto: provide access to a static Jitter RNG state
From: Stephan Müller @ 2020-01-15 10:34 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Greg Kroah-Hartman, linux-crypto-u79uwXL29TY76Z2rM5mHXA, LKML,
linux-api-u79uwXL29TY76Z2rM5mHXA, Eric W. Biederman,
Alexander E. Patrakov, Ahmed S. Darwish, Theodore Y. Ts'o,
Willy Tarreau, Matthew Garrett, Vito Caputo, Andreas Dilger,
Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange, "Peter, Matthias" <matth>
In-Reply-To: <5951792.lmNsirYsPE-jJGQKZiSfeo1haGO/jJMPxvVK+yQ3ZXh@public.gmane.org>
To support the LRNG operation which uses the Jitter RNG separately
from the kernel crypto API, at a time where potentially the regular
memory management is not yet initialized, the Jitter RNG needs to
provide a state whose memory is defined at compile time. As only once
instance will ever be needed by the LRNG, define once static memory
block which is solely to be used by the LRNG.
CC: "Eric W. Biederman" <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
CC: "Alexander E. Patrakov" <patrakov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
CC: "Ahmed S. Darwish" <darwish.07-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
CC: "Theodore Y. Ts'o" <tytso-3s7WtUTddSA@public.gmane.org>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59-1xO5oi07KQx4cg9Nei1l7Q@public.gmane.org>
CC: Vito Caputo <vcaputo-IiWei5kqaphBDgjK7y7TUQ@public.gmane.org>
CC: Andreas Dilger <adilger.kernel-m1MBpc4rdrD3fQ9qLvQP4Q@public.gmane.org>
CC: Jan Kara <jack-AlSwsSmVLrQ@public.gmane.org>
CC: Ray Strode <rstrode-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
CC: William Jon McCann <mccann-4GNroTWusrE@public.gmane.org>
CC: zhangjs <zachary-+jQ8zvd9CV5SzTKKI/wJ2A@public.gmane.org>
CC: Andy Lutomirski <luto-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
CC: Florian Weimer <fweimer-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
CC: Lennart Poettering <mzxreary-uLTowLwuiw4b1SvskN2V4Q@public.gmane.org>
CC: Nicolai Stange <nstange-l3A5Bk7waGM@public.gmane.org>
Reviewed-by: Roman Drahtmueller <draht-UTA6Eu5RHcbyVfKhlJ+g5A@public.gmane.org>
Tested-by: Roman Drahtmüller <draht-UTA6Eu5RHcbyVfKhlJ+g5A@public.gmane.org>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
Tested-by: Neil Horman <nhorman-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org>
---
crypto/jitterentropy-kcapi.c | 3 +--
crypto/jitterentropy.c | 25 ++++++++++++++++++-
.../crypto/internal}/jitterentropy.h | 3 +++
3 files changed, 28 insertions(+), 3 deletions(-)
rename {crypto => include/crypto/internal}/jitterentropy.h (84%)
diff --git a/crypto/jitterentropy-kcapi.c b/crypto/jitterentropy-kcapi.c
index a5ce8f96790f..11b00e9f37f3 100644
--- a/crypto/jitterentropy-kcapi.c
+++ b/crypto/jitterentropy-kcapi.c
@@ -43,8 +43,7 @@
#include <linux/time.h>
#include <linux/crypto.h>
#include <crypto/internal/rng.h>
-
-#include "jitterentropy.h"
+#include <crypto/internal/jitterentropy.h>
/***************************************************************************
* Helper function
diff --git a/crypto/jitterentropy.c b/crypto/jitterentropy.c
index 042157f0d28b..529c9db13e64 100644
--- a/crypto/jitterentropy.c
+++ b/crypto/jitterentropy.c
@@ -103,7 +103,7 @@ struct rand_data {
* Helper functions
***************************************************************************/
-#include "jitterentropy.h"
+#include <crypto/internal/jitterentropy.h>
/**
* Update of the loop count used for the next round of
@@ -639,3 +639,26 @@ int jent_entropy_init(void)
return 0;
}
+
+struct rand_data *jent_lrng_entropy_collector(void)
+{
+ static unsigned char lrng_jent_mem[JENT_MEMORY_SIZE];
+ static struct rand_data lrng_jent_state = {
+ .data = 0,
+ .old_data = 0,
+ .prev_time = 0,
+ .last_delta = 0,
+ .last_delta2 = 0,
+ .osr = 1,
+ .mem = lrng_jent_mem,
+ .memlocation = 0,
+ .memblocks = JENT_MEMORY_BLOCKSIZE,
+ .memblocksize = JENT_MEMORY_BLOCKS,
+ .memaccessloops = JENT_MEMORY_ACCESSLOOPS,
+ };
+
+ if (jent_entropy_init())
+ return NULL;
+
+ return &lrng_jent_state;
+}
diff --git a/crypto/jitterentropy.h b/include/crypto/internal/jitterentropy.h
similarity index 84%
rename from crypto/jitterentropy.h
rename to include/crypto/internal/jitterentropy.h
index c83fff32d130..6e07d86eac82 100644
--- a/crypto/jitterentropy.h
+++ b/include/crypto/internal/jitterentropy.h
@@ -15,3 +15,6 @@ extern int jent_read_entropy(struct rand_data *ec, unsigned char *data,
extern struct rand_data *jent_entropy_collector_alloc(unsigned int osr,
unsigned int flags);
extern void jent_entropy_collector_free(struct rand_data *entropy_collector);
+
+/* Access to statically allocated Jitter RNG instance */
+extern struct rand_data *jent_lrng_entropy_collector(void);
--
2.24.1
^ permalink raw reply related
* [PATCH v28 09/12] LRNG - add Jitter RNG fast noise source
From: Stephan Müller @ 2020-01-15 10:34 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Greg Kroah-Hartman, linux-crypto, LKML, linux-api,
Eric W. Biederman, Alexander E. Patrakov, Ahmed S. Darwish,
Theodore Y. Ts'o, Willy Tarreau, Matthew Garrett, Vito Caputo,
Andreas Dilger, Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange, "Peter, Matthias" <matth>
In-Reply-To: <5951792.lmNsirYsPE@positron.chronox.de>
From 1666398d358388d0476f79019e0a6116c30db8a4 Mon Sep 17 00:00:00 2001
From: Stephan Mueller <smueller@chronox.de>
Date: Sun, 12 Jan 2020 11:56:31 +0100
Subject: [PATCH v28 09/12] LRNG - add Jitter RNG fast noise source
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
The Jitter RNG fast noise source implemented as part of the kernel
crypto API is queried for 256 bits of entropy at the time the seed
buffer managed by the LRNG is about to be filled.
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
drivers/char/lrng/Kconfig | 11 +++++
drivers/char/lrng/Makefile | 1 +
drivers/char/lrng/lrng_jent.c | 89 +++++++++++++++++++++++++++++++++++
3 files changed, 101 insertions(+)
create mode 100644 drivers/char/lrng/lrng_jent.c
diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index 0d070a3897dd..10b7cbdb8c8e 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -92,4 +92,15 @@ config LRNG_KCAPI
provided by the selected kernel crypto API RNG.
endif # LRNG_DRNG_SWITCH
+config LRNG_JENT
+ bool "Enable Jitter RNG as LRNG Seed Source"
+ select CRYPTO_JITTERENTROPY
+ help
+ The Linux RNG may use the Jitter RNG as noise source. Enabling
+ this option enables the use of the Jitter RNG. Its default
+ entropy level is 16 bits of entropy per 256 data bits delivered
+ by the Jitter RNG. This entropy level can be changed at boot
+ time or at runtime with the lrng_base.jitterrng configuration
+ variable.
+
endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 94b2dfb2dfdb..4f5b6f38f0c4 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -13,3 +13,4 @@ obj-$(CONFIG_SYSCTL) += lrng_proc.o
obj-$(CONFIG_LRNG_DRNG_SWITCH) += lrng_switch.o
obj-$(CONFIG_LRNG_DRBG) += lrng_drbg.o
obj-$(CONFIG_LRNG_KCAPI) += lrng_kcapi.o
+obj-$(CONFIG_LRNG_JENT) += lrng_jent.o
diff --git a/drivers/char/lrng/lrng_jent.c b/drivers/char/lrng/lrng_jent.c
new file mode 100644
index 000000000000..ff0bbe2680c4
--- /dev/null
+++ b/drivers/char/lrng/lrng_jent.c
@@ -0,0 +1,89 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG Fast Noise Source: Jitter RNG
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/types.h>
+#include <crypto/internal/jitterentropy.h>
+
+#include "lrng_internal.h"
+
+/*
+ * Estimated entropy of data is a 16th of LRNG_DRNG_SECURITY_STRENGTH_BITS.
+ * Albeit a full entropy assessment is provided for the noise source indicating
+ * that it provides high entropy rates and considering that it deactivates
+ * when it detects insufficient hardware, the chosen under estimation of
+ * entropy is considered to be acceptable to all reviewers.
+ */
+static u32 jitterrng = LRNG_DRNG_SECURITY_STRENGTH_BITS>>4;
+module_param(jitterrng, uint, 0644);
+MODULE_PARM_DESC(jitterrng, "Entropy in bits of 256 data bits from Jitter "
+ "RNG noise source");
+
+/**
+ * lrng_get_jent() - Get Jitter RNG entropy
+ *
+ * @outbuf: buffer to store entropy
+ * @outbuflen: length of buffer
+ *
+ * Return:
+ * * > 0 on success where value provides the added entropy in bits
+ * * 0 if no fast source was available
+ */
+static struct rand_data *lrng_jent_state;
+
+u32 lrng_get_jent(u8 *outbuf, unsigned int outbuflen)
+{
+ int ret;
+ u32 ent_bits = jitterrng;
+ unsigned long flags;
+ static DEFINE_SPINLOCK(lrng_jent_lock);
+ static int lrng_jent_initialized = 0;
+
+ spin_lock_irqsave(&lrng_jent_lock, flags);
+
+ if (!ent_bits || (lrng_jent_initialized == -1)) {
+ spin_unlock_irqrestore(&lrng_jent_lock, flags);
+ return 0;
+ }
+
+ if (!lrng_jent_initialized) {
+ lrng_jent_state = jent_lrng_entropy_collector();
+ if (!lrng_jent_state) {
+ jitterrng = 0;
+ lrng_jent_initialized = -1;
+ spin_unlock_irqrestore(&lrng_jent_lock, flags);
+ pr_info("Jitter RNG unusable on current system\n");
+ return 0;
+ }
+ lrng_jent_initialized = 1;
+ pr_debug("Jitter RNG working on current system\n");
+ }
+ ret = jent_read_entropy(lrng_jent_state, outbuf, outbuflen);
+ spin_unlock_irqrestore(&lrng_jent_lock, flags);
+
+ if (ret) {
+ pr_debug("Jitter RNG failed with %d\n", ret);
+ return 0;
+ }
+
+ /* Obtain entropy statement */
+ if (outbuflen != LRNG_DRNG_SECURITY_STRENGTH_BYTES)
+ ent_bits = (ent_bits * outbuflen<<3) /
+ LRNG_DRNG_SECURITY_STRENGTH_BITS;
+ /* Cap entropy to buffer size in bits */
+ ent_bits = min_t(u32, ent_bits, outbuflen<<3);
+ pr_debug("obtained %u bits of entropy from Jitter RNG noise source\n",
+ ent_bits);
+
+ return ent_bits;
+}
+
+u32 lrng_jent_entropylevel(void)
+{
+ return min_t(u32, jitterrng, LRNG_DRNG_SECURITY_STRENGTH_BITS);
+}
--
2.24.1
^ permalink raw reply related
* [PATCH v28 10/12] LRNG - add SP800-90B compliant health tests
From: Stephan Müller @ 2020-01-15 10:35 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Greg Kroah-Hartman, linux-crypto, LKML, linux-api,
Eric W. Biederman, Alexander E. Patrakov, Ahmed S. Darwish,
Theodore Y. Ts'o, Willy Tarreau, Matthew Garrett, Vito Caputo,
Andreas Dilger, Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange, "Peter, Matthias" <matth>
In-Reply-To: <5951792.lmNsirYsPE@positron.chronox.de>
Implement health tests for LRNG's slow noise sources as mandated by
SP-800-90B The file contains the following health tests:
- stuck test: The stuck test calculates the first, second and third
discrete derivative of the time stamp to be processed by the LFSR.
Only if all three values are non-zero, the received time delta is
considered to be non-stuck.
- SP800-90B Repetition Count Test (RCT): The LRNG uses an enhanced
version of the RCT specified in SP800-90B section 4.4.1. Instead of
counting identical back-to-back values, the input to the RCT is the
counting of the stuck values during the processing of received
interrupt events. The RCT is applied with alpha=2^-30 compliant to
the recommendation of FIPS 140-2 IG 9.8. During the counting operation,
the LRNG always calculates the RCT cut-off value of C. If that value
exceeds the allowed cut-off value, the LRNG will trigger the health
test failure discussed below. An error is logged to the kernel log
that such RCT failure occurred. This test is only applied and
enforced in FIPS mode, i.e. when the kernel compiled with
CONFIG_CONFIG_FIPS is started with fips=1.
- SP800-90B Adaptive Proportion Test (APT): The LRNG implements the
APT as defined in SP800-90B section 4.4.2. The applied significance
level again is alpha=2^-30 compliant to the recommendation of FIPS
140-2 IG 9.8.
The aforementioned health tests are applied to the first 1,024 time stamps
obtained from interrupt events. In case one error is identified for either
the RCT, or the APT, the collected entropy is invalidated and the
SP800-90B startup health test is restarted.
As long as the SP800-90B startup health test is not completed, all LRNG
random number output interfaces that may block will block and not generate
any data. This implies that only those potentially blocking interfaces are
defined to provide random numbers that are seeded with the interrupt noise
source being SP800-90B compliant. All other output interfaces will not be
affected by the SP800-90B startup test and thus are not considered
SP800-90B compliant.
At runtime, the SP800-90B APT and RCT are applied to each time stamp
generated for a received interrupt. When either the APT and RCT indicates
a noise source failure, the LRNG is reset to a state it has immediately
after boot:
- all entropy counters are set to zero
- the SP800-90B startup tests are re-performed which implies that
getrandom(2) would block again until new entropy was collected
To summarize, the following rules apply:
• SP800-90B compliant output interfaces
- /dev/random
- getrandom(2) system call
- get_random_bytes kernel-internal interface when being triggered by
the callback registered with add_random_ready_callback
• SP800-90B non-compliant output interfaces
- /dev/urandom
- get_random_bytes kernel-internal interface called directly
- randomize_page kernel-internal interface
- get_random_u32 and get_random_u64 kernel-internal interfaces
- get_random_u32_wait, get_random_u64_wait, get_random_int_wait, and
get_random_long_wait kernel-internal interfaces
If either the RCT, or the APT health test fails irrespective whether
during initialization or runtime, the following actions occur:
1. The entropy of the entire entropy pool is invalidated.
2. All DRNGs are reset which imply that they are treated as being
not seeded and require a reseed during next invocation.
3. The SP800-90B startup health test are initiated with all
implications of the startup tests. That implies that from that point
on, new events must be observed and its entropy must be inserted into
the entropy pool before random numbers are calculated from the
entropy pool.
Further details on the SP800-90B compliance and the availability of all
test tools required to perform all tests mandated by SP800-90B are
provided at [1].
The entire health testing code is compile-time configurable.
The patch provides a CONFIG_BROKEN configuration of the APT / RCT cutoff
values which have a high likelihood to trigger the health test failure.
The BROKEN APT cutoff is set to the exact mean of the expected value if
the time stamps are equally distributed (512 time stamps divided by 16
possible values due to using the 4 LSB of the time stamp). The BROKEN
RCT cutoff value is set to 1 which is likely to be triggered during
regular operation.
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
drivers/char/lrng/Kconfig | 56 +++++
drivers/char/lrng/Makefile | 1 +
drivers/char/lrng/lrng_health.c | 409 ++++++++++++++++++++++++++++++++
3 files changed, 466 insertions(+)
create mode 100644 drivers/char/lrng/lrng_health.c
diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index 10b7cbdb8c8e..12b6b3439853 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -103,4 +103,60 @@ config LRNG_JENT
time or at runtime with the lrng_base.jitterrng configuration
variable.
+config LRNG_HEALTH_TESTS
+ bool "Enable noise source online health tests"
+ help
+ The online health tests validate the noise source at
+ runtime for fatal errors. These tests include SP800-90B
+ compliant tests which are invoked if the system is booted
+ with fips=1. In case of fatal errors during active
+ SP800-90B tests, the issue is logged and the noise
+ data is discarded. These tests are required for full
+ compliance with SP800-90B.
+
+ If unsure, say Y.
+
+config LRNG_RCT_BROKEN
+ bool "SP800-90B RCT with dangerous low cutoff value"
+ depends on LRNG_HEALTH_TESTS
+ depends on BROKEN
+ default n
+ help
+ This option enables a dangerously low SP800-90B repetitive
+ count test (RCT) cutoff value which makes it very likely
+ that the RCT is triggered to raise a self test failure.
+
+ This option is ONLY intended for developers wanting to
+ test the effectiveness of the SP800-90B RCT health test.
+
+ If unsure, say N.
+
+config LRNG_APT_BROKEN
+ bool "SP800-90B APT with dangerous low cutoff value"
+ depends on LRNG_HEALTH_TESTS
+ depends on BROKEN
+ default n
+ help
+ This option enables a dangerously low SP800-90B adaptive
+ proportion test (APT) cutoff value which makes it very
+ likely that the RCT is triggered to raise a self test
+ failure.
+
+ This option is ONLY intended for developers wanting to
+ test the effectiveness of the SP800-90B APT health test.
+
+ If unsure, say N.
+
+# Default taken from SP800-90B sec 4.4.1 - significance level 2^-30
+config LRNG_RCT_CUTOFF
+ int
+ default 31 if !LRNG_RCT_BROKEN
+ default 1 if LRNG_RCT_BROKEN
+
+# Default taken from SP800-90B sec 4.4.2 - significance level 2^-30
+config LRNG_APT_CUTOFF
+ int
+ default 325 if !LRNG_APT_BROKEN
+ default 32 if LRNG_APT_BROKEN
+
endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index 4f5b6f38f0c4..c3008763dd14 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -14,3 +14,4 @@ obj-$(CONFIG_LRNG_DRNG_SWITCH) += lrng_switch.o
obj-$(CONFIG_LRNG_DRBG) += lrng_drbg.o
obj-$(CONFIG_LRNG_KCAPI) += lrng_kcapi.o
obj-$(CONFIG_LRNG_JENT) += lrng_jent.o
+obj-$(CONFIG_LRNG_HEALTH_TESTS) += lrng_health.o
diff --git a/drivers/char/lrng/lrng_health.c b/drivers/char/lrng/lrng_health.c
new file mode 100644
index 000000000000..4aeca29d0b87
--- /dev/null
+++ b/drivers/char/lrng/lrng_health.c
@@ -0,0 +1,409 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Linux Random Number Generator (LRNG) Health Testing
+ *
+ * Copyright (C) 2019 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/fips.h>
+#include <linux/module.h>
+
+#include "lrng_internal.h"
+
+/* Stuck Test */
+struct lrng_stuck_test {
+ u32 last_time; /* Stuck test: time of previous IRQ */
+ u32 last_delta; /* Stuck test: delta of previous IRQ */
+ u32 last_delta2; /* Stuck test: 2. time derivation of prev IRQ */
+};
+
+/* Repetition Count Test */
+struct lrng_rct {
+ atomic_t rct_count; /* Number of stuck values */
+};
+
+/* Adaptive Proportion Test */
+struct lrng_apt {
+ /* Data window size */
+#define LRNG_APT_WINDOW_SIZE 512
+ /* LSB of time stamp to process */
+#define LRNG_APT_LSB 16
+#define LRNG_APT_WORD_MASK (LRNG_APT_LSB - 1)
+ atomic_t apt_count; /* APT counter */
+ atomic_t apt_base; /* APT base reference */
+
+ atomic_t apt_trigger;
+ bool apt_base_set; /* Is APT base set? */
+};
+
+/* The health test code must operate lock-less */
+struct lrng_health {
+ struct lrng_rct rct;
+ struct lrng_apt apt;
+
+ bool health_test_enabled;
+
+ /* SP800-90B startup health tests */
+#define LRNG_SP80090B_STARTUP_SAMPLES 1024
+#define LRNG_SP80090B_STARTUP_BLOCKS ((LRNG_SP80090B_STARTUP_SAMPLES + \
+ LRNG_APT_WINDOW_SIZE - 1) / \
+ LRNG_APT_WINDOW_SIZE)
+ bool sp80090b_startup_done;
+ atomic_t sp80090b_startup_blocks;
+};
+
+static struct lrng_health lrng_health = {
+ .rct.rct_count = ATOMIC_INIT(0),
+
+ .apt.apt_count = ATOMIC_INIT(0),
+ .apt.apt_base = ATOMIC_INIT(-1),
+ .apt.apt_trigger = ATOMIC_INIT(LRNG_APT_WINDOW_SIZE),
+ .apt.apt_base_set = false,
+
+ .health_test_enabled = true,
+
+ .sp80090b_startup_blocks = ATOMIC_INIT(LRNG_SP80090B_STARTUP_BLOCKS),
+ .sp80090b_startup_done = false,
+};
+
+static DEFINE_PER_CPU(struct lrng_stuck_test, lrng_stuck_test);
+
+static inline bool lrng_sp80090b_health_requested(void)
+{
+ /* Health tests are only requested in FIPS mode */
+ return fips_enabled;
+}
+
+static inline bool lrng_sp80090b_health_enabled(void)
+{
+ struct lrng_health *health = &lrng_health;
+
+ return lrng_sp80090b_health_requested() && health->health_test_enabled;
+}
+
+/***************************************************************************
+ * SP800-90B Compliance
+ *
+ * If the Linux-RNG is booted into FIPS mode, the following interfaces
+ * provide an SP800-90B compliant noise source:
+ *
+ * * /dev/random
+ * * getrandom(2)
+ * * get_random_bytes when using it in conjunction with
+ * add_random_ready_callback
+ *
+ * All other interfaces, including /dev/urandom or get_random_bytes without
+ * the add_random_ready_callback cannot claim to use an SP800-90B compliant
+ * noise source.
+ ***************************************************************************/
+
+/*
+ * Perform SP800-90B startup testing
+ */
+static inline void lrng_sp80090b_startup(struct lrng_health *health)
+{
+ if (!health->sp80090b_startup_done &&
+ atomic_dec_and_test(&health->sp80090b_startup_blocks)) {
+ health->sp80090b_startup_done = true;
+ pr_info("SP800-90B startup health tests completed\n");
+ lrng_init_ops(0);
+
+ /*
+ * Force a reseed of DRNGs to ensure they are seeded with
+ * entropy that passed the SP800-90B health tests.
+ * As the DRNG always will reseed before generating
+ * random numbers, it does not need a reseed trigger.
+ */
+ lrng_drng_force_reseed();
+ }
+}
+
+/*
+ * Handle failure of SP800-90B startup testing
+ */
+static inline void lrng_sp80090b_startup_failure(struct lrng_health *health)
+{
+ /* Reset of LRNG and its entropy - NOTE: we are in atomic context */
+ lrng_reset();
+
+ /*
+ * Reset the SP800-90B startup test.
+ *
+ * NOTE SP800-90B section 4.3 bullet 4 does not specify what
+ * exactly is to be done in case of failure! Thus, we do what
+ * makes sense, i.e. restarting the health test and thus gating
+ * the output function of /dev/random and getrandom(2).
+ */
+ atomic_set(&health->sp80090b_startup_blocks,
+ LRNG_SP80090B_STARTUP_BLOCKS);
+}
+
+/*
+ * Handle failure of SP800-90B runtime testing
+ */
+static inline void lrng_sp80090b_runtime_failure(struct lrng_health *health)
+{
+ lrng_sp80090b_startup_failure(health);
+ health->sp80090b_startup_done = false;
+}
+
+static inline void lrng_sp80090b_failure(struct lrng_health *health)
+{
+ if (health->sp80090b_startup_done) {
+ pr_err("SP800-90B runtime health test failure - invalidating "
+ "all existing entropy and initiate SP800-90B startup\n");
+ lrng_sp80090b_runtime_failure(health);
+ } else {
+ pr_err("SP800-90B startup test failure - resetting\n");
+ lrng_sp80090b_startup_failure(health);
+ }
+}
+
+/*
+ * Is the SP800-90B startup testing complete?
+ *
+ * This function is called by the LRNG to determine whether to unblock
+ * a certain user interface. Therefore, only the potentially blocking
+ * user interfaces are considered SP800-90B compliant.
+ */
+bool lrng_sp80090b_startup_complete(void)
+{
+ struct lrng_health *health = &lrng_health;
+
+ return (lrng_sp80090b_health_enabled()) ? health->sp80090b_startup_done:
+ true;
+}
+
+bool lrng_sp80090b_compliant(void)
+{
+ struct lrng_health *health = &lrng_health;
+
+ return lrng_sp80090b_health_enabled() && health->sp80090b_startup_done;
+}
+
+/***************************************************************************
+ * Adaptive Proportion Test
+ *
+ * This test complies with SP800-90B section 4.4.2.
+ ***************************************************************************/
+
+/*
+ * Reset the APT counter
+ *
+ * @health [in] Reference to health state
+ */
+static inline void lrng_apt_reset(struct lrng_health *health,
+ unsigned int time_masked)
+{
+ struct lrng_apt *apt = &health->apt;
+
+ pr_debug("APT value %d for base %d\n",
+ atomic_read(&apt->apt_count), atomic_read(&apt->apt_base));
+
+ /* Reset APT */
+ atomic_set(&apt->apt_count, 0);
+ atomic_set(&apt->apt_base, time_masked);
+}
+
+static inline void lrng_apt_restart(struct lrng_health *health)
+{
+ struct lrng_apt *apt = &health->apt;
+
+ atomic_set(&apt->apt_trigger, LRNG_APT_WINDOW_SIZE);
+}
+
+/*
+ * Insert a new entropy event into APT
+ *
+ * This function does is void as it does not decide about the fate of a time
+ * stamp. An APT failure can only happen at the same time of a stuck test
+ * failure. Thus, the stuck failure will already decide how the time stamp
+ * is handled.
+ *
+ * @health [in] Reference to health state
+ * @now_time [in] Time stamp to process
+ */
+static inline void lrng_apt_insert(struct lrng_health *health,
+ unsigned int now_time)
+{
+ struct lrng_apt *apt = &health->apt;
+
+ if (!lrng_sp80090b_health_requested())
+ return;
+
+ now_time &= LRNG_APT_WORD_MASK;
+
+ /* Initialization of APT */
+ if (!apt->apt_base_set) {
+ atomic_set(&apt->apt_base, now_time);
+ apt->apt_base_set = true;
+ return;
+ }
+
+ if (now_time == (unsigned int)atomic_read(&apt->apt_base)) {
+ u32 apt_val = (u32)atomic_inc_return_relaxed(&apt->apt_count);
+
+ if (apt_val >= CONFIG_LRNG_APT_CUTOFF)
+ lrng_sp80090b_failure(health);
+ }
+
+ if (atomic_dec_and_test(&apt->apt_trigger)) {
+ lrng_apt_restart(health);
+ lrng_apt_reset(health, now_time);
+ lrng_sp80090b_startup(health);
+ }
+}
+
+/***************************************************************************
+ * Repetition Count Test
+ *
+ * The LRNG uses an enhanced version of the Repetition Count Test
+ * (RCT) specified in SP800-90B section 4.4.1. Instead of counting identical
+ * back-to-back values, the input to the RCT is the counting of the stuck
+ * values while filling the entropy pool.
+ *
+ * The RCT is applied with an alpha of 2^-30 compliant to FIPS 140-2 IG 9.8.
+ *
+ * During the counting operation, the LRNG always calculates the RCT
+ * cut-off value of C. If that value exceeds the allowed cut-off value,
+ * the LRNG will invalidate all entropy for the entropy pool which implies
+ * that no data can be extracted from the entropy pool unless new entropy
+ * is received.
+ ***************************************************************************/
+
+/*
+ * Hot code path - Insert data for Repetition Count Test
+ *
+ * @health: Reference to health information
+ * @stuck: Decision of stuck test
+ */
+static inline void lrng_rct(struct lrng_health *health, int stuck)
+{
+ struct lrng_rct *rct = &health->rct;
+
+ if (!lrng_sp80090b_health_requested())
+ return;
+
+ if (stuck) {
+ u32 rct_count = atomic_add_return_relaxed(1, &rct->rct_count);
+
+ pr_debug("RCT count: %u\n", rct_count);
+
+ /*
+ * The cutoff value is based on the following consideration:
+ * alpha = 2^-30 as recommended in FIPS 140-2 IG 9.8.
+ * In addition, we imply an entropy value H of 1 bit as this
+ * is the minimum entropy required to provide full entropy.
+ *
+ * Note, rct_count (which equals to value B in the
+ * pseudo code of SP800-90B section 4.4.1) starts with zero.
+ * Hence we need to subtract one from the cutoff value as
+ * calculated following SP800-90B.
+ */
+ if (rct_count >= CONFIG_LRNG_RCT_CUTOFF) {
+ atomic_set(&rct->rct_count, 0);
+
+ /*
+ * APT must start anew as we consider all previously
+ * recorded data to contain no entropy.
+ */
+ lrng_apt_restart(health);
+
+ lrng_sp80090b_failure(health);
+ }
+ } else {
+ atomic_set(&rct->rct_count, 0);
+ }
+}
+
+/***************************************************************************
+ * Stuck Test
+ *
+ * Checking the:
+ * 1st derivative of the event occurrence (time delta)
+ * 2nd derivative of the event occurrence (delta of time deltas)
+ * 3rd derivative of the event occurrence (delta of delta of time deltas)
+ *
+ * All values must always be non-zero. The stuck test is only valid disabled if
+ * high-resolution time stamps are identified after initialization.
+ ***************************************************************************/
+
+static inline u32 lrng_delta(u32 prev, u32 next)
+{
+ /*
+ * Note that this (unsigned) subtraction does yield the correct value
+ * in the wraparound-case, i.e. when next < prev.
+ */
+ return (next - prev);
+}
+
+/*
+ * Hot code path
+ *
+ * @health: Reference to health information
+ * @now: Event time
+ * @return: 0 event occurrence not stuck (good time stamp)
+ * != 0 event occurrence stuck (reject time stamp)
+ */
+static inline int lrng_irq_stuck(struct lrng_stuck_test *stuck, u32 now_time)
+{
+ u32 delta = lrng_delta(stuck->last_time, now_time);
+ u32 delta2 = lrng_delta(stuck->last_delta, delta);
+ u32 delta3 = lrng_delta(stuck->last_delta2, delta2);
+
+ stuck->last_time = now_time;
+ stuck->last_delta = delta;
+ stuck->last_delta2 = delta2;
+
+ if (!delta || !delta2 || !delta3)
+ return 1;
+
+ return 0;
+}
+
+/***************************************************************************
+ * Health test interfaces
+ ***************************************************************************/
+
+/*
+ * Disable all health tests
+ */
+void lrng_health_disable(void)
+{
+ struct lrng_health *health = &lrng_health;
+
+ health->health_test_enabled = false;
+
+ if (lrng_sp80090b_health_requested())
+ pr_warn("SP800-90B compliance requested but the Linux RNG is "
+ "NOT SP800-90B compliant\n");
+}
+
+/*
+ * Hot code path - Perform health test on time stamp received from an event
+ *
+ * @now_time Time stap
+ */
+enum lrng_health_res lrng_health_test(u32 now_time)
+{
+ struct lrng_health *health = &lrng_health;
+ struct lrng_stuck_test *stuck_test = this_cpu_ptr(&lrng_stuck_test);
+ int stuck;
+
+ if (!health->health_test_enabled)
+ return lrng_health_pass;
+
+ lrng_apt_insert(health, now_time);
+
+ stuck = lrng_irq_stuck(stuck_test, now_time);
+ lrng_rct(health, stuck);
+ if (stuck) {
+ /* SP800-90B disallows using a failing health test time stamp */
+ return lrng_sp80090b_health_requested() ?
+ lrng_health_fail_drop : lrng_health_fail_use;
+ }
+
+ return lrng_health_pass;
+}
--
2.24.1
^ permalink raw reply related
* [PATCH v28 11/12] LRNG - add interface for gathering of raw entropy
From: Stephan Müller @ 2020-01-15 10:35 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Greg Kroah-Hartman, linux-crypto, LKML, linux-api,
Eric W. Biederman, Alexander E. Patrakov, Ahmed S. Darwish,
Theodore Y. Ts'o, Willy Tarreau, Matthew Garrett, Vito Caputo,
Andreas Dilger, Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange, "Peter, Matthias" <matth>
In-Reply-To: <5951792.lmNsirYsPE@positron.chronox.de>
The test interface allows a privileged process to capture the raw
unconditioned noise that is collected by the LRNG for statistical
analysis. Such testing allows the analysis how much entropy
the interrupt noise source provides on a given platform.
Extracted noise data is not used to seed the LRNG. This
is a test interface and not appropriate for production systems.
Yet, the interface is considered to be sufficiently secured for
production systems.
Access to the data is given through the lrng_raw debugfs file. The
data buffer should be multiples of sizeof(u32) to fill the entire
buffer. Using the option lrng_testing.boot_test=1 the raw noise of
the first 1000 entropy events since boot can be sampled.
This test interface allows generating the data required for
analysis whether the LRNG is in compliance with SP800-90B
sections 3.1.3 and 3.1.4.
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
Reviewed-by: Roman Drahtmueller <draht@schaltsekun.de>
Tested-by: Roman Drahtmüller <draht@schaltsekun.de>
Tested-by: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
Tested-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
drivers/char/lrng/Kconfig | 16 ++
drivers/char/lrng/Makefile | 1 +
drivers/char/lrng/lrng_testing.c | 271 +++++++++++++++++++++++++++++++
3 files changed, 288 insertions(+)
create mode 100644 drivers/char/lrng/lrng_testing.c
diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index 12b6b3439853..e503a4bb7475 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -159,4 +159,20 @@ config LRNG_APT_CUTOFF
default 325 if !LRNG_APT_BROKEN
default 32 if LRNG_APT_BROKEN
+config LRNG_TESTING
+ bool "Enable entropy test interface to LRNG noise source"
+ depends on DEBUG_FS
+ help
+ The test interface allows a privileged process to capture
+ the raw unconditioned noise that is collected by the LRNG
+ for statistical analysis. Extracted noise data is not used
+ to seed the LRNG.
+
+ The raw noise data can be obtained using the lrng_raw
+ debugfs file. Using the option lrng_testing.boot_test=1
+ the raw noise of the first 1000 entropy events since boot
+ can be sampled.
+
+ If unsure, say N.
+
endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index c3008763dd14..b2ce1979dc4b 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -15,3 +15,4 @@ obj-$(CONFIG_LRNG_DRBG) += lrng_drbg.o
obj-$(CONFIG_LRNG_KCAPI) += lrng_kcapi.o
obj-$(CONFIG_LRNG_JENT) += lrng_jent.o
obj-$(CONFIG_LRNG_HEALTH_TESTS) += lrng_health.o
+obj-$(CONFIG_LRNG_TESTING) += lrng_testing.o
diff --git a/drivers/char/lrng/lrng_testing.c b/drivers/char/lrng/lrng_testing.c
new file mode 100644
index 000000000000..0e287eccd622
--- /dev/null
+++ b/drivers/char/lrng/lrng_testing.c
@@ -0,0 +1,271 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * Linux Random Number Generator (LRNG) Raw entropy collection tool
+ *
+ * Copyright (C) 2019 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/atomic.h>
+#include <linux/bug.h>
+#include <linux/debugfs.h>
+#include <linux/module.h>
+#include <linux/sched.h>
+#include <linux/sched/signal.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/uaccess.h>
+#include <linux/workqueue.h>
+#include <asm/errno.h>
+
+#include "lrng_internal.h"
+
+#define LRNG_TESTING_RINGBUFFER_SIZE 1024
+#define LRNG_TESTING_RINGBUFFER_MASK (LRNG_TESTING_RINGBUFFER_SIZE - 1)
+
+static u32 lrng_testing_rb[LRNG_TESTING_RINGBUFFER_SIZE];
+static u32 lrng_rb_reader = 0;
+static u32 lrng_rb_writer = 0;
+static atomic_t lrng_testing_enabled = ATOMIC_INIT(0);
+
+static DECLARE_WAIT_QUEUE_HEAD(lrng_raw_read_wait);
+static DEFINE_SPINLOCK(lrng_raw_lock);
+
+/*
+ * 0 ==> No boot test, gathering of runtime data allowed
+ * 1 ==> Boot test enabled and ready for collecting data, gathering runtime
+ * data is disabled
+ * 2 ==> Boot test completed and disabled, gathering of runtime data is
+ * disabled
+ */
+static u32 boot_test = 0;
+module_param(boot_test, uint, 0644);
+MODULE_PARM_DESC(boot_test, "Enable gathering boot time entropy of the first"
+ " entropy events");
+
+static inline void lrng_raw_entropy_reset(void)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&lrng_raw_lock, flags);
+ lrng_rb_reader = 0;
+ lrng_rb_writer = 0;
+ spin_unlock_irqrestore(&lrng_raw_lock, flags);
+}
+
+static void lrng_raw_entropy_init(void)
+{
+ /*
+ * The boot time testing implies we have a running test. If the
+ * caller wants to clear it, he has to unset the boot_test flag
+ * at runtime via sysfs to enable regular runtime testing
+ */
+ if (boot_test)
+ return;
+
+ lrng_raw_entropy_reset();
+ atomic_set(&lrng_testing_enabled, 1);
+ pr_warn("Enabling raw entropy collection\n");
+}
+
+static void lrng_raw_entropy_fini(void)
+{
+ if (boot_test)
+ return;
+
+ atomic_set(&lrng_testing_enabled, 0);
+ lrng_raw_entropy_reset();
+ pr_warn("Disabling raw entropy collection\n");
+}
+
+bool lrng_raw_entropy_store(u32 value)
+{
+ unsigned long flags;
+
+ if (!atomic_read(&lrng_testing_enabled) && (boot_test != 1))
+ return false;
+
+ spin_lock_irqsave(&lrng_raw_lock, flags);
+
+ /*
+ * Disable entropy testing for boot time testing after ring buffer
+ * is filled.
+ */
+ if (boot_test) {
+ if (lrng_rb_writer > LRNG_TESTING_RINGBUFFER_SIZE) {
+ boot_test = 2;
+ pr_warn_once("Boot time entropy collection test "
+ "disabled\n");
+ spin_unlock_irqrestore(&lrng_raw_lock, flags);
+ return false;
+ }
+
+ if (lrng_rb_writer == 1)
+ pr_warn("Boot time entropy collection test enabled\n");
+ }
+
+ lrng_testing_rb[lrng_rb_writer & LRNG_TESTING_RINGBUFFER_MASK] = value;
+ lrng_rb_writer++;
+
+ spin_unlock_irqrestore(&lrng_raw_lock, flags);
+
+ if (wq_has_sleeper(&lrng_raw_read_wait))
+ wake_up_interruptible(&lrng_raw_read_wait);
+
+ return true;
+}
+
+static inline bool lrng_raw_have_data(void)
+{
+ return ((lrng_rb_writer & LRNG_TESTING_RINGBUFFER_MASK) !=
+ (lrng_rb_reader & LRNG_TESTING_RINGBUFFER_MASK));
+}
+
+static int lrng_raw_entropy_reader(u8 *outbuf, u32 outbuflen)
+{
+ unsigned long flags;
+ int collected_data = 0;
+
+ lrng_raw_entropy_init();
+
+ while (outbuflen) {
+ spin_lock_irqsave(&lrng_raw_lock, flags);
+
+ /* We have no data or reached the writer. */
+ if (!lrng_rb_writer || (lrng_rb_writer == lrng_rb_reader)) {
+
+ spin_unlock_irqrestore(&lrng_raw_lock, flags);
+
+ /*
+ * Now we gathered all boot data, enable regular data
+ * collection.
+ */
+ if (boot_test) {
+ boot_test = 0;
+ goto out;
+ }
+
+ wait_event_interruptible(lrng_raw_read_wait,
+ lrng_raw_have_data());
+ if (signal_pending(current)) {
+ collected_data = -ERESTARTSYS;
+ goto out;
+ }
+
+ continue;
+ }
+
+ /* We copy out word-wise */
+ if (outbuflen < sizeof(u32)) {
+ spin_unlock_irqrestore(&lrng_raw_lock, flags);
+ goto out;
+ }
+
+ memcpy(outbuf, &lrng_testing_rb[lrng_rb_reader], sizeof(u32));
+ lrng_rb_reader++;
+
+ spin_unlock_irqrestore(&lrng_raw_lock, flags);
+
+ outbuf += sizeof(u32);
+ outbuflen -= sizeof(u32);
+ collected_data += sizeof(u32);
+ }
+
+out:
+ lrng_raw_entropy_fini();
+ return collected_data;
+}
+
+/**************************************************************************
+ * Debugfs interface
+ **************************************************************************/
+static int lrng_raw_extract_user(char __user *buf, size_t nbytes)
+{
+ u8 *tmp, *tmp_aligned;
+ int ret = 0, large_request = (nbytes > 256);
+
+ /*
+ * The intention of this interface is for collecting at least
+ * 1000 samples due to the SP800-90B requirements. So, we make no
+ * effort in avoiding allocating more memory that actually needed
+ * by the user. Hence, we allocate sufficient memory to always hold
+ * that amount of data.
+ */
+ tmp = kmalloc(LRNG_TESTING_RINGBUFFER_SIZE + sizeof(u32), GFP_KERNEL);
+ if (!tmp)
+ return -ENOMEM;
+
+ tmp_aligned = PTR_ALIGN(tmp, sizeof(u32));
+
+ while (nbytes) {
+ int i;
+
+ if (large_request && need_resched()) {
+ if (signal_pending(current)) {
+ if (ret == 0)
+ ret = -ERESTARTSYS;
+ break;
+ }
+ schedule();
+ }
+
+ i = min_t(int, nbytes, LRNG_TESTING_RINGBUFFER_SIZE);
+ i = lrng_raw_entropy_reader(tmp_aligned, i);
+ if (i <= 0) {
+ if (i < 0)
+ ret = i;
+ break;
+ }
+ if (copy_to_user(buf, tmp_aligned, i)) {
+ ret = -EFAULT;
+ break;
+ }
+
+ nbytes -= i;
+ buf += i;
+ ret += i;
+ }
+
+ kzfree(tmp);
+ return ret;
+}
+
+/* DebugFS operations and definition of the debugfs files */
+static ssize_t lrng_raw_read(struct file *file, char __user *to,
+ size_t count, loff_t *ppos)
+{
+ loff_t pos = *ppos;
+ int ret;
+
+ if (!count)
+ return 0;
+
+ ret = lrng_raw_extract_user(to, count);
+ if (ret < 0)
+ return ret;
+
+ count -= ret;
+ *ppos = pos + count;
+
+ return ret;
+}
+
+static const struct file_operations lrng_raw_name_fops = {
+ .owner = THIS_MODULE,
+ .read = lrng_raw_read,
+};
+
+static int __init lrng_raw_init(void)
+{
+ struct dentry *lrng_raw_debugfs_root;
+
+ lrng_raw_debugfs_root = debugfs_create_dir(KBUILD_MODNAME, NULL);
+ debugfs_create_file_unsafe("lrng_raw", 0400, lrng_raw_debugfs_root,
+ NULL, &lrng_raw_name_fops);
+
+ return 0;
+}
+
+module_init(lrng_raw_init);
--
2.24.1
^ permalink raw reply related
* [PATCH v28 12/12] LRNG - add power-on and runtime self-tests
From: Stephan Müller @ 2020-01-15 10:36 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Greg Kroah-Hartman, linux-crypto, LKML, linux-api,
Eric W. Biederman, Alexander E. Patrakov, Ahmed S. Darwish,
Theodore Y. Ts'o, Willy Tarreau, Matthew Garrett, Vito Caputo,
Andreas Dilger, Jan Kara, Ray Strode, William Jon McCann, zhangjs,
Andy Lutomirski, Florian Weimer, Lennart Poettering,
Nicolai Stange, "Peter, Matthias" <matth>
In-Reply-To: <5951792.lmNsirYsPE@positron.chronox.de>
Parts of the LFSR are already covered by self-tests, including:
* Self-test of SP800-90A DRBG provided by the Linux kernel crypto API.
* Self-test of the PRNG provided by the Linux kernel crypto API.
* Raw noise source data testing including SP800-90B compliant
tests when enabling CONFIG_LRNG_HEALTH_TESTS
This patch adds the self-tests for the remaining critical functions of
the LRNG that are essential to maintain entropy and provide
cryptographic strong random numbers. The following self-tests are
implemented:
* Self-test of the time array maintenance. This test verifies whether
the time stamp array management to store multiple values in one integer
implements a concatenation of the data.
* Self-test of the LFSR operation. This test injects a monotonic
increasing counter into the LFSR. After completion of the injection of
the counter, 3 pool words are compared with known good values. The known
good values are calculated with the newly-developed tool
lfsr_testvector_generation provided as part of the LRNG test tool set at
[1].
* Self-test of the Hash_DF operation ensures that this function operates
compliant to the specification. The self-test performs a Hash_DF
operation of a zeroized entropy pool state. The test vectors are
generated using the newly-developed tool hash_df_testvector_generation
provided as part of the LRNG test tool set at [1].
* Self-test of the ChaCha20 DRNG is based on the self-tests that are
already present and implemented with the stand-alone user space
ChaCha20 DRNG implementation available at [2]. The self-tests cover
different use cases of the DRNG seeded with known seed data.
The status of the LRNG self-tests is provided with the selftest_status
SysFS file. If the file contains a zero, the self-tests passed. The
value 0xffffffff means that the self-tests were not executed. Any other
value indicates a self-test failure.
The self-test may be compiled to panic the system if the self-test
fails.
All self-tests operate on private state data structures. This implies
that none of the self-tests have any impact on the regular LRNG
operations. This allows the self-tests to be repeated at runtime by
writing anything into the selftest_status SysFS file.
[1] https://www.chronox.de/lrng.html
[2] https://www.chronox.de/chacha20.html
CC: "Eric W. Biederman" <ebiederm@xmission.com>
CC: "Alexander E. Patrakov" <patrakov@gmail.com>
CC: "Ahmed S. Darwish" <darwish.07@gmail.com>
CC: "Theodore Y. Ts'o" <tytso@mit.edu>
CC: Willy Tarreau <w@1wt.eu>
CC: Matthew Garrett <mjg59@srcf.ucam.org>
CC: Vito Caputo <vcaputo@pengaru.com>
CC: Andreas Dilger <adilger.kernel@dilger.ca>
CC: Jan Kara <jack@suse.cz>
CC: Ray Strode <rstrode@redhat.com>
CC: William Jon McCann <mccann@jhu.edu>
CC: zhangjs <zachary@baishancloud.com>
CC: Andy Lutomirski <luto@kernel.org>
CC: Florian Weimer <fweimer@redhat.com>
CC: Lennart Poettering <mzxreary@0pointer.de>
CC: Nicolai Stange <nstange@suse.de>
CC: Marcelo Henrique Cerri <marcelo.cerri@canonical.com>
CC: Neil Horman <nhorman@redhat.com>
Signed-off-by: Stephan Mueller <smueller@chronox.de>
---
drivers/char/lrng/Kconfig | 25 ++
drivers/char/lrng/Makefile | 1 +
drivers/char/lrng/lrng_selftest.c | 418 ++++++++++++++++++++++++++++++
3 files changed, 444 insertions(+)
create mode 100644 drivers/char/lrng/lrng_selftest.c
diff --git a/drivers/char/lrng/Kconfig b/drivers/char/lrng/Kconfig
index e503a4bb7475..0c9ad9563fb7 100644
--- a/drivers/char/lrng/Kconfig
+++ b/drivers/char/lrng/Kconfig
@@ -175,4 +175,29 @@ config LRNG_TESTING
If unsure, say N.
+config LRNG_SELFTEST
+ bool "Enable power-on and on-demand self-tests"
+ help
+ The power-on self-tests are executed during boot time
+ covering the ChaCha20 DRNG, the LFSR processing and the
+ time stamp management of the LRNG.
+
+ The on-demand self-tests are triggered by writing any
+ value into the SysFS file selftest_status. At the same
+ time, when reading this file, the test status is
+ returned. A zero indicates that all tests were executed
+ successfully.
+
+ If unsure, say Y.
+
+if LRNG_SELFTEST
+
+config LRNG_SELFTEST_PANIC
+ bool "Panic the kernel upon self-test failure"
+ help
+ If the option is enabled, the kernel is terminated if an
+ LRNG power-on self-test failure is detected.
+
+endif # LRNG_SELFTEST
+
endif # LRNG
diff --git a/drivers/char/lrng/Makefile b/drivers/char/lrng/Makefile
index b2ce1979dc4b..92219c565f66 100644
--- a/drivers/char/lrng/Makefile
+++ b/drivers/char/lrng/Makefile
@@ -16,3 +16,4 @@ obj-$(CONFIG_LRNG_KCAPI) += lrng_kcapi.o
obj-$(CONFIG_LRNG_JENT) += lrng_jent.o
obj-$(CONFIG_LRNG_HEALTH_TESTS) += lrng_health.o
obj-$(CONFIG_LRNG_TESTING) += lrng_testing.o
+obj-$(CONFIG_LRNG_SELFTEST) += lrng_selftest.o
diff --git a/drivers/char/lrng/lrng_selftest.c b/drivers/char/lrng/lrng_selftest.c
new file mode 100644
index 000000000000..da213572e1dd
--- /dev/null
+++ b/drivers/char/lrng/lrng_selftest.c
@@ -0,0 +1,418 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
+/*
+ * LRNG power-on and on-demand self-test
+ *
+ * Copyright (C) 2016 - 2020, Stephan Mueller <smueller@chronox.de>
+ */
+
+/*
+ * In addition to the self-tests below, the following LRNG components
+ * are covered with self-tests during regular operation:
+ *
+ * * power-on self-test: SP800-90A DRBG provided by the Linux kernel crypto API
+ * * power-on self-test: PRNG provided by the Linux kernel crypto API
+ * * runtime test: Raw noise source data testing including SP800-90B compliant
+ * tests when enabling CONFIG_LRNG_HEALTH_TESTS
+ *
+ * Additional developer tests present with LRNG code:
+ * * SP800-90B APT and RCT test enforcement validation when enabling
+ * CONFIG_LRNG_APT_BROKEN or CONFIG_LRNG_RCT_BROKEN.
+ * * Collection of raw entropy from the interrupt noise source when enabling
+ * CONFIG_LRNG_TESTING and pulling the data from the kernel with the provided
+ * interface.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/module.h>
+#include <linux/lrng.h>
+#include <linux/slab.h>
+
+#include "lrng_chacha20.h"
+#include "lrng_internal.h"
+#include "lrng_lfsr.h"
+#include "lrng_sw_noise.h"
+
+#define LRNG_SELFTEST_PASSED 0
+#define LRNG_SEFLTEST_ERROR_TIME (1 << 0)
+#define LRNG_SEFLTEST_ERROR_LFSR (1 << 1)
+#define LRNG_SEFLTEST_ERROR_CHACHA20 (1 << 2)
+#define LRNG_SEFLTEST_ERROR_HASHDF (1 << 3)
+#define LRNG_SELFTEST_NOT_EXECUTED 0xffffffff
+
+static u32 lrng_time_selftest[LRNG_TIME_ARRAY_SIZE];
+
+static unsigned int lrng_selftest_status = LRNG_SELFTEST_NOT_EXECUTED;
+
+static inline void lrng_time_process_selftest_insert(u32 time)
+{
+ static u32 lrng_time_selftest_ptr = 0;
+ u32 ptr = lrng_time_selftest_ptr++ & LRNG_TIME_WORD_MASK;
+
+ lrng_time_selftest[lrng_time_idx2array(ptr)] |=
+ lrng_time_slot_val(time & LRNG_TIME_SLOTSIZE_MASK,
+ lrng_time_idx2slot(ptr));
+}
+
+static unsigned int lrng_time_process_selftest(void)
+{
+ u32 time;
+ u32 idx_zero_compare = (0 << 0) | (1 << 8) | (2 << 16) | (3 << 24);
+ u32 idx_one_compare = (4 << 0) | (5 << 8) | (6 << 16) | (7 << 24);
+ u32 idx_last_compare = ((LRNG_TIME_NUM_VALUES - 4) << 0) |
+ ((LRNG_TIME_NUM_VALUES - 3) << 8) |
+ ((LRNG_TIME_NUM_VALUES - 2) << 16) |
+ ((LRNG_TIME_NUM_VALUES - 1) << 24);
+
+ (void)idx_one_compare;
+
+ for (time = 0; time < LRNG_TIME_NUM_VALUES; time++)
+ lrng_time_process_selftest_insert(time);
+
+ if ((lrng_time_selftest[0] != idx_zero_compare) ||
+#if (LRNG_TIME_ARRAY_SIZE > 1)
+ (lrng_time_selftest[1] != idx_one_compare) ||
+#endif
+ (lrng_time_selftest[LRNG_TIME_ARRAY_SIZE - 1] != idx_last_compare))
+ {
+ pr_err("LRNG time array self-test FAILED\n");
+ return LRNG_SEFLTEST_ERROR_TIME;
+ }
+
+ return LRNG_SELFTEST_PASSED;
+}
+
+/*
+ * The test vectors are generated with the lfsr_testvector_generation tool
+ * provided as part of the test tool set of the LRNG.
+ */
+static unsigned int lrng_pool_lfsr_selftest(void)
+{
+ /*
+ * First, 67th and last entry of entropy pool.
+ *
+ * The 67th entry is picked because this one is the first to receive
+ * an entry. As we start with 1 to inject into the LFSR, the
+ * 67th entry should be equal to rol(1, 7) >> 3 considering that
+ * all other values of the LFSR are zero and the the twist value of 0
+ * is applied.
+ */
+ static u32 const lrng_lfsr_selftest_result[][3] = {
+ { 0xf56df24a, 0x00000010, 0x0e014939 },
+ { 0x4b130726, 0x00000010, 0x2802f509 },
+ { 0x87279152, 0x00000010, 0x00150000 },
+ { 0x0b67f997, 0x00000010, 0x00150000 },
+ { 0x4fea174f, 0x00000010, 0xcbf4a6ae },
+ { 0x77149108, 0x00000010, 0x77bfadf2 },
+ { 0x1e96037e, 0x00000010, 0x18017e79 },
+ { 0xc84acef2, 0x00000010, 0x6345f7a8 },
+ { 0x6a2eb6df, 0x00000010, 0x03950000 },
+ };
+ struct lrng_pool *lrng_pool, *lrng_pool_aligned;
+ u32 i, ret = LRNG_SELFTEST_PASSED;
+
+ BUILD_BUG_ON(ARRAY_SIZE(lrng_lfsr_selftest_result) <
+ CONFIG_LRNG_POOL_SIZE);
+
+ lrng_pool = kzalloc(sizeof(struct lrng_pool) + LRNG_KCAPI_ALIGN,
+ GFP_KERNEL);
+ if (!lrng_pool)
+ return LRNG_SEFLTEST_ERROR_LFSR;
+ lrng_pool_aligned = PTR_ALIGN(lrng_pool, sizeof(u32));
+
+ for (i = 1; i <= LRNG_POOL_SIZE; i++)
+ _lrng_pool_lfsr_u32(lrng_pool_aligned, i);
+
+ if ((atomic_read_u32(&lrng_pool_aligned->pool[0]) !=
+ lrng_lfsr_selftest_result[CONFIG_LRNG_POOL_SIZE][0]) ||
+ (atomic_read_u32(&lrng_pool_aligned->pool[67 &
+ (LRNG_POOL_SIZE - 1)]) !=
+ lrng_lfsr_selftest_result[CONFIG_LRNG_POOL_SIZE][1]) ||
+ (atomic_read_u32(&lrng_pool_aligned->pool[LRNG_POOL_SIZE - 1]) !=
+ lrng_lfsr_selftest_result[CONFIG_LRNG_POOL_SIZE][2])) {
+ pr_err("LRNG LFSR self-test FAILED\n");
+ ret = LRNG_SEFLTEST_ERROR_LFSR;
+ }
+
+ kfree(lrng_pool);
+ return ret;
+}
+
+/*
+ * The test vectors are generated with the hash_df_testvector_generation tool
+ * provided as part of the test tool set of the LRNG.
+ */
+static unsigned int lrng_hash_df_selftest(void)
+{
+ const struct lrng_crypto_cb *crypto_cb = &lrng_cc20_crypto_cb;
+
+ /*
+ * The size of 45 bytes is chosen arbitrarily. Yet, this size should
+ * ensure that we have at least two hash blocks plus some fraction
+ * of a hash block generated.
+ */
+ static u8 const lrng_hash_df_selftest_result[][45] = {
+ {
+ 0x3b, 0xbe, 0x7a, 0xbd, 0x2b, 0x16, 0x02, 0x4c,
+ 0xfc, 0xd3, 0x02, 0x15, 0xf0, 0x86, 0xd4, 0xdb,
+ 0x49, 0xec, 0x26, 0x53, 0xd6, 0xc9, 0x6d, 0xad,
+ 0x24, 0xca, 0x72, 0x89, 0x2c, 0xfa, 0x48, 0x18,
+ 0xf7, 0x47, 0xb5, 0x2f, 0x92, 0xa2, 0x1b, 0xd9,
+ 0x24, 0xa7, 0x2f, 0xa2, 0x0b,
+ }, {
+ 0xd2, 0xaa, 0xf9, 0x76, 0x26, 0xc6, 0x13, 0xea,
+ 0xb8, 0xde, 0xe6, 0x88, 0x8f, 0xc4, 0x7a, 0x7d,
+ 0x9c, 0xb4, 0x1b, 0xd1, 0xd1, 0x8a, 0x40, 0xc9,
+ 0xaa, 0x45, 0xa6, 0xb6, 0xb5, 0x6f, 0xf6, 0xbc,
+ 0xbb, 0x77, 0x37, 0xbc, 0x5a, 0x2d, 0xcc, 0x84,
+ 0x25, 0x68, 0x5e, 0xba, 0x16,
+ }, {
+ 0x58, 0x66, 0x82, 0x88, 0x29, 0x19, 0xa4, 0xbb,
+ 0x33, 0x42, 0xc9, 0x72, 0x0d, 0x68, 0x6e, 0xb9,
+ 0xc6, 0xe0, 0x7a, 0xf9, 0x20, 0xca, 0x6d, 0x18,
+ 0x35, 0xec, 0xfa, 0x9e, 0xf6, 0x3a, 0xa7, 0xb6,
+ 0x92, 0x7a, 0xe5, 0xcd, 0xc5, 0x13, 0x9f, 0x65,
+ 0x6a, 0xe1, 0xe4, 0x3f, 0xb9,
+ }, {
+ 0xdd, 0xf1, 0x34, 0xca, 0x08, 0xe3, 0xce, 0x8a,
+ 0x26, 0x6b, 0xce, 0x99, 0x8a, 0x84, 0xd2, 0x21,
+ 0x98, 0x10, 0x95, 0x5f, 0x9f, 0xc3, 0xf2, 0xe4,
+ 0x79, 0x75, 0xb5, 0x15, 0xa7, 0xa2, 0xf1, 0xc4,
+ 0xdc, 0x67, 0xcb, 0x67, 0x8c, 0xb2, 0x1b, 0xd5,
+ 0xd6, 0x8b, 0xc2, 0x34, 0xd6,
+ }, {
+ 0xc3, 0x16, 0x9d, 0xf0, 0x78, 0x15, 0xab, 0xf2,
+ 0x2f, 0xc9, 0x2e, 0xe1, 0xc6, 0x5e, 0xfa, 0x03,
+ 0xaf, 0xd4, 0xd5, 0x47, 0x2a, 0xe8, 0x06, 0xe8,
+ 0x7e, 0x0a, 0x71, 0xc7, 0x0d, 0x39, 0xb1, 0xa9,
+ 0x5a, 0x49, 0xee, 0x8b, 0x2f, 0xcd, 0xea, 0x96,
+ 0xcc, 0x08, 0x71, 0xef, 0x9c,
+ }, {
+ 0x1a, 0x3d, 0x70, 0x39, 0xc2, 0x02, 0x4d, 0x3a,
+ 0xaa, 0x14, 0x20, 0x88, 0x96, 0x4c, 0x7c, 0xe4,
+ 0xaa, 0x49, 0x89, 0x30, 0x50, 0x96, 0xb6, 0xa7,
+ 0x55, 0x0a, 0xf8, 0xd2, 0x4e, 0x83, 0x9d, 0x1f,
+ 0x56, 0x49, 0x13, 0xc6, 0x46, 0x55, 0x73, 0x0d,
+ 0x74, 0xcd, 0x81, 0xe0, 0x65,
+ }, {
+ 0x4b, 0xf6, 0x49, 0x89, 0x2a, 0x9f, 0x67, 0xd7,
+ 0xb8, 0x1d, 0xbb, 0x5d, 0xf0, 0x1b, 0x60, 0xb6,
+ 0xb7, 0xf3, 0x86, 0x6d, 0xe0, 0x04, 0xa1, 0xbc,
+ 0x3b, 0xb0, 0x10, 0x91, 0xe8, 0x22, 0x67, 0x5b,
+ 0xe8, 0xf0, 0x4f, 0x82, 0x70, 0xc7, 0xe1, 0xc8,
+ 0xd8, 0xad, 0x70, 0xcf, 0xf6,
+ }, {
+ 0x60, 0x1f, 0x71, 0x07, 0x92, 0xae, 0xa0, 0x24,
+ 0xb6, 0xa4, 0x10, 0x70, 0x1f, 0x94, 0x51, 0x9a,
+ 0x5a, 0x81, 0xc4, 0x46, 0x78, 0x56, 0x71, 0xdd,
+ 0x45, 0x63, 0x01, 0x34, 0x87, 0x79, 0xb4, 0xd5,
+ 0x91, 0x79, 0xb9, 0x93, 0x11, 0x44, 0x50, 0xad,
+ 0x64, 0x7e, 0x5c, 0xec, 0x16,
+ }, {
+ 0x49, 0x2f, 0xa0, 0x45, 0xf8, 0xb0, 0x80, 0x88,
+ 0x79, 0xeb, 0xb6, 0x82, 0x1c, 0xf3, 0x67, 0xc4,
+ 0x88, 0x88, 0xe9, 0x75, 0x20, 0x54, 0x78, 0xc6,
+ 0x5c, 0x59, 0xcf, 0xd9, 0x73, 0x12, 0x17, 0xf4,
+ 0x30, 0x9c, 0xb7, 0x21, 0x45, 0xe2, 0xb6, 0x0c,
+ 0x0c, 0xeb, 0x1b, 0xdc, 0xdc,
+ }
+ };
+ struct lrng_pool *lrng_pool, *lrng_pool_aligned;
+ u8 hash_df[sizeof(lrng_hash_df_selftest_result[0])]
+ __aligned(sizeof(u32));
+ u32 generated;
+ int ret = 0;
+
+ BUILD_BUG_ON(ARRAY_SIZE(lrng_hash_df_selftest_result) <
+ CONFIG_LRNG_POOL_SIZE);
+
+ lrng_pool = kzalloc(sizeof(struct lrng_pool) + LRNG_KCAPI_ALIGN,
+ GFP_KERNEL);
+ if (!lrng_pool)
+ return LRNG_SEFLTEST_ERROR_HASHDF;
+ lrng_pool_aligned = PTR_ALIGN(lrng_pool, sizeof(u32));
+
+ generated = __lrng_pool_hash_df(crypto_cb, NULL, lrng_pool_aligned,
+ hash_df, sizeof(hash_df) << 3);
+
+ if ((generated >> 3) != sizeof(hash_df) ||
+ memcmp(hash_df, lrng_hash_df_selftest_result[CONFIG_LRNG_POOL_SIZE],
+ sizeof(hash_df))) {
+ pr_err("LRNG Hash DF self-test FAILED\n");
+ ret = LRNG_SEFLTEST_ERROR_HASHDF;
+ }
+
+ kfree(lrng_pool);
+ return ret;
+}
+
+/*
+ * The test vectors were generated using the ChaCha20 DRNG from
+ * https://www.chronox.de/chacha20.html
+ */
+static unsigned int lrng_chacha20_drng_selftest(void)
+{
+ const struct lrng_crypto_cb *crypto_cb = &lrng_cc20_crypto_cb;
+ static u8 const seed[CHACHA_KEY_SIZE * 2] = {
+ 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+ 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
+ 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17,
+ 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
+ 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27,
+ 0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f,
+ 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37,
+ 0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f,
+ };
+ struct chacha20_block chacha20;
+ int ret;
+ u8 outbuf[CHACHA_KEY_SIZE * 2] __aligned(sizeof(u32));
+
+ /*
+ * Expected result when ChaCha20 DRNG state is zero:
+ * * constants are set to "expand 32-byte k"
+ * * remaining state is 0
+ * and pulling one half ChaCha20 DRNG block.
+ */
+ static u8 const expected_halfblock[CHACHA_KEY_SIZE] = {
+ 0x76, 0xb8, 0xe0, 0xad, 0xa0, 0xf1, 0x3d, 0x90,
+ 0x40, 0x5d, 0x6a, 0xe5, 0x53, 0x86, 0xbd, 0x28,
+ 0xbd, 0xd2, 0x19, 0xb8, 0xa0, 0x8d, 0xed, 0x1a,
+ 0xa8, 0x36, 0xef, 0xcc, 0x8b, 0x77, 0x0d, 0xc7 };
+
+ /*
+ * Expected result when ChaCha20 DRNG state is zero:
+ * * constants are set to "expand 32-byte k"
+ * * remaining state is 0
+ * followed by a reseed with two keyblocks
+ * 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+ * 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
+ * 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17,
+ * 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
+ * 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27,
+ * 0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f,
+ * 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37,
+ * 0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f
+ * and pulling one ChaCha20 DRNG block.
+ */
+ static u8 const expected_oneblock[CHACHA_KEY_SIZE * 2] = {
+ 0xf5, 0xb4, 0xb6, 0x5a, 0xec, 0xcd, 0x5a, 0x65,
+ 0x87, 0x56, 0xe3, 0x86, 0x51, 0x54, 0xfc, 0x90,
+ 0x56, 0xff, 0x5e, 0xae, 0x58, 0xf2, 0x01, 0x88,
+ 0xb1, 0x7e, 0xb8, 0x2e, 0x17, 0x9a, 0x27, 0xe6,
+ 0x86, 0xb3, 0xed, 0x33, 0xf7, 0xb9, 0x06, 0x05,
+ 0x8a, 0x2d, 0x1a, 0x93, 0xc9, 0x0b, 0x80, 0x04,
+ 0x03, 0xaa, 0x60, 0xaf, 0xd5, 0x36, 0x40, 0x11,
+ 0x67, 0x89, 0xb1, 0x66, 0xd5, 0x88, 0x62, 0x6d };
+
+ /*
+ * Expected result when ChaCha20 DRNG state is zero:
+ * * constants are set to "expand 32-byte k"
+ * * remaining state is 0
+ * followed by a reseed with one key block plus one byte
+ * 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+ * 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
+ * 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17,
+ * 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
+ * 0x20
+ * and pulling less than one ChaCha20 DRNG block.
+ */
+ static u8 const expected_block_nonalinged[CHACHA_KEY_SIZE + 1] = {
+ 0x3d, 0x13, 0x47, 0x1e, 0x7f, 0x7c, 0x99, 0x33,
+ 0xfc, 0x44, 0xa4, 0xdd, 0xf9, 0x3d, 0xe1, 0x9a,
+ 0xd4, 0xe8, 0x7a, 0x7d, 0x42, 0xac, 0xd1, 0xcd,
+ 0x10, 0x69, 0xe7, 0xbf, 0xd4, 0xfd, 0x69, 0x4b,
+ 0xa7 };
+
+ memset(&chacha20, 0, sizeof(chacha20));
+ lrng_cc20_init_rfc7539(&chacha20);
+
+ /* Generate with zero state */
+ ret = crypto_cb->lrng_drng_generate_helper(&chacha20, outbuf,
+ sizeof(expected_halfblock));
+ if (ret != sizeof(expected_halfblock))
+ goto err;
+ if (memcmp(outbuf, expected_halfblock, sizeof(expected_halfblock)))
+ goto err;
+
+ /* Clear state of DRNG */
+ memset(&chacha20.key.u[0], 0, 48);
+
+ /* Reseed with 2 key blocks */
+ ret = crypto_cb->lrng_drng_seed_helper(&chacha20, seed,
+ sizeof(expected_oneblock));
+ if (ret < 0)
+ goto err;
+ ret = crypto_cb->lrng_drng_generate_helper(&chacha20, outbuf,
+ sizeof(expected_oneblock));
+ if (ret != sizeof(expected_oneblock))
+ goto err;
+ if (memcmp(outbuf, expected_oneblock, sizeof(expected_oneblock)))
+ goto err;
+
+ /* Clear state of DRNG */
+ memset(&chacha20.key.u[0], 0, 48);
+
+ /* Reseed with 1 key block and one byte */
+ ret = crypto_cb->lrng_drng_seed_helper(&chacha20, seed,
+ sizeof(expected_block_nonalinged));
+ if (ret < 0)
+ goto err;
+ ret = crypto_cb->lrng_drng_generate_helper(&chacha20, outbuf,
+ sizeof(expected_block_nonalinged));
+ if (ret != sizeof(expected_block_nonalinged))
+ goto err;
+ if (memcmp(outbuf, expected_block_nonalinged,
+ sizeof(expected_block_nonalinged)))
+ goto err;
+
+ return LRNG_SELFTEST_PASSED;
+
+err:
+ pr_err("LRNG ChaCha20 DRNG self-test FAILED\n");
+ return LRNG_SEFLTEST_ERROR_CHACHA20;
+}
+
+static int lrng_selftest(void)
+{
+ unsigned int ret = lrng_time_process_selftest();
+
+ ret |= lrng_pool_lfsr_selftest();
+ ret |= lrng_chacha20_drng_selftest();
+ ret |= lrng_hash_df_selftest();
+
+ if (ret) {
+ if (IS_ENABLED(CONFIG_LRNG_SELFTEST_PANIC))
+ panic("LRNG self-tests failed: %u\n", ret);
+ } else {
+ pr_info("LRNG self-tests passed\n");
+ }
+
+ lrng_selftest_status = ret;
+
+ if (lrng_selftest_status)
+ return -EFAULT;
+ return 0;
+}
+
+#ifdef CONFIG_SYSFS
+/* Re-perform self-test when any value is written to the sysfs file. */
+static int lrng_selftest_sysfs_set(const char *val,
+ const struct kernel_param *kp)
+{
+ return lrng_selftest();
+}
+
+static const struct kernel_param_ops lrng_selftest_sysfs = {
+ .set = lrng_selftest_sysfs_set,
+ .get = param_get_uint,
+};
+module_param_cb(selftest_status, &lrng_selftest_sysfs, &lrng_selftest_status,
+ 0644);
+#endif /* CONFIG_SYSFS */
+
+static int __init lrng_selftest_init(void)
+{
+ return lrng_selftest();
+}
+
+module_init(lrng_selftest_init);
--
2.24.1
^ permalink raw reply related
* [RFC PATCH 00/14] pipe: Keyrings, Block and USB notifications [ver #3]
From: David Howells @ 2020-01-15 13:30 UTC (permalink / raw)
To: torvalds
Cc: dhowells, Greg Kroah-Hartman, Casey Schaufler, Stephen Smalley,
nicolas.dichtel, raven, Christian Brauner
Here's a set of patches to add a general notification queue concept and to
add event sources such as:
(1) Keys/keyrings, such as linking and unlinking keys and changing their
attributes.
(2) General device events (single common queue) including:
- Block layer events, such as device errors
- USB subsystem events, such as device attach/remove, device reset,
device errors.
I have patches for adding superblock and mount topology watches also,
though those are not in this set as there are other dependencies.
LSM hooks are included:
(1) A set of hooks are provided that allow an LSM to rule on whether or
not a watch may be set. Each of these hooks takes a different
"watched object" parameter, so they're not really shareable. The LSM
should use current's credentials. [Wanted by SELinux & Smack]
(2) A hook is provided to allow an LSM to rule on whether or not a
particular message may be posted to a particular queue. This is given
the credentials from the event generator (which may be the system) and
the watch setter. [Wanted by Smack]
I've provided SELinux and Smack with implementations of some of these hooks.
Why:
(1) Key/keyring notifications.
If you have your kerberos tickets in a file/directory, your gnome desktop
will monitor that using something like fanotify and tell you if your
credentials cache changes.
We also have the ability to cache your kerberos tickets in the session,
user or persistent keyring so that it isn't left around on disk across a
reboot or logout. Keyrings, however, cannot currently be monitored
asynchronously, so the desktop has to poll for it - not so good on a
laptop.
This source will allow the desktop to avoid the need to poll.
(2) USB notifications.
GregKH was looking for a way to do USB notifications as I was looking to
find additional sources to implement. I'm not sure how he wants to use
them, but I'll let him speak to that himself.
(3) Block notifications.
This one I was thinking that I could make something like ddrescue better
by letting it get notifications this way. This was a target of
convenience since I had a dodgy disk I was trying to rescue.
It could also potentially be used help systemd, say, detect broken
devices and avoid trying to unmount them when trying to reboot the machine.
I can drop this for now if you prefer.
(4) Mount notifications.
This one is wanted to avoid repeated trawling of /proc/mounts or similar
to work out changes to the mount object attributes and mount topology.
I'm told that the proc file holding the namespace_sem is a point of
contention, especially as the process of generating the text descriptions
of the mounts/superblocks can be quite involved.
The notifications directly indicate the mounts involved in any particular
event and what the change was. You can poll /proc/mounts, but all you
know is that something changed; you don't know what and you don't know
how and reading that file may race with multiple changed being effected.
I pair this with a new fsinfo() system call that allows, amongst other
things, the ability to retrieve in one go an { id, change counter } tuple
from all the children of a specified mount, allowing buffer overruns to
be cleaned up quickly.
It's not just Red Hat that's potentially interested in this:
https://lore.kernel.org/linux-fsdevel/293c9bd3-f530-d75e-c353-ddeabac27cf6@6wind.com/
(5) Superblock notifications.
This one is provided to allow systemd or the desktop to more easily
detect events such as I/O errors and EDQUOT/ENOSPC.
Design decisions:
(1) The notification queue is built on top of a standard pipe. Messages
are effectively spliced in. The pipe is opened with a special flag:
pipe2(fds, O_NOTIFICATION_PIPE);
The special flag has the same value as O_EXCL (which doesn't seem like
it will ever be applicable in this context)[?]. It is given up front
to make it a lot easier to prohibit splice and co. from accessing the
pipe.
[?] Should this be done some other way? I'd rather not use up a new
O_* flag if I can avoid it - should I add a pipe3() system call
instead?
The pipe is then configured::
ioctl(fds[1], IOC_WATCH_QUEUE_SET_SIZE, queue_depth);
ioctl(fds[1], IOC_WATCH_QUEUE_SET_FILTER, &filter);
Messages are then read out of the pipe using read().
(2) It should be possible to allow write() to insert data into the
notification pipes too, but this is currently disabled as the kernel
has to be able to insert messages into the pipe *without* holding
pipe->mutex and the code to make this work needs careful auditing.
(3) sendfile(), splice() and vmsplice() are disabled on notification pipes
because of the pipe->mutex issue and also because they sometimes want
to revert what they just did - but one or more notification messages
might've been interleaved in the ring.
(4) The kernel inserts messages with the wait queue spinlock held. This
means that pipe_read() and pipe_write() have to take the spinlock to
update the queue pointers.
(5) Records in the buffer are binary, typed and have a length so that they
can be of varying size.
This allows multiple heterogeneous sources to share a common buffer;
there are 16 million types available, of which I've used just a few,
so there is scope for others to be used. Tags may be specified when a
watchpoint is created to help distinguish the sources.
(6) Records are filterable as types have up to 256 subtypes that can be
individually filtered. Other filtration is also available.
(7) Notification pipes don't interfere with each other; each may be bound
to a different set of watches. Any particular notification will be
copied to all the queues that are currently watching for it - and only
those that are watching for it.
(8) When recording a notification, the kernel will not sleep, but will
rather mark a queue as having lost a message if there's insufficient
space. read() will fabricate a loss notification message at an
appropriate point later.
(9) The notification pipe is created and then watchpoints are attached to
it, using one of:
keyctl_watch_key(KEY_SPEC_SESSION_KEYRING, fds[1], 0x01);
watch_devices(fds[1], 0x02, 0);
where in both cases, fd indicates the queue and the number after is a
tag between 0 and 255.
(10) Watches are removed if either the notification pipe is destroyed or
the watched object is destroyed. In the latter case, a message will
be generated indicating the enforced watch removal.
Things I want to avoid:
(1) Introducing features that make the core VFS dependent on the network
stack or networking namespaces (ie. usage of netlink).
(2) Dumping all this stuff into dmesg and having a daemon that sits there
parsing the output and distributing it as this then puts the
responsibility for security into userspace and makes handling
namespaces tricky. Further, dmesg might not exist or might be
inaccessible inside a container.
(3) Letting users see events they shouldn't be able to see.
Testing and manpages:
(*) The keyutils tree has a pipe-watch branch that has keyctl commands for
making use of notifications. Proposed manual pages can also be found
on this branch, though a couple of them really need to go to the main
manpages repository instead.
If the kernel supports the watching of keys, then running "make test"
on that branch will cause the testing infrastructure to spawn a
monitoring process on the side that monitors a notifications pipe for
all the key/keyring changes induced by the tests and they'll all be
checked off to make sure they happened.
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/keyutils.git/log/?h=pipe-watch
(*) A test program is provided (samples/watch_queue/watch_test) that can
be used to monitor for keyrings, some USB and some block device
events. Information on the notifications is simply logged to stdout.
The kernel patches can also be found here:
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=notifications-pipe-core
Changes:
ver #3:
(*) Rebase to after latest upstream pipe patches.
(*) Fix a missing ref get in add_watch_to_object().
ver #2:
(*) Declare O_NOTIFICATION_PIPE to use and switch it to be the same value
as O_EXCL rather then O_TMPFILE (the latter is a bit nasty in its
implementation).
ver #1:
(*) Build on top of standard pipes instead of having a driver.
David
---
David Howells (14):
uapi: General notification queue definitions
security: Add hooks to rule on setting a watch
security: Add a hook for the point of notification insertion
pipe: Add O_NOTIFICATION_PIPE
pipe: Add general notification queue support
keys: Add a notification facility
Add sample notification program
pipe: Allow buffers to be marked read-whole-or-error for notifications
pipe: Add notification lossage handling
Add a general, global device notification watch list
block: Add block layer notifications
usb: Add USB subsystem notifications
selinux: Implement the watch_key security hook
smack: Implement the watch_key and post_notification hooks
Documentation/security/keys/core.rst | 58 ++
Documentation/userspace-api/ioctl/ioctl-number.rst | 1
Documentation/watch_queue.rst | 385 ++++++++++++
arch/alpha/kernel/syscalls/syscall.tbl | 1
arch/arm/tools/syscall.tbl | 1
arch/arm64/include/asm/unistd.h | 2
arch/arm64/include/asm/unistd32.h | 2
arch/ia64/kernel/syscalls/syscall.tbl | 1
arch/m68k/kernel/syscalls/syscall.tbl | 1
arch/microblaze/kernel/syscalls/syscall.tbl | 1
arch/mips/kernel/syscalls/syscall_n32.tbl | 1
arch/mips/kernel/syscalls/syscall_n64.tbl | 1
arch/mips/kernel/syscalls/syscall_o32.tbl | 1
arch/parisc/kernel/syscalls/syscall.tbl | 1
arch/powerpc/kernel/syscalls/syscall.tbl | 1
arch/s390/kernel/syscalls/syscall.tbl | 1
arch/sh/kernel/syscalls/syscall.tbl | 1
arch/sparc/kernel/syscalls/syscall.tbl | 1
arch/x86/entry/syscalls/syscall_32.tbl | 1
arch/x86/entry/syscalls/syscall_64.tbl | 1
arch/xtensa/kernel/syscalls/syscall.tbl | 1
block/Kconfig | 9
block/blk-core.c | 29 +
drivers/base/Kconfig | 9
drivers/base/Makefile | 1
drivers/base/watch.c | 90 +++
drivers/usb/core/Kconfig | 9
drivers/usb/core/devio.c | 47 +
drivers/usb/core/hub.c | 4
fs/pipe.c | 242 +++++--
fs/splice.c | 12
include/linux/blkdev.h | 15
include/linux/device.h | 7
include/linux/key.h | 3
include/linux/lsm_audit.h | 1
include/linux/lsm_hooks.h | 38 +
include/linux/pipe_fs_i.h | 27 +
include/linux/security.h | 31 +
include/linux/syscalls.h | 1
include/linux/usb.h | 18 +
include/linux/watch_queue.h | 127 ++++
include/uapi/asm-generic/unistd.h | 4
include/uapi/linux/keyctl.h | 2
include/uapi/linux/watch_queue.h | 158 +++++
init/Kconfig | 12
kernel/Makefile | 1
kernel/sys_ni.c | 1
kernel/watch_queue.c | 659 ++++++++++++++++++++
samples/Kconfig | 6
samples/Makefile | 1
samples/watch_queue/Makefile | 7
samples/watch_queue/watch_test.c | 251 ++++++++
security/keys/Kconfig | 9
security/keys/compat.c | 3
security/keys/gc.c | 5
security/keys/internal.h | 30 +
security/keys/key.c | 38 +
security/keys/keyctl.c | 99 +++
security/keys/keyring.c | 20 -
security/keys/request_key.c | 4
security/security.c | 23 +
security/selinux/hooks.c | 14
security/smack/smack_lsm.c | 82 ++
63 files changed, 2506 insertions(+), 107 deletions(-)
create mode 100644 Documentation/watch_queue.rst
create mode 100644 drivers/base/watch.c
create mode 100644 include/linux/watch_queue.h
create mode 100644 include/uapi/linux/watch_queue.h
create mode 100644 kernel/watch_queue.c
create mode 100644 samples/watch_queue/Makefile
create mode 100644 samples/watch_queue/watch_test.c
^ permalink raw reply
* [RFC PATCH 01/14] uapi: General notification queue definitions [ver #3]
From: David Howells @ 2020-01-15 13:30 UTC (permalink / raw)
To: torvalds
Cc: dhowells, Greg Kroah-Hartman, Casey Schaufler, Stephen Smalley,
nicolas.dichtel, raven, Christian Brauner
In-Reply-To: <157909503552.20155.3030058841911628518.stgit@warthog.procyon.org.uk>
Add UAPI definitions for the general notification queue, including the
following pieces:
(*) struct watch_notification.
This is the metadata header for notification messages. It includes a
type and subtype that indicate the source of the message
(eg. WATCH_TYPE_MOUNT_NOTIFY) and the kind of the message
(eg. NOTIFY_MOUNT_NEW_MOUNT).
The header also contains an information field that conveys the
following information:
- WATCH_INFO_LENGTH. The size of the entry (entries are variable
length).
- WATCH_INFO_ID. The watch ID specified when the watchpoint was
set.
- WATCH_INFO_TYPE_INFO. (Sub)type-specific information.
- WATCH_INFO_FLAG_*. Flag bits overlain on the type-specific
information. For use by the type.
All the information in the header can be used in filtering messages at
the point of writing into the buffer.
(*) struct watch_notification_removal
This is an extended watch-removal notification record that includes an
'id' field that can indicate the identifier of the object being
removed if available (for instance, a keyring serial number).
Signed-off-by: David Howells <dhowells@redhat.com>
---
include/uapi/linux/watch_queue.h | 55 ++++++++++++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
create mode 100644 include/uapi/linux/watch_queue.h
diff --git a/include/uapi/linux/watch_queue.h b/include/uapi/linux/watch_queue.h
new file mode 100644
index 000000000000..5f3d21e8a34b
--- /dev/null
+++ b/include/uapi/linux/watch_queue.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _UAPI_LINUX_WATCH_QUEUE_H
+#define _UAPI_LINUX_WATCH_QUEUE_H
+
+#include <linux/types.h>
+
+enum watch_notification_type {
+ WATCH_TYPE_META = 0, /* Special record */
+ WATCH_TYPE__NR = 1
+};
+
+enum watch_meta_notification_subtype {
+ WATCH_META_REMOVAL_NOTIFICATION = 0, /* Watched object was removed */
+ WATCH_META_LOSS_NOTIFICATION = 1, /* Data loss occurred */
+};
+
+/*
+ * Notification record header. This is aligned to 64-bits so that subclasses
+ * can contain __u64 fields.
+ */
+struct watch_notification {
+ __u32 type:24; /* enum watch_notification_type */
+ __u32 subtype:8; /* Type-specific subtype (filterable) */
+ __u32 info;
+#define WATCH_INFO_LENGTH 0x0000007f /* Length of record */
+#define WATCH_INFO_LENGTH__SHIFT 0
+#define WATCH_INFO_ID 0x0000ff00 /* ID of watchpoint */
+#define WATCH_INFO_ID__SHIFT 8
+#define WATCH_INFO_TYPE_INFO 0xffff0000 /* Type-specific info */
+#define WATCH_INFO_TYPE_INFO__SHIFT 16
+#define WATCH_INFO_FLAG_0 0x00010000 /* Type-specific info, flag bit 0 */
+#define WATCH_INFO_FLAG_1 0x00020000 /* ... */
+#define WATCH_INFO_FLAG_2 0x00040000
+#define WATCH_INFO_FLAG_3 0x00080000
+#define WATCH_INFO_FLAG_4 0x00100000
+#define WATCH_INFO_FLAG_5 0x00200000
+#define WATCH_INFO_FLAG_6 0x00400000
+#define WATCH_INFO_FLAG_7 0x00800000
+};
+
+
+/*
+ * Extended watch removal notification. This is used optionally if the type
+ * wants to indicate an identifier for the object being watched, if there is
+ * such. This can be distinguished by the length.
+ *
+ * type -> WATCH_TYPE_META
+ * subtype -> WATCH_META_REMOVAL_NOTIFICATION
+ */
+struct watch_notification_removal {
+ struct watch_notification watch;
+ __u64 id; /* Type-dependent identifier */
+};
+
+#endif /* _UAPI_LINUX_WATCH_QUEUE_H */
^ permalink raw reply related
* [RFC PATCH 02/14] security: Add hooks to rule on setting a watch [ver #3]
From: David Howells @ 2020-01-15 13:30 UTC (permalink / raw)
To: torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Greg Kroah-Hartman,
Casey Schaufler, Stephen Smalley,
nicolas.dichtel-pdR9zngts4EAvxtiuMwx3w,
raven-PKsaG3nR2I+sTnJN9+BGXg, Christian Brauner
In-Reply-To: <157909503552.20155.3030058841911628518.stgit-S6HVgzuS8uM4Awkfq6JHfwNdhmdF6hFW@public.gmane.org>
Add security hooks that will allow an LSM to rule on whether or not a watch
may be set. More than one hook is required as the watches watch different
types of object.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
cc: Casey Schaufler <casey-iSGtlc1asvQWG2LlvL+J4A@public.gmane.org>
cc: Stephen Smalley <sds-+05T5uksL2qpZYMLLGbcSA@public.gmane.org>
cc: linux-security-module-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
---
include/linux/lsm_hooks.h | 24 ++++++++++++++++++++++++
include/linux/security.h | 17 +++++++++++++++++
security/security.c | 14 ++++++++++++++
3 files changed, 55 insertions(+)
diff --git a/include/linux/lsm_hooks.h b/include/linux/lsm_hooks.h
index 20d8cf194fb7..79d7c73676d7 100644
--- a/include/linux/lsm_hooks.h
+++ b/include/linux/lsm_hooks.h
@@ -1416,6 +1416,18 @@
* @ctx is a pointer in which to place the allocated security context.
* @ctxlen points to the place to put the length of @ctx.
*
+ * Security hooks for the general notification queue:
+ *
+ * @watch_key:
+ * Check to see if a process is allowed to watch for event notifications
+ * from a key or keyring.
+ * @key: The key to watch.
+ *
+ * @watch_devices:
+ * Check to see if a process is allowed to watch for event notifications
+ * from devices (as a global set).
+ *
+ *
* Security hooks for using the eBPF maps and programs functionalities through
* eBPF syscalls.
*
@@ -1698,6 +1710,12 @@ union security_list_options {
int (*inode_notifysecctx)(struct inode *inode, void *ctx, u32 ctxlen);
int (*inode_setsecctx)(struct dentry *dentry, void *ctx, u32 ctxlen);
int (*inode_getsecctx)(struct inode *inode, void **ctx, u32 *ctxlen);
+#ifdef CONFIG_KEY_NOTIFICATIONS
+ int (*watch_key)(struct key *key);
+#endif
+#ifdef CONFIG_DEVICE_NOTIFICATIONS
+ int (*watch_devices)(void);
+#endif
#ifdef CONFIG_SECURITY_NETWORK
int (*unix_stream_connect)(struct sock *sock, struct sock *other,
@@ -1985,6 +2003,12 @@ struct security_hook_heads {
struct hlist_head inode_notifysecctx;
struct hlist_head inode_setsecctx;
struct hlist_head inode_getsecctx;
+#ifdef CONFIG_KEY_NOTIFICATIONS
+ struct hlist_head watch_key;
+#endif
+#ifdef CONFIG_DEVICE_NOTIFICATIONS
+ struct hlist_head watch_devices;
+#endif
#ifdef CONFIG_SECURITY_NETWORK
struct hlist_head unix_stream_connect;
struct hlist_head unix_may_send;
diff --git a/include/linux/security.h b/include/linux/security.h
index 3e8d4bacd59d..8f2fa100d128 100644
--- a/include/linux/security.h
+++ b/include/linux/security.h
@@ -1274,6 +1274,23 @@ static inline int security_locked_down(enum lockdown_reason what)
}
#endif /* CONFIG_SECURITY */
+#if defined(CONFIG_SECURITY) && defined(CONFIG_KEY_NOTIFICATIONS)
+int security_watch_key(struct key *key);
+#else
+static inline int security_watch_key(struct key *key)
+{
+ return 0;
+}
+#endif
+#if defined(CONFIG_SECURITY) && defined(CONFIG_DEVICE_NOTIFICATIONS)
+int security_watch_devices(void);
+#else
+static inline int security_watch_devices(void)
+{
+ return 0;
+}
+#endif
+
#ifdef CONFIG_SECURITY_NETWORK
int security_unix_stream_connect(struct sock *sock, struct sock *other, struct sock *newsk);
diff --git a/security/security.c b/security/security.c
index cd2d18d2d279..578863f230a6 100644
--- a/security/security.c
+++ b/security/security.c
@@ -1956,6 +1956,20 @@ int security_inode_getsecctx(struct inode *inode, void **ctx, u32 *ctxlen)
}
EXPORT_SYMBOL(security_inode_getsecctx);
+#ifdef CONFIG_KEY_NOTIFICATIONS
+int security_watch_key(struct key *key)
+{
+ return call_int_hook(watch_key, 0, key);
+}
+#endif
+
+#ifdef CONFIG_DEVICE_NOTIFICATIONS
+int security_watch_devices(void)
+{
+ return call_int_hook(watch_devices, 0);
+}
+#endif
+
#ifdef CONFIG_SECURITY_NETWORK
int security_unix_stream_connect(struct sock *sock, struct sock *other, struct sock *newsk)
^ permalink raw reply related
* [RFC PATCH 03/14] security: Add a hook for the point of notification insertion [ver #3]
From: David Howells @ 2020-01-15 13:31 UTC (permalink / raw)
To: torvalds
Cc: dhowells, Greg Kroah-Hartman, Casey Schaufler, Stephen Smalley,
nicolas.dichtel, raven, Christian Brauner
In-Reply-To: <157909503552.20155.3030058841911628518.stgit@warthog.procyon.org.uk>
Add a security hook that allows an LSM to rule on whether a notification
message is allowed to be inserted into a particular watch queue.
The hook is given the following information:
(1) The credentials of the triggerer (which may be init_cred for a system
notification, eg. a hardware error).
(2) The credentials of the whoever set the watch.
(3) The notification message.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Casey Schaufler <casey@schaufler-ca.com>
cc: Stephen Smalley <sds@tycho.nsa.gov>
cc: linux-security-module@vger.kernel.org
---
include/linux/lsm_hooks.h | 14 ++++++++++++++
include/linux/security.h | 14 ++++++++++++++
security/security.c | 9 +++++++++
3 files changed, 37 insertions(+)
diff --git a/include/linux/lsm_hooks.h b/include/linux/lsm_hooks.h
index 79d7c73676d7..16530255dc11 100644
--- a/include/linux/lsm_hooks.h
+++ b/include/linux/lsm_hooks.h
@@ -1427,6 +1427,12 @@
* Check to see if a process is allowed to watch for event notifications
* from devices (as a global set).
*
+ * @post_notification:
+ * Check to see if a watch notification can be posted to a particular
+ * queue.
+ * @w_cred: The credentials of the whoever set the watch.
+ * @cred: The event-triggerer's credentials
+ * @n: The notification being posted
*
* Security hooks for using the eBPF maps and programs functionalities through
* eBPF syscalls.
@@ -1716,6 +1722,11 @@ union security_list_options {
#ifdef CONFIG_DEVICE_NOTIFICATIONS
int (*watch_devices)(void);
#endif
+#ifdef CONFIG_WATCH_QUEUE
+ int (*post_notification)(const struct cred *w_cred,
+ const struct cred *cred,
+ struct watch_notification *n);
+#endif
#ifdef CONFIG_SECURITY_NETWORK
int (*unix_stream_connect)(struct sock *sock, struct sock *other,
@@ -2009,6 +2020,9 @@ struct security_hook_heads {
#ifdef CONFIG_DEVICE_NOTIFICATIONS
struct hlist_head watch_devices;
#endif
+#ifdef CONFIG_WATCH_QUEUE
+ struct hlist_head post_notification;
+#endif /* CONFIG_WATCH_QUEUE */
#ifdef CONFIG_SECURITY_NETWORK
struct hlist_head unix_stream_connect;
struct hlist_head unix_may_send;
diff --git a/include/linux/security.h b/include/linux/security.h
index 8f2fa100d128..e03ee1164268 100644
--- a/include/linux/security.h
+++ b/include/linux/security.h
@@ -57,6 +57,8 @@ struct mm_struct;
struct fs_context;
struct fs_parameter;
enum fs_value_type;
+struct watch;
+struct watch_notification;
/* Default (no) options for the capable function */
#define CAP_OPT_NONE 0x0
@@ -1290,6 +1292,18 @@ static inline int security_watch_devices(void)
return 0;
}
#endif
+#if defined(CONFIG_SECURITY) && defined(CONFIG_WATCH_QUEUE)
+int security_post_notification(const struct cred *w_cred,
+ const struct cred *cred,
+ struct watch_notification *n);
+#else
+static inline int security_post_notification(const struct cred *w_cred,
+ const struct cred *cred,
+ struct watch_notification *n)
+{
+ return 0;
+}
+#endif
#ifdef CONFIG_SECURITY_NETWORK
diff --git a/security/security.c b/security/security.c
index 578863f230a6..9c1878a7fadf 100644
--- a/security/security.c
+++ b/security/security.c
@@ -1956,6 +1956,15 @@ int security_inode_getsecctx(struct inode *inode, void **ctx, u32 *ctxlen)
}
EXPORT_SYMBOL(security_inode_getsecctx);
+#ifdef CONFIG_WATCH_QUEUE
+int security_post_notification(const struct cred *w_cred,
+ const struct cred *cred,
+ struct watch_notification *n)
+{
+ return call_int_hook(post_notification, 0, w_cred, cred, n);
+}
+#endif /* CONFIG_WATCH_QUEUE */
+
#ifdef CONFIG_KEY_NOTIFICATIONS
int security_watch_key(struct key *key)
{
^ permalink raw reply related
* [RFC PATCH 04/14] pipe: Add O_NOTIFICATION_PIPE [ver #3]
From: David Howells @ 2020-01-15 13:31 UTC (permalink / raw)
To: torvalds
Cc: dhowells, Greg Kroah-Hartman, Casey Schaufler, Stephen Smalley,
nicolas.dichtel, raven, Christian Brauner
In-Reply-To: <157909503552.20155.3030058841911628518.stgit@warthog.procyon.org.uk>
Add an O_NOTIFICATION_PIPE flag that can be passed to pipe2() to indicate
that the pipe being created is going to be used for notifications. This
suppresses the use of splice(), vmsplice(), tee() and sendfile() on the
pipe as calling iov_iter_revert() on a pipe when a kernel notification
message has been inserted into the middle of a multi-buffer splice will be
messy.
The flag is given the same value as O_EXCL as it seems unlikely that
this flag will ever be applicable to pipes and I don't want to use up
another O_* bit unnecessarily. An alternative could be to add a pipe3()
system call.
Signed-off-by: David Howells <dhowells@redhat.com>
---
include/uapi/linux/watch_queue.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/include/uapi/linux/watch_queue.h b/include/uapi/linux/watch_queue.h
index 5f3d21e8a34b..9df72227f515 100644
--- a/include/uapi/linux/watch_queue.h
+++ b/include/uapi/linux/watch_queue.h
@@ -3,6 +3,9 @@
#define _UAPI_LINUX_WATCH_QUEUE_H
#include <linux/types.h>
+#include <linux/fcntl.h>
+
+#define O_NOTIFICATION_PIPE O_EXCL /* Parameter to pipe2() selecting notification pipe */
enum watch_notification_type {
WATCH_TYPE_META = 0, /* Special record */
^ permalink raw reply related
* [RFC PATCH 05/14] pipe: Add general notification queue support [ver #3]
From: David Howells @ 2020-01-15 13:31 UTC (permalink / raw)
To: torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Greg Kroah-Hartman,
Casey Schaufler, Stephen Smalley,
nicolas.dichtel-pdR9zngts4EAvxtiuMwx3w,
raven-PKsaG3nR2I+sTnJN9+BGXg, Christian Brauner
In-Reply-To: <157909503552.20155.3030058841911628518.stgit-S6HVgzuS8uM4Awkfq6JHfwNdhmdF6hFW@public.gmane.org>
Make it possible to have a general notification queue built on top of a
standard pipe. Notifications are 'spliced' into the pipe and then read
out. splice(), vmsplice() and sendfile() are forbidden on pipes used for
notifications as post_one_notification() cannot take pipe->mutex. This
means that notifications could be posted in between individual pipe
buffers, making iov_iter_revert() difficult to effect.
The way the notification queue is used is:
(1) An application opens a pipe with a special flag and indicates the
number of messages it wishes to be able to queue at once (this can
only be set once):
pipe2(fds, O_NOTIFICATION_PIPE);
ioctl(fds[0], IOC_WATCH_QUEUE_SET_SIZE, queue_depth);
(2) The application then uses poll() and read() as normal to extract data
from the pipe. read() will return multiple notifications if the
buffer is big enough, but it will not split a notification across
buffers - rather it will return a short read or EMSGSIZE.
Notification messages include a length in the header so that the
caller can split them up.
Each message has a header that describes it:
struct watch_notification {
__u32 type:24;
__u32 subtype:8;
__u32 info;
};
The type indicates the source (eg. mount tree changes, superblock events,
keyring changes, block layer events) and the subtype indicates the event
type (eg. mount, unmount; EIO, EDQUOT; link, unlink). The info field
indicates a number of things, including the entry length, an ID assigned to
a watchpoint contributing to this buffer and type-specific flags.
Supplementary data, such as the key ID that generated an event, can be
attached in additional slots. The maximum message size is 127 bytes.
Messages may not be padded or aligned, so there is no guarantee, for
example, that the notification type will be on a 4-byte bounary.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
Documentation/userspace-api/ioctl/ioctl-number.rst | 1
Documentation/watch_queue.rst | 354 +++++++++++
fs/pipe.c | 206 ++++--
fs/splice.c | 12
include/linux/pipe_fs_i.h | 19 +
include/linux/watch_queue.h | 127 ++++
include/uapi/linux/watch_queue.h | 20 +
init/Kconfig | 12
kernel/Makefile | 1
kernel/watch_queue.c | 657 ++++++++++++++++++++
10 files changed, 1333 insertions(+), 76 deletions(-)
create mode 100644 Documentation/watch_queue.rst
create mode 100644 include/linux/watch_queue.h
create mode 100644 kernel/watch_queue.c
diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
index 4ef86433bd67..dbdc1ccbe949 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -202,6 +202,7 @@ Code Seq# Include File Comments
'W' 00-1F linux/wanrouter.h conflict! (pre 3.9)
'W' 00-3F sound/asound.h conflict!
'W' 40-5F drivers/pci/switch/switchtec.c
+'W' 60-61 linux/watch_queue.h
'X' all fs/xfs/xfs_fs.h, conflict!
fs/xfs/linux-2.6/xfs_ioctl32.h,
include/linux/falloc.h,
diff --git a/Documentation/watch_queue.rst b/Documentation/watch_queue.rst
new file mode 100644
index 000000000000..d8f70282d247
--- /dev/null
+++ b/Documentation/watch_queue.rst
@@ -0,0 +1,354 @@
+==============================
+General notification mechanism
+==============================
+
+The general notification mechanism is built on top of the standard pipe driver
+whereby it effectively splices notification messages from the kernel into pipes
+opened by userspace. This can be used in conjunction with::
+
+ * Key/keyring notifications
+
+ * General device event notifications
+
+
+The notifications buffers can be enabled by:
+
+ "General setup"/"General notification queue"
+ (CONFIG_WATCH_QUEUE)
+
+This document has the following sections:
+
+.. contents:: :local:
+
+
+Overview
+========
+
+This facility appears as a pipe that is opened in a special mode. The pipe's
+internal ring buffer is used to hold messages that are generated by the kernel.
+These messages are then read out by read(). Splice and similar are disabled on
+such pipes due to them wanting to, under some circumstances, revert their
+additions to the ring - which might end up interleaved with notification
+messages.
+
+The owner of the pipe has to tell the kernel which sources it would like to
+watch through that pipe. Only sources that have been connected to a pipe will
+insert messages into it. Note that a source may be bound to multiple pipes and
+insert messages into all of them simultaneously.
+
+Filters may also be emplaced on a pipe so that certain source types and
+subevents can be ignored if they're not of interest.
+
+A message will be discarded if there isn't a slot available in the ring or if
+no preallocated message buffer is available. In both of these cases, read()
+will insert a WATCH_META_LOSS_NOTIFICATION message into the output buffer after
+the last message currently in the buffer has been read.
+
+Note that when producing a notification, the kernel does not wait for the
+consumers to collect it, but rather just continues on. This means that
+notifications can be generated whilst spinlocks are held and also protects the
+kernel from being held up indefinitely by a userspace malfunction.
+
+
+Message Structure
+=================
+
+Notification messages begin with a short header::
+
+ struct watch_notification {
+ __u32 type:24;
+ __u32 subtype:8;
+ __u32 info;
+ };
+
+"type" indicates the source of the notification record and "subtype" indicates
+the type of record from that source (see the Watch Sources section below). The
+type may also be "WATCH_TYPE_META". This is a special record type generated
+internally by the watch queue itself. There are two subtypes:
+
+ * WATCH_META_REMOVAL_NOTIFICATION
+ * WATCH_META_LOSS_NOTIFICATION
+
+The first indicates that an object on which a watch was installed was removed
+or destroyed and the second indicates that some messages have been lost.
+
+"info" indicates a bunch of things, including:
+
+ * The length of the message in bytes, including the header (mask with
+ WATCH_INFO_LENGTH and shift by WATCH_INFO_LENGTH__SHIFT). This indicates
+ the size of the record, which may be between 8 and 127 bytes.
+
+ * The watch ID (mask with WATCH_INFO_ID and shift by WATCH_INFO_ID__SHIFT).
+ This indicates that caller's ID of the watch, which may be between 0
+ and 255. Multiple watches may share a queue, and this provides a means to
+ distinguish them.
+
+ * A type-specific field (WATCH_INFO_TYPE_INFO). This is set by the
+ notification producer to indicate some meaning specific to the type and
+ subtype.
+
+Everything in info apart from the length can be used for filtering.
+
+The header can be followed by supplementary information. The format of this is
+at the discretion is defined by the type and subtype.
+
+
+Watch List (Notification Source) API
+====================================
+
+A "watch list" is a list of watchers that are subscribed to a source of
+notifications. A list may be attached to an object (say a key or a superblock)
+or may be global (say for device events). From a userspace perspective, a
+non-global watch list is typically referred to by reference to the object it
+belongs to (such as using KEYCTL_NOTIFY and giving it a key serial number to
+watch that specific key).
+
+To manage a watch list, the following functions are provided:
+
+ * ``void init_watch_list(struct watch_list *wlist,
+ void (*release_watch)(struct watch *wlist));``
+
+ Initialise a watch list. If ``release_watch`` is not NULL, then this
+ indicates a function that should be called when the watch_list object is
+ destroyed to discard any references the watch list holds on the watched
+ object.
+
+ * ``void remove_watch_list(struct watch_list *wlist);``
+
+ This removes all of the watches subscribed to a watch_list and frees them
+ and then destroys the watch_list object itself.
+
+
+Watch Queue (Notification Output) API
+=====================================
+
+A "watch queue" is the buffer allocated by an application that notification
+records will be written into. The workings of this are hidden entirely inside
+of the watch_queue device driver, but it is necessary to gain a reference to it
+to set a watch. These can be managed with:
+
+ * ``struct watch_queue *get_watch_queue(int fd);``
+
+ Since watch queues are indicated to the kernel by the fd of the pipe that
+ implements the buffer, userspace must hand that fd through a system call.
+ This can be used to look up an opaque pointer to the watch queue from the
+ system call.
+
+ * ``void put_watch_queue(struct watch_queue *wqueue);``
+
+ This discards the reference obtained from ``get_watch_queue()``.
+
+
+Watch Subscription API
+======================
+
+A "watch" is a subscription on a watch list, indicating the watch queue, and
+thus the buffer, into which notification records should be written. The watch
+queue object may also carry filtering rules for that object, as set by
+userspace. Some parts of the watch struct can be set by the driver::
+
+ struct watch {
+ union {
+ u32 info_id; /* ID to be OR'd in to info field */
+ ...
+ };
+ void *private; /* Private data for the watched object */
+ u64 id; /* Internal identifier */
+ ...
+ };
+
+The ``info_id`` value should be an 8-bit number obtained from userspace and
+shifted by WATCH_INFO_ID__SHIFT. This is OR'd into the WATCH_INFO_ID field of
+struct watch_notification::info when and if the notification is written into
+the associated watch queue buffer.
+
+The ``private`` field is the driver's data associated with the watch_list and
+is cleaned up by the ``watch_list::release_watch()`` method.
+
+The ``id`` field is the source's ID. Notifications that are posted with a
+different ID are ignored.
+
+The following functions are provided to manage watches:
+
+ * ``void init_watch(struct watch *watch, struct watch_queue *wqueue);``
+
+ Initialise a watch object, setting its pointer to the watch queue, using
+ appropriate barriering to avoid lockdep complaints.
+
+ * ``int add_watch_to_object(struct watch *watch, struct watch_list *wlist);``
+
+ Subscribe a watch to a watch list (notification source). The
+ driver-settable fields in the watch struct must have been set before this
+ is called.
+
+ * ``int remove_watch_from_object(struct watch_list *wlist,
+ struct watch_queue *wqueue,
+ u64 id, false);``
+
+ Remove a watch from a watch list, where the watch must match the specified
+ watch queue (``wqueue``) and object identifier (``id``). A notification
+ (``WATCH_META_REMOVAL_NOTIFICATION``) is sent to the watch queue to
+ indicate that the watch got removed.
+
+ * ``int remove_watch_from_object(struct watch_list *wlist, NULL, 0, true);``
+
+ Remove all the watches from a watch list. It is expected that this will be
+ called preparatory to destruction and that the watch list will be
+ inaccessible to new watches by this point. A notification
+ (``WATCH_META_REMOVAL_NOTIFICATION``) is sent to the watch queue of each
+ subscribed watch to indicate that the watch got removed.
+
+
+Notification Posting API
+========================
+
+To post a notification to watch list so that the subscribed watches can see it,
+the following function should be used::
+
+ void post_watch_notification(struct watch_list *wlist,
+ struct watch_notification *n,
+ const struct cred *cred,
+ u64 id);
+
+The notification should be preformatted and a pointer to the header (``n``)
+should be passed in. The notification may be larger than this and the size in
+units of buffer slots is noted in ``n->info & WATCH_INFO_LENGTH``.
+
+The ``cred`` struct indicates the credentials of the source (subject) and is
+passed to the LSMs, such as SELinux, to allow or suppress the recording of the
+note in each individual queue according to the credentials of that queue
+(object).
+
+The ``id`` is the ID of the source object (such as the serial number on a key).
+Only watches that have the same ID set in them will see this notification.
+
+
+Watch Sources
+=============
+
+Any particular buffer can be fed from multiple sources. Sources include:
+
+ * WATCH_TYPE_KEY_NOTIFY
+
+ Notifications of this type indicate changes to keys and keyrings, including
+ the changes of keyring contents or the attributes of keys.
+
+ See Documentation/security/keys/core.rst for more information.
+
+ * WATCH_TYPE_BLOCK_NOTIFY
+
+ Notifications of this type indicate block layer events, such as I/O errors
+ or temporary link loss. Watches of this type are set on a global queue.
+
+
+Event Filtering
+===============
+
+Once a watch queue has been created, a set of filters can be applied to limit
+the events that are received using::
+
+ struct watch_notification_filter filter = {
+ ...
+ };
+ ioctl(fd, IOC_WATCH_QUEUE_SET_FILTER, &filter)
+
+The filter description is a variable of type::
+
+ struct watch_notification_filter {
+ __u32 nr_filters;
+ __u32 __reserved;
+ struct watch_notification_type_filter filters[];
+ };
+
+Where "nr_filters" is the number of filters in filters[] and "__reserved"
+should be 0. The "filters" array has elements of the following type::
+
+ struct watch_notification_type_filter {
+ __u32 type;
+ __u32 info_filter;
+ __u32 info_mask;
+ __u32 subtype_filter[8];
+ };
+
+Where:
+
+ * ``type`` is the event type to filter for and should be something like
+ "WATCH_TYPE_KEY_NOTIFY"
+
+ * ``info_filter`` and ``info_mask`` act as a filter on the info field of the
+ notification record. The notification is only written into the buffer if::
+
+ (watch.info & info_mask) == info_filter
+
+ This could be used, for example, to ignore events that are not exactly on
+ the watched point in a mount tree.
+
+ * ``subtype_filter`` is a bitmask indicating the subtypes that are of
+ interest. Bit 0 of subtype_filter[0] corresponds to subtype 0, bit 1 to
+ subtype 1, and so on.
+
+If the argument to the ioctl() is NULL, then the filters will be removed and
+all events from the watched sources will come through.
+
+
+Userspace Code Example
+======================
+
+A buffer is created with something like the following::
+
+ pipe2(fds, O_TMPFILE);
+ ioctl(fds[1], IOC_WATCH_QUEUE_SET_SIZE, 256);
+
+It can then be set to receive keyring change notifications and device event
+notifications::
+
+ keyctl(KEYCTL_WATCH_KEY, KEY_SPEC_SESSION_KEYRING, fds[1], 0x01);
+ watch_devices(fds[1], 0x2);
+
+The notifications can then be consumed by something like the following::
+
+ static void consumer(int rfd, struct watch_queue_buffer *buf)
+ {
+ unsigned char buffer[128];
+ ssize_t buf_len;
+
+ while (buf_len = read(rfd, buffer, sizeof(buffer)),
+ buf_len > 0
+ ) {
+ void *p = buffer;
+ void *end = buffer + buf_len;
+ while (p < end) {
+ union {
+ struct watch_notification n;
+ unsigned char buf1[128];
+ } n;
+ size_t largest, len;
+
+ largest = end - p;
+ if (largest > 128)
+ largest = 128;
+ memcpy(&n, p, largest);
+
+ len = (n->info & WATCH_INFO_LENGTH) >>
+ WATCH_INFO_LENGTH__SHIFT;
+ if (len == 0 || len > largest)
+ return;
+
+ switch (n.n.type) {
+ case WATCH_TYPE_META:
+ got_meta(&n.n);
+ case WATCH_TYPE_KEY_NOTIFY:
+ saw_key_change(&n.n);
+ break;
+ case WATCH_TYPE_BLOCK_NOTIFY:
+ saw_block_event(&n.n);
+ break;
+ case WATCH_TYPE_USB_NOTIFY:
+ saw_usb_event(&n.n);
+ break;
+ }
+
+ p += len;
+ }
+ }
+ }
diff --git a/fs/pipe.c b/fs/pipe.c
index 57502c3c0fba..5352c07be47f 100644
--- a/fs/pipe.c
+++ b/fs/pipe.c
@@ -24,6 +24,7 @@
#include <linux/syscalls.h>
#include <linux/fcntl.h>
#include <linux/memcontrol.h>
+#include <linux/watch_queue.h>
#include <linux/uaccess.h>
#include <asm/ioctls.h>
@@ -450,6 +451,13 @@ pipe_write(struct kiocb *iocb, struct iov_iter *from)
goto out;
}
+#ifdef CONFIG_WATCH_QUEUE
+ if (pipe->watch_queue) {
+ ret = -EXDEV;
+ goto out;
+ }
+#endif
+
/*
* Only wake up if the pipe started out empty, since
* otherwise there should be no readers waiting.
@@ -614,22 +622,37 @@ static long pipe_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
int count, head, tail, mask;
switch (cmd) {
- case FIONREAD:
- __pipe_lock(pipe);
- count = 0;
- head = pipe->head;
- tail = pipe->tail;
- mask = pipe->ring_size - 1;
+ case FIONREAD:
+ __pipe_lock(pipe);
+ count = 0;
+ head = pipe->head;
+ tail = pipe->tail;
+ mask = pipe->ring_size - 1;
- while (tail != head) {
- count += pipe->bufs[tail & mask].len;
- tail++;
- }
- __pipe_unlock(pipe);
+ while (tail != head) {
+ count += pipe->bufs[tail & mask].len;
+ tail++;
+ }
+ __pipe_unlock(pipe);
- return put_user(count, (int __user *)arg);
- default:
- return -ENOIOCTLCMD;
+ return put_user(count, (int __user *)arg);
+
+#ifdef CONFIG_WATCH_QUEUE
+ case IOC_WATCH_QUEUE_SET_SIZE: {
+ int ret;
+ __pipe_lock(pipe);
+ ret = watch_queue_set_size(pipe, arg);
+ __pipe_unlock(pipe);
+ return ret;
+ }
+
+ case IOC_WATCH_QUEUE_SET_FILTER:
+ return watch_queue_set_filter(
+ pipe, (struct watch_notification_filter __user *)arg);
+#endif
+
+ default:
+ return -ENOIOCTLCMD;
}
}
@@ -735,27 +758,27 @@ pipe_fasync(int fd, struct file *filp, int on)
return retval;
}
-static unsigned long account_pipe_buffers(struct user_struct *user,
- unsigned long old, unsigned long new)
+unsigned long account_pipe_buffers(struct user_struct *user,
+ unsigned long old, unsigned long new)
{
return atomic_long_add_return(new - old, &user->pipe_bufs);
}
-static bool too_many_pipe_buffers_soft(unsigned long user_bufs)
+bool too_many_pipe_buffers_soft(unsigned long user_bufs)
{
unsigned long soft_limit = READ_ONCE(pipe_user_pages_soft);
return soft_limit && user_bufs > soft_limit;
}
-static bool too_many_pipe_buffers_hard(unsigned long user_bufs)
+bool too_many_pipe_buffers_hard(unsigned long user_bufs)
{
unsigned long hard_limit = READ_ONCE(pipe_user_pages_hard);
return hard_limit && user_bufs > hard_limit;
}
-static bool is_unprivileged_user(void)
+bool pipe_is_unprivileged_user(void)
{
return !capable(CAP_SYS_RESOURCE) && !capable(CAP_SYS_ADMIN);
}
@@ -777,12 +800,12 @@ struct pipe_inode_info *alloc_pipe_info(void)
user_bufs = account_pipe_buffers(user, 0, pipe_bufs);
- if (too_many_pipe_buffers_soft(user_bufs) && is_unprivileged_user()) {
+ if (too_many_pipe_buffers_soft(user_bufs) && pipe_is_unprivileged_user()) {
user_bufs = account_pipe_buffers(user, pipe_bufs, 1);
pipe_bufs = 1;
}
- if (too_many_pipe_buffers_hard(user_bufs) && is_unprivileged_user())
+ if (too_many_pipe_buffers_hard(user_bufs) && pipe_is_unprivileged_user())
goto out_revert_acct;
pipe->bufs = kcalloc(pipe_bufs, sizeof(struct pipe_buffer),
@@ -793,6 +816,7 @@ struct pipe_inode_info *alloc_pipe_info(void)
pipe->r_counter = pipe->w_counter = 1;
pipe->max_usage = pipe_bufs;
pipe->ring_size = pipe_bufs;
+ pipe->nr_accounted = pipe_bufs;
pipe->user = user;
mutex_init(&pipe->mutex);
return pipe;
@@ -810,7 +834,14 @@ void free_pipe_info(struct pipe_inode_info *pipe)
{
int i;
- (void) account_pipe_buffers(pipe->user, pipe->ring_size, 0);
+#ifdef CONFIG_WATCH_QUEUE
+ if (pipe->watch_queue) {
+ watch_queue_clear(pipe->watch_queue);
+ put_watch_queue(pipe->watch_queue);
+ }
+#endif
+
+ (void) account_pipe_buffers(pipe->user, pipe->nr_accounted, 0);
free_uid(pipe->user);
for (i = 0; i < pipe->ring_size; i++) {
struct pipe_buffer *buf = pipe->bufs + i;
@@ -886,6 +917,17 @@ int create_pipe_files(struct file **res, int flags)
if (!inode)
return -ENFILE;
+ if (flags & O_NOTIFICATION_PIPE) {
+#ifdef CONFIG_WATCH_QUEUE
+ if (watch_queue_init(inode->i_pipe) < 0) {
+ iput(inode);
+ return -ENOMEM;
+ }
+#else
+ return -ENOPKG;
+#endif
+ }
+
f = alloc_file_pseudo(inode, pipe_mnt, "",
O_WRONLY | (flags & (O_NONBLOCK | O_DIRECT)),
&pipefifo_fops);
@@ -916,7 +958,7 @@ static int __do_pipe_flags(int *fd, struct file **files, int flags)
int error;
int fdw, fdr;
- if (flags & ~(O_CLOEXEC | O_NONBLOCK | O_DIRECT))
+ if (flags & ~(O_CLOEXEC | O_NONBLOCK | O_DIRECT | O_NOTIFICATION_PIPE))
return -EINVAL;
error = create_pipe_files(files, flags);
@@ -1163,42 +1205,12 @@ unsigned int round_pipe_size(unsigned long size)
}
/*
- * Allocate a new array of pipe buffers and copy the info over. Returns the
- * pipe size if successful, or return -ERROR on error.
+ * Resize the pipe ring to a number of slots.
*/
-static long pipe_set_size(struct pipe_inode_info *pipe, unsigned long arg)
+int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots)
{
struct pipe_buffer *bufs;
- unsigned int size, nr_slots, head, tail, mask, n;
- unsigned long user_bufs;
- long ret = 0;
-
- size = round_pipe_size(arg);
- nr_slots = size >> PAGE_SHIFT;
-
- if (!nr_slots)
- return -EINVAL;
-
- /*
- * If trying to increase the pipe capacity, check that an
- * unprivileged user is not trying to exceed various limits
- * (soft limit check here, hard limit check just below).
- * Decreasing the pipe capacity is always permitted, even
- * if the user is currently over a limit.
- */
- if (nr_slots > pipe->ring_size &&
- size > pipe_max_size && !capable(CAP_SYS_RESOURCE))
- return -EPERM;
-
- user_bufs = account_pipe_buffers(pipe->user, pipe->ring_size, nr_slots);
-
- if (nr_slots > pipe->ring_size &&
- (too_many_pipe_buffers_hard(user_bufs) ||
- too_many_pipe_buffers_soft(user_bufs)) &&
- is_unprivileged_user()) {
- ret = -EPERM;
- goto out_revert_acct;
- }
+ unsigned int head, tail, mask, n;
/*
* We can shrink the pipe, if arg is greater than the ring occupancy.
@@ -1210,17 +1222,13 @@ static long pipe_set_size(struct pipe_inode_info *pipe, unsigned long arg)
head = pipe->head;
tail = pipe->tail;
n = pipe_occupancy(pipe->head, pipe->tail);
- if (nr_slots < n) {
- ret = -EBUSY;
- goto out_revert_acct;
- }
+ if (nr_slots < n)
+ return -EBUSY;
bufs = kcalloc(nr_slots, sizeof(*bufs),
GFP_KERNEL_ACCOUNT | __GFP_NOWARN);
- if (unlikely(!bufs)) {
- ret = -ENOMEM;
- goto out_revert_acct;
- }
+ if (unlikely(!bufs))
+ return -ENOMEM;
/*
* The pipe array wraps around, so just start the new one at zero
@@ -1248,14 +1256,66 @@ static long pipe_set_size(struct pipe_inode_info *pipe, unsigned long arg)
kfree(pipe->bufs);
pipe->bufs = bufs;
pipe->ring_size = nr_slots;
- pipe->max_usage = nr_slots;
+ if (pipe->max_usage > nr_slots)
+ pipe->max_usage = nr_slots;
pipe->tail = tail;
pipe->head = head;
wake_up_interruptible_all(&pipe->wait);
+ return 0;
+}
+
+/*
+ * Allocate a new array of pipe buffers and copy the info over. Returns the
+ * pipe size if successful, or return -ERROR on error.
+ */
+static long pipe_set_size(struct pipe_inode_info *pipe, unsigned long arg)
+{
+ unsigned long user_bufs;
+ unsigned int nr_slots, size;
+ long ret = 0;
+
+#ifdef CONFIG_WATCH_QUEUE
+ if (pipe->watch_queue)
+ return -EBUSY;
+#endif
+
+ size = round_pipe_size(arg);
+ nr_slots = size >> PAGE_SHIFT;
+
+ if (!nr_slots)
+ return -EINVAL;
+
+ /*
+ * If trying to increase the pipe capacity, check that an
+ * unprivileged user is not trying to exceed various limits
+ * (soft limit check here, hard limit check just below).
+ * Decreasing the pipe capacity is always permitted, even
+ * if the user is currently over a limit.
+ */
+ if (nr_slots > pipe->max_usage &&
+ size > pipe_max_size && !capable(CAP_SYS_RESOURCE))
+ return -EPERM;
+
+ user_bufs = account_pipe_buffers(pipe->user, pipe->nr_accounted, nr_slots);
+
+ if (nr_slots > pipe->max_usage &&
+ (too_many_pipe_buffers_hard(user_bufs) ||
+ too_many_pipe_buffers_soft(user_bufs)) &&
+ pipe_is_unprivileged_user()) {
+ ret = -EPERM;
+ goto out_revert_acct;
+ }
+
+ ret = pipe_resize_ring(pipe, nr_slots);
+ if (ret < 0)
+ goto out_revert_acct;
+
+ pipe->max_usage = nr_slots;
+ pipe->nr_accounted = nr_slots;
return pipe->max_usage * PAGE_SIZE;
out_revert_acct:
- (void) account_pipe_buffers(pipe->user, nr_slots, pipe->ring_size);
+ (void) account_pipe_buffers(pipe->user, nr_slots, pipe->nr_accounted);
return ret;
}
@@ -1264,9 +1324,17 @@ static long pipe_set_size(struct pipe_inode_info *pipe, unsigned long arg)
* location, so checking ->i_pipe is not enough to verify that this is a
* pipe.
*/
-struct pipe_inode_info *get_pipe_info(struct file *file)
+struct pipe_inode_info *get_pipe_info(struct file *file, bool for_splice)
{
- return file->f_op == &pipefifo_fops ? file->private_data : NULL;
+ struct pipe_inode_info *pipe = file->private_data;
+
+ if (file->f_op != &pipefifo_fops || !pipe)
+ return NULL;
+#ifdef CONFIG_WATCH_QUEUE
+ if (for_splice && pipe->watch_queue)
+ return NULL;
+#endif
+ return pipe;
}
long pipe_fcntl(struct file *file, unsigned int cmd, unsigned long arg)
@@ -1274,7 +1342,7 @@ long pipe_fcntl(struct file *file, unsigned int cmd, unsigned long arg)
struct pipe_inode_info *pipe;
long ret;
- pipe = get_pipe_info(file);
+ pipe = get_pipe_info(file, false);
if (!pipe)
return -EBADF;
diff --git a/fs/splice.c b/fs/splice.c
index 3009652a41c8..44a999d30f3f 100644
--- a/fs/splice.c
+++ b/fs/splice.c
@@ -1118,8 +1118,8 @@ static long do_splice(struct file *in, loff_t __user *off_in,
loff_t offset;
long ret;
- ipipe = get_pipe_info(in);
- opipe = get_pipe_info(out);
+ ipipe = get_pipe_info(in, true);
+ opipe = get_pipe_info(out, true);
if (ipipe && opipe) {
if (off_in || off_out)
@@ -1278,7 +1278,7 @@ static int pipe_to_user(struct pipe_inode_info *pipe, struct pipe_buffer *buf,
static long vmsplice_to_user(struct file *file, struct iov_iter *iter,
unsigned int flags)
{
- struct pipe_inode_info *pipe = get_pipe_info(file);
+ struct pipe_inode_info *pipe = get_pipe_info(file, true);
struct splice_desc sd = {
.total_len = iov_iter_count(iter),
.flags = flags,
@@ -1313,7 +1313,7 @@ static long vmsplice_to_pipe(struct file *file, struct iov_iter *iter,
if (flags & SPLICE_F_GIFT)
buf_flag = PIPE_BUF_FLAG_GIFT;
- pipe = get_pipe_info(file);
+ pipe = get_pipe_info(file, true);
if (!pipe)
return -EBADF;
@@ -1766,8 +1766,8 @@ static int link_pipe(struct pipe_inode_info *ipipe,
static long do_tee(struct file *in, struct file *out, size_t len,
unsigned int flags)
{
- struct pipe_inode_info *ipipe = get_pipe_info(in);
- struct pipe_inode_info *opipe = get_pipe_info(out);
+ struct pipe_inode_info *ipipe = get_pipe_info(in, true);
+ struct pipe_inode_info *opipe = get_pipe_info(out, true);
int ret = -EINVAL;
/*
diff --git a/include/linux/pipe_fs_i.h b/include/linux/pipe_fs_i.h
index dbcfa6892384..cef70acd99bf 100644
--- a/include/linux/pipe_fs_i.h
+++ b/include/linux/pipe_fs_i.h
@@ -34,6 +34,7 @@ struct pipe_buffer {
* @tail: The point of buffer consumption
* @max_usage: The maximum number of slots that may be used in the ring
* @ring_size: total number of buffers (should be a power of 2)
+ * @nr_accounted: The amount this pipe accounts for in user->pipe_bufs
* @tmp_page: cached released page
* @readers: number of current readers of this pipe
* @writers: number of current writers of this pipe
@@ -44,6 +45,7 @@ struct pipe_buffer {
* @fasync_writers: writer side fasync
* @bufs: the circular array of pipe buffers
* @user: the user who created this pipe
+ * @watch_queue: If this pipe is a watch_queue, this is the stuff for that
**/
struct pipe_inode_info {
struct mutex mutex;
@@ -52,6 +54,7 @@ struct pipe_inode_info {
unsigned int tail;
unsigned int max_usage;
unsigned int ring_size;
+ unsigned int nr_accounted;
unsigned int readers;
unsigned int writers;
unsigned int files;
@@ -62,6 +65,9 @@ struct pipe_inode_info {
struct fasync_struct *fasync_writers;
struct pipe_buffer *bufs;
struct user_struct *user;
+#ifdef CONFIG_WATCH_QUEUE
+ struct watch_queue *watch_queue;
+#endif
};
/*
@@ -236,9 +242,20 @@ void pipe_buf_mark_unmergeable(struct pipe_buffer *buf);
extern const struct pipe_buf_operations nosteal_pipe_buf_ops;
+#ifdef CONFIG_WATCH_QUEUE
+unsigned long account_pipe_buffers(struct user_struct *user,
+ unsigned long old, unsigned long new);
+bool too_many_pipe_buffers_soft(unsigned long user_bufs);
+bool too_many_pipe_buffers_hard(unsigned long user_bufs);
+bool pipe_is_unprivileged_user(void);
+#endif
+
/* for F_SETPIPE_SZ and F_GETPIPE_SZ */
+#ifdef CONFIG_WATCH_QUEUE
+int pipe_resize_ring(struct pipe_inode_info *pipe, unsigned int nr_slots);
+#endif
long pipe_fcntl(struct file *, unsigned int, unsigned long arg);
-struct pipe_inode_info *get_pipe_info(struct file *file);
+struct pipe_inode_info *get_pipe_info(struct file *file, bool for_splice);
int create_pipe_files(struct file **, int);
unsigned int round_pipe_size(unsigned long size);
diff --git a/include/linux/watch_queue.h b/include/linux/watch_queue.h
new file mode 100644
index 000000000000..5e08db2adc31
--- /dev/null
+++ b/include/linux/watch_queue.h
@@ -0,0 +1,127 @@
+// SPDX-License-Identifier: GPL-2.0
+/* User-mappable watch queue
+ *
+ * Copyright (C) 2020 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org)
+ *
+ * See Documentation/watch_queue.rst
+ */
+
+#ifndef _LINUX_WATCH_QUEUE_H
+#define _LINUX_WATCH_QUEUE_H
+
+#include <uapi/linux/watch_queue.h>
+#include <linux/kref.h>
+#include <linux/rcupdate.h>
+
+#ifdef CONFIG_WATCH_QUEUE
+
+struct cred;
+
+struct watch_type_filter {
+ enum watch_notification_type type;
+ __u32 subtype_filter[1]; /* Bitmask of subtypes to filter on */
+ __u32 info_filter; /* Filter on watch_notification::info */
+ __u32 info_mask; /* Mask of relevant bits in info_filter */
+};
+
+struct watch_filter {
+ union {
+ struct rcu_head rcu;
+ unsigned long type_filter[2]; /* Bitmask of accepted types */
+ };
+ u32 nr_filters; /* Number of filters */
+ struct watch_type_filter filters[];
+};
+
+struct watch_queue {
+ struct rcu_head rcu;
+ struct watch_filter __rcu *filter;
+ struct pipe_inode_info *pipe; /* The pipe we're using as a buffer */
+ struct hlist_head watches; /* Contributory watches */
+ struct page **notes; /* Preallocated notifications */
+ unsigned long *notes_bitmap; /* Allocation bitmap for notes */
+ struct kref usage; /* Object usage count */
+ spinlock_t lock;
+ unsigned int nr_notes; /* Number of notes */
+ unsigned int nr_pages; /* Number of pages in notes[] */
+ bool defunct; /* T when queues closed */
+};
+
+/*
+ * Representation of a watch on an object.
+ */
+struct watch {
+ union {
+ struct rcu_head rcu;
+ u32 info_id; /* ID to be OR'd in to info field */
+ };
+ struct watch_queue __rcu *queue; /* Queue to post events to */
+ struct hlist_node queue_node; /* Link in queue->watches */
+ struct watch_list __rcu *watch_list;
+ struct hlist_node list_node; /* Link in watch_list->watchers */
+ const struct cred *cred; /* Creds of the owner of the watch */
+ void *private; /* Private data for the watched object */
+ u64 id; /* Internal identifier */
+ struct kref usage; /* Object usage count */
+};
+
+/*
+ * List of watches on an object.
+ */
+struct watch_list {
+ struct rcu_head rcu;
+ struct hlist_head watchers;
+ void (*release_watch)(struct watch *);
+ spinlock_t lock;
+};
+
+extern void __post_watch_notification(struct watch_list *,
+ struct watch_notification *,
+ const struct cred *,
+ u64);
+extern struct watch_queue *get_watch_queue(int);
+extern void put_watch_queue(struct watch_queue *);
+extern void init_watch(struct watch *, struct watch_queue *);
+extern int add_watch_to_object(struct watch *, struct watch_list *);
+extern int remove_watch_from_object(struct watch_list *, struct watch_queue *, u64, bool);
+extern long watch_queue_set_size(struct pipe_inode_info *, unsigned int);
+extern long watch_queue_set_filter(struct pipe_inode_info *,
+ struct watch_notification_filter __user *);
+extern int watch_queue_init(struct pipe_inode_info *);
+extern void watch_queue_clear(struct watch_queue *);
+
+static inline void init_watch_list(struct watch_list *wlist,
+ void (*release_watch)(struct watch *))
+{
+ INIT_HLIST_HEAD(&wlist->watchers);
+ spin_lock_init(&wlist->lock);
+ wlist->release_watch = release_watch;
+}
+
+static inline void post_watch_notification(struct watch_list *wlist,
+ struct watch_notification *n,
+ const struct cred *cred,
+ u64 id)
+{
+ if (unlikely(wlist))
+ __post_watch_notification(wlist, n, cred, id);
+}
+
+static inline void remove_watch_list(struct watch_list *wlist, u64 id)
+{
+ if (wlist) {
+ remove_watch_from_object(wlist, NULL, id, true);
+ kfree_rcu(wlist, rcu);
+ }
+}
+
+/**
+ * watch_sizeof - Calculate the information part of the size of a watch record,
+ * given the structure size.
+ */
+#define watch_sizeof(STRUCT) (sizeof(STRUCT) << WATCH_INFO_LENGTH__SHIFT)
+
+#endif
+
+#endif /* _LINUX_WATCH_QUEUE_H */
diff --git a/include/uapi/linux/watch_queue.h b/include/uapi/linux/watch_queue.h
index 9df72227f515..3a5790f1f05d 100644
--- a/include/uapi/linux/watch_queue.h
+++ b/include/uapi/linux/watch_queue.h
@@ -4,9 +4,13 @@
#include <linux/types.h>
#include <linux/fcntl.h>
+#include <linux/ioctl.h>
#define O_NOTIFICATION_PIPE O_EXCL /* Parameter to pipe2() selecting notification pipe */
+#define IOC_WATCH_QUEUE_SET_SIZE _IO('W', 0x60) /* Set the size in pages */
+#define IOC_WATCH_QUEUE_SET_FILTER _IO('W', 0x61) /* Set the filter */
+
enum watch_notification_type {
WATCH_TYPE_META = 0, /* Special record */
WATCH_TYPE__NR = 1
@@ -41,6 +45,22 @@ struct watch_notification {
#define WATCH_INFO_FLAG_7 0x00800000
};
+/*
+ * Notification filtering rules (IOC_WATCH_QUEUE_SET_FILTER).
+ */
+struct watch_notification_type_filter {
+ __u32 type; /* Type to apply filter to */
+ __u32 info_filter; /* Filter on watch_notification::info */
+ __u32 info_mask; /* Mask of relevant bits in info_filter */
+ __u32 subtype_filter[8]; /* Bitmask of subtypes to filter on */
+};
+
+struct watch_notification_filter {
+ __u32 nr_filters; /* Number of filters */
+ __u32 __reserved; /* Must be 0 */
+ struct watch_notification_type_filter filters[];
+};
+
/*
* Extended watch removal notification. This is used optionally if the type
diff --git a/init/Kconfig b/init/Kconfig
index a34064a031a5..53cdcc25c920 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -338,6 +338,18 @@ config POSIX_MQUEUE_SYSCTL
depends on SYSCTL
default y
+config WATCH_QUEUE
+ bool "General notification queue"
+ default n
+ help
+
+ This is a general notification queue for the kernel to pass events to
+ userspace by splicing them into pipes. It can be used in conjunction
+ with watches for key/keyring change notifications and device
+ notifications.
+
+ See Documentation/watch_queue.rst
+
config CROSS_MEMORY_ATTACH
bool "Enable process_vm_readv/writev syscalls"
depends on MMU
diff --git a/kernel/Makefile b/kernel/Makefile
index f2cc0d118a0b..06d5255784b4 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -114,6 +114,7 @@ obj-$(CONFIG_TORTURE_TEST) += torture.o
obj-$(CONFIG_HAS_IOMEM) += iomem.o
obj-$(CONFIG_RSEQ) += rseq.o
+obj-$(CONFIG_WATCH_QUEUE) += watch_queue.o
obj-$(CONFIG_SYSCTL_KUNIT_TEST) += sysctl-test.o
diff --git a/kernel/watch_queue.c b/kernel/watch_queue.c
new file mode 100644
index 000000000000..e2e3344a2586
--- /dev/null
+++ b/kernel/watch_queue.c
@@ -0,0 +1,657 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Watch queue and general notification mechanism, built on pipes
+ *
+ * Copyright (C) 2020 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org)
+ *
+ * See Documentation/watch_queue.rst
+ */
+
+#define pr_fmt(fmt) "watchq: " fmt
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/printk.h>
+#include <linux/miscdevice.h>
+#include <linux/fs.h>
+#include <linux/mm.h>
+#include <linux/pagemap.h>
+#include <linux/poll.h>
+#include <linux/uaccess.h>
+#include <linux/vmalloc.h>
+#include <linux/file.h>
+#include <linux/security.h>
+#include <linux/cred.h>
+#include <linux/sched/signal.h>
+#include <linux/watch_queue.h>
+#include <linux/pipe_fs_i.h>
+
+MODULE_DESCRIPTION("Watch queue");
+MODULE_AUTHOR("Red Hat, Inc.");
+MODULE_LICENSE("GPL");
+
+#define WATCH_QUEUE_NOTE_SIZE 128
+#define WATCH_QUEUE_NOTES_PER_PAGE (PAGE_SIZE / WATCH_QUEUE_NOTE_SIZE)
+
+static void watch_queue_pipe_buf_release(struct pipe_inode_info *pipe,
+ struct pipe_buffer *buf)
+{
+ struct watch_queue *wqueue = (struct watch_queue *)buf->private;
+ struct page *page;
+ unsigned int bit;
+
+ /* We need to work out which note within the page this refers to, but
+ * the note might have been maximum size, so merely ANDing the offset
+ * off doesn't work. OTOH, the note must've been more than zero size.
+ */
+ bit = buf->offset + buf->len;
+ if ((bit & (WATCH_QUEUE_NOTE_SIZE - 1)) == 0)
+ bit -= WATCH_QUEUE_NOTE_SIZE;
+ bit /= WATCH_QUEUE_NOTE_SIZE;
+
+ page = buf->page;
+ bit += page->index;
+
+ set_bit(bit, wqueue->notes_bitmap);
+}
+
+static int watch_queue_pipe_buf_steal(struct pipe_inode_info *pipe,
+ struct pipe_buffer *buf)
+{
+ return -1; /* No. */
+}
+
+/* New data written to a pipe may be appended to a buffer with this type. */
+static const struct pipe_buf_operations watch_queue_pipe_buf_ops = {
+ .confirm = generic_pipe_buf_confirm,
+ .release = watch_queue_pipe_buf_release,
+ .steal = watch_queue_pipe_buf_steal,
+ .get = generic_pipe_buf_get,
+};
+
+/*
+ * Post a notification to a watch queue.
+ */
+static bool post_one_notification(struct watch_queue *wqueue,
+ struct watch_notification *n)
+{
+ void *p;
+ struct pipe_inode_info *pipe = wqueue->pipe;
+ struct pipe_buffer *buf;
+ struct page *page;
+ unsigned int head, tail, mask, note, offset, len;
+ bool done = false;
+
+ if (!pipe)
+ return false;
+
+ spin_lock_irq(&pipe->wait.lock);
+
+ if (wqueue->defunct)
+ goto out;
+
+ mask = pipe->ring_size - 1;
+ head = pipe->head;
+ tail = pipe->tail;
+ if (pipe_full(head, tail, pipe->ring_size))
+ goto lost;
+
+ note = find_first_bit(wqueue->notes_bitmap, wqueue->nr_notes);
+ if (note >= wqueue->nr_notes)
+ goto lost;
+
+ page = wqueue->notes[note / WATCH_QUEUE_NOTES_PER_PAGE];
+ offset = note % WATCH_QUEUE_NOTES_PER_PAGE * WATCH_QUEUE_NOTE_SIZE;
+ get_page(page);
+ len = n->info & WATCH_INFO_LENGTH;
+ p = kmap_atomic(page);
+ memcpy(p + offset, n, len);
+ kunmap_atomic(p);
+
+ buf = &pipe->bufs[head & mask];
+ buf->page = page;
+ buf->private = (unsigned long)wqueue;
+ buf->ops = &watch_queue_pipe_buf_ops;
+ buf->offset = offset;
+ buf->len = len;
+ buf->flags = 0;
+ pipe->head = head + 1;
+
+ if (!test_and_clear_bit(note, wqueue->notes_bitmap)) {
+ spin_unlock_irq(&pipe->wait.lock);
+ BUG();
+ }
+ wake_up_interruptible_sync_poll_locked(&pipe->wait, EPOLLIN | EPOLLRDNORM);
+ done = true;
+
+out:
+ spin_unlock_irq(&pipe->wait.lock);
+ if (done)
+ kill_fasync(&pipe->fasync_readers, SIGIO, POLL_IN);
+ return done;
+
+lost:
+ goto out;
+}
+
+/*
+ * Apply filter rules to a notification.
+ */
+static bool filter_watch_notification(const struct watch_filter *wf,
+ const struct watch_notification *n)
+{
+ const struct watch_type_filter *wt;
+ unsigned int st_bits = sizeof(wt->subtype_filter[0]) * 8;
+ unsigned int st_index = n->subtype / st_bits;
+ unsigned int st_bit = 1U << (n->subtype % st_bits);
+ int i;
+
+ if (!test_bit(n->type, wf->type_filter))
+ return false;
+
+ for (i = 0; i < wf->nr_filters; i++) {
+ wt = &wf->filters[i];
+ if (n->type == wt->type &&
+ (wt->subtype_filter[st_index] & st_bit) &&
+ (n->info & wt->info_mask) == wt->info_filter)
+ return true;
+ }
+
+ return false; /* If there is a filter, the default is to reject. */
+}
+
+/**
+ * __post_watch_notification - Post an event notification
+ * @wlist: The watch list to post the event to.
+ * @n: The notification record to post.
+ * @cred: The creds of the process that triggered the notification.
+ * @id: The ID to match on the watch.
+ *
+ * Post a notification of an event into a set of watch queues and let the users
+ * know.
+ *
+ * The size of the notification should be set in n->info & WATCH_INFO_LENGTH and
+ * should be in units of sizeof(*n).
+ */
+void __post_watch_notification(struct watch_list *wlist,
+ struct watch_notification *n,
+ const struct cred *cred,
+ u64 id)
+{
+ const struct watch_filter *wf;
+ struct watch_queue *wqueue;
+ struct watch *watch;
+
+ if (((n->info & WATCH_INFO_LENGTH) >> WATCH_INFO_LENGTH__SHIFT) == 0) {
+ WARN_ON(1);
+ return;
+ }
+
+ rcu_read_lock();
+
+ hlist_for_each_entry_rcu(watch, &wlist->watchers, list_node) {
+ if (watch->id != id)
+ continue;
+ n->info &= ~WATCH_INFO_ID;
+ n->info |= watch->info_id;
+
+ wqueue = rcu_dereference(watch->queue);
+ wf = rcu_dereference(wqueue->filter);
+ if (wf && !filter_watch_notification(wf, n))
+ continue;
+
+ if (security_post_notification(watch->cred, cred, n) < 0)
+ continue;
+
+ post_one_notification(wqueue, n);
+ }
+
+ rcu_read_unlock();
+}
+EXPORT_SYMBOL(__post_watch_notification);
+
+/*
+ * Allocate sufficient pages to preallocation for the requested number of
+ * notifications.
+ */
+long watch_queue_set_size(struct pipe_inode_info *pipe, unsigned int nr_notes)
+{
+ struct watch_queue *wqueue = pipe->watch_queue;
+ struct page **pages;
+ unsigned long *bitmap;
+ unsigned long user_bufs;
+ unsigned int bmsize;
+ int ret, i, nr_pages;
+
+ if (!wqueue)
+ return -ENODEV;
+ if (wqueue->notes)
+ return -EBUSY;
+
+ if (nr_notes < 1 ||
+ nr_notes > 512) /* TODO: choose a better hard limit */
+ return -EINVAL;
+
+ nr_pages = (nr_notes + WATCH_QUEUE_NOTES_PER_PAGE - 1);
+ nr_pages /= WATCH_QUEUE_NOTES_PER_PAGE;
+ user_bufs = account_pipe_buffers(pipe->user, pipe->nr_accounted, nr_pages);
+
+ if (nr_pages > pipe->max_usage &&
+ (too_many_pipe_buffers_hard(user_bufs) ||
+ too_many_pipe_buffers_soft(user_bufs)) &&
+ pipe_is_unprivileged_user()) {
+ ret = -EPERM;
+ goto error;
+ }
+
+ ret = pipe_resize_ring(pipe, nr_notes);
+ if (ret < 0)
+ goto error;
+
+ pages = kcalloc(sizeof(struct page *), nr_pages, GFP_KERNEL);
+ if (!pages)
+ goto error;
+
+ for (i = 0; i < nr_pages; i++) {
+ pages[i] = alloc_page(GFP_KERNEL);
+ if (!pages[i])
+ goto error_p;
+ pages[i]->index = i * WATCH_QUEUE_NOTES_PER_PAGE;
+ }
+
+ bmsize = (nr_notes + BITS_PER_LONG - 1) / BITS_PER_LONG;
+ bmsize *= sizeof(unsigned long);
+ bitmap = kmalloc(bmsize, GFP_KERNEL);
+ if (!bitmap)
+ goto error_p;
+
+ memset(bitmap, 0xff, bmsize);
+ wqueue->notes = pages;
+ wqueue->notes_bitmap = bitmap;
+ wqueue->nr_pages = nr_pages;
+ wqueue->nr_notes = nr_pages * WATCH_QUEUE_NOTES_PER_PAGE;
+ return 0;
+
+error_p:
+ for (i = 0; i < nr_pages; i++)
+ __free_page(pages[i]);
+ kfree(pages);
+error:
+ (void) account_pipe_buffers(pipe->user, nr_pages, pipe->nr_accounted);
+ return ret;
+}
+
+/*
+ * Set the filter on a watch queue.
+ */
+long watch_queue_set_filter(struct pipe_inode_info *pipe,
+ struct watch_notification_filter __user *_filter)
+{
+ struct watch_notification_type_filter *tf;
+ struct watch_notification_filter filter;
+ struct watch_type_filter *q;
+ struct watch_filter *wfilter;
+ struct watch_queue *wqueue = pipe->watch_queue;
+ int ret, nr_filter = 0, i;
+
+ if (!wqueue)
+ return -ENODEV;
+
+ if (!_filter) {
+ /* Remove the old filter */
+ wfilter = NULL;
+ goto set;
+ }
+
+ /* Grab the user's filter specification */
+ if (copy_from_user(&filter, _filter, sizeof(filter)) != 0)
+ return -EFAULT;
+ if (filter.nr_filters == 0 ||
+ filter.nr_filters > 16 ||
+ filter.__reserved != 0)
+ return -EINVAL;
+
+ tf = memdup_user(_filter->filters, filter.nr_filters * sizeof(*tf));
+ if (IS_ERR(tf))
+ return PTR_ERR(tf);
+
+ ret = -EINVAL;
+ for (i = 0; i < filter.nr_filters; i++) {
+ if ((tf[i].info_filter & ~tf[i].info_mask) ||
+ tf[i].info_mask & WATCH_INFO_LENGTH)
+ goto err_filter;
+ /* Ignore any unknown types */
+ if (tf[i].type >= sizeof(wfilter->type_filter) * 8)
+ continue;
+ nr_filter++;
+ }
+
+ /* Now we need to build the internal filter from only the relevant
+ * user-specified filters.
+ */
+ ret = -ENOMEM;
+ wfilter = kzalloc(struct_size(wfilter, filters, nr_filter), GFP_KERNEL);
+ if (!wfilter)
+ goto err_filter;
+ wfilter->nr_filters = nr_filter;
+
+ q = wfilter->filters;
+ for (i = 0; i < filter.nr_filters; i++) {
+ if (tf[i].type >= sizeof(wfilter->type_filter) * BITS_PER_LONG)
+ continue;
+
+ q->type = tf[i].type;
+ q->info_filter = tf[i].info_filter;
+ q->info_mask = tf[i].info_mask;
+ q->subtype_filter[0] = tf[i].subtype_filter[0];
+ __set_bit(q->type, wfilter->type_filter);
+ q++;
+ }
+
+ kfree(tf);
+set:
+ pipe_lock(pipe);
+ rcu_swap_protected(wqueue->filter, wfilter,
+ lockdep_is_held(&pipe->mutex));
+ pipe_unlock(pipe);
+ if (wfilter)
+ kfree_rcu(wfilter, rcu);
+ return 0;
+
+err_filter:
+ kfree(tf);
+ return ret;
+}
+
+static void __put_watch_queue(struct kref *kref)
+{
+ struct watch_queue *wqueue =
+ container_of(kref, struct watch_queue, usage);
+ struct watch_filter *wfilter;
+ int i;
+
+ for (i = 0; i < wqueue->nr_pages; i++)
+ __free_page(wqueue->notes[i]);
+
+ wfilter = rcu_access_pointer(wqueue->filter);
+ if (wfilter)
+ kfree_rcu(wfilter, rcu);
+ kfree_rcu(wqueue, rcu);
+}
+
+/**
+ * put_watch_queue - Dispose of a ref on a watchqueue.
+ * @wqueue: The watch queue to unref.
+ */
+void put_watch_queue(struct watch_queue *wqueue)
+{
+ kref_put(&wqueue->usage, __put_watch_queue);
+}
+EXPORT_SYMBOL(put_watch_queue);
+
+static void free_watch(struct rcu_head *rcu)
+{
+ struct watch *watch = container_of(rcu, struct watch, rcu);
+
+ put_watch_queue(rcu_access_pointer(watch->queue));
+ put_cred(watch->cred);
+}
+
+static void __put_watch(struct kref *kref)
+{
+ struct watch *watch = container_of(kref, struct watch, usage);
+
+ call_rcu(&watch->rcu, free_watch);
+}
+
+/*
+ * Discard a watch.
+ */
+static void put_watch(struct watch *watch)
+{
+ kref_put(&watch->usage, __put_watch);
+}
+
+/**
+ * init_watch_queue - Initialise a watch
+ * @watch: The watch to initialise.
+ * @wqueue: The queue to assign.
+ *
+ * Initialise a watch and set the watch queue.
+ */
+void init_watch(struct watch *watch, struct watch_queue *wqueue)
+{
+ kref_init(&watch->usage);
+ INIT_HLIST_NODE(&watch->list_node);
+ INIT_HLIST_NODE(&watch->queue_node);
+ rcu_assign_pointer(watch->queue, wqueue);
+}
+
+/**
+ * add_watch_to_object - Add a watch on an object to a watch list
+ * @watch: The watch to add
+ * @wlist: The watch list to add to
+ *
+ * @watch->queue must have been set to point to the queue to post notifications
+ * to and the watch list of the object to be watched. @watch->cred must also
+ * have been set to the appropriate credentials and a ref taken on them.
+ *
+ * The caller must pin the queue and the list both and must hold the list
+ * locked against racing watch additions/removals.
+ */
+int add_watch_to_object(struct watch *watch, struct watch_list *wlist)
+{
+ struct watch_queue *wqueue = rcu_access_pointer(watch->queue);
+ struct watch *w;
+
+ hlist_for_each_entry(w, &wlist->watchers, list_node) {
+ struct watch_queue *wq = rcu_access_pointer(w->queue);
+ if (wqueue == wq && watch->id == w->id)
+ return -EBUSY;
+ }
+
+ watch->cred = get_current_cred();
+ rcu_assign_pointer(watch->watch_list, wlist);
+
+ spin_lock_bh(&wqueue->lock);
+ kref_get(&wqueue->usage);
+ kref_get(&watch->usage);
+ hlist_add_head(&watch->queue_node, &wqueue->watches);
+ spin_unlock_bh(&wqueue->lock);
+
+ hlist_add_head(&watch->list_node, &wlist->watchers);
+ return 0;
+}
+EXPORT_SYMBOL(add_watch_to_object);
+
+/**
+ * remove_watch_from_object - Remove a watch or all watches from an object.
+ * @wlist: The watch list to remove from
+ * @wq: The watch queue of interest (ignored if @all is true)
+ * @id: The ID of the watch to remove (ignored if @all is true)
+ * @all: True to remove all objects
+ *
+ * Remove a specific watch or all watches from an object. A notification is
+ * sent to the watcher to tell them that this happened.
+ */
+int remove_watch_from_object(struct watch_list *wlist, struct watch_queue *wq,
+ u64 id, bool all)
+{
+ struct watch_notification_removal n;
+ struct watch_queue *wqueue;
+ struct watch *watch;
+ int ret = -EBADSLT;
+
+ rcu_read_lock();
+
+again:
+ spin_lock(&wlist->lock);
+ hlist_for_each_entry(watch, &wlist->watchers, list_node) {
+ if (all ||
+ (watch->id == id && rcu_access_pointer(watch->queue) == wq))
+ goto found;
+ }
+ spin_unlock(&wlist->lock);
+ goto out;
+
+found:
+ ret = 0;
+ hlist_del_init_rcu(&watch->list_node);
+ rcu_assign_pointer(watch->watch_list, NULL);
+ spin_unlock(&wlist->lock);
+
+ /* We now own the reference on watch that used to belong to wlist. */
+
+ n.watch.type = WATCH_TYPE_META;
+ n.watch.subtype = WATCH_META_REMOVAL_NOTIFICATION;
+ n.watch.info = watch->info_id | watch_sizeof(n.watch);
+ n.id = id;
+ if (id != 0)
+ n.watch.info = watch->info_id | watch_sizeof(n);
+
+ wqueue = rcu_dereference(watch->queue);
+
+ /* We don't need the watch list lock for the next bit as RCU is
+ * protecting *wqueue from deallocation.
+ */
+ if (wqueue) {
+ post_one_notification(wqueue, &n.watch);
+
+ spin_lock_bh(&wqueue->lock);
+
+ if (!hlist_unhashed(&watch->queue_node)) {
+ hlist_del_init_rcu(&watch->queue_node);
+ put_watch(watch);
+ }
+
+ spin_unlock_bh(&wqueue->lock);
+ }
+
+ if (wlist->release_watch) {
+ void (*release_watch)(struct watch *);
+
+ release_watch = wlist->release_watch;
+ rcu_read_unlock();
+ (*release_watch)(watch);
+ rcu_read_lock();
+ }
+ put_watch(watch);
+
+ if (all && !hlist_empty(&wlist->watchers))
+ goto again;
+out:
+ rcu_read_unlock();
+ return ret;
+}
+EXPORT_SYMBOL(remove_watch_from_object);
+
+/*
+ * Remove all the watches that are contributory to a queue. This has the
+ * potential to race with removal of the watches by the destruction of the
+ * objects being watched or with the distribution of notifications.
+ */
+void watch_queue_clear(struct watch_queue *wqueue)
+{
+ struct watch_list *wlist;
+ struct watch *watch;
+ bool release;
+
+ rcu_read_lock();
+ spin_lock_bh(&wqueue->lock);
+
+ /* Prevent new additions and prevent notifications from happening */
+ wqueue->defunct = true;
+
+ while (!hlist_empty(&wqueue->watches)) {
+ watch = hlist_entry(wqueue->watches.first, struct watch, queue_node);
+ hlist_del_init_rcu(&watch->queue_node);
+ /* We now own a ref on the watch. */
+ spin_unlock_bh(&wqueue->lock);
+
+ /* We can't do the next bit under the queue lock as we need to
+ * get the list lock - which would cause a deadlock if someone
+ * was removing from the opposite direction at the same time or
+ * posting a notification.
+ */
+ wlist = rcu_dereference(watch->watch_list);
+ if (wlist) {
+ void (*release_watch)(struct watch *);
+
+ spin_lock(&wlist->lock);
+
+ release = !hlist_unhashed(&watch->list_node);
+ if (release) {
+ hlist_del_init_rcu(&watch->list_node);
+ rcu_assign_pointer(watch->watch_list, NULL);
+
+ /* We now own a second ref on the watch. */
+ }
+
+ release_watch = wlist->release_watch;
+ spin_unlock(&wlist->lock);
+
+ if (release) {
+ if (release_watch) {
+ rcu_read_unlock();
+ /* This might need to call dput(), so
+ * we have to drop all the locks.
+ */
+ (*release_watch)(watch);
+ rcu_read_lock();
+ }
+ put_watch(watch);
+ }
+ }
+
+ put_watch(watch);
+ spin_lock_bh(&wqueue->lock);
+ }
+
+ spin_unlock_bh(&wqueue->lock);
+ rcu_read_unlock();
+}
+
+/**
+ * get_watch_queue - Get a watch queue from its file descriptor.
+ * @fd: The fd to query.
+ */
+struct watch_queue *get_watch_queue(int fd)
+{
+ struct pipe_inode_info *pipe;
+ struct watch_queue *wqueue = ERR_PTR(-EINVAL);
+ struct fd f;
+
+ f = fdget(fd);
+ if (f.file) {
+ pipe = get_pipe_info(f.file, false);
+ if (pipe && pipe->watch_queue) {
+ wqueue = pipe->watch_queue;
+ kref_get(&wqueue->usage);
+ }
+ fdput(f);
+ }
+
+ return wqueue;
+}
+EXPORT_SYMBOL(get_watch_queue);
+
+/*
+ * Initialise a watch queue
+ */
+int watch_queue_init(struct pipe_inode_info *pipe)
+{
+ struct watch_queue *wqueue;
+
+ wqueue = kzalloc(sizeof(*wqueue), GFP_KERNEL);
+ if (!wqueue)
+ return -ENOMEM;
+
+ wqueue->pipe = pipe;
+ kref_init(&wqueue->usage);
+ spin_lock_init(&wqueue->lock);
+ INIT_HLIST_HEAD(&wqueue->watches);
+
+ pipe->watch_queue = wqueue;
+ return 0;
+}
^ permalink raw reply related
* [RFC PATCH 06/14] keys: Add a notification facility [ver #3]
From: David Howells @ 2020-01-15 13:31 UTC (permalink / raw)
To: torvalds
Cc: dhowells, Greg Kroah-Hartman, Casey Schaufler, Stephen Smalley,
nicolas.dichtel, raven, Christian Brauner
In-Reply-To: <157909503552.20155.3030058841911628518.stgit@warthog.procyon.org.uk>
Add a key/keyring change notification facility whereby notifications about
changes in key and keyring content and attributes can be received.
Firstly, an event queue needs to be created:
pipe2(fds, O_NOTIFICATION_PIPE);
ioctl(fds[1], IOC_WATCH_QUEUE_SET_SIZE, 256);
then a notification can be set up to report notifications via that queue:
struct watch_notification_filter filter = {
.nr_filters = 1,
.filters = {
[0] = {
.type = WATCH_TYPE_KEY_NOTIFY,
.subtype_filter[0] = UINT_MAX,
},
},
};
ioctl(fds[1], IOC_WATCH_QUEUE_SET_FILTER, &filter);
keyctl_watch_key(KEY_SPEC_SESSION_KEYRING, fds[1], 0x01);
After that, records will be placed into the queue when events occur in
which keys are changed in some way. Records are of the following format:
struct key_notification {
struct watch_notification watch;
__u32 key_id;
__u32 aux;
} *n;
Where:
n->watch.type will be WATCH_TYPE_KEY_NOTIFY.
n->watch.subtype will indicate the type of event, such as
NOTIFY_KEY_REVOKED.
n->watch.info & WATCH_INFO_LENGTH will indicate the length of the
record.
n->watch.info & WATCH_INFO_ID will be the second argument to
keyctl_watch_key(), shifted.
n->key will be the ID of the affected key.
n->aux will hold subtype-dependent information, such as the key
being linked into the keyring specified by n->key in the case of
NOTIFY_KEY_LINKED.
Note that it is permissible for event records to be of variable length -
or, at least, the length may be dependent on the subtype. Note also that
the queue can be shared between multiple notifications of various types.
Signed-off-by: David Howells <dhowells@redhat.com>
---
Documentation/security/keys/core.rst | 58 ++++++++++++++++++++
include/linux/key.h | 3 +
include/uapi/linux/keyctl.h | 2 +
include/uapi/linux/watch_queue.h | 28 +++++++++-
security/keys/Kconfig | 9 +++
security/keys/compat.c | 3 +
security/keys/gc.c | 5 ++
security/keys/internal.h | 30 ++++++++++
security/keys/key.c | 38 ++++++++-----
security/keys/keyctl.c | 99 +++++++++++++++++++++++++++++++++-
security/keys/keyring.c | 20 ++++---
security/keys/request_key.c | 4 +
12 files changed, 271 insertions(+), 28 deletions(-)
diff --git a/Documentation/security/keys/core.rst b/Documentation/security/keys/core.rst
index d9b0b859018b..0154721b20b2 100644
--- a/Documentation/security/keys/core.rst
+++ b/Documentation/security/keys/core.rst
@@ -833,6 +833,7 @@ The keyctl syscall functions are:
A process must have search permission on the key for this function to be
successful.
+
* Compute a Diffie-Hellman shared secret or public key::
long keyctl(KEYCTL_DH_COMPUTE, struct keyctl_dh_params *params,
@@ -1026,6 +1027,63 @@ The keyctl syscall functions are:
written into the output buffer. Verification returns 0 on success.
+ * Watch a key or keyring for changes::
+
+ long keyctl(KEYCTL_WATCH_KEY, key_serial_t key, int queue_fd,
+ const struct watch_notification_filter *filter);
+
+ This will set or remove a watch for changes on the specified key or
+ keyring.
+
+ "key" is the ID of the key to be watched.
+
+ "queue_fd" is a file descriptor referring to an open "/dev/watch_queue"
+ which manages the buffer into which notifications will be delivered.
+
+ "filter" is either NULL to remove a watch or a filter specification to
+ indicate what events are required from the key.
+
+ See Documentation/watch_queue.rst for more information.
+
+ Note that only one watch may be emplaced for any particular { key,
+ queue_fd } combination.
+
+ Notification records look like::
+
+ struct key_notification {
+ struct watch_notification watch;
+ __u32 key_id;
+ __u32 aux;
+ };
+
+ In this, watch::type will be "WATCH_TYPE_KEY_NOTIFY" and subtype will be
+ one of::
+
+ NOTIFY_KEY_INSTANTIATED
+ NOTIFY_KEY_UPDATED
+ NOTIFY_KEY_LINKED
+ NOTIFY_KEY_UNLINKED
+ NOTIFY_KEY_CLEARED
+ NOTIFY_KEY_REVOKED
+ NOTIFY_KEY_INVALIDATED
+ NOTIFY_KEY_SETATTR
+
+ Where these indicate a key being instantiated/rejected, updated, a link
+ being made in a keyring, a link being removed from a keyring, a keyring
+ being cleared, a key being revoked, a key being invalidated or a key
+ having one of its attributes changed (user, group, perm, timeout,
+ restriction).
+
+ If a watched key is deleted, a basic watch_notification will be issued
+ with "type" set to WATCH_TYPE_META and "subtype" set to
+ watch_meta_removal_notification. The watchpoint ID will be set in the
+ "info" field.
+
+ This needs to be configured by enabling:
+
+ "Provide key/keyring change notifications" (KEY_NOTIFICATIONS)
+
+
Kernel Services
===============
diff --git a/include/linux/key.h b/include/linux/key.h
index 6cf8e71cf8b7..b99b40db08fc 100644
--- a/include/linux/key.h
+++ b/include/linux/key.h
@@ -176,6 +176,9 @@ struct key {
struct list_head graveyard_link;
struct rb_node serial_node;
};
+#ifdef CONFIG_KEY_NOTIFICATIONS
+ struct watch_list *watchers; /* Entities watching this key for changes */
+#endif
struct rw_semaphore sem; /* change vs change sem */
struct key_user *user; /* owner of this key */
void *security; /* security data for this key */
diff --git a/include/uapi/linux/keyctl.h b/include/uapi/linux/keyctl.h
index ed3d5893830d..4c8884eea808 100644
--- a/include/uapi/linux/keyctl.h
+++ b/include/uapi/linux/keyctl.h
@@ -69,6 +69,7 @@
#define KEYCTL_RESTRICT_KEYRING 29 /* Restrict keys allowed to link to a keyring */
#define KEYCTL_MOVE 30 /* Move keys between keyrings */
#define KEYCTL_CAPABILITIES 31 /* Find capabilities of keyrings subsystem */
+#define KEYCTL_WATCH_KEY 32 /* Watch a key or ring of keys for changes */
/* keyctl structures */
struct keyctl_dh_params {
@@ -130,5 +131,6 @@ struct keyctl_pkey_params {
#define KEYCTL_CAPS0_MOVE 0x80 /* KEYCTL_MOVE supported */
#define KEYCTL_CAPS1_NS_KEYRING_NAME 0x01 /* Keyring names are per-user_namespace */
#define KEYCTL_CAPS1_NS_KEY_TAG 0x02 /* Key indexing can include a namespace tag */
+#define KEYCTL_CAPS1_NOTIFICATIONS 0x04 /* Keys generate watchable notifications */
#endif /* _LINUX_KEYCTL_H */
diff --git a/include/uapi/linux/watch_queue.h b/include/uapi/linux/watch_queue.h
index 3a5790f1f05d..c3d8320b5d3a 100644
--- a/include/uapi/linux/watch_queue.h
+++ b/include/uapi/linux/watch_queue.h
@@ -13,7 +13,8 @@
enum watch_notification_type {
WATCH_TYPE_META = 0, /* Special record */
- WATCH_TYPE__NR = 1
+ WATCH_TYPE_KEY_NOTIFY = 1, /* Key change event notification */
+ WATCH_TYPE__NR = 2
};
enum watch_meta_notification_subtype {
@@ -75,4 +76,29 @@ struct watch_notification_removal {
__u64 id; /* Type-dependent identifier */
};
+/*
+ * Type of key/keyring change notification.
+ */
+enum key_notification_subtype {
+ NOTIFY_KEY_INSTANTIATED = 0, /* Key was instantiated (aux is error code) */
+ NOTIFY_KEY_UPDATED = 1, /* Key was updated */
+ NOTIFY_KEY_LINKED = 2, /* Key (aux) was added to watched keyring */
+ NOTIFY_KEY_UNLINKED = 3, /* Key (aux) was removed from watched keyring */
+ NOTIFY_KEY_CLEARED = 4, /* Keyring was cleared */
+ NOTIFY_KEY_REVOKED = 5, /* Key was revoked */
+ NOTIFY_KEY_INVALIDATED = 6, /* Key was invalidated */
+ NOTIFY_KEY_SETATTR = 7, /* Key's attributes got changed */
+};
+
+/*
+ * Key/keyring notification record.
+ * - watch.type = WATCH_TYPE_KEY_NOTIFY
+ * - watch.subtype = enum key_notification_type
+ */
+struct key_notification {
+ struct watch_notification watch;
+ __u32 key_id; /* The key/keyring affected */
+ __u32 aux; /* Per-type auxiliary data */
+};
+
#endif /* _UAPI_LINUX_WATCH_QUEUE_H */
diff --git a/security/keys/Kconfig b/security/keys/Kconfig
index 47c041563d41..d4dc5ea208af 100644
--- a/security/keys/Kconfig
+++ b/security/keys/Kconfig
@@ -116,3 +116,12 @@ config KEY_DH_OPERATIONS
in the kernel.
If you are unsure as to whether this is required, answer N.
+
+config KEY_NOTIFICATIONS
+ bool "Provide key/keyring change notifications"
+ depends on KEYS && WATCH_QUEUE
+ help
+ This option provides support for getting change notifications on keys
+ and keyrings on which the caller has View permission. This makes use
+ of the /dev/watch_queue misc device to handle the notification
+ buffer and provides KEYCTL_WATCH_KEY to enable/disable watches.
diff --git a/security/keys/compat.c b/security/keys/compat.c
index b975f8f11124..6ee9d8f6a4a5 100644
--- a/security/keys/compat.c
+++ b/security/keys/compat.c
@@ -156,6 +156,9 @@ COMPAT_SYSCALL_DEFINE5(keyctl, u32, option,
case KEYCTL_CAPABILITIES:
return keyctl_capabilities(compat_ptr(arg2), arg3);
+ case KEYCTL_WATCH_KEY:
+ return keyctl_watch_key(arg2, arg3, arg4);
+
default:
return -EOPNOTSUPP;
}
diff --git a/security/keys/gc.c b/security/keys/gc.c
index 671dd730ecfc..3c90807476eb 100644
--- a/security/keys/gc.c
+++ b/security/keys/gc.c
@@ -131,6 +131,11 @@ static noinline void key_gc_unused_keys(struct list_head *keys)
kdebug("- %u", key->serial);
key_check(key);
+#ifdef CONFIG_KEY_NOTIFICATIONS
+ remove_watch_list(key->watchers, key->serial);
+ key->watchers = NULL;
+#endif
+
/* Throw away the key data if the key is instantiated */
if (state == KEY_IS_POSITIVE && key->type->destroy)
key->type->destroy(key);
diff --git a/security/keys/internal.h b/security/keys/internal.h
index ba3e2da14cef..2a230c665162 100644
--- a/security/keys/internal.h
+++ b/security/keys/internal.h
@@ -15,6 +15,7 @@
#include <linux/task_work.h>
#include <linux/keyctl.h>
#include <linux/refcount.h>
+#include <linux/watch_queue.h>
#include <linux/compat.h>
struct iovec;
@@ -97,7 +98,8 @@ extern int __key_link_begin(struct key *keyring,
const struct keyring_index_key *index_key,
struct assoc_array_edit **_edit);
extern int __key_link_check_live_key(struct key *keyring, struct key *key);
-extern void __key_link(struct key *key, struct assoc_array_edit **_edit);
+extern void __key_link(struct key *keyring, struct key *key,
+ struct assoc_array_edit **_edit);
extern void __key_link_end(struct key *keyring,
const struct keyring_index_key *index_key,
struct assoc_array_edit *edit);
@@ -181,6 +183,23 @@ extern int key_task_permission(const key_ref_t key_ref,
const struct cred *cred,
key_perm_t perm);
+static inline void notify_key(struct key *key,
+ enum key_notification_subtype subtype, u32 aux)
+{
+#ifdef CONFIG_KEY_NOTIFICATIONS
+ struct key_notification n = {
+ .watch.type = WATCH_TYPE_KEY_NOTIFY,
+ .watch.subtype = subtype,
+ .watch.info = watch_sizeof(n),
+ .key_id = key_serial(key),
+ .aux = aux,
+ };
+
+ post_watch_notification(key->watchers, &n.watch, current_cred(),
+ n.key_id);
+#endif
+}
+
/*
* Check to see whether permission is granted to use a key in the desired way.
*/
@@ -331,6 +350,15 @@ static inline long keyctl_pkey_e_d_s(int op,
extern long keyctl_capabilities(unsigned char __user *_buffer, size_t buflen);
+#ifdef CONFIG_KEY_NOTIFICATIONS
+extern long keyctl_watch_key(key_serial_t, int, int);
+#else
+static inline long keyctl_watch_key(key_serial_t key_id, int watch_fd, int watch_id)
+{
+ return -EOPNOTSUPP;
+}
+#endif
+
/*
* Debugging key validation
*/
diff --git a/security/keys/key.c b/security/keys/key.c
index 764f4c57913e..83e8d7c4bb6f 100644
--- a/security/keys/key.c
+++ b/security/keys/key.c
@@ -443,6 +443,7 @@ static int __key_instantiate_and_link(struct key *key,
/* mark the key as being instantiated */
atomic_inc(&key->user->nikeys);
mark_key_instantiated(key, 0);
+ notify_key(key, NOTIFY_KEY_INSTANTIATED, 0);
if (test_and_clear_bit(KEY_FLAG_USER_CONSTRUCT, &key->flags))
awaken = 1;
@@ -452,7 +453,7 @@ static int __key_instantiate_and_link(struct key *key,
if (test_bit(KEY_FLAG_KEEP, &keyring->flags))
set_bit(KEY_FLAG_KEEP, &key->flags);
- __key_link(key, _edit);
+ __key_link(keyring, key, _edit);
}
/* disable the authorisation key */
@@ -600,6 +601,7 @@ int key_reject_and_link(struct key *key,
/* mark the key as being negatively instantiated */
atomic_inc(&key->user->nikeys);
mark_key_instantiated(key, -error);
+ notify_key(key, NOTIFY_KEY_INSTANTIATED, -error);
key->expiry = ktime_get_real_seconds() + timeout;
key_schedule_gc(key->expiry + key_gc_delay);
@@ -610,7 +612,7 @@ int key_reject_and_link(struct key *key,
/* and link it into the destination keyring */
if (keyring && link_ret == 0)
- __key_link(key, &edit);
+ __key_link(keyring, key, &edit);
/* disable the authorisation key */
if (authkey)
@@ -763,9 +765,11 @@ static inline key_ref_t __key_update(key_ref_t key_ref,
down_write(&key->sem);
ret = key->type->update(key, prep);
- if (ret == 0)
+ if (ret == 0) {
/* Updating a negative key positively instantiates it */
mark_key_instantiated(key, 0);
+ notify_key(key, NOTIFY_KEY_UPDATED, 0);
+ }
up_write(&key->sem);
@@ -1013,9 +1017,11 @@ int key_update(key_ref_t key_ref, const void *payload, size_t plen)
down_write(&key->sem);
ret = key->type->update(key, &prep);
- if (ret == 0)
+ if (ret == 0) {
/* Updating a negative key positively instantiates it */
mark_key_instantiated(key, 0);
+ notify_key(key, NOTIFY_KEY_UPDATED, 0);
+ }
up_write(&key->sem);
@@ -1047,15 +1053,17 @@ void key_revoke(struct key *key)
* instantiated
*/
down_write_nested(&key->sem, 1);
- if (!test_and_set_bit(KEY_FLAG_REVOKED, &key->flags) &&
- key->type->revoke)
- key->type->revoke(key);
-
- /* set the death time to no more than the expiry time */
- time = ktime_get_real_seconds();
- if (key->revoked_at == 0 || key->revoked_at > time) {
- key->revoked_at = time;
- key_schedule_gc(key->revoked_at + key_gc_delay);
+ if (!test_and_set_bit(KEY_FLAG_REVOKED, &key->flags)) {
+ notify_key(key, NOTIFY_KEY_REVOKED, 0);
+ if (key->type->revoke)
+ key->type->revoke(key);
+
+ /* set the death time to no more than the expiry time */
+ time = ktime_get_real_seconds();
+ if (key->revoked_at == 0 || key->revoked_at > time) {
+ key->revoked_at = time;
+ key_schedule_gc(key->revoked_at + key_gc_delay);
+ }
}
up_write(&key->sem);
@@ -1077,8 +1085,10 @@ void key_invalidate(struct key *key)
if (!test_bit(KEY_FLAG_INVALIDATED, &key->flags)) {
down_write_nested(&key->sem, 1);
- if (!test_and_set_bit(KEY_FLAG_INVALIDATED, &key->flags))
+ if (!test_and_set_bit(KEY_FLAG_INVALIDATED, &key->flags)) {
+ notify_key(key, NOTIFY_KEY_INVALIDATED, 0);
key_schedule_gc_links();
+ }
up_write(&key->sem);
}
}
diff --git a/security/keys/keyctl.c b/security/keys/keyctl.c
index 9b898c969558..6610649514fb 100644
--- a/security/keys/keyctl.c
+++ b/security/keys/keyctl.c
@@ -37,7 +37,9 @@ static const unsigned char keyrings_capabilities[2] = {
KEYCTL_CAPS0_MOVE
),
[1] = (KEYCTL_CAPS1_NS_KEYRING_NAME |
- KEYCTL_CAPS1_NS_KEY_TAG),
+ KEYCTL_CAPS1_NS_KEY_TAG |
+ (IS_ENABLED(CONFIG_KEY_NOTIFICATIONS) ? KEYCTL_CAPS1_NOTIFICATIONS : 0)
+ ),
};
static int key_get_type_from_user(char *type,
@@ -970,6 +972,7 @@ long keyctl_chown_key(key_serial_t id, uid_t user, gid_t group)
if (group != (gid_t) -1)
key->gid = gid;
+ notify_key(key, NOTIFY_KEY_SETATTR, 0);
ret = 0;
error_put:
@@ -1020,6 +1023,7 @@ long keyctl_setperm_key(key_serial_t id, key_perm_t perm)
/* if we're not the sysadmin, we can only change a key that we own */
if (capable(CAP_SYS_ADMIN) || uid_eq(key->uid, current_fsuid())) {
key->perm = perm;
+ notify_key(key, NOTIFY_KEY_SETATTR, 0);
ret = 0;
}
@@ -1411,10 +1415,12 @@ long keyctl_set_timeout(key_serial_t id, unsigned timeout)
okay:
key = key_ref_to_ptr(key_ref);
ret = 0;
- if (test_bit(KEY_FLAG_KEEP, &key->flags))
+ if (test_bit(KEY_FLAG_KEEP, &key->flags)) {
ret = -EPERM;
- else
+ } else {
key_set_timeout(key, timeout);
+ notify_key(key, NOTIFY_KEY_SETATTR, 0);
+ }
key_put(key);
error:
@@ -1688,6 +1694,90 @@ long keyctl_restrict_keyring(key_serial_t id, const char __user *_type,
return ret;
}
+#ifdef CONFIG_KEY_NOTIFICATIONS
+/*
+ * Watch for changes to a key.
+ *
+ * The caller must have View permission to watch a key or keyring.
+ */
+long keyctl_watch_key(key_serial_t id, int watch_queue_fd, int watch_id)
+{
+ struct watch_queue *wqueue;
+ struct watch_list *wlist = NULL;
+ struct watch *watch = NULL;
+ struct key *key;
+ key_ref_t key_ref;
+ long ret;
+
+ if (watch_id < -1 || watch_id > 0xff)
+ return -EINVAL;
+
+ key_ref = lookup_user_key(id, KEY_LOOKUP_CREATE, KEY_NEED_VIEW);
+ if (IS_ERR(key_ref))
+ return PTR_ERR(key_ref);
+ key = key_ref_to_ptr(key_ref);
+
+ wqueue = get_watch_queue(watch_queue_fd);
+ if (IS_ERR(wqueue)) {
+ ret = PTR_ERR(wqueue);
+ goto err_key;
+ }
+
+ if (watch_id >= 0) {
+ ret = -ENOMEM;
+ if (!key->watchers) {
+ wlist = kzalloc(sizeof(*wlist), GFP_KERNEL);
+ if (!wlist)
+ goto err_wqueue;
+ init_watch_list(wlist, NULL);
+ }
+
+ watch = kzalloc(sizeof(*watch), GFP_KERNEL);
+ if (!watch)
+ goto err_wlist;
+
+ init_watch(watch, wqueue);
+ watch->id = key->serial;
+ watch->info_id = (u32)watch_id << WATCH_INFO_ID__SHIFT;
+
+ ret = security_watch_key(key);
+ if (ret < 0)
+ goto err_watch;
+
+ down_write(&key->sem);
+ if (!key->watchers) {
+ key->watchers = wlist;
+ wlist = NULL;
+ }
+
+ ret = add_watch_to_object(watch, key->watchers);
+ up_write(&key->sem);
+
+ if (ret == 0)
+ watch = NULL;
+ } else {
+ ret = -EBADSLT;
+ if (key->watchers) {
+ down_write(&key->sem);
+ ret = remove_watch_from_object(key->watchers,
+ wqueue, key_serial(key),
+ false);
+ up_write(&key->sem);
+ }
+ }
+
+err_watch:
+ kfree(watch);
+err_wlist:
+ kfree(wlist);
+err_wqueue:
+ put_watch_queue(wqueue);
+err_key:
+ key_put(key);
+ return ret;
+}
+#endif /* CONFIG_KEY_NOTIFICATIONS */
+
/*
* Get keyrings subsystem capabilities.
*/
@@ -1857,6 +1947,9 @@ SYSCALL_DEFINE5(keyctl, int, option, unsigned long, arg2, unsigned long, arg3,
case KEYCTL_CAPABILITIES:
return keyctl_capabilities((unsigned char __user *)arg2, (size_t)arg3);
+ case KEYCTL_WATCH_KEY:
+ return keyctl_watch_key((key_serial_t)arg2, (int)arg3, (int)arg4);
+
default:
return -EOPNOTSUPP;
}
diff --git a/security/keys/keyring.c b/security/keys/keyring.c
index febf36c6ddc5..40a0dcdfda44 100644
--- a/security/keys/keyring.c
+++ b/security/keys/keyring.c
@@ -1060,12 +1060,14 @@ int keyring_restrict(key_ref_t keyring_ref, const char *type,
down_write(&keyring->sem);
down_write(&keyring_serialise_restrict_sem);
- if (keyring->restrict_link)
+ if (keyring->restrict_link) {
ret = -EEXIST;
- else if (keyring_detect_restriction_cycle(keyring, restrict_link))
+ } else if (keyring_detect_restriction_cycle(keyring, restrict_link)) {
ret = -EDEADLK;
- else
+ } else {
keyring->restrict_link = restrict_link;
+ notify_key(keyring, NOTIFY_KEY_SETATTR, 0);
+ }
up_write(&keyring_serialise_restrict_sem);
up_write(&keyring->sem);
@@ -1366,12 +1368,14 @@ int __key_link_check_live_key(struct key *keyring, struct key *key)
* holds at most one link to any given key of a particular type+description
* combination.
*/
-void __key_link(struct key *key, struct assoc_array_edit **_edit)
+void __key_link(struct key *keyring, struct key *key,
+ struct assoc_array_edit **_edit)
{
__key_get(key);
assoc_array_insert_set_object(*_edit, keyring_key_to_ptr(key));
assoc_array_apply_edit(*_edit);
*_edit = NULL;
+ notify_key(keyring, NOTIFY_KEY_LINKED, key_serial(key));
}
/*
@@ -1455,7 +1459,7 @@ int key_link(struct key *keyring, struct key *key)
if (ret == 0)
ret = __key_link_check_live_key(keyring, key);
if (ret == 0)
- __key_link(key, &edit);
+ __key_link(keyring, key, &edit);
error_end:
__key_link_end(keyring, &key->index_key, edit);
@@ -1487,7 +1491,7 @@ static int __key_unlink_begin(struct key *keyring, struct key *key,
struct assoc_array_edit *edit;
BUG_ON(*_edit != NULL);
-
+
edit = assoc_array_delete(&keyring->keys, &keyring_assoc_array_ops,
&key->index_key);
if (IS_ERR(edit))
@@ -1507,6 +1511,7 @@ static void __key_unlink(struct key *keyring, struct key *key,
struct assoc_array_edit **_edit)
{
assoc_array_apply_edit(*_edit);
+ notify_key(keyring, NOTIFY_KEY_UNLINKED, key_serial(key));
*_edit = NULL;
key_payload_reserve(keyring, keyring->datalen - KEYQUOTA_LINK_BYTES);
}
@@ -1625,7 +1630,7 @@ int key_move(struct key *key,
goto error;
__key_unlink(from_keyring, key, &from_edit);
- __key_link(key, &to_edit);
+ __key_link(to_keyring, key, &to_edit);
error:
__key_link_end(to_keyring, &key->index_key, to_edit);
__key_unlink_end(from_keyring, key, from_edit);
@@ -1659,6 +1664,7 @@ int keyring_clear(struct key *keyring)
} else {
if (edit)
assoc_array_apply_edit(edit);
+ notify_key(keyring, NOTIFY_KEY_CLEARED, 0);
key_payload_reserve(keyring, 0);
ret = 0;
}
diff --git a/security/keys/request_key.c b/security/keys/request_key.c
index 957b9e3e1492..e1b9f1a80676 100644
--- a/security/keys/request_key.c
+++ b/security/keys/request_key.c
@@ -418,7 +418,7 @@ static int construct_alloc_key(struct keyring_search_context *ctx,
goto key_already_present;
if (dest_keyring)
- __key_link(key, &edit);
+ __key_link(dest_keyring, key, &edit);
mutex_unlock(&key_construction_mutex);
if (dest_keyring)
@@ -437,7 +437,7 @@ static int construct_alloc_key(struct keyring_search_context *ctx,
if (dest_keyring) {
ret = __key_link_check_live_key(dest_keyring, key);
if (ret == 0)
- __key_link(key, &edit);
+ __key_link(dest_keyring, key, &edit);
__key_link_end(dest_keyring, &ctx->index_key, edit);
if (ret < 0)
goto link_check_failed;
^ permalink raw reply related
* [RFC PATCH 07/14] Add sample notification program [ver #3]
From: David Howells @ 2020-01-15 13:31 UTC (permalink / raw)
To: torvalds
Cc: dhowells, Greg Kroah-Hartman, Casey Schaufler, Stephen Smalley,
nicolas.dichtel, raven, Christian Brauner
In-Reply-To: <157909503552.20155.3030058841911628518.stgit@warthog.procyon.org.uk>
The sample program is run like:
./samples/watch_queue/watch_test
and watches "/" for mount changes and the current session keyring for key
changes:
# keyctl add user a a @s
1035096409
# keyctl unlink 1035096409 @s
producing:
# ./watch_test
read() = 16
NOTIFY[000]: ty=000001 sy=02 i=00000110
KEY 2ffc2e5d change=2[linked] aux=1035096409
read() = 16
NOTIFY[000]: ty=000001 sy=02 i=00000110
KEY 2ffc2e5d change=3[unlinked] aux=1035096409
Other events may be produced, such as with a failing disk:
read() = 22
NOTIFY[000]: ty=000003 sy=02 i=00000416
USB 3-7.7 dev-reset e=0 r=0
read() = 24
NOTIFY[000]: ty=000002 sy=06 i=00000418
BLOCK 00800050 e=6[critical medium] s=64000ef8
This corresponds to:
blk_update_request: critical medium error, dev sdf, sector 1677725432 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
in dmesg.
Signed-off-by: David Howells <dhowells@redhat.com>
---
samples/Kconfig | 6 +
samples/Makefile | 1
samples/watch_queue/Makefile | 7 +
samples/watch_queue/watch_test.c | 183 ++++++++++++++++++++++++++++++++++++++
4 files changed, 197 insertions(+)
create mode 100644 samples/watch_queue/Makefile
create mode 100644 samples/watch_queue/watch_test.c
diff --git a/samples/Kconfig b/samples/Kconfig
index 9d236c346de5..5c31971a5745 100644
--- a/samples/Kconfig
+++ b/samples/Kconfig
@@ -190,5 +190,11 @@ config SAMPLE_INTEL_MEI
help
Build a sample program to work with mei device.
+config SAMPLE_WATCH_QUEUE
+ bool "Build example /dev/watch_queue notification consumer"
+ depends on HEADERS_INSTALL
+ help
+ Build example userspace program to use the new mount_notify(),
+ sb_notify() syscalls and the KEYCTL_WATCH_KEY keyctl() function.
endif # SAMPLES
diff --git a/samples/Makefile b/samples/Makefile
index 5ce50ef0f2b2..a80ab2dbaae7 100644
--- a/samples/Makefile
+++ b/samples/Makefile
@@ -23,3 +23,4 @@ obj-$(CONFIG_VIDEO_PCI_SKELETON) += v4l/
obj-y += vfio-mdev/
subdir-$(CONFIG_SAMPLE_VFS) += vfs
obj-$(CONFIG_SAMPLE_INTEL_MEI) += mei/
+subdir-$(CONFIG_SAMPLE_WATCH_QUEUE) += watch_queue
diff --git a/samples/watch_queue/Makefile b/samples/watch_queue/Makefile
new file mode 100644
index 000000000000..eec00dd0a8df
--- /dev/null
+++ b/samples/watch_queue/Makefile
@@ -0,0 +1,7 @@
+# List of programs to build
+hostprogs-y := watch_test
+
+# Tell kbuild to always build the programs
+always := $(hostprogs-y)
+
+HOSTCFLAGS_watch_test.o += -I$(objtree)/usr/include
diff --git a/samples/watch_queue/watch_test.c b/samples/watch_queue/watch_test.c
new file mode 100644
index 000000000000..9bf60abf5c7e
--- /dev/null
+++ b/samples/watch_queue/watch_test.c
@@ -0,0 +1,183 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Use /dev/watch_queue to watch for notifications.
+ *
+ * Copyright (C) 2020 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#define _GNU_SOURCE
+#include <stdbool.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <signal.h>
+#include <unistd.h>
+#include <errno.h>
+#include <sys/ioctl.h>
+#include <limits.h>
+#include <linux/watch_queue.h>
+#include <linux/unistd.h>
+#include <linux/keyctl.h>
+
+#ifndef KEYCTL_WATCH_KEY
+#define KEYCTL_WATCH_KEY -1
+#endif
+#ifndef __NR_watch_devices
+#define __NR_watch_devices -1
+#endif
+
+#define BUF_SIZE 256
+
+static long keyctl_watch_key(int key, int watch_fd, int watch_id)
+{
+ return syscall(__NR_keyctl, KEYCTL_WATCH_KEY, key, watch_fd, watch_id);
+}
+
+static const char *key_subtypes[256] = {
+ [NOTIFY_KEY_INSTANTIATED] = "instantiated",
+ [NOTIFY_KEY_UPDATED] = "updated",
+ [NOTIFY_KEY_LINKED] = "linked",
+ [NOTIFY_KEY_UNLINKED] = "unlinked",
+ [NOTIFY_KEY_CLEARED] = "cleared",
+ [NOTIFY_KEY_REVOKED] = "revoked",
+ [NOTIFY_KEY_INVALIDATED] = "invalidated",
+ [NOTIFY_KEY_SETATTR] = "setattr",
+};
+
+static void saw_key_change(struct watch_notification *n, size_t len)
+{
+ struct key_notification *k = (struct key_notification *)n;
+
+ if (len != sizeof(struct key_notification)) {
+ fprintf(stderr, "Incorrect key message length\n");
+ return;
+ }
+
+ printf("KEY %08x change=%u[%s] aux=%u\n",
+ k->key_id, n->subtype, key_subtypes[n->subtype], k->aux);
+}
+
+/*
+ * Consume and display events.
+ */
+static void consumer(int fd)
+{
+ unsigned char buffer[4096], *p, *end;
+ union {
+ struct watch_notification n;
+ unsigned char buf1[128];
+ } n;
+ ssize_t buf_len;
+
+ for (;;) {
+ buf_len = read(fd, buffer, sizeof(buffer));
+ if (buf_len == -1) {
+ perror("read");
+ exit(1);
+ }
+
+ if (buf_len == 0) {
+ printf("-- END --\n");
+ return;
+ }
+
+ if (buf_len > sizeof(buffer)) {
+ fprintf(stderr, "Read buffer overrun: %zd\n", buf_len);
+ return;
+ }
+
+ printf("read() = %zd\n", buf_len);
+
+ p = buffer;
+ end = buffer + buf_len;
+ while (p < end) {
+ size_t largest, len;
+
+ largest = end - p;
+ if (largest > 128)
+ largest = 128;
+ if (largest < sizeof(struct watch_notification)) {
+ fprintf(stderr, "Short message header: %zu\n", largest);
+ return;
+ }
+ memcpy(&n, p, largest);
+
+ printf("NOTIFY[%03zx]: ty=%06x sy=%02x i=%08x\n",
+ p - buffer, n.n.type, n.n.subtype, n.n.info);
+
+ len = n.n.info & WATCH_INFO_LENGTH;
+ if (len < sizeof(n.n) || len > largest) {
+ fprintf(stderr, "Bad message length: %zu/%zu\n", len, largest);
+ exit(1);
+ }
+
+ switch (n.n.type) {
+ case WATCH_TYPE_META:
+ switch (n.n.subtype) {
+ case WATCH_META_REMOVAL_NOTIFICATION:
+ printf("REMOVAL of watchpoint %08x\n",
+ (n.n.info & WATCH_INFO_ID) >>
+ WATCH_INFO_ID__SHIFT);
+ break;
+ default:
+ printf("other meta record\n");
+ break;
+ }
+ break;
+ case WATCH_TYPE_KEY_NOTIFY:
+ saw_key_change(&n.n, len);
+ break;
+ default:
+ printf("other type\n");
+ break;
+ }
+
+ p += len;
+ }
+ }
+}
+
+static struct watch_notification_filter filter = {
+ .nr_filters = 1,
+ .filters = {
+ [0] = {
+ .type = WATCH_TYPE_KEY_NOTIFY,
+ .subtype_filter[0] = UINT_MAX,
+ },
+ },
+};
+
+int main(int argc, char **argv)
+{
+ int pipefd[2], fd;
+
+ if (pipe2(pipefd, O_NOTIFICATION_PIPE) == -1) {
+ perror("pipe2");
+ exit(1);
+ }
+ fd = pipefd[0];
+
+ if (ioctl(fd, IOC_WATCH_QUEUE_SET_SIZE, BUF_SIZE) == -1) {
+ perror("/dev/watch_queue(size)");
+ exit(1);
+ }
+
+ if (ioctl(fd, IOC_WATCH_QUEUE_SET_FILTER, &filter) == -1) {
+ perror("/dev/watch_queue(filter)");
+ exit(1);
+ }
+
+ if (keyctl_watch_key(KEY_SPEC_SESSION_KEYRING, fd, 0x01) == -1) {
+ perror("keyctl");
+ exit(1);
+ }
+
+ if (keyctl_watch_key(KEY_SPEC_USER_KEYRING, fd, 0x02) == -1) {
+ perror("keyctl");
+ exit(1);
+ }
+
+ consumer(fd);
+ exit(0);
+}
^ permalink raw reply related
* [RFC PATCH 08/14] pipe: Allow buffers to be marked read-whole-or-error for notifications [ver #3]
From: David Howells @ 2020-01-15 13:31 UTC (permalink / raw)
To: torvalds
Cc: dhowells, Greg Kroah-Hartman, Casey Schaufler, Stephen Smalley,
nicolas.dichtel, raven, Christian Brauner
In-Reply-To: <157909503552.20155.3030058841911628518.stgit@warthog.procyon.org.uk>
Allow a buffer to be marked such that read() must return the entire buffer
in one go or return ENOBUFS. Multiple buffers can be amalgamated into a
single read, but a short read will occur if the next "whole" buffer won't
fit.
This is useful for watch queue notifications to make sure we don't split a
notification across multiple reads, especially given that we need to
fabricate an overrun record under some circumstances - and that isn't in
the buffers.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/pipe.c | 8 +++++++-
include/linux/pipe_fs_i.h | 1 +
kernel/watch_queue.c | 2 +-
samples/watch_queue/watch_test.c | 2 +-
4 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/fs/pipe.c b/fs/pipe.c
index 5352c07be47f..05d0b02ed08d 100644
--- a/fs/pipe.c
+++ b/fs/pipe.c
@@ -317,8 +317,14 @@ pipe_read(struct kiocb *iocb, struct iov_iter *to)
size_t written;
int error;
- if (chars > total_len)
+ if (chars > total_len) {
+ if (buf->flags & PIPE_BUF_FLAG_WHOLE) {
+ if (ret == 0)
+ ret = -ENOBUFS;
+ break;
+ }
chars = total_len;
+ }
error = pipe_buf_confirm(pipe, buf);
if (error) {
diff --git a/include/linux/pipe_fs_i.h b/include/linux/pipe_fs_i.h
index cef70acd99bf..f86ae087aaca 100644
--- a/include/linux/pipe_fs_i.h
+++ b/include/linux/pipe_fs_i.h
@@ -8,6 +8,7 @@
#define PIPE_BUF_FLAG_ATOMIC 0x02 /* was atomically mapped */
#define PIPE_BUF_FLAG_GIFT 0x04 /* page is a gift */
#define PIPE_BUF_FLAG_PACKET 0x08 /* read() as a packet */
+#define PIPE_BUF_FLAG_WHOLE 0x10 /* read() must return entire buffer or error */
/**
* struct pipe_buffer - a linux kernel pipe buffer
diff --git a/kernel/watch_queue.c b/kernel/watch_queue.c
index e2e3344a2586..a01f2fed0983 100644
--- a/kernel/watch_queue.c
+++ b/kernel/watch_queue.c
@@ -115,7 +115,7 @@ static bool post_one_notification(struct watch_queue *wqueue,
buf->ops = &watch_queue_pipe_buf_ops;
buf->offset = offset;
buf->len = len;
- buf->flags = 0;
+ buf->flags = PIPE_BUF_FLAG_WHOLE;
pipe->head = head + 1;
if (!test_and_clear_bit(note, wqueue->notes_bitmap)) {
diff --git a/samples/watch_queue/watch_test.c b/samples/watch_queue/watch_test.c
index 9bf60abf5c7e..924e13a49c37 100644
--- a/samples/watch_queue/watch_test.c
+++ b/samples/watch_queue/watch_test.c
@@ -63,7 +63,7 @@ static void saw_key_change(struct watch_notification *n, size_t len)
*/
static void consumer(int fd)
{
- unsigned char buffer[4096], *p, *end;
+ unsigned char buffer[433], *p, *end;
union {
struct watch_notification n;
unsigned char buf1[128];
^ permalink raw reply related
* [RFC PATCH 09/14] pipe: Add notification lossage handling [ver #3]
From: David Howells @ 2020-01-15 13:31 UTC (permalink / raw)
To: torvalds
Cc: dhowells, Greg Kroah-Hartman, Casey Schaufler, Stephen Smalley,
nicolas.dichtel, raven, Christian Brauner
In-Reply-To: <157909503552.20155.3030058841911628518.stgit@warthog.procyon.org.uk>
Add handling for loss of notifications by having read() insert a
loss-notification message after it has read the pipe buffer that was last
in the ring when the loss occurred.
Lossage can come about either by running out of notification descriptors or
by running out of space in the pipe ring.
Signed-off-by: David Howells <dhowells@redhat.com>
---
fs/pipe.c | 28 ++++++++++++++++++++++++++++
include/linux/pipe_fs_i.h | 7 +++++++
kernel/watch_queue.c | 2 ++
samples/watch_queue/watch_test.c | 3 +++
4 files changed, 40 insertions(+)
diff --git a/fs/pipe.c b/fs/pipe.c
index 05d0b02ed08d..423aafca4338 100644
--- a/fs/pipe.c
+++ b/fs/pipe.c
@@ -311,6 +311,30 @@ pipe_read(struct kiocb *iocb, struct iov_iter *to)
unsigned int tail = pipe->tail;
unsigned int mask = pipe->ring_size - 1;
+#ifdef CONFIG_WATCH_QUEUE
+ if (pipe->note_loss) {
+ struct watch_notification n;
+
+ if (total_len < 8) {
+ if (ret == 0)
+ ret = -ENOBUFS;
+ break;
+ }
+
+ n.type = WATCH_TYPE_META;
+ n.subtype = WATCH_META_LOSS_NOTIFICATION;
+ n.info = watch_sizeof(n);
+ if (copy_to_iter(&n, sizeof(n), to) != sizeof(n)) {
+ if (ret == 0)
+ ret = -EFAULT;
+ break;
+ }
+ ret += sizeof(n);
+ total_len -= sizeof(n);
+ pipe->note_loss = false;
+ }
+#endif
+
if (!pipe_empty(head, tail)) {
struct pipe_buffer *buf = &pipe->bufs[tail & mask];
size_t chars = buf->len;
@@ -352,6 +376,10 @@ pipe_read(struct kiocb *iocb, struct iov_iter *to)
if (!buf->len) {
pipe_buf_release(pipe, buf);
spin_lock_irq(&pipe->wait.lock);
+#ifdef CONFIG_WATCH_QUEUE
+ if (buf->flags & PIPE_BUF_FLAG_LOSS)
+ pipe->note_loss = true;
+#endif
tail++;
pipe->tail = tail;
spin_unlock_irq(&pipe->wait.lock);
diff --git a/include/linux/pipe_fs_i.h b/include/linux/pipe_fs_i.h
index f86ae087aaca..810eb2b1efc6 100644
--- a/include/linux/pipe_fs_i.h
+++ b/include/linux/pipe_fs_i.h
@@ -9,6 +9,9 @@
#define PIPE_BUF_FLAG_GIFT 0x04 /* page is a gift */
#define PIPE_BUF_FLAG_PACKET 0x08 /* read() as a packet */
#define PIPE_BUF_FLAG_WHOLE 0x10 /* read() must return entire buffer or error */
+#ifdef CONFIG_WATCH_QUEUE
+#define PIPE_BUF_FLAG_LOSS 0x20 /* Message loss happened after this buffer */
+#endif
/**
* struct pipe_buffer - a linux kernel pipe buffer
@@ -33,6 +36,7 @@ struct pipe_buffer {
* @wait: reader/writer wait point in case of empty/full pipe
* @head: The point of buffer production
* @tail: The point of buffer consumption
+ * @note_loss: The next read() should insert a data-lost message
* @max_usage: The maximum number of slots that may be used in the ring
* @ring_size: total number of buffers (should be a power of 2)
* @nr_accounted: The amount this pipe accounts for in user->pipe_bufs
@@ -55,6 +59,9 @@ struct pipe_inode_info {
unsigned int tail;
unsigned int max_usage;
unsigned int ring_size;
+#ifdef CONFIG_WATCH_QUEUE
+ bool note_loss;
+#endif
unsigned int nr_accounted;
unsigned int readers;
unsigned int writers;
diff --git a/kernel/watch_queue.c b/kernel/watch_queue.c
index a01f2fed0983..d48f422f391a 100644
--- a/kernel/watch_queue.c
+++ b/kernel/watch_queue.c
@@ -132,6 +132,8 @@ static bool post_one_notification(struct watch_queue *wqueue,
return done;
lost:
+ buf = &pipe->bufs[(head - 1) & mask];
+ buf->flags |= PIPE_BUF_FLAG_LOSS;
goto out;
}
diff --git a/samples/watch_queue/watch_test.c b/samples/watch_queue/watch_test.c
index 924e13a49c37..0eaff5dc04c3 100644
--- a/samples/watch_queue/watch_test.c
+++ b/samples/watch_queue/watch_test.c
@@ -120,6 +120,9 @@ static void consumer(int fd)
(n.n.info & WATCH_INFO_ID) >>
WATCH_INFO_ID__SHIFT);
break;
+ case WATCH_META_LOSS_NOTIFICATION:
+ printf("-- LOSS --\n");
+ break;
default:
printf("other meta record\n");
break;
^ permalink raw reply related
* [RFC PATCH 10/14] Add a general, global device notification watch list [ver #3]
From: David Howells @ 2020-01-15 13:32 UTC (permalink / raw)
To: torvalds
Cc: dhowells, Greg Kroah-Hartman, Casey Schaufler, Stephen Smalley,
nicolas.dichtel, raven, Christian Brauner
In-Reply-To: <157909503552.20155.3030058841911628518.stgit@warthog.procyon.org.uk>
Create a general, global watch list that can be used for the posting of
device notification events, for such things as device attachment,
detachment and errors on sources such as block devices and USB devices.
This can be enabled with:
CONFIG_DEVICE_NOTIFICATIONS
To add a watch on this list, an event queue must be created and configured:
pipe2(fds, O_NOTIFICATION_PIPE);
ioctl(fds[1], IOC_WATCH_QUEUE_SET_SIZE, 256);
and then a watch can be placed upon it using a system call:
watch_devices(fds[1], 12, 0);
Unless the application wants to receive all events, it should employ
appropriate filters. For example, to receive just USB notifications, it
could do:
struct watch_notification_filter filter = {
.nr_filters = 1,
.filters = {
[0] = {
.type = WATCH_TYPE_USB_NOTIFY,
.subtype_filter[0] = UINT_MAX;
},
},
};
ioctl(fds[1], IOC_WATCH_QUEUE_SET_FILTER, &filter);
Signed-off-by: David Howells <dhowells@redhat.com>
---
Documentation/watch_queue.rst | 22 ++++++-
arch/alpha/kernel/syscalls/syscall.tbl | 1
arch/arm/tools/syscall.tbl | 1
arch/arm64/include/asm/unistd.h | 2 -
arch/arm64/include/asm/unistd32.h | 2 +
arch/ia64/kernel/syscalls/syscall.tbl | 1
arch/m68k/kernel/syscalls/syscall.tbl | 1
arch/microblaze/kernel/syscalls/syscall.tbl | 1
arch/mips/kernel/syscalls/syscall_n32.tbl | 1
arch/mips/kernel/syscalls/syscall_n64.tbl | 1
arch/mips/kernel/syscalls/syscall_o32.tbl | 1
arch/parisc/kernel/syscalls/syscall.tbl | 1
arch/powerpc/kernel/syscalls/syscall.tbl | 1
arch/s390/kernel/syscalls/syscall.tbl | 1
arch/sh/kernel/syscalls/syscall.tbl | 1
arch/sparc/kernel/syscalls/syscall.tbl | 1
arch/x86/entry/syscalls/syscall_32.tbl | 1
arch/x86/entry/syscalls/syscall_64.tbl | 1
arch/xtensa/kernel/syscalls/syscall.tbl | 1
drivers/base/Kconfig | 9 +++
drivers/base/Makefile | 1
drivers/base/watch.c | 90 +++++++++++++++++++++++++++
include/linux/device.h | 7 ++
include/linux/syscalls.h | 1
include/uapi/asm-generic/unistd.h | 4 +
kernel/sys_ni.c | 1
26 files changed, 152 insertions(+), 3 deletions(-)
create mode 100644 drivers/base/watch.c
diff --git a/Documentation/watch_queue.rst b/Documentation/watch_queue.rst
index d8f70282d247..ed592700be0e 100644
--- a/Documentation/watch_queue.rst
+++ b/Documentation/watch_queue.rst
@@ -223,6 +223,25 @@ The ``id`` is the ID of the source object (such as the serial number on a key).
Only watches that have the same ID set in them will see this notification.
+Global Device Watch List
+========================
+
+There is a global watch list that hardware generated events, such as device
+connection, disconnection, failure and error can be posted upon. It must be
+enabled using::
+
+ CONFIG_DEVICE_NOTIFICATIONS
+
+Watchpoints are set in userspace using the device_notify(2) system call.
+Within the kernel events are posted upon it using::
+
+ void post_device_notification(struct watch_notification *n, u64 id);
+
+where ``n`` is the formatted notification record to post. ``id`` is an
+identifier that can be used to direct to specific watches, but it should be 0
+for general use on this queue.
+
+
Watch Sources
=============
@@ -238,7 +257,8 @@ Any particular buffer can be fed from multiple sources. Sources include:
* WATCH_TYPE_BLOCK_NOTIFY
Notifications of this type indicate block layer events, such as I/O errors
- or temporary link loss. Watches of this type are set on a global queue.
+ or temporary link loss. Watches of this type are set on the global device
+ watch list.
Event Filtering
diff --git a/arch/alpha/kernel/syscalls/syscall.tbl b/arch/alpha/kernel/syscalls/syscall.tbl
index 8e13b0b2928d..3b355b0996a1 100644
--- a/arch/alpha/kernel/syscalls/syscall.tbl
+++ b/arch/alpha/kernel/syscalls/syscall.tbl
@@ -475,3 +475,4 @@
543 common fspick sys_fspick
544 common pidfd_open sys_pidfd_open
# 545 reserved for clone3
+546 common watch_devices sys_watch_devices
diff --git a/arch/arm/tools/syscall.tbl b/arch/arm/tools/syscall.tbl
index 6da7dc4d79cc..0f080cf44cc9 100644
--- a/arch/arm/tools/syscall.tbl
+++ b/arch/arm/tools/syscall.tbl
@@ -449,3 +449,4 @@
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
435 common clone3 sys_clone3
+436 common watch_devices sys_watch_devices
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index 2629a68b8724..368761302768 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -38,7 +38,7 @@
#define __ARM_NR_compat_set_tls (__ARM_NR_COMPAT_BASE + 5)
#define __ARM_NR_COMPAT_END (__ARM_NR_COMPAT_BASE + 0x800)
-#define __NR_compat_syscalls 436
+#define __NR_compat_syscalls 437
#endif
#define __ARCH_WANT_SYS_CLONE
diff --git a/arch/arm64/include/asm/unistd32.h b/arch/arm64/include/asm/unistd32.h
index 94ab29cf4f00..b5310789ce7a 100644
--- a/arch/arm64/include/asm/unistd32.h
+++ b/arch/arm64/include/asm/unistd32.h
@@ -879,6 +879,8 @@ __SYSCALL(__NR_fspick, sys_fspick)
__SYSCALL(__NR_pidfd_open, sys_pidfd_open)
#define __NR_clone3 435
__SYSCALL(__NR_clone3, sys_clone3)
+#define __NR_watch_devices 436
+__SYSCALL(__NR_watch_devices, sys_watch_devices)
/*
* Please add new compat syscalls above this comment and update
diff --git a/arch/ia64/kernel/syscalls/syscall.tbl b/arch/ia64/kernel/syscalls/syscall.tbl
index 36d5faf4c86c..2f33f5db2fed 100644
--- a/arch/ia64/kernel/syscalls/syscall.tbl
+++ b/arch/ia64/kernel/syscalls/syscall.tbl
@@ -356,3 +356,4 @@
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
# 435 reserved for clone3
+436 common watch_devices sys_watch_devices
diff --git a/arch/m68k/kernel/syscalls/syscall.tbl b/arch/m68k/kernel/syscalls/syscall.tbl
index a88a285a0e5f..83e4e8784b88 100644
--- a/arch/m68k/kernel/syscalls/syscall.tbl
+++ b/arch/m68k/kernel/syscalls/syscall.tbl
@@ -435,3 +435,4 @@
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
# 435 reserved for clone3
+436 common watch_devices sys_watch_devices
diff --git a/arch/microblaze/kernel/syscalls/syscall.tbl b/arch/microblaze/kernel/syscalls/syscall.tbl
index 09b0cd7dab0a..9a70a3be3b7b 100644
--- a/arch/microblaze/kernel/syscalls/syscall.tbl
+++ b/arch/microblaze/kernel/syscalls/syscall.tbl
@@ -441,3 +441,4 @@
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
435 common clone3 sys_clone3
+436 common watch_devices sys_watch_devices
diff --git a/arch/mips/kernel/syscalls/syscall_n32.tbl b/arch/mips/kernel/syscalls/syscall_n32.tbl
index e7c5ab38e403..b39527fc32c9 100644
--- a/arch/mips/kernel/syscalls/syscall_n32.tbl
+++ b/arch/mips/kernel/syscalls/syscall_n32.tbl
@@ -374,3 +374,4 @@
433 n32 fspick sys_fspick
434 n32 pidfd_open sys_pidfd_open
435 n32 clone3 __sys_clone3
+436 n32 watch_devices sys_watch_devices
diff --git a/arch/mips/kernel/syscalls/syscall_n64.tbl b/arch/mips/kernel/syscalls/syscall_n64.tbl
index 13cd66581f3b..a7f0c5e71768 100644
--- a/arch/mips/kernel/syscalls/syscall_n64.tbl
+++ b/arch/mips/kernel/syscalls/syscall_n64.tbl
@@ -350,3 +350,4 @@
433 n64 fspick sys_fspick
434 n64 pidfd_open sys_pidfd_open
435 n64 clone3 __sys_clone3
+436 n64 watch_devices sys_watch_devices
diff --git a/arch/mips/kernel/syscalls/syscall_o32.tbl b/arch/mips/kernel/syscalls/syscall_o32.tbl
index 353539ea4140..6f378288598c 100644
--- a/arch/mips/kernel/syscalls/syscall_o32.tbl
+++ b/arch/mips/kernel/syscalls/syscall_o32.tbl
@@ -423,3 +423,4 @@
433 o32 fspick sys_fspick
434 o32 pidfd_open sys_pidfd_open
435 o32 clone3 __sys_clone3
+436 o32 watch_devices sys_watch_devices
diff --git a/arch/parisc/kernel/syscalls/syscall.tbl b/arch/parisc/kernel/syscalls/syscall.tbl
index 285ff516150c..b64bbafa5919 100644
--- a/arch/parisc/kernel/syscalls/syscall.tbl
+++ b/arch/parisc/kernel/syscalls/syscall.tbl
@@ -433,3 +433,4 @@
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
435 common clone3 sys_clone3_wrapper
+436 common watch_devices sys_watch_devices
diff --git a/arch/powerpc/kernel/syscalls/syscall.tbl b/arch/powerpc/kernel/syscalls/syscall.tbl
index 43f736ed47f2..0a503239ab5c 100644
--- a/arch/powerpc/kernel/syscalls/syscall.tbl
+++ b/arch/powerpc/kernel/syscalls/syscall.tbl
@@ -517,3 +517,4 @@
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
435 nospu clone3 ppc_clone3
+436 common watch_devices sys_watch_devices
diff --git a/arch/s390/kernel/syscalls/syscall.tbl b/arch/s390/kernel/syscalls/syscall.tbl
index 3054e9c035a3..19b43c0d928a 100644
--- a/arch/s390/kernel/syscalls/syscall.tbl
+++ b/arch/s390/kernel/syscalls/syscall.tbl
@@ -438,3 +438,4 @@
433 common fspick sys_fspick sys_fspick
434 common pidfd_open sys_pidfd_open sys_pidfd_open
435 common clone3 sys_clone3 sys_clone3
+436 common watch_devices sys_watch_devices sys_watch_devices
diff --git a/arch/sh/kernel/syscalls/syscall.tbl b/arch/sh/kernel/syscalls/syscall.tbl
index b5ed26c4c005..b454e07c9372 100644
--- a/arch/sh/kernel/syscalls/syscall.tbl
+++ b/arch/sh/kernel/syscalls/syscall.tbl
@@ -438,3 +438,4 @@
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
# 435 reserved for clone3
+436 common watch_devices sys_watch_devices
diff --git a/arch/sparc/kernel/syscalls/syscall.tbl b/arch/sparc/kernel/syscalls/syscall.tbl
index 8c8cc7537fb2..8ef43c27457e 100644
--- a/arch/sparc/kernel/syscalls/syscall.tbl
+++ b/arch/sparc/kernel/syscalls/syscall.tbl
@@ -481,3 +481,4 @@
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
# 435 reserved for clone3
+436 common watch_devices sys_watch_devices
diff --git a/arch/x86/entry/syscalls/syscall_32.tbl b/arch/x86/entry/syscalls/syscall_32.tbl
index 15908eb9b17e..bdf7e845d1f8 100644
--- a/arch/x86/entry/syscalls/syscall_32.tbl
+++ b/arch/x86/entry/syscalls/syscall_32.tbl
@@ -440,3 +440,4 @@
433 i386 fspick sys_fspick __ia32_sys_fspick
434 i386 pidfd_open sys_pidfd_open __ia32_sys_pidfd_open
435 i386 clone3 sys_clone3 __ia32_sys_clone3
+436 i386 watch_devices sys_watch_devices __ia32_sys_watch_devices
diff --git a/arch/x86/entry/syscalls/syscall_64.tbl b/arch/x86/entry/syscalls/syscall_64.tbl
index c29976eca4a8..29293d103829 100644
--- a/arch/x86/entry/syscalls/syscall_64.tbl
+++ b/arch/x86/entry/syscalls/syscall_64.tbl
@@ -357,6 +357,7 @@
433 common fspick __x64_sys_fspick
434 common pidfd_open __x64_sys_pidfd_open
435 common clone3 __x64_sys_clone3/ptregs
+436 common watch_devices __x64_sys_watch_devices
#
# x32-specific system call numbers start at 512 to avoid cache impact
diff --git a/arch/xtensa/kernel/syscalls/syscall.tbl b/arch/xtensa/kernel/syscalls/syscall.tbl
index 25f4de729a6d..243fa18b8d1e 100644
--- a/arch/xtensa/kernel/syscalls/syscall.tbl
+++ b/arch/xtensa/kernel/syscalls/syscall.tbl
@@ -406,3 +406,4 @@
433 common fspick sys_fspick
434 common pidfd_open sys_pidfd_open
435 common clone3 sys_clone3
+436 common watch_devices sys_watch_devices
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
index c3b3b5c0b0da..d4ae1c1adf69 100644
--- a/drivers/base/Kconfig
+++ b/drivers/base/Kconfig
@@ -1,6 +1,15 @@
# SPDX-License-Identifier: GPL-2.0
menu "Generic Driver Options"
+config DEVICE_NOTIFICATIONS
+ bool "Provide device event notifications"
+ depends on WATCH_QUEUE
+ help
+ This option provides support for getting hardware event notifications
+ on devices, buses and interfaces. This makes use of the
+ /dev/watch_queue misc device to handle the notification buffer.
+ device_notify(2) is used to set/remove watches.
+
config UEVENT_HELPER
bool "Support for uevent helper"
help
diff --git a/drivers/base/Makefile b/drivers/base/Makefile
index 157452080f3d..4db2e8f1a1f4 100644
--- a/drivers/base/Makefile
+++ b/drivers/base/Makefile
@@ -7,6 +7,7 @@ obj-y := component.o core.o bus.o dd.o syscore.o \
attribute_container.o transport_class.o \
topology.o container.o property.o cacheinfo.o \
devcon.o swnode.o
+obj-$(CONFIG_DEVICE_NOTIFICATIONS) += watch.o
obj-$(CONFIG_DEVTMPFS) += devtmpfs.o
obj-y += power/
obj-$(CONFIG_ISA_BUS_API) += isa.o
diff --git a/drivers/base/watch.c b/drivers/base/watch.c
new file mode 100644
index 000000000000..725aaa24275b
--- /dev/null
+++ b/drivers/base/watch.c
@@ -0,0 +1,90 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Event notifications.
+ *
+ * Copyright (C) 2019 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#include <linux/device.h>
+#include <linux/watch_queue.h>
+#include <linux/syscalls.h>
+#include <linux/init_task.h>
+#include <linux/security.h>
+
+/*
+ * Global queue for watching for device layer events.
+ */
+static struct watch_list device_watchers = {
+ .watchers = HLIST_HEAD_INIT,
+ .lock = __SPIN_LOCK_UNLOCKED(&device_watchers.lock),
+};
+
+static DEFINE_SPINLOCK(device_watchers_lock);
+
+/**
+ * post_device_notification - Post notification of a device event
+ * @n - The notification to post
+ * @id - The device ID
+ *
+ * Note that there's only a global queue to which all events are posted. Might
+ * want to provide per-dev queues also.
+ */
+void post_device_notification(struct watch_notification *n, u64 id)
+{
+ post_watch_notification(&device_watchers, n, &init_cred, id);
+}
+EXPORT_SYMBOL(post_device_notification);
+
+/**
+ * sys_watch_devices - Watch for device events.
+ * @watch_fd: The watch queue to send notifications to.
+ * @watch_id: The watch ID to be placed in the notification (-1 to remove watch)
+ * @flags: Flags (reserved for future)
+ */
+SYSCALL_DEFINE3(watch_devices, int, watch_fd, int, watch_id, unsigned int, flags)
+{
+ struct watch_queue *wqueue;
+ struct watch *watch = NULL;
+ long ret = -ENOMEM;
+
+ if (watch_id < -1 || watch_id > 0xff || flags)
+ return -EINVAL;
+
+ wqueue = get_watch_queue(watch_fd);
+ if (IS_ERR(wqueue)) {
+ ret = PTR_ERR(wqueue);
+ goto err;
+ }
+
+ if (watch_id >= 0) {
+ watch = kzalloc(sizeof(*watch), GFP_KERNEL);
+ if (!watch)
+ goto err_wqueue;
+
+ init_watch(watch, wqueue);
+ watch->info_id = (u32)watch_id << WATCH_INFO_ID__SHIFT;
+
+ ret = security_watch_devices();
+ if (ret < 0)
+ goto err_watch;
+
+ spin_lock(&device_watchers_lock);
+ ret = add_watch_to_object(watch, &device_watchers);
+ spin_unlock(&device_watchers_lock);
+ if (ret == 0)
+ watch = NULL;
+ } else {
+ spin_lock(&device_watchers_lock);
+ ret = remove_watch_from_object(&device_watchers, wqueue, 0,
+ false);
+ spin_unlock(&device_watchers_lock);
+ }
+
+err_watch:
+ kfree(watch);
+err_wqueue:
+ put_watch_queue(wqueue);
+err:
+ return ret;
+}
diff --git a/include/linux/device.h b/include/linux/device.h
index 96ff76731e93..f32de6466092 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -43,6 +43,7 @@ struct iommu_group;
struct iommu_fwspec;
struct dev_pin_info;
struct iommu_param;
+struct watch_notification;
struct bus_attribute {
struct attribute attr;
@@ -1687,6 +1688,12 @@ void device_link_remove(void *consumer, struct device *supplier);
void device_links_supplier_sync_state_pause(void);
void device_links_supplier_sync_state_resume(void);
+#ifdef CONFIG_DEVICE_NOTIFICATIONS
+extern void post_device_notification(struct watch_notification *n, u64 id);
+#else
+static inline void post_device_notification(struct watch_notification *n, u64 id) {}
+#endif
+
#ifndef dev_fmt
#define dev_fmt(fmt) fmt
#endif
diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index 2960dedcfde8..393661015a30 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -1000,6 +1000,7 @@ asmlinkage long sys_fspick(int dfd, const char __user *path, unsigned int flags)
asmlinkage long sys_pidfd_send_signal(int pidfd, int sig,
siginfo_t __user *info,
unsigned int flags);
+asmlinkage long sys_watch_devices(int watch_fd, int watch_id, unsigned int flags);
/*
* Architecture-specific system calls
diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h
index 1fc8faa6e973..4794d3c2afd7 100644
--- a/include/uapi/asm-generic/unistd.h
+++ b/include/uapi/asm-generic/unistd.h
@@ -850,9 +850,11 @@ __SYSCALL(__NR_pidfd_open, sys_pidfd_open)
#define __NR_clone3 435
__SYSCALL(__NR_clone3, sys_clone3)
#endif
+#define __NR_watch_devices 436
+__SYSCALL(__NR_watch_devices, sys_watch_devices)
#undef __NR_syscalls
-#define __NR_syscalls 436
+#define __NR_syscalls 437
/*
* 32 bit systems traditionally used different
diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c
index 3b69a560a7ac..0e9b275260f8 100644
--- a/kernel/sys_ni.c
+++ b/kernel/sys_ni.c
@@ -51,6 +51,7 @@ COND_SYSCALL_COMPAT(io_pgetevents);
COND_SYSCALL(io_uring_setup);
COND_SYSCALL(io_uring_enter);
COND_SYSCALL(io_uring_register);
+COND_SYSCALL(watch_devices);
/* fs/xattr.c */
^ permalink raw reply related
* [RFC PATCH 11/14] block: Add block layer notifications [ver #3]
From: David Howells @ 2020-01-15 13:32 UTC (permalink / raw)
To: torvalds
Cc: dhowells, Greg Kroah-Hartman, Casey Schaufler, Stephen Smalley,
nicolas.dichtel, raven, Christian Brauner
In-Reply-To: <157909503552.20155.3030058841911628518.stgit@warthog.procyon.org.uk>
Add a block layer notification mechanism whereby notifications about
block-layer events such as I/O errors, can be reported to a monitoring
process asynchronously.
Firstly, an event queue needs to be created:
pipe2(fds, O_NOTIFICATION_PIPE);
ioctl(fds[1], IOC_WATCH_QUEUE_SET_SIZE, 256);
then a notification can be set up to report block notifications via that
queue:
struct watch_notification_filter filter = {
.nr_filters = 1,
.filters = {
[0] = {
.type = WATCH_TYPE_BLOCK_NOTIFY,
.subtype_filter[0] = UINT_MAX;
},
},
};
ioctl(fds[1], IOC_WATCH_QUEUE_SET_FILTER, &filter);
watch_devices(fds[1], 12);
After that, records will be placed into the queue when, for example, errors
occur on a block device. Records are of the following format:
struct block_notification {
struct watch_notification watch;
__u64 dev;
__u64 sector;
} *n;
Where:
n->watch.type will be WATCH_TYPE_BLOCK_NOTIFY
n->watch.subtype will be the type of notification, such as
NOTIFY_BLOCK_ERROR_CRITICAL_MEDIUM.
n->watch.info & WATCH_INFO_LENGTH will indicate the length of the
record.
n->watch.info & WATCH_INFO_ID will be the second argument to
watch_devices(), shifted.
n->dev will be the device numbers munged together.
n->sector will indicate the affected sector (if appropriate for the
event).
Note that it is permissible for event records to be of variable length -
or, at least, the length may be dependent on the subtype.
Signed-off-by: David Howells <dhowells@redhat.com>
---
Documentation/watch_queue.rst | 4 +++-
block/Kconfig | 9 +++++++++
block/blk-core.c | 29 ++++++++++++++++++++++++++++
include/linux/blkdev.h | 15 ++++++++++++++
include/uapi/linux/watch_queue.h | 30 ++++++++++++++++++++++++++++-
samples/watch_queue/watch_test.c | 40 +++++++++++++++++++++++++++++++++++++-
6 files changed, 124 insertions(+), 3 deletions(-)
diff --git a/Documentation/watch_queue.rst b/Documentation/watch_queue.rst
index ed592700be0e..f2299f631ae8 100644
--- a/Documentation/watch_queue.rst
+++ b/Documentation/watch_queue.rst
@@ -8,7 +8,9 @@ opened by userspace. This can be used in conjunction with::
* Key/keyring notifications
- * General device event notifications
+ * General device event notifications, including::
+
+ * Block layer event notifications
The notifications buffers can be enabled by:
diff --git a/block/Kconfig b/block/Kconfig
index c23094a14a2b..9fa6ce2177bc 100644
--- a/block/Kconfig
+++ b/block/Kconfig
@@ -181,6 +181,15 @@ config BLK_SED_OPAL
Enabling this option enables users to setup/unlock/lock
Locking ranges for SED devices using the Opal protocol.
+config BLK_NOTIFICATIONS
+ bool "Block layer event notifications"
+ depends on DEVICE_NOTIFICATIONS
+ help
+ This option provides support for getting block layer event
+ notifications. This makes use of the /dev/watch_queue misc device to
+ handle the notification buffer and provides the device_notify() system
+ call to enable/disable watches.
+
menu "Partition Types"
source "block/partitions/Kconfig"
diff --git a/block/blk-core.c b/block/blk-core.c
index 089e890ab208..50a5de025d5e 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -187,6 +187,22 @@ static const struct {
[BLK_STS_IOERR] = { -EIO, "I/O" },
};
+#ifdef CONFIG_BLK_NOTIFICATIONS
+static const
+enum block_notification_type blk_notifications[ARRAY_SIZE(blk_errors)] = {
+ [BLK_STS_TIMEOUT] = NOTIFY_BLOCK_ERROR_TIMEOUT,
+ [BLK_STS_NOSPC] = NOTIFY_BLOCK_ERROR_NO_SPACE,
+ [BLK_STS_TRANSPORT] = NOTIFY_BLOCK_ERROR_RECOVERABLE_TRANSPORT,
+ [BLK_STS_TARGET] = NOTIFY_BLOCK_ERROR_CRITICAL_TARGET,
+ [BLK_STS_NEXUS] = NOTIFY_BLOCK_ERROR_CRITICAL_NEXUS,
+ [BLK_STS_MEDIUM] = NOTIFY_BLOCK_ERROR_CRITICAL_MEDIUM,
+ [BLK_STS_PROTECTION] = NOTIFY_BLOCK_ERROR_PROTECTION,
+ [BLK_STS_RESOURCE] = NOTIFY_BLOCK_ERROR_KERNEL_RESOURCE,
+ [BLK_STS_DEV_RESOURCE] = NOTIFY_BLOCK_ERROR_DEVICE_RESOURCE,
+ [BLK_STS_IOERR] = NOTIFY_BLOCK_ERROR_IO,
+};
+#endif
+
blk_status_t errno_to_blk_status(int errno)
{
int i;
@@ -227,6 +243,19 @@ static void print_req_error(struct request *req, blk_status_t status,
req->cmd_flags & ~REQ_OP_MASK,
req->nr_phys_segments,
IOPRIO_PRIO_CLASS(req->ioprio));
+
+#ifdef CONFIG_BLK_NOTIFICATIONS
+ if (blk_notifications[idx]) {
+ struct block_notification n = {
+ .watch.type = WATCH_TYPE_BLOCK_NOTIFY,
+ .watch.subtype = blk_notifications[idx],
+ .watch.info = watch_sizeof(n),
+ .dev = req->rq_disk ? disk_devt(req->rq_disk) : 0,
+ .sector = blk_rq_pos(req),
+ };
+ post_block_notification(&n);
+ }
+#endif
}
static void req_bio_endio(struct request *rq, struct bio *bio,
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 47eb22a3b7f9..3cd1853dbdac 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -27,6 +27,7 @@
#include <linux/percpu-refcount.h>
#include <linux/scatterlist.h>
#include <linux/blkzoned.h>
+#include <linux/watch_queue.h>
struct module;
struct scsi_ioctl_command;
@@ -1770,6 +1771,20 @@ static inline bool blk_req_can_dispatch_to_zone(struct request *rq)
}
#endif /* CONFIG_BLK_DEV_ZONED */
+#ifdef CONFIG_BLK_NOTIFICATIONS
+static inline void post_block_notification(struct block_notification *n)
+{
+ u64 id = 0; /* Might want to allow dev# here. */
+
+ post_device_notification(&n->watch, id);
+}
+#else
+static inline void post_block_notification(struct block_notification *n)
+{
+}
+#endif
+
+
#else /* CONFIG_BLOCK */
struct block_device;
diff --git a/include/uapi/linux/watch_queue.h b/include/uapi/linux/watch_queue.h
index c3d8320b5d3a..557771413242 100644
--- a/include/uapi/linux/watch_queue.h
+++ b/include/uapi/linux/watch_queue.h
@@ -14,7 +14,8 @@
enum watch_notification_type {
WATCH_TYPE_META = 0, /* Special record */
WATCH_TYPE_KEY_NOTIFY = 1, /* Key change event notification */
- WATCH_TYPE__NR = 2
+ WATCH_TYPE_BLOCK_NOTIFY = 2, /* Block layer event notification */
+ WATCH_TYPE__NR = 3
};
enum watch_meta_notification_subtype {
@@ -101,4 +102,31 @@ struct key_notification {
__u32 aux; /* Per-type auxiliary data */
};
+/*
+ * Type of block layer notification.
+ */
+enum block_notification_type {
+ NOTIFY_BLOCK_ERROR_TIMEOUT = 1, /* Timeout error */
+ NOTIFY_BLOCK_ERROR_NO_SPACE = 2, /* Critical space allocation error */
+ NOTIFY_BLOCK_ERROR_RECOVERABLE_TRANSPORT = 3, /* Recoverable transport error */
+ NOTIFY_BLOCK_ERROR_CRITICAL_TARGET = 4, /* Critical target error */
+ NOTIFY_BLOCK_ERROR_CRITICAL_NEXUS = 5, /* Critical nexus error */
+ NOTIFY_BLOCK_ERROR_CRITICAL_MEDIUM = 6, /* Critical medium error */
+ NOTIFY_BLOCK_ERROR_PROTECTION = 7, /* Protection error */
+ NOTIFY_BLOCK_ERROR_KERNEL_RESOURCE = 8, /* Kernel resource error */
+ NOTIFY_BLOCK_ERROR_DEVICE_RESOURCE = 9, /* Device resource error */
+ NOTIFY_BLOCK_ERROR_IO = 10, /* Other I/O error */
+};
+
+/*
+ * Block layer notification record.
+ * - watch.type = WATCH_TYPE_BLOCK_NOTIFY
+ * - watch.subtype = enum block_notification_type
+ */
+struct block_notification {
+ struct watch_notification watch; /* WATCH_TYPE_BLOCK_NOTIFY */
+ __u64 dev; /* Device number */
+ __u64 sector; /* Affected sector */
+};
+
#endif /* _UAPI_LINUX_WATCH_QUEUE_H */
diff --git a/samples/watch_queue/watch_test.c b/samples/watch_queue/watch_test.c
index 0eaff5dc04c3..f5260fb792d1 100644
--- a/samples/watch_queue/watch_test.c
+++ b/samples/watch_queue/watch_test.c
@@ -58,6 +58,32 @@ static void saw_key_change(struct watch_notification *n, size_t len)
k->key_id, n->subtype, key_subtypes[n->subtype], k->aux);
}
+static const char *block_subtypes[256] = {
+ [NOTIFY_BLOCK_ERROR_TIMEOUT] = "timeout",
+ [NOTIFY_BLOCK_ERROR_NO_SPACE] = "critical space allocation",
+ [NOTIFY_BLOCK_ERROR_RECOVERABLE_TRANSPORT] = "recoverable transport",
+ [NOTIFY_BLOCK_ERROR_CRITICAL_TARGET] = "critical target",
+ [NOTIFY_BLOCK_ERROR_CRITICAL_NEXUS] = "critical nexus",
+ [NOTIFY_BLOCK_ERROR_CRITICAL_MEDIUM] = "critical medium",
+ [NOTIFY_BLOCK_ERROR_PROTECTION] = "protection",
+ [NOTIFY_BLOCK_ERROR_KERNEL_RESOURCE] = "kernel resource",
+ [NOTIFY_BLOCK_ERROR_DEVICE_RESOURCE] = "device resource",
+ [NOTIFY_BLOCK_ERROR_IO] = "I/O",
+};
+
+static void saw_block_change(struct watch_notification *n, size_t len)
+{
+ struct block_notification *b = (struct block_notification *)n;
+
+ if (len < sizeof(struct block_notification))
+ return;
+
+ printf("BLOCK %08llx e=%u[%s] s=%llx\n",
+ (unsigned long long)b->dev,
+ n->subtype, block_subtypes[n->subtype],
+ (unsigned long long)b->sector);
+}
+
/*
* Consume and display events.
*/
@@ -131,6 +157,9 @@ static void consumer(int fd)
case WATCH_TYPE_KEY_NOTIFY:
saw_key_change(&n.n, len);
break;
+ case WATCH_TYPE_BLOCK_NOTIFY:
+ saw_block_change(&n.n, len);
+ break;
default:
printf("other type\n");
break;
@@ -142,12 +171,16 @@ static void consumer(int fd)
}
static struct watch_notification_filter filter = {
- .nr_filters = 1,
+ .nr_filters = 2,
.filters = {
[0] = {
.type = WATCH_TYPE_KEY_NOTIFY,
.subtype_filter[0] = UINT_MAX,
},
+ [1] = {
+ .type = WATCH_TYPE_BLOCK_NOTIFY,
+ .subtype_filter[0] = UINT_MAX,
+ },
},
};
@@ -181,6 +214,11 @@ int main(int argc, char **argv)
exit(1);
}
+ if (syscall(__NR_watch_devices, fd, 0x04, 0) == -1) {
+ perror("watch_devices");
+ exit(1);
+ }
+
consumer(fd);
exit(0);
}
^ permalink raw reply related
* [RFC PATCH 12/14] usb: Add USB subsystem notifications [ver #3]
From: David Howells @ 2020-01-15 13:32 UTC (permalink / raw)
To: torvalds-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b
Cc: dhowells-H+wXaHxf7aLQT0dZR+AlfA, Greg Kroah-Hartman,
Casey Schaufler, Stephen Smalley,
nicolas.dichtel-pdR9zngts4EAvxtiuMwx3w,
raven-PKsaG3nR2I+sTnJN9+BGXg, Christian Brauner
In-Reply-To: <157909503552.20155.3030058841911628518.stgit-S6HVgzuS8uM4Awkfq6JHfwNdhmdF6hFW@public.gmane.org>
Add a USB subsystem notification mechanism whereby notifications about
hardware events such as device connection, disconnection, reset and I/O
errors, can be reported to a monitoring process asynchronously.
Firstly, an event queue needs to be created:
pipe2(fds, O_NOTIFICATION_PIPE);
ioctl(fds[1], IOC_WATCH_QUEUE_SET_SIZE, 256);
then a notification can be set up to report USB notifications via that
queue:
struct watch_notification_filter filter = {
.nr_filters = 1,
.filters = {
[0] = {
.type = WATCH_TYPE_USB_NOTIFY,
.subtype_filter[0] = UINT_MAX;
},
},
};
ioctl(fds[1], IOC_WATCH_QUEUE_SET_FILTER, &filter);
notify_devices(fds[1], 12);
After that, messages will be placed into the queue when events occur on a
USB device or bus. Messages are of the following format:
struct usb_notification {
struct watch_notification watch;
__u32 error;
__u32 reserved;
__u8 name_len;
__u8 name[0];
} *n;
Where:
n->watch.type will be WATCH_TYPE_USB_NOTIFY
n->watch.subtype will be the type of notification, such as
NOTIFY_USB_DEVICE_ADD.
n->watch.info & WATCH_INFO_LENGTH will indicate the length of the
message.
n->watch.info & WATCH_INFO_ID will be the second argument to
device_notify(), shifted.
n->error and n->reserved are intended to convey information such as
error codes, but are currently not used
n->name_len and n->name convey the USB device name as an
unterminated string. This may be truncated - it is currently
limited to a maximum 63 chars.
Note that it is permissible for messages to be of variable length - or, at
least, the length may be dependent on the subtype.
Signed-off-by: David Howells <dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
cc: Greg Kroah-Hartman <gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org>
cc: linux-usb-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
---
Documentation/watch_queue.rst | 9 +++++++
drivers/usb/core/Kconfig | 9 +++++++
drivers/usb/core/devio.c | 47 ++++++++++++++++++++++++++++++++++++++
drivers/usb/core/hub.c | 4 +++
include/linux/usb.h | 18 +++++++++++++++
include/uapi/linux/watch_queue.h | 28 ++++++++++++++++++++++-
samples/watch_queue/watch_test.c | 29 +++++++++++++++++++++++
7 files changed, 142 insertions(+), 2 deletions(-)
diff --git a/Documentation/watch_queue.rst b/Documentation/watch_queue.rst
index f2299f631ae8..5321a9cb1ab2 100644
--- a/Documentation/watch_queue.rst
+++ b/Documentation/watch_queue.rst
@@ -12,6 +12,8 @@ opened by userspace. This can be used in conjunction with::
* Block layer event notifications
+ * USB subsystem event notifications
+
The notifications buffers can be enabled by:
@@ -262,6 +264,13 @@ Any particular buffer can be fed from multiple sources. Sources include:
or temporary link loss. Watches of this type are set on the global device
watch list.
+ * WATCH_TYPE_USB_NOTIFY
+
+ Notifications of this type indicate USB subsystem events, such as
+ attachment, removal, reset and I/O errors. Separate events are generated
+ for buses and devices. Watchpoints of this type are set on the global
+ device watch list.
+
Event Filtering
===============
diff --git a/drivers/usb/core/Kconfig b/drivers/usb/core/Kconfig
index ecaacc8ed311..57e7b649e48b 100644
--- a/drivers/usb/core/Kconfig
+++ b/drivers/usb/core/Kconfig
@@ -102,3 +102,12 @@ config USB_AUTOSUSPEND_DELAY
The default value Linux has always had is 2 seconds. Change
this value if you want a different delay and cannot modify
the command line or module parameter.
+
+config USB_NOTIFICATIONS
+ bool "Provide USB hardware event notifications"
+ depends on USB && DEVICE_NOTIFICATIONS
+ help
+ This option provides support for getting hardware event notifications
+ on USB devices and interfaces. This makes use of the
+ /dev/watch_queue misc device to handle the notification buffer.
+ device_notify(2) is used to set/remove watches.
diff --git a/drivers/usb/core/devio.c b/drivers/usb/core/devio.c
index 12bb5722b420..3436a2bb6e98 100644
--- a/drivers/usb/core/devio.c
+++ b/drivers/usb/core/devio.c
@@ -41,6 +41,7 @@
#include <linux/dma-mapping.h>
#include <asm/byteorder.h>
#include <linux/moduleparam.h>
+#include <linux/watch_queue.h>
#include "usb.h"
@@ -2747,13 +2748,59 @@ static void usbdev_remove(struct usb_device *udev)
mutex_unlock(&usbfs_mutex);
}
+#ifdef CONFIG_USB_NOTIFICATIONS
+static noinline void post_usb_notification(const char *devname,
+ enum usb_notification_type subtype,
+ u32 error)
+{
+ unsigned int name_len, n_len;
+ u64 id = 0; /* We can put a device ID here for separate dev watches */
+
+ struct {
+ struct usb_notification n;
+ char more_name[USB_NOTIFICATION_MAX_NAME_LEN -
+ (sizeof(struct usb_notification) -
+ offsetof(struct usb_notification, name))];
+ } n;
+
+ name_len = strlen(devname);
+ name_len = min_t(size_t, name_len, USB_NOTIFICATION_MAX_NAME_LEN);
+ n_len = offsetof(struct usb_notification, name) + name_len;
+
+ memset(&n, 0, sizeof(n));
+ memcpy(n.n.name, devname, n_len);
+
+ n.n.watch.type = WATCH_TYPE_USB_NOTIFY;
+ n.n.watch.subtype = subtype;
+ n.n.watch.info = n_len;
+ n.n.error = error;
+ n.n.name_len = name_len;
+
+ post_device_notification(&n.n.watch, id);
+}
+
+void post_usb_device_notification(const struct usb_device *udev,
+ enum usb_notification_type subtype, u32 error)
+{
+ post_usb_notification(dev_name(&udev->dev), subtype, error);
+}
+
+void post_usb_bus_notification(const struct usb_bus *ubus,
+ enum usb_notification_type subtype, u32 error)
+{
+ post_usb_notification(ubus->bus_name, subtype, error);
+}
+#endif
+
static int usbdev_notify(struct notifier_block *self,
unsigned long action, void *dev)
{
switch (action) {
case USB_DEVICE_ADD:
+ post_usb_device_notification(dev, NOTIFY_USB_DEVICE_ADD, 0);
break;
case USB_DEVICE_REMOVE:
+ post_usb_device_notification(dev, NOTIFY_USB_DEVICE_REMOVE, 0);
usbdev_remove(dev);
break;
}
diff --git a/drivers/usb/core/hub.c b/drivers/usb/core/hub.c
index f229ad6952c0..eaf28eed51b0 100644
--- a/drivers/usb/core/hub.c
+++ b/drivers/usb/core/hub.c
@@ -30,6 +30,7 @@
#include <linux/random.h>
#include <linux/pm_qos.h>
#include <linux/kobject.h>
+#include <linux/watch_queue.h>
#include <linux/uaccess.h>
#include <asm/byteorder.h>
@@ -4606,6 +4607,9 @@ hub_port_init(struct usb_hub *hub, struct usb_device *udev, int port1,
(udev->config) ? "reset" : "new", speed,
devnum, driver_name);
+ if (udev->config)
+ post_usb_device_notification(udev, NOTIFY_USB_DEVICE_RESET, 0);
+
/* Set up TT records, if needed */
if (hdev->tt) {
udev->tt = hdev->tt;
diff --git a/include/linux/usb.h b/include/linux/usb.h
index e656e7b4b1e4..93fa0666f95a 100644
--- a/include/linux/usb.h
+++ b/include/linux/usb.h
@@ -26,6 +26,7 @@
struct usb_device;
struct usb_driver;
struct wusb_dev;
+enum usb_notification_type;
/*-------------------------------------------------------------------------*/
@@ -2015,6 +2016,23 @@ extern void usb_led_activity(enum usb_led_event ev);
static inline void usb_led_activity(enum usb_led_event ev) {}
#endif
+/*
+ * Notification functions.
+ */
+#ifdef CONFIG_USB_NOTIFICATIONS
+extern void post_usb_device_notification(const struct usb_device *udev,
+ enum usb_notification_type subtype,
+ u32 error);
+extern void post_usb_bus_notification(const struct usb_bus *ubus,
+ enum usb_notification_type subtype,
+ u32 error);
+#else
+static inline void post_usb_device_notification(const struct usb_device *udev,
+ unsigned int subtype, u32 error) {}
+static inline void post_usb_bus_notification(const struct usb_bus *ubus,
+ unsigned int subtype, u32 error) {}
+#endif
+
#endif /* __KERNEL__ */
#endif
diff --git a/include/uapi/linux/watch_queue.h b/include/uapi/linux/watch_queue.h
index 557771413242..ad1ae229674a 100644
--- a/include/uapi/linux/watch_queue.h
+++ b/include/uapi/linux/watch_queue.h
@@ -15,7 +15,8 @@ enum watch_notification_type {
WATCH_TYPE_META = 0, /* Special record */
WATCH_TYPE_KEY_NOTIFY = 1, /* Key change event notification */
WATCH_TYPE_BLOCK_NOTIFY = 2, /* Block layer event notification */
- WATCH_TYPE__NR = 3
+ WATCH_TYPE_USB_NOTIFY = 3, /* USB subsystem event notification */
+ WATCH_TYPE__NR = 4
};
enum watch_meta_notification_subtype {
@@ -129,4 +130,29 @@ struct block_notification {
__u64 sector; /* Affected sector */
};
+/*
+ * Type of USB layer notification.
+ */
+enum usb_notification_type {
+ NOTIFY_USB_DEVICE_ADD = 0, /* USB device added */
+ NOTIFY_USB_DEVICE_REMOVE = 1, /* USB device removed */
+ NOTIFY_USB_DEVICE_RESET = 2, /* USB device reset */
+ NOTIFY_USB_DEVICE_ERROR = 3, /* USB device error */
+};
+
+/*
+ * USB subsystem notification record.
+ * - watch.type = WATCH_TYPE_USB_NOTIFY
+ * - watch.subtype = enum usb_notification_type
+ */
+struct usb_notification {
+ struct watch_notification watch; /* WATCH_TYPE_USB_NOTIFY */
+ __u32 error;
+ __u32 reserved;
+ __u8 name_len; /* Length of device name */
+ __u8 name[0]; /* Device name (padded to __u64, truncated at 63 chars) */
+};
+
+#define USB_NOTIFICATION_MAX_NAME_LEN 63
+
#endif /* _UAPI_LINUX_WATCH_QUEUE_H */
diff --git a/samples/watch_queue/watch_test.c b/samples/watch_queue/watch_test.c
index f5260fb792d1..e4d47dfcc5d7 100644
--- a/samples/watch_queue/watch_test.c
+++ b/samples/watch_queue/watch_test.c
@@ -84,6 +84,26 @@ static void saw_block_change(struct watch_notification *n, size_t len)
(unsigned long long)b->sector);
}
+static const char *usb_subtypes[256] = {
+ [NOTIFY_USB_DEVICE_ADD] = "dev-add",
+ [NOTIFY_USB_DEVICE_REMOVE] = "dev-remove",
+ [NOTIFY_USB_DEVICE_RESET] = "dev-reset",
+ [NOTIFY_USB_DEVICE_ERROR] = "dev-error",
+};
+
+static void saw_usb_event(struct watch_notification *n, size_t len)
+{
+ struct usb_notification *u = (struct usb_notification *)n;
+
+ if (len < sizeof(struct usb_notification))
+ return;
+
+ printf("USB %*.*s %s e=%x r=%x\n",
+ u->name_len, u->name_len, u->name,
+ usb_subtypes[n->subtype],
+ u->error, u->reserved);
+}
+
/*
* Consume and display events.
*/
@@ -160,6 +180,9 @@ static void consumer(int fd)
case WATCH_TYPE_BLOCK_NOTIFY:
saw_block_change(&n.n, len);
break;
+ case WATCH_TYPE_USB_NOTIFY:
+ saw_usb_event(&n.n, len);
+ break;
default:
printf("other type\n");
break;
@@ -171,7 +194,7 @@ static void consumer(int fd)
}
static struct watch_notification_filter filter = {
- .nr_filters = 2,
+ .nr_filters = 3,
.filters = {
[0] = {
.type = WATCH_TYPE_KEY_NOTIFY,
@@ -181,6 +204,10 @@ static struct watch_notification_filter filter = {
.type = WATCH_TYPE_BLOCK_NOTIFY,
.subtype_filter[0] = UINT_MAX,
},
+ [2] = {
+ .type = WATCH_TYPE_USB_NOTIFY,
+ .subtype_filter[0] = UINT_MAX,
+ },
},
};
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox