[PATCH v2 0/4] RDSEED support for the Linux kernel

[PATCH v2 0/4] RDSEED support for the Linux kernel

[H. Peter Anvin]

Upcoming Intel silicon adds a new RDSEED instruction.  Whereas RDRAND
returns output from a PRNG, the RDSEED instruction returns fully
conditioned entropy that is suitable for use as seeds to a PRNG.

This patchset adds support for RDSEED in the Linux kernel in three
places:

1. During bootup, use RDSEED to initialize the entropy pool if
   available (we already use RDRAND for this).  We don't add any
   credit at this point, but it will give much better starting point.

2. In the slow path to add_interrupt_randomness, executed once per
   second, we take a single RDSEED sample and mix it into the entropy
   pool, crediting it at 50% of its rated entropy.  This was suggested
   by Linus.

3. If we are about to block on /dev/random due to lack of entropy,
   attempt an "emergency pool refill" using RDSEED.

Changes since version 1:

a. Rebased on top of random.git:dev.
b. Unbreak the PowerPC build (I had managed to miss that PowerPC had
   grown archrandom.h support.)
c. Remove duplicate dummy function definitions in <linux/random.h>.
d. Add a fourth patch containing a microoptimization: avoid the loop
   in arch_random_refill() if arch_get_random_seed*() is unavailable.

Comments are, of course, appreciated.

Ted, if you are OK with this could you add this to random.git:dev so
linux-next can pick it up?

	-hpa

[PATCH v2 1/4] x86, random: Enable the RDSEED instruction

[H. Peter Anvin]

Upcoming Intel silicon adds a new RDSEED instruction, which is similar
to RDRAND but provides a stronger guarantee: unlike RDRAND, RDSEED
will always reseed the PRNG from the true random number source between
each read.  Thus, the output of RDSEED is guaranteed to be 100%
entropic, unlike RDRAND which is only architecturally guaranteed to be
1/512 entropic (although in practice is much more.)

The RDSEED instruction takes the same time to execute as RDRAND, but
RDSEED unlike RDRAND can legitimately return failure (CF=0) due to
entropy exhaustion if too many threads on too many cores are hammering
the RDSEED instruction at the same time.  Therefore, we have to be
more conservative and only use it in places where we can tolerate
failures.

This patch introduces the primitives arch_get_random_seed_{int,long}()
but does not use it yet.

Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
Reviewed-by: Ingo Molnar <mingo@kernel.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <michael@ellerman.id.au>
---
 arch/powerpc/include/asm/archrandom.h |  9 +++++++++
 arch/x86/include/asm/archrandom.h     | 34 +++++++++++++++++++++++++++++++++-
 include/linux/random.h                |  8 ++++++++
 3 files changed, 50 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h
index d853d16..801beba 100644
--- a/arch/powerpc/include/asm/archrandom.h
+++ b/arch/powerpc/include/asm/archrandom.h
@@ -27,6 +27,15 @@ static inline int arch_get_random_int(unsigned int *v)
 
 int powernv_get_random_long(unsigned long *v);
 
+static inline int arch_get_random_seed_long(unsigned long *v)
+{
+	return 0;
+}
+static inline int arch_get_random_seed_int(unsigned int *v)
+{
+	return 0;
+}
+
 #endif /* CONFIG_ARCH_RANDOM */
 
 #endif /* _ASM_POWERPC_ARCHRANDOM_H */
diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h
index 0d9ec77..ba064d5 100644
--- a/arch/x86/include/asm/archrandom.h
+++ b/arch/x86/include/asm/archrandom.h
@@ -1,7 +1,7 @@
 /*
  * This file is part of the Linux kernel.
  *
- * Copyright (c) 2011, Intel Corporation
+ * Copyright (c) 2011-2014, Intel Corporation
  * Authors: Fenghua Yu <fenghua.yu@intel.com>,
  *          H. Peter Anvin <hpa@linux.intel.com>
  *
@@ -31,14 +31,27 @@
 #define RDRAND_RETRY_LOOPS	10
 
 #define RDRAND_INT	".byte 0x0f,0xc7,0xf0"
+#define RDSEED_INT	".byte 0x0f,0xc7,0xf8"
 #ifdef CONFIG_X86_64
 # define RDRAND_LONG	".byte 0x48,0x0f,0xc7,0xf0"
+# define RDSEED_LONG	".byte 0x48,0x0f,0xc7,0xf8"
 #else
 # define RDRAND_LONG	RDRAND_INT
+# define RDSEED_LONG	RDSEED_INT
 #endif
 
 #ifdef CONFIG_ARCH_RANDOM
 
+/* A single attempt at RDSEED */
+static inline bool rdseed_long(unsigned long *v)
+{
+	unsigned char ok;
+	asm volatile(RDSEED_LONG "\n\t"
+		     "setc %0"
+		     : "=qm" (ok), "=a" (*v));
+	return ok;
+}
+
 #define GET_RANDOM(name, type, rdrand, nop)			\
 static inline int name(type *v)					\
 {								\
@@ -56,16 +69,35 @@ static inline int name(type *v)					\
 	return ok;						\
 }
 
+#define GET_SEED(name, type, rdseed, nop)			\
+static inline int name(type *v)					\
+{								\
+	unsigned char ok;					\
+	alternative_io("movb $0, %0\n\t"			\
+		       nop,					\
+		       rdseed "\n\t"				\
+		       "setc %0",				\
+		       X86_FEATURE_RDSEED,                      \
+		       ASM_OUTPUT2("=q" (ok), "=a" (*v)));	\
+	return ok;						\
+}
+
 #ifdef CONFIG_X86_64
 
 GET_RANDOM(arch_get_random_long, unsigned long, RDRAND_LONG, ASM_NOP5);
 GET_RANDOM(arch_get_random_int, unsigned int, RDRAND_INT, ASM_NOP4);
 
+GET_SEED(arch_get_random_seed_long, unsigned long, RDSEED_LONG, ASM_NOP5);
+GET_SEED(arch_get_random_seed_int, unsigned int, RDSEED_INT, ASM_NOP4);
+
 #else
 
 GET_RANDOM(arch_get_random_long, unsigned long, RDRAND_LONG, ASM_NOP3);
 GET_RANDOM(arch_get_random_int, unsigned int, RDRAND_INT, ASM_NOP3);
 
+GET_SEED(arch_get_random_seed_long, unsigned long, RDSEED_LONG, ASM_NOP4);
+GET_SEED(arch_get_random_seed_int, unsigned int, RDSEED_INT, ASM_NOP4);
+
 #endif /* CONFIG_X86_64 */
 
 #endif  /* CONFIG_ARCH_RANDOM */
diff --git a/include/linux/random.h b/include/linux/random.h
index 4002b3d..ddaddec 100644
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -72,6 +72,14 @@ static inline int arch_get_random_int(unsigned int *v)
 {
 	return 0;
 }
+static inline int arch_get_random_seed_long(unsigned long *v)
+{
+	return 0;
+}
+static inline int arch_get_random_seed_int(unsigned int *v)
+{
+	return 0;
+}
 #endif
 
 /* Pseudo random number generator from numerical recipes. */
-- 
1.8.5.3

[PATCH v2 2/4] random: Use arch_get_random_seed*() at init time and once a second

[H. Peter Anvin]

Use arch_get_random_seed*() in two places in the Linux random
driver (drivers/char/random.c):

1. During entropy pool initialization, use RDSEED in favor of RDRAND,
   with a fallback to the latter.  Entropy exhaustion is unlikely to
   happen there on physical hardware as the machine is single-threaded
   at that point, but could happen in a virtual machine.  In that
   case, the fallback to RDRAND will still provide more than adequate
   entropy pool initialization.

2. Once a second, issue RDSEED and, if successful, feed it to the
   entropy pool.  To ensure an extra layer of security, only credit
   half the entropy just in case.

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Reviewed-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
---
 drivers/char/random.c | 24 ++++++++++++++++++++----
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index d07575c..a4bea77 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -844,6 +844,8 @@ void add_interrupt_randomness(int irq, int irq_flags)
 	cycles_t		cycles = random_get_entropy();
 	__u32			input[4], c_high, j_high;
 	__u64			ip;
+	unsigned long		seed;
+	int			credit;
 
 	c_high = (sizeof(cycles) > 4) ? cycles >> 32 : 0;
 	j_high = (sizeof(now) > 4) ? now >> 32 : 0;
@@ -862,20 +864,33 @@ void add_interrupt_randomness(int irq, int irq_flags)
 
 	r = nonblocking_pool.initialized ? &input_pool : &nonblocking_pool;
 	__mix_pool_bytes(r, &fast_pool->pool, sizeof(fast_pool->pool), NULL);
+
 	/*
 	 * If we don't have a valid cycle counter, and we see
 	 * back-to-back timer interrupts, then skip giving credit for
-	 * any entropy.
+	 * any entropy, otherwise credit 1 bit.
 	 */
+	credit = 1;
 	if (cycles == 0) {
 		if (irq_flags & __IRQF_TIMER) {
 			if (fast_pool->last_timer_intr)
-				return;
+				credit = 0;
 			fast_pool->last_timer_intr = 1;
 		} else
 			fast_pool->last_timer_intr = 0;
 	}
-	credit_entropy_bits(r, 1);
+
+	/*
+	 * If we have architectural seed generator, produce a seed and
+	 * add it to the pool.  For the sake of paranoia count it as
+	 * 50% entropic.
+	 */
+	if (arch_get_random_seed_long(&seed)) {
+		__mix_pool_bytes(r, &seed, sizeof(seed), NULL);
+		credit += sizeof(seed) * 4;
+	}
+
+	credit_entropy_bits(r, credit);
 }
 
 #ifdef CONFIG_BLOCK
@@ -1235,7 +1250,8 @@ static void init_std_data(struct entropy_store *r)
 	r->last_pulled = jiffies;
 	mix_pool_bytes(r, &now, sizeof(now), NULL);
 	for (i = r->poolinfo->poolbytes; i > 0; i -= sizeof(rv)) {
-		if (!arch_get_random_long(&rv))
+		if (!arch_get_random_seed_long(&rv) &&
+		    !arch_get_random_long(&rv))
 			rv = random_get_entropy();
 		mix_pool_bytes(r, &rv, sizeof(rv), NULL);
 	}
-- 
1.8.5.3

[PATCH v2 3/4] random: If we have arch_get_random_seed*(), try it before blocking

[H. Peter Anvin]

If we have arch_get_random_seed*(), try to use it for emergency refill
of the entropy pool before giving up and blocking on /dev/random.  It
may or may not work in the moment, but if it does work, it will give
the user better service than blocking will.

Reviewed-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
---
 drivers/char/random.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/drivers/char/random.c b/drivers/char/random.c
index a4bea77..c35cee2 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1294,6 +1294,34 @@ void rand_initialize_disk(struct gendisk *disk)
 }
 #endif
 
+/*
+ * Attempt an emergency refill using arch_get_random_seed_long().
+ *
+ * As with add_interrupt_randomness() be paranoid and only
+ * credit the output as 50% entropic.
+ */
+static int arch_random_refill(void)
+{
+	const unsigned int nlongs = 64;	/* Arbitrary number */
+	unsigned int n = 0;
+	unsigned int i;
+	unsigned long buf[nlongs];
+
+	for (i = 0; i < nlongs; i++) {
+		if (arch_get_random_seed_long(&buf[n]))
+			n++;
+	}
+
+	if (n) {
+		unsigned int rand_bytes = n * sizeof(unsigned long);
+
+		mix_pool_bytes(&input_pool, buf, rand_bytes, NULL);
+		credit_entropy_bits(&input_pool, rand_bytes*4);
+	}
+
+	return n;
+}
+
 static ssize_t
 random_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
 {
@@ -1312,8 +1340,13 @@ random_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos)
 				  ENTROPY_BITS(&input_pool));
 		if (n > 0)
 			return n;
+
 		/* Pool is (near) empty.  Maybe wait and retry. */
 
+		/* First try an emergency refill */
+		if (arch_random_refill())
+			continue;
+
 		if (file->f_flags & O_NONBLOCK)
 			return -EAGAIN;
 
-- 
1.8.5.3

[PATCH v2 4/4] random: Add arch_has_random[_seed]()

[H. Peter Anvin]

Add predicate functions for having arch_get_random[_seed]*().  The
only current use is to avoid the loop in arch_random_refill() when
arch_get_random_seed_long() is unavailable.

Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <michael@ellerman.id.au>
---
 arch/powerpc/include/asm/archrandom.h | 9 +++++++++
 arch/x86/include/asm/archrandom.h     | 3 +++
 drivers/char/random.c                 | 3 +++
 include/linux/random.h                | 8 ++++++++
 4 files changed, 23 insertions(+)

diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h
index 801beba..bde5311 100644
--- a/arch/powerpc/include/asm/archrandom.h
+++ b/arch/powerpc/include/asm/archrandom.h
@@ -25,6 +25,11 @@ static inline int arch_get_random_int(unsigned int *v)
 	return rc;
 }
 
+static inline int arch_has_random(void)
+{
+	return !!ppc_md.get_random_long;
+}
+
 int powernv_get_random_long(unsigned long *v);
 
 static inline int arch_get_random_seed_long(unsigned long *v)
@@ -35,6 +40,10 @@ static inline int arch_get_random_seed_int(unsigned int *v)
 {
 	return 0;
 }
+static inline int arch_has_random_seed(void)
+{
+	return 0;
+}
 
 #endif /* CONFIG_ARCH_RANDOM */
 
diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h
index ba064d5..c7ed4a6 100644
--- a/arch/x86/include/asm/archrandom.h
+++ b/arch/x86/include/asm/archrandom.h
@@ -100,6 +100,9 @@ GET_SEED(arch_get_random_seed_int, unsigned int, RDSEED_INT, ASM_NOP4);
 
 #endif /* CONFIG_X86_64 */
 
+#define arch_has_random()	static_cpu_has(X86_FEATURE_RDRAND)
+#define arch_has_random_seed()	static_cpu_has(X86_FEATURE_RDSEED)
+
 #endif  /* CONFIG_ARCH_RANDOM */
 
 extern void x86_init_rdrand(struct cpuinfo_x86 *c);
diff --git a/drivers/char/random.c b/drivers/char/random.c
index c35cee2..6b75713 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -1307,6 +1307,9 @@ static int arch_random_refill(void)
 	unsigned int i;
 	unsigned long buf[nlongs];
 
+	if (!arch_has_random_seed())
+		return 0;
+
 	for (i = 0; i < nlongs; i++) {
 		if (arch_get_random_seed_long(&buf[n]))
 			n++;
diff --git a/include/linux/random.h b/include/linux/random.h
index ddaddec..e7a221a 100644
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -72,6 +72,10 @@ static inline int arch_get_random_int(unsigned int *v)
 {
 	return 0;
 }
+static inline int arch_has_random(void)
+{
+	return 0;
+}
 static inline int arch_get_random_seed_long(unsigned long *v)
 {
 	return 0;
@@ -80,6 +84,10 @@ static inline int arch_get_random_seed_int(unsigned int *v)
 {
 	return 0;
 }
+static inline int arch_has_random_seed(void)
+{
+	return 0;
+}
 #endif
 
 /* Pseudo random number generator from numerical recipes. */
-- 
1.8.5.3

Re: [PATCH v2 4/4] random: Add arch_has_random[_seed]()

[Benjamin Herrenschmidt]

On Mon, 2014-03-17 at 16:36 -0700, H. Peter Anvin wrote:
> Add predicate functions for having arch_get_random[_seed]*().  The
> only current use is to avoid the loop in arch_random_refill() when
> arch_get_random_seed_long() is unavailable.

Paul, I think our HW rng on P7+ and later would qualify no ?

Cheers,
Ben.

> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Michael Ellerman <michael@ellerman.id.au>
> ---
>  arch/powerpc/include/asm/archrandom.h | 9 +++++++++
>  arch/x86/include/asm/archrandom.h     | 3 +++
>  drivers/char/random.c                 | 3 +++
>  include/linux/random.h                | 8 ++++++++
>  4 files changed, 23 insertions(+)
> 
> diff --git a/arch/powerpc/include/asm/archrandom.h b/arch/powerpc/include/asm/archrandom.h
> index 801beba..bde5311 100644
> --- a/arch/powerpc/include/asm/archrandom.h
> +++ b/arch/powerpc/include/asm/archrandom.h
> @@ -25,6 +25,11 @@ static inline int arch_get_random_int(unsigned int *v)
>  	return rc;
>  }
>  
> +static inline int arch_has_random(void)
> +{
> +	return !!ppc_md.get_random_long;
> +}
> +
>  int powernv_get_random_long(unsigned long *v);
>  
>  static inline int arch_get_random_seed_long(unsigned long *v)
> @@ -35,6 +40,10 @@ static inline int arch_get_random_seed_int(unsigned int *v)
>  {
>  	return 0;
>  }
> +static inline int arch_has_random_seed(void)
> +{
> +	return 0;
> +}
>  
>  #endif /* CONFIG_ARCH_RANDOM */
>  
> diff --git a/arch/x86/include/asm/archrandom.h b/arch/x86/include/asm/archrandom.h
> index ba064d5..c7ed4a6 100644
> --- a/arch/x86/include/asm/archrandom.h
> +++ b/arch/x86/include/asm/archrandom.h
> @@ -100,6 +100,9 @@ GET_SEED(arch_get_random_seed_int, unsigned int, RDSEED_INT, ASM_NOP4);
>  
>  #endif /* CONFIG_X86_64 */
>  
> +#define arch_has_random()	static_cpu_has(X86_FEATURE_RDRAND)
> +#define arch_has_random_seed()	static_cpu_has(X86_FEATURE_RDSEED)
> +
>  #endif  /* CONFIG_ARCH_RANDOM */
>  
>  extern void x86_init_rdrand(struct cpuinfo_x86 *c);
> diff --git a/drivers/char/random.c b/drivers/char/random.c
> index c35cee2..6b75713 100644
> --- a/drivers/char/random.c
> +++ b/drivers/char/random.c
> @@ -1307,6 +1307,9 @@ static int arch_random_refill(void)
>  	unsigned int i;
>  	unsigned long buf[nlongs];
>  
> +	if (!arch_has_random_seed())
> +		return 0;
> +
>  	for (i = 0; i < nlongs; i++) {
>  		if (arch_get_random_seed_long(&buf[n]))
>  			n++;
> diff --git a/include/linux/random.h b/include/linux/random.h
> index ddaddec..e7a221a 100644
> --- a/include/linux/random.h
> +++ b/include/linux/random.h
> @@ -72,6 +72,10 @@ static inline int arch_get_random_int(unsigned int *v)
>  {
>  	return 0;
>  }
> +static inline int arch_has_random(void)
> +{
> +	return 0;
> +}
>  static inline int arch_get_random_seed_long(unsigned long *v)
>  {
>  	return 0;
> @@ -80,6 +84,10 @@ static inline int arch_get_random_seed_int(unsigned int *v)
>  {
>  	return 0;
>  }
> +static inline int arch_has_random_seed(void)
> +{
> +	return 0;
> +}
>  #endif
>  
>  /* Pseudo random number generator from numerical recipes. */

Re: [PATCH v2 4/4] random: Add arch_has_random[_seed]()

[H. Peter Anvin]

On 03/17/2014 08:44 PM, Benjamin Herrenschmidt wrote:
> On Mon, 2014-03-17 at 16:36 -0700, H. Peter Anvin wrote:
>> Add predicate functions for having arch_get_random[_seed]*().  The
>> only current use is to avoid the loop in arch_random_refill() when
>> arch_get_random_seed_long() is unavailable.
> 
> Paul, I think our HW rng on P7+ and later would qualify no ?
> 
> Cheers,
> Ben.

Just to be clear: since I don't know the parameters of the PowerPC hwrng
I just put in the stub functions, leaving it up to you to analyze if it
is suitable and analyze it.  Do let me know if I can be of any
assistance, of course.

	-hpa

Re: [PATCH v2 0/4] RDSEED support for the Linux kernel

[tytso]

On Mon, Mar 17, 2014 at 04:36:26PM -0700, H. Peter Anvin wrote:
> Changes since version 1:
> 
> a. Rebased on top of random.git:dev.
> b. Unbreak the PowerPC build (I had managed to miss that PowerPC had
>    grown archrandom.h support.)
> c. Remove duplicate dummy function definitions in <linux/random.h>.
> d. Add a fourth patch containing a microoptimization: avoid the loop
>    in arch_random_refill() if arch_get_random_seed*() is unavailable.
> 
> Comments are, of course, appreciated.
> 
> Ted, if you are OK with this could you add this to random.git:dev so
> linux-next can pick it up?

Thanks, applied to the random.git tree.

					- Ted