arch_random_refill

arch_random_refill

[Stephan Mueller]

Hi Peter,

some time back when the RDRAND instruction was debated, a patch was offered 
for driver/char/random.c that in essence turned /dev/random into a frontend 
for RDRAND in case that instruction was available. The patch kind of 
monopolized the noise sources such that if a user space "random number hog" 
pulled data from /dev/random endlessly, the (almost) only noise source was 
RDRAND. As that patch treated RDRAND to provide entropy, the blocking behavior 
went away for /dev/random.

That patch did not sit well with some developers and it got finally changed 
such that the output of RDRAND is now just XORed with the output of the 
"classic" /dev/random behavior -- /dev/random is still blocking. 

With the current development cycle for 3.15, the function arch_random_refill 
is added as presented in [1]. It now uses RDSEED instead of RDRAND. Yet, the 
way this function is called in random_read seems (as I have no system with an 
RDSEED, I cannot test) to show the very same behavior as the aforementioned 
RDRAND patch: the blocking behavior of /dev/random will be gone and RDSEED 
will monopolize the noise sources in case of a user space hog.

Note, I do not see an issue with the patch that adds RDSEED as part of 
add_interrupt_randomness outlined in [2]. The reason is that this patch does 
not monopolizes the noise sources.

I do not want to imply that Intel (or any other chip manufacturer that will 
hook into arch_random_refill) intentionally provides bad entropy (and this 
email shall not start a discussion about entropy again), but I would like to 
be able to only use noise sources that I can fully audit. As it is with 
hardware, I am not able to see what it is doing.

Thus, may I ask that arch_random_refill is revised such that it will not 
monopolize the noise sources? If somebody wants that, he can easily use rngd.

[1] https://lkml.org/lkml/2014/3/4/968

[2] https://lkml.org/lkml/2014/3/4/934

Thanks
Stephan
-- 
| Cui bono? |

Re: arch_random_refill

[H. Peter Anvin]

On 05/11/2014 04:01 PM, Stephan Mueller wrote:
> Hi Peter,
> 
> some time back when the RDRAND instruction was debated, a patch was offered 
> for driver/char/random.c that in essence turned /dev/random into a frontend 
> for RDRAND in case that instruction was available. The patch kind of 
> monopolized the noise sources such that if a user space "random number hog" 
> pulled data from /dev/random endlessly, the (almost) only noise source was 
> RDRAND. As that patch treated RDRAND to provide entropy, the blocking behavior 
> went away for /dev/random.
> 

This is false in a number of ways.

First of all... we NEVER pulled either /dev/random or /dev/urandom
directly from RDRAND.  We used RDRAND directly for kernel internal
randomness uses.  Users did object to this.

> That patch did not sit well with some developers and it got finally changed 
> such that the output of RDRAND is now just XORed with the output of the 
> "classic" /dev/random behavior -- /dev/random is still blocking. 

Mixing in RDRAND into /dev/random and /dev/urandom is actually

> With the current development cycle for 3.15, the function arch_random_refill 
> is added as presented in [1]. It now uses RDSEED instead of RDRAND. Yet, the 
> way this function is called in random_read seems (as I have no system with an 
> RDSEED, I cannot test) to show the very same behavior as the aforementioned 
> RDRAND patch: the blocking behavior of /dev/random will be gone and RDSEED 
> will monopolize the noise sources in case of a user space hog.

There is a huge difference between this and what people objected to
earlier: we filter everything through the kernel random number pool
system, which would require a herculean mathematical effort to reverse
even if the output of RDSEED was 100% predictable.

> Note, I do not see an issue with the patch that adds RDSEED as part of 
> add_interrupt_randomness outlined in [2]. The reason is that this patch does 
> not monopolizes the noise sources.
> 
> I do not want to imply that Intel (or any other chip manufacturer that will 
> hook into arch_random_refill) intentionally provides bad entropy (and this 
> email shall not start a discussion about entropy again), but I would like to 
> be able to only use noise sources that I can fully audit. As it is with 
> hardware, I am not able to see what it is doing.

I have to point out the irony in this given your previous proposals,
however...

> Thus, may I ask that arch_random_refill is revised such that it will not 
> monopolize the noise sources? If somebody wants that, he can easily use rngd.

Feel free to build the kernel without CONFIG_ARCH_RANDOM, or use the
"nordrand" option to the kernel.  These options are there for a reason.

Now when you mention it, though, the nordrand option should turn off
RDSEED as well as RDRAND.  It currently doesn't; that is a bug, plain
and simple.

	-hpa

Re: arch_random_refill

[Stephan Mueller]

Am Sonntag, 11. Mai 2014, 20:22:28 schrieb H. Peter Anvin:

Hi Peter,
> 
> > Note, I do not see an issue with the patch that adds RDSEED as part of
> > add_interrupt_randomness outlined in [2]. The reason is that this patch
> > does not monopolizes the noise sources.
> > 
> > I do not want to imply that Intel (or any other chip manufacturer that
> > will
> > hook into arch_random_refill) intentionally provides bad entropy (and this
> > email shall not start a discussion about entropy again), but I would like
> > to be able to only use noise sources that I can fully audit. As it is
> > with hardware, I am not able to see what it is doing.
> 
> I have to point out the irony in this given your previous proposals,
> however...

I guess that is the funny nature of entropy :-)

But in our current predicament, not everybody trusts a few potentially easily 
manipulated gates that have no other purpose than produce white noise which 
are developed by the biggest chip vendor in the US. Gates which have other 
purposes may not be that easily manipulated.
> 
> > Thus, may I ask that arch_random_refill is revised such that it will not
> > monopolize the noise sources? If somebody wants that, he can easily use
> > rngd.
> Feel free to build the kernel without CONFIG_ARCH_RANDOM, or use the
> "nordrand" option to the kernel.  These options are there for a reason.
> 
> Now when you mention it, though, the nordrand option should turn off
> RDSEED as well as RDRAND.  It currently doesn't; that is a bug, plain
> and simple.

Ohh, ok, thanks for fixing that. :-) 

Though what makes me wonder is the following: why are some RNGs forced to use 
the hw_random framework whereas some others are not? What is the driver for 
that?

The current state of random.c vs. drivers/char/hw_random and the strong in-
kernel separation between both makes me wonder. Isn't that all kind of 
inconsistent?

Ciao
Stephan
-- 
| Cui bono? |

Re: arch_random_refill

[H. Peter Anvin]

On 05/11/2014 08:36 PM, Stephan Mueller wrote:
> 
> Ohh, ok, thanks for fixing that. :-) 
> 
> Though what makes me wonder is the following: why are some RNGs forced to use 
> the hw_random framework whereas some others are not? What is the driver for 
> that?
> 
> The current state of random.c vs. drivers/char/hw_random and the strong in-
> kernel separation between both makes me wonder. Isn't that all kind of 
> inconsistent?
> 

The main differences are speed of access, trivial interface, and
architectural guarantees.  You also don't have to deal with enumeration,
DMA engines, interrupts, indirect access, or bus drivers, which all are
utterly unacceptable on a synchronous path.

That being said, it is getting clear that we most likely would be better
off with the kernel directly feeding from at least a subset of the
hw_random drivers, rather than waiting for user space to come along and
launch a daemon... after $DEITY knows how many other processes have
already been launched.  There are patches being worked on to make that
happen, although there are a fair number of potential issues, including
the fact that some of the hw_random drivers are believed to be dodgy --
for example, the TPM driver: some TPMs are believed to not contain any
entropy element and simply rely on a factory-seeded nonvolatile counter
(since the TPM has to have support for nonvolatile counters anyway, this
hardware is already present.)

	-hpa

Re: arch_random_refill

[H. Peter Anvin]

On 05/11/2014 08:36 PM, Stephan Mueller wrote:
> 
> But in our current predicament, not everybody trusts a few potentially easily 
> manipulated gates that have no other purpose than produce white noise which 
> are developed by the biggest chip vendor in the US. Gates which have other 
> purposes may not be that easily manipulated.
>

Incidentally, I disagree with the "easily manipulated" bit.  Yes, I have
seen the paper which says that you can do it in such a way that it
doesn't show up on *visual* examination.  However, put an electrical
probe on it and it shows up immediately.

	-hpa