Re: [PATCH 2/2] ath9k: export HW random number generator

Re: [PATCH 2/2] ath9k: export HW random number generator

[Oleksij Rempel]

[-- Attachment #1: Type: text/plain, Size: 9254 bytes --]

Am 27.07.2015 um 08:50 schrieb Pan, Miaoqing:
>  “fips_run_rng_test”  is legacy code, recommend to disable 'FIPS 140-2' test if to use 'rngd-tools’.

Ok, lets try simple compression. will it find enough pattern to do
compression?
Here what i get on my system:
output from /dev/random
-rw-rw-r-- 1 lex lex 2501678 Jul 27 12:01 random.out
-rw-rw-r-- 1 lex lex 2512892 Jul 27 12:01 random.out.bz2

after compression we got bigger file. i would expect it since we need to
store bzip header somewhere.

output from /dev/hwrng
-rw-rw-r-- 1 lex lex 2564096 Jul 27 11:36 hwrng.out
-rw-rw-r-- 1 lex lex 2468394 Jul 27 11:36 hwrng.out.bz2

Do i understand it correctly, in case of hwrng bzip was able to find
enough pattern to compressed the data? Even with format overhead?

I'm no an expert, help of an expert would be welcome, added some more
people to CC

> -Miaoqing
> 
> -----Original Message-----
> From: Oleksij Rempel [mailto:linux-YEK0n+YFykbzxQdaRaTXBw@public.gmane.org] 
> Sent: Sunday, July 26, 2015 3:41 PM
> To: Pan, Miaoqing; linville-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org
> Cc: linux-wireless-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; ath9k-devel
> Subject: Re: [PATCH 2/2] ath9k: export HW random number generator
> 
> Hi all,
> 
> i did rngtest on top of this patch. The results are incredibly bad, right now it is more a pattern generator not random number generator. Is it possible to fix it?
> 
> /home/lex# cat /dev/hwrng | rngtest -c 1000 rngtest 5 Copyright (c) 2004 by Henrique de Moraes Holschuh This is free software; see the source for copying conditions.  There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> 
> rngtest: starting FIPS tests...
> rngtest: bits received from input: 20000032
> rngtest: FIPS 140-2 successes: 0
> rngtest: FIPS 140-2 failures: 1000
> rngtest: FIPS 140-2(2001-10-10) Monobit: 27
> rngtest: FIPS 140-2(2001-10-10) Poker: 1000
> rngtest: FIPS 140-2(2001-10-10) Runs: 1000
> rngtest: FIPS 140-2(2001-10-10) Long run: 2
> rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
> rngtest: input channel speed: (min=1.879; avg=871.897; max=19531250.000)Kibits/s
> rngtest: FIPS tests speed: (min=19.443; avg=48.374; max=70.123)Mibits/s
> rngtest: Program run time: 23423736 microseconds
> 
> 
> 
> Am 15.07.2015 um 09:54 schrieb miaoqing-Rm6X0d1/PG5y9aJCnZT0Uw@public.gmane.org:
>> From: Miaoqing Pan <miaoqing-A+ZNKFmMK5xy9aJCnZT0Uw@public.gmane.org>
>>
>> We measured the FFT-based entropy in 3 ways, Shannon entropy, 
>> collision entropy, and directly measured min-entropy. Just to be 
>> conservative, we recommend the estimated min-Entropy to be
>> 10 bits per 16-bit value.
>>
>> Analysis was done by Jacobson,David(djacobso-Rm6X0d1/PG5y9aJCnZT0Uw@public.gmane.org).
>>
>> Signed-off-by: Miaoqing Pan <miaoqing-A+ZNKFmMK5xy9aJCnZT0Uw@public.gmane.org>
>> ---
>>  drivers/net/wireless/ath/ath9k/Kconfig  |  7 +++  
>> drivers/net/wireless/ath/ath9k/Makefile |  1 +  
>> drivers/net/wireless/ath/ath9k/ath9k.h  | 23 ++++++++++
>>  drivers/net/wireless/ath/ath9k/main.c   |  4 ++
>>  drivers/net/wireless/ath/ath9k/rng.c    | 75 +++++++++++++++++++++++++++++++++
>>  5 files changed, 110 insertions(+)
>>  create mode 100644 drivers/net/wireless/ath/ath9k/rng.c
>>
>> diff --git a/drivers/net/wireless/ath/ath9k/Kconfig 
>> b/drivers/net/wireless/ath/ath9k/Kconfig
>> index fee0cad..bde62ec9 100644
>> --- a/drivers/net/wireless/ath/ath9k/Kconfig
>> +++ b/drivers/net/wireless/ath/ath9k/Kconfig
>> @@ -176,3 +176,10 @@ config ATH9K_HTC_DEBUGFS
>>  	depends on ATH9K_HTC && DEBUG_FS
>>  	---help---
>>  	  Say Y, if you need access to ath9k_htc's statistics.
>> +
>> +config ATH9K_HWRNG
>> +	bool "Random number generator support"
>> +	depends on ATH9K && (HW_RANDOM = y || HW_RANDOM = ATH9K)
>> +	default y
>> +	---help---
>> +	  Provides a hardware random number generator to the kernel.
>> diff --git a/drivers/net/wireless/ath/ath9k/Makefile 
>> b/drivers/net/wireless/ath/ath9k/Makefile
>> index ecda613..76f9dc3 100644
>> --- a/drivers/net/wireless/ath/ath9k/Makefile
>> +++ b/drivers/net/wireless/ath/ath9k/Makefile
>> @@ -15,6 +15,7 @@ ath9k-$(CONFIG_ATH9K_DFS_DEBUGFS) += dfs_debug.o
>>  ath9k-$(CONFIG_ATH9K_DFS_CERTIFIED) += dfs.o
>>  ath9k-$(CONFIG_ATH9K_TX99) += tx99.o
>>  ath9k-$(CONFIG_ATH9K_WOW) += wow.o
>> +ath9k-$(CONFIG_ATH9K_HWRNG) += rng.o
>>  
>>  ath9k-$(CONFIG_ATH9K_DEBUGFS) += debug.o
>>  
>> diff --git a/drivers/net/wireless/ath/ath9k/ath9k.h 
>> b/drivers/net/wireless/ath/ath9k/ath9k.h
>> index a7a81b3..45596e5 100644
>> --- a/drivers/net/wireless/ath/ath9k/ath9k.h
>> +++ b/drivers/net/wireless/ath/ath9k/ath9k.h
>> @@ -23,6 +23,7 @@
>>  #include <linux/leds.h>
>>  #include <linux/completion.h>
>>  #include <linux/time.h>
>> +#include <linux/hw_random.h>
>>  
>>  #include "common.h"
>>  #include "debug.h"
>> @@ -1041,6 +1042,12 @@ struct ath_softc {
>>  	u32 wow_intr_before_sleep;
>>  	bool force_wow;
>>  #endif
>> +
>> +#ifdef CONFIG_ATH9K_HWRNG
>> +	struct hwrng rng;
>> +	bool rng_initialized;
>> +	u32 rng_last;
>> +#endif
>>  };
>>  
>>  /********/
>> @@ -1063,6 +1070,22 @@ static inline int ath9k_tx99_send(struct 
>> ath_softc *sc,  }  #endif /* CONFIG_ATH9K_TX99 */
>>  
>> +/***************************/
>> +/* Random Number Generator */
>> +/***************************/
>> +#ifdef CONFIG_ATH9K_HWRNG
>> +void ath9k_rng_register(struct ath_softc *sc); void 
>> +ath9k_rng_unregister(struct ath_softc *sc); #else static inline void 
>> +ath9k_rng_register(struct ath_softc *sc) { }
>> +
>> +static inline void ath9k_rng_unregister(struct ath_softc *sc) { } 
>> +#endif
>> +
>>  static inline void ath_read_cachesize(struct ath_common *common, int 
>> *csz)  {
>>  	common->bus_ops->read_cachesize(common, csz); diff --git 
>> a/drivers/net/wireless/ath/ath9k/main.c 
>> b/drivers/net/wireless/ath/ath9k/main.c
>> index cfd45cb..5916ab2 100644
>> --- a/drivers/net/wireless/ath/ath9k/main.c
>> +++ b/drivers/net/wireless/ath/ath9k/main.c
>> @@ -739,6 +739,8 @@ static int ath9k_start(struct ieee80211_hw *hw)
>>  
>>  	ath9k_ps_restore(sc);
>>  
>> +	ath9k_rng_register(sc);
>> +
>>  	return 0;
>>  }
>>  
>> @@ -828,6 +830,8 @@ static void ath9k_stop(struct ieee80211_hw *hw)
>>  
>>  	ath9k_deinit_channel_context(sc);
>>  
>> +	ath9k_rng_unregister(sc);
>> +
>>  	mutex_lock(&sc->mutex);
>>  
>>  	ath_cancel_work(sc);
>> diff --git a/drivers/net/wireless/ath/ath9k/rng.c 
>> b/drivers/net/wireless/ath/ath9k/rng.c
>> new file mode 100644
>> index 0000000..d8fa7a5
>> --- /dev/null
>> +++ b/drivers/net/wireless/ath/ath9k/rng.c
>> @@ -0,0 +1,75 @@
>> +/*
>> + * Copyright (c) 2015 Qualcomm Atheros, Inc.
>> + *
>> + * Permission to use, copy, modify, and/or distribute this software 
>> +for any
>> + * purpose with or without fee is hereby granted, provided that the 
>> +above
>> + * copyright notice and this permission notice appear in all copies.
>> + *
>> + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL 
>> +WARRANTIES
>> + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
>> + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE 
>> +LIABLE FOR
>> + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY 
>> +DAMAGES
>> + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN 
>> +AN
>> + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING 
>> +OUT OF
>> + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
>> + */
>> +
>> +#include "ath9k.h"
>> +#include "hw.h"
>> +#include "ar9003_phy.h"
>> +
>> +static int ath9k_rng_data_read(struct hwrng *rng, u32 *data) {
>> +	u32 v1, v2;
>> +	struct ath_softc *sc = (struct ath_softc *)rng->priv;
>> +	struct ath_hw *ah = sc->sc_ah;
>> +
>> +	ath9k_ps_wakeup(sc);
>> +
>> +	REG_RMW_FIELD(ah, AR_PHY_TEST, AR_PHY_TEST_BBB_OBS_SEL, 5);
>> +	REG_CLR_BIT(ah, AR_PHY_TEST, AR_PHY_TEST_RX_OBS_SEL_BIT5);
>> +	REG_RMW_FIELD(ah, AR_PHY_TEST_CTL_STATUS, 
>> +AR_PHY_TEST_CTL_RX_OBS_SEL, 0);
>> +
>> +	v1 = REG_READ(ah, AR_PHY_TST_ADC);
>> +	v2 = REG_READ(ah, AR_PHY_TST_ADC);
>> +
>> +	ath9k_ps_restore(sc);
>> +
>> +	/* wait for data ready */
>> +	if (v1 && v2 && sc->rng_last != v1 && v1 != v2) {
>> +		*data = (v1 & 0xffff) | (v2 << 16);
>> +		sc->rng_last = v2;
>> +
>> +		return sizeof(u32);
>> +	}
>> +
>> +	sc->rng_last = v2;
>> +
>> +	return 0;
>> +}
>> +
>> +void ath9k_rng_register(struct ath_softc *sc) {
>> +	struct ath_hw *ah = sc->sc_ah;
>> +
>> +	if (WARN_ON(sc->rng_initialized))
>> +		return;
>> +
>> +	if (!AR_SREV_9300_20_OR_LATER(ah))
>> +		return;
>> +
>> +	sc->rng.name = "ath9k";
>> +	sc->rng.data_read = ath9k_rng_data_read;
>> +	sc->rng.priv = (unsigned long)sc;
>> +
>> +	if (!hwrng_register(&sc->rng))
>> +		sc->rng_initialized = true;
>> +}
>> +
>> +void ath9k_rng_unregister(struct ath_softc *sc) {
>> +	if (sc->rng_initialized) {
>> +		hwrng_unregister(&sc->rng);
>> +		sc->rng_initialized = false;
>> +	}
>> +}
>>
> 
> 
> --
> Regards,
> Oleksij
> 


-- 
Regards,
Oleksij


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 213 bytes --]

Re: [PATCH 2/2] ath9k: export HW random number generator

[Stephan Mueller]

Am Montag, 27. Juli 2015, 12:45:29 schrieb Oleksij Rempel:

Hi Oleksij,

>Am 27.07.2015 um 08:50 schrieb Pan, Miaoqing:
>>  “fips_run_rng_test”  is legacy code, recommend to disable 'FIPS 140-2'
>>  test if to use 'rngd-tools’.
>Ok, lets try simple compression. will it find enough pattern to do
>compression?
>Here what i get on my system:
>output from /dev/random
>-rw-rw-r-- 1 lex lex 2501678 Jul 27 12:01 random.out
>-rw-rw-r-- 1 lex lex 2512892 Jul 27 12:01 random.out.bz2
>
>after compression we got bigger file. i would expect it since we need to
>store bzip header somewhere.
>
>output from /dev/hwrng
>-rw-rw-r-- 1 lex lex 2564096 Jul 27 11:36 hwrng.out
>-rw-rw-r-- 1 lex lex 2468394 Jul 27 11:36 hwrng.out.bz2
>
>Do i understand it correctly, in case of hwrng bzip was able to find
>enough pattern to compressed the data? Even with format overhead?
>
>I'm no an expert, help of an expert would be welcome, added some more
>people to CC

This one does not look good for a claim that the RNG produces white noise. An 
RNG that is wired up to /dev/hwrng should produce white noise. Either by 
having an appropriate noise source or by conditioning the output of the noise 
source.

When conditioning the output, you have to be careful about the entropy claim. 
For example, you cannot state that the data stream from your noise source has 
close to one bit of entropy for each obtained bit. Thus, the conditioner must 
ensure that the data from the noise source is collected and its entropy is 
maintained and accumulated.

However, the hwrandom framework does not provide any conditioning logic. And I 
would say that such conditioner logic should not reside in a driver either. I 
would say that the discussed RNG does not seem fit for hooking it up with the 
hwrandom framework.

Ciao
Stephan

Re: [PATCH 2/2] ath9k: export HW random number generator

[Sandy Harris]

On Mon, Jul 27, 2015 at 7:01 AM, Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org> wrote:

> This one does not look good for a claim that the RNG produces white noise. An
> RNG that is wired up to /dev/hwrng should produce white noise. Either by
> having an appropriate noise source or by conditioning the output of the noise
> source.

Yes.

> When conditioning the output, you have to be careful about the entropy claim.

A very good analysis of how to deal with this is in Denker's Turbid paper:
http://www.av8n.com/turbid/

In particular, see section 4.2 on Saturation

> However, the hwrandom framework does not provide any conditioning logic.

At first sight, this sounds like a blunder to me, but I have not
looked at hwrandom at all. Is there a rationale?

For example, not building conditioning into that driver would make
perfect sense if the output were just being fed into the random(4)
which does plenty of mixing. The only problem then would be to make
sure of giving random(4) reasonable entropy estimates.
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] ath9k: export HW random number generator

[Stephan Mueller]

Am Dienstag, 28. Juli 2015, 13:41:57 schrieb Sandy Harris:

Hi Sandy,
>
>> However, the hwrandom framework does not provide any conditioning logic.
>
>At first sight, this sounds like a blunder to me, but I have not
>looked at hwrandom at all. Is there a rationale?

I think hwrandom is solely a framework to hook up RNG devices to user space. 
There is no massaging of data coming from the HW RNGs. Usually those HW RNGs 
all have their own conditioner and there is no need to do a conditioning 
again.
>
>For example, not building conditioning into that driver would make
>perfect sense if the output were just being fed into the random(4)
>which does plenty of mixing. The only problem then would be to make
>sure of giving random(4) reasonable entropy estimates.

hwrandom *may* be used to feed into the entropy pools. But there is no 
technical guarantee for that. Furthermore, I have seen use cases where the 
output of hwrandom is used for other purposes.

Ciao
Stephan
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] ath9k: export HW random number generator

[Kalle Valo]

Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org> writes:

>>-rw-rw-r-- 1 lex lex 2564096 Jul 27 11:36 hwrng.out
>>-rw-rw-r-- 1 lex lex 2468394 Jul 27 11:36 hwrng.out.bz2
>>
>>Do i understand it correctly, in case of hwrng bzip was able to find
>>enough pattern to compressed the data? Even with format overhead?
>>
>>I'm no an expert, help of an expert would be welcome, added some more
>>people to CC
>
> This one does not look good for a claim that the RNG produces white noise. An 
> RNG that is wired up to /dev/hwrng should produce white noise. Either by 
> having an appropriate noise source or by conditioning the output of the noise 
> source.
>
> When conditioning the output, you have to be careful about the entropy claim. 
> For example, you cannot state that the data stream from your noise source has 
> close to one bit of entropy for each obtained bit. Thus, the conditioner must 
> ensure that the data from the noise source is collected and its entropy is 
> maintained and accumulated.
>
> However, the hwrandom framework does not provide any conditioning logic. And I 
> would say that such conditioner logic should not reside in a driver either. I 
> would say that the discussed RNG does not seem fit for hooking it up with the 
> hwrandom framework.

Based on the discussion I'm going to revert this patch, at least for
now.

-- 
Kalle Valo
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH 2/2] ath9k: export HW random number generator

[Pan, Miaoqing]

+Pouyan & David.

-----Original Message-----
From: Kalle Valo [mailto:kvalo-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org] 
Sent: Friday, July 31, 2015 3:09 PM
To: Stephan Mueller
Cc: Oleksij Rempel; Pan, Miaoqing; linville-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org; linux-wireless-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Theodore Ts'o; linux-crypto-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; nhorman-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org
Subject: Re: [PATCH 2/2] ath9k: export HW random number generator

Stephan Mueller <smueller-T9tCv8IpfcWELgA04lAiVw@public.gmane.org> writes:

>>-rw-rw-r-- 1 lex lex 2564096 Jul 27 11:36 hwrng.out
>>-rw-rw-r-- 1 lex lex 2468394 Jul 27 11:36 hwrng.out.bz2
>>
>>Do i understand it correctly, in case of hwrng bzip was able to find 
>>enough pattern to compressed the data? Even with format overhead?
>>
>>I'm no an expert, help of an expert would be welcome, added some more 
>>people to CC
>
> This one does not look good for a claim that the RNG produces white 
> noise. An RNG that is wired up to /dev/hwrng should produce white 
> noise. Either by having an appropriate noise source or by conditioning 
> the output of the noise source.
>
> When conditioning the output, you have to be careful about the entropy claim. 
> For example, you cannot state that the data stream from your noise 
> source has close to one bit of entropy for each obtained bit. Thus, 
> the conditioner must ensure that the data from the noise source is 
> collected and its entropy is maintained and accumulated.
>
> However, the hwrandom framework does not provide any conditioning 
> logic. And I would say that such conditioner logic should not reside 
> in a driver either. I would say that the discussed RNG does not seem 
> fit for hooking it up with the hwrandom framework.

Based on the discussion I'm going to revert this patch, at least for now.

--
Kalle Valo
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] ath9k: export HW random number generator

[Nick Kossifidis]

Hello all,

That was partially my intention too when I submitted
2aa56cca3571fd08c0c38f3e2d4bb0bfb3def3c5 which mixes FFT measurements
to the entropy pool without providing any estimation on entropy
(entropy estimation is the wrong approach, read the papers on fortuna
for more information on that). I believe that this approach is better
than mine because FFT data is too much (high throughput) and may have
a negative impact on the pool when mixed like this (even without the
entropy count) so Kale please revert my patch also (I was about to
submit a request for reverting it and writing something similar when
this thread came up). I believe that exporting the card as a hw rng
and letting a userspace helper do the whitening and the post
processing is the proper approach.

The idea behind exposing a device as a hw random number generator is
not that it's output is ready to be used. It's just an entropy source.
Almost every entropy source has some bias and no it's not always
expected from a hw rng to do whitening on it's own, it's actually more
secure to get the raw data and perform the whitening yourself because
if what you get from the hw rng is already being post-processed then
there is no way to know if someone has hidden a backdoor there (that
was the big discussion about Intel's RDRAND and the possibility of an
NSA backdoor in there -it could just be AES on CTR mode with a known
key/iv-, we get whitened data so we don't know if we can trust them).
A userspace helper is needed for proper whitening and statistical
analysis (and no fips is not enough, try dieharder and ent and then go
for more custom stuff) because we should keep the kernel clean. That's
what rngd was supposed to do (do the whitening on userspace and then
submit it back to the kernel's pool).

What's wrong with exporting the atheros cards as a hw random number
generator ? Is the recently added jitter RNG (currently exposed as an
RNG through the CryptoAPI) any better ? Does it have stronger entropy
claims ? It just passes the statistical tests, it doesn't mean it has
good "randomness" (let me remind you hpa's comments on HAVEGEd on
LinuxCon 2012), a CPU is a deterministic system, the fact that there
are many parameters on this system -not on all CPUs by the way, not
everything out there is a cisc with branch prediction etc- turns it to
a very complex system to analyze (chaotic behavior) which is good
enough for what we want but it doesn't compare to an actual physical
phenomenon such as E/M noise (btw a type of "jitter" is already mixed
on the /dev/random's entropy pool since 2013 when I submitted this
http://lkml.iu.edu/hypermail/linux/kernel/1211.2/00863.html -again
with no entropy estimation and only by using timers- but it's not
random in any way, it's just "too complicated to analyze" -that's what
hpa also told about HAVEGEd and I totally agree with him-).

The truth is that it's very hard to get good quality entropy in a way
that's guaranteed e.g. even in the atheros's case an attacker can bias
the RNG by hitting the card with a sine wave signal on the IF, only
quantum phenomena such as photon detection -e.g. through radioactive
decay- or even better measurements of the void based on Heisenberg's
uncertainty principle can be truly classified as random and not
always, we still have Bell's correlations such as entanglement -so
e.g. something that seems random to us locally might not be random to
the attacker, our system might be entangled with the attacker's
system-. Even the current hw rngs supported by the framework might
provide non-random data in case of hw failure (think of an RNG based
on avalanche effect with a broken diode), that's why we should always
do extensive post-processing and kernel is not the place for that.

Some more comments here: I see people trusting HAVEGEd and entropy
from sound cards, video cards etc and even provide some "proof" about
how good that is. It's all wrong ! First of all there is no definition
of randomness, if we could define randomness it wouldn't be random any
more (we 'd have some information about it), there is only the
uncertainty principle. Second you can't have claims on entropy bounds
because such systems behave differently under different circumstances
or different configurations. HAVEGEd for example gathers entropy based
on the assumption that we have context switching and a complex CPU so
that it's too complicated to guess what's running on the system (again
no strong claims on entropy bounds it's just "it's messy enough, it
passes the tests, so it's ok"). Try running it on a real-time system
with a single process and you can say goodbye to any entropy claims.
Sound cards ? If you don't properly configure your sound card (even in
the case of having nothing connected on the mic, just gathering
thermal noise from the resistor) you 'll end up with no entropy at all
! Most sound cards have a noise cancellation system or they have a
volume threshold so they won't give anything below that (all zeroes,
no entropy), others will "loop" the signal when you raise the volume
enough and while you think that you get something random, you actually
get the same signal "looped" multiple times due to interference
between the input and output (so again no entropy claims can be made).
Video cards or web cams ? It depends on what they see, put a strong
light on them and again your entropy claims are invalid.

Trying to estimate the entropy is also very hard because you can only
measure the entropy from your point of view, not the attacker's. Take
a simple AES on CTR mode with a known key/iv to the attacker. You can
get its output, run your entropy estimator and find it full of entropy
but what looks random to you it won't be random to the attacker and
that's why we care about "high quality" randomness right ? It's for
cryptographic / security purposes, otherwise /dev/urandom is more than
enough (actually is good enough for crypto most of the times too).

So please let's stop arguing about what is random, more random, higher
quality random etc. We have an entropy source here, let's use it.
We'll never be able to know how good it is anyway.


2015-07-31 3:39 GMT-05:00 Pan, Miaoqing <miaoqing@qti.qualcomm.com>:
> +Pouyan & David.
>
> -----Original Message-----
> From: Kalle Valo [mailto:kvalo@codeaurora.org]
> Sent: Friday, July 31, 2015 3:09 PM
> To: Stephan Mueller
> Cc: Oleksij Rempel; Pan, Miaoqing; linville@tuxdriver.com; linux-wireless@vger.kernel.org; Theodore Ts'o; linux-crypto@vger.kernel.org; nhorman@tuxdriver.com
> Subject: Re: [PATCH 2/2] ath9k: export HW random number generator
>
> Stephan Mueller <smueller@chronox.de> writes:
>
>>>-rw-rw-r-- 1 lex lex 2564096 Jul 27 11:36 hwrng.out
>>>-rw-rw-r-- 1 lex lex 2468394 Jul 27 11:36 hwrng.out.bz2
>>>
>>>Do i understand it correctly, in case of hwrng bzip was able to find
>>>enough pattern to compressed the data? Even with format overhead?
>>>
>>>I'm no an expert, help of an expert would be welcome, added some more
>>>people to CC
>>
>> This one does not look good for a claim that the RNG produces white
>> noise. An RNG that is wired up to /dev/hwrng should produce white
>> noise. Either by having an appropriate noise source or by conditioning
>> the output of the noise source.
>>
>> When conditioning the output, you have to be careful about the entropy claim.
>> For example, you cannot state that the data stream from your noise
>> source has close to one bit of entropy for each obtained bit. Thus,
>> the conditioner must ensure that the data from the noise source is
>> collected and its entropy is maintained and accumulated.
>>
>> However, the hwrandom framework does not provide any conditioning
>> logic. And I would say that such conditioner logic should not reside
>> in a driver either. I would say that the discussed RNG does not seem
>> fit for hooking it up with the hwrandom framework.
>
> Based on the discussion I'm going to revert this patch, at least for now.
>
> --
> Kalle Valo
> --
> To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
GPG ID: 0xEE878588
As you read this post global entropy rises. Have Fun ;-)
Nick

Re: [PATCH 2/2] ath9k: export HW random number generator

[Nick Kossifidis]

Just a question to the Qualcomm devs: You are reading the TEST_DAC
register after switching the PHY to test mode or something. How would
that affect the card's behavior (e.g. if it gets called very
frequently) ? Is it possible to have packet loss or disconnections
because of that ? Also I notice that you read the register twice ?
What's the format of the data you get ?

My approach right now is to get FFT samples from the spectral scan
feature on "background" mode (when the card is scanning) so I get much
more throughput out of that (it's not MMIO, it's dma) and it's more
passive because it's data the card already gathers as part of its
radar detection loop. However I grab the data through debugfs (through
the current interface we have for spectral scan), I'd like to expose
this as a hw rng but I 'm still looking on the proper way of doing it
without having to buffer all this data of put them somewhere until a
consumer comes up. Maybe we could combine the two, provide high
throughput FFT samples (the raw data, without the formatting) when a
spectral scan is happening and when there are no spectral data -and if
your approach doesn't affect the card's operation- we could use that
for backup.


2015-11-07 17:39 GMT-06:00 Nick Kossifidis <mickflemm@gmail.com>:
> Hello all,
>
> That was partially my intention too when I submitted
> 2aa56cca3571fd08c0c38f3e2d4bb0bfb3def3c5 which mixes FFT measurements
> to the entropy pool without providing any estimation on entropy
> (entropy estimation is the wrong approach, read the papers on fortuna
> for more information on that). I believe that this approach is better
> than mine because FFT data is too much (high throughput) and may have
> a negative impact on the pool when mixed like this (even without the
> entropy count) so Kale please revert my patch also (I was about to
> submit a request for reverting it and writing something similar when
> this thread came up). I believe that exporting the card as a hw rng
> and letting a userspace helper do the whitening and the post
> processing is the proper approach.
>
> The idea behind exposing a device as a hw random number generator is
> not that it's output is ready to be used. It's just an entropy source.
> Almost every entropy source has some bias and no it's not always
> expected from a hw rng to do whitening on it's own, it's actually more
> secure to get the raw data and perform the whitening yourself because
> if what you get from the hw rng is already being post-processed then
> there is no way to know if someone has hidden a backdoor there (that
> was the big discussion about Intel's RDRAND and the possibility of an
> NSA backdoor in there -it could just be AES on CTR mode with a known
> key/iv-, we get whitened data so we don't know if we can trust them).
> A userspace helper is needed for proper whitening and statistical
> analysis (and no fips is not enough, try dieharder and ent and then go
> for more custom stuff) because we should keep the kernel clean. That's
> what rngd was supposed to do (do the whitening on userspace and then
> submit it back to the kernel's pool).
>
> What's wrong with exporting the atheros cards as a hw random number
> generator ? Is the recently added jitter RNG (currently exposed as an
> RNG through the CryptoAPI) any better ? Does it have stronger entropy
> claims ? It just passes the statistical tests, it doesn't mean it has
> good "randomness" (let me remind you hpa's comments on HAVEGEd on
> LinuxCon 2012), a CPU is a deterministic system, the fact that there
> are many parameters on this system -not on all CPUs by the way, not
> everything out there is a cisc with branch prediction etc- turns it to
> a very complex system to analyze (chaotic behavior) which is good
> enough for what we want but it doesn't compare to an actual physical
> phenomenon such as E/M noise (btw a type of "jitter" is already mixed
> on the /dev/random's entropy pool since 2013 when I submitted this
> http://lkml.iu.edu/hypermail/linux/kernel/1211.2/00863.html -again
> with no entropy estimation and only by using timers- but it's not
> random in any way, it's just "too complicated to analyze" -that's what
> hpa also told about HAVEGEd and I totally agree with him-).
>
> The truth is that it's very hard to get good quality entropy in a way
> that's guaranteed e.g. even in the atheros's case an attacker can bias
> the RNG by hitting the card with a sine wave signal on the IF, only
> quantum phenomena such as photon detection -e.g. through radioactive
> decay- or even better measurements of the void based on Heisenberg's
> uncertainty principle can be truly classified as random and not
> always, we still have Bell's correlations such as entanglement -so
> e.g. something that seems random to us locally might not be random to
> the attacker, our system might be entangled with the attacker's
> system-. Even the current hw rngs supported by the framework might
> provide non-random data in case of hw failure (think of an RNG based
> on avalanche effect with a broken diode), that's why we should always
> do extensive post-processing and kernel is not the place for that.
>
> Some more comments here: I see people trusting HAVEGEd and entropy
> from sound cards, video cards etc and even provide some "proof" about
> how good that is. It's all wrong ! First of all there is no definition
> of randomness, if we could define randomness it wouldn't be random any
> more (we 'd have some information about it), there is only the
> uncertainty principle. Second you can't have claims on entropy bounds
> because such systems behave differently under different circumstances
> or different configurations. HAVEGEd for example gathers entropy based
> on the assumption that we have context switching and a complex CPU so
> that it's too complicated to guess what's running on the system (again
> no strong claims on entropy bounds it's just "it's messy enough, it
> passes the tests, so it's ok"). Try running it on a real-time system
> with a single process and you can say goodbye to any entropy claims.
> Sound cards ? If you don't properly configure your sound card (even in
> the case of having nothing connected on the mic, just gathering
> thermal noise from the resistor) you 'll end up with no entropy at all
> ! Most sound cards have a noise cancellation system or they have a
> volume threshold so they won't give anything below that (all zeroes,
> no entropy), others will "loop" the signal when you raise the volume
> enough and while you think that you get something random, you actually
> get the same signal "looped" multiple times due to interference
> between the input and output (so again no entropy claims can be made).
> Video cards or web cams ? It depends on what they see, put a strong
> light on them and again your entropy claims are invalid.
>
> Trying to estimate the entropy is also very hard because you can only
> measure the entropy from your point of view, not the attacker's. Take
> a simple AES on CTR mode with a known key/iv to the attacker. You can
> get its output, run your entropy estimator and find it full of entropy
> but what looks random to you it won't be random to the attacker and
> that's why we care about "high quality" randomness right ? It's for
> cryptographic / security purposes, otherwise /dev/urandom is more than
> enough (actually is good enough for crypto most of the times too).
>
> So please let's stop arguing about what is random, more random, higher
> quality random etc. We have an entropy source here, let's use it.
> We'll never be able to know how good it is anyway.
>
>
> 2015-07-31 3:39 GMT-05:00 Pan, Miaoqing <miaoqing@qti.qualcomm.com>:
>> +Pouyan & David.
>>
>> -----Original Message-----
>> From: Kalle Valo [mailto:kvalo@codeaurora.org]
>> Sent: Friday, July 31, 2015 3:09 PM
>> To: Stephan Mueller
>> Cc: Oleksij Rempel; Pan, Miaoqing; linville@tuxdriver.com; linux-wireless@vger.kernel.org; Theodore Ts'o; linux-crypto@vger.kernel.org; nhorman@tuxdriver.com
>> Subject: Re: [PATCH 2/2] ath9k: export HW random number generator
>>
>> Stephan Mueller <smueller@chronox.de> writes:
>>
>>>>-rw-rw-r-- 1 lex lex 2564096 Jul 27 11:36 hwrng.out
>>>>-rw-rw-r-- 1 lex lex 2468394 Jul 27 11:36 hwrng.out.bz2
>>>>
>>>>Do i understand it correctly, in case of hwrng bzip was able to find
>>>>enough pattern to compressed the data? Even with format overhead?
>>>>
>>>>I'm no an expert, help of an expert would be welcome, added some more
>>>>people to CC
>>>
>>> This one does not look good for a claim that the RNG produces white
>>> noise. An RNG that is wired up to /dev/hwrng should produce white
>>> noise. Either by having an appropriate noise source or by conditioning
>>> the output of the noise source.
>>>
>>> When conditioning the output, you have to be careful about the entropy claim.
>>> For example, you cannot state that the data stream from your noise
>>> source has close to one bit of entropy for each obtained bit. Thus,
>>> the conditioner must ensure that the data from the noise source is
>>> collected and its entropy is maintained and accumulated.
>>>
>>> However, the hwrandom framework does not provide any conditioning
>>> logic. And I would say that such conditioner logic should not reside
>>> in a driver either. I would say that the discussed RNG does not seem
>>> fit for hooking it up with the hwrandom framework.
>>
>> Based on the discussion I'm going to revert this patch, at least for now.
>>
>> --
>> Kalle Valo
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>
> --
> GPG ID: 0xEE878588
> As you read this post global entropy rises. Have Fun ;-)
> Nick



-- 
GPG ID: 0xEE878588
As you read this post global entropy rises. Have Fun ;-)
Nick

Re: [PATCH 2/2] ath9k: export HW random number generator

[Kalle Valo]

Nick Kossifidis <mickflemm@gmail.com> writes:

> That was partially my intention too when I submitted
> 2aa56cca3571fd08c0c38f3e2d4bb0bfb3def3c5 which mixes FFT measurements
> to the entropy pool without providing any estimation on entropy
> (entropy estimation is the wrong approach, read the papers on fortuna
> for more information on that). I believe that this approach is better
> than mine because FFT data is too much (high throughput) and may have
> a negative impact on the pool when mixed like this (even without the
> entropy count) so Kale please revert my patch also (I was about to
> submit a request for reverting it and writing something similar when
> this thread came up).

Could you submit the revert as a patch and with the explonation in the
commit log, please?

-- 
Kalle Valo