From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756877AbbIXQwR (ORCPT ); Thu, 24 Sep 2015 12:52:17 -0400 Received: from zaphod.unpythonic.net ([76.79.27.186]:28513 "EHLO zaphod.unpythonic.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753189AbbIXQwQ (ORCPT ); Thu, 24 Sep 2015 12:52:16 -0400 Date: Thu, 24 Sep 2015 11:52:04 -0500 From: Jeff Epler To: Austin S Hemmelgarn Cc: "Theodore Ts'o" , Andi Kleen , linux-kernel@vger.kernel.org, kirill.shutemov@linux.intel.com, herbert@gondor.apana.org.au, Andi Kleen Subject: Re: [PATCH 1/3] Make /dev/urandom scalable Message-ID: <20150924165204.GA2835@unpythonic.net> References: <1442963767-14945-1-git-send-email-andi@firstfloor.org> <5603004A.20801@gmail.com> <20150923232841.GK1747@two.firstfloor.org> <5603E083.8020004@gmail.com> <20150924131235.GB6841@thunk.org> <56041E2C.2030602@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <56041E2C.2030602@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Sep 24, 2015 at 12:00:44PM -0400, Austin S Hemmelgarn wrote: > I've had cases where I've done thousands of dieharder runs, and it > failed almost 10% of the time, while stuff like mt19937 fails in > otherwise identical tests only about 1-2% of the time That is a startling result. Please say what architecture, kernel version, dieharder version and commandline arguments you are using to get 10% WEAK or FAILED assessments from dieharder on /dev/urandom. Since the structure of linux urandom involves taking a cryptographic hash the basic expectation is that it would fail statistical randomness tests at similar rates to e.g., dieharder's AES_OFB (-g 205) even in the absence of any entropy in the kernel pools. So if 10% failures at correct statistical tests can be replicated it is important and needs attention. I did take a few moments to look into this today and got starling failures (p-value 0.00000000) with e.g., dieharder -g 501 -d 10 (and a few other tests) using dieharder 3.31.1 on both debian linux-4.1-rt-amd64 and debian kfreebsd-10-amd64, but this seems to be an upstream bug known at least to debian and redhat, possibly fixed in current Fedora but apparently not in Debian. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=745742 https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=803292 if you have an affected version, these failures are seen only with -g 501, not with -g 200 < /dev/urandom. They are probably also not seen with 32-bit dieharder. diehard_parking_lot| 0| 12000| 100|0.00000000| FAILED diehard_2dsphere| 2| 8000| 100|0.00000000| FAILED diehard_3dsphere| 3| 4000| 100|0.00000000| FAILED diehard_squeeze| 0| 100000| 100|0.00000000| FAILED diehard_sums| 0| 100| 100|0.00000000| FAILED Jeff