Re: [PATCH] Add carta_random32() library routine

Re: [PATCH] Add carta_random32() library routine

[Jeff Garzik]

Linux Kernel Mailing List wrote:
> commit e0ab2928cc2202f13f0574d4c6f567f166d307eb
> tree 3df0b8e340b1a98cd8a2daa19672ff008e8fb7f9
> parent b611967de4dc5c52049676c4369dcac622a7cdfe
> author Stephane Eranian <eranian@hpl.hp.com> 1160554905 -0700
> committer Linus Torvalds <torvalds@g5.osdl.org> 1160590461 -0700
> 
> [PATCH] Add carta_random32() library routine
> 
> This is a follow-up patch based on the review for perfmon2.  This patch
> adds the carta_random32() library routine + carta_random32.h header file.
> 
> This is fast, simple, and efficient pseudo number generator algorithm.  We
> use it in perfmon2 to randomize the sampling periods.  In this context, we
> do not need any fancy randomizer.

hrm, does this really warrant inclusion into every kernel build, on 
every platform?

	Jeff

Re: [PATCH] Add carta_random32() library routine

[Stephen Hemminger]

On Wed, 11 Oct 2006 15:22:57 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> Linux Kernel Mailing List wrote:
> > commit e0ab2928cc2202f13f0574d4c6f567f166d307eb
> > tree 3df0b8e340b1a98cd8a2daa19672ff008e8fb7f9
> > parent b611967de4dc5c52049676c4369dcac622a7cdfe
> > author Stephane Eranian <eranian@hpl.hp.com> 1160554905 -0700
> > committer Linus Torvalds <torvalds@g5.osdl.org> 1160590461 -0700
> > 
> > [PATCH] Add carta_random32() library routine
> > 
> > This is a follow-up patch based on the review for perfmon2.  This patch
> > adds the carta_random32() library routine + carta_random32.h header file.
> > 
> > This is fast, simple, and efficient pseudo number generator algorithm.  We
> > use it in perfmon2 to randomize the sampling periods.  In this context, we
> > do not need any fancy randomizer.
> 
> hrm, does this really warrant inclusion into every kernel build, on 
> every platform?
> 
> 	Jeff
> 

Wouldn't existing net_random() work?

-- 
Stephen Hemminger <shemminger@osdl.org>

Re: [PATCH] Add carta_random32() library routine

[Andrew Morton]

On Wed, 11 Oct 2006 12:29:38 -0700
Stephen Hemminger <shemminger@osdl.org> wrote:

> On Wed, 11 Oct 2006 15:22:57 -0400
> Jeff Garzik <jeff@garzik.org> wrote:
> 
> > Linux Kernel Mailing List wrote:
> > > commit e0ab2928cc2202f13f0574d4c6f567f166d307eb
> > > tree 3df0b8e340b1a98cd8a2daa19672ff008e8fb7f9
> > > parent b611967de4dc5c52049676c4369dcac622a7cdfe
> > > author Stephane Eranian <eranian@hpl.hp.com> 1160554905 -0700
> > > committer Linus Torvalds <torvalds@g5.osdl.org> 1160590461 -0700
> > > 
> > > [PATCH] Add carta_random32() library routine
> > > 
> > > This is a follow-up patch based on the review for perfmon2.  This patch
> > > adds the carta_random32() library routine + carta_random32.h header file.
> > > 
> > > This is fast, simple, and efficient pseudo number generator algorithm.  We
> > > use it in perfmon2 to randomize the sampling periods.  In this context, we
> > > do not need any fancy randomizer.
> > 
> > hrm, does this really warrant inclusion into every kernel build, on 
> > every platform?
> > 
> > 	Jeff
> > 
> 
> Wouldn't existing net_random() work?
> 

It might do, but someone went and hid it in networking and nobody knew
about it.

Stephane?

[PATCH] rename net_random to random32

[Stephen Hemminger]

Make net_random() more widely available by calling it random32

Signed-off-by: Stephen Hemminger <shemminger@osdl.org>
---
 include/linux/net.h    |    5 +-
 include/linux/random.h |    3 +
 lib/Makefile           |    2 -
 lib/random32.c         |  135 ++++++++++++++++++++++++++++++++++++++++++++++++
 net/core/dev.c         |    2 -
 net/core/utils.c       |  116 -----------------------------------------
 6 files changed, 141 insertions(+), 122 deletions(-)

diff --git a/include/linux/net.h b/include/linux/net.h
index c257f71..5d4240e 100644
--- a/include/linux/net.h
+++ b/include/linux/net.h
@@ -193,9 +193,8 @@ extern int 	     sock_map_fd(struct sock
 extern struct socket *sockfd_lookup(int fd, int *err);
 #define		     sockfd_put(sock) fput(sock->file)
 extern int	     net_ratelimit(void);
-extern unsigned long net_random(void);
-extern void	     net_srandom(unsigned long);
-extern void	     net_random_init(void);
+
+#define net_random()	random32()
 
 extern int   	     kernel_sendmsg(struct socket *sock, struct msghdr *msg,
 				    struct kvec *vec, size_t num, size_t len);
diff --git a/include/linux/random.h b/include/linux/random.h
index 5d6456b..0248b30 100644
--- a/include/linux/random.h
+++ b/include/linux/random.h
@@ -69,6 +69,9 @@ #endif
 unsigned int get_random_int(void);
 unsigned long randomize_range(unsigned long start, unsigned long end, unsigned long len);
 
+u32 random32(void);
+void srandom32(u32 seed);
+
 #endif /* __KERNEL___ */
 
 #endif /* _LINUX_RANDOM_H */
diff --git a/lib/Makefile b/lib/Makefile
index 59070db..bffdc82 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -5,7 +5,7 @@ #
 lib-y := ctype.o string.o vsprintf.o cmdline.o \
 	 bust_spinlocks.o rbtree.o radix-tree.o dump_stack.o \
 	 idr.o div64.o int_sqrt.o bitmap.o extable.o prio_tree.o \
-	 sha1.o irq_regs.o carta_random32.o
+	 sha1.o irq_regs.o random32.o carta_random32.o
 
 lib-$(CONFIG_MMU) += ioremap.o
 lib-$(CONFIG_SMP) += cpumask.o
diff --git a/lib/random32.c b/lib/random32.c
new file mode 100644
index 0000000..a905769
--- /dev/null
+++ b/lib/random32.c
@@ -0,0 +1,135 @@
+/*
+  This is a maximally equidistributed combined Tausworthe generator
+  based on code from GNU Scientific Library 1.5 (30 Jun 2004)
+
+   x_n = (s1_n ^ s2_n ^ s3_n) 
+
+   s1_{n+1} = (((s1_n & 4294967294) <<12) ^ (((s1_n <<13) ^ s1_n) >>19))
+   s2_{n+1} = (((s2_n & 4294967288) << 4) ^ (((s2_n << 2) ^ s2_n) >>25))
+   s3_{n+1} = (((s3_n & 4294967280) <<17) ^ (((s3_n << 3) ^ s3_n) >>11))
+
+   The period of this generator is about 2^88.
+
+   From: P. L'Ecuyer, "Maximally Equidistributed Combined Tausworthe
+   Generators", Mathematics of Computation, 65, 213 (1996), 203--213.
+
+   This is available on the net from L'Ecuyer's home page,
+
+   http://www.iro.umontreal.ca/~lecuyer/myftp/papers/tausme.ps
+   ftp://ftp.iro.umontreal.ca/pub/simulation/lecuyer/papers/tausme.ps 
+
+   There is an erratum in the paper "Tables of Maximally
+   Equidistributed Combined LFSR Generators", Mathematics of
+   Computation, 68, 225 (1999), 261--269:
+   http://www.iro.umontreal.ca/~lecuyer/myftp/papers/tausme2.ps
+
+        ... the k_j most significant bits of z_j must be non-
+        zero, for each j. (Note: this restriction also applies to the 
+        computer code given in [4], but was mistakenly not mentioned in
+        that paper.)
+   
+   This affects the seeding procedure by imposing the requirement
+   s1 > 1, s2 > 7, s3 > 15.
+
+*/
+struct nrnd_state {
+	u32 s1, s2, s3;
+};
+
+static DEFINE_PER_CPU(struct nrnd_state, net_rand_state);
+
+static u32 __random32(struct nrnd_state *state)
+{
+#define TAUSWORTHE(s,a,b,c,d) ((s&c)<<d) ^ (((s <<a) ^ s)>>b)
+
+	state->s1 = TAUSWORTHE(state->s1, 13, 19, 4294967294UL, 12);
+	state->s2 = TAUSWORTHE(state->s2, 2, 25, 4294967288UL, 4);
+	state->s3 = TAUSWORTHE(state->s3, 3, 11, 4294967280UL, 17);
+
+	return (state->s1 ^ state->s2 ^ state->s3);
+}
+
+static void __set_random32(struct nrnd_state *state, unsigned long s)
+{
+	if (s == 0)
+		s = 1;      /* default seed is 1 */
+
+#define LCG(n) (69069 * n)
+	state->s1 = LCG(s);
+	state->s2 = LCG(state->s1);
+	state->s3 = LCG(state->s2);
+
+	/* "warm it up" */
+	__random32(state);
+	__random32(state);
+	__random32(state);
+	__random32(state);
+	__random32(state);
+	__random32(state);
+}
+
+/**
+ *	random32 - pseudo random number generator
+ *	
+ *	A 32 bit pseudo-random number is generated using a fast
+ *	algorithm suitable for simulation. This algorithm is NOT
+ *	considered safe for cryptographic use.
+ */
+u32 random32(void)
+{
+	unsigned long r;
+	struct nrnd_state *state = &get_cpu_var(net_rand_state);
+	r = __random32(state);
+	put_cpu_var(state);
+	return r;
+}
+EXPORT_SYMBOL(random32);
+
+/**
+ *	srandom32 - add entropy to pseudo random number generator
+ *	@seed: seed value
+ *
+ *	Add some additional seeding to the random32() pool.
+ *	Note: this pool is per cpu so it only affects current CPU.
+ */
+void srandom32(u32 entropy)
+{
+	struct nrnd_state *state = &get_cpu_var(net_rand_state);
+	__net_srandom(state, state->s1^entropy);
+	put_cpu_var(state);
+}
+EXPORT_SYMBOL(srandom32);
+
+/*
+ *	Generate some initially weak seeding values to allow
+ *	to start the random32() engine.
+ */
+static void __init random32_init(void)
+{
+	int i;
+
+	for_each_possible_cpu(i) {
+		struct nrnd_state *state = &per_cpu(net_rand_state,i);
+		__net_srandom(state, i+jiffies);
+	}
+}
+core_initcall(random32_init);
+
+/*
+ *	Generate better values after random number generator
+ *	is fully initalized.
+ */
+static void __init random32_reseed(void)
+{
+	int i;
+	unsigned long seed;
+
+	for_each_possible_cpu(i) {
+		struct nrnd_state *state = &per_cpu(net_rand_state,i);
+
+		get_random_bytes(&seed, sizeof(seed));
+		__net_srandom(state, seed);
+	}
+	return 0;
+}
+late_initcall(random32_reseed);
diff --git a/net/core/dev.c b/net/core/dev.c
index 4d891be..81c426a 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -3502,8 +3502,6 @@ static int __init net_dev_init(void)
 
 	BUG_ON(!dev_boot_phase);
 
-	net_random_init();
-
 	if (dev_proc_init())
 		goto out;
 
diff --git a/net/core/utils.c b/net/core/utils.c
index 94c5d76..d93fe64 100644
--- a/net/core/utils.c
+++ b/net/core/utils.c
@@ -30,119 +30,6 @@ #include <asm/byteorder.h>
 #include <asm/system.h>
 #include <asm/uaccess.h>
 
-/*
-  This is a maximally equidistributed combined Tausworthe generator
-  based on code from GNU Scientific Library 1.5 (30 Jun 2004)
-
-   x_n = (s1_n ^ s2_n ^ s3_n) 
-
-   s1_{n+1} = (((s1_n & 4294967294) <<12) ^ (((s1_n <<13) ^ s1_n) >>19))
-   s2_{n+1} = (((s2_n & 4294967288) << 4) ^ (((s2_n << 2) ^ s2_n) >>25))
-   s3_{n+1} = (((s3_n & 4294967280) <<17) ^ (((s3_n << 3) ^ s3_n) >>11))
-
-   The period of this generator is about 2^88.
-
-   From: P. L'Ecuyer, "Maximally Equidistributed Combined Tausworthe
-   Generators", Mathematics of Computation, 65, 213 (1996), 203--213.
-
-   This is available on the net from L'Ecuyer's home page,
-
-   http://www.iro.umontreal.ca/~lecuyer/myftp/papers/tausme.ps
-   ftp://ftp.iro.umontreal.ca/pub/simulation/lecuyer/papers/tausme.ps 
-
-   There is an erratum in the paper "Tables of Maximally
-   Equidistributed Combined LFSR Generators", Mathematics of
-   Computation, 68, 225 (1999), 261--269:
-   http://www.iro.umontreal.ca/~lecuyer/myftp/papers/tausme2.ps
-
-        ... the k_j most significant bits of z_j must be non-
-        zero, for each j. (Note: this restriction also applies to the 
-        computer code given in [4], but was mistakenly not mentioned in
-        that paper.)
-   
-   This affects the seeding procedure by imposing the requirement
-   s1 > 1, s2 > 7, s3 > 15.
-
-*/
-struct nrnd_state {
-	u32 s1, s2, s3;
-};
-
-static DEFINE_PER_CPU(struct nrnd_state, net_rand_state);
-
-static u32 __net_random(struct nrnd_state *state)
-{
-#define TAUSWORTHE(s,a,b,c,d) ((s&c)<<d) ^ (((s <<a) ^ s)>>b)
-
-	state->s1 = TAUSWORTHE(state->s1, 13, 19, 4294967294UL, 12);
-	state->s2 = TAUSWORTHE(state->s2, 2, 25, 4294967288UL, 4);
-	state->s3 = TAUSWORTHE(state->s3, 3, 11, 4294967280UL, 17);
-
-	return (state->s1 ^ state->s2 ^ state->s3);
-}
-
-static void __net_srandom(struct nrnd_state *state, unsigned long s)
-{
-	if (s == 0)
-		s = 1;      /* default seed is 1 */
-
-#define LCG(n) (69069 * n)
-	state->s1 = LCG(s);
-	state->s2 = LCG(state->s1);
-	state->s3 = LCG(state->s2);
-
-	/* "warm it up" */
-	__net_random(state);
-	__net_random(state);
-	__net_random(state);
-	__net_random(state);
-	__net_random(state);
-	__net_random(state);
-}
-
-
-unsigned long net_random(void)
-{
-	unsigned long r;
-	struct nrnd_state *state = &get_cpu_var(net_rand_state);
-	r = __net_random(state);
-	put_cpu_var(state);
-	return r;
-}
-
-
-void net_srandom(unsigned long entropy)
-{
-	struct nrnd_state *state = &get_cpu_var(net_rand_state);
-	__net_srandom(state, state->s1^entropy);
-	put_cpu_var(state);
-}
-
-void __init net_random_init(void)
-{
-	int i;
-
-	for_each_possible_cpu(i) {
-		struct nrnd_state *state = &per_cpu(net_rand_state,i);
-		__net_srandom(state, i+jiffies);
-	}
-}
-
-static int net_random_reseed(void)
-{
-	int i;
-	unsigned long seed;
-
-	for_each_possible_cpu(i) {
-		struct nrnd_state *state = &per_cpu(net_rand_state,i);
-
-		get_random_bytes(&seed, sizeof(seed));
-		__net_srandom(state, seed);
-	}
-	return 0;
-}
-late_initcall(net_random_reseed);
-
 int net_msg_cost = 5*HZ;
 int net_msg_burst = 10;
 
@@ -153,10 +40,7 @@ int net_ratelimit(void)
 {
 	return __printk_ratelimit(net_msg_cost, net_msg_burst);
 }
-
-EXPORT_SYMBOL(net_random);
 EXPORT_SYMBOL(net_ratelimit);
-EXPORT_SYMBOL(net_srandom);
 
 /*
  * Convert an ASCII string to binary IP.
-- 
1.4.1

Re: [PATCH] rename net_random to random32

[David Miller]

From: Stephen Hemminger <shemminger@osdl.org>
Date: Thu, 12 Oct 2006 10:26:38 -0700

> 
> Make net_random() more widely available by calling it random32
> 
> Signed-off-by: Stephen Hemminger <shemminger@osdl.org>

Signed-off-by: David S. Miller <davem@davemloft.net>

Re: [PATCH] rename net_random to random32

[Adrian Bunk]

On Thu, Oct 12, 2006 at 10:26:38AM -0700, Stephen Hemminger wrote:
> 
> Make net_random() more widely available by calling it random32
>...
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -5,7 +5,7 @@ #
>  lib-y := ctype.o string.o vsprintf.o cmdline.o \
>  	 bust_spinlocks.o rbtree.o radix-tree.o dump_stack.o \
>  	 idr.o div64.o int_sqrt.o bitmap.o extable.o prio_tree.o \
> -	 sha1.o irq_regs.o carta_random32.o
> +	 sha1.o irq_regs.o random32.o carta_random32.o
>...
> --- /dev/null
> +++ b/lib/random32.c
>...
> +EXPORT_SYMBOL(random32);
>...
> +EXPORT_SYMBOL(srandom32);
>...

EXPORT_SYMBOL's in lib-y are latent bugs (IMHO kbuild should error
on them):

The problem is that if only modules use these functions, they will be 
omitted from the kernel image.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH] rename net_random to random32

[Stephen Hemminger]

On Fri, 13 Oct 2006 20:19:22 +0200
Adrian Bunk <bunk@stusta.de> wrote:

> On Thu, Oct 12, 2006 at 10:26:38AM -0700, Stephen Hemminger wrote:
> > 
> > Make net_random() more widely available by calling it random32
> >...
> > --- a/lib/Makefile
> > +++ b/lib/Makefile
> > @@ -5,7 +5,7 @@ #
> >  lib-y := ctype.o string.o vsprintf.o cmdline.o \
> >  	 bust_spinlocks.o rbtree.o radix-tree.o dump_stack.o \
> >  	 idr.o div64.o int_sqrt.o bitmap.o extable.o prio_tree.o \
> > -	 sha1.o irq_regs.o carta_random32.o
> > +	 sha1.o irq_regs.o random32.o carta_random32.o
> >...
> > --- /dev/null
> > +++ b/lib/random32.c
> >...
> > +EXPORT_SYMBOL(random32);
> >...
> > +EXPORT_SYMBOL(srandom32);
> >...
> 
> EXPORT_SYMBOL's in lib-y are latent bugs (IMHO kbuild should error
> on them):
> 
> The problem is that if only modules use these functions, they will be 
> omitted from the kernel image.
> 
> cu
> Adrian
> 

The main net code uses the code so it isn't really an issue in this case.

Re: [PATCH] rename net_random to random32

[Andrew Morton]

On Fri, 13 Oct 2006 20:19:22 +0200
Adrian Bunk <bunk@stusta.de> wrote:

> On Thu, Oct 12, 2006 at 10:26:38AM -0700, Stephen Hemminger wrote:
> > 
> > Make net_random() more widely available by calling it random32
> >...
> > --- a/lib/Makefile
> > +++ b/lib/Makefile
> > @@ -5,7 +5,7 @@ #
> >  lib-y := ctype.o string.o vsprintf.o cmdline.o \
> >  	 bust_spinlocks.o rbtree.o radix-tree.o dump_stack.o \
> >  	 idr.o div64.o int_sqrt.o bitmap.o extable.o prio_tree.o \
> > -	 sha1.o irq_regs.o carta_random32.o
> > +	 sha1.o irq_regs.o random32.o carta_random32.o
> >...
> > --- /dev/null
> > +++ b/lib/random32.c
> >...
> > +EXPORT_SYMBOL(random32);
> >...
> > +EXPORT_SYMBOL(srandom32);
> >...
> 
> EXPORT_SYMBOL's in lib-y are latent bugs (IMHO kbuild should error
> on them):
> 
> The problem is that if only modules use these functions, they will be 
> omitted from the kernel image.
> 

Yeah, I always get those two confused.  I moved it to obj-y, thanks.

Re: [PATCH] rename net_random to random32

[David Miller]

From: Andrew Morton <akpm@osdl.org>
Date: Fri, 13 Oct 2006 12:09:56 -0700

> On Fri, 13 Oct 2006 20:19:22 +0200
> Adrian Bunk <bunk@stusta.de> wrote:
> 
> > EXPORT_SYMBOL's in lib-y are latent bugs (IMHO kbuild should error
> > on them):
> > 
> > The problem is that if only modules use these functions, they will be 
> > omitted from the kernel image.
> > 
> Yeah, I always get those two confused.  I moved it to obj-y, thanks.

Yep, good catch Adrian.  I've been burned by this one oto.

Re: [PATCH] rename net_random to random32

[Sam Ravnborg]

> EXPORT_SYMBOL's in lib-y are latent bugs (IMHO kbuild should error
> on them):

In practice module will not load becasue it cannot resolve symbol - right?

That aside we should not export symbols from lib-y. But I have not any good
idea how to catch this during build time unles doing some ugly grep magic.

One thing that could be done would be to check for a specific section
(__ksymtab_strings) in the .o file only for lib-y .o files.

The check could be something like:
objdump -h <file.o> | grep __ksymtab_strings

If grep give error code '0' then we exported a symbol.
I may try to cook up later this weekend.

        Sam

Re: [PATCH] rename net_random to random32

[Christoph Hellwig]

On Fri, Oct 13, 2006 at 08:19:22PM +0200, Adrian Bunk wrote:
> > --- /dev/null
> > +++ b/lib/random32.c
> >...
> > +EXPORT_SYMBOL(random32);
> >...
> > +EXPORT_SYMBOL(srandom32);
> >...
> 
> EXPORT_SYMBOL's in lib-y are latent bugs (IMHO kbuild should error
> on them):
> 
> The problem is that if only modules use these functions, they will be 
> omitted from the kernel image.

In fact we should really really just kill lib-y completely.  Leaving
the control of what gets built in to the linker shound like a really
bad idea in our enviroment.

Re: [PATCH] Add carta_random32() library routine

[Andrew Morton]

On Wed, 11 Oct 2006 15:22:57 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> Linux Kernel Mailing List wrote:
> > commit e0ab2928cc2202f13f0574d4c6f567f166d307eb
> > tree 3df0b8e340b1a98cd8a2daa19672ff008e8fb7f9
> > parent b611967de4dc5c52049676c4369dcac622a7cdfe
> > author Stephane Eranian <eranian@hpl.hp.com> 1160554905 -0700
> > committer Linus Torvalds <torvalds@g5.osdl.org> 1160590461 -0700
> > 
> > [PATCH] Add carta_random32() library routine
> > 
> > This is a follow-up patch based on the review for perfmon2.  This patch
> > adds the carta_random32() library routine + carta_random32.h header file.
> > 
> > This is fast, simple, and efficient pseudo number generator algorithm.  We
> > use it in perfmon2 to randomize the sampling periods.  In this context, we
> > do not need any fancy randomizer.
> 
> hrm, does this really warrant inclusion into every kernel build, on 
> every platform?
> 

probly not really.  But putting it into lib.a has problems, and making it a
loadable module has problems, and making it Kconfigurable has problems. 
And it's only 150-odd bytes.

It'd be better to just start to use it.  There are a number of callers of
get_random_bytes() who just don't need such fanciness (and degradation or
/dev/urandom), such as ext2/3/4's Orlov allocator.