[PATCH] char random: fix boot id uniqueness race

[PATCH] char random: fix boot id uniqueness race

[Mathieu Desnoyers]

The proc file /proc/sys/kernel/random/boot_id can be read concurrently
by user-space processes. If two (or more) user-space processes
concurrently read boot_id when sysctl_bootid is not yet assigned, a race
can occur making boot_id differ between the reads. Because the whole
point of the boot id is to be unique across a kernel execution, fix this
by protecting this operation with a mutex, and introduce a
boot_id_generated flag, along with appropriate memory barriers, to let
the fast-path know if the boot ID has been generated without having to
hold the mutex.

I propose this approach rather than setting it up within an initcall(),
because letting execution randomness add to entropy before populating
the boot id seems to be a wanted property. Also, populating it lazily
rather than at boot time only makes the performance hit be taken when
boot_id is being read.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC: "Theodore Ts'o" <tytso@mit.edu>
CC: Matt Mackall <mpm@selenic.com>
CC: Greg Kroah-Hartman <greg@kroah.com>
---
 drivers/char/random.c |   20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

Index: linux-2.6-lttng/drivers/char/random.c
===================================================================
--- linux-2.6-lttng.orig/drivers/char/random.c
+++ linux-2.6-lttng/drivers/char/random.c
@@ -1231,6 +1231,8 @@ static int min_read_thresh = 8, min_writ
 static int max_read_thresh = INPUT_POOL_WORDS * 32;
 static int max_write_thresh = INPUT_POOL_WORDS * 32;
 static char sysctl_bootid[16];
+static int boot_id_generated;
+static DEFINE_MUTEX(boot_id_mutex);
 
 /*
  * These functions is used to return both the bootid UUID, and random
@@ -1250,10 +1252,22 @@ static int proc_do_uuid(ctl_table *table
 	uuid = table->data;
 	if (!uuid) {
 		uuid = tmp_uuid;
-		uuid[8] = 0;
-	}
-	if (uuid[8] == 0)
 		generate_random_uuid(uuid);
+	} else {
+		if (unlikely(!ACCESS_ONCE(boot_id_generated))) {
+			mutex_lock(&boot_id_mutex);
+			if (!boot_id_generated) {
+				generate_random_uuid(uuid);
+				/* Store uuid before boot_id_generated. */
+				smp_wmb();
+				boot_id_generated = 1;
+			}
+			mutex_unlock(&boot_id_mutex);
+		} else {
+			/* Load boot_id_generated before uuid */
+			smp_rmb();
+		}
+	}
 
 	sprintf(buf, "%pU", uuid);
 

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

Re: [PATCH] char random: fix boot id uniqueness race

[Ted Ts'o]

On Sun, Jan 29, 2012 at 11:40:12PM -0500, Mathieu Desnoyers wrote:
> Index: linux-2.6-lttng/drivers/char/random.c
> ===================================================================
> --- linux-2.6-lttng.orig/drivers/char/random.c
> +++ linux-2.6-lttng/drivers/char/random.c
> @@ -1231,6 +1231,8 @@ static int min_read_thresh = 8, min_writ
>  static int max_read_thresh = INPUT_POOL_WORDS * 32;
>  static int max_write_thresh = INPUT_POOL_WORDS * 32;
>  static char sysctl_bootid[16];
> +static int boot_id_generated;
> +static DEFINE_MUTEX(boot_id_mutex);

My preference would be to put these statics in proc_do_uuid(), but
that's arguably a nit.

> +	} else {
> +		if (unlikely(!ACCESS_ONCE(boot_id_generated))) {
> +			mutex_lock(&boot_id_mutex);
> +			if (!boot_id_generated) {
> +				generate_random_uuid(uuid);
> +				/* Store uuid before boot_id_generated. */
> +				smp_wmb();
> +				boot_id_generated = 1;
> +			}
> +			mutex_unlock(&boot_id_mutex);
> +		} else {
> +			/* Load boot_id_generated before uuid */
> +			smp_rmb();
> +		}
> +	}

I don't believe the smp_wmb() and smp_rmb() are necessary here; the
mutex_lock() and mutex_unlock() should put the necessary memory
barriers in place.

Regards,

						- Ted

Re: [PATCH] char random: fix boot id uniqueness race

[Mathieu Desnoyers]

* Ted Ts'o (tytso@mit.edu) wrote:
> On Sun, Jan 29, 2012 at 11:40:12PM -0500, Mathieu Desnoyers wrote:
> > Index: linux-2.6-lttng/drivers/char/random.c
> > ===================================================================
> > --- linux-2.6-lttng.orig/drivers/char/random.c
> > +++ linux-2.6-lttng/drivers/char/random.c
> > @@ -1231,6 +1231,8 @@ static int min_read_thresh = 8, min_writ
> >  static int max_read_thresh = INPUT_POOL_WORDS * 32;
> >  static int max_write_thresh = INPUT_POOL_WORDS * 32;
> >  static char sysctl_bootid[16];
> > +static int boot_id_generated;
> > +static DEFINE_MUTEX(boot_id_mutex);

Hi Ted,

> 
> My preference would be to put these statics in proc_do_uuid(), but
> that's arguably a nit.

No problem, can do.

> 
> > +	} else {
> > +		if (unlikely(!ACCESS_ONCE(boot_id_generated))) {
> > +			mutex_lock(&boot_id_mutex);
> > +			if (!boot_id_generated) {
> > +				generate_random_uuid(uuid);
> > +				/* Store uuid before boot_id_generated. */
> > +				smp_wmb();
> > +				boot_id_generated = 1;
> > +			}
> > +			mutex_unlock(&boot_id_mutex);
> > +		} else {
> > +			/* Load boot_id_generated before uuid */
> > +			smp_rmb();
> > +		}
> > +	}
> 
> I don't believe the smp_wmb() and smp_rmb() are necessary here; the
> mutex_lock() and mutex_unlock() should put the necessary memory
> barriers in place.

The need for memory barriers is a consequence of letting the fast-path
run without holding this mutex.

Here is the race dealt with by the smp_rmb()/smp_wmb(). I'm showing the
result of reversed write order here:

CPU A                             CPU B

Load boot_id_generated            
  (test -> false)
mutex_lock(&boot_id_mutex)
  (implied memory barrier
   with acquire semantic)
Load boot_id_generated again
  (test -> false)
boot_id_generated = 1
  (both the compiler and
   CPU are free to reorder
   the boot_id_generated
   store before uuid stores)
                                  Load boot_id_generated
                                    (test -> true)
                                  Load uuid content
                                    (races with generate_random_uuid:
                                     result either 0 or corrupted)
                                  Return corrupted uuid.
generate_random_uuid(uuid)
mutex_unlock(&boot_id_mutex)

I prefer not requiring the fast-path to take a mutex, because this
would transform a read-mostly operation into an operation that
requires cache-line exchanges (the mutex). However, if we want the
fast-path to be mutex-free, we need to enforce order with
memory barriers: smp_rmb on the read-side, smp_wmb on the
update-side. Failure to do so leads to the race shown above,
where a corrupted boot_id can be returned.

Please let me know if there are aspects of your question I can
address better,

Thanks,

Mathieu

> 
> Regards,
> 
> 						- Ted

-- 
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com