All of lore.kernel.org
 help / color / mirror / Atom feed
From: Lukas Herbolt <lukas@herbolt.com>
To: "Darrick J. Wong" <djwong@kernel.org>
Cc: zlang@kernel.org, fstests@vger.kernel.org,
	linux-xfs@vger.kernel.org, linux-ext4@vger.kernel.org
Subject: Re: [PATCH 1/1] lib/random.c fix undefined behavior on signed integer overflow.
Date: Thu, 08 Oct 2026 10:23:59 +0200	[thread overview]
Message-ID: <d000ae8abba884b0a58be93e6e63f486@herbolt.com> (raw)
In-Reply-To: <20261007170058.GG2705364@frogsfrogsfrogs>

On 2026-10-07 19:00, Darrick J. Wong wrote:
> On Tue, Oct 06, 2026 at 03:51:52PM +0200, Lukas Herbolt wrote:
>> The random.c now expects that signed integer to overflow and
>> triggers the  ^MASK branch. But signed int overflow is undefined
>> behavior and GCC can optimize this branch out with certain
>> CFLAGS/LDFLAGS.
>> 
>> Signed-off-by: Lukas Herbolt <lukas@herbolt.com>
>> ---
>>  lib/random.c | 13 +++++++------
>>  1 file changed, 7 insertions(+), 6 deletions(-)
>> 
>> diff --git a/lib/random.c b/lib/random.c
>> index d5c81be817c8..f767ed1d1337 100644
>> --- a/lib/random.c
>> +++ b/lib/random.c
>> @@ -186,14 +186,15 @@ _random (int32_t is [2])
>>  int32_t
>>  _irandm (int32_t is [2])
>>  {
>> -	int32_t it, leh, nit;
>> -
>> +	int32_t it, leh, nit, apply_mask;
>>  	it = is [0];
>>  	leh = is [1];
>> -	if (it <= 0)
>> -		it = (it + it) ^ MASK;
>> -	else
>> -		it = it + it;
>> +
>> +/* 	evaluate on original value — no UB here */
>> +	apply_mask = (it <= 0);
>> +/*	double via unsigned shift — well-defined */
>> +	it = (unsigned)it << 1;
>> +	if (apply_mask) it ^= MASK;
> 
> Doesn't _random suffer the same flaw?
> 
> Oh.  It's dead code, maybe it should go away?
Good point.
> 
> Also, what about converting all the variables to unsigned?

It would break two signed checks: `it <= 0` (LFSR feedback -
only fires at zero instead of when MSB set) and `leh < 0`
(output folding — never fires). The `it` register degenerates
to 33 repeating values after 64 calls. Reducing the pool of
random numbers.

-- 
-lhe

  reply	other threads:[~2026-10-08  8:24 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-06 13:51 [PATCH 0/1] Fix random.c UB on int overflow Lukas Herbolt
2026-10-06 13:51 ` [PATCH 1/1] lib/random.c fix undefined behavior on signed integer overflow Lukas Herbolt
2026-10-07 17:00   ` Darrick J. Wong
2026-10-08  8:23     ` Lukas Herbolt [this message]
2026-10-08 15:17       ` Darrick J. Wong
2026-10-12 13:33       ` Zorro Lang
2026-10-12 13:44   ` Zorro Lang
2026-10-12 14:38     ` Lukas Herbolt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=d000ae8abba884b0a58be93e6e63f486@herbolt.com \
    --to=lukas@herbolt.com \
    --cc=djwong@kernel.org \
    --cc=fstests@vger.kernel.org \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=zlang@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.