Re: [dm-crypt] crypsetup segfaulting during luksFormat

[Milan Broz <mbroz@redhat.com>]

On 07/08/2010 08:44 PM, Sven Eschenberg wrote:
> On Thu, 2010-07-08 at 17:11 +0200, Milan Broz wrote:
>> On 07/08/2010 04:29 PM, Sven Eschenberg wrote:
>>> Hi Milan,
>>>
>>> Even worse, actually byte swappig is done to store the int in a big
>>> endian manner, unfortunately, since it is done wrong, on big endian
>>> systems all block indeces would be zero and they are part of the Derived
>>> Key. I wonder if this has a security impact as in quality of derived key
>>> on big endian systems.
>>
>> You mean when int is big endian and 64bit? Do you see system where it is wrong
>> or just guessing?
>>
> 
> Should happen on 32 Bit already, okay, let's see, AA BB CC DD, most
> significant byte is AA (mask BB CC DD, shift right logically, so we will
> get AA as value), which in little endian looks like AA 00 00 00, which
> is copied into position Slen+0, the 4 bytes will be filled with AA 00 00
> 00, next step would be copying BB 00 00 00 into the next position Slen
> +1, notice that here the last zero byte exceeds the buffer on the stack
> already ...

Swen,

please look at RFC2898, PBKDF2 definition, search for

"Here, INT (i) is a four-octet encoding of the integer i, most significant octet first."

Is it better now? And compiler interprets mask correctly according to endian type.
There should be no stack overflow, it works with char array, not int array.

FYI the same code is in gnutls, see lib/x509/pbkdf2-sha1.c

Do it still segfaults after your suggested change? Did you tried it?

Milan