Re: [dm-crypt] crypsetup segfaulting during luksFormat

[Milan Broz <mbroz@redhat.com>]

On 07/08/2010 03:37 PM, Sven Eschenberg wrote:
> Just for the record:
> 
> The crash happens with other gcc versions as well. As the gentoo bug
> report suggests, it seems to be a problem when the executeable is linked
> statically on hardened profiles.
> And yes, in my case compiling it dynamically resolves the segfault
> aswell.

I am compiling static version quite often, so hardened profile probably uses
some not common compiled switch for static version.

> In the src the following variables are used in the handler:
> 
> static volatile uint64_t __PBKDF2_global_j = 0;
> static volatile uint64_t __PBKDF2_performance = 0;
> 
> Since they are used in the sighandler, they would better not just be
> volatile but sig_atomic_t, to avoid possible races.

yes

> But this should not have any influence on the segfault as far as I can
> tell.
> 
> Oh, and better use sigaction() instead of signal().

why? should be no problem here. (that code is ugly anyway, I just polished
it some time ago when replacing pbkdf2 with gcrypt version...)


> I think I possibly found the problem:
> 
> In static int pkcs5_pbkdf2() in pbkdf.c:
> 
> size_t tmplen = Slen + 4;
> tmp = alloca(tmplen); // allocate Slen+4 bytes on the stack ...

so problem is implicit type cast? interesting...

seems to be some relict from former implementation, I am always
trying to avoid alloca() in code... :)
(wonder if valgrind find that)


Thanks!
Milan