From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <mbroz@redhat.com>
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28])
	by mail.saout.de (Postfix) with ESMTP
	for <dm-crypt@saout.de>; Thu,  8 Jul 2010 16:16:27 +0200 (CEST)
Message-ID: <4C35DDB2.6010803@redhat.com>
Date: Thu, 08 Jul 2010 16:16:18 +0200
From: Milan Broz <mbroz@redhat.com>
MIME-Version: 1.0
References: <1278523178.9943.5.camel@Koma-Station.localdomain>	<20100707173658.GA20180@tansi.org>	<1278524944.9943.9.camel@Koma-Station.localdomain>	<20100707201955.GA22353@tansi.org>
	<4C34E72E.8070809@redhat.com>	<1278536333.9943.17.camel@Koma-Station.localdomain>	<4C34F02C.10704@redhat.com>	<1278555755.9943.35.camel@Koma-Station.localdomain>	<20100708093055.GA5572@tansi.org>
	<1278596268.3147.56.camel@Koma-Station.localdomain>
In-Reply-To: <1278596268.3147.56.camel@Koma-Station.localdomain>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Subject: Re: [dm-crypt] crypsetup segfaulting during luksFormat
List-Id: <dm-crypt.saout.de>
List-Unsubscribe: <http://www.saout.de/mailman/options/dm-crypt>,
	<mailto:dm-crypt-request@saout.de?subject=unsubscribe>
List-Archive: <http://www.saout.de/pipermail/dm-crypt>
List-Post: <mailto:dm-crypt@saout.de>
List-Help: <mailto:dm-crypt-request@saout.de?subject=help>
List-Subscribe: <http://www.saout.de/mailman/listinfo/dm-crypt>,
	<mailto:dm-crypt-request@saout.de?subject=subscribe>
To: Sven Eschenberg <sven@whgl.uni-frankfurt.de>
Cc: dm-crypt@saout.de

On 07/08/2010 03:37 PM, Sven Eschenberg wrote:
> Just for the record:
> 
> The crash happens with other gcc versions as well. As the gentoo bug
> report suggests, it seems to be a problem when the executeable is linked
> statically on hardened profiles.
> And yes, in my case compiling it dynamically resolves the segfault
> aswell.

I am compiling static version quite often, so hardened profile probably uses
some not common compiled switch for static version.

> In the src the following variables are used in the handler:
> 
> static volatile uint64_t __PBKDF2_global_j = 0;
> static volatile uint64_t __PBKDF2_performance = 0;
> 
> Since they are used in the sighandler, they would better not just be
> volatile but sig_atomic_t, to avoid possible races.

yes

> But this should not have any influence on the segfault as far as I can
> tell.
> 
> Oh, and better use sigaction() instead of signal().

why? should be no problem here. (that code is ugly anyway, I just polished
it some time ago when replacing pbkdf2 with gcrypt version...)


> I think I possibly found the problem:
> 
> In static int pkcs5_pbkdf2() in pbkdf.c:
> 
> size_t tmplen = Slen + 4;
> tmp = alloca(tmplen); // allocate Slen+4 bytes on the stack ...

so problem is implicit type cast? interesting...

seems to be some relict from former implementation, I am always
trying to avoid alloca() in code... :)
(wonder if valgrind find that)


Thanks!
Milan