openssl environment variables

openssl environment variables

[Rasmus Villemoes]

Hi

I'm wondering if anybody has encountered this problem before, and if so,
if there is a clean solution:

When using openssl-native, there's machinery in place so that when
openssl-the-binary is called, it's done through a wrapper script that
sets

OPENSSL_CONF
SSL_CERT_DIR
SSL_CERT_FILE
OPENSSL_ENGINES
OPENSSL_MODULES

so that these point into the appropriate STAGING_DIR_NATIVE, and then
invokes openssl.real.

Similarly, when including nativesdk-openssl in the sdk, there's an env
snippet installed that has the same effect when the sdk setup script is
sourced.

However, when the build involves some tool, say (uboot-)mkimage, which
_links_ against libssl, no such env variables are automatically set
up. This means that if one tries to do something like using a pkcs11
engine, and has made sure that the appropriate pkcs11 .so file is
available in sysroot-native, libssl still won't find it because it
doesn't know to look in ${STAGING_DIR_NATIVE}/usr/lib/engines-3.

I can of course define and export these variables myself in the recipe,
or in a tiny openssl-env.bbclass helper class, but this feels like the
sort of thing that the build system should take care of automatically,
just as it already does for the openssl binary itself, and for the whole
sdk environment. But I suppose that by the time dependency resolution
has figured out that "hey, this recipe (transitively) depends on
openssl-native", it's way too late to inject something that sets+exports
these variables.

Rasmus

Re: [OE-core] openssl environment variables

[Richard Purdie]

On Tue, 2024-10-29 at 10:01 +0100, Rasmus Villemoes via lists.openembedded.org wrote:
> I'm wondering if anybody has encountered this problem before, and if so,
> if there is a clean solution:
> 
> When using openssl-native, there's machinery in place so that when
> openssl-the-binary is called, it's done through a wrapper script that
> sets
> 
> OPENSSL_CONF
> SSL_CERT_DIR
> SSL_CERT_FILE
> OPENSSL_ENGINES
> OPENSSL_MODULES
> 
> so that these point into the appropriate STAGING_DIR_NATIVE, and then
> invokes openssl.real.
> 
> Similarly, when including nativesdk-openssl in the sdk, there's an env
> snippet installed that has the same effect when the sdk setup script is
> sourced.
> 
> However, when the build involves some tool, say (uboot-)mkimage, which
> _links_ against libssl, no such env variables are automatically set
> up. This means that if one tries to do something like using a pkcs11
> engine, and has made sure that the appropriate pkcs11 .so file is
> available in sysroot-native, libssl still won't find it because it
> doesn't know to look in ${STAGING_DIR_NATIVE}/usr/lib/engines-3.
> 
> I can of course define and export these variables myself in the recipe,
> or in a tiny openssl-env.bbclass helper class, but this feels like the
> sort of thing that the build system should take care of automatically,
> just as it already does for the openssl binary itself, and for the whole
> sdk environment. But I suppose that by the time dependency resolution
> has figured out that "hey, this recipe (transitively) depends on
> openssl-native", it's way too late to inject something that sets+exports
> these variables.

https://github.com/openssl/openssl/pull/19260

We realised we could only probably fix this properly with upstream
help. We haven't managed to have anyone work through the process enough
to get patches accepted though.

So yes, we're aware but we need someone with time to work on it.

Cheers,

Richard

Re: [OE-core] openssl environment variables

[Rasmus Villemoes]

On Tue, Oct 29 2024, Richard Purdie <richard.purdie@linuxfoundation.org> wrote:

> On Tue, 2024-10-29 at 10:01 +0100, Rasmus Villemoes via lists.openembedded.org wrote:
>> I'm wondering if anybody has encountered this problem before, and if so,
>> if there is a clean solution:
>> 
>> When using openssl-native, there's machinery in place so that when
>> openssl-the-binary is called, it's done through a wrapper script that
>> sets
>> 
>> OPENSSL_CONF
>> SSL_CERT_DIR
>> SSL_CERT_FILE
>> OPENSSL_ENGINES
>> OPENSSL_MODULES
>> 
>> so that these point into the appropriate STAGING_DIR_NATIVE, and then
>> invokes openssl.real.
>> 
>> Similarly, when including nativesdk-openssl in the sdk, there's an env
>> snippet installed that has the same effect when the sdk setup script is
>> sourced.
>> 
>> However, when the build involves some tool, say (uboot-)mkimage, which
>> _links_ against libssl, no such env variables are automatically set
>> up. This means that if one tries to do something like using a pkcs11
>> engine, and has made sure that the appropriate pkcs11 .so file is
>> available in sysroot-native, libssl still won't find it because it
>> doesn't know to look in ${STAGING_DIR_NATIVE}/usr/lib/engines-3.
>> 
>> I can of course define and export these variables myself in the recipe,
>> or in a tiny openssl-env.bbclass helper class, but this feels like the
>> sort of thing that the build system should take care of automatically,
>> just as it already does for the openssl binary itself, and for the whole
>> sdk environment. But I suppose that by the time dependency resolution
>> has figured out that "hey, this recipe (transitively) depends on
>> openssl-native", it's way too late to inject something that sets+exports
>> these variables.
>
> https://github.com/openssl/openssl/pull/19260
>
> We realised we could only probably fix this properly with upstream
> help. We haven't managed to have anyone work through the process enough
> to get patches accepted though.
>
> So yes, we're aware but we need someone with time to work on it.

Ah, thanks for the pointer. OK, so it's a known, and hard, problem.

Rasmus

Re: [OE-core] openssl environment variables

[Mikko Rapeli]

Hi,

On Tue, Oct 29, 2024 at 10:56:43AM +0100, Rasmus Villemoes via lists.openembedded.org wrote:
> On Tue, Oct 29 2024, Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> 
> > On Tue, 2024-10-29 at 10:01 +0100, Rasmus Villemoes via lists.openembedded.org wrote:
> >> I'm wondering if anybody has encountered this problem before, and if so,
> >> if there is a clean solution:
> >> 
> >> When using openssl-native, there's machinery in place so that when
> >> openssl-the-binary is called, it's done through a wrapper script that
> >> sets
> >> 
> >> OPENSSL_CONF
> >> SSL_CERT_DIR
> >> SSL_CERT_FILE
> >> OPENSSL_ENGINES
> >> OPENSSL_MODULES
> >> 
> >> so that these point into the appropriate STAGING_DIR_NATIVE, and then
> >> invokes openssl.real.
> >> 
> >> Similarly, when including nativesdk-openssl in the sdk, there's an env
> >> snippet installed that has the same effect when the sdk setup script is
> >> sourced.
> >> 
> >> However, when the build involves some tool, say (uboot-)mkimage, which
> >> _links_ against libssl, no such env variables are automatically set
> >> up. This means that if one tries to do something like using a pkcs11
> >> engine, and has made sure that the appropriate pkcs11 .so file is
> >> available in sysroot-native, libssl still won't find it because it
> >> doesn't know to look in ${STAGING_DIR_NATIVE}/usr/lib/engines-3.
> >> 
> >> I can of course define and export these variables myself in the recipe,
> >> or in a tiny openssl-env.bbclass helper class, but this feels like the
> >> sort of thing that the build system should take care of automatically,
> >> just as it already does for the openssl binary itself, and for the whole
> >> sdk environment. But I suppose that by the time dependency resolution
> >> has figured out that "hey, this recipe (transitively) depends on
> >> openssl-native", it's way too late to inject something that sets+exports
> >> these variables.
> >
> > https://github.com/openssl/openssl/pull/19260
> >
> > We realised we could only probably fix this properly with upstream
> > help. We haven't managed to have anyone work through the process enough
> > to get patches accepted though.
> >
> > So yes, we're aware but we need someone with time to work on it.
> 
> Ah, thanks for the pointer. OK, so it's a known, and hard, problem.

As a workaround, you can do this in your own layer and a custom bbclass:

DEPENDS += "openssl-native"

export OPENSSL_MODULES="${STAGING_LIBDIR_NATIVE}/ossl-modules"
export OPENSSL_ENGINES="${STAGING_LIBDIR_NATIVE}/engines-3"
export OPENSSL_CONF="${STAGING_LIBDIR_NATIVE}/ssl-3/openssl.cnf"
export SSL_CERT_DIR="${STAGING_LIBDIR_NATIVE}/ssl-3/certs"
export SSL_CERT_FILE="${STAGING_LIBDIR_NATIVE}/ssl-3/cert.pem"

https://gitlab.com/Linaro/trustedsubstrate/meta-ts/-/blob/master/meta-trustedsubstrate/classes/openssl-native.bbclass?ref_type=heads

Cheers,

-Mikko