[PATCH 1/2] eoqa: use bash to execute SDK test commands

[PATCH 1/2] eoqa: use bash to execute SDK test commands

[Mikko Rapeli]

The commands only work with with bash. If /bin/sh is
dash like in Debian, the command execution fails with
errors like:

Standard Output: /bin/sh: 5: export: --sysroot: bad variable name

and all SDK tests fail.

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
---
 meta/lib/oeqa/sdk/case.py                  | 2 +-
 meta/lib/oeqa/sdk/utils/sdkbuildproject.py | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/sdk/case.py b/meta/lib/oeqa/sdk/case.py
index d8611c8..5334237 100644
--- a/meta/lib/oeqa/sdk/case.py
+++ b/meta/lib/oeqa/sdk/case.py
@@ -9,7 +9,7 @@ from oeqa.core.case import OETestCase
 class OESDKTestCase(OETestCase):
     def _run(self, cmd):
         return subprocess.check_output(". %s > /dev/null; %s;" % \
-                (self.tc.sdk_env, cmd), shell=True,
+                (self.tc.sdk_env, cmd), shell=True, executable="/bin/bash",
                 stderr=subprocess.STDOUT, universal_newlines=True)
 
     def fetch(self, workdir, dl_dir, url, archive=None):
diff --git a/meta/lib/oeqa/sdk/utils/sdkbuildproject.py b/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
index 6fed73e..eafbd7a 100644
--- a/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
+++ b/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
@@ -42,7 +42,8 @@ class SDKBuildProject(BuildProject):
     def _run(self, cmd):
         self.log("Running . %s; " % self.sdkenv + cmd)
         try:
-            output = subprocess.check_output(". %s; " % self.sdkenv + cmd, shell=True, stderr=subprocess.STDOUT)
+            output = subprocess.check_output(". %s; " % self.sdkenv + cmd, shell=True,
+                                             executable='/bin/bash', stderr=subprocess.STDOUT)
         except subprocess.CalledProcessError as exc:
             print(exc.output.decode('utf-8'))
             return exc.returncode
-- 
1.9.1

[PATCH 2/2] openssh: usable sshd depends on rngd from rng-tools

[Mikko Rapeli]

Since openssl 1.1.1 and openssh which uses it, sshd
startup is delayed. The delays range from few seconds
to minutes and even to hours. The delays are visible
in host keys generation and when sshd process is started
in response to incoming TCP connection but is failing
to provide SSH version string and clients or tests time out.

In all cases traces show that sshd is waiting for getentropy()
system call to return from Linux kernel, which returns only
after kernel side random number pool is initialized. The pool
is initialized via various entropy source which may be
missing on embedded development boards or via rngd from
rng-tools package from userspace. HW random number generation
and kernel support help but rngd is till needed to feed that data
back to the Linux kernel.

Example from an NXP imx8 board shows that kernel random number pool
initialization can take over 400 seconds without rngd,
and with rngd it is initialized at around 4 seconds after boot.
The completion of initialization is visible in kernel dmesg with line
"random: crng init done".

More details are available from:

 * https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=912087
 * https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=897572
 * https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=43838a23a05fbd13e47d750d3dfd77001536dd33
 * http://www.man7.org/linux/man-pages/man2/getrandom.2.html

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
---
 meta/recipes-connectivity/openssh/openssh_7.9p1.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
index b971b2b..567790c 100644
--- a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
+++ b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
@@ -148,6 +148,7 @@ FILES_${PN}-keygen = "${bindir}/ssh-keygen"
 
 RDEPENDS_${PN} += "${PN}-scp ${PN}-ssh ${PN}-sshd ${PN}-keygen"
 RDEPENDS_${PN}-sshd += "${PN}-keygen ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-keyinit pam-plugin-loginuid', '', d)}"
+RDEPENDS_${PN}-sshd += "rng-tools"
 RDEPENDS_${PN}-ptest += "${PN}-sftp ${PN}-misc ${PN}-sftp-server make sed"
 
 RPROVIDES_${PN}-ssh = "ssh"
-- 
1.9.1

Re: [PATCH 2/2] openssh: usable sshd depends on rngd from rng-tools

[Adrian Bunk]

On Wed, May 08, 2019 at 04:26:09PM +0300, Mikko Rapeli wrote:
> Since openssl 1.1.1 and openssh which uses it, sshd
> startup is delayed. The delays range from few seconds
> to minutes and even to hours. The delays are visible
> in host keys generation and when sshd process is started
> in response to incoming TCP connection but is failing
> to provide SSH version string and clients or tests time out.
> 
> In all cases traces show that sshd is waiting for getentropy()
> system call to return from Linux kernel, which returns only
> after kernel side random number pool is initialized. The pool
> is initialized via various entropy source which may be
> missing on embedded development boards or via rngd from
> rng-tools package from userspace. HW random number generation
> and kernel support help but rngd is till needed to feed that data
> back to the Linux kernel.
> 
> Example from an NXP imx8 board shows that kernel random number pool
> initialization can take over 400 seconds without rngd,
> and with rngd it is initialized at around 4 seconds after boot.
> The completion of initialization is visible in kernel dmesg with line
> "random: crng init done".
>...
> --- a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
> +++ b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
> @@ -148,6 +148,7 @@ FILES_${PN}-keygen = "${bindir}/ssh-keygen"
>  
>  RDEPENDS_${PN} += "${PN}-scp ${PN}-ssh ${PN}-sshd ${PN}-keygen"
>  RDEPENDS_${PN}-sshd += "${PN}-keygen ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-keyinit pam-plugin-loginuid', '', d)}"
> +RDEPENDS_${PN}-sshd += "rng-tools"
>...

This should only be an RRECOMMENDS so that people can opt out of it.

E.g. CONFIG_RANDOM_TRUST_CPU in the kernel can solve the same 
problem without using rng-tools on some platforms.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: [PATCH 2/2] openssh: usable sshd depends on rngd from rng-tools

[Mikko.Rapeli]

On Wed, May 08, 2019 at 05:07:08PM +0300, Adrian Bunk wrote:
> On Wed, May 08, 2019 at 04:26:09PM +0300, Mikko Rapeli wrote:
> > Since openssl 1.1.1 and openssh which uses it, sshd
> > startup is delayed. The delays range from few seconds
> > to minutes and even to hours. The delays are visible
> > in host keys generation and when sshd process is started
> > in response to incoming TCP connection but is failing
> > to provide SSH version string and clients or tests time out.
> > 
> > In all cases traces show that sshd is waiting for getentropy()
> > system call to return from Linux kernel, which returns only
> > after kernel side random number pool is initialized. The pool
> > is initialized via various entropy source which may be
> > missing on embedded development boards or via rngd from
> > rng-tools package from userspace. HW random number generation
> > and kernel support help but rngd is till needed to feed that data
> > back to the Linux kernel.
> > 
> > Example from an NXP imx8 board shows that kernel random number pool
> > initialization can take over 400 seconds without rngd,
> > and with rngd it is initialized at around 4 seconds after boot.
> > The completion of initialization is visible in kernel dmesg with line
> > "random: crng init done".
> >...
> > --- a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
> > +++ b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
> > @@ -148,6 +148,7 @@ FILES_${PN}-keygen = "${bindir}/ssh-keygen"
> >  
> >  RDEPENDS_${PN} += "${PN}-scp ${PN}-ssh ${PN}-sshd ${PN}-keygen"
> >  RDEPENDS_${PN}-sshd += "${PN}-keygen ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-keyinit pam-plugin-loginuid', '', d)}"
> > +RDEPENDS_${PN}-sshd += "rng-tools"
> >...
> 
> This should only be an RRECOMMENDS so that people can opt out of it.
> 
> E.g. CONFIG_RANDOM_TRUST_CPU in the kernel can solve the same 
> problem without using rng-tools on some platforms.

I think this is a stronger dependency than just RRECOMMENDS. We build
images and disable recommends but we care that sshd starts as fast as in
sumo and earlier yocto releases for testing etc purposes.

-Mikko

Re: [PATCH 2/2] openssh: usable sshd depends on rngd from rng-tools

[Rasmus Villemoes]

On 08/05/2019 16.22, Mikko.Rapeli@bmw.de wrote:
> On Wed, May 08, 2019 at 05:07:08PM +0300, Adrian Bunk wrote:
>> On Wed, May 08, 2019 at 04:26:09PM +0300, Mikko Rapeli wrote:
>>> Since openssl 1.1.1 and openssh which uses it, sshd
>>> startup is delayed. The delays range from few seconds
>>> to minutes and even to hours. The delays are visible
>>> in host keys generation and when sshd process is started
>>> in response to incoming TCP connection but is failing
>>> to provide SSH version string and clients or tests time out.
>>>
>>> In all cases traces show that sshd is waiting for getentropy()
>>> system call to return from Linux kernel, which returns only
>>> after kernel side random number pool is initialized. The pool
>>> is initialized via various entropy source which may be
>>> missing on embedded development boards or via rngd from
>>> rng-tools package from userspace. HW random number generation
>>> and kernel support help but rngd is till needed to feed that data
>>> back to the Linux kernel.
>>>
>>> Example from an NXP imx8 board shows that kernel random number pool
>>> initialization can take over 400 seconds without rngd,
>>> and with rngd it is initialized at around 4 seconds after boot.
>>> The completion of initialization is visible in kernel dmesg with line
>>> "random: crng init done".
>>> ...
>>> --- a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
>>> +++ b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
>>> @@ -148,6 +148,7 @@ FILES_${PN}-keygen = "${bindir}/ssh-keygen"
>>>  
>>>  RDEPENDS_${PN} += "${PN}-scp ${PN}-ssh ${PN}-sshd ${PN}-keygen"
>>>  RDEPENDS_${PN}-sshd += "${PN}-keygen ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-keyinit pam-plugin-loginuid', '', d)}"
>>> +RDEPENDS_${PN}-sshd += "rng-tools"
>>> ...
>>
>> This should only be an RRECOMMENDS so that people can opt out of it.
>>
>> E.g. CONFIG_RANDOM_TRUST_CPU in the kernel can solve the same 
>> problem without using rng-tools on some platforms.
> 
> I think this is a stronger dependency than just RRECOMMENDS. We build
> images and disable recommends but we care that sshd starts as fast as in
> sumo and earlier yocto releases for testing etc purposes.

But why should boards without a hwrng be forced to spend disk space and
run-time resources on a daemon which they don't benefit from at all?

I don't think RANDOM_TRUST_CPU works, though. That's just for stuff like
rdrand(), i.e. instructions built into the CPU - not for some other
on-chip hwrng. Whether those are used for seeding early on (i.e.,
without rngd doing its thing) depends on the ->quality parameter set by
the individual hwrng drivers. Very few set one, so they get assigned the
default_quality, which is a module parameter that defaults to 0.

IOW, I think (but I haven't got around to testing this) one should set
rng_core.default_quality=512 (or something) on the kernel command line
to make the kernel start the hwrng_fill thread that will seed the
entropy pool from the hwrng. At least if I'm reading
drivers/char/hw_random/core.c correctly.

Rasmus

Re: [PATCH 2/2] openssh: usable sshd depends on rngd from rng-tools

[Burton, Ross]

I'm very dubious of the need to make this a dependency, as the
hardware RNG should be used.  Note that there's been a slew of fixes
to the kernel to enable this with modern stacks, for example:

https://github.com/torvalds/linux/commit/62f95ae805fa9e1e84d47d3219adddd97b2654b7

Maybe the IMX driver needs the same patch?

Ross

On Thu, 9 May 2019 at 09:46, Rasmus Villemoes
<rasmus.villemoes@prevas.dk> wrote:
>
> On 08/05/2019 16.22, Mikko.Rapeli@bmw.de wrote:
> > On Wed, May 08, 2019 at 05:07:08PM +0300, Adrian Bunk wrote:
> >> On Wed, May 08, 2019 at 04:26:09PM +0300, Mikko Rapeli wrote:
> >>> Since openssl 1.1.1 and openssh which uses it, sshd
> >>> startup is delayed. The delays range from few seconds
> >>> to minutes and even to hours. The delays are visible
> >>> in host keys generation and when sshd process is started
> >>> in response to incoming TCP connection but is failing
> >>> to provide SSH version string and clients or tests time out.
> >>>
> >>> In all cases traces show that sshd is waiting for getentropy()
> >>> system call to return from Linux kernel, which returns only
> >>> after kernel side random number pool is initialized. The pool
> >>> is initialized via various entropy source which may be
> >>> missing on embedded development boards or via rngd from
> >>> rng-tools package from userspace. HW random number generation
> >>> and kernel support help but rngd is till needed to feed that data
> >>> back to the Linux kernel.
> >>>
> >>> Example from an NXP imx8 board shows that kernel random number pool
> >>> initialization can take over 400 seconds without rngd,
> >>> and with rngd it is initialized at around 4 seconds after boot.
> >>> The completion of initialization is visible in kernel dmesg with line
> >>> "random: crng init done".
> >>> ...
> >>> --- a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
> >>> +++ b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
> >>> @@ -148,6 +148,7 @@ FILES_${PN}-keygen = "${bindir}/ssh-keygen"
> >>>
> >>>  RDEPENDS_${PN} += "${PN}-scp ${PN}-ssh ${PN}-sshd ${PN}-keygen"
> >>>  RDEPENDS_${PN}-sshd += "${PN}-keygen ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-keyinit pam-plugin-loginuid', '', d)}"
> >>> +RDEPENDS_${PN}-sshd += "rng-tools"
> >>> ...
> >>
> >> This should only be an RRECOMMENDS so that people can opt out of it.
> >>
> >> E.g. CONFIG_RANDOM_TRUST_CPU in the kernel can solve the same
> >> problem without using rng-tools on some platforms.
> >
> > I think this is a stronger dependency than just RRECOMMENDS. We build
> > images and disable recommends but we care that sshd starts as fast as in
> > sumo and earlier yocto releases for testing etc purposes.
>
> But why should boards without a hwrng be forced to spend disk space and
> run-time resources on a daemon which they don't benefit from at all?
>
> I don't think RANDOM_TRUST_CPU works, though. That's just for stuff like
> rdrand(), i.e. instructions built into the CPU - not for some other
> on-chip hwrng. Whether those are used for seeding early on (i.e.,
> without rngd doing its thing) depends on the ->quality parameter set by
> the individual hwrng drivers. Very few set one, so they get assigned the
> default_quality, which is a module parameter that defaults to 0.
>
> IOW, I think (but I haven't got around to testing this) one should set
> rng_core.default_quality=512 (or something) on the kernel command line
> to make the kernel start the hwrng_fill thread that will seed the
> entropy pool from the hwrng. At least if I'm reading
> drivers/char/hw_random/core.c correctly.
>
> Rasmus
>
>
>
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH 2/2] openssh: usable sshd depends on rngd from rng-tools

[Mikko.Rapeli]

On Fri, May 10, 2019 at 01:18:21PM +0100, Burton, Ross wrote:
> I'm very dubious of the need to make this a dependency, as the
> hardware RNG should be used.  Note that there's been a slew of fixes
> to the kernel to enable this with modern stacks, for example:
> 
> https://github.com/torvalds/linux/commit/62f95ae805fa9e1e84d47d3219adddd97b2654b7
> 
> Maybe the IMX driver needs the same patch?

Possibly. Was running 4.14.89 kernel from NXP.

I would have benefitted from this patch, but it's ok if you reject it.

-Mikko

> Ross
> 
> On Thu, 9 May 2019 at 09:46, Rasmus Villemoes
> <rasmus.villemoes@prevas.dk> wrote:
> >
> > On 08/05/2019 16.22, Mikko.Rapeli@bmw.de wrote:
> > > On Wed, May 08, 2019 at 05:07:08PM +0300, Adrian Bunk wrote:
> > >> On Wed, May 08, 2019 at 04:26:09PM +0300, Mikko Rapeli wrote:
> > >>> Since openssl 1.1.1 and openssh which uses it, sshd
> > >>> startup is delayed. The delays range from few seconds
> > >>> to minutes and even to hours. The delays are visible
> > >>> in host keys generation and when sshd process is started
> > >>> in response to incoming TCP connection but is failing
> > >>> to provide SSH version string and clients or tests time out.
> > >>>
> > >>> In all cases traces show that sshd is waiting for getentropy()
> > >>> system call to return from Linux kernel, which returns only
> > >>> after kernel side random number pool is initialized. The pool
> > >>> is initialized via various entropy source which may be
> > >>> missing on embedded development boards or via rngd from
> > >>> rng-tools package from userspace. HW random number generation
> > >>> and kernel support help but rngd is till needed to feed that data
> > >>> back to the Linux kernel.
> > >>>
> > >>> Example from an NXP imx8 board shows that kernel random number pool
> > >>> initialization can take over 400 seconds without rngd,
> > >>> and with rngd it is initialized at around 4 seconds after boot.
> > >>> The completion of initialization is visible in kernel dmesg with line
> > >>> "random: crng init done".
> > >>> ...
> > >>> --- a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
> > >>> +++ b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
> > >>> @@ -148,6 +148,7 @@ FILES_${PN}-keygen = "${bindir}/ssh-keygen"
> > >>>
> > >>>  RDEPENDS_${PN} += "${PN}-scp ${PN}-ssh ${PN}-sshd ${PN}-keygen"
> > >>>  RDEPENDS_${PN}-sshd += "${PN}-keygen ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-keyinit pam-plugin-loginuid', '', d)}"
> > >>> +RDEPENDS_${PN}-sshd += "rng-tools"
> > >>> ...
> > >>
> > >> This should only be an RRECOMMENDS so that people can opt out of it.
> > >>
> > >> E.g. CONFIG_RANDOM_TRUST_CPU in the kernel can solve the same
> > >> problem without using rng-tools on some platforms.
> > >
> > > I think this is a stronger dependency than just RRECOMMENDS. We build
> > > images and disable recommends but we care that sshd starts as fast as in
> > > sumo and earlier yocto releases for testing etc purposes.
> >
> > But why should boards without a hwrng be forced to spend disk space and
> > run-time resources on a daemon which they don't benefit from at all?
> >
> > I don't think RANDOM_TRUST_CPU works, though. That's just for stuff like
> > rdrand(), i.e. instructions built into the CPU - not for some other
> > on-chip hwrng. Whether those are used for seeding early on (i.e.,
> > without rngd doing its thing) depends on the ->quality parameter set by
> > the individual hwrng drivers. Very few set one, so they get assigned the
> > default_quality, which is a module parameter that defaults to 0.
> >
> > IOW, I think (but I haven't got around to testing this) one should set
> > rng_core.default_quality=512 (or something) on the kernel command line
> > to make the kernel start the hwrng_fill thread that will seed the
> > entropy pool from the hwrng. At least if I'm reading
> > drivers/char/hw_random/core.c correctly.
> >
> > Rasmus
> >
> >
> >
> >
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH 2/2] openssh: usable sshd depends on rngd from rng-tools

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 4883 bytes --]

On Fri, May 10, 2019 at 8:18 AM Burton, Ross <ross.burton@intel.com> wrote:

> I'm very dubious of the need to make this a dependency, as the
> hardware RNG should be used.  Note that there's been a slew of fixes
> to the kernel to enable this with modern stacks, for example:
>
>
> https://github.com/torvalds/linux/commit/62f95ae805fa9e1e84d47d3219adddd97b2654b7
>
> Maybe the IMX driver needs the same patch?
>

In my experience, not all boards have a usable hw RNG that can solve the
problem.

That being said, I have no strong opinion on it being a RRECOMMENDS, or not
mentioned
at all. Since a BSP really is the place that knows if a usable hwrng is
available, and some sort
of machine feature or distro layer could pull in the dependency for boards
that know they
need the daemons.

Bruce



>
> Ross
>
> On Thu, 9 May 2019 at 09:46, Rasmus Villemoes
> <rasmus.villemoes@prevas.dk> wrote:
> >
> > On 08/05/2019 16.22, Mikko.Rapeli@bmw.de wrote:
> > > On Wed, May 08, 2019 at 05:07:08PM +0300, Adrian Bunk wrote:
> > >> On Wed, May 08, 2019 at 04:26:09PM +0300, Mikko Rapeli wrote:
> > >>> Since openssl 1.1.1 and openssh which uses it, sshd
> > >>> startup is delayed. The delays range from few seconds
> > >>> to minutes and even to hours. The delays are visible
> > >>> in host keys generation and when sshd process is started
> > >>> in response to incoming TCP connection but is failing
> > >>> to provide SSH version string and clients or tests time out.
> > >>>
> > >>> In all cases traces show that sshd is waiting for getentropy()
> > >>> system call to return from Linux kernel, which returns only
> > >>> after kernel side random number pool is initialized. The pool
> > >>> is initialized via various entropy source which may be
> > >>> missing on embedded development boards or via rngd from
> > >>> rng-tools package from userspace. HW random number generation
> > >>> and kernel support help but rngd is till needed to feed that data
> > >>> back to the Linux kernel.
> > >>>
> > >>> Example from an NXP imx8 board shows that kernel random number pool
> > >>> initialization can take over 400 seconds without rngd,
> > >>> and with rngd it is initialized at around 4 seconds after boot.
> > >>> The completion of initialization is visible in kernel dmesg with line
> > >>> "random: crng init done".
> > >>> ...
> > >>> --- a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
> > >>> +++ b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
> > >>> @@ -148,6 +148,7 @@ FILES_${PN}-keygen = "${bindir}/ssh-keygen"
> > >>>
> > >>>  RDEPENDS_${PN} += "${PN}-scp ${PN}-ssh ${PN}-sshd ${PN}-keygen"
> > >>>  RDEPENDS_${PN}-sshd += "${PN}-keygen
> ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-keyinit
> pam-plugin-loginuid', '', d)}"
> > >>> +RDEPENDS_${PN}-sshd += "rng-tools"
> > >>> ...
> > >>
> > >> This should only be an RRECOMMENDS so that people can opt out of it.
> > >>
> > >> E.g. CONFIG_RANDOM_TRUST_CPU in the kernel can solve the same
> > >> problem without using rng-tools on some platforms.
> > >
> > > I think this is a stronger dependency than just RRECOMMENDS. We build
> > > images and disable recommends but we care that sshd starts as fast as
> in
> > > sumo and earlier yocto releases for testing etc purposes.
> >
> > But why should boards without a hwrng be forced to spend disk space and
> > run-time resources on a daemon which they don't benefit from at all?
> >
> > I don't think RANDOM_TRUST_CPU works, though. That's just for stuff like
> > rdrand(), i.e. instructions built into the CPU - not for some other
> > on-chip hwrng. Whether those are used for seeding early on (i.e.,
> > without rngd doing its thing) depends on the ->quality parameter set by
> > the individual hwrng drivers. Very few set one, so they get assigned the
> > default_quality, which is a module parameter that defaults to 0.
> >
> > IOW, I think (but I haven't got around to testing this) one should set
> > rng_core.default_quality=512 (or something) on the kernel command line
> > to make the kernel start the hwrng_fill thread that will seed the
> > entropy pool from the hwrng. At least if I'm reading
> > drivers/char/hw_random/core.c correctly.
> >
> > Rasmus
> >
> >
> >
> >
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 7536 bytes --]

Re: [PATCH 2/2] openssh: usable sshd depends on rngd from rng-tools

[Mark Hatle]

On 5/8/19 5:22 PM, Mikko.Rapeli@bmw.de wrote:
> On Wed, May 08, 2019 at 05:07:08PM +0300, Adrian Bunk wrote:
>> On Wed, May 08, 2019 at 04:26:09PM +0300, Mikko Rapeli wrote:
>>> Since openssl 1.1.1 and openssh which uses it, sshd
>>> startup is delayed. The delays range from few seconds
>>> to minutes and even to hours. The delays are visible
>>> in host keys generation and when sshd process is started
>>> in response to incoming TCP connection but is failing
>>> to provide SSH version string and clients or tests time out.
>>>
>>> In all cases traces show that sshd is waiting for getentropy()
>>> system call to return from Linux kernel, which returns only
>>> after kernel side random number pool is initialized. The pool
>>> is initialized via various entropy source which may be
>>> missing on embedded development boards or via rngd from
>>> rng-tools package from userspace. HW random number generation
>>> and kernel support help but rngd is till needed to feed that data
>>> back to the Linux kernel.
>>>
>>> Example from an NXP imx8 board shows that kernel random number pool
>>> initialization can take over 400 seconds without rngd,
>>> and with rngd it is initialized at around 4 seconds after boot.
>>> The completion of initialization is visible in kernel dmesg with line
>>> "random: crng init done".
>>> ...
>>> --- a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
>>> +++ b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
>>> @@ -148,6 +148,7 @@ FILES_${PN}-keygen = "${bindir}/ssh-keygen"
>>>  
>>>  RDEPENDS_${PN} += "${PN}-scp ${PN}-ssh ${PN}-sshd ${PN}-keygen"
>>>  RDEPENDS_${PN}-sshd += "${PN}-keygen ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-keyinit pam-plugin-loginuid', '', d)}"
>>> +RDEPENDS_${PN}-sshd += "rng-tools"
>>> ...
>>
>> This should only be an RRECOMMENDS so that people can opt out of it.
>>
>> E.g. CONFIG_RANDOM_TRUST_CPU in the kernel can solve the same 
>> problem without using rng-tools on some platforms.
> 
> I think this is a stronger dependency than just RRECOMMENDS. We build
> images and disable recommends but we care that sshd starts as fast as in
> sumo and earlier yocto releases for testing etc purposes.

I agree with Adrian here.  It should be a recommend.  The system works without
this, and there are valid use-cases without rngd existing on the system.  (In
fact I have a couple of customers that would rather the system stall waiting for
'real' entropy then use the values from rngd.)

Note the warning on this page:  https://wiki.archlinux.org/index.php/Rng-tools

In a lot of cases, this dependency on urandom on an embedded target without even
a clock or entropy sources results in the system having effectively the same
entropy each time it starts up -- even with rngd.  So you get a false sense of
security.

Once you have a hardware (or other) rng source, the tool can be useful to
increase the amount of entropy available however... but it all starts with
having a reasonable starting source.

In your case, if using rngd has the entropy your device requires, based on your
system configuration (and you do not want recommends), then I think it's
reasonable that you need to manually include it as an image dependency.

--Mark

> -Mikko
> 

Re: [PATCH 2/2] openssh: usable sshd depends on rngd from rng-tools

[Mikko.Rapeli]

On Wed, May 08, 2019 at 06:50:54PM +0300, Mark Hatle wrote:
> On 5/8/19 5:22 PM, Mikko.Rapeli@bmw.de wrote:
> > On Wed, May 08, 2019 at 05:07:08PM +0300, Adrian Bunk wrote:
> >> On Wed, May 08, 2019 at 04:26:09PM +0300, Mikko Rapeli wrote:
> >>> Since openssl 1.1.1 and openssh which uses it, sshd
> >>> startup is delayed. The delays range from few seconds
> >>> to minutes and even to hours. The delays are visible
> >>> in host keys generation and when sshd process is started
> >>> in response to incoming TCP connection but is failing
> >>> to provide SSH version string and clients or tests time out.
> >>>
> >>> In all cases traces show that sshd is waiting for getentropy()
> >>> system call to return from Linux kernel, which returns only
> >>> after kernel side random number pool is initialized. The pool
> >>> is initialized via various entropy source which may be
> >>> missing on embedded development boards or via rngd from
> >>> rng-tools package from userspace. HW random number generation
> >>> and kernel support help but rngd is till needed to feed that data
> >>> back to the Linux kernel.
> >>>
> >>> Example from an NXP imx8 board shows that kernel random number pool
> >>> initialization can take over 400 seconds without rngd,
> >>> and with rngd it is initialized at around 4 seconds after boot.
> >>> The completion of initialization is visible in kernel dmesg with line
> >>> "random: crng init done".
> >>> ...
> >>> --- a/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
> >>> +++ b/meta/recipes-connectivity/openssh/openssh_7.9p1.bb
> >>> @@ -148,6 +148,7 @@ FILES_${PN}-keygen = "${bindir}/ssh-keygen"
> >>>  
> >>>  RDEPENDS_${PN} += "${PN}-scp ${PN}-ssh ${PN}-sshd ${PN}-keygen"
> >>>  RDEPENDS_${PN}-sshd += "${PN}-keygen ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-keyinit pam-plugin-loginuid', '', d)}"
> >>> +RDEPENDS_${PN}-sshd += "rng-tools"
> >>> ...
> >>
> >> This should only be an RRECOMMENDS so that people can opt out of it.
> >>
> >> E.g. CONFIG_RANDOM_TRUST_CPU in the kernel can solve the same 
> >> problem without using rng-tools on some platforms.
> > 
> > I think this is a stronger dependency than just RRECOMMENDS. We build
> > images and disable recommends but we care that sshd starts as fast as in
> > sumo and earlier yocto releases for testing etc purposes.
> 
> I agree with Adrian here.  It should be a recommend.  The system works without
> this, and there are valid use-cases without rngd existing on the system.  (In
> fact I have a couple of customers that would rather the system stall waiting for
> 'real' entropy then use the values from rngd.)

Ok, at least I tried.

I can send a v2 with RRECOMMENDS instead though it is useless for me
since enabling recommends causes images to explode in size, complexity,
licensing with GPLv3 components etc.

> Note the warning on this page:  https://wiki.archlinux.org/index.php/Rng-tools
> 
> In a lot of cases, this dependency on urandom on an embedded target without even
> a clock or entropy sources results in the system having effectively the same
> entropy each time it starts up -- even with rngd.  So you get a false sense of
> security.
> 
> Once you have a hardware (or other) rng source, the tool can be useful to
> increase the amount of entropy available however... but it all starts with
> having a reasonable starting source.
> 
> In your case, if using rngd has the entropy your device requires, based on your
> system configuration (and you do not want recommends), then I think it's
> reasonable that you need to manually include it as an image dependency.

Yes, this lack of entropy on embedded devices is understood but in my case
rngd is the one reading /dev/hwrnd and pushing that back to kernel...

-Mikko

Re: [PATCH 1/2] eoqa: use bash to execute SDK test commands

[Joshua Watt]

On Wed, 2019-05-08 at 16:26 +0300, Mikko Rapeli wrote:
> The commands only work with with bash. If /bin/sh is
> dash like in Debian, the command execution fails with
> errors like:

This might possibly be related to 
https://bugzilla.yoctoproject.org/show_bug.cgi?id=11775 where it was
discovered that in order to use dash, the CWD must be the same
directory where the init script lives?

> 
> Standard Output: /bin/sh: 5: export: --sysroot: bad variable name
> 
> and all SDK tests fail.
> 
> Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
> ---
>  meta/lib/oeqa/sdk/case.py                  | 2 +-
>  meta/lib/oeqa/sdk/utils/sdkbuildproject.py | 3 ++-
>  2 files changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/lib/oeqa/sdk/case.py b/meta/lib/oeqa/sdk/case.py
> index d8611c8..5334237 100644
> --- a/meta/lib/oeqa/sdk/case.py
> +++ b/meta/lib/oeqa/sdk/case.py
> @@ -9,7 +9,7 @@ from oeqa.core.case import OETestCase
>  class OESDKTestCase(OETestCase):
>      def _run(self, cmd):
>          return subprocess.check_output(". %s > /dev/null; %s;" % \
> -                (self.tc.sdk_env, cmd), shell=True,
> +                (self.tc.sdk_env, cmd), shell=True,
> executable="/bin/bash",
>                  stderr=subprocess.STDOUT, universal_newlines=True)
>  
>      def fetch(self, workdir, dl_dir, url, archive=None):
> diff --git a/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
> b/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
> index 6fed73e..eafbd7a 100644
> --- a/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
> +++ b/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
> @@ -42,7 +42,8 @@ class SDKBuildProject(BuildProject):
>      def _run(self, cmd):
>          self.log("Running . %s; " % self.sdkenv + cmd)
>          try:
> -            output = subprocess.check_output(". %s; " % self.sdkenv
> + cmd, shell=True, stderr=subprocess.STDOUT)
> +            output = subprocess.check_output(". %s; " % self.sdkenv
> + cmd, shell=True,
> +                                             executable='/bin/bash',
> stderr=subprocess.STDOUT)
>          except subprocess.CalledProcessError as exc:
>              print(exc.output.decode('utf-8'))
>              return exc.returncode
> -- 
> 1.9.1
> 
-- 
Joshua Watt <JPEWhacker@gmail.com>

Re: [PATCH 1/2] eoqa: use bash to execute SDK test commands

[Mikko.Rapeli]

On Wed, May 08, 2019 at 08:41:21AM -0500, Joshua Watt wrote:
> On Wed, 2019-05-08 at 16:26 +0300, Mikko Rapeli wrote:
> > The commands only work with with bash. If /bin/sh is
> > dash like in Debian, the command execution fails with
> > errors like:
> 
> This might possibly be related to 
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=11775 where it was
> discovered that in order to use dash, the CWD must be the same
> directory where the init script lives?

Yes, this is related.

In my case bitbake build is already called from bash and all scripts
seem to be calling bash correctly as /bin/bash, except for these
SDK tests.

Also on sumo, there is no meaningful debug output when these tests fail.

from oeqa.utils.subprocesstweak import errors_have_output
errors_have_output()

should maybe be used wherewhere with python subprocess...

-Mikko

> > 
> > Standard Output: /bin/sh: 5: export: --sysroot: bad variable name
> > 
> > and all SDK tests fail.
> > 
> > Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
> > ---
> >  meta/lib/oeqa/sdk/case.py                  | 2 +-
> >  meta/lib/oeqa/sdk/utils/sdkbuildproject.py | 3 ++-
> >  2 files changed, 3 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/lib/oeqa/sdk/case.py b/meta/lib/oeqa/sdk/case.py
> > index d8611c8..5334237 100644
> > --- a/meta/lib/oeqa/sdk/case.py
> > +++ b/meta/lib/oeqa/sdk/case.py
> > @@ -9,7 +9,7 @@ from oeqa.core.case import OETestCase
> >  class OESDKTestCase(OETestCase):
> >      def _run(self, cmd):
> >          return subprocess.check_output(". %s > /dev/null; %s;" % \
> > -                (self.tc.sdk_env, cmd), shell=True,
> > +                (self.tc.sdk_env, cmd), shell=True,
> > executable="/bin/bash",
> >                  stderr=subprocess.STDOUT, universal_newlines=True)
> >  
> >      def fetch(self, workdir, dl_dir, url, archive=None):
> > diff --git a/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
> > b/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
> > index 6fed73e..eafbd7a 100644
> > --- a/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
> > +++ b/meta/lib/oeqa/sdk/utils/sdkbuildproject.py
> > @@ -42,7 +42,8 @@ class SDKBuildProject(BuildProject):
> >      def _run(self, cmd):
> >          self.log("Running . %s; " % self.sdkenv + cmd)
> >          try:
> > -            output = subprocess.check_output(". %s; " % self.sdkenv
> > + cmd, shell=True, stderr=subprocess.STDOUT)
> > +            output = subprocess.check_output(". %s; " % self.sdkenv
> > + cmd, shell=True,
> > +                                             executable='/bin/bash',
> > stderr=subprocess.STDOUT)
> >          except subprocess.CalledProcessError as exc:
> >              print(exc.output.decode('utf-8'))
> >              return exc.returncode
> > -- 
> > 1.9.1
> > 
> -- 
> Joshua Watt <JPEWhacker@gmail.com>