[PATCH 0/3] oeqa related: 3 fixes

[PATCH 0/3] oeqa related: 3 fixes

[Robert Yang]

The following changes since commit 7bbc453ac2b8d63680855789948a145fc448017f:

  glibc: Enable --with-default-link (2018-12-26 11:19:47 +0000)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib rbt/oeqa
  http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/oeqa

Robert Yang (3):
  oeqa: make it work for multiple users
  runqemu-gen-tapdevs: Allow run --help without sudo
  oeqa/utils/qemurunner: Print output when failed to login

 meta/classes/testimage.bbclass    |  2 +-
 meta/lib/oeqa/runtime/context.py  |  7 +++----
 meta/lib/oeqa/utils/dump.py       |  5 ++++-
 meta/lib/oeqa/utils/qemurunner.py |  1 +
 scripts/runqemu-gen-tapdevs       | 12 ++++++------
 5 files changed, 15 insertions(+), 12 deletions(-)

-- 
2.10.2

[PATCH 1/3] oeqa: make it work for multiple users

[Robert Yang]

There are failures when multiple users run oe-selftest on the same host:

PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved-tests/201812250324_qemu'

This is because /tmp/oe-saved-tests was created by user A, while user B tries
to write data in it, then the error will happen. This patch can fix the problem.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/testimage.bbclass   | 2 +-
 meta/lib/oeqa/runtime/context.py | 7 +++----
 meta/lib/oeqa/utils/dump.py      | 5 ++++-
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 8a15c30..78b7b5e 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -77,7 +77,7 @@ TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'deb', 'apt-native:do
 TESTIMAGELOCK = "${TMPDIR}/testimage.lock"
 TESTIMAGELOCK_qemuall = ""
 
-TESTIMAGE_DUMP_DIR ?= "/tmp/oe-saved-tests/"
+TESTIMAGE_DUMP_DIR ?= "/tmp/oe-saved-tests.${USER}/"
 
 TESTIMAGE_UPDATE_VARS ?= "DL_DIR WORKDIR DEPLOY_DIR"
 
diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa/runtime/context.py
index 943e29b..96c231d 100644
--- a/meta/lib/oeqa/runtime/context.py
+++ b/meta/lib/oeqa/runtime/context.py
@@ -6,7 +6,7 @@ import os
 from oeqa.core.context import OETestContext, OETestContextExecutor
 from oeqa.core.target.ssh import OESSHTarget
 from oeqa.core.target.qemu import OEQemuTarget
-from oeqa.utils.dump import HostDumper
+from oeqa.utils.dump import HostDumper, default_host_dumper_dir
 
 from oeqa.runtime.loader import OERuntimeTestLoader
 
@@ -49,7 +49,6 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
     default_manifest = 'data/manifest'
     default_server_ip = '192.168.7.1'
     default_target_ip = '192.168.7.2'
-    default_host_dumper_dir = '/tmp/oe-saved-tests'
     default_extract_dir = 'packages/extracted'
 
     def register_commands(self, logger, subparsers):
@@ -71,9 +70,9 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
                 % self.default_server_ip)
 
         runtime_group.add_argument('--host-dumper-dir', action='store',
-                default=self.default_host_dumper_dir,
+                default=default_host_dumper_dir,
                 help="Directory where host status is dumped, if tests fails, default: %s" \
-                % self.default_host_dumper_dir)
+                % default_host_dumper_dir)
 
         runtime_group.add_argument('--packages-manifest', action='store',
                 default=self.default_manifest,
diff --git a/meta/lib/oeqa/utils/dump.py b/meta/lib/oeqa/utils/dump.py
index 5a7edc1..3ce8dd1 100644
--- a/meta/lib/oeqa/utils/dump.py
+++ b/meta/lib/oeqa/utils/dump.py
@@ -3,8 +3,11 @@ import sys
 import errno
 import datetime
 import itertools
+import getpass
 from .commands import runCmd
 
+default_host_dumper_dir = "/tmp/oe-saved-tests.%s" % getpass.getuser()
+
 class BaseDumper(object):
     """ Base class to dump commands from host/target """
 
@@ -12,7 +15,7 @@ class BaseDumper(object):
         self.cmds = []
         # Some testing doesn't inherit testimage, so it is needed
         # to set some defaults.
-        self.parent_dir = parent_dir or "/tmp/oe-saved-tests"
+        self.parent_dir = parent_dir or default_host_dumper_dir
         dft_cmds = """  top -bn1
                         iostat -x -z -N -d -p ALL 20 2
                         ps -ef
-- 
2.10.2

[PATCH 2/3] runqemu-gen-tapdevs: Allow run --help without sudo

[Robert Yang]

Then we can get user's UID and GID rather than hardcode to 1000, e.g.:

$ sudo ../poky/scripts/runqemu-gen-tapdevs 15220 100 4 tmp/sysroots-components/x86_64/qemu-helper-native/usr/bin

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 scripts/runqemu-gen-tapdevs | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/scripts/runqemu-gen-tapdevs b/scripts/runqemu-gen-tapdevs
index 11de318..67c4fcc 100755
--- a/scripts/runqemu-gen-tapdevs
+++ b/scripts/runqemu-gen-tapdevs
@@ -29,15 +29,10 @@ usage() {
 	echo "<num> is the number of tap devices to create (0 to remove all)"
 	echo "<native-sysroot-basedir> is the path to the build system's native sysroot"
 	echo "e.g. $ bitbake qemu-helper-native"
-	echo "$ sudo $0 1000 1000 4 tmp/sysroots-components/x86_64/qemu-helper-native/usr/bin"
+	echo "$ sudo $0 `id -u` `id -g` 4 tmp/sysroots-components/x86_64/qemu-helper-native/usr/bin"
 	exit 1
 }
 
-if [ $EUID -ne 0 ]; then
-	echo "Error: This script must be run with root privileges"
-	exit
-fi
-
 if [ $# -ne 4 ]; then
 	echo "Error: Incorrect number of arguments"
 	usage
@@ -54,6 +49,11 @@ if [[ ! -x "$TUNCTL" || -d "$TUNCTL" ]]; then
 	usage
 fi
 
+if [ $EUID -ne 0 ]; then
+	echo "Error: This script must be run with root privileges"
+	exit
+fi
+
 SCRIPT_DIR=`dirname $0`
 RUNQEMU_IFUP="$SCRIPT_DIR/runqemu-ifup"
 if [ ! -x "$RUNQEMU_IFUP" ]; then
-- 
2.10.2

[PATCH 3/3] oeqa/utils/qemurunner: Print output when failed to login

[Robert Yang]

This is useful for debugging.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/lib/oeqa/utils/qemurunner.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 3a9ba02..cce95e1 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -354,6 +354,7 @@ class QemuRunner:
             else:
                 self.logger.debug("Couldn't login into serial console"
                             " as root using blank password")
+                self.logger.debug("The output:\n%s" % output)
         except:
             self.logger.debug("Serial console failed while trying to login")
         return True
-- 
2.10.2

Re: [PATCH 2/3] runqemu-gen-tapdevs: Allow run --help without sudo

[Richard Purdie]

On Thu, 2018-12-27 at 00:59 -0800, Robert Yang wrote:
> Then we can get user's UID and GID rather than hardcode to 1000,
> e.g.:
> 
> $ sudo ../poky/scripts/runqemu-gen-tapdevs 15220 100 4 tmp/sysroots-
> components/x86_64/qemu-helper-native/usr/bin
> 
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  scripts/runqemu-gen-tapdevs | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)

This fixes the help output without sudo but breaks the help output
under sudo. How about we check SUDO_UID and SUDO_GID? Something like:

uid=`id -u`
if [ -n "$SUDO_UID" ]; then
    uid=$SUDO_UID
fi

?

Cheers,

Richard

Re: [PATCH 1/3] oeqa: make it work for multiple users

[Richard Purdie]

On Thu, 2018-12-27 at 00:59 -0800, Robert Yang wrote:
> There are failures when multiple users run oe-selftest on the same
> host:
> 
> PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved-
> tests/201812250324_qemu'
> 
> This is because /tmp/oe-saved-tests was created by user A, while user
> B tries
> to write data in it, then the error will happen. This patch can fix
> the problem.
> 
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  meta/classes/testimage.bbclass   | 2 +-
>  meta/lib/oeqa/runtime/context.py | 7 +++----
>  meta/lib/oeqa/utils/dump.py      | 5 ++++-
>  3 files changed, 8 insertions(+), 6 deletions(-)

Putting this into /tmp isn't good. Why don't we move this to
LOG_DIR/runtime-hostdump/?

Cheers,

Richard

Re: [PATCH 1/3] oeqa: make it work for multiple users

[Robert Yang]

On 12/27/18 7:37 PM, Richard Purdie wrote:
> On Thu, 2018-12-27 at 00:59 -0800, Robert Yang wrote:
>> There are failures when multiple users run oe-selftest on the same
>> host:
>>
>> PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved-
>> tests/201812250324_qemu'
>>
>> This is because /tmp/oe-saved-tests was created by user A, while user
>> B tries
>> to write data in it, then the error will happen. This patch can fix
>> the problem.
>>
>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> ---
>>   meta/classes/testimage.bbclass   | 2 +-
>>   meta/lib/oeqa/runtime/context.py | 7 +++----
>>   meta/lib/oeqa/utils/dump.py      | 5 ++++-
>>   3 files changed, 8 insertions(+), 6 deletions(-)
> 
> Putting this into /tmp isn't good. Why don't we move this to
> LOG_DIR/runtime-hostdump/?

Thanks, make sense, I will send a V2 for patch 1 and 2.

// Robert

> 
> Cheers,
> 
> Richard
> 
> 

Re: [PATCH 1/3] oeqa: make it work for multiple users

[Robert Yang]

On 12/27/18 7:37 PM, Richard Purdie wrote:
> On Thu, 2018-12-27 at 00:59 -0800, Robert Yang wrote:
>> There are failures when multiple users run oe-selftest on the same
>> host:
>>
>> PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved-
>> tests/201812250324_qemu'
>>
>> This is because /tmp/oe-saved-tests was created by user A, while user
>> B tries
>> to write data in it, then the error will happen. This patch can fix
>> the problem.
>>
>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> ---
>>   meta/classes/testimage.bbclass   | 2 +-
>>   meta/lib/oeqa/runtime/context.py | 7 +++----
>>   meta/lib/oeqa/utils/dump.py      | 5 ++++-
>>   3 files changed, 8 insertions(+), 6 deletions(-)
> 
> Putting this into /tmp isn't good. Why don't we move this to
> LOG_DIR/runtime-hostdump/?

I'm afraid that we can't do this, the problem is that we need call get_bb_vars()
to get LOG_DIR (d.getVar('LOG_DIR') doesn't work), but get_bb_vars() runs
"bitbake -e" which doesn't work when "bitbake <image> -ctestimage" since there
is a bitbake server running. We can only get BUILDDIR if we don't call
get_bb_vars(), so I think that we have the following 2 solutions:

1) Save it to /tmp/runtime-hostdump

Or:

2) Save it to BUILDDIR/runtime-hostdump

What's your opinion, please ?

// Robert

> 
> Cheers,
> 
> Richard
> 
> 

Re: [PATCH 1/3] oeqa: make it work for multiple users

[richard.purdie]

On Fri, 2018-12-28 at 15:57 +0800, Robert Yang wrote:
> 
> On 12/27/18 7:37 PM, Richard Purdie wrote:
> > On Thu, 2018-12-27 at 00:59 -0800, Robert Yang wrote:
> > > There are failures when multiple users run oe-selftest on the
> > > same
> > > host:
> > > 
> > > PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved-
> > > tests/201812250324_qemu'
> > > 
> > > This is because /tmp/oe-saved-tests was created by user A, while
> > > user
> > > B tries
> > > to write data in it, then the error will happen. This patch can
> > > fix
> > > the problem.
> > > 
> > > Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> > > ---
> > >   meta/classes/testimage.bbclass   | 2 +-
> > >   meta/lib/oeqa/runtime/context.py | 7 +++----
> > >   meta/lib/oeqa/utils/dump.py      | 5 ++++-
> > >   3 files changed, 8 insertions(+), 6 deletions(-)
> > 
> > Putting this into /tmp isn't good. Why don't we move this to
> > LOG_DIR/runtime-hostdump/?
> 
> I'm afraid that we can't do this, the problem is that we need call
> get_bb_vars()
> to get LOG_DIR (d.getVar('LOG_DIR') doesn't work), but get_bb_vars()
> runs
> "bitbake -e" which doesn't work when "bitbake <image> -ctestimage"
> since there
> is a bitbake server running. We can only get BUILDDIR if we don't
> call
> get_bb_vars(), so I think that we have the following 2 solutions:
> 
> 1) Save it to /tmp/runtime-hostdump
> 
> Or:
> 
> 2) Save it to BUILDDIR/runtime-hostdump
> 
> What's your opinion, please ?

I think it should be possible to use LOG_DIR for most of the usecases.
It won't work for the "on target" mode but it should be possible
elsewhere even if we have to tweak the code a bit to ensure the right
variables are passed around?

Cheers,

Richard

Re: [PATCH 1/3] oeqa: make it work for multiple users

[Robert Yang]

On 12/29/18 7:09 AM, richard.purdie@linuxfoundation.org wrote:
> On Fri, 2018-12-28 at 15:57 +0800, Robert Yang wrote:
>>
>> On 12/27/18 7:37 PM, Richard Purdie wrote:
>>> On Thu, 2018-12-27 at 00:59 -0800, Robert Yang wrote:
>>>> There are failures when multiple users run oe-selftest on the
>>>> same
>>>> host:
>>>>
>>>> PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved-
>>>> tests/201812250324_qemu'
>>>>
>>>> This is because /tmp/oe-saved-tests was created by user A, while
>>>> user
>>>> B tries
>>>> to write data in it, then the error will happen. This patch can
>>>> fix
>>>> the problem.
>>>>
>>>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>>>> ---
>>>>    meta/classes/testimage.bbclass   | 2 +-
>>>>    meta/lib/oeqa/runtime/context.py | 7 +++----
>>>>    meta/lib/oeqa/utils/dump.py      | 5 ++++-
>>>>    3 files changed, 8 insertions(+), 6 deletions(-)
>>>
>>> Putting this into /tmp isn't good. Why don't we move this to
>>> LOG_DIR/runtime-hostdump/?
>>
>> I'm afraid that we can't do this, the problem is that we need call
>> get_bb_vars()
>> to get LOG_DIR (d.getVar('LOG_DIR') doesn't work), but get_bb_vars()
>> runs
>> "bitbake -e" which doesn't work when "bitbake <image> -ctestimage"
>> since there
>> is a bitbake server running. We can only get BUILDDIR if we don't
>> call
>> get_bb_vars(), so I think that we have the following 2 solutions:
>>
>> 1) Save it to /tmp/runtime-hostdump
>>
>> Or:
>>
>> 2) Save it to BUILDDIR/runtime-hostdump
>>
>> What's your opinion, please ?
> 
> I think it should be possible to use LOG_DIR for most of the usecases.
> It won't work for the "on target" mode but it should be possible
> elsewhere even if we have to tweak the code a bit to ensure the right
> variables are passed around?

Yes, I agree with that, I will try it.

// Robert

> 
> Cheers,
> 
> Richard
> 
>