[meta-oe][PATCH v2 0/4] runqemu: support non-bootable wic images

[meta-oe][PATCH v2 0/4] runqemu: support non-bootable wic images

[Adrian Freihofer]

Resolves [Yocto 13336]
Compared to v1, no new wic-nb image type is added (this caused problems
with bitbake -c testimage).

runqemu now evaluates the QB_MACHINE setting to determine whether a wic image
should be booted as a vmtype image (kernel loaded from file passed by -kernel
option) or as a fstype image (kernel in image). If QB_MACHINE contains
"-machine virt" runqemu passes a "-kernel" parameter to qemu. Otherwise runqemu
treats the wic image as vmtype image as before. 

The last patch additionally allows the user to enforce one behavior by passing
wic-vm or wic-fs to runqemu. 

Adrian Freihofer (4):
  qemurunner: fix undefined variable
  testimage: consider QB_DEFAULT_FSTYPE
  runqemu: support non-bootable wic images
  runqemu: add wic-fs and wic-wm paramters

 meta/classes/testimage.bbclass    |  6 +++++-
 meta/lib/oeqa/utils/qemurunner.py |  1 +
 scripts/runqemu                   | 40 +++++++++++++++++++++++++++++++++------
 3 files changed, 40 insertions(+), 7 deletions(-)

-- 
2.11.0

[meta-oe][PATCH v2 1/4] qemurunner: fix undefined variable

[Adrian Freihofer]

While hacking on this I got an Exception. It's better to define
variables also in python.

Signe:-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/lib/oeqa/utils/qemurunner.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 6d2860c106..c16227fc38 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -60,6 +60,7 @@ class QemuRunner:
         self.runqemutime = 120
         self.qemu_pidfile = 'pidfile_'+str(os.getpid())
         self.host_dumper = HostDumper(dump_host_cmds, dump_dir)
+        self.monitorpipe = None
 
         self.logger = logger
 
-- 
2.11.0

[meta-oe][PATCH v2 2/4] testimage: consider QB_DEFAULT_FSTYPE

[Adrian Freihofer]

testimage.bbclass starts qemu with the first image type found in
the IMAGE_FSTYPES list. It's weird: this ['wic', 'tar'] works but
this ['tar'. 'wic'] does not. If QB_DEFAULT_FSTYPE is defined,
this fstype is booted.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/classes/testimage.bbclass | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 9bb5a5cb0b..525c5a6173 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -210,7 +210,11 @@ def testimage_main(d):
             bb.fatal('Unsupported image type built. Add a comptible image to '
                      'IMAGE_FSTYPES. Supported types: %s' %
                      ', '.join(supported_fstypes))
-    rootfs = '%s.%s' % (image_name, fstypes[0])
+    qfstype = fstypes[0]
+    qdeffstype = d.getVar("QB_DEFAULT_FSTYPE")
+    if qdeffstype:
+        qfstype = qdeffstype
+    rootfs = '%s.%s' % (image_name, qfstype)
 
     # Get tmpdir (not really used, just for compatibility)
     tmpdir = d.getVar("TMPDIR")
-- 
2.11.0

[meta-oe][PATCH v2 3/4] runqemu: support non-bootable wic images

[Adrian Freihofer]

wic images are handled as vmtype images. Starting qemu with "-kernel"
parameter and an image of type wic is not supported. Especially for
"-machine virt" the combination of wic with -kernel parameter would
be beneficial.

This patch changes the runqemu script to support this. If QB_MACHINE
contains "-machine virt" and the image is of type wic a -kernel
parameter is expected. Otherwise wic images are handled as before.

Example:
QB_DEFAULT_FSTYPE = "wic"
QB_KERNEL_ROOT = "/dev/vda1"
QB_SYSTEM_NAME = "qemu-system-aarch64"
QB_MACHINE = "-machine virt"
...

[YOCTO #13336]

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 scripts/runqemu | 34 ++++++++++++++++++++++++++++------
 1 file changed, 28 insertions(+), 6 deletions(-)

diff --git a/scripts/runqemu b/scripts/runqemu
index af90c010da..71894c9ca8 100755
--- a/scripts/runqemu
+++ b/scripts/runqemu
@@ -185,10 +185,11 @@ class BaseConfig(object):
         self.lock_descriptor = None
         self.bitbake_e = ''
         self.snapshot = False
+        self.wic_fs = None
+        self.wictypes = ('wic', 'wic.vmdk', 'wic.qcow2', 'wic.vdi')
         self.fstypes = ('ext2', 'ext3', 'ext4', 'jffs2', 'nfs', 'btrfs',
                         'cpio.gz', 'cpio', 'ramfs', 'tar.bz2', 'tar.gz')
-        self.vmtypes = ('hddimg', 'hdddirect', 'wic', 'wic.vmdk',
-                        'wic.qcow2', 'wic.vdi', 'iso')
+        self.vmtypes = ('hddimg', 'hdddirect', 'iso')
         self.network_device = "-device e1000,netdev=net0,mac=@MAC@"
         # Use different mac section for tap and slirp to avoid
         # conflicts, e.g., when one is running with tap, the other is
@@ -253,7 +254,7 @@ class BaseConfig(object):
 
     def check_arg_fstype(self, fst):
         """Check and set FSTYPE"""
-        if fst not in self.fstypes + self.vmtypes:
+        if fst not in self.fstypes + self.vmtypes + self.wictypes:
             logger.warning("Maybe unsupported FSTYPE: %s" % fst)
         if not self.fstype or self.fstype == fst:
             if fst == 'ramfs':
@@ -300,7 +301,7 @@ class BaseConfig(object):
             # Check filename against self.fstypes can hanlde <file>.cpio.gz,
             # otherwise, its type would be "gz", which is incorrect.
             fst = ""
-            for t in self.fstypes:
+            for t in self.fstypes + self.wictypes:
                 if p.endswith(t):
                     fst = t
                     break
@@ -390,7 +391,7 @@ class BaseConfig(object):
 
         unknown_arg = ""
         for arg in sys.argv[1:]:
-            if arg in self.fstypes + self.vmtypes:
+            if arg in self.fstypes + self.vmtypes + self.wictypes:
                 self.check_arg_fstype(arg)
             elif arg == 'nographic':
                 self.qemu_opt_script += ' -nographic'
@@ -695,6 +696,21 @@ class BaseConfig(object):
 
     def check_and_set(self):
         """Check configs sanity and set when needed"""
+
+        # Decide how wic images are handled: as vm or with -kernel parameter
+        if self.wic_fs is None:
+            self.wic_fs = False
+            try:
+                qbm = self.get('QB_MACHINE')
+                if re.search('\s+virt(?![^ ])', qbm):
+                    self.wic_fs = True
+            except AttributeError:
+                pass
+        if self.wic_fs is True:
+            self.fstypes = self.fstypes + self.wictypes
+        else:
+            self.vmtypes = self.vmtypes + self.wictypes
+
         self.validate_paths()
         if not self.slirp_enabled:
             check_tun()
@@ -832,7 +848,13 @@ class BaseConfig(object):
             if self.dtb:
                 print('DTB: [%s]' % self.dtb)
         print('MACHINE: [%s]' % self.get('MACHINE'))
-        print('FSTYPE: [%s]' % self.fstype)
+        wic_mode = ''
+        if self.fstype == 'wic':
+            if self.wic_fs:
+                wic_mode = ' (fs)'
+            else:
+                wic_mode = ' (vm)'
+        print('FSTYPE: [%s%s]' % (self.fstype, wic_mode))
         if self.fstype  == 'nfs':
             print('NFS_DIR: [%s]' % self.rootfs)
         else:
-- 
2.11.0

[meta-oe][PATCH v2 4/4] runqemu: add wic-fs and wic-wm paramters

[Adrian Freihofer]

This is an optional follow up for the previous commit. It allows
to define if wic images are handled as vm or as fs type images by
command line argument.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 scripts/runqemu | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/scripts/runqemu b/scripts/runqemu
index 71894c9ca8..c730a72168 100755
--- a/scripts/runqemu
+++ b/scripts/runqemu
@@ -76,6 +76,8 @@ of the following environment variables (in any order):
     publicvnc - enable a VNC server open to all hosts
     audio - enable audio
     [*/]ovmf* - OVMF firmware file or base name for booting with UEFI
+    wic-fs - Start Qemu with -kernel parameter for wic images (default if -machine virt)
+    wic-vm - Start Qemu without -kernel parameter for wic images (default otherwise)
   tcpserial=<port> - specify tcp serial port number
   biosdir=<dir> - specify custom bios dir
   biosfilename=<filename> - specify bios filename
@@ -455,6 +457,10 @@ class BaseConfig(object):
                 self.rootfs = arg
             elif arg.startswith('ovmf'):
                 self.ovmf_bios.append(arg)
+            elif arg == 'wic-fs':
+                self.wic_fs = True
+            elif arg == 'wic-vm':
+                self.wic_fs = False
             else:
                 # At last, assume it is the MACHINE
                 if (not unknown_arg) or unknown_arg == arg:
-- 
2.11.0

✗ patchtest: failure for runqemu: support non-bootable wic images

[Patchwork]

== Series Details ==

Series: runqemu: support non-bootable wic images
Revision: 1
URL   : https://patchwork.openembedded.org/series/18046/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Patch            [meta-oe,v2,1/4] qemurunner: fix undefined variable
 Issue             Patch is missing Signed-off-by [test_signed_off_by_presence] 
  Suggested fix    Sign off the patch (either manually or with "git commit --amend -s")



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

[meta-oe][PATCH v3 0/4] runqemu: support non-bootable wic images

[Adrian Freihofer]

Please ignore v2. This version 3 only clears up the mess in version 2.

Resolves [Yocto 13336]
Compared to v1, no new wic-nb image type is added (this caused problems
with bitbake -c testimage).

runqemu now evaluates the QB_MACHINE setting to determine whether a wic image
should be booted as a vmtype image (kernel loaded from file passed by -kernel
option) or as a fstype image (kernel in image). If QB_MACHINE contains
"-machine virt" runqemu passes a "-kernel" parameter to qemu. Otherwise
runqemu treats the wic image as vmtype image as before. 

The last patch additionally allows the user to enforce one behavior by passing
wic-vm or wic-fs to runqemu. 


One more comment on v2: The problem with v2 probably arose because I sent the
series from a new email address before registering the address. The patches
seemed to be lost. Half an hour after registering the new e-mail address I
resent them. Then this mess appeared on the mailing list.


Adrian Freihofer (4):
  qemurunner: fix undefined variable
  testimage: consider QB_DEFAULT_FSTYPE
  runqemu: support non-bootable wic images
  runqemu: add wic-fs and wic-wm paramters

 meta/classes/testimage.bbclass    |  6 +++++-
 meta/lib/oeqa/utils/qemurunner.py |  1 +
 scripts/runqemu                   | 40 +++++++++++++++++++++++++++++++++------
 3 files changed, 40 insertions(+), 7 deletions(-)

-- 
2.11.0

✗ patchtest: failure for runqemu: support non-bootable wic images

[Patchwork]

== Series Details ==

Series: runqemu: support non-bootable wic images
Revision: 1
URL   : https://patchwork.openembedded.org/series/18051/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Patch            [meta-oe,v3,1/4] qemurunner: fix undefined variable
 Issue             Patch is missing Signed-off-by [test_signed_off_by_presence] 
  Suggested fix    Sign off the patch (either manually or with "git commit --amend -s")



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

[meta-oe][PATCH v4 0/3] runqemu: support non-bootable wic images

[Adrian Freihofer]

Compared to V3 this is a new implementation, as suggested by Richard.
Thanks for the hint, it makes perfect sense.

Adrian Freihofer (3):
  qemurunner: fix undefined variable
  testimage: consider QB_DEFAULT_FSTYPE
  runqemu: QB_FSINFO to support fstype wic images

 meta/classes/testimage.bbclass    |  6 ++++-
 meta/lib/oeqa/utils/qemurunner.py |  1 +
 scripts/runqemu                   | 49 +++++++++++++++++++++++++++++++++++----
 3 files changed, 50 insertions(+), 6 deletions(-)

-- 
2.11.0

✗ patchtest: failure for runqemu: support non-bootable wic images

[Patchwork]

== Series Details ==

Series: runqemu: support non-bootable wic images
Revision: 1
URL   : https://patchwork.openembedded.org/series/18057/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Patch            [meta-oe,v4,1/3] qemurunner: fix undefined variable
 Issue             Series sent to the wrong mailing list [test_target_mailing_list] 
  Suggested fix    Check the project's README (meta-oe,v4,1/3) and send the patch to the indicated list

* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at 00fa839136)

* Patch            [meta-oe,v4,1/3] qemurunner: fix undefined variable
 Issue             Patch is missing Signed-off-by [test_signed_off_by_presence] 
  Suggested fix    Sign off the patch (either manually or with "git commit --amend -s")



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe