[PATCH 0/5] Enable wic in eSDK

[PATCH 0/5] Enable wic in eSDK

[rebecca.swee.fun.chang]

From: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>

Hi all,

As the subject called out: this patch series enable wic in eSDK.
The details of what I have done are documented within the commit message.
Basically wic requires an OE build environment, but we are using a
different environment setup script in eSDK. Hence, I have added some
code for wic to explicitly export bitbake variables within eSDK. I
have also make wic to use the shared code in scriptpath for oe lib
and bitbake path addition to sys.path.

I have run the changes on wic oe-selftest and the tests are passing.
What's next: I think it would better to have some test cases
for wic within eSDK if this series are merged.

Thanks.

Regards,
Rebecca


The following changes since commit 205cfd702190026e64eed9cae27c05ff62d1637e:

  bitbake: fetch2/__init__: Disable pseudo in runfetchcmd() (2018-01-08 08:48:54 +0000)

are available in the git repository at:

  git://push.yoctoproject.org/poky-contrib rebeccas/wic-dev

Chang Rebecca Swee Fun (5):
  scripts/wic: use scriptpath module to find bitbake path and oe lib
    path
  scripts/wic: append bitbake executable file path in eSDK environment
  scripts/wic: fix error of import wic module in eSDK environment
  scripts/wic: explicitly set BUILDDIR within eSDK
  classes/populate_sdk_ext: support wic in eSDK

 meta/classes/populate_sdk_ext.bbclass |  2 +-
 scripts/wic                           | 23 ++++++++++++++++++-----
 2 files changed, 19 insertions(+), 6 deletions(-)

-- 
2.7.4

[PATCH 1/5] scripts/wic: use scriptpath module to find bitbake path and oe lib path

[rebecca.swee.fun.chang]

From: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>

Use the scriptpath module in order to standardize the adding of
bitbake and meta/lib path to sys.path.

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
---
 scripts/wic | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/scripts/wic b/scripts/wic
index 097084a..0d98875 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -43,13 +43,12 @@ from distutils import spawn
 scripts_path = os.path.abspath(os.path.dirname(__file__))
 lib_path = scripts_path + '/lib'
 sys.path.insert(0, lib_path)
-oe_lib_path = os.path.join(os.path.dirname(scripts_path), 'meta', 'lib')
-sys.path.insert(0, oe_lib_path)
+import scriptpath
+scriptpath.add_oe_lib_path()
 
 bitbake_exe = spawn.find_executable('bitbake')
 if bitbake_exe:
-    bitbake_path = os.path.join(os.path.dirname(bitbake_exe), '../lib')
-    sys.path.insert(0, bitbake_path)
+    bitbake_path = scriptpath.add_bitbake_lib_path()
     from bb import cookerdata
     from bb.main import bitbake_main, BitBakeConfigParameters
 else:
-- 
2.7.4

[PATCH 2/5] scripts/wic: append bitbake executable file path in eSDK environment

[rebecca.swee.fun.chang]

From: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>

wic needs a set of tools to be available from sysroots.
wic will find bitbake executable within the environment,
and wic was unable to locate bitbake executable within eSDK
because it wasn't setup with the OE build environment script.
Hence, we need to add bitbake file path into the environment
PATH for wic to be able to discover it and import bb modules.

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
---
 scripts/wic | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/scripts/wic b/scripts/wic
index 0d98875..293a216 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -46,6 +46,18 @@ sys.path.insert(0, lib_path)
 import scriptpath
 scriptpath.add_oe_lib_path()
 
+# Check whether wic is running within eSDK environment
+sdkroot = scripts_path
+if os.environ.get('SDKTARGETSYSROOT'):
+    while sdkroot != '' and sdkroot != os.sep:
+        if os.path.exists(os.path.join(sdkroot, '.devtoolbase')):
+            # .devtoolbase only exists within eSDK
+            # If found, initialize bitbake path for eSDK environment and append to PATH
+            sdkroot = os.path.join(os.path.dirname(scripts_path), 'bitbake', 'bin')
+            os.environ['PATH'] += ":" + sdkroot
+            break
+        sdkroot = os.path.dirname(sdkroot)
+
 bitbake_exe = spawn.find_executable('bitbake')
 if bitbake_exe:
     bitbake_path = scriptpath.add_bitbake_lib_path()
-- 
2.7.4

[PATCH 3/5] scripts/wic: fix error of import wic module in eSDK environment

[rebecca.swee.fun.chang]

From: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>

wic modules in scripts/lib/ are needed for wic to work, but path to
the python module is not exported in eSDK environment and we were
using an absolutized path of wic script within the sysroots.

We now changed to use real script path instead, where the wic modules
are located. This will also resolved the tracebacks found when running
wic from within the eSDK environment.

Traceback (most recent call last):
  File "/tmp/deploy/sdk/poky_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/wic", line 58, in <module>
    from wic import WicError
ImportError: No module named 'wic'

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
---
 scripts/wic | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/wic b/scripts/wic
index 293a216..d9bea22 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -40,7 +40,7 @@ from collections import namedtuple
 from distutils import spawn
 
 # External modules
-scripts_path = os.path.abspath(os.path.dirname(__file__))
+scripts_path = os.path.dirname(os.path.realpath(__file__))
 lib_path = scripts_path + '/lib'
 sys.path.insert(0, lib_path)
 import scriptpath
-- 
2.7.4

[PATCH 4/5] scripts/wic: explicitly set BUILDDIR within eSDK

[rebecca.swee.fun.chang]

From: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>

When we run wic within eSDK:
$ wic create mkefidisk -e core-image-minimal

ERROR: BUILDDIR not found, exiting. (Did you forget to source oe-init-build-env?)

In order to figure out variable values, one must have sourced
the OE build environment setup script. However, when we are in
within the eSDK environment which isn't initialised like the
normal OE build environment, we can't use wic utility with eSDK.

Reference:
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#wic-requirements

While wic ought to be fixed to be able to run without bitbake
& native tools [YOCTO #11281], but this is a workaround to set
BUILDDIR in the environment so that bitbake environment is setup
for wic to build its required native tools.

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
---
 scripts/wic | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/wic b/scripts/wic
index d9bea22..b02d907 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -55,6 +55,8 @@ if os.environ.get('SDKTARGETSYSROOT'):
             # If found, initialize bitbake path for eSDK environment and append to PATH
             sdkroot = os.path.join(os.path.dirname(scripts_path), 'bitbake', 'bin')
             os.environ['PATH'] += ":" + sdkroot
+            # Set BUILDDIR for wic to work within eSDK
+            os.environ['BUILDDIR'] = sdkroot
             break
         sdkroot = os.path.dirname(sdkroot)
 
-- 
2.7.4

[PATCH 5/5] classes/populate_sdk_ext: support wic in eSDK

[rebecca.swee.fun.chang]

From: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>

Make 'wic' image creation tool/command available in eSDK
environment. This would allow eSDK users to manipulate
images within eSDK environment.

[YOCTO #12177]

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
---
 meta/classes/populate_sdk_ext.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/populate_sdk_ext.bbclass b/meta/classes/populate_sdk_ext.bbclass
index eabc365..4f7100d 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -533,7 +533,7 @@ def get_sdk_required_utilities(buildtools_fn, d):
 
 install_tools() {
 	install -d ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}
-	scripts="devtool recipetool oe-find-native-sysroot runqemu*"
+	scripts="devtool recipetool oe-find-native-sysroot runqemu* wic"
 	for script in $scripts; do
 		for scriptfn in `find ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath} -maxdepth 1 -executable -name "$script"`; do
 			lnr ${scriptfn} ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/`basename $scriptfn`
-- 
2.7.4