[PATCHv2 0/5] Enable wic in eSDK

[PATCHv2 0/5] Enable wic in eSDK

[rebecca.swee.fun.chang]

From: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>

Hi all,

Resend as v2 as I realized an issue with BUILDDIR being set
was not correct. I made the correction on Patch 4 by setting
BUILDDIR to sdkroot variable (eSDK base path) before we alter
the variable for bitbake executable file path.

Regards,
Rebecca

-----

Hi all,

As the subject called out: this patch series enable wic in eSDK.
The details of what I have done are documented within the commit message.
Basically wic requires an OE build environment, but we are using a
different environment setup script in eSDK. Hence, I have added some
code for wic to explicitly export bitbake variables within eSDK. I
have also make wic to use the shared code in scriptpath for oe lib
and bitbake path addition to sys.path.

I have run the changes on wic oe-selftest and the tests are passing.
What's next: I think it would better to have some test cases
for wic within eSDK if this series are merged.

Thanks.

Regards,
Rebecca


The following changes since commit 364f8bcfcbd04e722490f363ad36a15fb7066ba7:

  linux-firmware: Bump revision to 65b1c68c (2018-01-11 10:26:07 +0000)

are available in the Git repository at:

  git://push.yoctoproject.org/poky-contrib rebeccas/wic-dev

Chang Rebecca Swee Fun (5):
  scripts/wic: use scriptpath module to find bitbake path and oe lib
    path
  scripts/wic: append bitbake executable file path in eSDK environment
  scripts/wic: fix error of import wic module in eSDK environment
  scripts/wic: explicitly set BUILDDIR within eSDK
  classes/populate_sdk_ext: support wic in eSDK

 meta/classes/populate_sdk_ext.bbclass |  2 +-
 scripts/wic                           | 23 ++++++++++++++++++-----
 2 files changed, 19 insertions(+), 6 deletions(-)

-- 
2.15.0

[PATCHv2 1/5] scripts/wic: use scriptpath module to find bitbake path and oe lib path

[rebecca.swee.fun.chang]

From: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>

Use the scriptpath module in order to standardize the adding of
bitbake and meta/lib path to sys.path.

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
---
 scripts/wic | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/scripts/wic b/scripts/wic
index 097084a6033..0d988757150 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -43,13 +43,12 @@ from distutils import spawn
 scripts_path = os.path.abspath(os.path.dirname(__file__))
 lib_path = scripts_path + '/lib'
 sys.path.insert(0, lib_path)
-oe_lib_path = os.path.join(os.path.dirname(scripts_path), 'meta', 'lib')
-sys.path.insert(0, oe_lib_path)
+import scriptpath
+scriptpath.add_oe_lib_path()
 
 bitbake_exe = spawn.find_executable('bitbake')
 if bitbake_exe:
-    bitbake_path = os.path.join(os.path.dirname(bitbake_exe), '../lib')
-    sys.path.insert(0, bitbake_path)
+    bitbake_path = scriptpath.add_bitbake_lib_path()
     from bb import cookerdata
     from bb.main import bitbake_main, BitBakeConfigParameters
 else:
-- 
2.15.0

[PATCHv2 2/5] scripts/wic: append bitbake executable file path in eSDK environment

[rebecca.swee.fun.chang]

From: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>

wic needs a set of tools to be available from sysroots.
wic will find bitbake executable within the environment,
and wic was unable to locate bitbake executable within eSDK
because it wasn't setup with the OE build environment script.
Hence, we need to add bitbake file path into the environment
PATH for wic to be able to discover it and import bb modules.

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
---
 scripts/wic | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/scripts/wic b/scripts/wic
index 0d988757150..293a216d71d 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -46,6 +46,18 @@ sys.path.insert(0, lib_path)
 import scriptpath
 scriptpath.add_oe_lib_path()
 
+# Check whether wic is running within eSDK environment
+sdkroot = scripts_path
+if os.environ.get('SDKTARGETSYSROOT'):
+    while sdkroot != '' and sdkroot != os.sep:
+        if os.path.exists(os.path.join(sdkroot, '.devtoolbase')):
+            # .devtoolbase only exists within eSDK
+            # If found, initialize bitbake path for eSDK environment and append to PATH
+            sdkroot = os.path.join(os.path.dirname(scripts_path), 'bitbake', 'bin')
+            os.environ['PATH'] += ":" + sdkroot
+            break
+        sdkroot = os.path.dirname(sdkroot)
+
 bitbake_exe = spawn.find_executable('bitbake')
 if bitbake_exe:
     bitbake_path = scriptpath.add_bitbake_lib_path()
-- 
2.15.0

[PATCHv2 3/5] scripts/wic: fix error of import wic module in eSDK environment

[rebecca.swee.fun.chang]

From: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>

wic modules in scripts/lib/ are needed for wic to work, but path to
the python module is not exported in eSDK environment and we were
using an absolutized path of wic script within the sysroots.

We now changed to use real script path instead, where the wic modules
are located. This will also resolved the tracebacks found when running
wic from within the eSDK environment.

Traceback (most recent call last):
  File "/tmp/deploy/sdk/poky_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/wic", line 58, in <module>
    from wic import WicError
ImportError: No module named 'wic'

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
---
 scripts/wic | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/wic b/scripts/wic
index 293a216d71d..d9bea228ad5 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -40,7 +40,7 @@ from collections import namedtuple
 from distutils import spawn
 
 # External modules
-scripts_path = os.path.abspath(os.path.dirname(__file__))
+scripts_path = os.path.dirname(os.path.realpath(__file__))
 lib_path = scripts_path + '/lib'
 sys.path.insert(0, lib_path)
 import scriptpath
-- 
2.15.0

[PATCHv2 4/5] scripts/wic: explicitly set BUILDDIR within eSDK

[rebecca.swee.fun.chang]

From: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>

When we run wic within eSDK:
$ wic create mkefidisk -e core-image-minimal

ERROR: BUILDDIR not found, exiting. (Did you forget to source oe-init-build-env?)

In order to figure out variable values, one must have sourced
the OE build environment setup script. However, when we are in
within the eSDK environment which isn't initialised like the
normal OE build environment, we can't use wic utility with eSDK.

Reference:
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#wic-requirements

While wic ought to be fixed to be able to run without bitbake
& native tools [YOCTO #11281], but this is a workaround to set
BUILDDIR in the environment so that bitbake environment is setup
for wic to build its required native tools.

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
---
 scripts/wic | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/wic b/scripts/wic
index d9bea228ad5..7392bc4e7f4 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -51,6 +51,8 @@ sdkroot = scripts_path
 if os.environ.get('SDKTARGETSYSROOT'):
     while sdkroot != '' and sdkroot != os.sep:
         if os.path.exists(os.path.join(sdkroot, '.devtoolbase')):
+            # Set BUILDDIR for wic to work within eSDK
+            os.environ['BUILDDIR'] = sdkroot
             # .devtoolbase only exists within eSDK
             # If found, initialize bitbake path for eSDK environment and append to PATH
             sdkroot = os.path.join(os.path.dirname(scripts_path), 'bitbake', 'bin')
-- 
2.15.0

[PATCHv2 5/5] classes/populate_sdk_ext: support wic in eSDK

[rebecca.swee.fun.chang]

From: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>

Make 'wic' image creation tool/command available in eSDK
environment. This would allow eSDK users to manipulate
images within eSDK environment.

[YOCTO #12177]

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
---
 meta/classes/populate_sdk_ext.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/populate_sdk_ext.bbclass b/meta/classes/populate_sdk_ext.bbclass
index eabc365300a..4f7100dd7e1 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -533,7 +533,7 @@ def get_sdk_required_utilities(buildtools_fn, d):
 
 install_tools() {
 	install -d ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}
-	scripts="devtool recipetool oe-find-native-sysroot runqemu*"
+	scripts="devtool recipetool oe-find-native-sysroot runqemu* wic"
 	for script in $scripts; do
 		for scriptfn in `find ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath} -maxdepth 1 -executable -name "$script"`; do
 			lnr ${scriptfn} ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/`basename $scriptfn`
-- 
2.15.0

Re: [PATCHv2 0/5] Enable wic in eSDK

[Ed Bartosh]

On Thu, Jan 11, 2018 at 10:55:18PM +0800, rebecca.swee.fun.chang@intel.com wrote:
> Hi all,
> 
> As the subject called out: this patch series enable wic in eSDK.
> The details of what I have done are documented within the commit message.
> Basically wic requires an OE build environment, but we are using a
> different environment setup script in eSDK. Hence, I have added some
> code for wic to explicitly export bitbake variables within eSDK. I
> have also make wic to use the shared code in scriptpath for oe lib
> and bitbake path addition to sys.path.
> 
> I have run the changes on wic oe-selftest and the tests are passing.
> What's next: I think it would better to have some test cases
> for wic within eSDK if this series are merged.
> 

Would it make sense to cover this new functionality by tests?

> 
> The following changes since commit 364f8bcfcbd04e722490f363ad36a15fb7066ba7:
> 
>   linux-firmware: Bump revision to 65b1c68c (2018-01-11 10:26:07 +0000)
> 
> are available in the Git repository at:
> 
>   git://push.yoctoproject.org/poky-contrib rebeccas/wic-dev
> 
> Chang Rebecca Swee Fun (5):
>   scripts/wic: use scriptpath module to find bitbake path and oe lib
>     path
>   scripts/wic: append bitbake executable file path in eSDK environment
>   scripts/wic: fix error of import wic module in eSDK environment
>   scripts/wic: explicitly set BUILDDIR within eSDK
>   classes/populate_sdk_ext: support wic in eSDK
> 
>  meta/classes/populate_sdk_ext.bbclass |  2 +-
>  scripts/wic                           | 23 ++++++++++++++++++-----
>  2 files changed, 19 insertions(+), 6 deletions(-)
> 
> -- 
> 2.15.0
> 
> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

-- 
--
Regards,
Ed

Re: [PATCHv2 0/5] Enable wic in eSDK

[Chang, Rebecca Swee Fun]

Yes. The tests are in development. Stay tuned.

> -----Original Message-----
> From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com]
> Sent: Friday, January 12, 2018 5:21 PM
> To: Chang, Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCHv2 0/5] Enable wic in eSDK
> 
> On Thu, Jan 11, 2018 at 10:55:18PM +0800,
> rebecca.swee.fun.chang@intel.com wrote:
> > Hi all,
> >
> > As the subject called out: this patch series enable wic in eSDK.
> > The details of what I have done are documented within the commit message.
> > Basically wic requires an OE build environment, but we are using a
> > different environment setup script in eSDK. Hence, I have added some
> > code for wic to explicitly export bitbake variables within eSDK. I
> > have also make wic to use the shared code in scriptpath for oe lib and
> > bitbake path addition to sys.path.
> >
> > I have run the changes on wic oe-selftest and the tests are passing.
> > What's next: I think it would better to have some test cases for wic
> > within eSDK if this series are merged.
> >
> 
> Would it make sense to cover this new functionality by tests?
> 
> >
> > The following changes since commit
> 364f8bcfcbd04e722490f363ad36a15fb7066ba7:
> >
> >   linux-firmware: Bump revision to 65b1c68c (2018-01-11 10:26:07
> > +0000)
> >
> > are available in the Git repository at:
> >
> >   git://push.yoctoproject.org/poky-contrib rebeccas/wic-dev
> >
> > Chang Rebecca Swee Fun (5):
> >   scripts/wic: use scriptpath module to find bitbake path and oe lib
> >     path
> >   scripts/wic: append bitbake executable file path in eSDK environment
> >   scripts/wic: fix error of import wic module in eSDK environment
> >   scripts/wic: explicitly set BUILDDIR within eSDK
> >   classes/populate_sdk_ext: support wic in eSDK
> >
> >  meta/classes/populate_sdk_ext.bbclass |  2 +-
> >  scripts/wic                           | 23 ++++++++++++++++++-----
> >  2 files changed, 19 insertions(+), 6 deletions(-)
> >
> > --
> > 2.15.0
> >
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> --
> --
> Regards,
> Ed