[PATCH 0/4] Fix some errors in archiver class

[PATCH 0/4] Fix some errors in archiver class

[mariano.lopez]

From: Mariano Lopez <mariano.lopez@linux.intel.com>

There were some errors in the archiver class that are fixed with
this series:

- Don't expand python functions in dumpdata
    Fixed a datastore expansion for python functions
- Fix use of ARCHIVER_WORKDIR and ARCHIVER_OUTDIR
    In unpack and patch task the outdir was used as workdir
- Fix gcc-source corner case
    gcc-source doesn't have do_configure task used for do_ar_configure
- Fix tar name for git repositories
    For git repositories the file generated for the original
    source had the domain in the file name.

The following changes since commit 8debfea81e69d038bd2d56314b272cb74f5582ed:

  local.conf.sample: Disable prelink by default (2016-03-13 22:09:05 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib mariano/bug9245
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug9245

Mariano Lopez (4):
  archiver.bbclass: Don't expand python functions in dumpdata
  archiver.bbclass: Fix use of ARCHIVER_WORKDIR and ARCHIVER_OUTDIR
  archiver.bbclass: Fix gcc-source corner case
  archiver.bbclass: Fix tar name for git repositories

 meta/classes/archiver.bbclass | 76 +++++++++++++++++++++++--------------------
 1 file changed, 40 insertions(+), 36 deletions(-)

-- 
2.6.2

[PATCH 1/4] archiver.bbclass: Don't expand python functions in dumpdata

[mariano.lopez]

From: Mariano Lopez <mariano.lopez@linux.intel.com>

Currently the dumpdata task expands python data in the datastore, in
some functions this causes a silent error and the task will fail.

The change also rewrite the function to make a bit clearer.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
 meta/classes/archiver.bbclass | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index b95176b..221c7f4 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -335,14 +335,13 @@ python do_dumpdata () {
     dumpfile = os.path.join(d.getVar('ARCHIVER_OUTDIR', True), \
         '%s-showdata.dump' % d.getVar('PF', True))
     bb.note('Dumping metadata into %s' % dumpfile)
-    f = open(dumpfile, 'w')
-    # emit variables and shell functions
-    bb.data.emit_env(f, d, True)
-    # emit the metadata which isn't valid shell
-    for e in d.keys():
-        if bb.data.getVarFlag(e, 'python', d):
-            f.write("\npython %s () {\n%s}\n" % (e, bb.data.getVar(e, d, True)))
-    f.close()
+    with open(dumpfile, "w") as f:
+        # emit variables and shell functions
+        bb.data.emit_env(f, d, True)
+        # emit the metadata which isn't valid shell
+        for e in d.keys():
+            if d.getVarFlag(e, "python", False):
+                f.write("\npython %s () {\n%s}\n" % (e, d.getVar(e, False)))
 }
 
 SSTATETASKS += "do_deploy_archives"
-- 
2.6.2

[PATCH 2/4] archiver.bbclass: Fix use of ARCHIVER_WORKDIR and ARCHIVER_OUTDIR

[mariano.lopez]

From: Mariano Lopez <mariano.lopez@linux.intel.com>

Currently do_unpack_and_patch() and do_ar_configured() are using
the ARCHIVER_OUTDIR as the ARCHIVER_WORKDIR, this lead to have
duplicated files inside the tars when using the archiver class
for patched and configured source.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
 meta/classes/archiver.bbclass | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index 221c7f4..4064674 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -156,8 +156,9 @@ python do_ar_patched() {
 
     # Get the ARCHIVER_OUTDIR before we reset the WORKDIR
     ar_outdir = d.getVar('ARCHIVER_OUTDIR', True)
+    ar_workdir = d.getVar('ARCHIVER_WORKDIR', True)
     bb.note('Archiving the patched source...')
-    d.setVar('WORKDIR', ar_outdir)
+    d.setVar('WORKDIR', ar_workdir)
     create_tarball(d, d.getVar('S', True), 'patched', ar_outdir)
 }
 
@@ -250,21 +251,20 @@ python do_unpack_and_patch() {
             [ 'patched', 'configured'] and \
             d.getVarFlag('ARCHIVER_MODE', 'diff', True) != '1':
         return
-    # Change the WORKDIR to make do_unpack do_patch run in another dir.
     ar_outdir = d.getVar('ARCHIVER_OUTDIR', True)
-    d.setVar('WORKDIR', ar_outdir)
-
-    # The changed 'WORKDIR' also caused 'B' changed, create dir 'B' for the
-    # possibly requiring of the following tasks (such as some recipes's
-    # do_patch required 'B' existed).
-    bb.utils.mkdirhier(d.getVar('B', True))
+    ar_workdir = d.getVar('ARCHIVER_WORKDIR', True)
 
     # The kernel class functions require it to be on work-shared, so we dont change WORKDIR
     if not bb.data.inherits_class('kernel-yocto', d):
-        ar_outdir = d.getVar('ARCHIVER_OUTDIR', True)
-        d.setVar('WORKDIR', ar_outdir)
-        bb.build.exec_func('do_unpack', d)
+        # Change the WORKDIR to make do_unpack do_patch run in another dir.
+        d.setVar('WORKDIR', ar_workdir)
 
+        # The changed 'WORKDIR' also caused 'B' changed, create dir 'B' for the
+        # possibly requiring of the following tasks (such as some recipes's
+        # do_patch required 'B' existed).
+        bb.utils.mkdirhier(d.getVar('B', True))
+
+        bb.build.exec_func('do_unpack', d)
 
     # Save the original source for creating the patches
     if d.getVarFlag('ARCHIVER_MODE', 'diff', True) == '1':
-- 
2.6.2

[PATCH 3/4] archiver.bbclass: Fix gcc-source corner case

[mariano.lopez]

From: Mariano Lopez <mariano.lopez@linux.intel.com>

Bitbake couldn't add the task ar_configured when
trying to archive the configured source for
gcc-source-${PV} recipes. This is because the task
depended in the do_configure and this task doesn't
exist for gcc-source.

This fix allows to archive configured gcc-source recipe.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
 meta/classes/archiver.bbclass | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index 4064674..d709f78 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -73,8 +73,15 @@ python () {
         # We can't use "addtask do_ar_configured after do_configure" since it
         # will cause the deptask of do_populate_sysroot to run not matter what
         # archives we need, so we add the depends here.
-        d.appendVarFlag('do_ar_configured', 'depends', ' %s:do_configure' % pn)
+
+        # There is a corner case with "gcc-source-${PV}" recipes, they don't have
+        # the "do_configure" task, so we need to use "do_preconfigure"
+        if pn.startswith("gcc-source-"):
+            d.appendVarFlag('do_ar_configured', 'depends', ' %s:do_preconfigure' % pn)
+        else:
+            d.appendVarFlag('do_ar_configured', 'depends', ' %s:do_configure' % pn)
         d.appendVarFlag('do_deploy_archives', 'depends', ' %s:do_ar_configured' % pn)
+
     elif ar_src:
         bb.fatal("Invalid ARCHIVER_MODE[src]: %s" % ar_src)
 
@@ -168,11 +175,18 @@ python do_ar_configured() {
     ar_outdir = d.getVar('ARCHIVER_OUTDIR', True)
     if d.getVarFlag('ARCHIVER_MODE', 'src', True) == 'configured':
         bb.note('Archiving the configured source...')
+        pn = d.getVar('PN', True)
+        # "gcc-source-${PV}" recipes don't have "do_configure"
+        # task, so we need to run "do_preconfigure" instead
+        if pn.startswith("gcc-source-"):
+            d.setVar('WORKDIR', d.getVar('ARCHIVER_WORKDIR', True))
+            bb.build.exec_func('do_preconfigure', d)
+
         # The libtool-native's do_configure will remove the
         # ${STAGING_DATADIR}/aclocal/libtool.m4, so we can't re-run the
         # do_configure, we archive the already configured ${S} to
         # instead of.
-        if d.getVar('PN', True) != 'libtool-native':
+        elif pn != 'libtool-native':
             # Change the WORKDIR to make do_configure run in another dir.
             d.setVar('WORKDIR', d.getVar('ARCHIVER_WORKDIR', True))
             if bb.data.inherits_class('kernel-yocto', d):
-- 
2.6.2

[PATCH 4/4] archiver.bbclass: Fix tar name for git repositories

[mariano.lopez]

From: Mariano Lopez <mariano.lopez@linux.intel.com>

When archiving the original source, the git repositories have the name as
they are in the $DL_DIR plus the source revision; i.e.
"git.yoctoproject.org.linux-yocto-4.4.git.89419d8b90_dadb436904.tar.gz".
This change set the tar name to $PF.tar.gz instead, to have consistency with
the others archives created by the class.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
 meta/classes/archiver.bbclass | 21 ++++++---------------
 1 file changed, 6 insertions(+), 15 deletions(-)

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index d709f78..7758f3c 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -126,21 +126,9 @@ python do_ar_original() {
         if os.path.isfile(local):
             shutil.copy(local, ar_outdir)
         elif os.path.isdir(local):
-            basename = os.path.basename(local)
-
             tmpdir = tempfile.mkdtemp(dir=d.getVar('ARCHIVER_WORKDIR', True))
             fetch.unpack(tmpdir, (url,))
-
-            os.chdir(tmpdir)
-            # We eliminate any AUTOINC+ in the revision.
-            try:
-                src_rev = bb.fetch2.get_srcrev(d).replace('AUTOINC+','')
-            except:
-                src_rev = 'NOREV'
-            tarname = os.path.join(ar_outdir, basename + '.' + src_rev + '.tar.gz')
-            tar = tarfile.open(tarname, 'w:gz')
-            tar.add('.')
-            tar.close()
+            create_tarball(d, tmpdir + '/.', '', ar_outdir)
 
     # Emit patch series files for 'original'
     bb.note('Writing patch series files...')
@@ -222,8 +210,11 @@ def create_tarball(d, srcdir, suffix, ar_outdir):
         return
 
     bb.utils.mkdirhier(ar_outdir)
-    tarname = os.path.join(ar_outdir, '%s-%s.tar.gz' % \
-            (d.getVar('PF', True), suffix))
+    if suffix:
+        filename = '%s-%s.tar.gz' % (d.getVar('PF', True), suffix)
+    else:
+        filename = '%s.tar.gz' % d.getVar('PF', True)
+    tarname = os.path.join(ar_outdir, filename)
 
     srcdir = srcdir.rstrip('/')
     dirname = os.path.dirname(srcdir)
-- 
2.6.2