[PATCH 0/2] Fix oe.path.copyhardlinktree()

[PATCH 0/2] Fix oe.path.copyhardlinktree()

[Joshua Lock]

My recent change to preserve extended attributes in oe.path resulted in a
copyhardlinktree() being broken when the source contains hidden files.

This series includes a fix for copyhardlinktree() and a simple selftest for 
copyhardlinktree() which ensures the source and destination have the same number
of files after a copy.

The following changes since commit 9b08503eabf78bc1b114416523b41dcce3449f58:

  toaster: fire TaskArtifacts event (2016-09-04 00:05:13 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib joshuagl/copyhard
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=joshuagl/copyhard

Joshua Lock (2):
  oe.path: fix copyhardlinktree()
  selftest/liboe: add a test for copyhardlinktree()

 meta/lib/oe/path.py             |  9 ++++++---
 meta/lib/oeqa/selftest/liboe.py | 29 +++++++++++++++++++++++++++++
 2 files changed, 35 insertions(+), 3 deletions(-)

-- 
2.7.4

[PATCH 1/2] oe.path: fix copyhardlinktree()

[Joshua Lock]

The change to preserve extended attributes in copytree() and
copyhardlinktree() (e591d69103a40ec4f76d1132a6039d9cb1555103)
resulted in an incorrect cp invocation in copyhardlinktree() when
the source directory contained hidden files.

This was because the passed src was modified in place but some code
paths expected it to remain unmodified from the passed value.
Resolve the issue by constructing a new source string, rather than
modifying the passed in string.

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
---
 meta/lib/oe/path.py | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py
index 631c3b4..06a5af2 100644
--- a/meta/lib/oe/path.py
+++ b/meta/lib/oe/path.py
@@ -79,12 +79,15 @@ def copyhardlinktree(src, dst):
         # writers try and create a directory at the same time
         cmd = "cd %s; find . -type d -print | tar --xattrs --xattrs-include='*' -cf - -C %s -p --no-recursion --files-from - | tar --xattrs --xattrs-include='*' -xf - -C %s" % (src, src, dst)
         subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
+        source = ''
         if os.path.isdir(src):
             import glob
             if len(glob.glob('%s/.??*' % src)) > 0:
-                src = src + '/.??* '
-            src = src + '/*'
-        cmd = 'cp -afl --preserve=xattr %s %s' % (src, dst)
+                source = '%s/.??* ' % src
+            source = source + '%s/*' % src
+        else:
+            source = src
+        cmd = 'cp -afl --preserve=xattr %s %s' % (source, dst)
         subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
     else:
         copytree(src, dst)
-- 
2.7.4

[PATCH 2/2] selftest/liboe: add a test for copyhardlinktree()

[Joshua Lock]

Add a simple test to validate that the number of files in the
destination matches the number of files in the source after the
copyhardlinktree() has been performed.

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
---
 meta/lib/oeqa/selftest/liboe.py | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/meta/lib/oeqa/selftest/liboe.py b/meta/lib/oeqa/selftest/liboe.py
index 5c93069..35131eb 100644
--- a/meta/lib/oeqa/selftest/liboe.py
+++ b/meta/lib/oeqa/selftest/liboe.py
@@ -62,3 +62,32 @@ class LibOE(oeSelfTest):
         self.assertIn('user.oetest="testing liboe"', result.output, 'Extended attribute not sert in dst')
 
         oe.path.remove(testloc)
+
+    def test_copy_hardlink_tree_count(self):
+        """
+        Summary:    oe.path.copyhardlinktree() shouldn't miss out files
+        Expected:   src and dst should have the same number of files
+        Product:    OE-Core
+        Author:     Joshua Lock <joshua.g.lock@intel.com>
+        """
+        tmp_dir = get_bb_var('TMPDIR')
+        testloc = oe.path.join(tmp_dir, 'liboetests')
+        src = oe.path.join(testloc, 'src')
+        dst = oe.path.join(testloc, 'dst')
+        bb.utils.mkdirhier(testloc)
+        bb.utils.mkdirhier(src)
+        testfiles = ['foo', 'bar', '.baz', 'quux']
+
+        def touchfile(tf):
+            open(oe.path.join(src, tf), 'w+b').close()
+
+        for f in testfiles:
+            touchfile(f)
+
+        oe.path.copyhardlinktree(src, dst)
+
+        dstcnt = len(os.listdir(dst))
+        srccnt = len(os.listdir(src))
+        self.assertEquals(dstcnt, len(testfiles), "Number of files in dst (%s) differs from number of files in src(%s)." % (dstcnt, srccnt))
+
+        oe.path.remove(testloc)
-- 
2.7.4