[1.52][PATCH 0/2] Review request

[1.52][PATCH 0/2] Review request

[Anuj Mittal]

Please review these changes for 1.52. No problems seen while testing.

Thanks,

Anuj

The following changes since commit c1938abf51b57938a21948bb414ad0467e4368d9:

  cooker: Fix task-depends.dot for multiconfig targets (2021-11-16 13:13:20 +0800)

are available in the Git repository at:

  git://push.openembedded.org/bitbake-contrib stable/1.52-next

Richard Purdie (2):
  utils: Handle lockfile filenames that are too long for filesystems
  fetch2: Fix url remap issue and add testcase

 lib/bb/fetch2/__init__.py | 2 +-
 lib/bb/tests/fetch.py     | 1 +
 lib/bb/utils.py           | 6 +++++-
 3 files changed, 7 insertions(+), 2 deletions(-)

-- 
2.33.1

[1.52][PATCH 1/2] utils: Handle lockfile filenames that are too long for filesystems

[Anuj Mittal]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

The fetcher mirror code can go crazy creating lock filenames which exceed the
filesystem limits. When this happens, the code will loop/hang.

Handle the filename too long exception correctly but also truncate lockfile
lengths to under 256 since the worst case situation is lockfile overlap
and lack of parallelism.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 63baf3440b16e41ac6601de21ced94a94bdf1509)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
---
 lib/bb/utils.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/lib/bb/utils.py b/lib/bb/utils.py
index 70634910..d890ea83 100644
--- a/lib/bb/utils.py
+++ b/lib/bb/utils.py
@@ -451,6 +451,10 @@ def lockfile(name, shared=False, retry=True, block=False):
     consider the possibility of sending a signal to the process to break
     out - at which point you want block=True rather than retry=True.
     """
+    if len(name) > 255:
+        root, ext = os.path.splitext(name)
+        name = root[:255 - len(ext)] + ext
+
     dirname = os.path.dirname(name)
     mkdirhier(dirname)
 
@@ -487,7 +491,7 @@ def lockfile(name, shared=False, retry=True, block=False):
                     return lf
             lf.close()
         except OSError as e:
-            if e.errno == errno.EACCES:
+            if e.errno == errno.EACCES or e.errno == errno.ENAMETOOLONG:
                 logger.error("Unable to acquire lock '%s', %s",
                              e.strerror, name)
                 sys.exit(1)
-- 
2.33.1

[1.52][PATCH 2/2] fetch2: Fix url remap issue and add testcase

[Anuj Mittal]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

Using "" as a target for .replace() is a really bad idea as it duplicates the replacement
for every character in the string. Add a testcase which triggered this and correct the
code to return the correct result.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3af1ecf049d2eed56f6d319dc7df6eb4a3d4eebc)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
---
 lib/bb/fetch2/__init__.py | 2 +-
 lib/bb/tests/fetch.py     | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index b73b3695..25179bc2 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -473,7 +473,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
                     basename = os.path.basename(ud.localpath)
                 if basename:
                     uri_basename = os.path.basename(uri_decoded[loc])
-                    if basename != uri_basename and result_decoded[loc].endswith(uri_basename):
+                    if uri_basename and basename != uri_basename and result_decoded[loc].endswith(uri_basename):
                         result_decoded[loc] = result_decoded[loc].replace(uri_basename, basename)
                     elif not result_decoded[loc].endswith(basename):
                         result_decoded[loc] = os.path.join(result_decoded[loc], basename)
diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py
index 7edbf50b..a85ebdf4 100644
--- a/lib/bb/tests/fetch.py
+++ b/lib/bb/tests/fetch.py
@@ -434,6 +434,7 @@ class MirrorUriTest(FetcherTest):
         ("git://user1@someserver.org/bitbake;tag=1234567890123456789012345678901234567890;branch=master", "git://someserver.org/bitbake;branch=master", "git://user2@git.openembedded.org/bitbake;protocol=http")
             : "git://user2@git.openembedded.org/bitbake;tag=1234567890123456789012345678901234567890;branch=master;protocol=http",
 
+        ("gitsm://git.qemu.org/git/seabios.git/;protocol=https;name=roms/seabios;subpath=roms/seabios;bareclone=1;nobranch=1;rev=1234567890123456789012345678901234567890", "gitsm://.*/.*", "http://petalinux.xilinx.com/sswreleases/rel-v${XILINX_VER_MAIN}/downloads") : "http://petalinux.xilinx.com/sswreleases/rel-v%24%7BXILINX_VER_MAIN%7D/downloads/git2_git.qemu.org.git.seabios.git..tar.gz",
 
         #Renaming files doesn't work
         #("http://somewhere.org/somedir1/somefile_1.2.3.tar.gz", "http://somewhere.org/somedir1/somefile_1.2.3.tar.gz", "http://somewhere2.org/somedir3/somefile_2.3.4.tar.gz") : "http://somewhere2.org/somedir3/somefile_2.3.4.tar.gz"
-- 
2.33.1

[1.52][PATCH 0/2] Review request

[Anuj Mittal]

Please review and merge these changes for 1.52.

Thanks,

Anuj

The following changes since commit 20eae05fdd6cb7ace87ad005f72c256e2fddb3d0:

  fetch2/perforce: Fix typo (2021-10-26 13:47:06 +0100)

are available in the Git repository at:

  git://push.openembedded.org/bitbake-contrib stable/1.52-next

Richard Purdie (2):
  tests/runqueue: Ensure hashserv exits before deleting files
  bitbake-worker: Add debug when unpickle fails

 bin/bitbake-worker       | 6 +++++-
 lib/bb/tests/runqueue.py | 2 +-
 2 files changed, 6 insertions(+), 2 deletions(-)

-- 
2.31.1