* [bitbake][kirkstone][2.0][PATCH 1/5] gitsm: Add call_process_submodules() to remove duplicated code
2024-10-30 16:49 [bitbake][kirkstone][2.0][PATCH 0/5] Patch review Steve Sakoman
@ 2024-10-30 16:49 ` Steve Sakoman
2024-10-30 16:49 ` [bitbake][kirkstone][2.0][PATCH 2/5] gitsm: Remove downloads/tmpdir when failed Steve Sakoman
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Steve Sakoman @ 2024-10-30 16:49 UTC (permalink / raw)
To: bitbake-devel
From: Robert Yang <liezhi.yang@windriver.com>
There are 14 lines can be removed, and can make it easy to maintain.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0ea2c1ac079d63349407a69172ff80cd9acc7252)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
lib/bb/fetch2/gitsm.py | 42 ++++++++++++++----------------------------
1 file changed, 14 insertions(+), 28 deletions(-)
diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch2/gitsm.py
index c5f7c03c4..0699645fb 100644
--- a/lib/bb/fetch2/gitsm.py
+++ b/lib/bb/fetch2/gitsm.py
@@ -139,6 +139,17 @@ class GitSM(Git):
return submodules != []
+ def call_process_submodules(self, ud, d, extra_check, subfunc):
+ # If we're using a shallow mirror tarball it needs to be
+ # unpacked temporarily so that we can examine the .gitmodules file
+ if ud.shallow and os.path.exists(ud.fullshallow) and extra_check:
+ tmpdir = tempfile.mkdtemp(dir=d.getVar("DL_DIR"))
+ runfetchcmd("tar -xzf %s" % ud.fullshallow, d, workdir=tmpdir)
+ self.process_submodules(ud, tmpdir, subfunc, d)
+ shutil.rmtree(tmpdir)
+ else:
+ self.process_submodules(ud, ud.clonedir, subfunc, d)
+
def need_update(self, ud, d):
if Git.need_update(self, ud, d):
return True
@@ -156,15 +167,7 @@ class GitSM(Git):
logger.error('gitsm: submodule update check failed: %s %s' % (type(e).__name__, str(e)))
need_update_result = True
- # If we're using a shallow mirror tarball it needs to be unpacked
- # temporarily so that we can examine the .gitmodules file
- if ud.shallow and os.path.exists(ud.fullshallow) and not os.path.exists(ud.clonedir):
- tmpdir = tempfile.mkdtemp(dir=d.getVar("DL_DIR"))
- runfetchcmd("tar -xzf %s" % ud.fullshallow, d, workdir=tmpdir)
- self.process_submodules(ud, tmpdir, need_update_submodule, d)
- shutil.rmtree(tmpdir)
- else:
- self.process_submodules(ud, ud.clonedir, need_update_submodule, d)
+ self.call_process_submodules(ud, d, not os.path.exists(ud.clonedir), need_update_submodule)
if need_update_list:
logger.debug('gitsm: Submodules requiring update: %s' % (' '.join(need_update_list)))
@@ -187,16 +190,7 @@ class GitSM(Git):
raise
Git.download(self, ud, d)
-
- # If we're using a shallow mirror tarball it needs to be unpacked
- # temporarily so that we can examine the .gitmodules file
- if ud.shallow and os.path.exists(ud.fullshallow) and self.need_update(ud, d):
- tmpdir = tempfile.mkdtemp(dir=d.getVar("DL_DIR"))
- runfetchcmd("tar -xzf %s" % ud.fullshallow, d, workdir=tmpdir)
- self.process_submodules(ud, tmpdir, download_submodule, d)
- shutil.rmtree(tmpdir)
- else:
- self.process_submodules(ud, ud.clonedir, download_submodule, d)
+ self.call_process_submodules(ud, d, self.need_update(ud, d), download_submodule)
def unpack(self, ud, destdir, d):
def unpack_submodules(ud, url, module, modpath, workdir, d):
@@ -249,14 +243,6 @@ class GitSM(Git):
newfetch = Fetch([url], d, cache=False)
urldata.extend(newfetch.expanded_urldata())
- # If we're using a shallow mirror tarball it needs to be unpacked
- # temporarily so that we can examine the .gitmodules file
- if ud.shallow and os.path.exists(ud.fullshallow) and ud.method.need_update(ud, d):
- tmpdir = tempfile.mkdtemp(dir=d.getVar("DL_DIR"))
- subprocess.check_call("tar -xzf %s" % ud.fullshallow, cwd=tmpdir, shell=True)
- self.process_submodules(ud, tmpdir, add_submodule, d)
- shutil.rmtree(tmpdir)
- else:
- self.process_submodules(ud, ud.clonedir, add_submodule, d)
+ self.call_process_submodules(ud, d, ud.method.need_update(ud, d), add_submodule)
return urldata
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [bitbake][kirkstone][2.0][PATCH 2/5] gitsm: Remove downloads/tmpdir when failed
2024-10-30 16:49 [bitbake][kirkstone][2.0][PATCH 0/5] Patch review Steve Sakoman
2024-10-30 16:49 ` [bitbake][kirkstone][2.0][PATCH 1/5] gitsm: Add call_process_submodules() to remove duplicated code Steve Sakoman
@ 2024-10-30 16:49 ` Steve Sakoman
2024-10-30 16:49 ` [bitbake][kirkstone][2.0][PATCH 3/5] tests/fetch: Use our own mirror of sysprof to decouple from gnome gitlab Steve Sakoman
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Steve Sakoman @ 2024-10-30 16:49 UTC (permalink / raw)
To: bitbake-devel
From: Robert Yang <liezhi.yang@windriver.com>
The tmpdir such as downloads/tmplp3cnemv won't be removed without this fix.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2ba8d3214759142afc11f0a88d80eb30a8bcde3a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
lib/bb/fetch2/gitsm.py | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch2/gitsm.py
index 0699645fb..488e99cde 100644
--- a/lib/bb/fetch2/gitsm.py
+++ b/lib/bb/fetch2/gitsm.py
@@ -144,9 +144,11 @@ class GitSM(Git):
# unpacked temporarily so that we can examine the .gitmodules file
if ud.shallow and os.path.exists(ud.fullshallow) and extra_check:
tmpdir = tempfile.mkdtemp(dir=d.getVar("DL_DIR"))
- runfetchcmd("tar -xzf %s" % ud.fullshallow, d, workdir=tmpdir)
- self.process_submodules(ud, tmpdir, subfunc, d)
- shutil.rmtree(tmpdir)
+ try:
+ runfetchcmd("tar -xzf %s" % ud.fullshallow, d, workdir=tmpdir)
+ self.process_submodules(ud, tmpdir, subfunc, d)
+ finally:
+ shutil.rmtree(tmpdir)
else:
self.process_submodules(ud, ud.clonedir, subfunc, d)
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [bitbake][kirkstone][2.0][PATCH 3/5] tests/fetch: Use our own mirror of sysprof to decouple from gnome gitlab
2024-10-30 16:49 [bitbake][kirkstone][2.0][PATCH 0/5] Patch review Steve Sakoman
2024-10-30 16:49 ` [bitbake][kirkstone][2.0][PATCH 1/5] gitsm: Add call_process_submodules() to remove duplicated code Steve Sakoman
2024-10-30 16:49 ` [bitbake][kirkstone][2.0][PATCH 2/5] gitsm: Remove downloads/tmpdir when failed Steve Sakoman
@ 2024-10-30 16:49 ` Steve Sakoman
2024-10-30 16:49 ` [bitbake][kirkstone][2.0][PATCH 4/5] tests/fetch: Use our own mirror of mobile-broadband-provider " Steve Sakoman
2024-10-30 16:49 ` [bitbake][kirkstone][2.0][PATCH 5/5] bitbake-user-manual-ref-variables: update BB_HASHSERVE_UPSTREAM for new infrastructure Steve Sakoman
4 siblings, 0 replies; 6+ messages in thread
From: Steve Sakoman @ 2024-10-30 16:49 UTC (permalink / raw)
To: bitbake-devel
From: Richard Purdie <richard.purdie@linuxfoundation.org>
GNOME gitlab has occasional downtime which impacts bitbake-selftest
and causes autobuilder failures. Switch to our own mirror for test
purposes to avoid those issues.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 008808755ed6cfeb6c41273e69ce718f0833c26c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
lib/bb/tests/fetch.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py
index efe5479a6..8595d9295 100644
--- a/lib/bb/tests/fetch.py
+++ b/lib/bb/tests/fetch.py
@@ -1333,7 +1333,7 @@ class FetchLatestVersionTest(FetcherTest):
("dtc", "git://git.yoctoproject.org/bbfetchtests-dtc.git;branch=master", "65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf", "")
: "1.4.0",
# combination version pattern
- ("sysprof", "git://gitlab.gnome.org/GNOME/sysprof.git;protocol=https;branch=master", "cd44ee6644c3641507fb53b8a2a69137f2971219", "")
+ ("sysprof", "git://git.yoctoproject.org/sysprof.git;protocol=https;branch=master", "cd44ee6644c3641507fb53b8a2a69137f2971219", "")
: "1.2.0",
("u-boot-mkimage", "git://source.denx.de/u-boot/u-boot.git;branch=master;protocol=https", "62c175fbb8a0f9a926c88294ea9f7e88eb898f6c", "")
: "2014.01",
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread