* [bitbake][kirkstone][2.0][PATCH 1/3] fetch2/osc: Small fixes for osc fetcher
2022-05-23 14:40 [bitbake][kirkstone][2.0][PATCH 0/3] Patch review Steve Sakoman
@ 2022-05-23 14:40 ` Steve Sakoman
2022-05-23 14:40 ` [bitbake][kirkstone][2.0][PATCH 2/3] data: Do not depend on vardepvalueexclude flag Steve Sakoman
2022-05-23 14:40 ` [bitbake][kirkstone][2.0][PATCH 3/3] build: Add clean_stamp API function to allow removal of task stamps Steve Sakoman
2 siblings, 0 replies; 5+ messages in thread
From: Steve Sakoman @ 2022-05-23 14:40 UTC (permalink / raw)
To: bitbake-devel
From: Gunjan Gupta <viraniac@gmail.com>
The current fetcher seemed to have some issues that made it difficult when
trying to use the same. This patch fixes the following
* Make consistent use of the path that needs to be used as oscdir
* The path mentioned in os.access in download function was not same as
ud.moddir which would result into invoking of fetch command instead of
update command even if directory already existed
* Before creating oscrc, make sure oscdir exists and create it if it does
not exist
* Updated the configuration to use apiurl and added a new parameter to
control whether http or https needs to be used to connect to apiurl
Signed-off-by: Gunjan Gupta <viraniac@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3ec78686f3c0ea2304097b86a965f9be4b0cb879)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
lib/bb/fetch2/osc.py | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/lib/bb/fetch2/osc.py b/lib/bb/fetch2/osc.py
index 99a529e5..eb0f82c8 100644
--- a/lib/bb/fetch2/osc.py
+++ b/lib/bb/fetch2/osc.py
@@ -36,6 +36,7 @@ class Osc(FetchMethod):
# Create paths to osc checkouts
oscdir = d.getVar("OSCDIR") or (d.getVar("DL_DIR") + "/osc")
relpath = self._strip_leading_slashes(ud.path)
+ ud.oscdir = oscdir
ud.pkgdir = os.path.join(oscdir, ud.host)
ud.moddir = os.path.join(ud.pkgdir, relpath, ud.module)
@@ -49,7 +50,7 @@ class Osc(FetchMethod):
else:
ud.revision = ""
- ud.localfile = d.expand('%s_%s_%s.tar.gz' % (ud.module.replace('/', '.'), ud.path.replace('/', '.'), ud.revision))
+ ud.localfile = d.expand('%s_%s_%s.tar.gz' % (ud.module.replace('/', '.'), relpath.replace('/', '.'), ud.revision))
def _buildosccommand(self, ud, d, command):
"""
@@ -86,7 +87,7 @@ class Osc(FetchMethod):
logger.debug2("Fetch: checking for module directory '" + ud.moddir + "'")
- if os.access(os.path.join(d.getVar('OSCDIR'), ud.path, ud.module), os.R_OK):
+ if os.access(ud.moddir, os.R_OK):
oscupdatecmd = self._buildosccommand(ud, d, "update")
logger.info("Update "+ ud.url)
# update sources there
@@ -114,20 +115,23 @@ class Osc(FetchMethod):
Generate a .oscrc to be used for this run.
"""
- config_path = os.path.join(d.getVar('OSCDIR'), "oscrc")
+ config_path = os.path.join(ud.oscdir, "oscrc")
+ if not os.path.exists(ud.oscdir):
+ bb.utils.mkdirhier(ud.oscdir)
+
if (os.path.exists(config_path)):
os.remove(config_path)
f = open(config_path, 'w')
+ proto = ud.parm.get('proto', 'https')
f.write("[general]\n")
- f.write("apisrv = %s\n" % ud.host)
- f.write("scheme = http\n")
+ f.write("apiurl = %s://%s\n" % (proto, ud.host))
f.write("su-wrapper = su -c\n")
f.write("build-root = %s\n" % d.getVar('WORKDIR'))
f.write("urllist = %s\n" % d.getVar("OSCURLLIST"))
f.write("extra-pkgs = gzip\n")
f.write("\n")
- f.write("[%s]\n" % ud.host)
+ f.write("[%s://%s]\n" % (proto, ud.host))
f.write("user = %s\n" % ud.parm["user"])
f.write("pass = %s\n" % ud.parm["pswd"])
f.close()
--
2.25.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* [bitbake][kirkstone][2.0][PATCH 2/3] data: Do not depend on vardepvalueexclude flag
2022-05-23 14:40 [bitbake][kirkstone][2.0][PATCH 0/3] Patch review Steve Sakoman
2022-05-23 14:40 ` [bitbake][kirkstone][2.0][PATCH 1/3] fetch2/osc: Small fixes for osc fetcher Steve Sakoman
@ 2022-05-23 14:40 ` Steve Sakoman
2022-05-23 14:40 ` [bitbake][kirkstone][2.0][PATCH 3/3] build: Add clean_stamp API function to allow removal of task stamps Steve Sakoman
2 siblings, 0 replies; 5+ messages in thread
From: Steve Sakoman @ 2022-05-23 14:40 UTC (permalink / raw)
To: bitbake-devel
From: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
If SRC_URI contains python function that extends vardepvalueexclude its
value is being tracked by sstate-cache, which can lead to rebuilds if
value is set dynamically (for example gerrit replicas).
Return empty string if vardepvalueexclude is checked to fix this
behaviour.
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f5f9a7b89a7d8321f03184e61ad6d5ed8d0f840e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
lib/bb/data.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/lib/bb/data.py b/lib/bb/data.py
index 62ec2147..c09d9b04 100644
--- a/lib/bb/data.py
+++ b/lib/bb/data.py
@@ -277,6 +277,8 @@ def build_dependencies(key, keys, shelldeps, varflagsexcl, ignored_vars, d):
try:
if key[-1] == ']':
vf = key[:-1].split('[')
+ if vf[1] == "vardepvalueexclude":
+ return deps, ""
value, parser = d.getVarFlag(vf[0], vf[1], False, retparser=True)
deps |= parser.references
deps = deps | (keys & parser.execs)
--
2.25.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* [bitbake][kirkstone][2.0][PATCH 3/3] build: Add clean_stamp API function to allow removal of task stamps
2022-05-23 14:40 [bitbake][kirkstone][2.0][PATCH 0/3] Patch review Steve Sakoman
2022-05-23 14:40 ` [bitbake][kirkstone][2.0][PATCH 1/3] fetch2/osc: Small fixes for osc fetcher Steve Sakoman
2022-05-23 14:40 ` [bitbake][kirkstone][2.0][PATCH 2/3] data: Do not depend on vardepvalueexclude flag Steve Sakoman
@ 2022-05-23 14:40 ` Steve Sakoman
2 siblings, 0 replies; 5+ messages in thread
From: Steve Sakoman @ 2022-05-23 14:40 UTC (permalink / raw)
To: bitbake-devel
From: Richard Purdie <richard.purdie@linuxfoundation.org>
We currently have no API to be able to remove all the potential stamps of a
task. It is unusual to need to do this, particularly as you could race against
other things happening in the system but we do have a use case for this in
cleaning up sysroots in OE-Core. The alternative is to mess with CLEANMASK in
OE-Core but that is just going to add potential for errors.
We need the first part of the make_stamp() function so separate that out so
it can be called seperately.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4d671504a25863018ac51c21c005cef0a4d8f05c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
lib/bb/build.py | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/lib/bb/build.py b/lib/bb/build.py
index af60c3d8..55f68b98 100644
--- a/lib/bb/build.py
+++ b/lib/bb/build.py
@@ -835,11 +835,7 @@ def stamp_cleanmask_internal(taskname, d, file_name):
return [cleanmask, cleanmask.replace(taskflagname, taskflagname + "_setscene")]
-def make_stamp(task, d, file_name = None):
- """
- Creates/updates a stamp for a given task
- (d can be a data dict or dataCache)
- """
+def clean_stamp(task, d, file_name = None):
cleanmask = stamp_cleanmask_internal(task, d, file_name)
for mask in cleanmask:
for name in glob.glob(mask):
@@ -850,6 +846,14 @@ def make_stamp(task, d, file_name = None):
if name.endswith('.taint'):
continue
os.unlink(name)
+ return
+
+def make_stamp(task, d, file_name = None):
+ """
+ Creates/updates a stamp for a given task
+ (d can be a data dict or dataCache)
+ """
+ clean_stamp(task, d, file_name)
stamp = stamp_internal(task, d, file_name)
# Remove the file and recreate to force timestamp
--
2.25.1
^ permalink raw reply related [flat|nested] 5+ messages in thread