[bitbake][kirkstone][2.0][PATCH 0/3] Patch review

[bitbake][kirkstone][2.0][PATCH 0/3] Patch review

[Steve Sakoman]

Please review this set of patches for kirkstone/2.0 and have comments back
by end of day Wednesday.

Passed a-full on autobuilder:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/3692

The following changes since commit 59c16ae6c55c607c56efd2287537a1b97ba2bf52:

  fetch/git : Use cat as pager (2022-05-12 13:41:35 +0100)

are available in the Git repository at:

  git://git.openembedded.org/bitbake-contrib stable/2.0-nut
  http://cgit.openembedded.org/bitbake-contrib/log/?h=stable/2.0-nut

Gunjan Gupta (1):
  fetch2/osc: Small fixes for osc fetcher

Richard Purdie (1):
  build: Add clean_stamp API function to allow removal of task stamps

Tomasz Dziendzielski (1):
  data: Do not depend on vardepvalueexclude flag

 lib/bb/build.py      | 14 +++++++++-----
 lib/bb/data.py       |  2 ++
 lib/bb/fetch2/osc.py | 16 ++++++++++------
 3 files changed, 21 insertions(+), 11 deletions(-)

-- 
2.25.1

[bitbake][kirkstone][2.0][PATCH 1/3] fetch2/osc: Small fixes for osc fetcher

[Steve Sakoman]

From: Gunjan Gupta <viraniac@gmail.com>

The current fetcher seemed to have some issues that made it difficult when
trying to use the same. This patch fixes the following

* Make consistent use of the path that needs to be used as oscdir
* The path mentioned in os.access in download function was not same as
  ud.moddir which would result into invoking of fetch command instead of
  update command even if directory already existed
* Before creating oscrc, make sure oscdir exists and create it if it does
  not exist
* Updated the configuration to use apiurl and added a new parameter to
  control whether http or https needs to be used to connect to apiurl

Signed-off-by: Gunjan Gupta <viraniac@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3ec78686f3c0ea2304097b86a965f9be4b0cb879)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/fetch2/osc.py | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/lib/bb/fetch2/osc.py b/lib/bb/fetch2/osc.py
index 99a529e5..eb0f82c8 100644
--- a/lib/bb/fetch2/osc.py
+++ b/lib/bb/fetch2/osc.py
@@ -36,6 +36,7 @@ class Osc(FetchMethod):
         # Create paths to osc checkouts
         oscdir = d.getVar("OSCDIR") or (d.getVar("DL_DIR") + "/osc")
         relpath = self._strip_leading_slashes(ud.path)
+        ud.oscdir = oscdir
         ud.pkgdir = os.path.join(oscdir, ud.host)
         ud.moddir = os.path.join(ud.pkgdir, relpath, ud.module)
 
@@ -49,7 +50,7 @@ class Osc(FetchMethod):
             else:
                 ud.revision = ""
 
-        ud.localfile = d.expand('%s_%s_%s.tar.gz' % (ud.module.replace('/', '.'), ud.path.replace('/', '.'), ud.revision))
+        ud.localfile = d.expand('%s_%s_%s.tar.gz' % (ud.module.replace('/', '.'), relpath.replace('/', '.'), ud.revision))
 
     def _buildosccommand(self, ud, d, command):
         """
@@ -86,7 +87,7 @@ class Osc(FetchMethod):
 
         logger.debug2("Fetch: checking for module directory '" + ud.moddir + "'")
 
-        if os.access(os.path.join(d.getVar('OSCDIR'), ud.path, ud.module), os.R_OK):
+        if os.access(ud.moddir, os.R_OK):
             oscupdatecmd = self._buildosccommand(ud, d, "update")
             logger.info("Update "+ ud.url)
             # update sources there
@@ -114,20 +115,23 @@ class Osc(FetchMethod):
         Generate a .oscrc to be used for this run.
         """
 
-        config_path = os.path.join(d.getVar('OSCDIR'), "oscrc")
+        config_path = os.path.join(ud.oscdir, "oscrc")
+        if not os.path.exists(ud.oscdir):
+            bb.utils.mkdirhier(ud.oscdir)
+
         if (os.path.exists(config_path)):
             os.remove(config_path)
 
         f = open(config_path, 'w')
+        proto = ud.parm.get('proto', 'https')
         f.write("[general]\n")
-        f.write("apisrv = %s\n" % ud.host)
-        f.write("scheme = http\n")
+        f.write("apiurl = %s://%s\n" % (proto, ud.host))
         f.write("su-wrapper = su -c\n")
         f.write("build-root = %s\n" % d.getVar('WORKDIR'))
         f.write("urllist = %s\n" % d.getVar("OSCURLLIST"))
         f.write("extra-pkgs = gzip\n")
         f.write("\n")
-        f.write("[%s]\n" % ud.host)
+        f.write("[%s://%s]\n" % (proto, ud.host))
         f.write("user = %s\n" % ud.parm["user"])
         f.write("pass = %s\n" % ud.parm["pswd"])
         f.close()
-- 
2.25.1

[bitbake][kirkstone][2.0][PATCH 2/3] data: Do not depend on vardepvalueexclude flag

[Steve Sakoman]

From: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>

If SRC_URI contains python function that extends vardepvalueexclude its
value is being tracked by sstate-cache, which can lead to rebuilds if
value is set dynamically (for example gerrit replicas).

Return empty string if vardepvalueexclude is checked to fix this
behaviour.

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f5f9a7b89a7d8321f03184e61ad6d5ed8d0f840e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/data.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/lib/bb/data.py b/lib/bb/data.py
index 62ec2147..c09d9b04 100644
--- a/lib/bb/data.py
+++ b/lib/bb/data.py
@@ -277,6 +277,8 @@ def build_dependencies(key, keys, shelldeps, varflagsexcl, ignored_vars, d):
     try:
         if key[-1] == ']':
             vf = key[:-1].split('[')
+            if vf[1] == "vardepvalueexclude":
+                return deps, ""
             value, parser = d.getVarFlag(vf[0], vf[1], False, retparser=True)
             deps |= parser.references
             deps = deps | (keys & parser.execs)
-- 
2.25.1

[bitbake][kirkstone][2.0][PATCH 3/3] build: Add clean_stamp API function to allow removal of task stamps

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

We currently have no API to be able to remove all the potential stamps of a
task. It is unusual to need to do this, particularly as you could race against
other things happening in the system but we do have a use case for this in
cleaning up sysroots in OE-Core. The alternative is to mess with CLEANMASK in
OE-Core but that is just going to add potential for errors.

We need the first part of the make_stamp() function so separate that out so
it can be called seperately.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4d671504a25863018ac51c21c005cef0a4d8f05c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/build.py | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/lib/bb/build.py b/lib/bb/build.py
index af60c3d8..55f68b98 100644
--- a/lib/bb/build.py
+++ b/lib/bb/build.py
@@ -835,11 +835,7 @@ def stamp_cleanmask_internal(taskname, d, file_name):
 
     return [cleanmask, cleanmask.replace(taskflagname, taskflagname + "_setscene")]
 
-def make_stamp(task, d, file_name = None):
-    """
-    Creates/updates a stamp for a given task
-    (d can be a data dict or dataCache)
-    """
+def clean_stamp(task, d, file_name = None):
     cleanmask = stamp_cleanmask_internal(task, d, file_name)
     for mask in cleanmask:
         for name in glob.glob(mask):
@@ -850,6 +846,14 @@ def make_stamp(task, d, file_name = None):
             if name.endswith('.taint'):
                 continue
             os.unlink(name)
+    return
+
+def make_stamp(task, d, file_name = None):
+    """
+    Creates/updates a stamp for a given task
+    (d can be a data dict or dataCache)
+    """
+    clean_stamp(task, d, file_name)
 
     stamp = stamp_internal(task, d, file_name)
     # Remove the file and recreate to force timestamp
-- 
2.25.1

[bitbake][kirkstone][2.0][PATCH 0/3] Patch review

[Steve Sakoman]

Please review this set of changes for 2.0/kirkstone and have comments back
by end of day Friday, November 10

Passed a-full on autobuilder:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6158

The following changes since commit 6c1ffa9091d0c53a100e8c8c15122d28642034bd:

  SECURITY.md: add file (2023-10-24 12:50:43 +0100)

are available in the Git repository at:

  https://git.openembedded.org/bitbake-contrib stable/2.0-nut
  https://git.openembedded.org/bitbake-contrib/log/?h=stable/2.0-nut

Denys Dmytriyenko (1):
  runqueue: convert deferral messages from bb.note to bb.debug

Peter Kjellerstedt (2):
  bitbake-getvar: Make --quiet work with --recipe
  tinfoil: Do not fail when logging is disabled and full config is used

 bin/bitbake-getvar | 5 +++--
 lib/bb/runqueue.py | 6 +++---
 lib/bb/tinfoil.py  | 2 +-
 3 files changed, 7 insertions(+), 6 deletions(-)

-- 
2.34.1