[bitbake][langdale][2.2][PATCH 0/4] Patch review

[bitbake][langdale][2.2][PATCH 0/4] Patch review

[Steve Sakoman]

Please review this set of changes for 2.2/langdale and have comments back
by end of day Friday.

Passed a-full on autobuilder:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/4447

With the exception of a known autobuilder intermittent issue on qemumips64:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=14931

which passed on subsequent retest:

https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/6080

The following changes since commit 209f7ba352b60722830157054e3fc56cb9c693eb:

  tests/fetch: Allow handling of a file:// url within a submodule (2022-10-26 23:01:35 +0100)

are available in the Git repository at:

  https://git.openembedded.org/bitbake-contrib stable/2.2-nut
  http://cgit.openembedded.org/bitbake-contrib/log/?h=stable/2.2-nut

Justin Bronder (1):
  asyncrpc: serv: correct closed client socket detection

Mark Asselstine (1):
  bitbake: bitbake-layers: checkout layer(s) branch when clone exists

Michael Opdenacker (1):
  bitbake-user-manual: details about variable flags starting with
    underscore

Ross Burton (1):
  fetch2/git: don't set core.fsyncobjectfiles=0

 .../bitbake-user-manual-metadata.rst          |  6 +++++
 lib/bb/asyncrpc/serv.py                       |  4 +--
 lib/bb/fetch2/git.py                          |  2 +-
 lib/bblayers/layerindex.py                    | 25 +++++++++++++++++++
 4 files changed, 34 insertions(+), 3 deletions(-)

-- 
2.25.1

[bitbake][langdale][2.2][PATCH 1/4] bitbake-user-manual: details about variable flags starting with underscore

[Steve Sakoman]

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

Fixes [YOCTO #14140]

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0f3e9d87168813ce49995ff04bccdce11c5f7b47)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 doc/bitbake-user-manual/bitbake-user-manual-metadata.rst | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
index b533d9dc..b7c3d809 100644
--- a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
+++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
@@ -422,6 +422,12 @@ documentation to a BitBake variable as follows::
 
    CACHE[doc] = "The directory holding the cache of the metadata."
 
+.. note::
+
+   Variable flag names starting with an underscore (``_``) character
+   are allowed but are ignored by ``d.getVarFlags("VAR")``
+   in Python code. Such flag names are used internally by BitBake.
+
 Inline Python Variable Expansion
 --------------------------------
 
-- 
2.25.1

[bitbake][langdale][2.2][PATCH 2/4] fetch2/git: don't set core.fsyncobjectfiles=0

[Steve Sakoman]

From: Ross Burton <ross.burton@arm.com>

This git configuration variable is deprecated in 2.36.0 onwards, so git
warns in the logs for every git call.

Luckily the default value has always been false[1], so we can just remove
this.

[ YOCTO #14939 ]

[1] https://github.com/git/git/commit/aafe9fbaf4f1d1f27a6f6e3eb3e246fff81240ef

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8ad310633e0c5d5593631c1196cbdde30147efce)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/fetch2/git.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
index 17d49049..578edc59 100644
--- a/lib/bb/fetch2/git.py
+++ b/lib/bb/fetch2/git.py
@@ -243,7 +243,7 @@ class Git(FetchMethod):
             for name in ud.names:
                 ud.unresolvedrev[name] = 'HEAD'
 
-        ud.basecmd = d.getVar("FETCHCMD_git") or "git -c core.fsyncobjectfiles=0 -c gc.autoDetach=false -c core.pager=cat"
+        ud.basecmd = d.getVar("FETCHCMD_git") or "git -c gc.autoDetach=false -c core.pager=cat"
 
         write_tarballs = d.getVar("BB_GENERATE_MIRROR_TARBALLS") or "0"
         ud.write_tarballs = write_tarballs != "0" or ud.rebaseable
-- 
2.25.1

[bitbake][langdale][2.2][PATCH 3/4] asyncrpc: serv: correct closed client socket detection

[Steve Sakoman]

From: Justin Bronder <jsbronder@cold-front.org>

If the client socket is closed, asyncio.StreamReader.readline() will
return an empty bytes object, not None.

This prevents multiple tracebacks being logged by bitbake-hashserv each
time bitbake is started and performs a connection check.

Signed-off-by: Justin Bronder <jsbronder@cold-front.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2d07f252704dff7747fa1f9adf223a452806717f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/asyncrpc/serv.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/bb/asyncrpc/serv.py b/lib/bb/asyncrpc/serv.py
index 5cf45f90..d2de4891 100644
--- a/lib/bb/asyncrpc/serv.py
+++ b/lib/bb/asyncrpc/serv.py
@@ -42,7 +42,7 @@ class AsyncServerConnection(object):
 
             # Read protocol and version
             client_protocol = await self.reader.readline()
-            if client_protocol is None:
+            if not client_protocol:
                 return
 
             (client_proto_name, client_proto_version) = client_protocol.decode('utf-8').rstrip().split()
@@ -59,7 +59,7 @@ class AsyncServerConnection(object):
             # an empty line to signal the end of the headers
             while True:
                 line = await self.reader.readline()
-                if line is None:
+                if not line:
                     return
 
                 line = line.decode('utf-8').rstrip()
-- 
2.25.1

[bitbake][langdale][2.2][PATCH 4/4] bitbake: bitbake-layers: checkout layer(s) branch when clone exists

[Steve Sakoman]

From: Mark Asselstine <mark.asselstine@windriver.com>

[YOCTO #7852]

Fixes 'bitbake-layers layerindex-fetch --branch kirkstone meta-arm'
not checking out the branch if the repo is already cloned and on a
different branch.

If a clone of a layer being added already exists check what branch it
is on and if necessary attempt to switch to the given branch. If the
switch fails to happen the git error will be reported. We also warn if
there are uncommitted changes as the changes might go unnoticed and
result in unexpected behaviors.

Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d2cb388f58a37db2149fad34e4572d954e6e5441)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bblayers/layerindex.py | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/lib/bblayers/layerindex.py b/lib/bblayers/layerindex.py
index 0ac8fd2e..ba91fac6 100644
--- a/lib/bblayers/layerindex.py
+++ b/lib/bblayers/layerindex.py
@@ -49,6 +49,31 @@ class LayerIndexPlugin(ActionPlugin):
             else:
                 logger.plain("Repository %s needs to be fetched" % url)
                 return subdir, layername, layerdir
+        elif os.path.exists(repodir) and branch:
+            """
+            If the repo is already cloned, ensure it is on the correct branch,
+            switching branches if necessary and possible.
+            """
+            base_cmd = ['git', '--git-dir=%s/.git' % repodir, '--work-tree=%s' % repodir]
+            cmd = base_cmd + ['branch']
+            completed_proc = subprocess.run(cmd, text=True, capture_output=True)
+            if completed_proc.returncode:
+                logger.error("Unable to validate repo %s (%s)" % (repodir, stderr))
+                return None, None, None
+            else:
+                if branch != completed_proc.stdout[2:-1]:
+                    cmd = base_cmd + ['status', '--short']
+                    completed_proc = subprocess.run(cmd, text=True, capture_output=True)
+                    if completed_proc.stdout.count('\n') != 0:
+                        logger.warning("There are uncommitted changes in repo %s" % repodir)
+                    cmd = base_cmd + ['checkout', branch]
+                    completed_proc = subprocess.run(cmd, text=True, capture_output=True)
+                    if completed_proc.returncode:
+                        # Could be due to original shallow clone on a different branch for example
+                        logger.error("Unable to automatically switch %s to desired branch '%s' (%s)"
+                                     % (repodir, branch, completed_proc.stderr))
+                        return None, None, None
+            return subdir, layername, layerdir
         elif os.path.exists(layerdir):
             return subdir, layername, layerdir
         else:
-- 
2.25.1