[bitbake][langdale][2.2][PATCH 0/2] Patch review

[bitbake][langdale][2.2][PATCH 0/2] Patch review

[Steve Sakoman]

We're seeing some race issues on the autobuilder where langdale builds are
causing master builds to fail. This is likely due to langdale
missing these patches.

The following changes since commit f0f166aee766b4bb1f8cf8b35dfc7d406c75e6a4:

  gitsm: Fix regression in gitsm submodule path parsing (2023-01-06 17:27:39 +0000)

are available in the Git repository at:

  https://git.openembedded.org/bitbake-contrib stable/2.2-nut
  http://cgit.openembedded.org/bitbake-contrib/log/?h=stable/2.2-nut

Frank de Brabander (1):
  process: log odd unlink events with bitbake.sock

Richard Purdie (1):
  server/process: Add bitbake.sock race handling

 lib/bb/server/process.py | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

-- 
2.25.1

[bitbake][langdale][2.2][PATCH 1/2] process: log odd unlink events with bitbake.sock

[Steve Sakoman]

From: Frank de Brabander <debrabander@gmail.com>

Log when the socket file already exists and is removed before
recreating a new socket.

Log when unlinking the socket file failed.

Signed-off-by: Frank de Brabander <debrabander@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit cfd7c9899f988bab6d9fe7bbfbdb60603fb5ed34)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/server/process.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py
index 5d02c0b9..2a9fb662 100644
--- a/lib/bb/server/process.py
+++ b/lib/bb/server/process.py
@@ -248,8 +248,8 @@ class ProcessServer():
         # Remove the socket file so we don't get any more connections to avoid races
         try:
             os.unlink(self.sockname)
-        except:
-            pass
+        except Exception as err:
+            serverlog("Removing socket file '%s' failed (%s)" % (self.sockname, err))
         self.sock.close()
 
         try:
@@ -532,6 +532,7 @@ def execServer(lockfd, readypipeinfd, lockname, sockname, server_timeout, xmlrpc
 
         # Create server control socket
         if os.path.exists(sockname):
+            serverlog("WARNING: removing existing socket file '%s'" % sockname)
             os.unlink(sockname)
 
         sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
-- 
2.25.1

[bitbake][langdale][2.2][PATCH 2/2] server/process: Add bitbake.sock race handling

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

We've seen cases where the bitbake.sock file appears to disappear but the
server continues to hold bitbake.lock. The most likely explaination is
that some previous build directory was moved out the way, a server there
kept running, eventually exited and removed the sock file from the wrong
directory.

To guard against this, save the inode information for the sock file and check
it before deleting the file. The new code isn't entirely race free but should
guard against what is a rare but annoying potential issue.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b02ebbffdae27e564450446bf84c4e98d094ee4a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/server/process.py | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py
index 2a9fb662..3668a32b 100644
--- a/lib/bb/server/process.py
+++ b/lib/bb/server/process.py
@@ -28,6 +28,7 @@ import datetime
 import pickle
 import traceback
 import gc
+import stat
 import bb.server.xmlrpcserver
 from bb import daemonize
 from multiprocessing import queues
@@ -64,6 +65,9 @@ class ProcessServer():
         self.bitbake_lock_name = lockname
         self.sock = sock
         self.sockname = sockname
+        # It is possible the directory may be renamed. Cache the inode of the socket file
+        # so we can tell if things changed.
+        self.sockinode = os.stat(self.sockname)[stat.ST_INO]
 
         self.server_timeout = server_timeout
         self.timeout = self.server_timeout
@@ -246,8 +250,14 @@ class ProcessServer():
 
         serverlog("Exiting")
         # Remove the socket file so we don't get any more connections to avoid races
+        # The build directory could have been renamed so if the file isn't the one we created
+        # we shouldn't delete it.
         try:
-            os.unlink(self.sockname)
+            sockinode = os.stat(self.sockname)[stat.ST_INO]
+            if sockinode == self.sockinode:
+                os.unlink(self.sockname)
+            else:
+                serverlog("bitbake.sock inode mismatch (%s vs %s), not deleting." % (sockinode, self.sockinode))
         except Exception as err:
             serverlog("Removing socket file '%s' failed (%s)" % (self.sockname, err))
         self.sock.close()
-- 
2.25.1

[bitbake][langdale][2.2][PATCH 0/2] Patch review

[Steve Sakoman]

Please review this set of patches for 2.2/langdale and have comments back by
end of day Friday.

Passed a-full on autobuilder:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/4871

The following changes since commit 28ba566c25af72afe62ea3cb62b0bafeffc47de8:

  bb/utils: include SSL certificate paths in export_proxies (2023-01-30 08:37:04 +0000)

are available in the Git repository at:

  https://git.openembedded.org/bitbake-contrib stable/2.2-nut
  http://cgit.openembedded.org/bitbake-contrib/log/?h=stable/2.2-nut

Marek Vasut (2):
  fetch2/git: Prevent git fetcher from fetching gitlab repository
    metadata
  fetch2/git: Clarify the meaning of namespace

 doc/bitbake-user-manual/bitbake-user-manual-fetching.rst | 4 ++--
 lib/bb/fetch2/git.py                                     | 9 +++++++--
 2 files changed, 9 insertions(+), 4 deletions(-)

-- 
2.25.1