[PATCH yocto-autobuilder2 0/3] runconfig: Add build configuration as properties

[PATCH yocto-autobuilder2 0/3] runconfig: Add build configuration as properties

[Mathieu Dubois-Briand]

It was recently brought to my attention that the "Build properties" tab
on buildbot could contain more data. One is the build configuration,
such as MACHINE or DISTRO, the other is about the poky-ci-archive tag
that is automatically created on a-full and a-quick builds.

This series aims to improve a bit the situation, by extracting DISTRO,
MACHINE, SDKMACHINE and PACKAGE_CLASSES from the config.json file
describing builds configuration, and adding them as build properties.
More properties can be added if any appears to be useful.

It also stores poky-ci-archive tag as well as a web git URL in
"poky-ci-archive_tag" and "poky-ci-archive_url" properties, on a-full
and a-quick builds, copied to all sub-builds. I'm not convinced by the
need to have both properties, so I'm open for any feedback.

This series has a dependency on run-config modifications, in
yocto-autobuilder-helper git [1].

Also add b4 configuration file, to ease a bit the patch submission
process.

[1] https://lists.yoctoproject.org/g/yocto-patches/message/2129

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
---
Mathieu Dubois-Briand (3):
      runconfig: Add build configuration as properties
      builders: Add poky-ci-archive tag in build properties
      b4-config: Add basic b4 configuration

 .b4-config         |  4 ++++
 builders.py        | 10 +++++++++-
 steps/runconfig.py | 12 +++++++++++-
 3 files changed, 24 insertions(+), 2 deletions(-)
---
base-commit: 45b64625b3d58449fc6e18bf2cbe473b63745656
change-id: 20250904-contrib-mathieu-build_props-a30fbeef06d3

Best regards,
-- 
Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>

[PATCH yocto-autobuilder2 1/3] runconfig: Add build configuration as properties

[Mathieu Dubois-Briand]

Add some important build configuration such as MACHINE or DISTRO to
buildbot build properties.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
---
 steps/runconfig.py | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/steps/runconfig.py b/steps/runconfig.py
index a6d0dbd42efe..4e1e704f9de8 100644
--- a/steps/runconfig.py
+++ b/steps/runconfig.py
@@ -160,17 +160,22 @@ class RunConfigCheckSteps(shell.ShellCommand):
     haltOnFailure = False
     flunkOnFailure = True
     jsonFileName = util.Interpolate("%(prop:builddir)s/runconfig.json")
-    logfiles = {'json': jsonFileName}
+    propertiesFileName = util.Interpolate("%(prop:builddir)s/runconfig-properties.json")
+    logfiles = {'json': jsonFileName, 'properties': propertiesFileName}
 
     def __init__(self, *args, **kwargs):
         self.posttrigger = kwargs.pop("posttrigger")
         self.command = get_runconfig_command(self.posttrigger)
         self.command.append("--json-outputfile")
         self.command.append(self.jsonFileName)
+        self.command.append("--properties-outputfile")
+        self.command.append(self.propertiesFileName)
         super().__init__(*args, **kwargs)
 
         self.log_observer_json = logobserver.BufferLogObserver()
         self.addLogObserver('json', self.log_observer_json)
+        self.log_observer_properties = logobserver.BufferLogObserver()
+        self.addLogObserver('properties', self.log_observer_properties)
 
     @defer.inlineCallbacks
     def run(self):
@@ -188,12 +193,17 @@ class RunConfigCheckSteps(shell.ShellCommand):
         # If the command fails, fall back to old style run-config execution
         rc = cmd.results()
         logLines = self.log_observer_json.getStdout()
+        propertiesLines = self.log_observer_properties.getStdout()
         jsonconfig = None
         try:
             jsonconfig = json.loads(logLines)
+            properties = json.loads(propertiesLines)
         except Exception as ex:
             self._addToLog('stderr', 'ERROR: unable to parse data, exception {}: {}'.format(ex.__class__, ex))
 
+        for k, v in properties.items():
+            self.setProperty(k, v, "run-config output")
+
         if rc == FAILURE or not jsonconfig:
             steps = [get_runconfig_legacy_step(self.posttrigger)]
         else:

-- 
2.39.5

[PATCH yocto-autobuilder2 2/3] builders: Add poky-ci-archive tag in build properties

[Mathieu Dubois-Briand]

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
---
 builders.py | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/builders.py b/builders.py
index e4763d5325aa..0fa6e0035385 100644
--- a/builders.py
+++ b/builders.py
@@ -307,6 +307,12 @@ def create_parent_builder_factory(buildername, waitname):
         ],
         haltOnFailure=True,
         name="Prepare shared repositories"))
+    factory.addStep(steps.SetProperty(
+        property="poky-ci-archive_tag", value=createBuildTag))
+    tagurl = util.Interpolate('https://git.yoctoproject.org/poky-ci-archive/log/?h=%(kw:tag)s',
+                              tag=createBuildTag)
+    factory.addStep(steps.SetProperty(
+        property="poky-ci-archive_url", value=tagurl))
     factory.addStep(steps.SetProperty(
         property="sharedrepolocation",
         value=util.Interpolate("{}/%(prop:buildername)s-%(prop:buildnumber)s".format(config.sharedrepodir))
@@ -358,7 +364,9 @@ def create_parent_builder_factory(buildername, waitname):
             "milestone_number": util.Property("milestone_number"),
             "rc_number": util.Property("rc_number"),
             "yp_build_revision": util.Property("yp_build_revision"),
-            "yp_build_branch": util.Property("yp_build_branch")
+            "yp_build_branch": util.Property("yp_build_branch"),
+            "poky-ci-archive_tag": util.Property("poky-ci-archive_tag"),
+            "poky-ci-archive_url": util.Property("poky-ci-archive_url")
         }
 
         for repo in config.buildertorepos[buildername]:

-- 
2.39.5

[PATCH yocto-autobuilder2 3/3] b4-config: Add basic b4 configuration

[Mathieu Dubois-Briand]

Allow to easily prepare mail series with b4. Most of the configuration
was shamelessly stolen from openembedded-core git.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
---
 .b4-config | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/.b4-config b/.b4-config
new file mode 100644
index 000000000000..0f1f430c74f3
--- /dev/null
+++ b/.b4-config
@@ -0,0 +1,4 @@
+[b4]
+  send-series-to = yocto-patches@lists.yoctoproject.org
+  prep-pre-flight-checks = disable-needs-auto-to-cc, disable-needs-checking
+  send-prefixes = yocto-autobuilder2

-- 
2.39.5

Re: [yocto-patches] [PATCH yocto-autobuilder2 0/3] runconfig: Add build configuration as properties

[Richard Purdie]

On Thu, 2025-09-04 at 17:48 +0200, Mathieu Dubois-Briand via lists.yoctoproject.org wrote:
> It was recently brought to my attention that the "Build properties" tab
> on buildbot could contain more data. One is the build configuration,
> such as MACHINE or DISTRO, the other is about the poky-ci-archive tag
> that is automatically created on a-full and a-quick builds.
> 
> This series aims to improve a bit the situation, by extracting DISTRO,
> MACHINE, SDKMACHINE and PACKAGE_CLASSES from the config.json file
> describing builds configuration, and adding them as build properties.
> More properties can be added if any appears to be useful.
> 
> It also stores poky-ci-archive tag as well as a web git URL in
> "poky-ci-archive_tag" and "poky-ci-archive_url" properties, on a-full
> and a-quick builds, copied to all sub-builds. I'm not convinced by the
> need to have both properties, so I'm open for any feedback.
> 
> This series has a dependency on run-config modifications, in
> yocto-autobuilder-helper git [1].

The challenge is we have to patch all helper branches we're still using
when we make a change like that, which makes this kind of change a lot
harder to merge.

Cheers,

Richard

Re: [yocto-patches] [PATCH yocto-autobuilder2 0/3] runconfig: Add build configuration as properties

[Mathieu Dubois-Briand]

On Mon Sep 8, 2025 at 4:24 PM CEST, Richard Purdie wrote:
> On Thu, 2025-09-04 at 17:48 +0200, Mathieu Dubois-Briand via lists.yoctoproject.org wrote:
>> It was recently brought to my attention that the "Build properties" tab
>> on buildbot could contain more data. One is the build configuration,
>> such as MACHINE or DISTRO, the other is about the poky-ci-archive tag
>> that is automatically created on a-full and a-quick builds.
>> 
>> This series aims to improve a bit the situation, by extracting DISTRO,
>> MACHINE, SDKMACHINE and PACKAGE_CLASSES from the config.json file
>> describing builds configuration, and adding them as build properties.
>> More properties can be added if any appears to be useful.
>> 
>> It also stores poky-ci-archive tag as well as a web git URL in
>> "poky-ci-archive_tag" and "poky-ci-archive_url" properties, on a-full
>> and a-quick builds, copied to all sub-builds. I'm not convinced by the
>> need to have both properties, so I'm open for any feedback.
>> 
>> This series has a dependency on run-config modifications, in
>> yocto-autobuilder-helper git [1].
>
> The challenge is we have to patch all helper branches we're still using
> when we make a change like that, which makes this kind of change a lot
> harder to merge.
>
> Cheers,
>
> Richard

Oops, I had the dependency order wrong for a second. So I confirm I had
no intention to mandate an update of all helper branches, but only
master.

I will try to come with a better idea on the yocto-autobuilder2 side, so
it can work both with old and new helper code.

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com