[PATCH v3 0/2] classes: Print bitbake branch and revision

[PATCH v3 0/2] classes: Print bitbake branch and revision

[liezhi.yang]

From: Robert Yang <liezhi.yang@windriver.com>

* V3:
  - Add is_bitbake_in_separate_repo() as Alexander suggested

* V2:
  - Move the code out of get_layer_revisions() as Alexander suggested 
  - Always print bitbake branch and revision to make the output consistent
    whenever bitbake is in a separate repo or not.

* V1:
  - Initial version

// Robert

The following changes since commit 918e2b266eba6779f19f65349f85caa880ba45e7:

  libedit: Make docs generation deterministic (2024-09-11 11:56:34 +0100)

are available in the Git repository at:

  https://github.com/robertlinux/yocto rbt/bbrev
  https://github.com/robertlinux/yocto/tree/rbt/bbrev

Robert Yang (2):
  classes: Print bitbake branch and revision
  lib/buildcfg: Add is_bitbake_in_separate_repo()

 meta/classes-global/base.bbclass     | 10 +++++++---
 meta/classes/buildhistory.bbclass    |  5 +++--
 meta/classes/image-buildinfo.bbclass | 13 +++++++++++--
 meta/lib/oe/buildcfg.py              | 23 +++++++++++++++++++++++
 4 files changed, 44 insertions(+), 7 deletions(-)

-- 
2.44.1

[PATCH v3 1/2] classes: Print bitbake branch and revision

[liezhi.yang]

From: Robert Yang <liezhi.yang@windriver.com>

There is no bitbake revision when it is in a separate repo, then we have no way
to know which version of bitbake is used from the log, this patch always prints
it.

* console-latest.log
bitbake              = "master:165368bad152a14b32b5216b938aa4b915d72a70"
meta
meta-poky
meta-yocto-bsp       = "master:165368bad152a14b32b5216b938aa4b915d72a70"

* buildhistory/metadata-revs
bitbake           = master:165368bad152a14b32b5216b938aa4b915d72a70 -- modified
[snip]

* buildhistory/metadata-revs
-----------------------
Build Configuration:  |
-----------------------
DISTRO = poky
DISTRO_VERSION = 5.0+snapshot-165368bad152a14b32b5216b938aa4b915d72a70
-----------------------
Bitbake Revision:      |
-----------------------
bitbake           = master:165368bad152a14b32b5216b938aa4b915d72a70 -- modified
-----------------------
Layer Revisions:      |
-----------------------
[snip]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes-global/base.bbclass     | 10 +++++++---
 meta/classes/buildhistory.bbclass    |  5 +++--
 meta/classes/image-buildinfo.bbclass | 13 +++++++++++--
 meta/lib/oe/buildcfg.py              | 13 +++++++++++++
 4 files changed, 34 insertions(+), 7 deletions(-)

diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass
index b6940bbb6f..85c7cb8c24 100644
--- a/meta/classes-global/base.bbclass
+++ b/meta/classes-global/base.bbclass
@@ -226,9 +226,13 @@ do_unpack[postfuncs] += "create_source_date_epoch_stamp"
 def get_source_date_epoch_value(d):
     return oe.reproducible.epochfile_read(d.getVar('SDE_FILE'), d)
 
+def get_bitbake_branch_rev(d):
+    revisions = oe.buildcfg.get_bitbake_revision()
+    return oe.buildcfg.get_branch_rev3(revisions)
+
 def get_layers_branch_rev(d):
     revisions = oe.buildcfg.get_layer_revisions(d)
-    layers_branch_rev = ["%-20s = \"%s:%s\"" % (r[1], r[2], r[3]) for r in revisions]
+    layers_branch_rev = oe.buildcfg.get_branch_rev3(revisions)
     i = len(layers_branch_rev)-1
     p1 = layers_branch_rev[i].find("=")
     s1 = layers_branch_rev[i][p1:]
@@ -241,11 +245,11 @@ def get_layers_branch_rev(d):
         else:
             i -= 1
             p1 = layers_branch_rev[i].find("=")
-            s1= layers_branch_rev[i][p1:]
+            s1 = layers_branch_rev[i][p1:]
     return layers_branch_rev
 
 
-BUILDCFG_FUNCS ??= "buildcfg_vars get_layers_branch_rev buildcfg_neededvars"
+BUILDCFG_FUNCS ??= "buildcfg_vars get_bitbake_branch_rev get_layers_branch_rev buildcfg_neededvars"
 BUILDCFG_FUNCS[type] = "list"
 
 def buildcfg_vars(d):
diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index ce3abaa69d..50516e6533 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -764,8 +764,9 @@ def buildhistory_get_build_id(d):
 
 def buildhistory_get_metadata_revs(d):
     # We want an easily machine-readable format here
-    revisions = oe.buildcfg.get_layer_revisions(d)
-    medadata_revs = ["%-17s = %s:%s%s" % (r[1], r[2], r[3], r[4]) for r in revisions]
+    revisions = oe.buildcfg.get_bitbake_revision()
+    revisions.extend(oe.buildcfg.get_layer_revisions(d))
+    medadata_revs = oe.buildcfg.get_branch_rev4(revisions)
     return '\n'.join(medadata_revs)
 
 def outputvars(vars, listvars, d):
diff --git a/meta/classes/image-buildinfo.bbclass b/meta/classes/image-buildinfo.bbclass
index b83ce650ad..7b5f84854c 100644
--- a/meta/classes/image-buildinfo.bbclass
+++ b/meta/classes/image-buildinfo.bbclass
@@ -27,11 +27,14 @@ def image_buildinfo_outputvars(vars, d):
         ret += "%s = %s\n" % (var, value)
     return ret.rstrip('\n')
 
+def get_bitbake_rev(d):
+    revision = oe.buildcfg.get_bitbake_revision()
+    return '\n'.join(oe.buildcfg.get_branch_rev4(revision))
+
 # Returns layer revisions along with their respective status
 def get_layer_revs(d):
     revisions = oe.buildcfg.get_layer_revisions(d)
-    medadata_revs = ["%-17s = %s:%s%s" % (r[1], r[2], r[3], r[4]) for r in revisions]
-    return '\n'.join(medadata_revs)
+    return '\n'.join(oe.buildcfg.get_branch_rev4(revisions))
 
 def buildinfo_target(d):
         # Get context
@@ -55,6 +58,12 @@ Build Configuration:  |
             buildinfo_target(d),
             '''
 -----------------------
+Bitbake Revision:      |
+-----------------------
+''',
+            get_bitbake_rev(d),
+            '''
+-----------------------
 Layer Revisions:      |
 -----------------------
 ''',
diff --git a/meta/lib/oe/buildcfg.py b/meta/lib/oe/buildcfg.py
index 4b22f18f36..dab4aa7831 100644
--- a/meta/lib/oe/buildcfg.py
+++ b/meta/lib/oe/buildcfg.py
@@ -71,6 +71,19 @@ def is_layer_modified(path):
         # output and a 129 return code when a layer isn't a git repo at all.
         return " -- modified"
 
+def get_branch_rev3(revisions):
+    # Return 3 items for each revision
+    return ["%-20s = \"%s:%s\"" % (r[1], r[2], r[3]) for r in revisions]
+
+def get_branch_rev4(revisions):
+    # Return 4 items for each revision
+    return ["%-17s = %s:%s%s" % (r[1], r[2], r[3], r[4]) for r in revisions]
+
+def get_bitbake_revision():
+    bbdir = bb.__file__.rsplit('/', 3)[0]
+    return [(bbdir, os.path.basename(bbdir), get_metadata_git_branch(bbdir).strip(), \
+            get_metadata_git_revision(bbdir), is_layer_modified(bbdir))]
+
 def get_layer_revisions(d):
     layers = (d.getVar("BBLAYERS") or "").split()
     revisions = []
-- 
2.44.1

Re: [OE-core] [PATCH v3 1/2] classes: Print bitbake branch and revision

[Richard Purdie]

On Wed, 2024-09-11 at 07:35 -0700, Robert Yang via lists.openembedded.org wrote:
> From: Robert Yang <liezhi.yang@windriver.com>
> 
> There is no bitbake revision when it is in a separate repo, then we have no way
> to know which version of bitbake is used from the log, this patch always prints
> it.
> 
> * console-latest.log
> bitbake              = "master:165368bad152a14b32b5216b938aa4b915d72a70"
> meta
> meta-poky
> meta-yocto-bsp       = "master:165368bad152a14b32b5216b938aa4b915d72a70"

I think this is confusing as it implies that revision is in the bitbake
repository which it isn't, that is a poky revision. The current output
tries to make that clear, this patch starts to confuse that.

We're really close to release and I'm not going to rush into changing
this so close to a release build. I don't mind printing a bitbake
revision but it should probably only be when it is a separate repo. The
change is too late for 5.1 IMO.

Cheers,

Richard

Re: [OE-core] [PATCH v3 1/2] classes: Print bitbake branch and revision

[Robert Yang]

On 9/17/24 19:12, Richard Purdie wrote:
> On Wed, 2024-09-11 at 07:35 -0700, Robert Yang via lists.openembedded.org wrote:
>> From: Robert Yang <liezhi.yang@windriver.com>
>>
>> There is no bitbake revision when it is in a separate repo, then we have no way
>> to know which version of bitbake is used from the log, this patch always prints
>> it.
>>
>> * console-latest.log
>> bitbake              = "master:165368bad152a14b32b5216b938aa4b915d72a70"
>> meta
>> meta-poky
>> meta-yocto-bsp       = "master:165368bad152a14b32b5216b938aa4b915d72a70"
> 
> I think this is confusing as it implies that revision is in the bitbake
> repository which it isn't, that is a poky revision. The current output
> tries to make that clear, this patch starts to confuse that.
> 
> We're really close to release and I'm not going to rush into changing
> this so close to a release build. I don't mind printing a bitbake
> revision but it should probably only be when it is a separate repo. The
> change is too late for 5.1 IMO.

Thanks, I will send a V4 for 5.2.

// Robert

> 
> Cheers,
> 
> Richard

[PATCH v3 2/2] lib/buildcfg: Add is_bitbake_in_separate_repo()

[liezhi.yang]

From: Robert Yang <liezhi.yang@windriver.com>

This would help various tooling that is being developed for setting up layers
and builds, or creating configurations for those tasks out of pre-existing
yocto setups.

Suggested-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/lib/oe/buildcfg.py | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/meta/lib/oe/buildcfg.py b/meta/lib/oe/buildcfg.py
index dab4aa7831..9256b327ed 100644
--- a/meta/lib/oe/buildcfg.py
+++ b/meta/lib/oe/buildcfg.py
@@ -71,6 +71,16 @@ def is_layer_modified(path):
         # output and a 129 return code when a layer isn't a git repo at all.
         return " -- modified"
 
+def is_bitbake_in_separate_repo(bitbake_dir):
+    """
+    Check whether bitbake is in a separate git repo
+    """
+    bitbake_git_dir = os.path.join(bitbake_dir, '.git')
+    if os.path.exists(bitbake_git_dir):
+        return True
+    else:
+        return False
+
 def get_branch_rev3(revisions):
     # Return 3 items for each revision
     return ["%-20s = \"%s:%s\"" % (r[1], r[2], r[3]) for r in revisions]
-- 
2.44.1

RE: [OE-core] [PATCH v3 2/2] lib/buildcfg: Add is_bitbake_in_separate_repo()

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Robert Yang via lists.openembedded.org
> Sent: den 11 september 2024 16:36
> To: openembedded-core@lists.openembedded.org
> Cc: alex.kanavin@gmail.com
> Subject: [OE-core] [PATCH v3 2/2] lib/buildcfg: Add is_bitbake_in_separate_repo()
> 
> From: Robert Yang <liezhi.yang@windriver.com>
> 
> This would help various tooling that is being developed for setting up layers
> and builds, or creating configurations for those tasks out of pre-existing
> yocto setups.
> 
> Suggested-by: Alexander Kanavin <alex.kanavin@gmail.com>
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  meta/lib/oe/buildcfg.py | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/meta/lib/oe/buildcfg.py b/meta/lib/oe/buildcfg.py
> index dab4aa7831..9256b327ed 100644
> --- a/meta/lib/oe/buildcfg.py
> +++ b/meta/lib/oe/buildcfg.py
> @@ -71,6 +71,16 @@ def is_layer_modified(path):
>          # output and a 129 return code when a layer isn't a git repo at all.
>          return " -- modified"
> 
> +def is_bitbake_in_separate_repo(bitbake_dir):
> +    """
> +    Check whether bitbake is in a separate git repo
> +    """
> +    bitbake_git_dir = os.path.join(bitbake_dir, '.git')
> +    if os.path.exists(bitbake_git_dir):
> +        return True
> +    else:
> +        return False

That if-statement can be simplified to:

    return os.path.exists(bitbake_git_dir)

> +
>  def get_branch_rev3(revisions):
>      # Return 3 items for each revision
>      return ["%-20s = \"%s:%s\"" % (r[1], r[2], r[3]) for r in revisions]
> --
> 2.44.1

//Peter

Re: [OE-core] [PATCH v3 2/2] lib/buildcfg: Add is_bitbake_in_separate_repo()

[Robert Yang]

On 9/12/24 13:13, Peter Kjellerstedt wrote:
>> -----Original Message-----
>> From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Robert Yang via lists.openembedded.org
>> Sent: den 11 september 2024 16:36
>> To: openembedded-core@lists.openembedded.org
>> Cc: alex.kanavin@gmail.com
>> Subject: [OE-core] [PATCH v3 2/2] lib/buildcfg: Add is_bitbake_in_separate_repo()
>>
>> From: Robert Yang <liezhi.yang@windriver.com>
>>
>> This would help various tooling that is being developed for setting up layers
>> and builds, or creating configurations for those tasks out of pre-existing
>> yocto setups.
>>
>> Suggested-by: Alexander Kanavin <alex.kanavin@gmail.com>
>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> ---
>>   meta/lib/oe/buildcfg.py | 10 ++++++++++
>>   1 file changed, 10 insertions(+)
>>
>> diff --git a/meta/lib/oe/buildcfg.py b/meta/lib/oe/buildcfg.py
>> index dab4aa7831..9256b327ed 100644
>> --- a/meta/lib/oe/buildcfg.py
>> +++ b/meta/lib/oe/buildcfg.py
>> @@ -71,6 +71,16 @@ def is_layer_modified(path):
>>           # output and a 129 return code when a layer isn't a git repo at all.
>>           return " -- modified"
>>
>> +def is_bitbake_in_separate_repo(bitbake_dir):
>> +    """
>> +    Check whether bitbake is in a separate git repo
>> +    """
>> +    bitbake_git_dir = os.path.join(bitbake_dir, '.git')
>> +    if os.path.exists(bitbake_git_dir):
>> +        return True
>> +    else:
>> +        return False
> 
> That if-statement can be simplified to:
> 
>      return os.path.exists(bitbake_git_dir)

Thanks, updated in the pull request:

   https://github.com/robertlinux/yocto rbt/bbrev
   https://github.com/robertlinux/yocto/tree/rbt/bbrev

// Robert

> 
>> +
>>   def get_branch_rev3(revisions):
>>       # Return 3 items for each revision
>>       return ["%-20s = \"%s:%s\"" % (r[1], r[2], r[3]) for r in revisions]
>> --
>> 2.44.1
> 
> //Peter
>