[meta-oe][PATCH v2] gitpkgv.bbclass: inspect repository in UNPACKDIR

[meta-oe][PATCH v2] gitpkgv.bbclass: inspect repository in UNPACKDIR

[Gyorgy Sarvari]

When BB_GIT_SHALLOW = "1" is used, the unpacked gir repository doesn't
exist in the download folder, and the class isn't able to inspect the
details of the repository.

Instead inspect the repository it the UNPACKDIR.

Beside this, since BitBake fetcher performs an actual initial shallow
clone of the repository when this feature is enabled, it is not possible
to determine the exact number of commits. Add a warning about this.

Reported-by: WXbet <WXbet@proton.me>
Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
---
 meta-oe/classes/gitpkgv.bbclass | 25 ++++++++++++++++++++-----
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/meta-oe/classes/gitpkgv.bbclass b/meta-oe/classes/gitpkgv.bbclass
index eb4b1eae9a..c3d98cb2e1 100644
--- a/meta-oe/classes/gitpkgv.bbclass
+++ b/meta-oe/classes/gitpkgv.bbclass
@@ -21,6 +21,11 @@
 # v1.2, xtest, v2.0" will force you to increment PE to get upgradeable
 # path to v2.0 revisions
 #
+# Another WARNING: Since Walnascar release BB_SHALLOW_GIT will actually
+# perform a shallow initial checkout, which makes it impossible to determine
+# the correct number of commits in the repository - thus using this class
+# is not recommended when shallow cloning is enabled.
+#
 # use example:
 #
 # inherit gitpkgv
@@ -59,6 +64,9 @@ def get_git_pkgv(d, use_tags):
     from shlex import quote
 
     src_uri = d.getVar('SRC_URI').split()
+    unpackdir = d.getVar('UNPACKDIR')
+    def_destsuffix = (d.getVar("BB_GIT_DEFAULT_DESTSUFFIX") or "git") + "/"
+
     fetcher = bb.fetch2.Fetch(src_uri, d)
     ud = fetcher.ud
 
@@ -78,20 +86,27 @@ def get_git_pkgv(d, use_tags):
     found = False
     for url in ud.values():
         if url.type == 'git' or url.type == 'gitsm':
-            if not os.path.exists(url.localpath):
+            destsuffix = url.parm.get("destsuffix", def_destsuffix)
+            subdir = url.parm.get('subdir', '')
+            destdir = os.path.join(unpackdir, destsuffix, subdir)
+
+            if not os.path.exists(destdir):
                 return None
 
+            if d.getVar('BB_GIT_SHALLOW') == '1':
+                bb.warnonce('%s: Shallow cloning enabled - gitpkgv.bbclass will not generate sortable versions' % d.getVar('PN'))
+
             found = True
 
-            vars = { 'repodir' : quote(url.localpath),
+            vars = { 'repodir' : quote(destdir),
                      'rev' : quote(url.revision) }
 
             rev = bb.fetch2.get_srcrev(d).split('+')[1]
-            rev_file = os.path.join(url.localpath, "oe-gitpkgv_" + url.revision)
+            rev_file = os.path.join(destdir, "oe-gitpkgv_" + url.revision)
 
             if not os.path.exists(rev_file) or os.path.getsize(rev_file)==0:
                 commits = bb.fetch2.runfetchcmd(
-                    "git --git-dir=%(repodir)s rev-list %(rev)s -- 2>/dev/null | wc -l"
+                    "git -C %(repodir)s rev-list %(rev)s -- 2>/dev/null | wc -l"
                     % vars, d, quiet=True).strip().lstrip('0')
 
                 if commits != "":
@@ -107,7 +122,7 @@ def get_git_pkgv(d, use_tags):
             if use_tags:
                 try:
                     output = bb.fetch2.runfetchcmd(
-                        "git --git-dir=%(repodir)s describe %(rev)s --tags --exact-match 2>/dev/null"
+                        "git -C %(repodir)s describe %(rev)s --tags --exact-match 2>/dev/null"
                         % vars, d, quiet=True).strip()
                     ver = gitpkgv_drop_tag_prefix(d, output)
                 except Exception:

AW: [meta-oe][PATCH v2] gitpkgv.bbclass: inspect repository in UNPACKDIR

[WXbet]

[-- Attachment #1: Type: text/plain, Size: 3547 bytes --]

Idea for shallow clones: Why you don’t convert the last commit‘s date/time to a UNIX timestamp number ( e.g. git show -s --format=%ct HEAD ) and use this instead of the commit count number?

Regards WXbet

Am Montag, 18. August 2025 um 07:18, Gyorgy Sarvari <[skandigraun@gmail.com](mailto:Am Montag, 18. August 2025 um 07:18, Gyorgy Sarvari <<a href=)> schrieb:

> When BB_GIT_SHALLOW = "1" is used, the unpacked gir repository doesn't
> exist in the download folder, and the class isn't able to inspect the
> details of the repository.
>
> Instead inspect the repository it the UNPACKDIR.
>
> Beside this, since BitBake fetcher performs an actual initial shallow
> clone of the repository when this feature is enabled, it is not possible
> to determine the exact number of commits. Add a warning about this.
>
> Reported-by: WXbet <WXbet@proton.me>
> Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
> ---
> meta-oe/classes/gitpkgv.bbclass | 25 ++++++++++++++++++++-----
> 1 file changed, 20 insertions(+), 5 deletions(-)
>
> diff --git a/meta-oe/classes/gitpkgv.bbclass b/meta-oe/classes/gitpkgv.bbclass
> index eb4b1eae9a..c3d98cb2e1 100644
> --- a/meta-oe/classes/gitpkgv.bbclass
> +++ b/meta-oe/classes/gitpkgv.bbclass
> @@ -21,6 +21,11 @@
> # v1.2, xtest, v2.0" will force you to increment PE to get upgradeable
> # path to v2.0 revisions
> #
> +# Another WARNING: Since Walnascar release BB_SHALLOW_GIT will actually
> +# perform a shallow initial checkout, which makes it impossible to determine
> +# the correct number of commits in the repository - thus using this class
> +# is not recommended when shallow cloning is enabled.
> +#
> # use example:
> #
> # inherit gitpkgv
> @@ -59,6 +64,9 @@ def get_git_pkgv(d, use_tags):
> from shlex import quote
>
> src_uri = d.getVar('SRC_URI').split()
> + unpackdir = d.getVar('UNPACKDIR')
> + def_destsuffix = (d.getVar("BB_GIT_DEFAULT_DESTSUFFIX") or "git") + "/"
> +
> fetcher = bb.fetch2.Fetch(src_uri, d)
> ud = fetcher.ud
>
> @@ -78,20 +86,27 @@ def get_git_pkgv(d, use_tags):
> found = False
> for url in ud.values():
> if url.type == 'git' or url.type == 'gitsm':
> - if not os.path.exists(url.localpath):
> + destsuffix = url.parm.get("destsuffix", def_destsuffix)
> + subdir = url.parm.get('subdir', '')
> + destdir = os.path.join(unpackdir, destsuffix, subdir)
> +
> + if not os.path.exists(destdir):
> return None
>
> + if d.getVar('BB_GIT_SHALLOW') == '1':
> + bb.warnonce('%s: Shallow cloning enabled - gitpkgv.bbclass will not generate sortable versions' % d.getVar('PN'))
> +
> found = True
>
> - vars = { 'repodir' : quote(url.localpath),
> + vars = { 'repodir' : quote(destdir),
> 'rev' : quote(url.revision) }
>
> rev = bb.fetch2.get_srcrev(d).split('+')[1]
> - rev_file = os.path.join(url.localpath, "oe-gitpkgv_" + url.revision)
> + rev_file = os.path.join(destdir, "oe-gitpkgv_" + url.revision)
>
> if not os.path.exists(rev_file) or os.path.getsize(rev_file)==0:
> commits = bb.fetch2.runfetchcmd(
> - "git --git-dir=%(repodir)s rev-list %(rev)s -- 2>/dev/null | wc -l"
> + "git -C %(repodir)s rev-list %(rev)s -- 2>/dev/null | wc -l"
> % vars, d, quiet=True).strip().lstrip('0')
>
> if commits != "":
> @@ -107,7 +122,7 @@ def get_git_pkgv(d, use_tags):
> if use_tags:
> try:
> output = bb.fetch2.runfetchcmd(
> - "git --git-dir=%(repodir)s describe %(rev)s --tags --exact-match 2>/dev/null"
> + "git -C %(repodir)s describe %(rev)s --tags --exact-match 2>/dev/null"
> % vars, d, quiet=True).strip()
> ver = gitpkgv_drop_tag_prefix(d, output)
> except Exception:

[-- Attachment #2: Type: text/html, Size: 4149 bytes --]

Re: [oe] [meta-oe][PATCH v2] gitpkgv.bbclass: inspect repository in UNPACKDIR

[Khem Raj]

On Mon, Aug 18, 2025 at 8:00 AM WXbet via lists.openembedded.org
<WXbet=proton.me@lists.openembedded.org> wrote:
>
> Idea for shallow clones: Why you don’t convert the last commit‘s date/time to a UNIX timestamp number ( e.g. git show -s --format=%ct HEAD ) and use this instead of the commit count number?

sometimes commit times are not applied chronologically, perhaps
because it preserves the creation time.
>
> Regards WXbet
>
>
> Am Montag, 18. August 2025 um 07:18, Gyorgy Sarvari <skandigraun@gmail.com> schrieb:
>
> When BB_GIT_SHALLOW = "1" is used, the unpacked gir repository doesn't
> exist in the download folder, and the class isn't able to inspect the
> details of the repository.
>
> Instead inspect the repository it the UNPACKDIR.
>
> Beside this, since BitBake fetcher performs an actual initial shallow
> clone of the repository when this feature is enabled, it is not possible
> to determine the exact number of commits. Add a warning about this.
>
> Reported-by: WXbet <WXbet@proton.me>
> Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
> ---
> meta-oe/classes/gitpkgv.bbclass | 25 ++++++++++++++++++++-----
> 1 file changed, 20 insertions(+), 5 deletions(-)
>
> diff --git a/meta-oe/classes/gitpkgv.bbclass b/meta-oe/classes/gitpkgv.bbclass
> index eb4b1eae9a..c3d98cb2e1 100644
> --- a/meta-oe/classes/gitpkgv.bbclass
> +++ b/meta-oe/classes/gitpkgv.bbclass
> @@ -21,6 +21,11 @@
> # v1.2, xtest, v2.0" will force you to increment PE to get upgradeable
> # path to v2.0 revisions
> #
> +# Another WARNING: Since Walnascar release BB_SHALLOW_GIT will actually
> +# perform a shallow initial checkout, which makes it impossible to determine
> +# the correct number of commits in the repository - thus using this class
> +# is not recommended when shallow cloning is enabled.
> +#
> # use example:
> #
> # inherit gitpkgv
> @@ -59,6 +64,9 @@ def get_git_pkgv(d, use_tags):
> from shlex import quote
>
> src_uri = d.getVar('SRC_URI').split()
> + unpackdir = d.getVar('UNPACKDIR')
> + def_destsuffix = (d.getVar("BB_GIT_DEFAULT_DESTSUFFIX") or "git") + "/"
> +
> fetcher = bb.fetch2.Fetch(src_uri, d)
> ud = fetcher.ud
>
> @@ -78,20 +86,27 @@ def get_git_pkgv(d, use_tags):
> found = False
> for url in ud.values():
> if url.type == 'git' or url.type == 'gitsm':
> - if not os.path.exists(url.localpath):
> + destsuffix = url.parm.get("destsuffix", def_destsuffix)
> + subdir = url.parm.get('subdir', '')
> + destdir = os.path.join(unpackdir, destsuffix, subdir)
> +
> + if not os.path.exists(destdir):
> return None
>
> + if d.getVar('BB_GIT_SHALLOW') == '1':
> + bb.warnonce('%s: Shallow cloning enabled - gitpkgv.bbclass will not generate sortable versions' % d.getVar('PN'))
> +
> found = True
>
> - vars = { 'repodir' : quote(url.localpath),
> + vars = { 'repodir' : quote(destdir),
> 'rev' : quote(url.revision) }
>
> rev = bb.fetch2.get_srcrev(d).split('+')[1]
> - rev_file = os.path.join(url.localpath, "oe-gitpkgv_" + url.revision)
> + rev_file = os.path.join(destdir, "oe-gitpkgv_" + url.revision)
>
> if not os.path.exists(rev_file) or os.path.getsize(rev_file)==0:
> commits = bb.fetch2.runfetchcmd(
> - "git --git-dir=%(repodir)s rev-list %(rev)s -- 2>/dev/null | wc -l"
> + "git -C %(repodir)s rev-list %(rev)s -- 2>/dev/null | wc -l"
> % vars, d, quiet=True).strip().lstrip('0')
>
> if commits != "":
> @@ -107,7 +122,7 @@ def get_git_pkgv(d, use_tags):
> if use_tags:
> try:
> output = bb.fetch2.runfetchcmd(
> - "git --git-dir=%(repodir)s describe %(rev)s --tags --exact-match 2>/dev/null"
> + "git -C %(repodir)s describe %(rev)s --tags --exact-match 2>/dev/null"
> % vars, d, quiet=True).strip()
> ver = gitpkgv_drop_tag_prefix(d, output)
> except Exception:
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#118994): https://lists.openembedded.org/g/openembedded-devel/message/118994
> Mute This Topic: https://lists.openembedded.org/mt/114758958/1997914
> Group Owner: openembedded-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

AW: Re: [oe] [meta-oe][PATCH v2] gitpkgv.bbclass: inspect repository in UNPACKDIR

[WXbet]

[-- Attachment #1: Type: text/plain, Size: 4674 bytes --]

As every commit stores two distinct timestamps: the author date (creation) and the commit date (recording), I meant the commit date.

I know, my idea isn’t perfect, it’s an offer to think about, how to get an increasing number instead the value of BB_GIT_SHALLOW_DEPTH.

Am Montag, 18. August 2025 um 17:54, Khem Raj <[raj.khem@gmail.com](mailto:Am Montag, 18. August 2025 um 17:54, Khem Raj <<a href=)> schrieb:

> On Mon, Aug 18, 2025 at 8:00 AM WXbet via lists.openembedded.org
> <WXbet=proton.me@lists.openembedded.org> wrote:
>>
>> Idea for shallow clones: Why you don’t convert the last commit‘s date/time to a UNIX timestamp number ( e.g. git show -s --format=%ct HEAD ) and use this instead of the commit count number?
>
> sometimes commit times are not applied chronologically, perhaps
> because it preserves the creation time.
>>
>> Regards WXbet
>>
>>
>> Am Montag, 18. August 2025 um 07:18, Gyorgy Sarvari <skandigraun@gmail.com> schrieb:
>>
>> When BB_GIT_SHALLOW = "1" is used, the unpacked gir repository doesn't
>> exist in the download folder, and the class isn't able to inspect the
>> details of the repository.
>>
>> Instead inspect the repository it the UNPACKDIR.
>>
>> Beside this, since BitBake fetcher performs an actual initial shallow
>> clone of the repository when this feature is enabled, it is not possible
>> to determine the exact number of commits. Add a warning about this.
>>
>> Reported-by: WXbet <WXbet@proton.me>
>> Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
>> ---
>> meta-oe/classes/gitpkgv.bbclass | 25 ++++++++++++++++++++-----
>> 1 file changed, 20 insertions(+), 5 deletions(-)
>>
>> diff --git a/meta-oe/classes/gitpkgv.bbclass b/meta-oe/classes/gitpkgv.bbclass
>> index eb4b1eae9a..c3d98cb2e1 100644
>> --- a/meta-oe/classes/gitpkgv.bbclass
>> +++ b/meta-oe/classes/gitpkgv.bbclass
>> @@ -21,6 +21,11 @@
>> # v1.2, xtest, v2.0" will force you to increment PE to get upgradeable
>> # path to v2.0 revisions
>> #
>> +# Another WARNING: Since Walnascar release BB_SHALLOW_GIT will actually
>> +# perform a shallow initial checkout, which makes it impossible to determine
>> +# the correct number of commits in the repository - thus using this class
>> +# is not recommended when shallow cloning is enabled.
>> +#
>> # use example:
>> #
>> # inherit gitpkgv
>> @@ -59,6 +64,9 @@ def get_git_pkgv(d, use_tags):
>> from shlex import quote
>>
>> src_uri = d.getVar('SRC_URI').split()
>> + unpackdir = d.getVar('UNPACKDIR')
>> + def_destsuffix = (d.getVar("BB_GIT_DEFAULT_DESTSUFFIX") or "git") + "/"
>> +
>> fetcher = bb.fetch2.Fetch(src_uri, d)
>> ud = fetcher.ud
>>
>> @@ -78,20 +86,27 @@ def get_git_pkgv(d, use_tags):
>> found = False
>> for url in ud.values():
>> if url.type == 'git' or url.type == 'gitsm':
>> - if not os.path.exists(url.localpath):
>> + destsuffix = url.parm.get("destsuffix", def_destsuffix)
>> + subdir = url.parm.get('subdir', '')
>> + destdir = os.path.join(unpackdir, destsuffix, subdir)
>> +
>> + if not os.path.exists(destdir):
>> return None
>>
>> + if d.getVar('BB_GIT_SHALLOW') == '1':
>> + bb.warnonce('%s: Shallow cloning enabled - gitpkgv.bbclass will not generate sortable versions' % d.getVar('PN'))
>> +
>> found = True
>>
>> - vars = { 'repodir' : quote(url.localpath),
>> + vars = { 'repodir' : quote(destdir),
>> 'rev' : quote(url.revision) }
>>
>> rev = bb.fetch2.get_srcrev(d).split('+')[1]
>> - rev_file = os.path.join(url.localpath, "oe-gitpkgv_" + url.revision)
>> + rev_file = os.path.join(destdir, "oe-gitpkgv_" + url.revision)
>>
>> if not os.path.exists(rev_file) or os.path.getsize(rev_file)==0:
>> commits = bb.fetch2.runfetchcmd(
>> - "git --git-dir=%(repodir)s rev-list %(rev)s -- 2>/dev/null | wc -l"
>> + "git -C %(repodir)s rev-list %(rev)s -- 2>/dev/null | wc -l"
>> % vars, d, quiet=True).strip().lstrip('0')
>>
>> if commits != "":
>> @@ -107,7 +122,7 @@ def get_git_pkgv(d, use_tags):
>> if use_tags:
>> try:
>> output = bb.fetch2.runfetchcmd(
>> - "git --git-dir=%(repodir)s describe %(rev)s --tags --exact-match 2>/dev/null"
>> + "git -C %(repodir)s describe %(rev)s --tags --exact-match 2>/dev/null"
>> % vars, d, quiet=True).strip()
>> ver = gitpkgv_drop_tag_prefix(d, output)
>> except Exception:
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#118994): https://lists.openembedded.org/g/openembedded-devel/message/118994
>> Mute This Topic: https://lists.openembedded.org/mt/114758958/1997914
>> Group Owner: openembedded-devel+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [raj.khem@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>

[-- Attachment #2: Type: text/html, Size: 5913 bytes --]