Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

["Antonin Godard" <antonin.godard@bootlin.com>]

Hi Quentin,

On Mon Apr 14, 2025 at 2:18 PM CEST, Quentin Schulz wrote:
> Hi Antonin,
>
> On 4/9/25 11:55 AM, Antonin Godard via lists.yoctoproject.org wrote:
>> Introduce the DISTRO_LATEST_TAG macro, which should always point to the
>> latest existing tag in the documentation, unlike DISTRO which may point
>> to A.B.999 to represent the tip of a branch.
>> 
>> This variable is needed to fix dead links in the documentation that
>> currently use the DISTRO macro.
>> 
>> Also, make DISTRO_REL_TAG use the DISTRO macro directly, to avoid
>> repetition, and add a DISTRO_REL_LATEST_TAG macro that has the same role
>> as DISTRO_LATEST_TAG but with "yocto-" prepended to it.
>> 
>> In set_versions.py, run the "git describe --abbrev=0 --tags
>> --match='yocto-*'" command to get the latest existing tag on the
>> currently checked out commit. Fallback to ourversion in case we didn't
>> find any.
>> 
>> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
>> ---
>>   documentation/poky.yaml.in    | 11 ++++++++++-
>>   documentation/set_versions.py | 16 +++++++++++++++-
>>   2 files changed, 25 insertions(+), 2 deletions(-)
>> 
>> diff --git a/documentation/poky.yaml.in b/documentation/poky.yaml.in
>> index 836f11454..26c21e346 100644
>> --- a/documentation/poky.yaml.in
>> +++ b/documentation/poky.yaml.in
>> @@ -2,13 +2,22 @@
>>   # Macros used in the documentation
>>   #
>>   
>> +# The DISTRO variable represents the current docs version. It should be used
>> +# when referring to the current docs version. See also DISTRO_LATEST_TAG.
>>   DISTRO : "5.1"
>> +# The DISTRO_LATEST_TAG represents the latest tag on the current branch. It
>> +# should be used in HTTP link referring to the current docs version. In these
>> +# cases, the DISTRO may point to A.B.999 which does not exist (just used to
>> +# represent the latest HEAD revision on the branch). DISTRO_LATEST_TAG should
>> +# always point to an existing tag.
>
> I don't remember why/where we need A.B.999? I assume most shouldn't 
> point at .999, ever?

The default landing page of docs.yoctoproject.org is a 999 version (tip of the
stable release branch).

> Also, it would be nice to provide an example as to what is supposed to
> be stored in each variable based on a specific example.
>
> e.g.
>
> """
> # If building from master, e.g. commit 
> 3b50193fa0c9acf4a601aeae6e1c78d0e4a05aef
> # DISTRO will be 5.2.999
> # DISTRO_LATEST_TAG will be 5.1
> #
> # If building from a release branch, e.g. styhead (e.g. commit 
> 85e738e4c0e62f69699fff4bb0482ee3e3121496)
> # DISTRO will be 5.1.999
> # DISTRO_LATEST_TAG will be 5.1.4
> #
> # If building from a tag, e.g. yocto-5.1.4
> # DISTRO will be 5.1.4
> # DISTRO_LATEST_TAG will be 5.1.4
> """
>
> Note that I haven't thoroughly checked the above is correct.

That's it. Although, as you see from set_versions.py, these variables are not to
be set manually in poky.yaml.in, and are automatically set by the script.
I could add a comment on that in poky.yaml.in.

> If that is correct, do we really want to return 5.1 for LATEST_TAG if 
> building from master?

DISTRO_LATEST_TAG is used in URLs and in most cases point to
downloads.yoctoproject.org. Now I'm not sure for master we want to point to a
dead link, or the latest tag across all possible branches, or just the previous
tag on the master branch. For simplicity I'd favor the last option, especially
because master is a dev branch so we don't expect any sort of stable information
on there.

> If you always want to return the latest tag for a release, it's a bit 
> more involved. For example, I did the following for finding out the 
> latest tag in TF-A, including lts branches:

DISTRO_LATEST_TAG only represents the current branch, and I don't think it's
worth trying to find the latest tag accross all branches, considering the master
branch is not a release branch (development branch) and so we don't expect URLs
to point to a valid link anyway.

> """
> # Use most recent release, the latest annotated tag reachable from master
> LAST_V_TAG=$(git describe --abbrev=0 --match "v?*.?*.?*")
> # Find all tags from different branches that contain that latest tag 
> reachable from master.
> # This will return lts tags, of which we want to take the latest available.
> # If no LTS tag, take the latest non-rc tag reachable from master.
> LAST_LTS_TAG=$(git tag --sort -version:refname --contains "$LAST_V_TAG" 
> 'lts-v?*.?*.?*' | head -1)
> TAG=${LAST_LTS_TAG:-$LAST_V_TAG}
> """
>
> I assume we could something similar here.
>
>> +DISTRO_LATEST_TAG : "5.1"
>>   DISTRO_NAME_NO_CAP : "styhead"
>>   DISTRO_NAME : "Styhead"
>>   DISTRO_NAME_NO_CAP_MINUS_ONE : "scarthgap"
>>   DISTRO_NAME_NO_CAP_LTS : "scarthgap"
>>   YOCTO_DOC_VERSION : "5.1"
>> -DISTRO_REL_TAG : "yocto-5.1"
>> +DISTRO_REL_TAG : "yocto-$DISTRO;"
>
> Separate patch for doing that (with the removal of it in the 
> replacements dict in the python script).
>
>> +DISTRO_REL_LATEST_TAG : "yocto-&DISTRO_LATEST_TAG;">   DOCCONF_VERSION : "dev"
>>   BITBAKE_SERIES : ""
>>   YOCTO_DL_URL : "https://downloads.yoctoproject.org/"
>> diff --git a/documentation/set_versions.py b/documentation/set_versions.py
>> index 5c55f470d..b94a7daad 100755
>> --- a/documentation/set_versions.py
>> +++ b/documentation/set_versions.py
>> @@ -170,17 +170,29 @@ series = [k for k in release_series]
>>   previousseries = series[series.index(ourseries)+1:] or [""]
>>   lastlts = [k for k in previousseries if k in ltsseries] or "dunfell"
>>   
>> +latestreltag = subprocess.run(["git", "describe", "--abbrev=0", "--tags", "--match", "yocto-*"], capture_output=True, text=True).stdout
>> +latestreltag = latestreltag.strip()
>> +if latestreltag:
>> +    if latestreltag.startswith("yocto-"):
>
> This is guaranteed, because of how git describe --match works, no?
>
> """
>         --match <pattern>
>             Only consider tags matching the given glob(7) pattern
> """
>
> So we can avoid the check.

True, I think I just felt off not checking. :)

>> +        latesttag = latestreltag[6:]
>
> You could use len("yocto-") instead of 6 to be more explicit then.
>
>> +else:
>> +    # fallback on the calculated version
>> +    print("Did not find a tag with 'git describe', falling back to %s" % ourversion)
>> +    latestreltag = "yocto-" + ourversion
>> +    latesttag = ourversion
>> +
>>   print("Version calculated to be %s" % ourversion)
>> +print("Latest release tag found is %s" % latestreltag)
>>   print("Release series calculated to be %s" % ourseries)
>>   
>>   replacements = {
>>       "DISTRO" : ourversion,
>> +    "DISTRO_LATEST_TAG": latesttag,
>>       "DISTRO_NAME_NO_CAP" : ourseries,
>>       "DISTRO_NAME" : ourseries.capitalize(),
>>       "DISTRO_NAME_NO_CAP_MINUS_ONE" : previousseries[0],
>>       "DISTRO_NAME_NO_CAP_LTS" : lastlts[0],
>>       "YOCTO_DOC_VERSION" : ourversion,
>> -    "DISTRO_REL_TAG" : "yocto-" + ourversion,
>>       "DOCCONF_VERSION" : docconfver,
>>       "BITBAKE_SERIES" : bitbakeversion,
>>   }
>> @@ -318,3 +330,5 @@ with open('releases.rst', 'w') as f:
>>               if tag == release_series[series] or tag.startswith('%s.' % release_series[series]):
>>                   f.write('- :yocto_docs:`%s Documentation </%s>`\n' % (tag, tag))
>>           f.write('\n')
>> +
>> +
>> 
>
> Not sure we need those two additional new lines?

My bad, a mistake that slipped in.

Thanks,
Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com