[PATCH 0/2] Fix broken links when building on branch tip

[PATCH 0/2] Fix broken links when building on branch tip

[Antonin Godard]

When building on a branch tip, some generated links are created with the
A.B.999 version inside them. Fix this by adding the DISTRO_LATEST_TAG
macro and using this instead in the existing links.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
Antonin Godard (2):
      poky.yaml: introduce DISTRO_LATEST_TAG
      Fix dead links that use the DISTRO macro

 documentation/bsp-guide/bsp.rst                  |  2 +-
 documentation/dev-manual/qemu.rst                |  2 +-
 documentation/dev-manual/start.rst               |  2 +-
 documentation/poky.yaml.in                       | 11 ++++++++++-
 documentation/ref-manual/system-requirements.rst |  2 +-
 documentation/ref-manual/terms.rst               |  2 +-
 documentation/sdk-manual/appendix-obtain.rst     |  6 +++---
 documentation/sdk-manual/extensible.rst          |  2 +-
 documentation/sdk-manual/intro.rst               |  4 ++--
 documentation/sdk-manual/using.rst               |  2 +-
 documentation/set_versions.py                    | 16 +++++++++++++++-
 11 files changed, 37 insertions(+), 14 deletions(-)
---
base-commit: 3b50193fa0c9acf4a601aeae6e1c78d0e4a05aef
change-id: 20250409-fix-distro-dead-links-4b37bd8d54ac

Best regards,
--
Antonin Godard <antonin.godard@bootlin.com>

[PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Antonin Godard]

Introduce the DISTRO_LATEST_TAG macro, which should always point to the
latest existing tag in the documentation, unlike DISTRO which may point
to A.B.999 to represent the tip of a branch.

This variable is needed to fix dead links in the documentation that
currently use the DISTRO macro.

Also, make DISTRO_REL_TAG use the DISTRO macro directly, to avoid
repetition, and add a DISTRO_REL_LATEST_TAG macro that has the same role
as DISTRO_LATEST_TAG but with "yocto-" prepended to it.

In set_versions.py, run the "git describe --abbrev=0 --tags
--match='yocto-*'" command to get the latest existing tag on the
currently checked out commit. Fallback to ourversion in case we didn't
find any.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/poky.yaml.in    | 11 ++++++++++-
 documentation/set_versions.py | 16 +++++++++++++++-
 2 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/documentation/poky.yaml.in b/documentation/poky.yaml.in
index 836f11454..26c21e346 100644
--- a/documentation/poky.yaml.in
+++ b/documentation/poky.yaml.in
@@ -2,13 +2,22 @@
 # Macros used in the documentation
 #
 
+# The DISTRO variable represents the current docs version. It should be used
+# when referring to the current docs version. See also DISTRO_LATEST_TAG.
 DISTRO : "5.1"
+# The DISTRO_LATEST_TAG represents the latest tag on the current branch. It
+# should be used in HTTP link referring to the current docs version. In these
+# cases, the DISTRO may point to A.B.999 which does not exist (just used to
+# represent the latest HEAD revision on the branch). DISTRO_LATEST_TAG should
+# always point to an existing tag.
+DISTRO_LATEST_TAG : "5.1"
 DISTRO_NAME_NO_CAP : "styhead"
 DISTRO_NAME : "Styhead"
 DISTRO_NAME_NO_CAP_MINUS_ONE : "scarthgap"
 DISTRO_NAME_NO_CAP_LTS : "scarthgap"
 YOCTO_DOC_VERSION : "5.1"
-DISTRO_REL_TAG : "yocto-5.1"
+DISTRO_REL_TAG : "yocto-$DISTRO;"
+DISTRO_REL_LATEST_TAG : "yocto-&DISTRO_LATEST_TAG;"
 DOCCONF_VERSION : "dev"
 BITBAKE_SERIES : ""
 YOCTO_DL_URL : "https://downloads.yoctoproject.org"
diff --git a/documentation/set_versions.py b/documentation/set_versions.py
index 5c55f470d..b94a7daad 100755
--- a/documentation/set_versions.py
+++ b/documentation/set_versions.py
@@ -170,17 +170,29 @@ series = [k for k in release_series]
 previousseries = series[series.index(ourseries)+1:] or [""]
 lastlts = [k for k in previousseries if k in ltsseries] or "dunfell"
 
+latestreltag = subprocess.run(["git", "describe", "--abbrev=0", "--tags", "--match", "yocto-*"], capture_output=True, text=True).stdout
+latestreltag = latestreltag.strip()
+if latestreltag:
+    if latestreltag.startswith("yocto-"):
+        latesttag = latestreltag[6:]
+else:
+    # fallback on the calculated version
+    print("Did not find a tag with 'git describe', falling back to %s" % ourversion)
+    latestreltag = "yocto-" + ourversion
+    latesttag = ourversion
+
 print("Version calculated to be %s" % ourversion)
+print("Latest release tag found is %s" % latestreltag)
 print("Release series calculated to be %s" % ourseries)
 
 replacements = {
     "DISTRO" : ourversion,
+    "DISTRO_LATEST_TAG": latesttag,
     "DISTRO_NAME_NO_CAP" : ourseries,
     "DISTRO_NAME" : ourseries.capitalize(),
     "DISTRO_NAME_NO_CAP_MINUS_ONE" : previousseries[0],
     "DISTRO_NAME_NO_CAP_LTS" : lastlts[0],
     "YOCTO_DOC_VERSION" : ourversion,
-    "DISTRO_REL_TAG" : "yocto-" + ourversion,
     "DOCCONF_VERSION" : docconfver,
     "BITBAKE_SERIES" : bitbakeversion,
 }
@@ -318,3 +330,5 @@ with open('releases.rst', 'w') as f:
             if tag == release_series[series] or tag.startswith('%s.' % release_series[series]):
                 f.write('- :yocto_docs:`%s Documentation </%s>`\n' % (tag, tag))
         f.write('\n')
+
+

-- 
2.47.0

Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Quentin Schulz]

Hi Antonin,

On 4/9/25 11:55 AM, Antonin Godard via lists.yoctoproject.org wrote:
> Introduce the DISTRO_LATEST_TAG macro, which should always point to the
> latest existing tag in the documentation, unlike DISTRO which may point
> to A.B.999 to represent the tip of a branch.
> 
> This variable is needed to fix dead links in the documentation that
> currently use the DISTRO macro.
> 
> Also, make DISTRO_REL_TAG use the DISTRO macro directly, to avoid
> repetition, and add a DISTRO_REL_LATEST_TAG macro that has the same role
> as DISTRO_LATEST_TAG but with "yocto-" prepended to it.
> 
> In set_versions.py, run the "git describe --abbrev=0 --tags
> --match='yocto-*'" command to get the latest existing tag on the
> currently checked out commit. Fallback to ourversion in case we didn't
> find any.
> 
> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
> ---
>   documentation/poky.yaml.in    | 11 ++++++++++-
>   documentation/set_versions.py | 16 +++++++++++++++-
>   2 files changed, 25 insertions(+), 2 deletions(-)
> 
> diff --git a/documentation/poky.yaml.in b/documentation/poky.yaml.in
> index 836f11454..26c21e346 100644
> --- a/documentation/poky.yaml.in
> +++ b/documentation/poky.yaml.in
> @@ -2,13 +2,22 @@
>   # Macros used in the documentation
>   #
>   
> +# The DISTRO variable represents the current docs version. It should be used
> +# when referring to the current docs version. See also DISTRO_LATEST_TAG.
>   DISTRO : "5.1"
> +# The DISTRO_LATEST_TAG represents the latest tag on the current branch. It
> +# should be used in HTTP link referring to the current docs version. In these
> +# cases, the DISTRO may point to A.B.999 which does not exist (just used to
> +# represent the latest HEAD revision on the branch). DISTRO_LATEST_TAG should
> +# always point to an existing tag.

I don't remember why/where we need A.B.999? I assume most shouldn't 
point at .999, ever?

Also, it would be nice to provide an example as to what is supposed to 
be stored in each variable based on a specific example.

e.g.

"""
# If building from master, e.g. commit 
3b50193fa0c9acf4a601aeae6e1c78d0e4a05aef
# DISTRO will be 5.2.999
# DISTRO_LATEST_TAG will be 5.1
#
# If building from a release branch, e.g. styhead (e.g. commit 
85e738e4c0e62f69699fff4bb0482ee3e3121496)
# DISTRO will be 5.1.999
# DISTRO_LATEST_TAG will be 5.1.4
#
# If building from a tag, e.g. yocto-5.1.4
# DISTRO will be 5.1.4
# DISTRO_LATEST_TAG will be 5.1.4
"""

Note that I haven't thoroughly checked the above is correct.

If that is correct, do we really want to return 5.1 for LATEST_TAG if 
building from master?

If you always want to return the latest tag for a release, it's a bit 
more involved. For example, I did the following for finding out the 
latest tag in TF-A, including lts branches:

"""
# Use most recent release, the latest annotated tag reachable from master
LAST_V_TAG=$(git describe --abbrev=0 --match "v?*.?*.?*")
# Find all tags from different branches that contain that latest tag 
reachable from master.
# This will return lts tags, of which we want to take the latest available.
# If no LTS tag, take the latest non-rc tag reachable from master.
LAST_LTS_TAG=$(git tag --sort -version:refname --contains "$LAST_V_TAG" 
'lts-v?*.?*.?*' | head -1)
TAG=${LAST_LTS_TAG:-$LAST_V_TAG}
"""

I assume we could something similar here.

> +DISTRO_LATEST_TAG : "5.1"
>   DISTRO_NAME_NO_CAP : "styhead"
>   DISTRO_NAME : "Styhead"
>   DISTRO_NAME_NO_CAP_MINUS_ONE : "scarthgap"
>   DISTRO_NAME_NO_CAP_LTS : "scarthgap"
>   YOCTO_DOC_VERSION : "5.1"
> -DISTRO_REL_TAG : "yocto-5.1"
> +DISTRO_REL_TAG : "yocto-$DISTRO;"

Separate patch for doing that (with the removal of it in the 
replacements dict in the python script).

> +DISTRO_REL_LATEST_TAG : "yocto-&DISTRO_LATEST_TAG;">   DOCCONF_VERSION : "dev"
>   BITBAKE_SERIES : ""
>   YOCTO_DL_URL : "https://downloads.yoctoproject.org/"
> diff --git a/documentation/set_versions.py b/documentation/set_versions.py
> index 5c55f470d..b94a7daad 100755
> --- a/documentation/set_versions.py
> +++ b/documentation/set_versions.py
> @@ -170,17 +170,29 @@ series = [k for k in release_series]
>   previousseries = series[series.index(ourseries)+1:] or [""]
>   lastlts = [k for k in previousseries if k in ltsseries] or "dunfell"
>   
> +latestreltag = subprocess.run(["git", "describe", "--abbrev=0", "--tags", "--match", "yocto-*"], capture_output=True, text=True).stdout
> +latestreltag = latestreltag.strip()
> +if latestreltag:
> +    if latestreltag.startswith("yocto-"):

This is guaranteed, because of how git describe --match works, no?

"""
        --match <pattern>
            Only consider tags matching the given glob(7) pattern
"""

So we can avoid the check.

> +        latesttag = latestreltag[6:]

You could use len("yocto-") instead of 6 to be more explicit then.

> +else:
> +    # fallback on the calculated version
> +    print("Did not find a tag with 'git describe', falling back to %s" % ourversion)
> +    latestreltag = "yocto-" + ourversion
> +    latesttag = ourversion
> +
>   print("Version calculated to be %s" % ourversion)
> +print("Latest release tag found is %s" % latestreltag)
>   print("Release series calculated to be %s" % ourseries)
>   
>   replacements = {
>       "DISTRO" : ourversion,
> +    "DISTRO_LATEST_TAG": latesttag,
>       "DISTRO_NAME_NO_CAP" : ourseries,
>       "DISTRO_NAME" : ourseries.capitalize(),
>       "DISTRO_NAME_NO_CAP_MINUS_ONE" : previousseries[0],
>       "DISTRO_NAME_NO_CAP_LTS" : lastlts[0],
>       "YOCTO_DOC_VERSION" : ourversion,
> -    "DISTRO_REL_TAG" : "yocto-" + ourversion,
>       "DOCCONF_VERSION" : docconfver,
>       "BITBAKE_SERIES" : bitbakeversion,
>   }
> @@ -318,3 +330,5 @@ with open('releases.rst', 'w') as f:
>               if tag == release_series[series] or tag.startswith('%s.' % release_series[series]):
>                   f.write('- :yocto_docs:`%s Documentation </%s>`\n' % (tag, tag))
>           f.write('\n')
> +
> +
> 

Not sure we need those two additional new lines?

Cheers,
Quentin

Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Antonin Godard]

Hi Quentin,

On Mon Apr 14, 2025 at 2:18 PM CEST, Quentin Schulz wrote:
> Hi Antonin,
>
> On 4/9/25 11:55 AM, Antonin Godard via lists.yoctoproject.org wrote:
>> Introduce the DISTRO_LATEST_TAG macro, which should always point to the
>> latest existing tag in the documentation, unlike DISTRO which may point
>> to A.B.999 to represent the tip of a branch.
>> 
>> This variable is needed to fix dead links in the documentation that
>> currently use the DISTRO macro.
>> 
>> Also, make DISTRO_REL_TAG use the DISTRO macro directly, to avoid
>> repetition, and add a DISTRO_REL_LATEST_TAG macro that has the same role
>> as DISTRO_LATEST_TAG but with "yocto-" prepended to it.
>> 
>> In set_versions.py, run the "git describe --abbrev=0 --tags
>> --match='yocto-*'" command to get the latest existing tag on the
>> currently checked out commit. Fallback to ourversion in case we didn't
>> find any.
>> 
>> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
>> ---
>>   documentation/poky.yaml.in    | 11 ++++++++++-
>>   documentation/set_versions.py | 16 +++++++++++++++-
>>   2 files changed, 25 insertions(+), 2 deletions(-)
>> 
>> diff --git a/documentation/poky.yaml.in b/documentation/poky.yaml.in
>> index 836f11454..26c21e346 100644
>> --- a/documentation/poky.yaml.in
>> +++ b/documentation/poky.yaml.in
>> @@ -2,13 +2,22 @@
>>   # Macros used in the documentation
>>   #
>>   
>> +# The DISTRO variable represents the current docs version. It should be used
>> +# when referring to the current docs version. See also DISTRO_LATEST_TAG.
>>   DISTRO : "5.1"
>> +# The DISTRO_LATEST_TAG represents the latest tag on the current branch. It
>> +# should be used in HTTP link referring to the current docs version. In these
>> +# cases, the DISTRO may point to A.B.999 which does not exist (just used to
>> +# represent the latest HEAD revision on the branch). DISTRO_LATEST_TAG should
>> +# always point to an existing tag.
>
> I don't remember why/where we need A.B.999? I assume most shouldn't 
> point at .999, ever?

The default landing page of docs.yoctoproject.org is a 999 version (tip of the
stable release branch).

> Also, it would be nice to provide an example as to what is supposed to
> be stored in each variable based on a specific example.
>
> e.g.
>
> """
> # If building from master, e.g. commit 
> 3b50193fa0c9acf4a601aeae6e1c78d0e4a05aef
> # DISTRO will be 5.2.999
> # DISTRO_LATEST_TAG will be 5.1
> #
> # If building from a release branch, e.g. styhead (e.g. commit 
> 85e738e4c0e62f69699fff4bb0482ee3e3121496)
> # DISTRO will be 5.1.999
> # DISTRO_LATEST_TAG will be 5.1.4
> #
> # If building from a tag, e.g. yocto-5.1.4
> # DISTRO will be 5.1.4
> # DISTRO_LATEST_TAG will be 5.1.4
> """
>
> Note that I haven't thoroughly checked the above is correct.

That's it. Although, as you see from set_versions.py, these variables are not to
be set manually in poky.yaml.in, and are automatically set by the script.
I could add a comment on that in poky.yaml.in.

> If that is correct, do we really want to return 5.1 for LATEST_TAG if 
> building from master?

DISTRO_LATEST_TAG is used in URLs and in most cases point to
downloads.yoctoproject.org. Now I'm not sure for master we want to point to a
dead link, or the latest tag across all possible branches, or just the previous
tag on the master branch. For simplicity I'd favor the last option, especially
because master is a dev branch so we don't expect any sort of stable information
on there.

> If you always want to return the latest tag for a release, it's a bit 
> more involved. For example, I did the following for finding out the 
> latest tag in TF-A, including lts branches:

DISTRO_LATEST_TAG only represents the current branch, and I don't think it's
worth trying to find the latest tag accross all branches, considering the master
branch is not a release branch (development branch) and so we don't expect URLs
to point to a valid link anyway.

> """
> # Use most recent release, the latest annotated tag reachable from master
> LAST_V_TAG=$(git describe --abbrev=0 --match "v?*.?*.?*")
> # Find all tags from different branches that contain that latest tag 
> reachable from master.
> # This will return lts tags, of which we want to take the latest available.
> # If no LTS tag, take the latest non-rc tag reachable from master.
> LAST_LTS_TAG=$(git tag --sort -version:refname --contains "$LAST_V_TAG" 
> 'lts-v?*.?*.?*' | head -1)
> TAG=${LAST_LTS_TAG:-$LAST_V_TAG}
> """
>
> I assume we could something similar here.
>
>> +DISTRO_LATEST_TAG : "5.1"
>>   DISTRO_NAME_NO_CAP : "styhead"
>>   DISTRO_NAME : "Styhead"
>>   DISTRO_NAME_NO_CAP_MINUS_ONE : "scarthgap"
>>   DISTRO_NAME_NO_CAP_LTS : "scarthgap"
>>   YOCTO_DOC_VERSION : "5.1"
>> -DISTRO_REL_TAG : "yocto-5.1"
>> +DISTRO_REL_TAG : "yocto-$DISTRO;"
>
> Separate patch for doing that (with the removal of it in the 
> replacements dict in the python script).
>
>> +DISTRO_REL_LATEST_TAG : "yocto-&DISTRO_LATEST_TAG;">   DOCCONF_VERSION : "dev"
>>   BITBAKE_SERIES : ""
>>   YOCTO_DL_URL : "https://downloads.yoctoproject.org/"
>> diff --git a/documentation/set_versions.py b/documentation/set_versions.py
>> index 5c55f470d..b94a7daad 100755
>> --- a/documentation/set_versions.py
>> +++ b/documentation/set_versions.py
>> @@ -170,17 +170,29 @@ series = [k for k in release_series]
>>   previousseries = series[series.index(ourseries)+1:] or [""]
>>   lastlts = [k for k in previousseries if k in ltsseries] or "dunfell"
>>   
>> +latestreltag = subprocess.run(["git", "describe", "--abbrev=0", "--tags", "--match", "yocto-*"], capture_output=True, text=True).stdout
>> +latestreltag = latestreltag.strip()
>> +if latestreltag:
>> +    if latestreltag.startswith("yocto-"):
>
> This is guaranteed, because of how git describe --match works, no?
>
> """
>         --match <pattern>
>             Only consider tags matching the given glob(7) pattern
> """
>
> So we can avoid the check.

True, I think I just felt off not checking. :)

>> +        latesttag = latestreltag[6:]
>
> You could use len("yocto-") instead of 6 to be more explicit then.
>
>> +else:
>> +    # fallback on the calculated version
>> +    print("Did not find a tag with 'git describe', falling back to %s" % ourversion)
>> +    latestreltag = "yocto-" + ourversion
>> +    latesttag = ourversion
>> +
>>   print("Version calculated to be %s" % ourversion)
>> +print("Latest release tag found is %s" % latestreltag)
>>   print("Release series calculated to be %s" % ourseries)
>>   
>>   replacements = {
>>       "DISTRO" : ourversion,
>> +    "DISTRO_LATEST_TAG": latesttag,
>>       "DISTRO_NAME_NO_CAP" : ourseries,
>>       "DISTRO_NAME" : ourseries.capitalize(),
>>       "DISTRO_NAME_NO_CAP_MINUS_ONE" : previousseries[0],
>>       "DISTRO_NAME_NO_CAP_LTS" : lastlts[0],
>>       "YOCTO_DOC_VERSION" : ourversion,
>> -    "DISTRO_REL_TAG" : "yocto-" + ourversion,
>>       "DOCCONF_VERSION" : docconfver,
>>       "BITBAKE_SERIES" : bitbakeversion,
>>   }
>> @@ -318,3 +330,5 @@ with open('releases.rst', 'w') as f:
>>               if tag == release_series[series] or tag.startswith('%s.' % release_series[series]):
>>                   f.write('- :yocto_docs:`%s Documentation </%s>`\n' % (tag, tag))
>>           f.write('\n')
>> +
>> +
>> 
>
> Not sure we need those two additional new lines?

My bad, a mistake that slipped in.

Thanks,
Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Quentin Schulz]

Hi Antonin,

On 4/16/25 9:46 AM, Antonin Godard wrote:
> Hi Quentin,
> 
> On Mon Apr 14, 2025 at 2:18 PM CEST, Quentin Schulz wrote:
>> Hi Antonin,
>>
>> On 4/9/25 11:55 AM, Antonin Godard via lists.yoctoproject.org wrote:
>>> Introduce the DISTRO_LATEST_TAG macro, which should always point to the
>>> latest existing tag in the documentation, unlike DISTRO which may point
>>> to A.B.999 to represent the tip of a branch.
>>>
>>> This variable is needed to fix dead links in the documentation that
>>> currently use the DISTRO macro.
>>>
>>> Also, make DISTRO_REL_TAG use the DISTRO macro directly, to avoid
>>> repetition, and add a DISTRO_REL_LATEST_TAG macro that has the same role
>>> as DISTRO_LATEST_TAG but with "yocto-" prepended to it.
>>>
>>> In set_versions.py, run the "git describe --abbrev=0 --tags
>>> --match='yocto-*'" command to get the latest existing tag on the
>>> currently checked out commit. Fallback to ourversion in case we didn't
>>> find any.
>>>
>>> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
>>> ---
>>>    documentation/poky.yaml.in    | 11 ++++++++++-
>>>    documentation/set_versions.py | 16 +++++++++++++++-
>>>    2 files changed, 25 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/documentation/poky.yaml.in b/documentation/poky.yaml.in
>>> index 836f11454..26c21e346 100644
>>> --- a/documentation/poky.yaml.in
>>> +++ b/documentation/poky.yaml.in
>>> @@ -2,13 +2,22 @@
>>>    # Macros used in the documentation
>>>    #
>>>    
>>> +# The DISTRO variable represents the current docs version. It should be used
>>> +# when referring to the current docs version. See also DISTRO_LATEST_TAG.
>>>    DISTRO : "5.1"
>>> +# The DISTRO_LATEST_TAG represents the latest tag on the current branch. It
>>> +# should be used in HTTP link referring to the current docs version. In these
>>> +# cases, the DISTRO may point to A.B.999 which does not exist (just used to
>>> +# represent the latest HEAD revision on the branch). DISTRO_LATEST_TAG should
>>> +# always point to an existing tag.
>>
>> I don't remember why/where we need A.B.999? I assume most shouldn't
>> point at .999, ever?
> 
> The default landing page of docs.yoctoproject.org is a 999 version (tip of the
> stable release branch).
> 

That essentially should be for the version number in text and in the JS 
dropdown? We should really check our use of &DISTRO; in the docs.

For example:

documentation/contributor-guide/report-defect.rst, not sure we don't 
want to use &DISTRO_LATEST_TAG; (or another variable) instead, as it'll 
always be wrong on the default landing page of the docs (always x.x.999)?

documentation/dev-manual/multiconfig.rst, I assume we also do not want 
to have x.x.999 there? I'm also not entirely sure why we explicitly list 
the version or if it's still appropriate?

documentation/dev-manual/start.rst:774 we should probably get rid of 
&DISTRO; (but we can keep &DISTRO_NAME; as that matches the code block 
under?)

documentation/dev-manual/start.rst:843 is plain wrong as the tag does 
not exist so we should be using DISTRO_LATEST_TAG. Same for
documentation/dev-manual/start.rst:854, same for 
documentation/overview-manual/development-environment.rst:459.

documentation/ref-manual/release-process.rst:15 should probably use 
DISTRO_NAME or DISTRO_LATEST_TAG? (or probably another variable which is 
only X.Y and not X.Y.Z if there's one, since X.Y.Z is a minor release).

documentation/ref-manual/release-process.rst:59 probably should be using 
DISTRO_LATEST_TAG too?

documentation/ref-manual/system-requirements.rst:58, could be using 
DISTRO_LATEST_TAG but probably not a big deal if we keep DISTRO instead?

documentation/ref-manual/system-requirements.rst:349, I woudl assume is 
completely wrong and should be DISTRO_LATEST_TAG instead? Same for 
documentation/ref-manual/system-requirements.rst:350?

documentation/ref-manual/system-requirements.rst:403,407,411,474,478,482 
need to use DISTRO_LATEST_TAG to match the name of the buildtools.

documentation/sdk-manual/appendix-obtain.rst:66,72,176 needs to match 
the toolchain you can download, so needs to be DISTRO_LATEST_TAG.

documentation/sdk-manual/appendix-obtain.rst:275 probably wants 
DISTRO_LATEST_TAG too.

documentation/sdk-manual/extensible.rst:120,124,126 probably wants 
DISTRO_LATEST_TAG too.

documentation/sdk-manual/using.rst:74,78,80,101,102,104,105,110,130,141, 
probably wants DISTRO_LATEST_TAG too.

documentation/sdk-manual/working-projects.rst:85,88,219,223,225,280 
probably wants DISTRO_LATEST_TAG too.

documentation/toaster-manual/reference.rst:161,295 DISTRO_LATEST_TAG

Nothing uses YOCTO_RELEASE_DL_URL, maybe we can remove it entirely?

>> Also, it would be nice to provide an example as to what is supposed to
>> be stored in each variable based on a specific example.
>>
>> e.g.
>>
>> """
>> # If building from master, e.g. commit
>> 3b50193fa0c9acf4a601aeae6e1c78d0e4a05aef
>> # DISTRO will be 5.2.999
>> # DISTRO_LATEST_TAG will be 5.1
>> #
>> # If building from a release branch, e.g. styhead (e.g. commit
>> 85e738e4c0e62f69699fff4bb0482ee3e3121496)
>> # DISTRO will be 5.1.999
>> # DISTRO_LATEST_TAG will be 5.1.4
>> #
>> # If building from a tag, e.g. yocto-5.1.4
>> # DISTRO will be 5.1.4
>> # DISTRO_LATEST_TAG will be 5.1.4
>> """
>>
>> Note that I haven't thoroughly checked the above is correct.
> 
> That's it. Although, as you see from set_versions.py, these variables are not to
> be set manually in poky.yaml.in, and are automatically set by the script.
> I could add a comment on that in poky.yaml.in.
> 

It's actually been bothering me a lot that we have some parts of 
poky.yaml.in that are taken verbatim and some modified before making it 
to poky.yaml.

I seem to recall we wanted the variables set by set_versions.py to still 
be able to build the docs from tarballs and not force the use of git 
repos? But it seems like we force to fetch git tags at the very 
beginning of the Python script.

So I'm wondering if we should only have variables that aren't replaced 
in poky.yaml.in and eventually have the ones set_versions.py adds in 
poky.yaml (instead of replacing for example) documented in comments to 
explain what they are and in which context(s) to use them?

>> If that is correct, do we really want to return 5.1 for LATEST_TAG if
>> building from master?
> 
> DISTRO_LATEST_TAG is used in URLs and in most cases point to
> downloads.yoctoproject.org. Now I'm not sure for master we want to point to a
> dead link, or the latest tag across all possible branches, or just the previous
> tag on the master branch. For simplicity I'd favor the last option, especially
> because master is a dev branch so we don't expect any sort of stable information
> on there.
> 

And the dev/ branch isn't the one accessed by default when going to 
docs.yoctoproject.org/ so I think we can afford not linking to the 
latest tag in the latest release branch and instead the latest tag in 
the master branch (i.e. the last release's first tag).

[...]

Cheers,
Quentin

Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Antonin Godard]

Hi Quentin,

On Wed Apr 16, 2025 at 11:58 AM CEST, Quentin Schulz wrote:
> Hi Antonin,
>
> On 4/16/25 9:46 AM, Antonin Godard wrote:
>> Hi Quentin,
>> 
>> On Mon Apr 14, 2025 at 2:18 PM CEST, Quentin Schulz wrote:
>>> Hi Antonin,
>>>
>>> On 4/9/25 11:55 AM, Antonin Godard via lists.yoctoproject.org wrote:
>>>> Introduce the DISTRO_LATEST_TAG macro, which should always point to the
>>>> latest existing tag in the documentation, unlike DISTRO which may point
>>>> to A.B.999 to represent the tip of a branch.
>>>>
>>>> This variable is needed to fix dead links in the documentation that
>>>> currently use the DISTRO macro.
>>>>
>>>> Also, make DISTRO_REL_TAG use the DISTRO macro directly, to avoid
>>>> repetition, and add a DISTRO_REL_LATEST_TAG macro that has the same role
>>>> as DISTRO_LATEST_TAG but with "yocto-" prepended to it.
>>>>
>>>> In set_versions.py, run the "git describe --abbrev=0 --tags
>>>> --match='yocto-*'" command to get the latest existing tag on the
>>>> currently checked out commit. Fallback to ourversion in case we didn't
>>>> find any.
>>>>
>>>> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
>>>> ---
>>>>    documentation/poky.yaml.in    | 11 ++++++++++-
>>>>    documentation/set_versions.py | 16 +++++++++++++++-
>>>>    2 files changed, 25 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/documentation/poky.yaml.in b/documentation/poky.yaml.in
>>>> index 836f11454..26c21e346 100644
>>>> --- a/documentation/poky.yaml.in
>>>> +++ b/documentation/poky.yaml.in
>>>> @@ -2,13 +2,22 @@
>>>>    # Macros used in the documentation
>>>>    #
>>>>    
>>>> +# The DISTRO variable represents the current docs version. It should be used
>>>> +# when referring to the current docs version. See also DISTRO_LATEST_TAG.
>>>>    DISTRO : "5.1"
>>>> +# The DISTRO_LATEST_TAG represents the latest tag on the current branch. It
>>>> +# should be used in HTTP link referring to the current docs version. In these
>>>> +# cases, the DISTRO may point to A.B.999 which does not exist (just used to
>>>> +# represent the latest HEAD revision on the branch). DISTRO_LATEST_TAG should
>>>> +# always point to an existing tag.
>>>
>>> I don't remember why/where we need A.B.999? I assume most shouldn't
>>> point at .999, ever?
>> 
>> The default landing page of docs.yoctoproject.org is a 999 version (tip of the
>> stable release branch).
>> 
>
> That essentially should be for the version number in text and in the JS 
> dropdown? We should really check our use of &DISTRO; in the docs.
>
> For example:
>
> documentation/contributor-guide/report-defect.rst, not sure we don't 
> want to use &DISTRO_LATEST_TAG; (or another variable) instead, as it'll 
> always be wrong on the default landing page of the docs (always x.x.999)?
>
> documentation/dev-manual/multiconfig.rst, I assume we also do not want 
> to have x.x.999 there? I'm also not entirely sure why we explicitly list 
> the version or if it's still appropriate?
>
> documentation/dev-manual/start.rst:774 we should probably get rid of 
> &DISTRO; (but we can keep &DISTRO_NAME; as that matches the code block 
> under?)
>
> documentation/dev-manual/start.rst:843 is plain wrong as the tag does 
> not exist so we should be using DISTRO_LATEST_TAG. Same for
> documentation/dev-manual/start.rst:854, same for 
> documentation/overview-manual/development-environment.rst:459.
>
> documentation/ref-manual/release-process.rst:15 should probably use 
> DISTRO_NAME or DISTRO_LATEST_TAG? (or probably another variable which is 
> only X.Y and not X.Y.Z if there's one, since X.Y.Z is a minor release).
>
> documentation/ref-manual/release-process.rst:59 probably should be using 
> DISTRO_LATEST_TAG too?
>
> documentation/ref-manual/system-requirements.rst:58, could be using 
> DISTRO_LATEST_TAG but probably not a big deal if we keep DISTRO instead?
>
> documentation/ref-manual/system-requirements.rst:349, I woudl assume is 
> completely wrong and should be DISTRO_LATEST_TAG instead? Same for 
> documentation/ref-manual/system-requirements.rst:350?
>
> documentation/ref-manual/system-requirements.rst:403,407,411,474,478,482 
> need to use DISTRO_LATEST_TAG to match the name of the buildtools.
>
> documentation/sdk-manual/appendix-obtain.rst:66,72,176 needs to match 
> the toolchain you can download, so needs to be DISTRO_LATEST_TAG.
>
> documentation/sdk-manual/appendix-obtain.rst:275 probably wants 
> DISTRO_LATEST_TAG too.
>
> documentation/sdk-manual/extensible.rst:120,124,126 probably wants 
> DISTRO_LATEST_TAG too.
>
> documentation/sdk-manual/using.rst:74,78,80,101,102,104,105,110,130,141, 
> probably wants DISTRO_LATEST_TAG too.
>
> documentation/sdk-manual/working-projects.rst:85,88,219,223,225,280 
> probably wants DISTRO_LATEST_TAG too.
>
> documentation/toaster-manual/reference.rst:161,295 DISTRO_LATEST_TAG

Thanks for going through the different usages of DISTRO. You're right, with
DISTRO_LATEST_TAG now I also think it would make more sense to use it rather
than DISTRO. I will try to work on that a bit later (or feel free to send a
patch :), the initial focus for this series was to fix dead links .

> Nothing uses YOCTO_RELEASE_DL_URL, maybe we can remove it entirely?

Indeed we can remove this variable.

>>> Also, it would be nice to provide an example as to what is supposed to
>>> be stored in each variable based on a specific example.
>>>
>>> e.g.
>>>
>>> """
>>> # If building from master, e.g. commit
>>> 3b50193fa0c9acf4a601aeae6e1c78d0e4a05aef
>>> # DISTRO will be 5.2.999
>>> # DISTRO_LATEST_TAG will be 5.1
>>> #
>>> # If building from a release branch, e.g. styhead (e.g. commit
>>> 85e738e4c0e62f69699fff4bb0482ee3e3121496)
>>> # DISTRO will be 5.1.999
>>> # DISTRO_LATEST_TAG will be 5.1.4
>>> #
>>> # If building from a tag, e.g. yocto-5.1.4
>>> # DISTRO will be 5.1.4
>>> # DISTRO_LATEST_TAG will be 5.1.4
>>> """
>>>
>>> Note that I haven't thoroughly checked the above is correct.
>> 
>> That's it. Although, as you see from set_versions.py, these variables are not to
>> be set manually in poky.yaml.in, and are automatically set by the script.
>> I could add a comment on that in poky.yaml.in.
>> 
>
> It's actually been bothering me a lot that we have some parts of 
> poky.yaml.in that are taken verbatim and some modified before making it 
> to poky.yaml.
>
> I seem to recall we wanted the variables set by set_versions.py to still 
> be able to build the docs from tarballs and not force the use of git 
> repos? But it seems like we force to fetch git tags at the very 
> beginning of the Python script.

  $ make html
  ./set_versions.py
  Please run 'git fetch --tags' before building the documentation
  make: *** [Makefile:72: html] Error 1

It is indeed a bug!
Created one here: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15834

> So I'm wondering if we should only have variables that aren't replaced 
> in poky.yaml.in and eventually have the ones set_versions.py adds in 
> poky.yaml (instead of replacing for example) documented in comments to 
> explain what they are and in which context(s) to use them?

That would be a bit cleaner yes.

Maybe standards.md would be a good place? i.e.:
"Use &DISTRO; in the docs when wanting to refer to the latest possible version"
"Use &DISTRO_LATEST_TAG; in the docs when wanting to refer to the latest
existing tag"
etc.

>>> If that is correct, do we really want to return 5.1 for LATEST_TAG if
>>> building from master?
>> 
>> DISTRO_LATEST_TAG is used in URLs and in most cases point to
>> downloads.yoctoproject.org. Now I'm not sure for master we want to point to a
>> dead link, or the latest tag across all possible branches, or just the previous
>> tag on the master branch. For simplicity I'd favor the last option, especially
>> because master is a dev branch so we don't expect any sort of stable information
>> on there.
>> 
>
> And the dev/ branch isn't the one accessed by default when going to 
> docs.yoctoproject.org/ so I think we can afford not linking to the 
> latest tag in the latest release branch and instead the latest tag in 
> the master branch (i.e. the last release's first tag).
>
> [...]
>
> Cheers,
> Quentin

Thanks,
Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Quentin Schulz]

Hi Antonin,

On 4/17/25 9:21 AM, Antonin Godard wrote:
> Hi Quentin,
> 
> On Wed Apr 16, 2025 at 11:58 AM CEST, Quentin Schulz wrote:
>> Hi Antonin,
>>
>> On 4/16/25 9:46 AM, Antonin Godard wrote:
>>> Hi Quentin,
>>>
>>> On Mon Apr 14, 2025 at 2:18 PM CEST, Quentin Schulz wrote:
>>>> Hi Antonin,
>>>>
>>>> On 4/9/25 11:55 AM, Antonin Godard via lists.yoctoproject.org wrote:
>>>>> Introduce the DISTRO_LATEST_TAG macro, which should always point to the
>>>>> latest existing tag in the documentation, unlike DISTRO which may point
>>>>> to A.B.999 to represent the tip of a branch.
>>>>>
>>>>> This variable is needed to fix dead links in the documentation that
>>>>> currently use the DISTRO macro.
>>>>>
>>>>> Also, make DISTRO_REL_TAG use the DISTRO macro directly, to avoid
>>>>> repetition, and add a DISTRO_REL_LATEST_TAG macro that has the same role
>>>>> as DISTRO_LATEST_TAG but with "yocto-" prepended to it.
>>>>>
>>>>> In set_versions.py, run the "git describe --abbrev=0 --tags
>>>>> --match='yocto-*'" command to get the latest existing tag on the
>>>>> currently checked out commit. Fallback to ourversion in case we didn't
>>>>> find any.
>>>>>
>>>>> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
>>>>> ---
>>>>>     documentation/poky.yaml.in    | 11 ++++++++++-
>>>>>     documentation/set_versions.py | 16 +++++++++++++++-
>>>>>     2 files changed, 25 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/documentation/poky.yaml.in b/documentation/poky.yaml.in
>>>>> index 836f11454..26c21e346 100644
>>>>> --- a/documentation/poky.yaml.in
>>>>> +++ b/documentation/poky.yaml.in
>>>>> @@ -2,13 +2,22 @@
>>>>>     # Macros used in the documentation
>>>>>     #
>>>>>     
>>>>> +# The DISTRO variable represents the current docs version. It should be used
>>>>> +# when referring to the current docs version. See also DISTRO_LATEST_TAG.
>>>>>     DISTRO : "5.1"
>>>>> +# The DISTRO_LATEST_TAG represents the latest tag on the current branch. It
>>>>> +# should be used in HTTP link referring to the current docs version. In these
>>>>> +# cases, the DISTRO may point to A.B.999 which does not exist (just used to
>>>>> +# represent the latest HEAD revision on the branch). DISTRO_LATEST_TAG should
>>>>> +# always point to an existing tag.
>>>>
>>>> I don't remember why/where we need A.B.999? I assume most shouldn't
>>>> point at .999, ever?
>>>
>>> The default landing page of docs.yoctoproject.org is a 999 version (tip of the
>>> stable release branch).
>>>
>>
>> That essentially should be for the version number in text and in the JS
>> dropdown? We should really check our use of &DISTRO; in the docs.
>>
>> For example:
>>
>> documentation/contributor-guide/report-defect.rst, not sure we don't
>> want to use &DISTRO_LATEST_TAG; (or another variable) instead, as it'll
>> always be wrong on the default landing page of the docs (always x.x.999)?
>>
>> documentation/dev-manual/multiconfig.rst, I assume we also do not want
>> to have x.x.999 there? I'm also not entirely sure why we explicitly list
>> the version or if it's still appropriate?
>>
>> documentation/dev-manual/start.rst:774 we should probably get rid of
>> &DISTRO; (but we can keep &DISTRO_NAME; as that matches the code block
>> under?)
>>
>> documentation/dev-manual/start.rst:843 is plain wrong as the tag does
>> not exist so we should be using DISTRO_LATEST_TAG. Same for
>> documentation/dev-manual/start.rst:854, same for
>> documentation/overview-manual/development-environment.rst:459.
>>
>> documentation/ref-manual/release-process.rst:15 should probably use
>> DISTRO_NAME or DISTRO_LATEST_TAG? (or probably another variable which is
>> only X.Y and not X.Y.Z if there's one, since X.Y.Z is a minor release).
>>
>> documentation/ref-manual/release-process.rst:59 probably should be using
>> DISTRO_LATEST_TAG too?
>>
>> documentation/ref-manual/system-requirements.rst:58, could be using
>> DISTRO_LATEST_TAG but probably not a big deal if we keep DISTRO instead?
>>
>> documentation/ref-manual/system-requirements.rst:349, I woudl assume is
>> completely wrong and should be DISTRO_LATEST_TAG instead? Same for
>> documentation/ref-manual/system-requirements.rst:350?
>>
>> documentation/ref-manual/system-requirements.rst:403,407,411,474,478,482
>> need to use DISTRO_LATEST_TAG to match the name of the buildtools.
>>
>> documentation/sdk-manual/appendix-obtain.rst:66,72,176 needs to match
>> the toolchain you can download, so needs to be DISTRO_LATEST_TAG.
>>
>> documentation/sdk-manual/appendix-obtain.rst:275 probably wants
>> DISTRO_LATEST_TAG too.
>>
>> documentation/sdk-manual/extensible.rst:120,124,126 probably wants
>> DISTRO_LATEST_TAG too.
>>
>> documentation/sdk-manual/using.rst:74,78,80,101,102,104,105,110,130,141,
>> probably wants DISTRO_LATEST_TAG too.
>>
>> documentation/sdk-manual/working-projects.rst:85,88,219,223,225,280
>> probably wants DISTRO_LATEST_TAG too.
>>
>> documentation/toaster-manual/reference.rst:161,295 DISTRO_LATEST_TAG
> 
> Thanks for going through the different usages of DISTRO. You're right, with
> DISTRO_LATEST_TAG now I also think it would make more sense to use it rather
> than DISTRO. I will try to work on that a bit later (or feel free to send a
> patch :), the initial focus for this series was to fix dead links .
> 

I haven't checked if all links are now valid, but for sure some of our 
instructions aren't. I think they are as important :) or even more so 
because linkcheck wouldn't be able to catch those issues.

[...]

>> I seem to recall we wanted the variables set by set_versions.py to still
>> be able to build the docs from tarballs and not force the use of git
>> repos? But it seems like we force to fetch git tags at the very
>> beginning of the Python script.
> 
>    $ make html
>    ./set_versions.py
>    Please run 'git fetch --tags' before building the documentation
>    make: *** [Makefile:72: html] Error 1
> 
> It is indeed a bug!
> Created one here: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15834
> 
>> So I'm wondering if we should only have variables that aren't replaced
>> in poky.yaml.in and eventually have the ones set_versions.py adds in
>> poky.yaml (instead of replacing for example) documented in comments to
>> explain what they are and in which context(s) to use them?
> 
> That would be a bit cleaner yes.
> 
> Maybe standards.md would be a good place? i.e.:
> "Use &DISTRO; in the docs when wanting to refer to the latest possible version"
> "Use &DISTRO_LATEST_TAG; in the docs when wanting to refer to the latest
> existing tag"
> etc.
> 

If we need to restore building the docs from tarballs, then we need 
default values (and appropriate ones!) in poky.yaml.in. We could still 
explain proper usage in standards.md , though, up to you.

I would simply split the file in two with a big comment before the 
second section where we would define overridden variables?

I don't think we should be using &DISTRO; anymore? Most of the time we 
don't want to be printing .999 on the landing page of 
docs.yoctoproject.org! For the dev page, that's different.

Cheers,
Quentin

Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Antonin Godard]

Hi Quentin,

On Fri Apr 18, 2025 at 1:28 PM CEST, Quentin Schulz wrote:
[...]
>>> I seem to recall we wanted the variables set by set_versions.py to still
>>> be able to build the docs from tarballs and not force the use of git
>>> repos? But it seems like we force to fetch git tags at the very
>>> beginning of the Python script.
>> 
>>    $ make html
>>    ./set_versions.py
>>    Please run 'git fetch --tags' before building the documentation
>>    make: *** [Makefile:72: html] Error 1
>> 
>> It is indeed a bug!
>> Created one here: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15834
>> 
>>> So I'm wondering if we should only have variables that aren't replaced
>>> in poky.yaml.in and eventually have the ones set_versions.py adds in
>>> poky.yaml (instead of replacing for example) documented in comments to
>>> explain what they are and in which context(s) to use them?
>> 
>> That would be a bit cleaner yes.
>> 
>> Maybe standards.md would be a good place? i.e.:
>> "Use &DISTRO; in the docs when wanting to refer to the latest possible version"
>> "Use &DISTRO_LATEST_TAG; in the docs when wanting to refer to the latest
>> existing tag"
>> etc.
>> 
>
> If we need to restore building the docs from tarballs, then we need 
> default values (and appropriate ones!) in poky.yaml.in. We could still 
> explain proper usage in standards.md , though, up to you.

I discussed this with Richard the other day, and it was decided to not support
building the docs from tarballs in the end. As mentioned in the ticket above, I
will simply state that the docs must be built from the Git repository in a
message when trying to build from a tarball.

> I would simply split the file in two with a big comment before the 
> second section where we would define overridden variables?

Yep, re-organizing the file seems like a good approach to me too - easy and
effective.

> I don't think we should be using &DISTRO; anymore? Most of the time we 
> don't want to be printing .999 on the landing page of 
> docs.yoctoproject.org! For the dev page, that's different.

I would like to agree, but at the same time, wouldn't it be confusing to have a
real tag (e.g. 5.1.8) when reading for the latest tip of the branch? It would be
somewhat accurate, but not _stricty_ accurate as to which version of the doc you
are actually reading (the latest tip).

I'm trying to think of a way to make it clear that 999 represent the branch's
tip.

Some ideas to replace or append to the 999:

"5.1 (<short commit sha>)"
"5.1.999 (<short commit sha>)"
"5.1+<short commit sha>"
"5.1+<short commit sha>+<branch name>"

Maybe just using an output from `git describe` (modified to render what we want)
could be enough?

> Cheers,
> Quentin

Thanks,
Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Quentin Schulz]

Hi Antonin,

On 4/23/25 11:33 AM, Antonin Godard wrote:
[...]
>> I don't think we should be using &DISTRO; anymore? Most of the time we
>> don't want to be printing .999 on the landing page of
>> docs.yoctoproject.org! For the dev page, that's different.
> 
> I would like to agree, but at the same time, wouldn't it be confusing to have a
> real tag (e.g. 5.1.8) when reading for the latest tip of the branch? It would be

Does it matter? We should only be doing bug fixing and ideally without 
breaking backward compatibility in a given release branch.
Again, the development (master) branch is a different story here.

> somewhat accurate, but not _stricty_ accurate as to which version of the doc you
> are actually reading (the latest tip).
> 

The issue is that this tag simply doesn't exist so we should absolutely 
never use it when code is involved, as the instructions will simply not 
work.

> I'm trying to think of a way to make it clear that 999 represent the branch's
> tip.
> 

I'm failing to see how it is relevant to the user that the docs are 
built from the branch's tip instead of the tag?

BTW, it seems like once you're on a tagged release, you cannot go back 
to the .999 from the dropdown menu, is that intended?

Cheers,
Quentin

Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Antonin Godard]

On Wed Apr 23, 2025 at 11:56 AM CEST, Quentin Schulz wrote:
> Hi Antonin,
>
> On 4/23/25 11:33 AM, Antonin Godard wrote:
> [...]
>>> I don't think we should be using &DISTRO; anymore? Most of the time we
>>> don't want to be printing .999 on the landing page of
>>> docs.yoctoproject.org! For the dev page, that's different.
>> 
>> I would like to agree, but at the same time, wouldn't it be confusing to have a
>> real tag (e.g. 5.1.8) when reading for the latest tip of the branch? It would be
>
> Does it matter? We should only be doing bug fixing and ideally without 
> breaking backward compatibility in a given release branch.
> Again, the development (master) branch is a different story here.
>
>> somewhat accurate, but not _stricty_ accurate as to which version of the doc you
>> are actually reading (the latest tip).
>> 
>
> The issue is that this tag simply doesn't exist so we should absolutely 
> never use it when code is involved, as the instructions will simply not 
> work.

Right, so the code parts of the docs should be adapted to use DISTRO_LATEST_TAG,
but I still think using DISTRO in the textual parts is useful to distinguish a
branch tip version of the docs and a tagged version of the docs.

>> I'm trying to think of a way to make it clear that 999 represent the branch's
>> tip.
>> 
>
> I'm failing to see how it is relevant to the user that the docs are 
> built from the branch's tip instead of the tag?

There might be a reason that goes beyond my work on the docs, but it is kind of
useful to me to monitor changes as we go, or just to provides up-to-date links
to people.

It is questionable that this should be the default landing page though.

> BTW, it seems like once you're on a tagged release, you cannot go back 
> to the .999 from the dropdown menu, is that intended?

Sounds like a bug to me!

Thanks,
Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Quentin Schulz]

Hi Antonin,

On 5/13/25 10:28 AM, Antonin Godard wrote:
> On Wed Apr 23, 2025 at 11:56 AM CEST, Quentin Schulz wrote:
>> Hi Antonin,
>>
>> On 4/23/25 11:33 AM, Antonin Godard wrote:
>> [...]
>>>> I don't think we should be using &DISTRO; anymore? Most of the time we
>>>> don't want to be printing .999 on the landing page of
>>>> docs.yoctoproject.org! For the dev page, that's different.
>>>
>>> I would like to agree, but at the same time, wouldn't it be confusing to have a
>>> real tag (e.g. 5.1.8) when reading for the latest tip of the branch? It would be
>>
>> Does it matter? We should only be doing bug fixing and ideally without
>> breaking backward compatibility in a given release branch.
>> Again, the development (master) branch is a different story here.
>>
>>> somewhat accurate, but not _stricty_ accurate as to which version of the doc you
>>> are actually reading (the latest tip).
>>>
>>
>> The issue is that this tag simply doesn't exist so we should absolutely
>> never use it when code is involved, as the instructions will simply not
>> work.
> 
> Right, so the code parts of the docs should be adapted to use DISTRO_LATEST_TAG,
> but I still think using DISTRO in the textual parts is useful to distinguish a
> branch tip version of the docs and a tagged version of the docs.
> 

I don't see the benefit. But we can start by fixing the code blocks to 
not use DISTRO and postpone the rest of the discussion for another time 
:) Progress is still progress even if it's not perfect now.

>>> I'm trying to think of a way to make it clear that 999 represent the branch's
>>> tip.
>>>
>>
>> I'm failing to see how it is relevant to the user that the docs are
>> built from the branch's tip instead of the tag?
> 
> There might be a reason that goes beyond my work on the docs, but it is kind of
> useful to me to monitor changes as we go, or just to provides up-to-date links
> to people.
> 
> It is questionable that this should be the default landing page though.
> 

Ah, this I disagree with :) And I think this is the reason why we have 
this in the first place.

There is a very simple reason for that. We never have (or at least had, 
I don't know if this has changed) the docs ready for the release before 
the release is tagged (including the docs). This means if we were to 
always show the latest tag we would be missing information, even for the 
tagged release (e.g. recipe version bumps, cves fixed, known issues, etc.)!

Even more so, we want to avoid having to wait ~1+ month for the next 
release if there are important fixes in the documentation (e.g. 
misleading information). This we can only achieve by using the tip of 
the branch and not the release. This also apply to outdated releases 
which will never receive a new tag but we may still want to patch 
documentation (e.g. links have gone bad, misleading info, etc.).

>> BTW, it seems like once you're on a tagged release, you cannot go back
>> to the .999 from the dropdown menu, is that intended?
> 
> Sounds like a bug to me!
> 

I vaguely remember discussing this years ago. I think the issue is how 
to represent this in the menu? Should we really have

scarthgap (5.0.9)
scarthgap (dev)

for each release?

Cheers,
Quentin

Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Antonin Godard]

On Thu May 15, 2025 at 9:50 AM CEST, Quentin Schulz wrote:
> Hi Antonin,
>
> On 5/13/25 10:28 AM, Antonin Godard wrote:
>> On Wed Apr 23, 2025 at 11:56 AM CEST, Quentin Schulz wrote:
>>> Hi Antonin,
>>>
>>> On 4/23/25 11:33 AM, Antonin Godard wrote:
>>> [...]
>>>>> I don't think we should be using &DISTRO; anymore? Most of the time we
>>>>> don't want to be printing .999 on the landing page of
>>>>> docs.yoctoproject.org! For the dev page, that's different.
>>>>
>>>> I would like to agree, but at the same time, wouldn't it be confusing to have a
>>>> real tag (e.g. 5.1.8) when reading for the latest tip of the branch? It would be
>>>
>>> Does it matter? We should only be doing bug fixing and ideally without
>>> breaking backward compatibility in a given release branch.
>>> Again, the development (master) branch is a different story here.
>>>
>>>> somewhat accurate, but not _stricty_ accurate as to which version of the doc you
>>>> are actually reading (the latest tip).
>>>>
>>>
>>> The issue is that this tag simply doesn't exist so we should absolutely
>>> never use it when code is involved, as the instructions will simply not
>>> work.
>> 
>> Right, so the code parts of the docs should be adapted to use DISTRO_LATEST_TAG,
>> but I still think using DISTRO in the textual parts is useful to distinguish a
>> branch tip version of the docs and a tagged version of the docs.
>> 
>
> I don't see the benefit. But we can start by fixing the code blocks to 
> not use DISTRO and postpone the rest of the discussion for another time 
> :) Progress is still progress even if it's not perfect now.
>
>>>> I'm trying to think of a way to make it clear that 999 represent the branch's
>>>> tip.
>>>>
>>>
>>> I'm failing to see how it is relevant to the user that the docs are
>>> built from the branch's tip instead of the tag?
>> 
>> There might be a reason that goes beyond my work on the docs, but it is kind of
>> useful to me to monitor changes as we go, or just to provides up-to-date links
>> to people.
>> 
>> It is questionable that this should be the default landing page though.
>> 
>
> Ah, this I disagree with :) And I think this is the reason why we have 
> this in the first place.

I think we agree actually, I also think it is a good default for the reasons you
stated below. Maybe I wasn't very clear about that. :)

> There is a very simple reason for that. We never have (or at least had, 
> I don't know if this has changed) the docs ready for the release before 
> the release is tagged (including the docs). This means if we were to 
> always show the latest tag we would be missing information, even for the 
> tagged release (e.g. recipe version bumps, cves fixed, known issues, etc.)!
>
> Even more so, we want to avoid having to wait ~1+ month for the next 
> release if there are important fixes in the documentation (e.g. 
> misleading information). This we can only achieve by using the tip of 
> the branch and not the release. This also apply to outdated releases 
> which will never receive a new tag but we may still want to patch 
> documentation (e.g. links have gone bad, misleading info, etc.).
>
>>> BTW, it seems like once you're on a tagged release, you cannot go back
>>> to the .999 from the dropdown menu, is that intended?
>> 
>> Sounds like a bug to me!
>> 
>
> I vaguely remember discussing this years ago. I think the issue is how 
> to represent this in the menu? Should we really have
>
> scarthgap (5.0.9)
> scarthgap (dev)
>
> for each release?

I don't think it's necessary to do so for each release, but only for the current
stable release, as is done today. So I think the only real bug here is that we
don't see "Walnascar (5.2.999)" in the list as it should be.

I don't really like renaming from "Walnascar (5.2.999)" to "Walnascar (dev)" (if
that's what you implied). I would rather put "latest" instead.

Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Quentin Schulz]

Hi Antonin,

On 5/22/25 11:51 AM, Antonin Godard wrote:
> On Thu May 15, 2025 at 9:50 AM CEST, Quentin Schulz wrote:
>> Hi Antonin,
>>
>> On 5/13/25 10:28 AM, Antonin Godard wrote:
>>> On Wed Apr 23, 2025 at 11:56 AM CEST, Quentin Schulz wrote:
>>>> Hi Antonin,
>>>>
>>>> On 4/23/25 11:33 AM, Antonin Godard wrote:
[...]
>>>> BTW, it seems like once you're on a tagged release, you cannot go back
>>>> to the .999 from the dropdown menu, is that intended?
>>>
>>> Sounds like a bug to me!
>>>
>>
>> I vaguely remember discussing this years ago. I think the issue is how
>> to represent this in the menu? Should we really have
>>
>> scarthgap (5.0.9)
>> scarthgap (dev)
>>
>> for each release?
> 
> I don't think it's necessary to do so for each release, but only for the current
> stable release, as is done today. So I think the only real bug here is that we
> don't see "Walnascar (5.2.999)" in the list as it should be.
> 

I think we should have it also when looking at old unmaintained releases.

e.g. https://docs.yoctoproject.org/4.1.4/ but I would think we want to 
show https://docs.yoctoproject.org/langdale/ in the dropdown menu from 
that page.

> I don't really like renaming from "Walnascar (5.2.999)" to "Walnascar (dev)" (if
> that's what you implied). I would rather put "latest" instead.
> 

Latest for me implies the latest tag, but this is not something 
important IMO as it's easy to fix later on if it turns out whatever term 
we picked is confusing.

Cheers,
Quentin

[PATCH 2/2] Fix dead links that use the DISTRO macro

[Antonin Godard]

After introducing the DISTRO_LATEST_TAG and DISTRO_REL_LATEST_TAG
macros, use them in links that currently use DISTRO/DISTRO_REL_TAG. When
building for the tip of a branch, this will replace the current A.B.999
in links to the latest existing tag.

The links were found across the documentation by running 'grep -r
"http.*5\.2\.999"' inside the _build/html output after building the
docs.

[YOCTO #14802]

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
---
 documentation/bsp-guide/bsp.rst                  | 2 +-
 documentation/dev-manual/qemu.rst                | 2 +-
 documentation/dev-manual/start.rst               | 2 +-
 documentation/ref-manual/system-requirements.rst | 2 +-
 documentation/ref-manual/terms.rst               | 2 +-
 documentation/sdk-manual/appendix-obtain.rst     | 6 +++---
 documentation/sdk-manual/extensible.rst          | 2 +-
 documentation/sdk-manual/intro.rst               | 4 ++--
 documentation/sdk-manual/using.rst               | 2 +-
 9 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/documentation/bsp-guide/bsp.rst b/documentation/bsp-guide/bsp.rst
index 11ca5d8b76..3e484aa473 100644
--- a/documentation/bsp-guide/bsp.rst
+++ b/documentation/bsp-guide/bsp.rst
@@ -166,7 +166,7 @@ section.
    BSPs, which are maintained in their own layers or in layers designed
    to contain several BSPs. To get an idea of machine support through
    BSP layers, you can look at the
-   :yocto_dl:`index of machines </releases/yocto/yocto-&DISTRO;/machines>`
+   :yocto_dl:`index of machines </releases/yocto/&DISTRO_REL_LATEST_TAG;/machines>`
    for the release.
 
 #. *Optionally Clone the meta-intel BSP Layer:* If your hardware is
diff --git a/documentation/dev-manual/qemu.rst b/documentation/dev-manual/qemu.rst
index 030bd55143..92c93a82ab 100644
--- a/documentation/dev-manual/qemu.rst
+++ b/documentation/dev-manual/qemu.rst
@@ -75,7 +75,7 @@ available. Follow these general steps to run QEMU:
       your :term:`Build Directory`.
 
    -  If you have not built an image, you can go to the
-      :yocto_dl:`machines/qemu </releases/yocto/yocto-&DISTRO;/machines/qemu/>` area and download a
+      :yocto_dl:`machines/qemu </releases/yocto/&DISTRO_REL_LATEST_TAG;/machines/qemu/>` area and download a
       pre-built image that matches your architecture and can be run on
       QEMU.
 
diff --git a/documentation/dev-manual/start.rst b/documentation/dev-manual/start.rst
index 05827610b7..0f015d4340 100644
--- a/documentation/dev-manual/start.rst
+++ b/documentation/dev-manual/start.rst
@@ -615,7 +615,7 @@ Accessing Source Archives
 The Yocto Project also provides source archives of its releases, which
 are available on :yocto_dl:`/releases/yocto/`. Then, choose the subdirectory
 containing the release you wish to use, for example
-:yocto_dl:`yocto-&DISTRO; </releases/yocto/yocto-&DISTRO;/>`.
+:yocto_dl:`&DISTRO_REL_LATEST_TAG; </releases/yocto/&DISTRO_REL_LATEST_TAG;/>`.
 
 You will find there source archives of individual components (if you wish
 to use them individually), and of the corresponding Poky release bundling
diff --git a/documentation/ref-manual/system-requirements.rst b/documentation/ref-manual/system-requirements.rst
index 04d2674efd..b64a13320a 100644
--- a/documentation/ref-manual/system-requirements.rst
+++ b/documentation/ref-manual/system-requirements.rst
@@ -393,7 +393,7 @@ If you would prefer not to use the ``install-buildtools`` script, you can instea
 download and run a pre-built :term:`buildtools` installer yourself with the following
 steps:
 
-#. Go to :yocto_dl:`/releases/yocto/yocto-&DISTRO;/buildtools/`, locate and
+#. Go to :yocto_dl:`/releases/yocto/&DISTRO_REL_LATEST_TAG;/buildtools/`, locate and
    download the ``.sh`` file corresponding to your host architecture
    and to :term:`buildtools`, :term:`buildtools-extended` or :term:`buildtools-make`.
 
diff --git a/documentation/ref-manual/terms.rst b/documentation/ref-manual/terms.rst
index b18c4183b6..9d765c89c8 100644
--- a/documentation/ref-manual/terms.rst
+++ b/documentation/ref-manual/terms.rst
@@ -452,7 +452,7 @@ universal, the list includes them just in case:
      the Source Directory, if you do, the top-level directory name of the
      Source Directory is derived from the Yocto Project release tarball.
      For example, downloading and unpacking poky tarballs from
-     :yocto_dl:`/releases/yocto/&DISTRO_REL_TAG;/`
+     :yocto_dl:`/releases/yocto/&DISTRO_REL_LATEST_TAG;/`
      results in a Source Directory whose root folder is named poky.
 
 
diff --git a/documentation/sdk-manual/appendix-obtain.rst b/documentation/sdk-manual/appendix-obtain.rst
index d06d6ec6b5..a42cbc31bb 100644
--- a/documentation/sdk-manual/appendix-obtain.rst
+++ b/documentation/sdk-manual/appendix-obtain.rst
@@ -29,7 +29,7 @@ and then run the script to hand-install the toolchain.
 Follow these steps to locate and hand-install the toolchain:
 
 #. *Go to the Installers Directory:* Go to
-   :yocto_dl:`/releases/yocto/yocto-&DISTRO;/toolchain/`
+   :yocto_dl:`/releases/yocto/&DISTRO_REL_LATEST_TAG;/toolchain/`
 
 #. *Open the Folder for Your Build Host:* Open the folder that matches
    your :term:`Build Host` (i.e.
@@ -201,7 +201,7 @@ Follow these steps to extract the root filesystem:
    Image File:* You need to find and download the root filesystem image
    file that is appropriate for your target system. These files are kept
    in machine-specific folders in the
-   :yocto_dl:`Index of Releases </releases/yocto/yocto-&DISTRO;/machines/>`
+   :yocto_dl:`Index of Releases </releases/yocto/&DISTRO_REL_LATEST_TAG;/machines/>`
    in the "machines" directory.
 
    The machine-specific folders of the "machines" directory contain
@@ -245,7 +245,7 @@ Follow these steps to extract the root filesystem:
 
    Here is an example command that extracts the root filesystem
    from a previously built root filesystem image that was downloaded
-   from the :yocto_dl:`Index of Releases </releases/yocto/yocto-&DISTRO;/machines/>`.
+   from the :yocto_dl:`Index of Releases </releases/yocto/&DISTRO_REL_LATEST_TAG;/machines/>`.
    This command extracts the root filesystem into the ``core2-64-sato``
    directory::
 
diff --git a/documentation/sdk-manual/extensible.rst b/documentation/sdk-manual/extensible.rst
index dcb02c4e00..e5e9e4a03b 100644
--- a/documentation/sdk-manual/extensible.rst
+++ b/documentation/sdk-manual/extensible.rst
@@ -87,7 +87,7 @@ Host` by running the ``*.sh`` installation script.
 You can download a tarball installer, which includes the pre-built
 toolchain, the ``runqemu`` script, the internal build system,
 ``devtool``, and support files from the appropriate
-:yocto_dl:`toolchain </releases/yocto/yocto-&DISTRO;/toolchain/>` directory within the Index of
+:yocto_dl:`toolchain </releases/yocto/&DISTRO_REL_LATEST_TAG;/toolchain/>` directory within the Index of
 Releases. Toolchains are available for several 32-bit and 64-bit
 architectures with the ``x86_64`` directories, respectively. The
 toolchains the Yocto Project provides are based off the
diff --git a/documentation/sdk-manual/intro.rst b/documentation/sdk-manual/intro.rst
index e8fd191dbc..fbfc8c2ac7 100644
--- a/documentation/sdk-manual/intro.rst
+++ b/documentation/sdk-manual/intro.rst
@@ -173,7 +173,7 @@ You just need to follow these general steps:
    root filesystem images.
 
    If you are going to develop your application on hardware, go to the
-   :yocto_dl:`machines </releases/yocto/yocto-&DISTRO;/machines/>` download area and choose a
+   :yocto_dl:`machines </releases/yocto/&DISTRO_REL_LATEST_TAG;/machines/>` download area and choose a
    target machine area from which to download the kernel image and root
    filesystem. This download area could have several files in it that
    support development using actual hardware. For example, the area
@@ -183,7 +183,7 @@ You just need to follow these general steps:
 
    If you are going to develop your application and then run and test it
    using the QEMU emulator, go to the
-   :yocto_dl:`machines/qemu </releases/yocto/yocto-&DISTRO;/machines/qemu>` download area. From this
+   :yocto_dl:`machines/qemu </releases/yocto/&DISTRO_REL_LATEST_TAG;/machines/qemu>` download area. From this
    area, go down into the directory for your target architecture (e.g.
    ``qemux86_64`` for an Intel-based 64-bit architecture). Download the
    kernel, root filesystem, and any other files you need for your
diff --git a/documentation/sdk-manual/using.rst b/documentation/sdk-manual/using.rst
index f1ff0c76ca..bfb306abf5 100644
--- a/documentation/sdk-manual/using.rst
+++ b/documentation/sdk-manual/using.rst
@@ -43,7 +43,7 @@ Host` by running the ``*.sh`` installation script.
 
 You can download a tarball installer, which includes the pre-built
 toolchain, the ``runqemu`` script, and support files from the
-appropriate :yocto_dl:`toolchain </releases/yocto/yocto-&DISTRO;/toolchain/>` directory within
+appropriate :yocto_dl:`toolchain </releases/yocto/&DISTRO_REL_LATEST_TAG;/toolchain/>` directory within
 the Index of Releases. Toolchains are available for several 32-bit and
 64-bit architectures with the ``x86_64`` directories, respectively. The
 toolchains the Yocto Project provides are based off the

-- 
2.47.0