Re: [docs] [PATCH 1/2] poky.yaml: introduce DISTRO_LATEST_TAG

[Quentin Schulz <quentin.schulz@cherry.de>]

Hi Antonin,

On 4/16/25 9:46 AM, Antonin Godard wrote:
> Hi Quentin,
> 
> On Mon Apr 14, 2025 at 2:18 PM CEST, Quentin Schulz wrote:
>> Hi Antonin,
>>
>> On 4/9/25 11:55 AM, Antonin Godard via lists.yoctoproject.org wrote:
>>> Introduce the DISTRO_LATEST_TAG macro, which should always point to the
>>> latest existing tag in the documentation, unlike DISTRO which may point
>>> to A.B.999 to represent the tip of a branch.
>>>
>>> This variable is needed to fix dead links in the documentation that
>>> currently use the DISTRO macro.
>>>
>>> Also, make DISTRO_REL_TAG use the DISTRO macro directly, to avoid
>>> repetition, and add a DISTRO_REL_LATEST_TAG macro that has the same role
>>> as DISTRO_LATEST_TAG but with "yocto-" prepended to it.
>>>
>>> In set_versions.py, run the "git describe --abbrev=0 --tags
>>> --match='yocto-*'" command to get the latest existing tag on the
>>> currently checked out commit. Fallback to ourversion in case we didn't
>>> find any.
>>>
>>> Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
>>> ---
>>>    documentation/poky.yaml.in    | 11 ++++++++++-
>>>    documentation/set_versions.py | 16 +++++++++++++++-
>>>    2 files changed, 25 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/documentation/poky.yaml.in b/documentation/poky.yaml.in
>>> index 836f11454..26c21e346 100644
>>> --- a/documentation/poky.yaml.in
>>> +++ b/documentation/poky.yaml.in
>>> @@ -2,13 +2,22 @@
>>>    # Macros used in the documentation
>>>    #
>>>    
>>> +# The DISTRO variable represents the current docs version. It should be used
>>> +# when referring to the current docs version. See also DISTRO_LATEST_TAG.
>>>    DISTRO : "5.1"
>>> +# The DISTRO_LATEST_TAG represents the latest tag on the current branch. It
>>> +# should be used in HTTP link referring to the current docs version. In these
>>> +# cases, the DISTRO may point to A.B.999 which does not exist (just used to
>>> +# represent the latest HEAD revision on the branch). DISTRO_LATEST_TAG should
>>> +# always point to an existing tag.
>>
>> I don't remember why/where we need A.B.999? I assume most shouldn't
>> point at .999, ever?
> 
> The default landing page of docs.yoctoproject.org is a 999 version (tip of the
> stable release branch).
> 

That essentially should be for the version number in text and in the JS 
dropdown? We should really check our use of &DISTRO; in the docs.

For example:

documentation/contributor-guide/report-defect.rst, not sure we don't 
want to use &DISTRO_LATEST_TAG; (or another variable) instead, as it'll 
always be wrong on the default landing page of the docs (always x.x.999)?

documentation/dev-manual/multiconfig.rst, I assume we also do not want 
to have x.x.999 there? I'm also not entirely sure why we explicitly list 
the version or if it's still appropriate?

documentation/dev-manual/start.rst:774 we should probably get rid of 
&DISTRO; (but we can keep &DISTRO_NAME; as that matches the code block 
under?)

documentation/dev-manual/start.rst:843 is plain wrong as the tag does 
not exist so we should be using DISTRO_LATEST_TAG. Same for
documentation/dev-manual/start.rst:854, same for 
documentation/overview-manual/development-environment.rst:459.

documentation/ref-manual/release-process.rst:15 should probably use 
DISTRO_NAME or DISTRO_LATEST_TAG? (or probably another variable which is 
only X.Y and not X.Y.Z if there's one, since X.Y.Z is a minor release).

documentation/ref-manual/release-process.rst:59 probably should be using 
DISTRO_LATEST_TAG too?

documentation/ref-manual/system-requirements.rst:58, could be using 
DISTRO_LATEST_TAG but probably not a big deal if we keep DISTRO instead?

documentation/ref-manual/system-requirements.rst:349, I woudl assume is 
completely wrong and should be DISTRO_LATEST_TAG instead? Same for 
documentation/ref-manual/system-requirements.rst:350?

documentation/ref-manual/system-requirements.rst:403,407,411,474,478,482 
need to use DISTRO_LATEST_TAG to match the name of the buildtools.

documentation/sdk-manual/appendix-obtain.rst:66,72,176 needs to match 
the toolchain you can download, so needs to be DISTRO_LATEST_TAG.

documentation/sdk-manual/appendix-obtain.rst:275 probably wants 
DISTRO_LATEST_TAG too.

documentation/sdk-manual/extensible.rst:120,124,126 probably wants 
DISTRO_LATEST_TAG too.

documentation/sdk-manual/using.rst:74,78,80,101,102,104,105,110,130,141, 
probably wants DISTRO_LATEST_TAG too.

documentation/sdk-manual/working-projects.rst:85,88,219,223,225,280 
probably wants DISTRO_LATEST_TAG too.

documentation/toaster-manual/reference.rst:161,295 DISTRO_LATEST_TAG

Nothing uses YOCTO_RELEASE_DL_URL, maybe we can remove it entirely?

>> Also, it would be nice to provide an example as to what is supposed to
>> be stored in each variable based on a specific example.
>>
>> e.g.
>>
>> """
>> # If building from master, e.g. commit
>> 3b50193fa0c9acf4a601aeae6e1c78d0e4a05aef
>> # DISTRO will be 5.2.999
>> # DISTRO_LATEST_TAG will be 5.1
>> #
>> # If building from a release branch, e.g. styhead (e.g. commit
>> 85e738e4c0e62f69699fff4bb0482ee3e3121496)
>> # DISTRO will be 5.1.999
>> # DISTRO_LATEST_TAG will be 5.1.4
>> #
>> # If building from a tag, e.g. yocto-5.1.4
>> # DISTRO will be 5.1.4
>> # DISTRO_LATEST_TAG will be 5.1.4
>> """
>>
>> Note that I haven't thoroughly checked the above is correct.
> 
> That's it. Although, as you see from set_versions.py, these variables are not to
> be set manually in poky.yaml.in, and are automatically set by the script.
> I could add a comment on that in poky.yaml.in.
> 

It's actually been bothering me a lot that we have some parts of 
poky.yaml.in that are taken verbatim and some modified before making it 
to poky.yaml.

I seem to recall we wanted the variables set by set_versions.py to still 
be able to build the docs from tarballs and not force the use of git 
repos? But it seems like we force to fetch git tags at the very 
beginning of the Python script.

So I'm wondering if we should only have variables that aren't replaced 
in poky.yaml.in and eventually have the ones set_versions.py adds in 
poky.yaml (instead of replacing for example) documented in comments to 
explain what they are and in which context(s) to use them?

>> If that is correct, do we really want to return 5.1 for LATEST_TAG if
>> building from master?
> 
> DISTRO_LATEST_TAG is used in URLs and in most cases point to
> downloads.yoctoproject.org. Now I'm not sure for master we want to point to a
> dead link, or the latest tag across all possible branches, or just the previous
> tag on the master branch. For simplicity I'd favor the last option, especially
> because master is a dev branch so we don't expect any sort of stable information
> on there.
> 

And the dev/ branch isn't the one accessed by default when going to 
docs.yoctoproject.org/ so I think we can afford not linking to the 
latest tag in the latest release branch and instead the latest tag in 
the master branch (i.e. the last release's first tag).

[...]

Cheers,
Quentin