Generated master docs not up to date - issue in run-docs-build?

Generated master docs not up to date - issue in run-docs-build?

[Michael Opdenacker]

Greetings

I've found issues with https://docs.yoctoproject.org/, generated from 
the master branch:

  * The 4.0.1 and 4.0.2 release notes don't appear on
    https://docs.yoctoproject.org/migration-guides/index.html while they
    are in the master branch
    (https://git.yoctoproject.org/yocto-docs/commit/?id=027a58616db9f3e8ee5992b57fd99cb3c284ece9
    and
    https://git.yoctoproject.org/yocto-docs/commit/?id=a17bfe2636c11ce8e3266d9add804d795991e2ea)
  * PEP517_BUILD_API is still documented on
    https://docs.yoctoproject.org/ref-manual/variables.html#term-PEP517_BUILD_API
    while it was removed by
    https://git.yoctoproject.org/yocto-docs/commit/?id=afa895871e8ec18f4861f48f70b8a54cc2e4cb98

So, some commits in master doesn't seem to have been used for generating 
the docs. However, the latest commit 
(https://git.yoctoproject.org/yocto-docs/commit/?id=5004957d1c65d03d3e296bc9f0a00982457a27f8) 
was used because you have the new hash server (hashserv.yocto.io:8687) 
on 
https://docs.yoctoproject.org/brief-yoctoprojectqs/index.html#building-your-image

On the contrary, https://docs.yoctoproject.org/next generated from the 
master-next branch looks OK as it contains the 3 above changes, and the 
changes from the latest commit.

Weird, isn't it? Could there be an issue in 
https://git.yoctoproject.org/yocto-autobuilder-helper/tree/scripts/run-docs-build 
, the scripts which generates our docs?

I started looking at the Autobuilder logs 
(https://autobuilder.yoctoproject.org/typhoon/#/builders/114), but 
haven't found anything suspicious yet. Investigations may take time though.

Any guesses?

Cheers
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [docs] Generated master docs not up to date - issue in run-docs-build?

[Quentin Schulz]

Hi Michael,

Sorry for Office365 mangling the URLs, I wish it wasn't on........

On 8/3/22 14:59, Michael Opdenacker via lists.yoctoproject.org wrote:
> Greetings
> 
> I've found issues with 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.yoctoproject.org_&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=pFIoanEyazz3qTtKZ31so443Fv7Vpfnj5fZnx3H14oQ&e= 
> , generated from the master branch:
> 
>   * The 4.0.1 and 4.0.2 release notes don't appear on
>     
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.yoctoproject.org_migration-2Dguides_index.html&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=8l5ZSIFCB5w2dI1Hi2WadpcRCQLbdi5_SgEaJQ9b9bw&e=  
> while they
>     are in the master branch
>     
> (https://urldefense.proofpoint.com/v2/url?u=https-3A__git.yoctoproject.org_yocto-2Ddocs_commit_-3Fid-3D027a58616db9f3e8ee5992b57fd99cb3c284ece9&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=Xhvjm3mSHW3Tv-1GKzIzVWMdX4ZsBOiGabMGhSbv_yc&e= 
>     and
>     
> https://urldefense.proofpoint.com/v2/url?u=https-3A__git.yoctoproject.org_yocto-2Ddocs_commit_-3Fid-3Da17bfe2636c11ce8e3266d9add804d795991e2ea&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=5RhBg7FSba6PsHFejKTGnq-KXoYKhPHU1xAMwsTZ5zg&e= 
> )

Because you're not looking at the master branch.
https://docs.yoctoproject.org/dev/migration-guides/index.html
has it.

>   * PEP517_BUILD_API is still documented on
>     
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.yoctoproject.org_ref-2Dmanual_variables.html-23term-2DPEP517-5FBUILD-5FAPI&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=Z72HE5ahwiHuHzuO6dkA0rbQQKwHM5BdD7A5YzUzMJU&e= 
>     while it was removed by
>     
> https://urldefense.proofpoint.com/v2/url?u=https-3A__git.yoctoproject.org_yocto-2Ddocs_commit_-3Fid-3Dafa895871e8ec18f4861f48f70b8a54cc2e4cb98&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=2DMYIlsNiG_tPJ9GRzF6XuEjTPbgP7CBO_TgZ8gmEgs&e= 
> 

Ditto.

> So, some commits in master doesn't seem to have been used for generating 
> the docs. However, the latest commit 
> (https://urldefense.proofpoint.com/v2/url?u=https-3A__git.yoctoproject.org_yocto-2Ddocs_commit_-3Fid-3D5004957d1c65d03d3e296bc9f0a00982457a27f8&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=cSKVy0IkfnU5lIiP7ywCf-i6RsLkyjrUcROT160Dn8M&e= 
> ) was used because you have the new hash server (hashserv.yocto.io:8687) 

No, commit f9a9c5cc721a79df7a39a6df8441739e3377fdc4 is actually used, 
because you're looking at the kirkstone branch and not master.

> on 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.yoctoproject.org_brief-2Dyoctoprojectqs_index.html-23building-2Dyour-2Dimage&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=ZscUeWWce0XqCZw86gXHzGP7krugyL07NWNnhuWGIVQ&e= 
> 
> On the contrary, 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.yoctoproject.org_next&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=9dr7-M7-VQ2FVie76y_7895g8beuIJolI-KxnYvDIMo&e=  
> generated from the master-next branch looks OK as it contains the 3 
> above changes, and the changes from the latest commit.
> 
> Weird, isn't it? Could there be an issue in 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__git.yoctoproject.org_yocto-2Dautobuilder-2Dhelper_tree_scripts_run-2Ddocs-2Dbuild&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=dkIKElbngTDiX6Iryp8iUHSfLeLX6T9R6-HZNv5SEL8&e= 
> , the scripts which generates our docs?
> 

Nope, expected.

> I started looking at the Autobuilder logs 
> (https://urldefense.proofpoint.com/v2/url?u=https-3A__autobuilder.yoctoproject.org_typhoon_-23_builders_114&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=O9gSB3rFAHS_gj5OGIzBEXjBpU5FLQjSjoCaO_PCbr4&e= 
> ), but haven't found anything suspicious yet. Investigations may take 
> time though.
> 

You won't find anything :)

So.. we decided some months ago to make the "default" branch of the docs 
on docs.yoctoproject.org point to the latest release to date instead of 
the master branch because we believe most people will use the latest 
release rather than the master branch and we don't want to misdocument 
variables, features or behaviors for them.

If we should actually redirect the "default" branch to the latest 
release branch from an HTTP point of view is a different topic. The idea 
would have https://docs.yoctoproject.org/ redirect to 
https://docs.yoctoproject.org/kirkstone until the next release and then 
redirect to langdale. I am not sure this is going to make web search 
engine SEO happy though...

Finally, you can see the branch of the documentation you're looking at 
from the dropdown menu at the top of the page.

Cheers,
Quentin

Re: [docs] Generated master docs not up to date - issue in run-docs-build?

[Quentin Schulz]

On 8/3/22 15:11, Quentin Schulz wrote:
> Hi Michael,
> 
> Sorry for Office365 mangling the URLs, I wish it wasn't on........
> 
> On 8/3/22 14:59, Michael Opdenacker via lists.yoctoproject.org wrote:
>> Greetings
>>
>> I've found issues with 
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.yoctoproject.org_&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=pFIoanEyazz3qTtKZ31so443Fv7Vpfnj5fZnx3H14oQ&e= 
>> , generated from the master branch:
>>
>>   * The 4.0.1 and 4.0.2 release notes don't appear on
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.yoctoproject.org_migration-2Dguides_index.html&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=8l5ZSIFCB5w2dI1Hi2WadpcRCQLbdi5_SgEaJQ9b9bw&e= 
>> while they
>>     are in the master branch
>> (https://urldefense.proofpoint.com/v2/url?u=https-3A__git.yoctoproject.org_yocto-2Ddocs_commit_-3Fid-3D027a58616db9f3e8ee5992b57fd99cb3c284ece9&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=Xhvjm3mSHW3Tv-1GKzIzVWMdX4ZsBOiGabMGhSbv_yc&e= 
>>     and
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__git.yoctoproject.org_yocto-2Ddocs_commit_-3Fid-3Da17bfe2636c11ce8e3266d9add804d795991e2ea&d=DwICaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=omMbxGmNQJRezZOXvOUrTA_kKPNZV4A2oVcrJjd86fPT8aVHAs52Av4OvW3-7lse&s=5RhBg7FSba6PsHFejKTGnq-KXoYKhPHU1xAMwsTZ5zg&e= 
>> )
> 
> Because you're not looking at the master branch.
> https://docs.yoctoproject.org/dev/migration-guides/index.html
> has it.
> 

But this is actually a very good point. I personally believe the Release 
Manuals section should be the same for all releases and branches.

I had in mind to force the use of the Release Manuals from the master 
branch when building with the auto-builder so that they'd always be in 
sync. I haven't had the time to do it yet, but something like rm -rf of 
the previous location of the release manuals for older releases, + a 
patch for using the master release manuals, then git checkout master 
migration-guides/ and whatnot.

This would actually be a substantial improvement to the docs IMO. Which 
would also allow us to use the latest tag of the latest release instead 
of the latest release (which was changed for kirkstone because the tags 
are created before the migration manuals are committed to the branch, 
thus not making it to the tag, which meant no migration manual available 
for kirkstone, which was bad).

Cheers,
Quentin

Re: [docs] Generated master docs not up to date - issue in run-docs-build?

[Michael Opdenacker]

On 8/3/22 15:20, Quentin Schulz wrote:
>
> But this is actually a very good point. I personally believe the 
> Release Manuals section should be the same for all releases and branches.
>
> I had in mind to force the use of the Release Manuals from the master 
> branch when building with the auto-builder so that they'd always be in 
> sync. I haven't had the time to do it yet, but something like rm -rf 
> of the previous location of the release manuals for older releases, + 
> a patch for using the master release manuals, then git checkout master 
> migration-guides/ and whatnot.


Strongly voting for this good idea!
Cheers
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com