Re: [yocto] [PATCH yocto-autobuilder-helper] scripts: run-docs-build: make the workdir pristine between builds

[Luca Ceresoli <luca.ceresoli@bootlin.com>]

Hi Quentin,

On Wed, 5 Oct 2022 10:55:44 +0200
Quentin Schulz <quentin.schulz@theobroma-systems.com> wrote:

> Hi Luka,
> 
> On 10/4/22 22:54, Luca Ceresoli wrote:
> > Hi Quentin,
> > 
> > On Tue, 4 Oct 2022 10:15:20 +0200
> > Quentin Schulz <quentin.schulz@theobroma-systems.com> wrote:
> >   
> >> Hi Luca,
> >>
> >> On 10/3/22 23:15, Luca Ceresoli wrote:  
> >>> Hi Quentin,
> >>>
> >>> On Mon,  3 Oct 2022 19:04:01 +0200
> >>> "Quentin Schulz" <foss@0leil.net> wrote:
> >>>      
> >>>> From: Quentin Schulz <quentin.schulz@theobroma-systems.com>
> >>>>
> >>>> It happened that the git repositories were dirty and resulted in
> >>>> incorrect files being used. Let's use git clean -ffdx to force a
> >>>> completely clean git repositories before and after checking out a branch
> >>>> so that nothing is left from or to another branch build
> >>>>
> >>>> Cc: Quentin Schulz <foss+yocto@0leil.net>
> >>>> Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
> >>>> ---
> >>>>    scripts/run-docs-build | 10 +++++-----
> >>>>    1 file changed, 5 insertions(+), 5 deletions(-)
> >>>>
> >>>> diff --git a/scripts/run-docs-build b/scripts/run-docs-build
> >>>> index c6b3965..69e3257 100755
> >>>> --- a/scripts/run-docs-build
> >>>> +++ b/scripts/run-docs-build
> >>>> @@ -61,6 +61,7 @@ for branch in 1.46 $(git branch --remote --contains "$first_sphinx_commit" --for
> >>>>    
> >>>>        echo Building bitbake $branch branch
> >>>>        git checkout $branch
> >>>> +    git clean -ffdx
> >>>>        git checkout origin/master releases.rst
> >>>>        make clean
> >>>>        SPHINXOPTS="-j auto" make publish
> >>>> @@ -80,7 +81,7 @@ for branch in 1.46 $(git branch --remote --contains "$first_sphinx_commit" --for
> >>>>        fi
> >>>>    
> >>>>        cp -r ./_build/final/* $outputdir/bitbake/$branch
> >>>> -    git reset --hard
> >>>> +    git clean -ffdx  
> >>>
> >>> Sure this is correct? 'git clean -ffdx' does not revert changes to
> >>> tracked files, be them staged or not.
> >>>      
> >>
> >> Nope, not sure this is correct. I misread git clean manpage, we should
> >> have a git reset --hard and git clean -ffdx. Now the question is when
> >> those are necessary because with this patch we do it twice, before and
> >> after the git checkout. I did this because I remember doing checkouts
> >> between branches of U-Boot/kernel and while the pre-checkout branch was
> >> not dirty, the after-checkout branch was dirty. I assume this might have
> >> something to do with build artifacts of the pre-checkout build that
> >> weren't .gitignored in the afer-checkout branch? Something that git
> >> clean -ffdx should tackle I think.
> >>
> >> Sooo, I guess only having git reset --hard and git clean -ffdx before a
> >> checkout should be enough and we don't need them both before and after
> >> the checkout like I did in this patch?  
> > 
> > I think 'reset --hard' + 'clean -ffdx' only before the checkout should
> > be enough. However I'm not sure whether there are corner cases such as
> > a file that is .gitignored in commit A and versioned in commit B or
> > similar. Perhaps worth trying with reset+clean only before, and see  
> 
> I guess it does not hurt to be on the safe side by having them before 
> and after the git checkout then? Since the current issue went unnoticed 
> for months...

Sorry for the delayed reply. It took a while before I found a little
time to look at the script code...

Indeed cleaning before _and_ after would be safe, even though perhaps
unneeded.

> > what happens. However I don't know exactly the initial problem you're
> > trying to fix.
> >   
> 
> https://lore.kernel.org/yocto-docs/e50abe3c777e4a23a752a3ec25ad0b2a@axis.com/T/#t

Ah, interesting. Thanks for the link. Why not adding it to your commit
message, for reference?

Best regards.
-- 
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com