Re: [bitbake-devel][PATCH] fetch2: git: Check if directory is a bare git repo

[Alexandre Belloni <alexandre.belloni@bootlin.com>]

Hello Joshua,

I'm fairly sure this causes the following bitbake-selftest failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/56/builds/2307/steps/11/logs/stdio

On 04/08/2023 09:54:24-0600, Joshua Watt wrote:
> If the clone target directory exists, but isn't the valid top level of a
> bare git repo, it needs to be erased and re-cloned. One example of how
> this can happen is if a clone creates the directory, but then fails to
> actual clone and make it a git repository. This left-over directory can
> be particularly problematic if the download directory is a descent of
> some top layer git repo (e.g. the default with poky), as the commands
> that operate on the remote later will then mangle the layers git
> repository instead of the download git repo.
> 
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
>  bitbake/lib/bb/fetch2/git.py | 24 +++++++++++++++++++++++-
>  1 file changed, 23 insertions(+), 1 deletion(-)
> 
> diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
> index 2a3c06fe4e9..112bed294f0 100644
> --- a/bitbake/lib/bb/fetch2/git.py
> +++ b/bitbake/lib/bb/fetch2/git.py
> @@ -65,6 +65,7 @@ import fnmatch
>  import os
>  import re
>  import shlex
> +import shutil
>  import subprocess
>  import tempfile
>  import bb
> @@ -365,8 +366,29 @@ class Git(FetchMethod):
>                  runfetchcmd(fetch_cmd, d, workdir=ud.clonedir)
>          repourl = self._get_repo_url(ud)
>  
> +        needs_clone = False
> +        if os.path.exists(ud.clonedir):
> +            # The directory may exist, but not be the top level of a bare git
> +            # repository in which case it needs to be deleted and re-cloned.
> +            try:
> +                # Since clones are bare, use --absolute-git-dir instead of --show-toplevel
> +                output = runfetchcmd("LANG=C %s rev-parse --absolute-git-dir" % ud.basecmd, d, workdir=ud.clonedir)
> +            except bb.fetch2.FetchError as e:
> +                logger.warning("Unable to get top level for %s (not a git directory?): %s", ud.clonedir, e)
> +                needs_clone = True
> +            else:
> +                toplevel = output.rstrip()
> +                if os.path.abspath(toplevel) != os.path.abspath(ud.clonedir):
> +                    logger.warning("Top level directory '%s' doesn't match expected '%s'. Re-cloning", toplevel, ud.clonedir)
> +                    needs_clone = True
> +
> +            if needs_clone:
> +                shutil.rmtree(ud.clonedir)
> +        else:
> +            needs_clone = True
> +
>          # If the repo still doesn't exist, fallback to cloning it
> -        if not os.path.exists(ud.clonedir):
> +        if needs_clone:
>              # We do this since git will use a "-l" option automatically for local urls where possible,
>              # but it doesn't work when git/objects is a symlink, only works when it is a directory.
>              if repourl.startswith("file://"):
> -- 
> 2.33.0
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#14915): https://lists.openembedded.org/g/bitbake-devel/message/14915
> Mute This Topic: https://lists.openembedded.org/mt/100548945/3617179
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com