Re: [bitbake-devel] [PATCH v3 1/4] fetch2/git: Add support for fast initial shallow fetch

["Mathieu Dubois-Briand" <mathieu.dubois-briand@bootlin.com>]

On Wed Feb 26, 2025 at 9:39 AM CET, Mathieu Dubois-Briand wrote:
> On Tue Feb 25, 2025 at 6:04 PM CET, Stefan Koch wrote:
> > So far, I can't reproduce the issue using a ubuntu 20.04 container.
> > git clone git://git.yoctoproject.org/poky
> > Patching subdir bitbake/
> > oe-selftest -r devtool.DevtoolUpdateTests.test_devtool_git_submodules
> > bitbake git-submodule-test
> >
> > The current master-next branch from bitbake contains the v4 patches,
> > now. Do you still have the issue?
> > >
> >
> > --
> > Stefan Koch
> > Siemens AG
> > http://www.siemens.com/
>
>
> Hi Stefan,
>
> Yes, it does look like Richard had the same issue on his branch:
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/54/builds/1045/steps/14/logs/stdio
>
> I will try to provide you with an easy way to reproduce this failure.

I successfully reproduced this on uubuntu 20.04 with the following
commands:

git clone git://git.yoctoproject.org/poky-ci-archive -b autobuilder.yoctoproject.org/valkyrie/a-full-1069
. poky-ci-archive/oe-init-build-env
echo "SANITY_TESTED_DISTROS = ''" >> conf/local.conf
oe-selftest -r archiver.Archiver.test_archiver_mode_mirror_gitsm_shallow

WARNING: git-submodule-test-1.0-r0 do_fetch: Creating fast initial shallow clone failed, try initial regular clone now.
WARNING: git-submodule-test-1.0-r0 do_fetch: Failed to fetch URL gitsm://git.openembedded.org/bitbake;protocol=git;name=bitbake;subpath=bitbake;nobranch=1;lfs=False;bareclone=1;nobranch=1, attempting MIRRORS if available
ERROR: git-submodule-test-1.0-r0 do_fetch: Fetcher failure: Fetch command export PSEUDO_DISABLED=1; export PATH="/home/mdubois-briand/swat/fetch2-issue/build-st/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/mdubois-briand/swat/fetch2-issue/poky-ci-archive/scripts:/home/mdubois-briand/swat/fetch2-issue/build-st/tmp/work/core2-64-poky-linux/git-submodule-test/1.0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/home/mdubois-briand/swat/fetch2-issue/build-st/tmp/work/core2-64-poky-linux/git-submodule-test/1.0/recipe-sysroot/usr/bin/crossscripts:/home/mdubois-briand/swat/fetch2-issue/build-st/tmp/work/core2-64-poky-linux/git-submodule-test/1.0/recipe-sysroot-native/usr/sbin:/home/mdubois-briand/swat/fetch2-issue/build-st/tmp/work/core2-64-poky-linux/git-submodule-test/1.0/recipe-sysroot-native/usr/bin:/home/mdubois-briand/swat/fetch2-issue/build-st/tmp/work/core2-64-poky-linux/git-submodule-test/1.0/recipe-sysroot-native/sbin:/home/mdubois-briand/swat/fetch2-issue/build-st/tmp/work/core2-64-poky-linux/git-submodule-test/1.0/recipe-sysroot-native/bin:/home/mdubois-briand/swat/fetch2-issue/poky-ci-archive/bitbake/bin:/home/mdubois-briand/swat/fetch2-issue/build-st/tmp/hosttools"; export HOME="/home/build"; git -c gc.autoDetach=false -c core.pager=cat -c safe.bareRepository=all -c clone.defaultRemoteName=origin fetch origin 52a144a7daa94b2bd239d582cb71d1f03119918f --depth 1 failed with exit code 1, output:
error: Server does not allow request for unadvertised object 52a144a7daa94b2bd239d582cb71d1f03119918f

ERROR: git-submodule-test-1.0-r0 do_fetch: gitsm: submodule download failed: FetchError Fetcher failure for URL: 'gitsm://git.openembedded.org/bitbake;protocol=git;name=bitbake;subpath=bitbake;nobranch=1;lfs=False;bareclone=1;nobranch=1'. Unable to fetch URL from any source.
WARNING: git-submodule-test-1.0-r0 do_fetch: Failed to fetch URL gitsm://git.yoctoproject.org/bitbake-gitsm-test1;protocol=git;name=bitbake-gitsm-test1;subpath=bitbake-gitsm-test1;nobranch=1;lfs=True;bareclone=1;nobranch=1, attempting MIRRORS if available
NOTE: recipe git-submodule-test-1.0-r0: task do_fetch: Succeeded
NOTE: Running task 3 of 4 (/home/mdubois-briand/swat/fetch2-issue/build-st/meta-selftest/recipes-test/git-submodule-test/git-submodule-test.bb:do_ar_mirror)
NOTE: recipe git-submodule-test-1.0-r0: task do_ar_mirror: Started
WARNING: git-submodule-test-1.0-r0 do_ar_mirror: Mirror tarballs are listed for a source but none are present. Falling back to original download.
SRC_URI = gitsm://git.openembedded.org/bitbake;protocol=git;name=bitbake;subpath=bitbake;nobranch=1;lfs=False;bareclone=1;nobranch=1
WARNING: git-submodule-test-1.0-r0 do_ar_mirror: Mirror tarballs are listed for a source but none are present. Falling back to original download.
SRC_URI = gitsm://git.yoctoproject.org/bitbake-gitsm-test2;protocol=git;name=bitbake-gitsm-test2;subpath=bitbake-gitsm-test2;nobranch=1;lfs=False;bareclone=1;nobranch=1
ERROR: git-submodule-test-1.0-r0 do_ar_mirror: Original download is missing for a source.
SRC_URI = gitsm://git.yoctoproject.org/bitbake-gitsm-test2;protocol=git;name=bitbake-gitsm-test2;subpath=bitbake-gitsm-test2;nobranch=1;lfs=False;bareclone=1;nobranch=1
ERROR: Logfile of failure stored in: /home/mdubois-briand/swat/fetch2-issue/build-st/tmp/work/core2-64-poky-linux/git-submodule-test/1.0/temp/log.do_ar_mirror.982
NOTE: recipe git-submodule-test-1.0-r0: task do_ar_mirror: Failed
ERROR: Task (/home/mdubois-briand/swat/fetch2-issue/build-st/meta-selftest/recipes-test/git-submodule-test/git-submodule-test.bb:do_ar_mirror) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3 tasks of which 0 didn't need to be rerun and 1 failed.


-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com