[PATCH] musl: Drop MIRRORS

[PATCH] musl: Drop MIRRORS

[Khem Raj]

When using mirrors fetcher gets into a weird state when shallow clones
are enables.

ERROR: Bitbake Fetcher Error: UnpackError('No up to date source found: clone directory not available or not up to date: /mnt/b/yoe/master/downloads/git2/git.musl-libc.org.git.musl.git; shallow clone not available: /mnt/b/yoe/master/downloads/gitshallow_git.musl-libc.org.git.musl.git_79bdacf-1_master.tar.gz', 'git://git.musl-libc.org/git/musl.git;branch=master;protocol=https')

Anyway the mirrors were added as backup when we had issues with musl git
in past.

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/recipes-core/musl/musl_git.bb | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/meta/recipes-core/musl/musl_git.bb b/meta/recipes-core/musl/musl_git.bb
index dbc31e1d83b..72253636dc8 100644
--- a/meta/recipes-core/musl/musl_git.bb
+++ b/meta/recipes-core/musl/musl_git.bb
@@ -10,15 +10,11 @@ BASEVER = "1.2.4"
 
 PV = "${BASEVER}+git"
 
-# mirror is at git://github.com/kraj/musl.git
-
-SRC_URI = "git://git.musl-libc.org/git/musl.git;branch=master;protocol=https \
+SRC_URI = "git://git.musl-libc.org/git/musl;branch=master;protocol=https \
            file://0001-Make-dynamic-linker-a-relative-symlink-to-libc.patch \
            file://0002-ldso-Use-syslibdir-and-libdir-as-default-pathes-to-l.patch \
           "
 
-MIRRORS += "git://git.musl-libc.org/git/musl.git git://github.com/kraj/musl.git"
-
 S = "${WORKDIR}/git"
 
 PROVIDES += "virtual/libc virtual/libiconv virtual/libintl virtual/crypt"
-- 
2.42.0

Re: [OE-core] [PATCH] musl: Drop MIRRORS

[Richard Purdie]

On Thu, 2023-09-21 at 18:15 -0700, Khem Raj wrote:
> When using mirrors fetcher gets into a weird state when shallow clones
> are enables.
> 
> ERROR: Bitbake Fetcher Error: UnpackError('No up to date source
> found: clone directory not available or not up to date:
> /mnt/b/yoe/master/downloads/git2/git.musl-libc.org.git.musl.git;
> shallow clone not available:
> /mnt/b/yoe/master/downloads/gitshallow_git.musl-
> libc.org.git.musl.git_79bdacf-1_master.tar.gz', 'git://git.musl-
> libc.org/git/musl.git;branch=master;protocol=https')

That sounds like something we should look into and fix?

I don't think it makes sense as a justification in a commit message.

Cheers,

Richard

Re: [OE-core] [PATCH] musl: Drop MIRRORS

[Khem Raj]

On Fri, Sep 22, 2023 at 3:11 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Thu, 2023-09-21 at 18:15 -0700, Khem Raj wrote:
> > When using mirrors fetcher gets into a weird state when shallow clones
> > are enables.
> >
> > ERROR: Bitbake Fetcher Error: UnpackError('No up to date source
> > found: clone directory not available or not up to date:
> > /mnt/b/yoe/master/downloads/git2/git.musl-libc.org.git.musl.git;
> > shallow clone not available:
> > /mnt/b/yoe/master/downloads/gitshallow_git.musl-
> > libc.org.git.musl.git_79bdacf-1_master.tar.gz', 'git://git.musl-
> > libc.org/git/musl.git;branch=master;protocol=https')
>
> That sounds like something we should look into and fix?
>
> I don't think it makes sense as a justification in a commit message.

I tried to do it in several ways and somehow fetch is failing. Only
way I see it work once was when removing the mirrors and rebuilding
after -ccleanall
see - https://autobuilder.yoctoproject.org/typhoon/#/builders/64/builds/7836

if I git fetch these repos from any of mirrors or upstream clones work
totally fine but git fetcher always complains. No idea whats going on
>
> Cheers,
>
> Richard

Re: [OE-core] [PATCH] musl: Drop MIRRORS

[Richard Purdie]

On Fri, 2023-09-22 at 08:10 -0700, Khem Raj wrote:
> On Fri, Sep 22, 2023 at 3:11 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > 
> > On Thu, 2023-09-21 at 18:15 -0700, Khem Raj wrote:
> > > When using mirrors fetcher gets into a weird state when shallow clones
> > > are enables.
> > > 
> > > ERROR: Bitbake Fetcher Error: UnpackError('No up to date source
> > > found: clone directory not available or not up to date:
> > > /mnt/b/yoe/master/downloads/git2/git.musl-libc.org.git.musl.git;
> > > shallow clone not available:
> > > /mnt/b/yoe/master/downloads/gitshallow_git.musl-
> > > libc.org.git.musl.git_79bdacf-1_master.tar.gz', 'git://git.musl-
> > > libc.org/git/musl.git;branch=master;protocol=https')
> > 
> > That sounds like something we should look into and fix?
> > 
> > I don't think it makes sense as a justification in a commit message.
> 
> I tried to do it in several ways and somehow fetch is failing. Only
> way I see it work once was when removing the mirrors and rebuilding
> after -ccleanall
> see - https://autobuilder.yoctoproject.org/typhoon/#/builders/64/builds/7836
> 
> if I git fetch these repos from any of mirrors or upstream clones work
> totally fine but git fetcher always complains. No idea whats going on
> 

Please tell me you're not running -c cleanall on the autobuilder
infrastructure?

Cheers,

Richard

Re: [OE-core] [PATCH] musl: Drop MIRRORS

[Khem Raj]

[-- Attachment #1.1.1: Type: text/plain, Size: 1476 bytes --]

On 9/22/23 8:16 AM, Richard Purdie wrote:
> On Fri, 2023-09-22 at 08:10 -0700, Khem Raj wrote:
>> On Fri, Sep 22, 2023 at 3:11 AM Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>>>
>>> On Thu, 2023-09-21 at 18:15 -0700, Khem Raj wrote:
>>>> When using mirrors fetcher gets into a weird state when shallow clones
>>>> are enables.
>>>>
>>>> ERROR: Bitbake Fetcher Error: UnpackError('No up to date source
>>>> found: clone directory not available or not up to date:
>>>> /mnt/b/yoe/master/downloads/git2/git.musl-libc.org.git.musl.git;
>>>> shallow clone not available:
>>>> /mnt/b/yoe/master/downloads/gitshallow_git.musl-
>>>> libc.org.git.musl.git_79bdacf-1_master.tar.gz', 'git://git.musl-
>>>> libc.org/git/musl.git;branch=master;protocol=https')
>>>
>>> That sounds like something we should look into and fix?
>>>
>>> I don't think it makes sense as a justification in a commit message.
>>
>> I tried to do it in several ways and somehow fetch is failing. Only
>> way I see it work once was when removing the mirrors and rebuilding
>> after -ccleanall
>> see - https://autobuilder.yoctoproject.org/typhoon/#/builders/64/builds/7836
>>
>> if I git fetch these repos from any of mirrors or upstream clones work
>> totally fine but git fetcher always complains. No idea whats going on
>>
> 
> Please tell me you're not running -c cleanall on the autobuilder
> infrastructure?

ofcourse not.

> 
> Cheers,
> 
> Richard

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 2613 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 203 bytes --]

Re: [OE-core] [PATCH] musl: Drop MIRRORS

[Alexandre Belloni]

https://autobuilder.yoctoproject.org/typhoon/#/builders/15/builds/8149/steps/11/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/64/builds/7837/steps/11/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/7859/steps/11/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/97/builds/7193/steps/13/logs/stdio

On 21/09/2023 18:15:20-0700, Khem Raj wrote:
> When using mirrors fetcher gets into a weird state when shallow clones
> are enables.
> 
> ERROR: Bitbake Fetcher Error: UnpackError('No up to date source found: clone directory not available or not up to date: /mnt/b/yoe/master/downloads/git2/git.musl-libc.org.git.musl.git; shallow clone not available: /mnt/b/yoe/master/downloads/gitshallow_git.musl-libc.org.git.musl.git_79bdacf-1_master.tar.gz', 'git://git.musl-libc.org/git/musl.git;branch=master;protocol=https')
> 
> Anyway the mirrors were added as backup when we had issues with musl git
> in past.
> 
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---
>  meta/recipes-core/musl/musl_git.bb | 6 +-----
>  1 file changed, 1 insertion(+), 5 deletions(-)
> 
> diff --git a/meta/recipes-core/musl/musl_git.bb b/meta/recipes-core/musl/musl_git.bb
> index dbc31e1d83b..72253636dc8 100644
> --- a/meta/recipes-core/musl/musl_git.bb
> +++ b/meta/recipes-core/musl/musl_git.bb
> @@ -10,15 +10,11 @@ BASEVER = "1.2.4"
>  
>  PV = "${BASEVER}+git"
>  
> -# mirror is at git://github.com/kraj/musl.git
> -
> -SRC_URI = "git://git.musl-libc.org/git/musl.git;branch=master;protocol=https \
> +SRC_URI = "git://git.musl-libc.org/git/musl;branch=master;protocol=https \
>             file://0001-Make-dynamic-linker-a-relative-symlink-to-libc.patch \
>             file://0002-ldso-Use-syslibdir-and-libdir-as-default-pathes-to-l.patch \
>            "
>  
> -MIRRORS += "git://git.musl-libc.org/git/musl.git git://github.com/kraj/musl.git"
> -
>  S = "${WORKDIR}/git"
>  
>  PROVIDES += "virtual/libc virtual/libiconv virtual/libintl virtual/crypt"
> -- 
> 2.42.0
> 

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#188034): https://lists.openembedded.org/g/openembedded-core/message/188034
> Mute This Topic: https://lists.openembedded.org/mt/101513420/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH] musl: Drop MIRRORS

[Khem Raj]

yeah I am trying to investigate, locally I reproduced it once but then
I did bitbake -ccleanall musl and then it started to work.

I see this in log.do_fetch on AB

http://sprunge.us/c5J8K3

says

fatal: protocol error: bad line length character: erro


On Fri, Sep 22, 2023 at 9:07 AM Alexandre Belloni
<alexandre.belloni@bootlin.com> wrote:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/15/builds/8149/steps/11/logs/stdio
> https://autobuilder.yoctoproject.org/typhoon/#/builders/64/builds/7837/steps/11/logs/stdio
> https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/7859/steps/11/logs/stdio
> https://autobuilder.yoctoproject.org/typhoon/#/builders/97/builds/7193/steps/13/logs/stdio
>
> On 21/09/2023 18:15:20-0700, Khem Raj wrote:
> > When using mirrors fetcher gets into a weird state when shallow clones
> > are enables.
> >
> > ERROR: Bitbake Fetcher Error: UnpackError('No up to date source found: clone directory not available or not up to date: /mnt/b/yoe/master/downloads/git2/git.musl-libc.org.git.musl.git; shallow clone not available: /mnt/b/yoe/master/downloads/gitshallow_git.musl-libc.org.git.musl.git_79bdacf-1_master.tar.gz', 'git://git.musl-libc.org/git/musl.git;branch=master;protocol=https')
> >
> > Anyway the mirrors were added as backup when we had issues with musl git
> > in past.
> >
> > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > ---
> >  meta/recipes-core/musl/musl_git.bb | 6 +-----
> >  1 file changed, 1 insertion(+), 5 deletions(-)
> >
> > diff --git a/meta/recipes-core/musl/musl_git.bb b/meta/recipes-core/musl/musl_git.bb
> > index dbc31e1d83b..72253636dc8 100644
> > --- a/meta/recipes-core/musl/musl_git.bb
> > +++ b/meta/recipes-core/musl/musl_git.bb
> > @@ -10,15 +10,11 @@ BASEVER = "1.2.4"
> >
> >  PV = "${BASEVER}+git"
> >
> > -# mirror is at git://github.com/kraj/musl.git
> > -
> > -SRC_URI = "git://git.musl-libc.org/git/musl.git;branch=master;protocol=https \
> > +SRC_URI = "git://git.musl-libc.org/git/musl;branch=master;protocol=https \
> >             file://0001-Make-dynamic-linker-a-relative-symlink-to-libc.patch \
> >             file://0002-ldso-Use-syslibdir-and-libdir-as-default-pathes-to-l.patch \
> >            "
> >
> > -MIRRORS += "git://git.musl-libc.org/git/musl.git git://github.com/kraj/musl.git"
> > -
> >  S = "${WORKDIR}/git"
> >
> >  PROVIDES += "virtual/libc virtual/libiconv virtual/libintl virtual/crypt"
> > --
> > 2.42.0
> >
>
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#188034): https://lists.openembedded.org/g/openembedded-core/message/188034
> > Mute This Topic: https://lists.openembedded.org/mt/101513420/3617179
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>
>
> --
> Alexandre Belloni, co-owner and COO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com