[PATCH] mirrors: Add uninative mirror on kernel.org

[PATCH] mirrors: Add uninative mirror on kernel.org

[Richard Purdie]

At the last nas outage, we realised that we don't have good mirrors of the
uninative tarball if our main system can't be accessed. kernel.org mirrors
some Yocto Project data so we've ensured uninative is there. Add the appropriate
mirror url to make use of that.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/mirrors.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/mirrors.bbclass b/meta/classes/mirrors.bbclass
index ba325a658bc..75eb86a7243 100644
--- a/meta/classes/mirrors.bbclass
+++ b/meta/classes/mirrors.bbclass
@@ -62,6 +62,7 @@ ftp://.*/.*     http://sources.openembedded.org/ \n \
 npm://.*/?.*    http://sources.openembedded.org/ \n \
 ${CPAN_MIRROR}  http://cpan.metacpan.org/ \n \
 ${CPAN_MIRROR}  http://search.cpan.org/CPAN/ \n \
+https?$://downloads.yoctoproject.org/releases/uninative/ https://mirrors.kernel.org/yocto/uninative/ \n \
 "
 
 # Use MIRRORS to provide git repo fallbacks using the https protocol, for cases
-- 
2.32.0

RE: [OE-core] [PATCH] mirrors: Add uninative mirror on kernel.org

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-
> core@lists.openembedded.org> On Behalf Of Richard Purdie
> Sent: den 9 november 2021 14:22
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH] mirrors: Add uninative mirror on kernel.org
> 
> At the last nas outage, we realised that we don't have good mirrors of the
> uninative tarball if our main system can't be accessed. kernel.org mirrors
> some Yocto Project data so we've ensured uninative is there. Add the appropriate
> mirror url to make use of that.
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/classes/mirrors.bbclass | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/classes/mirrors.bbclass b/meta/classes/mirrors.bbclass
> index ba325a658bc..75eb86a7243 100644
> --- a/meta/classes/mirrors.bbclass
> +++ b/meta/classes/mirrors.bbclass
> @@ -62,6 +62,7 @@ ftp://.*/.*     http://sources.openembedded.org/ \n \
>  npm://.*/?.*    http://sources.openembedded.org/ \n \
>  ${CPAN_MIRROR}  http://cpan.metacpan.org/ \n \
>  ${CPAN_MIRROR}  http://search.cpan.org/CPAN/ \n \
> +https?$://downloads.yoctoproject.org/releases/uninative/ https://mirrors.kernel.org/yocto/uninative/ \n \

The above looked very odd to me, having that dollar sign in the middle of 
the regular expression. It wasn't until I was deep into the fetcher's code 
that I realized that it is not actually one regular expression above, but 
three, formatted as a URL...

However, based on the code in bitbake/lib/bb/fetch2/__init__.py:uri_replace(), 
AFAICT, the dollar sign should not be needed above, as it will be added for 
the URI type regexp automatically. The same goes for the other two URLs in 
mirrors.bbclass that use that formatting.

While investigating this, I also tried to look in the bitbake documentation 
for a description of the format used for the PREMIRRORS and MIRRORS variables, 
but I could not find any. There is only a simple example, without any 
explanation for the actual format that is being used.

>  "
> 
>  # Use MIRRORS to provide git repo fallbacks using the https protocol, for cases
> --
> 2.32.0

//Peter

Re: [OE-core] [PATCH] mirrors: Add uninative mirror on kernel.org

[Richard Purdie]

On Wed, 2021-11-10 at 20:44 +0000, Peter Kjellerstedt wrote:
> > -----Original Message-----
> > From: openembedded-core@lists.openembedded.org <openembedded-
> > core@lists.openembedded.org> On Behalf Of Richard Purdie
> > Sent: den 9 november 2021 14:22
> > To: openembedded-core@lists.openembedded.org
> > Subject: [OE-core] [PATCH] mirrors: Add uninative mirror on kernel.org
> > 
> > At the last nas outage, we realised that we don't have good mirrors of the
> > uninative tarball if our main system can't be accessed. kernel.org mirrors
> > some Yocto Project data so we've ensured uninative is there. Add the
> > appropriate
> > mirror url to make use of that.
> > 
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> >  meta/classes/mirrors.bbclass | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/meta/classes/mirrors.bbclass b/meta/classes/mirrors.bbclass
> > index ba325a658bc..75eb86a7243 100644
> > --- a/meta/classes/mirrors.bbclass
> > +++ b/meta/classes/mirrors.bbclass
> > @@ -62,6 +62,7 @@ ftp://.*/.*     http://sources.openembedded.org/ \n \
> >  npm://.*/?.*    http://sources.openembedded.org/ \n \
> >  ${CPAN_MIRROR}  http://cpan.metacpan.org/ \n \
> >  ${CPAN_MIRROR}  http://search.cpan.org/CPAN/ \n \
> > +https?$://downloads.yoctoproject.org/releases/uninative/
> > https://mirrors.kernel.org/yocto/uninative/ \n \
> 
> The above looked very odd to me, having that dollar sign in the middle of 
> the regular expression. It wasn't until I was deep into the fetcher's code 
> that I realized that it is not actually one regular expression above, but 
> three, formatted as a URL...

It is all a bit crazy. This format is from the days of the fetcher before my
involvement and before it was the fetcher.

> However, based on the code in bitbake/lib/bb/fetch2/__init__.py:uri_replace(),
> AFAICT, the dollar sign should not be needed above, as it will be added for 
> the URI type regexp automatically. The same goes for the other two URLs in 
> mirrors.bbclass that use that formatting.

Right, I think I remember added the anchoring fix. We haven't cleaned up all the
urls out in the wild and I was being consistent with the other entries in this
case.

> While investigating this, I also tried to look in the bitbake documentation 
> for a description of the format used for the PREMIRRORS and MIRRORS variables,
> but I could not find any. There is only a simple example, without any 
> explanation for the actual format that is being used.

That doesn't surprise me, I still have trouble remembering all the
idiosyncrasies when trying to fix the mirror url code too. It would be good to
document it if anyone fancies trying.

Cheers,

Richard