[Buildroot] Inquiry About Mirroring Methods for Buildroot Packages

[Buildroot] Inquiry About Mirroring Methods for Buildroot Packages

[Yuan Tan]

Hi,

I am reaching out to inquire about the best practices for mirroring
Buildroot packages. We aim to establish a reliable mirror for these
packages, particularly to facilitate access for users in regions like
mainland China, where downloading packages can sometimes be challenging.

I have two suggestions and would like to know if either is possible:

1. Could you enable rsync for sources.buildroot.net? This would make it
easier for mirror sites to stay updated.

2. If rsync isn't an option, would it be okay to use wget to clone the
entire sources.buildroot.net?

I would also like to inquire if there are any licensing or authorization
considerations we should be aware of when mirroring Buildroot.

Thank you for taking the time to consider these options. I greatly
appreciate any guidance or suggestions you can provide.

Best regards,

Yuan

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Inquiry About Mirroring Methods for Buildroot Packages

[Arnout Vandecappelle via buildroot]

On 11/12/2023 10:56, Yuan Tan wrote:
> Hi,
> 
> I am reaching out to inquire about the best practices for mirroring
> Buildroot packages. We aim to establish a reliable mirror for these
> packages, particularly to facilitate access for users in regions like
> mainland China, where downloading packages can sometimes be challenging.

  We use cloudflare CDN. I just read up on it and it seems there's a separate 
CDN that needs to be enabled to serve China. Perhaps that just needs to be enabled?
Peter?

> I have two suggestions and would like to know if either is possible:
> 
> 1. Could you enable rsync for sources.buildroot.net? This would make it
> easier for mirror sites to stay updated.
> 
> 2. If rsync isn't an option, would it be okay to use wget to clone the
> entire sources.buildroot.net?

  Wouldn't it be easier to construct your own version, by running the same 
script that Peter uses to update actual sources.buildroot.net? Though IIUC that 
script is a bit of a hack.

  Regards,
  Arnout

> I would also like to inquire if there are any licensing or authorization
> considerations we should be aware of when mirroring Buildroot.
> 
> Thank you for taking the time to consider these options. I greatly
> appreciate any guidance or suggestions you can provide.
> 
> Best regards,
> 
> Yuan
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Inquiry About Mirroring Methods for Buildroot Packages

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

 > On 11/12/2023 10:56, Yuan Tan wrote:
 >> Hi,
 >> I am reaching out to inquire about the best practices for mirroring
 >> Buildroot packages. We aim to establish a reliable mirror for these
 >> packages, particularly to facilitate access for users in regions like
 >> mainland China, where downloading packages can sometimes be challenging.

 >  We use cloudflare CDN. I just read up on it and it seems there's a
 >  separate CDN that needs to be enabled to serve China. Perhaps that
 > just needs to be enabled?
 > Peter?

Hmm, I guess you are referring to
https://www.cloudflare.com/application-services/products/china-network/?

If so, that doesn't look as if that is just something you can enable for
their free offering, E.G. you apparently need a "ICP license":

What is an ICP license

ICP is the short name for "Internet Content Provider". According to
Chinese regulations, all websites hosted in Mainland China or content
served from Chinese data centers (which is true for our China Network
service) needs to obtain an ICP "BeiAn备案" or "License 证" number for
every domain that serves content to your pages. This number needs to be
displayed on the footer of your pages.

 >> I have two suggestions and would like to know if either is possible:
 >> 1. Could you enable rsync for sources.buildroot.net? This would make
 >> it
 >> easier for mirror sites to stay updated.
 >> 2. If rsync isn't an option, would it be okay to use wget to clone
 >> the
 >> entire sources.buildroot.net?

 >  Wouldn't it be easier to construct your own version, by running the
 >  same script that Peter uses to update actual sources.buildroot.net?
 >  Though IIUC that script is a bit of a hack.

yes, it is a big hack. It basically runs make allyespackageconfig &&
make source, but then does a number of mutations as well to try to catch
per-architecture packages or packages with multiple versions.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Inquiry About Mirroring Methods for Buildroot Packages

[Yuan Tan]

Hi,
> 
>  > On 11/12/2023 10:56, Yuan Tan wrote:
>  >> Hi,
>  >> I am reaching out to inquire about the best practices for mirroring
>  >> Buildroot packages. We aim to establish a reliable mirror for these
>  >> packages, particularly to facilitate access for users in regions like
>  >> mainland China, where downloading packages can sometimes be challenging.
> 
>  >  We use cloudflare CDN. I just read up on it and it seems there's a
>  >  separate CDN that needs to be enabled to serve China. Perhaps that
>  > just needs to be enabled?
>  > Peter?
> 
> Hmm, I guess you are referring to
> https://www.cloudflare.com/application-services/products/china-network/?
> 
> If so, that doesn't look as if that is just something you can enable for
> their free offering, E.G. you apparently need a "ICP license":
> 
> What is an ICP license
> 
> ICP is the short name for "Internet Content Provider". According to
> Chinese regulations, all websites hosted in Mainland China or content
> served from Chinese data centers (which is true for our China Network
> service) needs to obtain an ICP "BeiAn=E5=A4=87=E6=A1=88" or "License =E8=
> =AF=81" number for
> every domain that serves content to your pages. This number needs to be
> displayed on the footer of your pages.
> 

Yes, providing internet services in China is quite troublesome :). Some
universities have set up some mirror sites, and I am trying to set up one
for buildroot as well.

>  >> I have two suggestions and would like to know if either is possible:
>  >> 1. Could you enable rsync for sources.buildroot.net? This would make
>  >> it
>  >> easier for mirror sites to stay updated.
>  >> 2. If rsync isn't an option, would it be okay to use wget to clone
>  >> the
>  >> entire sources.buildroot.net?
> 
>  >  Wouldn't it be easier to construct your own version, by running the
>  >  same script that Peter uses to update actual sources.buildroot.net?
>  >  Though IIUC that script is a bit of a hack.
> 
> yes, it is a big hack. It basically runs make allyespackageconfig &&
> make source, but then does a number of mutations as well to try to catch
> per-architecture packages or packages with multiple versions.

May I ask if this script can be shared?

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] Inquiry About Mirroring Methods for Buildroot Packages

[Yuan Tan]

Hi,

It seems that my last email was ignored, I am resending it now, and I hope
it does not disturb anyone.

> 
>  > On 11/12/2023 10:56, Yuan Tan wrote:
>  >> Hi,
>  >> I am reaching out to inquire about the best practices for mirroring
>  >> Buildroot packages. We aim to establish a reliable mirror for these
>  >> packages, particularly to facilitate access for users in regions like
>  >> mainland China, where downloading packages can sometimes be challenging.
> 
>  >  We use cloudflare CDN. I just read up on it and it seems there's a
>  >  separate CDN that needs to be enabled to serve China. Perhaps that
>  > just needs to be enabled?
>  > Peter?
> 
> Hmm, I guess you are referring to
> https://www.cloudflare.com/application-services/products/china-network/?
> 
> If so, that doesn't look as if that is just something you can enable for
> their free offering, E.G. you apparently need a "ICP license":
> 
> What is an ICP license
> 
> ICP is the short name for "Internet Content Provider". According to
> Chinese regulations, all websites hosted in Mainland China or content
> served from Chinese data centers (which is true for our China Network
> service) needs to obtain an ICP "BeiAn=E5=A4=87=E6=A1=88" or "License =E8=
> =AF=81" number for
> every domain that serves content to your pages. This number needs to be
> displayed on the footer of your pages.
> 

Yes, providing internet services in China is quite troublesome :). Some
universities have set up some mirror sites, and I am trying to set up one
for buildroot as well.

>  >> I have two suggestions and would like to know if either is possible:
>  >> 1. Could you enable rsync for sources.buildroot.net? This would make
>  >> it
>  >> easier for mirror sites to stay updated.
>  >> 2. If rsync isn't an option, would it be okay to use wget to clone
>  >> the
>  >> entire sources.buildroot.net?
> 
>  >  Wouldn't it be easier to construct your own version, by running the
>  >  same script that Peter uses to update actual sources.buildroot.net?
>  >  Though IIUC that script is a bit of a hack.
> 
> yes, it is a big hack. It basically runs make allyespackageconfig &&
> make source, but then does a number of mutations as well to try to catch
> per-architecture packages or packages with multiple versions.

May I ask if this script can be shared?
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot