[Buildroot] [PATCH] utils/genrandconfig: always use BR2_BACKUP_SITE in the autobuilders

[Buildroot] [PATCH] utils/genrandconfig: always use BR2_BACKUP_SITE in the autobuilders

[Thomas Petazzoni via buildroot]

In commit da5c25c9f91b17a3c00ff0b35164881f2d1aa425, we decided to
generate 10% of configurations without BR2_BACKUP_SITE to test
upstream locations and see if they fail.

In recent times, we have seen a significant increase of temporary
failures of upstream locations, which cause build failures for which
there is not much we can do. They cause noise in the build results and
consume autobuilders CPU time for builds that don't lead to any useful
conclusion.

So in the end let's revert back to using BR2_BACKUP_SITE in all
configurations to avoid those build failures. If we want to detect
really stale upstream locations, as suggested by Arnout, we can
instrument the autobuilder runs to inject a special HTTP header and
then look at the logs of sources.buildroot.net to see which files are
grabbed by the autobuilders and determine if upstream locations are
really dead and packages need to be fixed accordingly.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
---
Note: this is a proposal. If accepted, it should be merged into all
branches currently tested by autobuild.buildroot.org.
---
 utils/genrandconfig | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/utils/genrandconfig b/utils/genrandconfig
index 727c8193c4..fad3a20786 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -556,11 +556,6 @@ async def gen_config(args):
         configlines.append("BR2_REPRODUCIBLE=y\n")
         configlines.append("BR2_TARGET_ROOTFS_TAR=y\n")
 
-    # From time to time, ignore sources.buildroot.net to really fetch
-    # from upstream
-    if randint(0, 1) == 0:
-        configlines.append("""BR2_BACKUP_SITE=""\n""")
-
     with open(configfile, "a") as configf:
         configf.writelines(configlines)
 
-- 
2.55.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] utils/genrandconfig: always use BR2_BACKUP_SITE in the autobuilders

[Thomas Perale via buildroot]

Hello Thomas,

In reply of:
> So in the end let's revert back to using BR2_BACKUP_SITE in all
> configurations to avoid those build failures. If we want to detect
> really stale upstream locations, as suggested by Arnout, we can
> instrument the autobuilder runs to inject a special HTTP header and
> then look at the logs of sources.buildroot.net to see which files are
> grabbed by the autobuilders and determine if upstream locations are
> really dead and packages need to be fixed accordingly.

How and how often are the 'source.buildroot.net' backups generated right now ?
By reading this paragraph I'm thinking the upstream location check could be
coupled to the generation of the backups instead.

Otherwise I think this change make sense.

PERALE Thomas
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] utils/genrandconfig: always use BR2_BACKUP_SITE in the autobuilders

[Thomas Petazzoni via buildroot]

On Thu, Aug 27, 2026 at 05:29:22PM +0200, Thomas Perale wrote:

> How and how often are the 'source.buildroot.net' backups generated right now ?

Pretty much constantly:

3 */2 * * * flock -n /home/brmirror/br-mirror.lock /home/brmirror/bin/update-br-mirror.sh >/dev/null 2>&1 || true

> By reading this paragraph I'm thinking the upstream location check could be
> coupled to the generation of the backups instead.

I'm not sure to understand what you mean. The "upstream location
check" is done by the build machines, which are completely distinct
from the machine hosting sources.b.o and doing the mirroring.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] utils/genrandconfig: always use BR2_BACKUP_SITE in the autobuilders

[Thomas Perale via buildroot]

In reply of:
> > How and how often are the 'source.buildroot.net' backups generated right now ?
> 
> Pretty much constantly:
> 
> 3 */2 * * * flock -n /home/brmirror/br-mirror.lock /home/brmirror/bin/update-br-mirror.sh >/dev/null 2>&1 || true

Is the update-br-mirror.sh script available somewhere ?

> > By reading this paragraph I'm thinking the upstream location check could be
> > coupled to the generation of the backups instead.
> 
> I'm not sure to understand what you mean. The "upstream location
> check" is done by the build machines, which are completely distinct
> from the machine hosting sources.b.o and doing the mirroring.

Yes that's what I meant. Since doing this verification by the autobuilders is
an issue you could move the verification and publishing of the stale entries
from the machine hosting the sources. 

Since it already has the responsability to download all the entries, it seems
like a perfect candidate to detect stale locations.
Unless 'update-br-mirror' script only trigger a the download if a new location
is detected ?

What I'm afraid of with the HTTP header solution proposed is that it's gonna be
burried in logs that would be quickly forgotten about. The current solution
while generating a lot of false positive manage to catch some real stale
locations publicly.

PERALE Thomas
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH] utils/genrandconfig: always use BR2_BACKUP_SITE in the autobuilders

[Thomas Petazzoni via buildroot]

On Thu, Aug 27, 2026 at 07:33:28PM +0200, Thomas Perale wrote:

> Yes that's what I meant. Since doing this verification by the autobuilders is
> an issue you could move the verification and publishing of the stale entries
> from the machine hosting the sources. 
> 
> Since it already has the responsability to download all the entries, it seems
> like a perfect candidate to detect stale locations.
> Unless 'update-br-mirror' script only trigger a the download if a new location
> is detected ?

It does "make source" so it only triggers a download if the file is
not already in the cache. And typically dead upstream servers are used
to host dead projects, which don't see releases very often, and
therefore we likely already have the tarball in our cache.

> What I'm afraid of with the HTTP header solution proposed is that it's gonna be
> burried in logs that would be quickly forgotten about. The current solution
> while generating a lot of false positive manage to catch some real stale
> locations publicly.

True, but it's also generating a lot of noise.

To be honest, I am not 100% sure my proposal is the best solution, but
if we're aiming to reduce the number of build failures, dropping false
positives is important.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot