* [Buildroot] [PATCH v7 01/23] utils/genrandconfig: new script
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 02/23] genrandconfig: use subprocess.check_output instead of Popen Arnout Vandecappelle
` (23 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
This script will be used by the autobuild-run script to generate the
configuration to test. It is put in the utils directory because it can
also be called directly to allow users to test things.
For now, it is a direct copy of the relevant functions from the
autobuild-run script. The only changes are:
- unneeded import statements are removed;
- code/decode wrappers are limited to decode_byte_list;
- __main__ handling is added.
For now, the only supported arguments are the ones needed for
autobuild-run. Follow-up patches will refactor things and also change
the way the script is called. In this version, it can be called from the
autobuild-run script as:
subprocess.call([os.path.join(srcdir, "utils/genrandconfig"),
"-i", str(kwargs['instance']),
"--toolchains-url", kwargs['toolchains_url']],
stdout=log, stderr=log)
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
I elected to go for argparse rather than docopt because it is much
more standard and I find it easier to work with.
---
v7: new patch
---
utils/genrandconfig | 445 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 445 insertions(+)
create mode 100755 utils/genrandconfig
diff --git a/utils/genrandconfig b/utils/genrandconfig
new file mode 100755
index 0000000000..6d3269225d
--- /dev/null
+++ b/utils/genrandconfig
@@ -0,0 +1,445 @@
+#!/usr/bin/env python
+
+# Copyright (C) 2014 by Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+# This script generates a random configuration for testing Buildroot.
+
+from __future__ import print_function
+
+import contextlib
+import csv
+import os
+from random import randint
+import subprocess
+import sys
+from time import localtime, strftime
+from distutils.version import StrictVersion
+import platform
+
+if sys.hexversion >= 0x3000000:
+ import urllib.request as _urllib
+else:
+ import urllib2 as _urllib
+
+urlopen = _urllib.urlopen
+urlopen_closing = lambda uri: contextlib.closing(urlopen(uri))
+
+if sys.hexversion >= 0x3000000:
+ def decode_byte_list(bl):
+ return [b.decode() for b in bl]
+else:
+ def decode_byte_list(e):
+ return e
+
+def log_write(logf, msg):
+ logf.write("[%s] %s\n" % (strftime("%a, %d %b %Y %H:%M:%S", localtime()), msg))
+ logf.flush()
+
+class SystemInfo:
+ DEFAULT_NEEDED_PROGS = ["make", "git", "gcc", "timeout"]
+ DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar"]
+
+ def __init__(self):
+ self.needed_progs = list(self.__class__.DEFAULT_NEEDED_PROGS)
+ self.optional_progs = list(self.__class__.DEFAULT_OPTIONAL_PROGS)
+ self.progs = {}
+
+ def find_prog(self, name, flags=os.X_OK, env=os.environ):
+ if not name or name[0] == os.sep: raise ValueError(name)
+
+ prog_path = env.get("PATH", None)
+ # for windows compatibility, we'd need to take PATHEXT into account
+
+ if prog_path:
+ for prog_dir in filter(None, prog_path.split(os.pathsep)):
+ # os.join() not necessary: non-empty prog_dir
+ # and name[0] != os.sep
+ prog = prog_dir + os.sep + name
+ if os.access(prog, flags):
+ return prog
+ # --
+ return None
+
+ def has(self, prog):
+ """Checks whether a program is available.
+ Lazily evaluates missing entries.
+
+ Returns: None if prog not found, else path to the program [evaluates to True]
+ """
+ try:
+ return self.progs[prog]
+ except KeyError:
+ pass
+
+ have_it = self.find_prog(prog)
+ # java[c] needs special care
+ if have_it and prog in ('java', 'javac'):
+ with open(os.devnull, "w") as devnull:
+ if subprocess.call("%s -version | grep gcj" % prog, shell=True,
+ stdout=devnull, stderr=devnull) != 1:
+ have_it = False
+ # --
+ self.progs[prog] = have_it
+ return have_it
+
+ def check_requirements(self):
+ """Checks program dependencies.
+
+ Returns: True if all mandatory programs are present, else False.
+ """
+ do_check_has_prog = self.has
+
+ missing_requirements = False
+ for prog in self.needed_progs:
+ if not do_check_has_prog(prog):
+ print("ERROR: your system lacks the '%s' program" % prog)
+ missing_requirements = True
+
+ # check optional programs here,
+ # else they'd get checked by each worker instance
+ for prog in self.optional_progs:
+ do_check_has_prog(prog)
+
+ return not missing_requirements
+
+def get_toolchain_configs(**kwargs):
+ """Fetch and return the possible toolchain configurations
+
+ This function returns an array of toolchain configurations. Each
+ toolchain configuration is itself an array of lines of the defconfig.
+ """
+ toolchains_url = kwargs['toolchains_url']
+
+ with urlopen_closing(toolchains_url) as r:
+ l = decode_byte_list(r.readlines())
+ configs = []
+
+ (_, _, _, _, hostarch) = os.uname()
+ # ~2015 distros report x86 when on a 32bit install
+ if hostarch == 'i686' or hostarch == 'i386' or hostarch == 'x86':
+ hostarch = 'x86'
+
+ for row in csv.reader(l):
+ config = {}
+ url = row[0]
+ config_hostarch = row[1]
+ keep = False
+
+ # Keep all toolchain configs that work regardless of the host
+ # architecture
+ if config_hostarch == "any":
+ keep = True
+
+ # Keep all toolchain configs that can work on the current host
+ # architecture
+ if hostarch == config_hostarch:
+ keep = True
+
+ # Assume that x86 32 bits toolchains work on x86_64 build
+ # machines
+ if hostarch == 'x86_64' and config_hostarch == "x86":
+ keep = True
+
+ if not keep:
+ continue
+
+ with urlopen_closing(url) as r:
+ config = decode_byte_list(r.readlines())
+ configs.append(config)
+ return configs
+
+def is_toolchain_usable(**kwargs):
+ """Check if the toolchain is actually usable."""
+
+ idir = "instance-%d" % kwargs['instance']
+ sysinfo = kwargs['sysinfo']
+ log = kwargs['log']
+
+ outputdir = os.path.join(idir, "output")
+ with open(os.path.join(outputdir, ".config")) as configf:
+ configlines = configf.readlines()
+
+ # Check that the toolchain configuration is still present
+ for toolchainline in kwargs['config']:
+ if toolchainline not in configlines:
+ return False
+
+ # The latest Linaro toolchains on x86-64 hosts requires glibc
+ # 2.14+ on the host.
+ if platform.machine() == 'x86_64':
+ if 'BR2_TOOLCHAIN_EXTERNAL_LINARO_ARM=y\n' in configlines or \
+ 'BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64=y\n' in configlines or \
+ 'BR2_TOOLCHAIN_EXTERNAL_LINARO_ARMEB=y\n' in configlines:
+ ldd_version_output = subprocess.Popen(['ldd', '--version'], stdout=subprocess.PIPE).communicate()[0]
+ glibc_version = ldd_version_output.splitlines()[0].split()[-1]
+ if StrictVersion('2.14') > StrictVersion(glibc_version):
+ log_write(log, "WARN: ignoring the Linaro ARM toolchains becausee too old host glibc")
+ return False
+
+ return True
+
+def fixup_config(**kwargs):
+ """Finalize the configuration and reject any problematic combinations
+
+ This function returns 'True' when the configuration has been
+ accepted, and 'False' when the configuration has not been accepted because
+ it is known to fail (in which case another random configuration will be
+ generated).
+ """
+
+ idir = "instance-%d" % kwargs['instance']
+ sysinfo = kwargs['sysinfo']
+
+ outputdir = os.path.join(idir, "output")
+ with open(os.path.join(outputdir, ".config")) as configf:
+ configlines = configf.readlines()
+
+ if "BR2_NEEDS_HOST_JAVA=y\n" in configlines and not sysinfo.has("java"):
+ return False
+ if "BR2_NEEDS_HOST_JAVAC=y\n" in configlines and not sysinfo.has("javac"):
+ return False
+ if "BR2_NEEDS_HOST_JAR=y\n" in configlines and not sysinfo.has("jar"):
+ return False
+ # python-nfc needs bzr
+ if 'BR2_PACKAGE_PYTHON_NFC=y\n' in configlines and not sysinfo.has("bzr"):
+ return False
+ # The ctng toolchain is affected by PR58854
+ if 'BR2_PACKAGE_LTTNG_TOOLS=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/armv5-ctng-linux-gnueabi.tar.xz"\n' in configlines:
+ return False
+ # The ctng toolchain tigger an assembler error with guile package when compiled with -Os (same issue as for CS ARM 2014.05-29)
+ if 'BR2_PACKAGE_GUILE=y\n' in configlines and 'BR2_OPTIMIZE_S=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/armv5-ctng-linux-gnueabi.tar.xz"\n' in configlines:
+ return False
+ # The ctng toolchain is affected by PR58854
+ if 'BR2_PACKAGE_LTTNG_TOOLS=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/armv6-ctng-linux-uclibcgnueabi.tar.xz"\n' in configlines:
+ return False
+ # The ctng toolchain is affected by PR58854
+ if 'BR2_PACKAGE_LTTNG_TOOLS=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/armv7-ctng-linux-gnueabihf.tar.xz"\n' in configlines:
+ return False
+ # The ctng toolchain is affected by PR60155
+ if 'BR2_PACKAGE_SDL=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/powerpc-ctng-linux-uclibc.tar.xz"\n' in configlines:
+ return False
+ # The ctng toolchain is affected by PR60155
+ if 'BR2_PACKAGE_LIBMPEG2=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/powerpc-ctng-linux-uclibc.tar.xz"\n' in configlines:
+ return False
+ # This MIPS toolchain uses eglibc-2.18 which lacks SYS_getdents64
+ if 'BR2_PACKAGE_STRONGSWAN=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mips64el-ctng_n64-linux-gnu.tar.xz"\n' in configlines:
+ return False
+ # This MIPS toolchain uses eglibc-2.18 which lacks SYS_getdents64
+ if 'BR2_PACKAGE_PYTHON3=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mips64el-ctng_n64-linux-gnu.tar.xz"\n' in configlines:
+ return False
+ # libffi not available on sh2a and ARMv7-M, but propagating libffi
+ # arch dependencies in Buildroot is really too much work, so we
+ # handle this here.
+ if 'BR2_sh2a=y\n' in configlines and 'BR2_PACKAGE_LIBFFI=y\n' in configlines:
+ return False
+ if 'BR2_ARM_CPU_ARMV7M=y\n' in configlines and 'BR2_PACKAGE_LIBFFI=y\n' in configlines:
+ return False
+ if 'BR2_PACKAGE_SUNXI_BOARDS=y\n' in configlines:
+ configlines.remove('BR2_PACKAGE_SUNXI_BOARDS_FEX_FILE=""\n')
+ configlines.append('BR2_PACKAGE_SUNXI_BOARDS_FEX_FILE="a10/hackberry.fex"\n')
+ # This MIPS uClibc toolchain fails to build the gdb package
+ if 'BR2_PACKAGE_GDB=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mipsel-ctng-linux-uclibc.tar.xz"\n' in configlines:
+ return False
+ # This MIPS uClibc toolchain fails to build the rt-tests package
+ if 'BR2_PACKAGE_RT_TESTS=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mipsel-ctng-linux-uclibc.tar.xz"\n' in configlines:
+ return False
+ # This MIPS uClibc toolchain fails to build the civetweb package
+ if 'BR2_PACKAGE_CIVETWEB=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mipsel-ctng-linux-uclibc.tar.xz"\n' in configlines:
+ return False
+ # This MIPS ctng toolchain fails to build the python3 package
+ if 'BR2_PACKAGE_PYTHON3=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mips64el-ctng_n64-linux-gnu.tar.xz"\n' in configlines:
+ return False
+ # This MIPS uClibc toolchain fails to build the strace package
+ if 'BR2_PACKAGE_STRACE=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mipsel-ctng-linux-uclibc.tar.xz"\n' in configlines:
+ return False
+ # This MIPS uClibc toolchain fails to build the cdrkit package
+ if 'BR2_PACKAGE_CDRKIT=y\n' in configlines and \
+ 'BR2_STATIC_LIBS=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mipsel-ctng-linux-uclibc.tar.xz"\n' in configlines:
+ return False
+ # uClibc vfork static linking issue
+ if 'BR2_PACKAGE_ALSA_LIB=y\n' in configlines and \
+ 'BR2_STATIC_LIBS=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/i486-ctng-linux-uclibc.tar.xz"\n' in configlines:
+ return False
+ # This MIPS uClibc toolchain fails to build the weston package
+ if 'BR2_PACKAGE_WESTON=y\n' in configlines and \
+ 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mipsel-ctng-linux-uclibc.tar.xz"\n' in configlines:
+ return False
+ # The cs nios2 2017.02 toolchain is affected by binutils PR19405
+ if 'BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_NIOSII=y\n' in configlines and \
+ 'BR2_PACKAGE_BOOST=y\n' in configlines:
+ return False
+ # The cs nios2 2017.02 toolchain is affected by binutils PR19405
+ if 'BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_NIOSII=y\n' in configlines and \
+ 'BR2_PACKAGE_QT5BASE_GUI=y\n' in configlines:
+ return False
+ # The cs nios2 2017.02 toolchain is affected by binutils PR19405
+ if 'BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_NIOSII=y\n' in configlines and \
+ 'BR2_PACKAGE_QT_GUI_MODULE=y\n' in configlines:
+ return False
+ # The cs nios2 2017.02 toolchain is affected by binutils PR19405
+ if 'BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_NIOSII=y\n' in configlines and \
+ 'BR2_PACKAGE_FLANN=y\n' in configlines:
+ return False
+ # or1k affected by binutils PR21464
+ if 'BR2_or1k=y\n' in configlines and \
+ 'BR2_PACKAGE_QT_GUI_MODULE=y\n' in configlines:
+ return False
+
+ with open(os.path.join(outputdir, ".config"), "w+") as configf:
+ configf.writelines(configlines)
+
+ return True
+
+def gen_config(**kwargs):
+ """Generate a new random configuration
+
+ This function generates the configuration, by choosing a random
+ toolchain configuration and then generating a random selection of
+ packages.
+ """
+
+ idir = "instance-%d" % kwargs['instance']
+ log = kwargs['log']
+
+ # We need the absolute path to use with O=, because the relative
+ # path to the output directory here is not relative to the
+ # Buildroot sources, but to the location of the autobuilder
+ # script.
+ outputdir = os.path.abspath(os.path.join(idir, "output"))
+ srcdir = os.path.join(idir, "buildroot")
+
+ log_write(log, "INFO: generate the configuration")
+
+ # Select a random toolchain configuration
+ try:
+ configs = get_toolchain_configs(**kwargs)
+ except:
+ return -1
+
+ i = randint(0, len(configs) - 1)
+ config = configs[i]
+
+ configlines = config
+
+ # Amend the configuration with a few things.
+ configlines.append("BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y\n")
+ configlines.append("# BR2_TARGET_ROOTFS_TAR is not set\n")
+ configlines.append("BR2_COMPILER_PARANOID_UNSAFE_PATH=y\n")
+ if randint(0, 20) == 0:
+ configlines.append("BR2_ENABLE_DEBUG=y\n")
+ if randint(0, 30) == 0:
+ configlines.append("BR2_INIT_SYSTEMD=y\n")
+ elif randint(0, 20) == 0:
+ configlines.append("BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV=y\n")
+ if randint(0, 20) == 0:
+ configlines.append("BR2_STATIC_LIBS=y\n")
+ if randint(0, 20) == 0:
+ configlines.append("BR2_PACKAGE_PYTHON_PY_ONLY=y\n")
+
+ # Write out the configuration file
+ with open(os.path.join(outputdir, ".config"), "w+") as configf:
+ configf.writelines(configlines)
+
+ devnull = open(os.devnull, "w")
+
+ ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir, "olddefconfig"],
+ stdout=devnull, stderr=devnull)
+ if ret != 0:
+ log_write(log, "ERROR: cannot oldconfig")
+ return -1
+
+ if not is_toolchain_usable(config=config, **kwargs):
+ return -1
+
+ # Now, generate the random selection of packages, and fixup
+ # things if needed.
+ # Safe-guard, in case we can not quickly come to a valid
+ # configuration: allow at most 100 (arbitrary) iterations.
+ bounded_loop = 100
+ while True:
+ if bounded_loop == 0:
+ log_write(log, "ERROR: cannot generate random configuration after 100 iterations")
+ return -1
+ bounded_loop -= 1
+ ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
+ "KCONFIG_PROBABILITY=%d" % randint(1,30), "randpackageconfig"],
+ stdout=devnull, stderr=devnull)
+ if ret != 0:
+ log_write(log, "ERROR: cannot generate random configuration")
+ return -1
+ if fixup_config(**kwargs):
+ break
+
+ ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir, "olddefconfig"],
+ stdout=devnull, stderr=devnull)
+ if ret != 0:
+ log_write(log, "ERROR: cannot oldconfig")
+ return -1
+
+ ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir, "savedefconfig"],
+ stdout=devnull, stderr=devnull)
+ if ret != 0:
+ log_write(log, "ERROR: cannot savedefconfig")
+ return -1
+
+ return 0
+
+
+if __name__ == '__main__':
+ import argparse
+ parser = argparse.ArgumentParser(description="Generate a random configuration")
+ parser.add_argument("--instance", "-i",
+ help="Instance number for creating unique directories",
+ type=int, default=0)
+ parser.add_argument("--toolchains-url",
+ help="URL of toolchain configuration file",
+ type=str,
+ default="http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv")
+ args = parser.parse_args()
+
+ # Arguments expected by gen_config for which we just set a default here
+ args.log = sys.stdout
+ args.sysinfo = SystemInfo()
+
+ # Output directory is already created by autobuild-run so emulate it here
+ idir = "instance-%d" % args.instance
+ if not os.path.exists(idir):
+ os.mkdir(idir)
+ os.mkdir(os.path.join(idir, "output"))
+ # gen_config expects "buildroot" directory under idir
+ os.symlink("..", os.path.join(idir, "buildroot"))
+
+ # gen_config expects a dict, but args is a class object
+ ret = gen_config(**args.__dict__)
+
+ if ret != 0:
+ parser.exit(1)
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 02/23] genrandconfig: use subprocess.check_output instead of Popen
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 01/23] utils/genrandconfig: new script Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 03/23] genrandconfig: fix (some) pep8 warnings Arnout Vandecappelle
` (22 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
Popen is more complicated and more difficult to understand.
check_output raises an exception if the exit code is non-zero, but
that's probably what we want if ldd can't be executed.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
This is actually part of the following patch, to reduce the line length :-)
---
v7: new patch
---
utils/genrandconfig | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/utils/genrandconfig b/utils/genrandconfig
index 6d3269225d..a7fe7ceca9 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -184,7 +184,7 @@ def is_toolchain_usable(**kwargs):
if 'BR2_TOOLCHAIN_EXTERNAL_LINARO_ARM=y\n' in configlines or \
'BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64=y\n' in configlines or \
'BR2_TOOLCHAIN_EXTERNAL_LINARO_ARMEB=y\n' in configlines:
- ldd_version_output = subprocess.Popen(['ldd', '--version'], stdout=subprocess.PIPE).communicate()[0]
+ ldd_version_output = subprocess.check_output(['ldd', '--version'])
glibc_version = ldd_version_output.splitlines()[0].split()[-1]
if StrictVersion('2.14') > StrictVersion(glibc_version):
log_write(log, "WARN: ignoring the Linaro ARM toolchains becausee too old host glibc")
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 03/23] genrandconfig: fix (some) pep8 warnings
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 01/23] utils/genrandconfig: new script Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 02/23] genrandconfig: use subprocess.check_output instead of Popen Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 04/23] genrandconfig: replace kwargs with explicit arguments Arnout Vandecappelle
` (21 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
Warnings fixed:
E731 do not assign a lambda expression, use a def
-> urlopen_closing is defined with a def. urlopen is not used
elsewhere so inlined.
E302 expected 2 blank lines
E501 line too long
-> long lines due to a long string are NOT split
E701 multiple statements on one line (colon)
E722 do not use bare except'
-> use "except Exception", so KeyInterrupt and SystemExit are still
passed. We never intended to catch those.
E741 ambiguous variable name 'l'
-> variable name is replaced with the much more descriptive
toolchains_csv
E271 multiple spaces after keyword
E231 missing whitespace after ','
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
utils/genrandconfig | 53 ++++++++++++++++++++++++++++++++++++-----------------
1 file changed, 36 insertions(+), 17 deletions(-)
diff --git a/utils/genrandconfig b/utils/genrandconfig
index a7fe7ceca9..fad60cdea7 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -35,8 +35,10 @@ if sys.hexversion >= 0x3000000:
else:
import urllib2 as _urllib
-urlopen = _urllib.urlopen
-urlopen_closing = lambda uri: contextlib.closing(urlopen(uri))
+
+def urlopen_closing(uri):
+ return contextlib.closing(_urllib.urlopen(uri))
+
if sys.hexversion >= 0x3000000:
def decode_byte_list(bl):
@@ -45,10 +47,13 @@ else:
def decode_byte_list(e):
return e
+
def log_write(logf, msg):
- logf.write("[%s] %s\n" % (strftime("%a, %d %b %Y %H:%M:%S", localtime()), msg))
+ logf.write("[%s] %s\n" % (strftime("%a, %d %b %Y %H:%M:%S", localtime()),
+ msg))
logf.flush()
+
class SystemInfo:
DEFAULT_NEEDED_PROGS = ["make", "git", "gcc", "timeout"]
DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar"]
@@ -59,7 +64,8 @@ class SystemInfo:
self.progs = {}
def find_prog(self, name, flags=os.X_OK, env=os.environ):
- if not name or name[0] == os.sep: raise ValueError(name)
+ if not name or name[0] == os.sep:
+ raise ValueError(name)
prog_path = env.get("PATH", None)
# for windows compatibility, we'd need to take PATHEXT into account
@@ -78,7 +84,8 @@ class SystemInfo:
"""Checks whether a program is available.
Lazily evaluates missing entries.
- Returns: None if prog not found, else path to the program [evaluates to True]
+ Returns: None if prog not found, else path to the program [evaluates
+ to True]
"""
try:
return self.progs[prog]
@@ -89,7 +96,8 @@ class SystemInfo:
# java[c] needs special care
if have_it and prog in ('java', 'javac'):
with open(os.devnull, "w") as devnull:
- if subprocess.call("%s -version | grep gcj" % prog, shell=True,
+ if subprocess.call("%s -version | grep gcj" % prog,
+ shell=True,
stdout=devnull, stderr=devnull) != 1:
have_it = False
# --
@@ -116,6 +124,7 @@ class SystemInfo:
return not missing_requirements
+
def get_toolchain_configs(**kwargs):
"""Fetch and return the possible toolchain configurations
@@ -125,7 +134,7 @@ def get_toolchain_configs(**kwargs):
toolchains_url = kwargs['toolchains_url']
with urlopen_closing(toolchains_url) as r:
- l = decode_byte_list(r.readlines())
+ toolchains_csv = decode_byte_list(r.readlines())
configs = []
(_, _, _, _, hostarch) = os.uname()
@@ -133,7 +142,7 @@ def get_toolchain_configs(**kwargs):
if hostarch == 'i686' or hostarch == 'i386' or hostarch == 'x86':
hostarch = 'x86'
- for row in csv.reader(l):
+ for row in csv.reader(toolchains_csv):
config = {}
url = row[0]
config_hostarch = row[1]
@@ -162,6 +171,7 @@ def get_toolchain_configs(**kwargs):
configs.append(config)
return configs
+
def is_toolchain_usable(**kwargs):
"""Check if the toolchain is actually usable."""
@@ -187,11 +197,12 @@ def is_toolchain_usable(**kwargs):
ldd_version_output = subprocess.check_output(['ldd', '--version'])
glibc_version = ldd_version_output.splitlines()[0].split()[-1]
if StrictVersion('2.14') > StrictVersion(glibc_version):
- log_write(log, "WARN: ignoring the Linaro ARM toolchains becausee too old host glibc")
+ log_write(log, "WARN: ignoring the Linaro ARM toolchains because too old host glibc")
return False
return True
+
def fixup_config(**kwargs):
"""Finalize the configuration and reject any problematic combinations
@@ -222,7 +233,8 @@ def fixup_config(**kwargs):
'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/armv5-ctng-linux-gnueabi.tar.xz"\n' in configlines:
return False
# The ctng toolchain tigger an assembler error with guile package when compiled with -Os (same issue as for CS ARM 2014.05-29)
- if 'BR2_PACKAGE_GUILE=y\n' in configlines and 'BR2_OPTIMIZE_S=y\n' in configlines and \
+ if 'BR2_PACKAGE_GUILE=y\n' in configlines and \
+ 'BR2_OPTIMIZE_S=y\n' in configlines and \
'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/armv5-ctng-linux-gnueabi.tar.xz"\n' in configlines:
return False
# The ctng toolchain is affected by PR58854
@@ -252,9 +264,11 @@ def fixup_config(**kwargs):
# libffi not available on sh2a and ARMv7-M, but propagating libffi
# arch dependencies in Buildroot is really too much work, so we
# handle this here.
- if 'BR2_sh2a=y\n' in configlines and 'BR2_PACKAGE_LIBFFI=y\n' in configlines:
+ if 'BR2_sh2a=y\n' in configlines and \
+ 'BR2_PACKAGE_LIBFFI=y\n' in configlines:
return False
- if 'BR2_ARM_CPU_ARMV7M=y\n' in configlines and 'BR2_PACKAGE_LIBFFI=y\n' in configlines:
+ if 'BR2_ARM_CPU_ARMV7M=y\n' in configlines and \
+ 'BR2_PACKAGE_LIBFFI=y\n' in configlines:
return False
if 'BR2_PACKAGE_SUNXI_BOARDS=y\n' in configlines:
configlines.remove('BR2_PACKAGE_SUNXI_BOARDS_FEX_FILE=""\n')
@@ -319,6 +333,7 @@ def fixup_config(**kwargs):
return True
+
def gen_config(**kwargs):
"""Generate a new random configuration
@@ -342,7 +357,7 @@ def gen_config(**kwargs):
# Select a random toolchain configuration
try:
configs = get_toolchain_configs(**kwargs)
- except:
+ except Exception:
return -1
i = randint(0, len(configs) - 1)
@@ -371,7 +386,8 @@ def gen_config(**kwargs):
devnull = open(os.devnull, "w")
- ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir, "olddefconfig"],
+ ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
+ "olddefconfig"],
stdout=devnull, stderr=devnull)
if ret != 0:
log_write(log, "ERROR: cannot oldconfig")
@@ -391,7 +407,8 @@ def gen_config(**kwargs):
return -1
bounded_loop -= 1
ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
- "KCONFIG_PROBABILITY=%d" % randint(1,30), "randpackageconfig"],
+ "KCONFIG_PROBABILITY=%d" % randint(1, 30),
+ "randpackageconfig"],
stdout=devnull, stderr=devnull)
if ret != 0:
log_write(log, "ERROR: cannot generate random configuration")
@@ -399,13 +416,15 @@ def gen_config(**kwargs):
if fixup_config(**kwargs):
break
- ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir, "olddefconfig"],
+ ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
+ "olddefconfig"],
stdout=devnull, stderr=devnull)
if ret != 0:
log_write(log, "ERROR: cannot oldconfig")
return -1
- ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir, "savedefconfig"],
+ ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
+ "savedefconfig"],
stdout=devnull, stderr=devnull)
if ret != 0:
log_write(log, "ERROR: cannot savedefconfig")
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 04/23] genrandconfig: replace kwargs with explicit arguments
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (2 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 03/23] genrandconfig: fix (some) pep8 warnings Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 05/23] genrandconfig: move instantiation of SystemInfo down Arnout Vandecappelle
` (20 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
kwargs is a left-over from the use of docopt, it's better to use
argparse's Namespace object directly.
In addition, most functions use just one or two fields of args, so
these can just as well be passed directly as arguments to the function.
Particularly for outputdir it doesn't make sense to reconstruct it all
the time.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
utils/genrandconfig | 44 ++++++++++++++++----------------------------
1 file changed, 16 insertions(+), 28 deletions(-)
diff --git a/utils/genrandconfig b/utils/genrandconfig
index fad60cdea7..a9519b5f2f 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -125,13 +125,12 @@ class SystemInfo:
return not missing_requirements
-def get_toolchain_configs(**kwargs):
+def get_toolchain_configs(toolchains_url):
"""Fetch and return the possible toolchain configurations
This function returns an array of toolchain configurations. Each
toolchain configuration is itself an array of lines of the defconfig.
"""
- toolchains_url = kwargs['toolchains_url']
with urlopen_closing(toolchains_url) as r:
toolchains_csv = decode_byte_list(r.readlines())
@@ -172,19 +171,14 @@ def get_toolchain_configs(**kwargs):
return configs
-def is_toolchain_usable(**kwargs):
+def is_toolchain_usable(outputdir, config):
"""Check if the toolchain is actually usable."""
- idir = "instance-%d" % kwargs['instance']
- sysinfo = kwargs['sysinfo']
- log = kwargs['log']
-
- outputdir = os.path.join(idir, "output")
with open(os.path.join(outputdir, ".config")) as configf:
configlines = configf.readlines()
# Check that the toolchain configuration is still present
- for toolchainline in kwargs['config']:
+ for toolchainline in config:
if toolchainline not in configlines:
return False
@@ -203,7 +197,7 @@ def is_toolchain_usable(**kwargs):
return True
-def fixup_config(**kwargs):
+def fixup_config(outputdir, sysinfo):
"""Finalize the configuration and reject any problematic combinations
This function returns 'True' when the configuration has been
@@ -212,10 +206,6 @@ def fixup_config(**kwargs):
generated).
"""
- idir = "instance-%d" % kwargs['instance']
- sysinfo = kwargs['sysinfo']
-
- outputdir = os.path.join(idir, "output")
with open(os.path.join(outputdir, ".config")) as configf:
configlines = configf.readlines()
@@ -334,7 +324,7 @@ def fixup_config(**kwargs):
return True
-def gen_config(**kwargs):
+def gen_config(args):
"""Generate a new random configuration
This function generates the configuration, by choosing a random
@@ -342,8 +332,7 @@ def gen_config(**kwargs):
packages.
"""
- idir = "instance-%d" % kwargs['instance']
- log = kwargs['log']
+ idir = "instance-%d" % args.instance
# We need the absolute path to use with O=, because the relative
# path to the output directory here is not relative to the
@@ -352,11 +341,11 @@ def gen_config(**kwargs):
outputdir = os.path.abspath(os.path.join(idir, "output"))
srcdir = os.path.join(idir, "buildroot")
- log_write(log, "INFO: generate the configuration")
+ log_write(args.log, "INFO: generate the configuration")
# Select a random toolchain configuration
try:
- configs = get_toolchain_configs(**kwargs)
+ configs = get_toolchain_configs(args.toolchains_url)
except Exception:
return -1
@@ -390,10 +379,10 @@ def gen_config(**kwargs):
"olddefconfig"],
stdout=devnull, stderr=devnull)
if ret != 0:
- log_write(log, "ERROR: cannot oldconfig")
+ log_write(args.log, "ERROR: cannot oldconfig")
return -1
- if not is_toolchain_usable(config=config, **kwargs):
+ if not is_toolchain_usable(outputdir, config):
return -1
# Now, generate the random selection of packages, and fixup
@@ -403,7 +392,7 @@ def gen_config(**kwargs):
bounded_loop = 100
while True:
if bounded_loop == 0:
- log_write(log, "ERROR: cannot generate random configuration after 100 iterations")
+ log_write(args.log, "ERROR: cannot generate random configuration after 100 iterations")
return -1
bounded_loop -= 1
ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
@@ -411,23 +400,23 @@ def gen_config(**kwargs):
"randpackageconfig"],
stdout=devnull, stderr=devnull)
if ret != 0:
- log_write(log, "ERROR: cannot generate random configuration")
+ log_write(args.log, "ERROR: cannot generate random configuration")
return -1
- if fixup_config(**kwargs):
+ if fixup_config(outputdir, args.sysinfo):
break
ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
"olddefconfig"],
stdout=devnull, stderr=devnull)
if ret != 0:
- log_write(log, "ERROR: cannot oldconfig")
+ log_write(args.log, "ERROR: cannot oldconfig")
return -1
ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
"savedefconfig"],
stdout=devnull, stderr=devnull)
if ret != 0:
- log_write(log, "ERROR: cannot savedefconfig")
+ log_write(args.log, "ERROR: cannot savedefconfig")
return -1
return 0
@@ -457,8 +446,7 @@ if __name__ == '__main__':
# gen_config expects "buildroot" directory under idir
os.symlink("..", os.path.join(idir, "buildroot"))
- # gen_config expects a dict, but args is a class object
- ret = gen_config(**args.__dict__)
+ ret = gen_config(args)
if ret != 0:
parser.exit(1)
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 05/23] genrandconfig: move instantiation of SystemInfo down
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (3 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 04/23] genrandconfig: replace kwargs with explicit arguments Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 06/23] genrandconfig: verbose output and use stderr Arnout Vandecappelle
` (19 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
The SystemInfo class is instantiated globally and passed down to all
functions, but it is really only used in fixup_config. So instead,
instantiate it there.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
utils/genrandconfig | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/utils/genrandconfig b/utils/genrandconfig
index a9519b5f2f..79916fef2c 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -197,7 +197,7 @@ def is_toolchain_usable(outputdir, config):
return True
-def fixup_config(outputdir, sysinfo):
+def fixup_config(outputdir):
"""Finalize the configuration and reject any problematic combinations
This function returns 'True' when the configuration has been
@@ -206,6 +206,7 @@ def fixup_config(outputdir, sysinfo):
generated).
"""
+ sysinfo = SystemInfo()
with open(os.path.join(outputdir, ".config")) as configf:
configlines = configf.readlines()
@@ -402,7 +403,7 @@ def gen_config(args):
if ret != 0:
log_write(args.log, "ERROR: cannot generate random configuration")
return -1
- if fixup_config(outputdir, args.sysinfo):
+ if fixup_config(outputdir):
break
ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
@@ -436,7 +437,6 @@ if __name__ == '__main__':
# Arguments expected by gen_config for which we just set a default here
args.log = sys.stdout
- args.sysinfo = SystemInfo()
# Output directory is already created by autobuild-run so emulate it here
idir = "instance-%d" % args.instance
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 06/23] genrandconfig: verbose output and use stderr
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (4 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 05/23] genrandconfig: move instantiation of SystemInfo down Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 07/23] genrandconfig: calculate outputdir in __main__ Arnout Vandecappelle
` (18 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
The output of genrandconfig is currently very terse, which is annoying
for debugging the script or generally seeing what is going on. Also the
timing information added by log_write isn't very useful when the script
is used stand-alone.
In the new setup, (verbose) output goes to stdout and error output goes
to stderr. Also the "INFO: generate the configuration" message is
eliminated - it should go in the autobuild-run script.
We also add an explicit message when a toolchain can't be used after
the first defconfig, otherwise autobuild-run will just silently
restart.
Note that, since the output of make is no longer redirected to
/dev/null, we get one more message on stderr that will be recorded in
the autobuilder's log file: KCONFIG_SEED=0xXXXXXXXX.
This approach allows us to optimise the error handling to use
exceptions, where appropriate, which can be caught at the top level and
converted to an error message to stderr. This, in turn, allows us to use
subprocess.check_call, which eliminates a lot of conditions.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
utils/genrandconfig | 71 +++++++++++++++++------------------------------------
1 file changed, 22 insertions(+), 49 deletions(-)
diff --git a/utils/genrandconfig b/utils/genrandconfig
index 79916fef2c..a35c86f65e 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -26,7 +26,6 @@ import os
from random import randint
import subprocess
import sys
-from time import localtime, strftime
from distutils.version import StrictVersion
import platform
@@ -48,12 +47,6 @@ else:
return e
-def log_write(logf, msg):
- logf.write("[%s] %s\n" % (strftime("%a, %d %b %Y %H:%M:%S", localtime()),
- msg))
- logf.flush()
-
-
class SystemInfo:
DEFAULT_NEEDED_PROGS = ["make", "git", "gcc", "timeout"]
DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar"]
@@ -180,6 +173,8 @@ def is_toolchain_usable(outputdir, config):
# Check that the toolchain configuration is still present
for toolchainline in config:
if toolchainline not in configlines:
+ print("WARN: toolchain can't be used", file=sys.stderr)
+ print(" Missing: %s" % toolchainline.strip(), file=sys.stderr)
return False
# The latest Linaro toolchains on x86-64 hosts requires glibc
@@ -191,7 +186,7 @@ def is_toolchain_usable(outputdir, config):
ldd_version_output = subprocess.check_output(['ldd', '--version'])
glibc_version = ldd_version_output.splitlines()[0].split()[-1]
if StrictVersion('2.14') > StrictVersion(glibc_version):
- log_write(log, "WARN: ignoring the Linaro ARM toolchains because too old host glibc")
+ print("WARN: ignoring the Linaro ARM toolchains because too old host glibc", file=sys.stderr)
return False
return True
@@ -342,13 +337,8 @@ def gen_config(args):
outputdir = os.path.abspath(os.path.join(idir, "output"))
srcdir = os.path.join(idir, "buildroot")
- log_write(args.log, "INFO: generate the configuration")
-
# Select a random toolchain configuration
- try:
- configs = get_toolchain_configs(args.toolchains_url)
- except Exception:
- return -1
+ configs = get_toolchain_configs(args.toolchains_url)
i = randint(0, len(configs) - 1)
config = configs[i]
@@ -374,17 +364,11 @@ def gen_config(args):
with open(os.path.join(outputdir, ".config"), "w+") as configf:
configf.writelines(configlines)
- devnull = open(os.devnull, "w")
-
- ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
- "olddefconfig"],
- stdout=devnull, stderr=devnull)
- if ret != 0:
- log_write(args.log, "ERROR: cannot oldconfig")
- return -1
+ subprocess.check_call(["make", "O=%s" % outputdir, "-C", srcdir,
+ "olddefconfig"])
if not is_toolchain_usable(outputdir, config):
- return -1
+ return 2
# Now, generate the random selection of packages, and fixup
# things if needed.
@@ -393,32 +377,22 @@ def gen_config(args):
bounded_loop = 100
while True:
if bounded_loop == 0:
- log_write(args.log, "ERROR: cannot generate random configuration after 100 iterations")
- return -1
+ print("ERROR: cannot generate random configuration after 100 iterations",
+ file=sys.stderr)
+ return 1
bounded_loop -= 1
- ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
+ subprocess.check_call(["make", "O=%s" % outputdir, "-C", srcdir,
"KCONFIG_PROBABILITY=%d" % randint(1, 30),
- "randpackageconfig"],
- stdout=devnull, stderr=devnull)
- if ret != 0:
- log_write(args.log, "ERROR: cannot generate random configuration")
- return -1
+ "randpackageconfig"])
+
if fixup_config(outputdir):
break
- ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
- "olddefconfig"],
- stdout=devnull, stderr=devnull)
- if ret != 0:
- log_write(args.log, "ERROR: cannot oldconfig")
- return -1
+ subprocess.check_call(["make", "O=%s" % outputdir, "-C", srcdir,
+ "olddefconfig"])
- ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
- "savedefconfig"],
- stdout=devnull, stderr=devnull)
- if ret != 0:
- log_write(args.log, "ERROR: cannot savedefconfig")
- return -1
+ subprocess.check_call(["make", "O=%s" % outputdir, "-C", srcdir,
+ "savedefconfig"])
return 0
@@ -435,9 +409,6 @@ if __name__ == '__main__':
default="http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv")
args = parser.parse_args()
- # Arguments expected by gen_config for which we just set a default here
- args.log = sys.stdout
-
# Output directory is already created by autobuild-run so emulate it here
idir = "instance-%d" % args.instance
if not os.path.exists(idir):
@@ -446,7 +417,9 @@ if __name__ == '__main__':
# gen_config expects "buildroot" directory under idir
os.symlink("..", os.path.join(idir, "buildroot"))
- ret = gen_config(args)
-
- if ret != 0:
+ try:
+ ret = gen_config(args)
+ except Exception as e:
+ print(str(e), file=sys.stderr)
parser.exit(1)
+ parser.exit(ret)
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 07/23] genrandconfig: calculate outputdir in __main__
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (5 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 06/23] genrandconfig: verbose output and use stderr Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 08/23] genrandconfig: calculate buildrootdir " Arnout Vandecappelle
` (17 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
This prepares for passing outputdir as an argument.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
utils/genrandconfig | 28 ++++++++++++++--------------
1 file changed, 14 insertions(+), 14 deletions(-)
diff --git a/utils/genrandconfig b/utils/genrandconfig
index a35c86f65e..3770b16018 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -329,12 +329,6 @@ def gen_config(args):
"""
idir = "instance-%d" % args.instance
-
- # We need the absolute path to use with O=, because the relative
- # path to the output directory here is not relative to the
- # Buildroot sources, but to the location of the autobuilder
- # script.
- outputdir = os.path.abspath(os.path.join(idir, "output"))
srcdir = os.path.join(idir, "buildroot")
# Select a random toolchain configuration
@@ -361,13 +355,13 @@ def gen_config(args):
configlines.append("BR2_PACKAGE_PYTHON_PY_ONLY=y\n")
# Write out the configuration file
- with open(os.path.join(outputdir, ".config"), "w+") as configf:
+ with open(os.path.join(args.outputdir, ".config"), "w+") as configf:
configf.writelines(configlines)
- subprocess.check_call(["make", "O=%s" % outputdir, "-C", srcdir,
+ subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", srcdir,
"olddefconfig"])
- if not is_toolchain_usable(outputdir, config):
+ if not is_toolchain_usable(args.outputdir, config):
return 2
# Now, generate the random selection of packages, and fixup
@@ -381,17 +375,17 @@ def gen_config(args):
file=sys.stderr)
return 1
bounded_loop -= 1
- subprocess.check_call(["make", "O=%s" % outputdir, "-C", srcdir,
+ subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", srcdir,
"KCONFIG_PROBABILITY=%d" % randint(1, 30),
"randpackageconfig"])
- if fixup_config(outputdir):
+ if fixup_config(args.outputdir):
break
- subprocess.check_call(["make", "O=%s" % outputdir, "-C", srcdir,
+ subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", srcdir,
"olddefconfig"])
- subprocess.check_call(["make", "O=%s" % outputdir, "-C", srcdir,
+ subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", srcdir,
"savedefconfig"])
return 0
@@ -411,9 +405,15 @@ if __name__ == '__main__':
# Output directory is already created by autobuild-run so emulate it here
idir = "instance-%d" % args.instance
+ # We need the absolute path to use with O=, because the relative
+ # path to the output directory here is not relative to the
+ # Buildroot sources, but to the location of the autobuilder
+ # script.
+ args.outputdir = os.path.abspath(os.path.join(idir, "output"))
+
if not os.path.exists(idir):
os.mkdir(idir)
- os.mkdir(os.path.join(idir, "output"))
+ os.mkdir(args.outputdir)
# gen_config expects "buildroot" directory under idir
os.symlink("..", os.path.join(idir, "buildroot"))
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 08/23] genrandconfig: calculate buildrootdir in __main__
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (6 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 07/23] genrandconfig: calculate outputdir in __main__ Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 09/23] genrandconfig: pass outputdir and buildrootdir as arguments Arnout Vandecappelle
` (16 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
This prepares for passing buildrootdir as an argument.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
utils/genrandconfig | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/utils/genrandconfig b/utils/genrandconfig
index 3770b16018..880f1f63cd 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -328,9 +328,6 @@ def gen_config(args):
packages.
"""
- idir = "instance-%d" % args.instance
- srcdir = os.path.join(idir, "buildroot")
-
# Select a random toolchain configuration
configs = get_toolchain_configs(args.toolchains_url)
@@ -358,7 +355,7 @@ def gen_config(args):
with open(os.path.join(args.outputdir, ".config"), "w+") as configf:
configf.writelines(configlines)
- subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", srcdir,
+ subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", args.buildrootdir,
"olddefconfig"])
if not is_toolchain_usable(args.outputdir, config):
@@ -375,17 +372,17 @@ def gen_config(args):
file=sys.stderr)
return 1
bounded_loop -= 1
- subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", srcdir,
+ subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", args.buildrootdir,
"KCONFIG_PROBABILITY=%d" % randint(1, 30),
"randpackageconfig"])
if fixup_config(args.outputdir):
break
- subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", srcdir,
+ subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", args.buildrootdir,
"olddefconfig"])
- subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", srcdir,
+ subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", args.buildrootdir,
"savedefconfig"])
return 0
@@ -410,12 +407,13 @@ if __name__ == '__main__':
# Buildroot sources, but to the location of the autobuilder
# script.
args.outputdir = os.path.abspath(os.path.join(idir, "output"))
+ args.buildrootdir = os.path.join(idir, "buildroot")
if not os.path.exists(idir):
os.mkdir(idir)
os.mkdir(args.outputdir)
# gen_config expects "buildroot" directory under idir
- os.symlink("..", os.path.join(idir, "buildroot"))
+ os.symlink("..", args.buildrootdir)
try:
ret = gen_config(args)
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 09/23] genrandconfig: pass outputdir and buildrootdir as arguments
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (7 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 08/23] genrandconfig: calculate buildrootdir " Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 10/23] genrandconfig: calculate configfile only once Arnout Vandecappelle
` (15 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
The --instance argument is just an artifact of genrandconfig's
history as part of autobuild-run. It is much more logical to pass
the output directory and the buildroot directory as arguments, with
sane defaults.
This also allows us to remove the hack of creating a symlink in the
instance directory if it doesn't exist yet.
Note that the default outputdir 'output' doesn't work yet, because in
that case Buildroot will put the config file in the buildroot directory
instead of the output directory. This will be fixed in a follow-up
patch.
After this change, the script should be called from autobuild-run as:
subprocess.call([os.path.join(srcdir, "utils/genrandconfig"),
"-o", outputdir, "-b", srcdir,
"--toolchains-url", kwargs['toolchains_url']],
stdout=devnull, stderr=log)
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
utils/genrandconfig | 25 ++++++++++---------------
1 file changed, 10 insertions(+), 15 deletions(-)
diff --git a/utils/genrandconfig b/utils/genrandconfig
index 880f1f63cd..4893c43f41 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -352,6 +352,8 @@ def gen_config(args):
configlines.append("BR2_PACKAGE_PYTHON_PY_ONLY=y\n")
# Write out the configuration file
+ if not os.path.exists(args.outputdir):
+ os.makedirs(args.outputdir)
with open(os.path.join(args.outputdir, ".config"), "w+") as configf:
configf.writelines(configlines)
@@ -391,29 +393,22 @@ def gen_config(args):
if __name__ == '__main__':
import argparse
parser = argparse.ArgumentParser(description="Generate a random configuration")
- parser.add_argument("--instance", "-i",
- help="Instance number for creating unique directories",
- type=int, default=0)
+ parser.add_argument("--outputdir", "-o",
+ help="Output directory (relative to current directory)",
+ type=str, default='output')
+ parser.add_argument("--buildrootdir", "-b",
+ help="Buildroot directory (relative to current directory)",
+ type=str, default='.')
parser.add_argument("--toolchains-url",
help="URL of toolchain configuration file",
type=str,
default="http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv")
args = parser.parse_args()
- # Output directory is already created by autobuild-run so emulate it here
- idir = "instance-%d" % args.instance
# We need the absolute path to use with O=, because the relative
# path to the output directory here is not relative to the
- # Buildroot sources, but to the location of the autobuilder
- # script.
- args.outputdir = os.path.abspath(os.path.join(idir, "output"))
- args.buildrootdir = os.path.join(idir, "buildroot")
-
- if not os.path.exists(idir):
- os.mkdir(idir)
- os.mkdir(args.outputdir)
- # gen_config expects "buildroot" directory under idir
- os.symlink("..", args.buildrootdir)
+ # Buildroot sources, but to the current directory.
+ args.outputdir = os.path.abspath(args.outputdir)
try:
ret = gen_config(args)
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 10/23] genrandconfig: calculate configfile only once
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (8 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 09/23] genrandconfig: pass outputdir and buildrootdir as arguments Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 11/23] genrandconfig: fix the case when outputdir is 'output' Arnout Vandecappelle
` (14 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
The path to the .config file is calculated in several places - replace
it with a single calculation, and pass configfile as an argument
to is_toolchain_usable and fixup_config. These functions also don't
need outputdir any more.
This makes it easier to fix the case when configfile is not in
outputdir.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
utils/genrandconfig | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/utils/genrandconfig b/utils/genrandconfig
index 4893c43f41..eaca6cff8c 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -164,10 +164,10 @@ def get_toolchain_configs(toolchains_url):
return configs
-def is_toolchain_usable(outputdir, config):
+def is_toolchain_usable(configfile, config):
"""Check if the toolchain is actually usable."""
- with open(os.path.join(outputdir, ".config")) as configf:
+ with open(configfile) as configf:
configlines = configf.readlines()
# Check that the toolchain configuration is still present
@@ -192,7 +192,7 @@ def is_toolchain_usable(outputdir, config):
return True
-def fixup_config(outputdir):
+def fixup_config(configfile):
"""Finalize the configuration and reject any problematic combinations
This function returns 'True' when the configuration has been
@@ -202,7 +202,7 @@ def fixup_config(outputdir):
"""
sysinfo = SystemInfo()
- with open(os.path.join(outputdir, ".config")) as configf:
+ with open(configfile) as configf:
configlines = configf.readlines()
if "BR2_NEEDS_HOST_JAVA=y\n" in configlines and not sysinfo.has("java"):
@@ -314,7 +314,7 @@ def fixup_config(outputdir):
'BR2_PACKAGE_QT_GUI_MODULE=y\n' in configlines:
return False
- with open(os.path.join(outputdir, ".config"), "w+") as configf:
+ with open(configfile, "w+") as configf:
configf.writelines(configlines)
return True
@@ -354,13 +354,14 @@ def gen_config(args):
# Write out the configuration file
if not os.path.exists(args.outputdir):
os.makedirs(args.outputdir)
- with open(os.path.join(args.outputdir, ".config"), "w+") as configf:
+ configfile = os.path.join(args.outputdir, ".config")
+ with open(configfile, "w+") as configf:
configf.writelines(configlines)
subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", args.buildrootdir,
"olddefconfig"])
- if not is_toolchain_usable(args.outputdir, config):
+ if not is_toolchain_usable(configfile, config):
return 2
# Now, generate the random selection of packages, and fixup
@@ -378,7 +379,7 @@ def gen_config(args):
"KCONFIG_PROBABILITY=%d" % randint(1, 30),
"randpackageconfig"])
- if fixup_config(args.outputdir):
+ if fixup_config(configfile):
break
subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", args.buildrootdir,
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 11/23] genrandconfig: fix the case when outputdir is 'output'
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (9 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 10/23] genrandconfig: calculate configfile only once Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 12/23] support/test-pkg: move minimal.config into a separate file Arnout Vandecappelle
` (13 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
When outputdir is 'output' (the default), genrandconfig didn't work
correctly because it expects the configfile in outputdir, while
Buildroot puts it in the buildroot directory.
Fix this by explicitly checking if outputdir == buildrootdir/output.
Because abspath is used for both paths, string comparison works
reliably.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
utils/genrandconfig | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/utils/genrandconfig b/utils/genrandconfig
index eaca6cff8c..4ff7deb26a 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -354,7 +354,10 @@ def gen_config(args):
# Write out the configuration file
if not os.path.exists(args.outputdir):
os.makedirs(args.outputdir)
- configfile = os.path.join(args.outputdir, ".config")
+ if args.outputdir == os.path.abspath(os.path.join(args.buildrootdir, "output")):
+ configfile = os.path.join(args.buildrootdir, ".config")
+ else:
+ configfile = os.path.join(args.outputdir, ".config")
with open(configfile, "w+") as configf:
configf.writelines(configlines)
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 12/23] support/test-pkg: move minimal.config into a separate file
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (10 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 11/23] genrandconfig: fix the case when outputdir is 'output' Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 7:01 ` Thomas Petazzoni
2017-07-21 1:05 ` [Buildroot] [PATCH v7 13/23] minimal.config: add BR2_COMPILER_PARANOID_UNSAFE_PATH=y Arnout Vandecappelle
` (12 subsequent siblings)
24 siblings, 1 reply; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
This minimal configuration is also very useful outside test-pkg. In
addition, it will simplify the config merge in a later patch.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
---
v7: Re-ordered the patch
v6: Use support/config-fragments instead of support/misc
v5: First version
---
support/config-fragments/minimal.config | 6 ++++++
utils/test-pkg | 8 +-------
2 files changed, 7 insertions(+), 7 deletions(-)
create mode 100644 support/config-fragments/minimal.config
diff --git a/support/config-fragments/minimal.config b/support/config-fragments/minimal.config
new file mode 100644
index 0000000000..0f20847f6d
--- /dev/null
+++ b/support/config-fragments/minimal.config
@@ -0,0 +1,6 @@
+# This config fragment disables Buildroot options that are turned on by
+# default, in order to arrive at minimal build time.
+BR2_INIT_NONE=y
+BR2_SYSTEM_BIN_SH_NONE=y
+# BR2_PACKAGE_BUSYBOX is not set
+# BR2_TARGET_ROOTFS_TAR is not set
diff --git a/utils/test-pkg b/utils/test-pkg
index 7633a21e53..25a25e2048 100755
--- a/utils/test-pkg
+++ b/utils/test-pkg
@@ -100,13 +100,7 @@ build_one() {
return 2
fi
- cat >>"${dir}/.config" <<-_EOF_
- BR2_INIT_NONE=y
- BR2_SYSTEM_BIN_SH_NONE=y
- # BR2_PACKAGE_BUSYBOX is not set
- # BR2_TARGET_ROOTFS_TAR is not set
- _EOF_
- cat "${cfg}" >>"${dir}/.config"
+ cat "support/config-fragments/minimal.config" "${cfg}" >>"${dir}/.config"
if ! make O="${dir}" olddefconfig > "${dir}/logfile" 2>&1; then
return 2
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 12/23] support/test-pkg: move minimal.config into a separate file
2017-07-21 1:05 ` [Buildroot] [PATCH v7 12/23] support/test-pkg: move minimal.config into a separate file Arnout Vandecappelle
@ 2017-07-21 7:01 ` Thomas Petazzoni
0 siblings, 0 replies; 34+ messages in thread
From: Thomas Petazzoni @ 2017-07-21 7:01 UTC (permalink / raw)
To: buildroot
Hello,
On Fri, 21 Jul 2017 03:05:19 +0200, Arnout Vandecappelle
(Essensium/Mind) wrote:
> This minimal configuration is also very useful outside test-pkg. In
> addition, it will simplify the config merge in a later patch.
>
> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> Cc: Yann E. MORIN <yann.morin.1998@free.fr>
> ---
> v7: Re-ordered the patch
> v6: Use support/config-fragments instead of support/misc
> v5: First version
> ---
> support/config-fragments/minimal.config | 6 ++++++
> utils/test-pkg | 8 +-------
> 2 files changed, 7 insertions(+), 7 deletions(-)
> create mode 100644 support/config-fragments/minimal.config
Applied to master, thanks.
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 34+ messages in thread
* [Buildroot] [PATCH v7 13/23] minimal.config: add BR2_COMPILER_PARANOID_UNSAFE_PATH=y
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (11 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 12/23] support/test-pkg: move minimal.config into a separate file Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 7:01 ` Thomas Petazzoni
2017-07-21 1:05 ` [Buildroot] [PATCH v7 14/23] minimal.config: add BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y Arnout Vandecappelle
` (11 subsequent siblings)
24 siblings, 1 reply; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
We really want test-pkg to do the test with a paranoid unsafe path.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
support/config-fragments/minimal.config | 1 +
1 file changed, 1 insertion(+)
diff --git a/support/config-fragments/minimal.config b/support/config-fragments/minimal.config
index 0f20847f6d..43fef06f49 100644
--- a/support/config-fragments/minimal.config
+++ b/support/config-fragments/minimal.config
@@ -4,3 +4,4 @@ BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
# BR2_PACKAGE_BUSYBOX is not set
# BR2_TARGET_ROOTFS_TAR is not set
+BR2_COMPILER_PARANOID_UNSAFE_PATH=y
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 14/23] minimal.config: add BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (12 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 13/23] minimal.config: add BR2_COMPILER_PARANOID_UNSAFE_PATH=y Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 7:01 ` Thomas Petazzoni
2017-07-21 1:05 ` [Buildroot] [PATCH v7 15/23] genrandconfig: use minimal.config Arnout Vandecappelle
` (10 subsequent siblings)
24 siblings, 1 reply; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
The user shouldn't need to pass this manually when creating a test-pkg
config file. It's an absolutely harmless option to enable always.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
support/config-fragments/minimal.config | 1 +
1 file changed, 1 insertion(+)
diff --git a/support/config-fragments/minimal.config b/support/config-fragments/minimal.config
index 43fef06f49..71344e2c69 100644
--- a/support/config-fragments/minimal.config
+++ b/support/config-fragments/minimal.config
@@ -5,3 +5,4 @@ BR2_SYSTEM_BIN_SH_NONE=y
# BR2_PACKAGE_BUSYBOX is not set
# BR2_TARGET_ROOTFS_TAR is not set
BR2_COMPILER_PARANOID_UNSAFE_PATH=y
+BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 15/23] genrandconfig: use minimal.config
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (13 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 14/23] minimal.config: add BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 16/23] support: add autobuild toolchain config fragments Arnout Vandecappelle
` (9 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
This has a number of side-effects which must be handled.
The lines in minimal.config may be overridden by the random lines added
by amending the configuration, so is_toolchain_usable() shouldn't take
those into account, or indeed the random lines added. Therefore, make
a copy of the config before appending minimal.config and the random
lines. While we're at it, rename the variable to the more appropriate
toolchainconfig.
minimal.config sets BR2_INIT_NONE=y, but we really also want to test
with BR2_INIT_BUSYBOX=y. Therefore, add a random line to use the
busybox init system. We set its probability rather high. The
probabilities of systemd and eudev are increased since they're now
in the else branch of BR2_INIT_BUSYBOX, which halves the probability
that we even get there.
We now also generate configurations without busybox. Previously,
busybox was almost always selected due to BR2_INIT_BUSYBOX=y. Only if
systemd is selected there was a chance to build without busybox.
We now set BR2_SYSTEM_BIN_SH_NONE=y, the other /bin/sh options are
never tested. However, this is not really something that is relevant
to test in the autobuilders.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
utils/genrandconfig | 21 +++++++++++++--------
1 file changed, 13 insertions(+), 8 deletions(-)
diff --git a/utils/genrandconfig b/utils/genrandconfig
index 4ff7deb26a..d397b23f36 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -332,19 +332,24 @@ def gen_config(args):
configs = get_toolchain_configs(args.toolchains_url)
i = randint(0, len(configs) - 1)
- config = configs[i]
+ toolchainconfig = configs[i]
- configlines = config
+ configlines = list(toolchainconfig)
+
+ # Combine with the minimal configuration
+ minimalconfigfile = os.path.join(args.buildrootdir,
+ 'support/config-fragments/minimal.config')
+ with open(minimalconfigfile) as minimalf:
+ configlines += minimalf.readlines()
# Amend the configuration with a few things.
- configlines.append("BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y\n")
- configlines.append("# BR2_TARGET_ROOTFS_TAR is not set\n")
- configlines.append("BR2_COMPILER_PARANOID_UNSAFE_PATH=y\n")
if randint(0, 20) == 0:
configlines.append("BR2_ENABLE_DEBUG=y\n")
- if randint(0, 30) == 0:
+ if randint(0, 1) == 0:
+ configlines.append("BR2_INIT_BUSYBOX=y\n")
+ elif randint(0, 15) == 0:
configlines.append("BR2_INIT_SYSTEMD=y\n")
- elif randint(0, 20) == 0:
+ elif randint(0, 10) == 0:
configlines.append("BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV=y\n")
if randint(0, 20) == 0:
configlines.append("BR2_STATIC_LIBS=y\n")
@@ -364,7 +369,7 @@ def gen_config(args):
subprocess.check_call(["make", "O=%s" % args.outputdir, "-C", args.buildrootdir,
"olddefconfig"])
- if not is_toolchain_usable(configfile, config):
+ if not is_toolchain_usable(configfile, toolchainconfig):
return 2
# Now, generate the random selection of packages, and fixup
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 16/23] support: add autobuild toolchain config fragments
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (14 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 15/23] genrandconfig: use minimal.config Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 17/23] test-pkg: get configs from in-tree toolchain-configs.csv Arnout Vandecappelle
` (8 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
We currently have a list of toolchain configurations that are used by
the autobuilders at [1]. However, this makes it a little more difficult
for people to use these configurations, and also to have a different
list of configurations for different branches. For example if a new
architecture is introduced, the 2017.02.x branch doesn't have support
for this architecture yet so it shouldn't try to run those configs.
Therefore, include the autobuild config fragments directly in Buildroot,
so they can be branched together with the rest. We create a new
directory under support/ to store them.
Generated with
wget -nd --no-parent --recursive http://autobuild.buildroot.net/toolchains/configs/
The index.html file is removed.
The toolchain-configs.csv file is adapted so the URLs become relative
paths pointing to the config fragments.
[1] http://autobuild.buildroot.net/toolchains/configs/toolchain-configs.csv
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: update the configs; include toolchain-configs.csv
v6: Move to support/config-fragments/autobuild
v5: Refresh after rebuild of the br-* toolchains
Don't add HOSTARCH, it's not needed as explained above
v2-v4 don't exist (due to merging 2 series)
---
.../autobuild/armv5-ctng-linux-gnueabi.config | 10 +++++
.../autobuild/armv7-ctng-linux-gnueabihf.config | 13 ++++++
.../autobuild/br-aarch64-glibc.config | 9 ++++
.../autobuild/br-arc-full-internal.config | 3 ++
.../autobuild/br-arcle-hs38.config | 11 +++++
.../config-fragments/autobuild/br-arm-basic.config | 8 ++++
.../autobuild/br-arm-cortex-a9-glibc.config | 11 +++++
.../autobuild/br-arm-cortex-a9-musl.config | 11 +++++
.../autobuild/br-arm-cortex-m4-full.config | 12 +++++
.../autobuild/br-arm-full-nothread.config | 11 +++++
.../autobuild/br-arm-full-static.config | 11 +++++
.../config-fragments/autobuild/br-arm-full.config | 10 +++++
.../autobuild/br-arm-internal-full.config | 6 +++
.../config-fragments/autobuild/br-bfin-full.config | 12 +++++
.../autobuild/br-i386-pentium-mmx-musl.config | 10 +++++
.../autobuild/br-i386-pentium4-full.config | 10 +++++
.../autobuild/br-m68k-5208-full.config | 11 +++++
.../autobuild/br-m68k-68040-full.config | 10 +++++
.../autobuild/br-microblazeel-full-internal.config | 1 +
.../autobuild/br-microblazeel-full.config | 10 +++++
.../autobuild/br-mips32r6-el-hf-glibc.config | 11 +++++
.../autobuild/br-mips64-n64-full.config | 10 +++++
.../autobuild/br-mips64r6-el-hf-glibc.config | 12 +++++
.../autobuild/br-mipsel-o32-full.config | 10 +++++
.../autobuild/br-nios2-glibc.config | 9 ++++
.../autobuild/br-openrisc-uclibc.config | 10 +++++
.../autobuild/br-powerpc-603e-basic-cpp.config | 10 +++++
.../autobuild/br-powerpc-e500mc-full.config | 11 +++++
.../autobuild/br-powerpc-internal-full.config | 5 +++
.../autobuild/br-powerpc64-power7-glibc.config | 9 ++++
.../autobuild/br-powerpc64le-power8-glibc.config | 9 ++++
.../config-fragments/autobuild/br-sh4-full.config | 10 +++++
.../autobuild/br-sparc-uclibc.config | 9 ++++
.../autobuild/br-sparc64-glibc.config | 8 ++++
.../autobuild/br-x86-64-core2-full.config | 11 +++++
.../autobuild/br-x86-64-musl.config | 10 +++++
.../autobuild/br-xtensa-full-internal.config | 4 ++
.../autobuild/br-xtensa-full.config | 9 ++++
.../autobuild/i686-ctng-linux-gnu.config | 10 +++++
.../autobuild/linaro-aarch64.config | 3 ++
.../config-fragments/autobuild/linaro-arm.config | 5 +++
.../autobuild/mips64el-ctng_n32-linux-gnu.config | 9 ++++
.../autobuild/mips64el-ctng_n64-linux-gnu.config | 11 +++++
.../powerpc-ctng_e500v2-linux-gnuspe.config | 11 +++++
.../autobuild/sourcery-arm-armv4t.config | 4 ++
.../autobuild/sourcery-arm-thumb2.config | 7 +++
.../config-fragments/autobuild/sourcery-arm.config | 3 ++
.../autobuild/sourcery-mips.config | 4 ++
.../autobuild/sourcery-mips64.config | 5 +++
.../autobuild/sourcery-nios2.config | 3 ++
.../autobuild/sourcery-x86-64.config | 4 ++
.../autobuild/toolchain-configs.csv | 52 ++++++++++++++++++++++
.../autobuild/x86_64-ctng_locales-linux-gnu.config | 11 +++++
53 files changed, 499 insertions(+)
create mode 100644 support/config-fragments/autobuild/armv5-ctng-linux-gnueabi.config
create mode 100644 support/config-fragments/autobuild/armv7-ctng-linux-gnueabihf.config
create mode 100644 support/config-fragments/autobuild/br-aarch64-glibc.config
create mode 100644 support/config-fragments/autobuild/br-arc-full-internal.config
create mode 100644 support/config-fragments/autobuild/br-arcle-hs38.config
create mode 100644 support/config-fragments/autobuild/br-arm-basic.config
create mode 100644 support/config-fragments/autobuild/br-arm-cortex-a9-glibc.config
create mode 100644 support/config-fragments/autobuild/br-arm-cortex-a9-musl.config
create mode 100644 support/config-fragments/autobuild/br-arm-cortex-m4-full.config
create mode 100644 support/config-fragments/autobuild/br-arm-full-nothread.config
create mode 100644 support/config-fragments/autobuild/br-arm-full-static.config
create mode 100644 support/config-fragments/autobuild/br-arm-full.config
create mode 100644 support/config-fragments/autobuild/br-arm-internal-full.config
create mode 100644 support/config-fragments/autobuild/br-bfin-full.config
create mode 100644 support/config-fragments/autobuild/br-i386-pentium-mmx-musl.config
create mode 100644 support/config-fragments/autobuild/br-i386-pentium4-full.config
create mode 100644 support/config-fragments/autobuild/br-m68k-5208-full.config
create mode 100644 support/config-fragments/autobuild/br-m68k-68040-full.config
create mode 100644 support/config-fragments/autobuild/br-microblazeel-full-internal.config
create mode 100644 support/config-fragments/autobuild/br-microblazeel-full.config
create mode 100644 support/config-fragments/autobuild/br-mips32r6-el-hf-glibc.config
create mode 100644 support/config-fragments/autobuild/br-mips64-n64-full.config
create mode 100644 support/config-fragments/autobuild/br-mips64r6-el-hf-glibc.config
create mode 100644 support/config-fragments/autobuild/br-mipsel-o32-full.config
create mode 100644 support/config-fragments/autobuild/br-nios2-glibc.config
create mode 100644 support/config-fragments/autobuild/br-openrisc-uclibc.config
create mode 100644 support/config-fragments/autobuild/br-powerpc-603e-basic-cpp.config
create mode 100644 support/config-fragments/autobuild/br-powerpc-e500mc-full.config
create mode 100644 support/config-fragments/autobuild/br-powerpc-internal-full.config
create mode 100644 support/config-fragments/autobuild/br-powerpc64-power7-glibc.config
create mode 100644 support/config-fragments/autobuild/br-powerpc64le-power8-glibc.config
create mode 100644 support/config-fragments/autobuild/br-sh4-full.config
create mode 100644 support/config-fragments/autobuild/br-sparc-uclibc.config
create mode 100644 support/config-fragments/autobuild/br-sparc64-glibc.config
create mode 100644 support/config-fragments/autobuild/br-x86-64-core2-full.config
create mode 100644 support/config-fragments/autobuild/br-x86-64-musl.config
create mode 100644 support/config-fragments/autobuild/br-xtensa-full-internal.config
create mode 100644 support/config-fragments/autobuild/br-xtensa-full.config
create mode 100644 support/config-fragments/autobuild/i686-ctng-linux-gnu.config
create mode 100644 support/config-fragments/autobuild/linaro-aarch64.config
create mode 100644 support/config-fragments/autobuild/linaro-arm.config
create mode 100644 support/config-fragments/autobuild/mips64el-ctng_n32-linux-gnu.config
create mode 100644 support/config-fragments/autobuild/mips64el-ctng_n64-linux-gnu.config
create mode 100644 support/config-fragments/autobuild/powerpc-ctng_e500v2-linux-gnuspe.config
create mode 100644 support/config-fragments/autobuild/sourcery-arm-armv4t.config
create mode 100644 support/config-fragments/autobuild/sourcery-arm-thumb2.config
create mode 100644 support/config-fragments/autobuild/sourcery-arm.config
create mode 100644 support/config-fragments/autobuild/sourcery-mips.config
create mode 100644 support/config-fragments/autobuild/sourcery-mips64.config
create mode 100644 support/config-fragments/autobuild/sourcery-nios2.config
create mode 100644 support/config-fragments/autobuild/sourcery-x86-64.config
create mode 100644 support/config-fragments/autobuild/toolchain-configs.csv
create mode 100644 support/config-fragments/autobuild/x86_64-ctng_locales-linux-gnu.config
diff --git a/support/config-fragments/autobuild/armv5-ctng-linux-gnueabi.config b/support/config-fragments/autobuild/armv5-ctng-linux-gnueabi.config
new file mode 100644
index 0000000000..c2f4cb62ca
--- /dev/null
+++ b/support/config-fragments/autobuild/armv5-ctng-linux-gnueabi.config
@@ -0,0 +1,10 @@
+BR2_arm=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/armv5-ctng-linux-gnueabi.tar.xz"
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="armv5-ctng-linux-gnueabi"
+BR2_TOOLCHAIN_EXTERNAL_GCC_4_8=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_12=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/armv7-ctng-linux-gnueabihf.config b/support/config-fragments/autobuild/armv7-ctng-linux-gnueabihf.config
new file mode 100644
index 0000000000..60ec952940
--- /dev/null
+++ b/support/config-fragments/autobuild/armv7-ctng-linux-gnueabihf.config
@@ -0,0 +1,13 @@
+BR2_arm=y
+BR2_cortex_a9=y
+BR2_ARM_ENABLE_NEON=y
+BR2_ARM_ENABLE_VFP=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/armv7-ctng-linux-gnueabihf.tar.xz"
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="armv7-ctng-linux-gnueabihf"
+BR2_TOOLCHAIN_EXTERNAL_GCC_4_8=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_12=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-aarch64-glibc.config b/support/config-fragments/autobuild/br-aarch64-glibc.config
new file mode 100644
index 0000000000..6501f15ec5
--- /dev/null
+++ b/support/config-fragments/autobuild/br-aarch64-glibc.config
@@ -0,0 +1,9 @@
+BR2_aarch64=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-aarch64-glibc-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_1=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-arc-full-internal.config b/support/config-fragments/autobuild/br-arc-full-internal.config
new file mode 100644
index 0000000000..4755c9e45b
--- /dev/null
+++ b/support/config-fragments/autobuild/br-arc-full-internal.config
@@ -0,0 +1,3 @@
+BR2_arcle=y
+BR2_TOOLCHAIN_BUILDROOT_LOCALE=y
+BR2_TOOLCHAIN_BUILDROOT_CXX=y
diff --git a/support/config-fragments/autobuild/br-arcle-hs38.config b/support/config-fragments/autobuild/br-arcle-hs38.config
new file mode 100644
index 0000000000..b9f5764603
--- /dev/null
+++ b/support/config-fragments/autobuild/br-arcle-hs38.config
@@ -0,0 +1,11 @@
+BR2_arcle=y
+BR2_archs38=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arcle-hs38-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_1=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-arm-basic.config b/support/config-fragments/autobuild/br-arm-basic.config
new file mode 100644
index 0000000000..e47c742c6c
--- /dev/null
+++ b/support/config-fragments/autobuild/br-arm-basic.config
@@ -0,0 +1,8 @@
+BR2_arm=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-basic-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
diff --git a/support/config-fragments/autobuild/br-arm-cortex-a9-glibc.config b/support/config-fragments/autobuild/br-arm-cortex-a9-glibc.config
new file mode 100644
index 0000000000..662de17b37
--- /dev/null
+++ b/support/config-fragments/autobuild/br-arm-cortex-a9-glibc.config
@@ -0,0 +1,11 @@
+BR2_arm=y
+BR2_cortex_a9=y
+BR2_ARM_ENABLE_VFP=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-cortex-a9-glibc-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_7=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-arm-cortex-a9-musl.config b/support/config-fragments/autobuild/br-arm-cortex-a9-musl.config
new file mode 100644
index 0000000000..65e2edfa86
--- /dev/null
+++ b/support/config-fragments/autobuild/br-arm-cortex-a9-musl.config
@@ -0,0 +1,11 @@
+BR2_arm=y
+BR2_cortex_a9=y
+BR2_ARM_ENABLE_VFP=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-cortex-a9-musl-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-arm-cortex-m4-full.config b/support/config-fragments/autobuild/br-arm-cortex-m4-full.config
new file mode 100644
index 0000000000..fc514b1d97
--- /dev/null
+++ b/support/config-fragments/autobuild/br-arm-cortex-m4-full.config
@@ -0,0 +1,12 @@
+BR2_arm=y
+BR2_cortex_m4=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-cortex-m4-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-arm-full-nothread.config b/support/config-fragments/autobuild/br-arm-full-nothread.config
new file mode 100644
index 0000000000..1e260ec90d
--- /dev/null
+++ b/support/config-fragments/autobuild/br-arm-full-nothread.config
@@ -0,0 +1,11 @@
+BR2_arm=y
+BR2_arm1176jzf_s=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm11-full-nothread-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-arm-full-static.config b/support/config-fragments/autobuild/br-arm-full-static.config
new file mode 100644
index 0000000000..065c454653
--- /dev/null
+++ b/support/config-fragments/autobuild/br-arm-full-static.config
@@ -0,0 +1,11 @@
+BR2_arm=y
+BR2_STATIC_LIBS=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-static-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-arm-full.config b/support/config-fragments/autobuild/br-arm-full.config
new file mode 100644
index 0000000000..84c225d883
--- /dev/null
+++ b/support/config-fragments/autobuild/br-arm-full.config
@@ -0,0 +1,10 @@
+BR2_arm=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_4_9=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_10=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-arm-internal-full.config b/support/config-fragments/autobuild/br-arm-internal-full.config
new file mode 100644
index 0000000000..2e70534882
--- /dev/null
+++ b/support/config-fragments/autobuild/br-arm-internal-full.config
@@ -0,0 +1,6 @@
+BR2_arm=y
+BR2_ARM_EABI=y
+BR2_UCLIBC_VERSION_NG=y
+BR2_TOOLCHAIN_BUILDROOT_LOCALE=y
+BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
+BR2_TOOLCHAIN_BUILDROOT_CXX=y
diff --git a/support/config-fragments/autobuild/br-bfin-full.config b/support/config-fragments/autobuild/br-bfin-full.config
new file mode 100644
index 0000000000..7389bb53c3
--- /dev/null
+++ b/support/config-fragments/autobuild/br-bfin-full.config
@@ -0,0 +1,12 @@
+BR2_bfin=y
+BR2_bf512=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.net/toolchains/tarballs/br-bfin-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-i386-pentium-mmx-musl.config b/support/config-fragments/autobuild/br-i386-pentium-mmx-musl.config
new file mode 100644
index 0000000000..65119c321e
--- /dev/null
+++ b/support/config-fragments/autobuild/br-i386-pentium-mmx-musl.config
@@ -0,0 +1,10 @@
+BR2_x86_pentium_mmx=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.net/toolchains/tarballs/br-i386-pentium-mmx-musl-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_12=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_SSP is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-i386-pentium4-full.config b/support/config-fragments/autobuild/br-i386-pentium4-full.config
new file mode 100644
index 0000000000..37233c5975
--- /dev/null
+++ b/support/config-fragments/autobuild/br-i386-pentium4-full.config
@@ -0,0 +1,10 @@
+BR2_x86_pentium4=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-i386-pentium4-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_2=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-m68k-5208-full.config b/support/config-fragments/autobuild/br-m68k-5208-full.config
new file mode 100644
index 0000000000..e6c6bdabad
--- /dev/null
+++ b/support/config-fragments/autobuild/br-m68k-5208-full.config
@@ -0,0 +1,11 @@
+BR2_m68k=y
+BR2_m68k_cf5208=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.net/toolchains/tarballs/br-m68k-5208-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-m68k-68040-full.config b/support/config-fragments/autobuild/br-m68k-68040-full.config
new file mode 100644
index 0000000000..5cf68def9f
--- /dev/null
+++ b/support/config-fragments/autobuild/br-m68k-68040-full.config
@@ -0,0 +1,10 @@
+BR2_m68k=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.net/toolchains/tarballs/br-m68k-68040-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-microblazeel-full-internal.config b/support/config-fragments/autobuild/br-microblazeel-full-internal.config
new file mode 100644
index 0000000000..0d2d7df8f1
--- /dev/null
+++ b/support/config-fragments/autobuild/br-microblazeel-full-internal.config
@@ -0,0 +1 @@
+BR2_microblazeel=y
diff --git a/support/config-fragments/autobuild/br-microblazeel-full.config b/support/config-fragments/autobuild/br-microblazeel-full.config
new file mode 100644
index 0000000000..4cd72d1ef2
--- /dev/null
+++ b/support/config-fragments/autobuild/br-microblazeel-full.config
@@ -0,0 +1,10 @@
+BR2_microblazeel=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-microblaze-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_1=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-mips32r6-el-hf-glibc.config b/support/config-fragments/autobuild/br-mips32r6-el-hf-glibc.config
new file mode 100644
index 0000000000..86fb0148ec
--- /dev/null
+++ b/support/config-fragments/autobuild/br-mips32r6-el-hf-glibc.config
@@ -0,0 +1,11 @@
+BR2_mipsel=y
+BR2_mips_32r6=y
+# BR2_MIPS_SOFT_FLOAT is not set
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-mips32r6-el-hf-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-mips64-n64-full.config b/support/config-fragments/autobuild/br-mips64-n64-full.config
new file mode 100644
index 0000000000..503f0a2e7c
--- /dev/null
+++ b/support/config-fragments/autobuild/br-mips64-n64-full.config
@@ -0,0 +1,10 @@
+BR2_mips64el=y
+BR2_MIPS_NABI64=y
+# BR2_MIPS_SOFT_FLOAT is not set
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-mips64-n64-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_4_9=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-mips64r6-el-hf-glibc.config b/support/config-fragments/autobuild/br-mips64r6-el-hf-glibc.config
new file mode 100644
index 0000000000..a8f9cb7f34
--- /dev/null
+++ b/support/config-fragments/autobuild/br-mips64r6-el-hf-glibc.config
@@ -0,0 +1,12 @@
+BR2_mips64el=y
+BR2_mips_64r6=y
+BR2_MIPS_NABI64=y
+# BR2_MIPS_SOFT_FLOAT is not set
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-mips64r6-n64-el-hf-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-mipsel-o32-full.config b/support/config-fragments/autobuild/br-mipsel-o32-full.config
new file mode 100644
index 0000000000..e6cedfa225
--- /dev/null
+++ b/support/config-fragments/autobuild/br-mipsel-o32-full.config
@@ -0,0 +1,10 @@
+BR2_mipsel=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-mipsel-o32-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-nios2-glibc.config b/support/config-fragments/autobuild/br-nios2-glibc.config
new file mode 100644
index 0000000000..c63e4a22cd
--- /dev/null
+++ b/support/config-fragments/autobuild/br-nios2-glibc.config
@@ -0,0 +1,9 @@
+BR2_nios2=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-nios2-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-openrisc-uclibc.config b/support/config-fragments/autobuild/br-openrisc-uclibc.config
new file mode 100644
index 0000000000..04d6b3c8bd
--- /dev/null
+++ b/support/config-fragments/autobuild/br-openrisc-uclibc.config
@@ -0,0 +1,10 @@
+BR2_or1k=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.net/toolchains/tarballs/br-openrisc-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_5=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-powerpc-603e-basic-cpp.config b/support/config-fragments/autobuild/br-powerpc-603e-basic-cpp.config
new file mode 100644
index 0000000000..eaac058923
--- /dev/null
+++ b/support/config-fragments/autobuild/br-powerpc-603e-basic-cpp.config
@@ -0,0 +1,10 @@
+BR2_powerpc=y
+BR2_powerpc_603e=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-powerpc-603e-basic-cpp-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-powerpc-e500mc-full.config b/support/config-fragments/autobuild/br-powerpc-e500mc-full.config
new file mode 100644
index 0000000000..f4c073779e
--- /dev/null
+++ b/support/config-fragments/autobuild/br-powerpc-e500mc-full.config
@@ -0,0 +1,11 @@
+BR2_powerpc=y
+BR2_powerpc_e500mc=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-powerpc-e500mc-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-powerpc-internal-full.config b/support/config-fragments/autobuild/br-powerpc-internal-full.config
new file mode 100644
index 0000000000..2621e9fb41
--- /dev/null
+++ b/support/config-fragments/autobuild/br-powerpc-internal-full.config
@@ -0,0 +1,5 @@
+BR2_powerpc=y
+BR2_TOOLCHAIN_BUILDROOT_INET_IPV6=y
+BR2_TOOLCHAIN_BUILDROOT_LOCALE=y
+BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
+BR2_TOOLCHAIN_BUILDROOT_CXX=y
diff --git a/support/config-fragments/autobuild/br-powerpc64-power7-glibc.config b/support/config-fragments/autobuild/br-powerpc64-power7-glibc.config
new file mode 100644
index 0000000000..6e31cbb7c7
--- /dev/null
+++ b/support/config-fragments/autobuild/br-powerpc64-power7-glibc.config
@@ -0,0 +1,9 @@
+BR2_powerpc64=y
+BR2_powerpc_power7=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.net/toolchains/tarballs/br-powerpc64-power7-glibc-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-powerpc64le-power8-glibc.config b/support/config-fragments/autobuild/br-powerpc64le-power8-glibc.config
new file mode 100644
index 0000000000..80d5b3c133
--- /dev/null
+++ b/support/config-fragments/autobuild/br-powerpc64le-power8-glibc.config
@@ -0,0 +1,9 @@
+BR2_powerpc64le=y
+BR2_powerpc_power8=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.net/toolchains/tarballs/br-powerpc64le-power8-glibc-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-sh4-full.config b/support/config-fragments/autobuild/br-sh4-full.config
new file mode 100644
index 0000000000..b50e3528b6
--- /dev/null
+++ b/support/config-fragments/autobuild/br-sh4-full.config
@@ -0,0 +1,10 @@
+BR2_sh=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-sh4-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-sparc-uclibc.config b/support/config-fragments/autobuild/br-sparc-uclibc.config
new file mode 100644
index 0000000000..e2a69bb56d
--- /dev/null
+++ b/support/config-fragments/autobuild/br-sparc-uclibc.config
@@ -0,0 +1,9 @@
+BR2_sparc=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-sparc-uclibc-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_1=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-sparc64-glibc.config b/support/config-fragments/autobuild/br-sparc64-glibc.config
new file mode 100644
index 0000000000..c1d272141a
--- /dev/null
+++ b/support/config-fragments/autobuild/br-sparc64-glibc.config
@@ -0,0 +1,8 @@
+BR2_sparc64=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-sparc64-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_1=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-x86-64-core2-full.config b/support/config-fragments/autobuild/br-x86-64-core2-full.config
new file mode 100644
index 0000000000..56e2238731
--- /dev/null
+++ b/support/config-fragments/autobuild/br-x86-64-core2-full.config
@@ -0,0 +1,11 @@
+BR2_x86_64=y
+BR2_x86_core2=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-x86-64-core2-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_4=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-x86-64-musl.config b/support/config-fragments/autobuild/br-x86-64-musl.config
new file mode 100644
index 0000000000..48f84cc79a
--- /dev/null
+++ b/support/config-fragments/autobuild/br-x86-64-musl.config
@@ -0,0 +1,10 @@
+BR2_x86_64=y
+BR2_x86_atom=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-x86-64-musl-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_MUSL=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/br-xtensa-full-internal.config b/support/config-fragments/autobuild/br-xtensa-full-internal.config
new file mode 100644
index 0000000000..f642a4d76b
--- /dev/null
+++ b/support/config-fragments/autobuild/br-xtensa-full-internal.config
@@ -0,0 +1,4 @@
+BR2_xtensa=y
+BR2_JLEVEL=8
+BR2_TOOLCHAIN_BUILDROOT_LOCALE=y
+BR2_TOOLCHAIN_BUILDROOT_CXX=y
diff --git a/support/config-fragments/autobuild/br-xtensa-full.config b/support/config-fragments/autobuild/br-xtensa-full.config
new file mode 100644
index 0000000000..c98b3057cf
--- /dev/null
+++ b/support/config-fragments/autobuild/br-xtensa-full.config
@@ -0,0 +1,9 @@
+BR2_xtensa=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-xtensa-full-2017.05-1078-g95b1dae.tar.bz2"
+BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_12=y
+BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/i686-ctng-linux-gnu.config b/support/config-fragments/autobuild/i686-ctng-linux-gnu.config
new file mode 100644
index 0000000000..9a961123c3
--- /dev/null
+++ b/support/config-fragments/autobuild/i686-ctng-linux-gnu.config
@@ -0,0 +1,10 @@
+BR2_x86_i686=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/i686-ctng-linux-gnu.tar.xz"
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="i686-ctng-linux-gnu"
+BR2_TOOLCHAIN_EXTERNAL_GCC_4_8=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_9=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/linaro-aarch64.config b/support/config-fragments/autobuild/linaro-aarch64.config
new file mode 100644
index 0000000000..f65f78a39b
--- /dev/null
+++ b/support/config-fragments/autobuild/linaro-aarch64.config
@@ -0,0 +1,3 @@
+BR2_aarch64=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64=y
diff --git a/support/config-fragments/autobuild/linaro-arm.config b/support/config-fragments/autobuild/linaro-arm.config
new file mode 100644
index 0000000000..d72e19b3ad
--- /dev/null
+++ b/support/config-fragments/autobuild/linaro-arm.config
@@ -0,0 +1,5 @@
+BR2_arm=y
+BR2_cortex_a8=y
+BR2_ARM_EABIHF=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_LINARO_ARM=y
diff --git a/support/config-fragments/autobuild/mips64el-ctng_n32-linux-gnu.config b/support/config-fragments/autobuild/mips64el-ctng_n32-linux-gnu.config
new file mode 100644
index 0000000000..cf33ea26f3
--- /dev/null
+++ b/support/config-fragments/autobuild/mips64el-ctng_n32-linux-gnu.config
@@ -0,0 +1,9 @@
+BR2_mips64el=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mips64el-ctng_n32-linux-gnu.tar.xz"
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="mips64el-ctng_n32-linux-gnu"
+BR2_TOOLCHAIN_EXTERNAL_GCC_4_8=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_9=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/mips64el-ctng_n64-linux-gnu.config b/support/config-fragments/autobuild/mips64el-ctng_n64-linux-gnu.config
new file mode 100644
index 0000000000..942a1b13b5
--- /dev/null
+++ b/support/config-fragments/autobuild/mips64el-ctng_n64-linux-gnu.config
@@ -0,0 +1,11 @@
+BR2_mips64el=y
+BR2_MIPS_NABI64=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mips64el-ctng_n64-linux-gnu.tar.xz"
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="mips64el-ctng_n64-linux-gnu"
+BR2_TOOLCHAIN_EXTERNAL_GCC_4_8=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_9=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/powerpc-ctng_e500v2-linux-gnuspe.config b/support/config-fragments/autobuild/powerpc-ctng_e500v2-linux-gnuspe.config
new file mode 100644
index 0000000000..e82d3fbda7
--- /dev/null
+++ b/support/config-fragments/autobuild/powerpc-ctng_e500v2-linux-gnuspe.config
@@ -0,0 +1,11 @@
+BR2_powerpc=y
+BR2_powerpc_8548=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/powerpc-ctng_e500v2-linux-gnuspe.tar.xz"
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="powerpc-ctng_e500v2-linux-gnuspe"
+BR2_TOOLCHAIN_EXTERNAL_GCC_4_7=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_12=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
diff --git a/support/config-fragments/autobuild/sourcery-arm-armv4t.config b/support/config-fragments/autobuild/sourcery-arm-armv4t.config
new file mode 100644
index 0000000000..4c0e01fecb
--- /dev/null
+++ b/support/config-fragments/autobuild/sourcery-arm-armv4t.config
@@ -0,0 +1,4 @@
+BR2_arm=y
+BR2_arm920t=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM=y
diff --git a/support/config-fragments/autobuild/sourcery-arm-thumb2.config b/support/config-fragments/autobuild/sourcery-arm-thumb2.config
new file mode 100644
index 0000000000..e726757a16
--- /dev/null
+++ b/support/config-fragments/autobuild/sourcery-arm-thumb2.config
@@ -0,0 +1,7 @@
+BR2_arm=y
+BR2_cortex_a8=y
+BR2_ARM_EABI=y
+BR2_ARM_INSTRUCTIONS_THUMB2=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM=y
+BR2_TARGET_OPTIMIZATION=""
diff --git a/support/config-fragments/autobuild/sourcery-arm.config b/support/config-fragments/autobuild/sourcery-arm.config
new file mode 100644
index 0000000000..8ade4647f2
--- /dev/null
+++ b/support/config-fragments/autobuild/sourcery-arm.config
@@ -0,0 +1,3 @@
+BR2_arm=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_ARM=y
diff --git a/support/config-fragments/autobuild/sourcery-mips.config b/support/config-fragments/autobuild/sourcery-mips.config
new file mode 100644
index 0000000000..103e20bfc2
--- /dev/null
+++ b/support/config-fragments/autobuild/sourcery-mips.config
@@ -0,0 +1,4 @@
+BR2_mips=y
+BR2_mips_32r2=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_MIPS=y
diff --git a/support/config-fragments/autobuild/sourcery-mips64.config b/support/config-fragments/autobuild/sourcery-mips64.config
new file mode 100644
index 0000000000..77e3a853e1
--- /dev/null
+++ b/support/config-fragments/autobuild/sourcery-mips64.config
@@ -0,0 +1,5 @@
+BR2_mips64el=y
+BR2_mips_64r2=y
+BR2_MIPS_NABI64=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_MIPS=y
diff --git a/support/config-fragments/autobuild/sourcery-nios2.config b/support/config-fragments/autobuild/sourcery-nios2.config
new file mode 100644
index 0000000000..d58407d513
--- /dev/null
+++ b/support/config-fragments/autobuild/sourcery-nios2.config
@@ -0,0 +1,3 @@
+BR2_nios2=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_NIOSII=y
diff --git a/support/config-fragments/autobuild/sourcery-x86-64.config b/support/config-fragments/autobuild/sourcery-x86-64.config
new file mode 100644
index 0000000000..6c7ad4c5b7
--- /dev/null
+++ b/support/config-fragments/autobuild/sourcery-x86-64.config
@@ -0,0 +1,4 @@
+BR2_x86_64=y
+BR2_x86_steamroller=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_AMD64=y
diff --git a/support/config-fragments/autobuild/toolchain-configs.csv b/support/config-fragments/autobuild/toolchain-configs.csv
new file mode 100644
index 0000000000..15f6546cef
--- /dev/null
+++ b/support/config-fragments/autobuild/toolchain-configs.csv
@@ -0,0 +1,52 @@
+support/config-fragments/autobuild/armv5-ctng-linux-gnueabi.config,x86,glibc
+support/config-fragments/autobuild/armv7-ctng-linux-gnueabihf.config,x86,glibc
+support/config-fragments/autobuild/br-aarch64-glibc.config,x86_64,glibc
+support/config-fragments/autobuild/br-arc-full-internal.config,any,uclibc
+support/config-fragments/autobuild/br-arcle-hs38.config,x86_64,uclibc
+support/config-fragments/autobuild/br-arm-basic.config,x86_64,uclibc
+support/config-fragments/autobuild/br-arm-cortex-a9-glibc.config,x86_64,glibc
+support/config-fragments/autobuild/br-arm-cortex-a9-musl.config,x86_64,musl
+support/config-fragments/autobuild/br-arm-cortex-m4-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-arm-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-arm-full-nothread.config,x86_64,uclibc
+support/config-fragments/autobuild/br-arm-full-static.config,x86_64,uclibc
+support/config-fragments/autobuild/br-arm-internal-full.config,any,uclibc
+support/config-fragments/autobuild/br-bfin-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-i386-pentium4-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-i386-pentium-mmx-musl.config,x86_64,musl
+support/config-fragments/autobuild/br-m68k-5208-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-m68k-68040-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-microblazeel-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-microblazeel-full-internal.config,any,glibc
+support/config-fragments/autobuild/br-mips64-n64-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-mips32r6-el-hf-glibc.config,x86_64,glibc
+support/config-fragments/autobuild/br-mips64r6-el-hf-glibc.config,x86_64,glibc
+support/config-fragments/autobuild/br-mipsel-o32-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-nios2-glibc.config,x86_64,glibc
+support/config-fragments/autobuild/br-openrisc-uclibc.config,x86_64,uclibc
+support/config-fragments/autobuild/br-powerpc-603e-basic-cpp.config,x86_64,uclibc
+support/config-fragments/autobuild/br-powerpc64le-power8-glibc.config,x86_64,glibc
+support/config-fragments/autobuild/br-powerpc64-power7-glibc.config,x86_64,glibc
+support/config-fragments/autobuild/br-powerpc-e500mc-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-powerpc-internal-full.config,any,uclibc
+support/config-fragments/autobuild/br-sh4-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-sparc-uclibc.config,x86_64,uclibc
+support/config-fragments/autobuild/br-sparc64-glibc.config,x86_64,glibc
+support/config-fragments/autobuild/br-x86-64-core2-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-x86-64-musl.config,x86_64,musl
+support/config-fragments/autobuild/br-xtensa-full.config,x86_64,uclibc
+support/config-fragments/autobuild/br-xtensa-full-internal.config,any,uclibc
+support/config-fragments/autobuild/i686-ctng-linux-gnu.config,x86,glibc
+support/config-fragments/autobuild/linaro-aarch64.config,x86,glibc
+support/config-fragments/autobuild/linaro-arm.config,x86,glibc
+support/config-fragments/autobuild/mips64el-ctng_n32-linux-gnu.config,x86,glibc
+support/config-fragments/autobuild/mips64el-ctng_n64-linux-gnu.config,x86,glibc
+support/config-fragments/autobuild/powerpc-ctng_e500v2-linux-gnuspe.config,x86,glibc
+support/config-fragments/autobuild/sourcery-arm-armv4t.config,x86,glibc
+support/config-fragments/autobuild/sourcery-arm.config,x86,glibc
+support/config-fragments/autobuild/sourcery-arm-thumb2.config,x86,glibc
+support/config-fragments/autobuild/sourcery-mips64.config,x86,glibc
+support/config-fragments/autobuild/sourcery-mips.config,x86,glibc
+support/config-fragments/autobuild/sourcery-nios2.config,x86,glibc
+support/config-fragments/autobuild/sourcery-x86-64.config,x86,glibc
+support/config-fragments/autobuild/x86_64-ctng_locales-linux-gnu.config,x86,glibc
diff --git a/support/config-fragments/autobuild/x86_64-ctng_locales-linux-gnu.config b/support/config-fragments/autobuild/x86_64-ctng_locales-linux-gnu.config
new file mode 100644
index 0000000000..435034be27
--- /dev/null
+++ b/support/config-fragments/autobuild/x86_64-ctng_locales-linux-gnu.config
@@ -0,0 +1,11 @@
+BR2_x86_64=y
+BR2_x86_corei7=y
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/x86_64-ctng_locales-linux-gnu.tar.xz"
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="x86_64-ctng_locales-linux-gnu"
+BR2_TOOLCHAIN_EXTERNAL_GCC_4_8=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_9=y
+BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
+BR2_TOOLCHAIN_EXTERNAL_CXX=y
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 17/23] test-pkg: get configs from in-tree toolchain-configs.csv
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (15 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 16/23] support: add autobuild toolchain config fragments Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 18/23] support/test-pkg: add option to use an alternate toolchains CSV file Arnout Vandecappelle
` (7 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
Now we have the toolchain config fragments in the buildroot directory
itself, it is no longer necessary to fetch it from the toolchain URL.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
---
v7: Still use CSV file (instead of listing directory contents), but
use the in-tree one. Don't bother adding an option to specify an
alternative CSV file.
v6: Moved configs to support/config-fragments/autobuild
v5: first version
---
utils/test-pkg | 14 ++++----------
1 file changed, 4 insertions(+), 10 deletions(-)
diff --git a/utils/test-pkg b/utils/test-pkg
index 25a25e2048..8c60649b32 100755
--- a/utils/test-pkg
+++ b/utils/test-pkg
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
set -e
-TOOLCHAINS_URL='http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv'
+TOOLCHAINS_CSV='support/config-fragments/autobuild/toolchain-configs.csv'
main() {
local o O opts
@@ -50,8 +50,7 @@ main() {
# Extract the URLs of the toolchains; drop internal toolchains
# E.g.: http://server/path/to/name.config,arch,libc
# --> http://server/path/to/name.config
- toolchains=($(curl -s "${TOOLCHAINS_URL}" \
- |sed -r -e 's/,.*//; /internal/d;' \
+ toolchains=($(sed -r -e 's/,.*//; /internal/d;' "${TOOLCHAINS_CSV}" \
|if [ ${random} -gt 0 ]; then \
sort -R |head -n ${random}
else
@@ -71,7 +70,6 @@ main() {
nb_legal=0
for toolchainconfig in "${toolchains[@]}"; do
: $((nb++))
- # Using basename(1) on a URL works nicely
toolchain="$(basename "${toolchainconfig}" .config)"
build_dir="${dir}/${toolchain}"
printf "%40s [%*d/%d]: " "${toolchain}" ${#nb_tc} ${nb} ${nb_tc}
@@ -90,17 +88,13 @@ main() {
build_one() {
local dir="${1}"
- local url="${2}"
+ local toolchainconfig="${2}"
local cfg="${3}"
local pkg="${4}"
mkdir -p "${dir}"
- if ! curl -s "${url}" >"${dir}/.config"; then
- return 2
- fi
-
- cat "support/config-fragments/minimal.config" "${cfg}" >>"${dir}/.config"
+ cat "${toolchainconfig}" "support/config-fragments/minimal.config" "${cfg}" >>"${dir}/.config"
if ! make O="${dir}" olddefconfig > "${dir}/logfile" 2>&1; then
return 2
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 18/23] support/test-pkg: add option to use an alternate toolchains CSV file
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (16 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 17/23] test-pkg: get configs from in-tree toolchain-configs.csv Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-25 21:05 ` Thomas Petazzoni
2017-07-21 1:05 ` [Buildroot] [PATCH v7 19/23] genrandconfig: get configs from in-tree toolchain-configs.csv Arnout Vandecappelle
` (6 subsequent siblings)
24 siblings, 1 reply; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
For now, testing a package requires network access. However, there are
situations where everything is already cached locally (especially the
toolchains tarballs) and network is not available (e.g. in the train,
travelling back from FOSDEM...)
Alternatively, one may also want to test against a subset of the default
toolchains (e.g. the ones known to have a specific issue), or against
additional toolchains (e.g. the ones used within the company).
Add an option to use an alternate CSV file containing the config
fragments of toolchains to try.
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: CSV file instead of directory
v6: s/defconfig/config fragment/g
v5: First version
Loosely based on http://patchwork.ozlabs.org/patch/728395/ but with a
CSV file instead of URL, and completely rewritten help text.
---
utils/test-pkg | 26 ++++++++++++++++++++------
1 file changed, 20 insertions(+), 6 deletions(-)
diff --git a/utils/test-pkg b/utils/test-pkg
index 8c60649b32..7eb9c535c3 100755
--- a/utils/test-pkg
+++ b/utils/test-pkg
@@ -5,16 +5,17 @@ TOOLCHAINS_CSV='support/config-fragments/autobuild/toolchain-configs.csv'
main() {
local o O opts
- local cfg dir pkg random toolchain
+ local cfg dir pkg random toolchains_dir toolchain
local ret nb nb_skip nb_fail nb_legal nb_tc build_dir
local -a toolchains
- o='hc:d:p:r:'
- O='help,config-snippet:build-dir:package:,random:'
+ o='hc:d:p:r:t:'
+ O='help,config-snippet:build-dir:package:,random:,toolchains-dir:'
opts="$(getopt -n "${my_name}" -o "${o}" -l "${O}" -- "${@}")"
eval set -- "${opts}"
random=0
+ toolchains_csv="${TOOLCHAINS_CSV}"
while [ ${#} -gt 0 ]; do
case "${1}" in
(-h|--help)
@@ -32,6 +33,9 @@ main() {
(-r|--random)
random="${2}"; shift 2
;;
+ (-t|--toolchains-csv)
+ toolchains_csv="${2}"; shift 2
+ ;;
(--)
shift; break
;;
@@ -50,7 +54,7 @@ main() {
# Extract the URLs of the toolchains; drop internal toolchains
# E.g.: http://server/path/to/name.config,arch,libc
# --> http://server/path/to/name.config
- toolchains=($(sed -r -e 's/,.*//; /internal/d;' "${TOOLCHAINS_CSV}" \
+ toolchains=($(sed -r -e 's/,.*//; /internal/d;' "${toolchains_csv}" \
|if [ ${random} -gt 0 ]; then \
sort -R |head -n ${random}
else
@@ -146,8 +150,13 @@ In case failures are noticed, you can fix the package and just re-run the
same command again; it will re-run the test where it failed. If you did
specify a package (with -p), the package build dir will be removed first.
-The list of toolchains is retrieved from the Buildroot autobuilders, available
-at ${TOOLCHAINS_URL}.
+The list of toolchains is retrieved from ${TOOLCHAINS_CSV}.
+Only the external toolchains are tried, because building a Buildroot toolchain
+would take too long. An alternative toolchains CSV file can be specified with
+the -t option. This file should have lines consisting of the path to the
+toolchain config fragment and the required host architecture, separated by a
+comma. The config fragments should contain only the toolchain and architecture
+settings.
Options:
@@ -169,6 +178,11 @@ Options:
Limit the tests to the N randomly selected toolchains, instead of
building with all toolchains.
+ -t CSVFILE, --toolchains-csv CSVFILE
+ CSV file containing the paths to config fragments of toolchains to
+ try. If not specified, the toolchains in ${TOOLCHAINS_CSV} will be
+ used.
+
Example:
Testing libcec would require a config snippet that contains:
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 18/23] support/test-pkg: add option to use an alternate toolchains CSV file
2017-07-21 1:05 ` [Buildroot] [PATCH v7 18/23] support/test-pkg: add option to use an alternate toolchains CSV file Arnout Vandecappelle
@ 2017-07-25 21:05 ` Thomas Petazzoni
0 siblings, 0 replies; 34+ messages in thread
From: Thomas Petazzoni @ 2017-07-25 21:05 UTC (permalink / raw)
To: buildroot
Hello,
On Fri, 21 Jul 2017 03:05:25 +0200, Arnout Vandecappelle
(Essensium/Mind) wrote:
> For now, testing a package requires network access. However, there are
I don't see what network access has to do with this. Thanks to your
previous commit, the toolchain config CSV file is local, so there is no
longer the need for network access.
Isn't that part of the commit log somewhat outdated ?
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 34+ messages in thread
* [Buildroot] [PATCH v7 19/23] genrandconfig: get configs from in-tree toolchain-configs.csv
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (17 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 18/23] support/test-pkg: add option to use an alternate toolchains CSV file Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 20/23] toolchain-configs.csv: remove unused libc column Arnout Vandecappelle
` (5 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
Now we have the toolchain config fragments in the buildroot directory
itself, it is no longer necessary to fetch it from the toolchain URL.
The --toolchains-url option is renamed to --toolchains-csv.
The paths in the toolchains_csv file should be either absolute, or
relative to buildrootdir.
After this change, the script should be called from autobuild-run as:
subprocess.call([os.path.join(srcdir, "utils/genrandconfig"),
"-o", outputdir, "-b", srcdir,
"--toolchains-csv", kwargs['toolchains_csv']],
stdout=devnull, stderr=log)
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
utils/genrandconfig | 25 ++++++++++++++-----------
1 file changed, 14 insertions(+), 11 deletions(-)
diff --git a/utils/genrandconfig b/utils/genrandconfig
index d397b23f36..a67d46fad9 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -118,15 +118,15 @@ class SystemInfo:
return not missing_requirements
-def get_toolchain_configs(toolchains_url):
+def get_toolchain_configs(toolchains_csv, buildrootdir):
"""Fetch and return the possible toolchain configurations
This function returns an array of toolchain configurations. Each
toolchain configuration is itself an array of lines of the defconfig.
"""
- with urlopen_closing(toolchains_url) as r:
- toolchains_csv = decode_byte_list(r.readlines())
+ with open(toolchains_csv) as r:
+ toolchains = decode_byte_list(r.readlines())
configs = []
(_, _, _, _, hostarch) = os.uname()
@@ -134,9 +134,9 @@ def get_toolchain_configs(toolchains_url):
if hostarch == 'i686' or hostarch == 'i386' or hostarch == 'x86':
hostarch = 'x86'
- for row in csv.reader(toolchains_csv):
+ for row in csv.reader(toolchains):
config = {}
- url = row[0]
+ configfile = row[0]
config_hostarch = row[1]
keep = False
@@ -158,8 +158,11 @@ def get_toolchain_configs(toolchains_url):
if not keep:
continue
- with urlopen_closing(url) as r:
- config = decode_byte_list(r.readlines())
+ if not os.path.isabs(configfile):
+ configfile = os.path.join(buildrootdir, configfile)
+
+ with open(configfile) as r:
+ config = r.readlines()
configs.append(config)
return configs
@@ -329,7 +332,7 @@ def gen_config(args):
"""
# Select a random toolchain configuration
- configs = get_toolchain_configs(args.toolchains_url)
+ configs = get_toolchain_configs(args.toolchains_csv, args.buildrootdir)
i = randint(0, len(configs) - 1)
toolchainconfig = configs[i]
@@ -408,10 +411,10 @@ if __name__ == '__main__':
parser.add_argument("--buildrootdir", "-b",
help="Buildroot directory (relative to current directory)",
type=str, default='.')
- parser.add_argument("--toolchains-url",
- help="URL of toolchain configuration file",
+ parser.add_argument("--toolchains-csv",
+ help="Path of the toolchain configuration file",
type=str,
- default="http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv")
+ default="support/config-fragments/autobuild/toolchain-configs.csv")
args = parser.parse_args()
# We need the absolute path to use with O=, because the relative
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 20/23] toolchain-configs.csv: remove unused libc column
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (18 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 19/23] genrandconfig: get configs from in-tree toolchain-configs.csv Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 21/23] Makefile: refactor *config targets Arnout Vandecappelle
` (4 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
This column is not used by either genrandconfig or test-pkg, so remove
it.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: new patch
---
.../autobuild/toolchain-configs.csv | 104 ++++++++++-----------
1 file changed, 52 insertions(+), 52 deletions(-)
diff --git a/support/config-fragments/autobuild/toolchain-configs.csv b/support/config-fragments/autobuild/toolchain-configs.csv
index 15f6546cef..80a6d04a35 100644
--- a/support/config-fragments/autobuild/toolchain-configs.csv
+++ b/support/config-fragments/autobuild/toolchain-configs.csv
@@ -1,52 +1,52 @@
-support/config-fragments/autobuild/armv5-ctng-linux-gnueabi.config,x86,glibc
-support/config-fragments/autobuild/armv7-ctng-linux-gnueabihf.config,x86,glibc
-support/config-fragments/autobuild/br-aarch64-glibc.config,x86_64,glibc
-support/config-fragments/autobuild/br-arc-full-internal.config,any,uclibc
-support/config-fragments/autobuild/br-arcle-hs38.config,x86_64,uclibc
-support/config-fragments/autobuild/br-arm-basic.config,x86_64,uclibc
-support/config-fragments/autobuild/br-arm-cortex-a9-glibc.config,x86_64,glibc
-support/config-fragments/autobuild/br-arm-cortex-a9-musl.config,x86_64,musl
-support/config-fragments/autobuild/br-arm-cortex-m4-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-arm-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-arm-full-nothread.config,x86_64,uclibc
-support/config-fragments/autobuild/br-arm-full-static.config,x86_64,uclibc
-support/config-fragments/autobuild/br-arm-internal-full.config,any,uclibc
-support/config-fragments/autobuild/br-bfin-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-i386-pentium4-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-i386-pentium-mmx-musl.config,x86_64,musl
-support/config-fragments/autobuild/br-m68k-5208-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-m68k-68040-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-microblazeel-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-microblazeel-full-internal.config,any,glibc
-support/config-fragments/autobuild/br-mips64-n64-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-mips32r6-el-hf-glibc.config,x86_64,glibc
-support/config-fragments/autobuild/br-mips64r6-el-hf-glibc.config,x86_64,glibc
-support/config-fragments/autobuild/br-mipsel-o32-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-nios2-glibc.config,x86_64,glibc
-support/config-fragments/autobuild/br-openrisc-uclibc.config,x86_64,uclibc
-support/config-fragments/autobuild/br-powerpc-603e-basic-cpp.config,x86_64,uclibc
-support/config-fragments/autobuild/br-powerpc64le-power8-glibc.config,x86_64,glibc
-support/config-fragments/autobuild/br-powerpc64-power7-glibc.config,x86_64,glibc
-support/config-fragments/autobuild/br-powerpc-e500mc-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-powerpc-internal-full.config,any,uclibc
-support/config-fragments/autobuild/br-sh4-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-sparc-uclibc.config,x86_64,uclibc
-support/config-fragments/autobuild/br-sparc64-glibc.config,x86_64,glibc
-support/config-fragments/autobuild/br-x86-64-core2-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-x86-64-musl.config,x86_64,musl
-support/config-fragments/autobuild/br-xtensa-full.config,x86_64,uclibc
-support/config-fragments/autobuild/br-xtensa-full-internal.config,any,uclibc
-support/config-fragments/autobuild/i686-ctng-linux-gnu.config,x86,glibc
-support/config-fragments/autobuild/linaro-aarch64.config,x86,glibc
-support/config-fragments/autobuild/linaro-arm.config,x86,glibc
-support/config-fragments/autobuild/mips64el-ctng_n32-linux-gnu.config,x86,glibc
-support/config-fragments/autobuild/mips64el-ctng_n64-linux-gnu.config,x86,glibc
-support/config-fragments/autobuild/powerpc-ctng_e500v2-linux-gnuspe.config,x86,glibc
-support/config-fragments/autobuild/sourcery-arm-armv4t.config,x86,glibc
-support/config-fragments/autobuild/sourcery-arm.config,x86,glibc
-support/config-fragments/autobuild/sourcery-arm-thumb2.config,x86,glibc
-support/config-fragments/autobuild/sourcery-mips64.config,x86,glibc
-support/config-fragments/autobuild/sourcery-mips.config,x86,glibc
-support/config-fragments/autobuild/sourcery-nios2.config,x86,glibc
-support/config-fragments/autobuild/sourcery-x86-64.config,x86,glibc
-support/config-fragments/autobuild/x86_64-ctng_locales-linux-gnu.config,x86,glibc
+support/config-fragments/autobuild/armv5-ctng-linux-gnueabi.config,x86
+support/config-fragments/autobuild/armv7-ctng-linux-gnueabihf.config,x86
+support/config-fragments/autobuild/br-aarch64-glibc.config,x86_64
+support/config-fragments/autobuild/br-arc-full-internal.config,any
+support/config-fragments/autobuild/br-arcle-hs38.config,x86_64
+support/config-fragments/autobuild/br-arm-basic.config,x86_64
+support/config-fragments/autobuild/br-arm-cortex-a9-glibc.config,x86_64
+support/config-fragments/autobuild/br-arm-cortex-a9-musl.config,x86_64
+support/config-fragments/autobuild/br-arm-cortex-m4-full.config,x86_64
+support/config-fragments/autobuild/br-arm-full.config,x86_64
+support/config-fragments/autobuild/br-arm-full-nothread.config,x86_64
+support/config-fragments/autobuild/br-arm-full-static.config,x86_64
+support/config-fragments/autobuild/br-arm-internal-full.config,any
+support/config-fragments/autobuild/br-bfin-full.config,x86_64
+support/config-fragments/autobuild/br-i386-pentium4-full.config,x86_64
+support/config-fragments/autobuild/br-i386-pentium-mmx-musl.config,x86_64
+support/config-fragments/autobuild/br-m68k-5208-full.config,x86_64
+support/config-fragments/autobuild/br-m68k-68040-full.config,x86_64
+support/config-fragments/autobuild/br-microblazeel-full.config,x86_64
+support/config-fragments/autobuild/br-microblazeel-full-internal.config,any
+support/config-fragments/autobuild/br-mips64-n64-full.config,x86_64
+support/config-fragments/autobuild/br-mips32r6-el-hf-glibc.config,x86_64
+support/config-fragments/autobuild/br-mips64r6-el-hf-glibc.config,x86_64
+support/config-fragments/autobuild/br-mipsel-o32-full.config,x86_64
+support/config-fragments/autobuild/br-nios2-glibc.config,x86_64
+support/config-fragments/autobuild/br-openrisc-uclibc.config,x86_64
+support/config-fragments/autobuild/br-powerpc-603e-basic-cpp.config,x86_64
+support/config-fragments/autobuild/br-powerpc64le-power8-glibc.config,x86_64
+support/config-fragments/autobuild/br-powerpc64-power7-glibc.config,x86_64
+support/config-fragments/autobuild/br-powerpc-e500mc-full.config,x86_64
+support/config-fragments/autobuild/br-powerpc-internal-full.config,any
+support/config-fragments/autobuild/br-sh4-full.config,x86_64
+support/config-fragments/autobuild/br-sparc-uclibc.config,x86_64
+support/config-fragments/autobuild/br-sparc64-glibc.config,x86_64
+support/config-fragments/autobuild/br-x86-64-core2-full.config,x86_64
+support/config-fragments/autobuild/br-x86-64-musl.config,x86_64
+support/config-fragments/autobuild/br-xtensa-full.config,x86_64
+support/config-fragments/autobuild/br-xtensa-full-internal.config,any
+support/config-fragments/autobuild/i686-ctng-linux-gnu.config,x86
+support/config-fragments/autobuild/linaro-aarch64.config,x86
+support/config-fragments/autobuild/linaro-arm.config,x86
+support/config-fragments/autobuild/mips64el-ctng_n32-linux-gnu.config,x86
+support/config-fragments/autobuild/mips64el-ctng_n64-linux-gnu.config,x86
+support/config-fragments/autobuild/powerpc-ctng_e500v2-linux-gnuspe.config,x86
+support/config-fragments/autobuild/sourcery-arm-armv4t.config,x86
+support/config-fragments/autobuild/sourcery-arm.config,x86
+support/config-fragments/autobuild/sourcery-arm-thumb2.config,x86
+support/config-fragments/autobuild/sourcery-mips64.config,x86
+support/config-fragments/autobuild/sourcery-mips.config,x86
+support/config-fragments/autobuild/sourcery-nios2.config,x86
+support/config-fragments/autobuild/sourcery-x86-64.config,x86
+support/config-fragments/autobuild/x86_64-ctng_locales-linux-gnu.config,x86
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 21/23] Makefile: refactor *config targets
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (19 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 20/23] toolchain-configs.csv: remove unused libc column Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 22/23] Makefile: add alldefconfig target Arnout Vandecappelle
` (3 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
The rules for the *config targets are all very similar, so factor them
together using $@.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: No changes
v6: No changes
v5: First version
---
Makefile | 42 ++++++------------------------------------
1 file changed, 6 insertions(+), 36 deletions(-)
diff --git a/Makefile b/Makefile
index d4faa021cd..c8912c9da4 100644
--- a/Makefile
+++ b/Makefile
@@ -877,50 +877,20 @@ config: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
# no values are set for the legacy options so a subsequent oldconfig
# will query them. Therefore, run an additional olddefconfig.
-oldconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
- @$(COMMON_CONFIG_ENV) $< --oldconfig $(CONFIG_CONFIG_IN)
-
-randconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
- @$(COMMON_CONFIG_ENV) SKIP_LEGACY=y $< --randconfig $(CONFIG_CONFIG_IN)
- @$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
-
-allyesconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
- @$(COMMON_CONFIG_ENV) SKIP_LEGACY=y $< --allyesconfig $(CONFIG_CONFIG_IN)
- @$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
-
-allnoconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
- @$(COMMON_CONFIG_ENV) SKIP_LEGACY=y $< --allnoconfig $(CONFIG_CONFIG_IN)
- @$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
-
-randpackageconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
- @grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
- @$(COMMON_CONFIG_ENV) SKIP_LEGACY=y \
- KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
- $< --randconfig $(CONFIG_CONFIG_IN)
- @rm -f $(CONFIG_DIR)/.config.nopkg
+randconfig allyesconfig allnoconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
+ @$(COMMON_CONFIG_ENV) SKIP_LEGACY=y $< --$@ $(CONFIG_CONFIG_IN)
@$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
-allyespackageconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
+randpackageconfig allyespackageconfig allnopackageconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
@grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
@$(COMMON_CONFIG_ENV) SKIP_LEGACY=y \
KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
- $< --allyesconfig $(CONFIG_CONFIG_IN)
+ $< --$(subst package,,$@) $(CONFIG_CONFIG_IN)
@rm -f $(CONFIG_DIR)/.config.nopkg
@$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
-allnopackageconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
- @grep -v BR2_PACKAGE_ $(BR2_CONFIG) > $(CONFIG_DIR)/.config.nopkg
- @$(COMMON_CONFIG_ENV) SKIP_LEGACY=y \
- KCONFIG_ALLCONFIG=$(CONFIG_DIR)/.config.nopkg \
- $< --allnoconfig $(CONFIG_CONFIG_IN)
- @rm -f $(CONFIG_DIR)/.config.nopkg
- @$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
-
-silentoldconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
- $(COMMON_CONFIG_ENV) $< --silentoldconfig $(CONFIG_CONFIG_IN)
-
-olddefconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
- $(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN)
+oldconfig silentoldconfig olddefconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
+ @$(COMMON_CONFIG_ENV) $< --$@ $(CONFIG_CONFIG_IN)
defconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 22/23] Makefile: add alldefconfig target
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (20 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 21/23] Makefile: refactor *config targets Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:05 ` [Buildroot] [PATCH v7 23/23] test-pkg: use merge_config.sh to merge the fragments Arnout Vandecappelle
` (2 subsequent siblings)
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
It is used by Kconfig's merge_config.sh.
No alldefpackageconfig is added, since it's rather pointless: it would
only enable busybox.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
v7: No changes
v6: No changes
v5: First version
---
Makefile | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/Makefile b/Makefile
index c8912c9da4..66b62e2a18 100644
--- a/Makefile
+++ b/Makefile
@@ -128,7 +128,7 @@ export BR2_VERSION_FULL := $(BR2_VERSION)$(shell $(TOPDIR)/support/scripts/setlo
# List of targets and target patterns for which .config doesn't need to be read in
noconfig_targets := menuconfig nconfig gconfig xconfig config oldconfig randconfig \
- defconfig %_defconfig allyesconfig allnoconfig silentoldconfig release \
+ defconfig %_defconfig allyesconfig allnoconfig alldefconfig silentoldconfig release \
randpackageconfig allyespackageconfig allnopackageconfig \
print-version olddefconfig distclean manual manual-%
@@ -877,7 +877,7 @@ config: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
# no values are set for the legacy options so a subsequent oldconfig
# will query them. Therefore, run an additional olddefconfig.
-randconfig allyesconfig allnoconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
+randconfig allyesconfig alldefconfig allnoconfig: $(BUILD_DIR)/buildroot-config/conf prepare-kconfig
@$(COMMON_CONFIG_ENV) SKIP_LEGACY=y $< --$@ $(CONFIG_CONFIG_IN)
@$(COMMON_CONFIG_ENV) $< --olddefconfig $(CONFIG_CONFIG_IN) >/dev/null
@@ -986,6 +986,7 @@ help:
@echo ' savedefconfig - Save current config to BR2_DEFCONFIG (minimal config)'
@echo ' allyesconfig - New config where all options are accepted with yes'
@echo ' allnoconfig - New config where all options are answered with no'
+ @echo ' alldefconfig - New config where all options are set to default'
@echo ' randpackageconfig - New config with random answer to package options'
@echo ' allyespackageconfig - New config where pkg options are accepted with yes'
@echo ' allnopackageconfig - New config where package options are answered with no'
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v7 23/23] test-pkg: use merge_config.sh to merge the fragments
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (21 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 22/23] Makefile: add alldefconfig target Arnout Vandecappelle
@ 2017-07-21 1:05 ` Arnout Vandecappelle
2017-07-21 1:06 ` [Buildroot] [PATCH v3 1/4] autobuild-run: use in-tree genrandconfig script (initial version) Arnout Vandecappelle
2017-07-25 21:10 ` [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Thomas Petazzoni
24 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:05 UTC (permalink / raw)
To: buildroot
It is supposedly more robust than just concatenating.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
---
v7: Rebased
v6: Rebased
v5: First version
---
utils/test-pkg | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/utils/test-pkg b/utils/test-pkg
index 7eb9c535c3..6899d44f7c 100755
--- a/utils/test-pkg
+++ b/utils/test-pkg
@@ -98,11 +98,9 @@ build_one() {
mkdir -p "${dir}"
- cat "${toolchainconfig}" "support/config-fragments/minimal.config" "${cfg}" >>"${dir}/.config"
-
- if ! make O="${dir}" olddefconfig > "${dir}/logfile" 2>&1; then
- return 2
- fi
+ support/kconfig/merge_config.sh -O "${dir}" \
+ "${toolchainconfig}" "support/config-fragments/minimal.config" "${cfg}" \
+ > /dev/null
# We want all the options from the snippet to be present as-is (set
# or not set) in the actual .config; if one of them is not, it means
# some dependency from the toolchain or arch is not available, in
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v3 1/4] autobuild-run: use in-tree genrandconfig script (initial version)
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (22 preceding siblings ...)
2017-07-21 1:05 ` [Buildroot] [PATCH v7 23/23] test-pkg: use merge_config.sh to merge the fragments Arnout Vandecappelle
@ 2017-07-21 1:06 ` Arnout Vandecappelle
2017-07-21 1:06 ` [Buildroot] [PATCH v3 2/4] autobuild-run: adapt to genrandconfig without log_write Arnout Vandecappelle
` (3 more replies)
2017-07-25 21:10 ` [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Thomas Petazzoni
24 siblings, 4 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:06 UTC (permalink / raw)
To: buildroot
The Buildroot tree now has a genrandconfig script that replaces a
large part of the autobuild-run script. Call this script rather than
having it in autobuild-run.
The genrandconfig script inside the Buildroot tree has the advantage
that it can be adapted to the changes in the source, without the
need to update the autobuild-run script all the time. Also, it makes
it feasible to support autobuilding multiple branches.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
scripts/autobuild-run | 301 +-------------------------------------------------
1 file changed, 6 insertions(+), 295 deletions(-)
diff --git a/scripts/autobuild-run b/scripts/autobuild-run
index a7d7d4f..879c9c3 100755
--- a/scripts/autobuild-run
+++ b/scripts/autobuild-run
@@ -155,9 +155,6 @@ if sys.hexversion >= 0x3000000:
def decode_bytes(b):
return b.decode()
- def decode_byte_list(bl):
- return [b.decode() for b in bl]
-
def encode_str(s):
return s.encode()
else:
@@ -165,7 +162,6 @@ else:
return e
decode_bytes = _identity
- decode_byte_list = _identity
encode_str = _identity
MAX_DURATION = 60 * 60 * 8
@@ -249,52 +245,6 @@ class SystemInfo:
return not missing_requirements
-def get_toolchain_configs(**kwargs):
- """Fetch and return the possible toolchain configurations
-
- This function returns an array of toolchain configurations. Each
- toolchain configuration is itself an array of lines of the defconfig.
- """
- toolchains_url = kwargs['toolchains_url']
-
- with urlopen_closing(toolchains_url) as r:
- l = decode_byte_list(r.readlines())
- configs = []
-
- (_, _, _, _, hostarch) = os.uname()
- # ~2015 distros report x86 when on a 32bit install
- if hostarch == 'i686' or hostarch == 'i386' or hostarch == 'x86':
- hostarch = 'x86'
-
- for row in csv.reader(l):
- config = {}
- url = row[0]
- config_hostarch = row[1]
- keep = False
-
- # Keep all toolchain configs that work regardless of the host
- # architecture
- if config_hostarch == "any":
- keep = True
-
- # Keep all toolchain configs that can work on the current host
- # architecture
- if hostarch == config_hostarch:
- keep = True
-
- # Assume that x86 32 bits toolchains work on x86_64 build
- # machines
- if hostarch == 'x86_64' and config_hostarch == "x86":
- keep = True
-
- if not keep:
- continue
-
- with urlopen_closing(url) as r:
- config = decode_byte_list(r.readlines())
- configs.append(config)
- return configs
-
def prepare_build(**kwargs):
"""Prepare for the next build of the specified instance
@@ -351,256 +301,17 @@ def prepare_build(**kwargs):
return 0
-def is_toolchain_usable(**kwargs):
- """Check if the toolchain is actually usable."""
-
- idir = "instance-%d" % kwargs['instance']
- sysinfo = kwargs['sysinfo']
- log = kwargs['log']
-
- outputdir = os.path.join(idir, "output")
- with open(os.path.join(outputdir, ".config")) as configf:
- configlines = configf.readlines()
-
- # Check that the toolchain configuration is still present
- for toolchainline in kwargs['config']:
- if toolchainline not in configlines:
- return False
-
- # The latest Linaro toolchains on x86-64 hosts requires glibc
- # 2.14+ on the host.
- if platform.machine() == 'x86_64':
- if 'BR2_TOOLCHAIN_EXTERNAL_LINARO_ARM=y\n' in configlines or \
- 'BR2_TOOLCHAIN_EXTERNAL_LINARO_AARCH64=y\n' in configlines or \
- 'BR2_TOOLCHAIN_EXTERNAL_LINARO_ARMEB=y\n' in configlines:
- ldd_version_output = subprocess.Popen(['ldd', '--version'], stdout=subprocess.PIPE).communicate()[0]
- glibc_version = ldd_version_output.splitlines()[0].split()[-1]
- if StrictVersion('2.14') > StrictVersion(glibc_version):
- log_write(log, "WARN: ignoring the Linaro ARM toolchains becausee too old host glibc")
- return False
-
- return True
-
-def fixup_config(**kwargs):
- """Finalize the configuration and reject any problematic combinations
-
- This function returns 'True' when the configuration has been
- accepted, and 'False' when the configuration has not been accepted because
- it is known to fail (in which case another random configuration will be
- generated).
- """
-
- idir = "instance-%d" % kwargs['instance']
- sysinfo = kwargs['sysinfo']
-
- outputdir = os.path.join(idir, "output")
- with open(os.path.join(outputdir, ".config")) as configf:
- configlines = configf.readlines()
-
- if "BR2_NEEDS_HOST_JAVA=y\n" in configlines and not sysinfo.has("java"):
- return False
- if "BR2_NEEDS_HOST_JAVAC=y\n" in configlines and not sysinfo.has("javac"):
- return False
- if "BR2_NEEDS_HOST_JAR=y\n" in configlines and not sysinfo.has("jar"):
- return False
- # python-nfc needs bzr
- if 'BR2_PACKAGE_PYTHON_NFC=y\n' in configlines and not sysinfo.has("bzr"):
- return False
- # The ctng toolchain is affected by PR58854
- if 'BR2_PACKAGE_LTTNG_TOOLS=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/armv5-ctng-linux-gnueabi.tar.xz"\n' in configlines:
- return False
- # The ctng toolchain tigger an assembler error with guile package when compiled with -Os (same issue as for CS ARM 2014.05-29)
- if 'BR2_PACKAGE_GUILE=y\n' in configlines and 'BR2_OPTIMIZE_S=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/armv5-ctng-linux-gnueabi.tar.xz"\n' in configlines:
- return False
- # The ctng toolchain is affected by PR58854
- if 'BR2_PACKAGE_LTTNG_TOOLS=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/armv6-ctng-linux-uclibcgnueabi.tar.xz"\n' in configlines:
- return False
- # The ctng toolchain is affected by PR58854
- if 'BR2_PACKAGE_LTTNG_TOOLS=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/armv7-ctng-linux-gnueabihf.tar.xz"\n' in configlines:
- return False
- # The ctng toolchain is affected by PR60155
- if 'BR2_PACKAGE_SDL=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/powerpc-ctng-linux-uclibc.tar.xz"\n' in configlines:
- return False
- # The ctng toolchain is affected by PR60155
- if 'BR2_PACKAGE_LIBMPEG2=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/powerpc-ctng-linux-uclibc.tar.xz"\n' in configlines:
- return False
- # This MIPS toolchain uses eglibc-2.18 which lacks SYS_getdents64
- if 'BR2_PACKAGE_STRONGSWAN=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mips64el-ctng_n64-linux-gnu.tar.xz"\n' in configlines:
- return False
- # This MIPS toolchain uses eglibc-2.18 which lacks SYS_getdents64
- if 'BR2_PACKAGE_PYTHON3=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mips64el-ctng_n64-linux-gnu.tar.xz"\n' in configlines:
- return False
- # libffi not available on sh2a and ARMv7-M, but propagating libffi
- # arch dependencies in Buildroot is really too much work, so we
- # handle this here.
- if 'BR2_sh2a=y\n' in configlines and 'BR2_PACKAGE_LIBFFI=y\n' in configlines:
- return False
- if 'BR2_ARM_CPU_ARMV7M=y\n' in configlines and 'BR2_PACKAGE_LIBFFI=y\n' in configlines:
- return False
- if 'BR2_PACKAGE_SUNXI_BOARDS=y\n' in configlines:
- configlines.remove('BR2_PACKAGE_SUNXI_BOARDS_FEX_FILE=""\n')
- configlines.append('BR2_PACKAGE_SUNXI_BOARDS_FEX_FILE="a10/hackberry.fex"\n')
- # This MIPS uClibc toolchain fails to build the gdb package
- if 'BR2_PACKAGE_GDB=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mipsel-ctng-linux-uclibc.tar.xz"\n' in configlines:
- return False
- # This MIPS uClibc toolchain fails to build the rt-tests package
- if 'BR2_PACKAGE_RT_TESTS=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mipsel-ctng-linux-uclibc.tar.xz"\n' in configlines:
- return False
- # This MIPS uClibc toolchain fails to build the civetweb package
- if 'BR2_PACKAGE_CIVETWEB=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mipsel-ctng-linux-uclibc.tar.xz"\n' in configlines:
- return False
- # This MIPS ctng toolchain fails to build the python3 package
- if 'BR2_PACKAGE_PYTHON3=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mips64el-ctng_n64-linux-gnu.tar.xz"\n' in configlines:
- return False
- # This MIPS uClibc toolchain fails to build the strace package
- if 'BR2_PACKAGE_STRACE=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mipsel-ctng-linux-uclibc.tar.xz"\n' in configlines:
- return False
- # This MIPS uClibc toolchain fails to build the cdrkit package
- if 'BR2_PACKAGE_CDRKIT=y\n' in configlines and \
- 'BR2_STATIC_LIBS=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mipsel-ctng-linux-uclibc.tar.xz"\n' in configlines:
- return False
- # uClibc vfork static linking issue
- if 'BR2_PACKAGE_ALSA_LIB=y\n' in configlines and \
- 'BR2_STATIC_LIBS=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/i486-ctng-linux-uclibc.tar.xz"\n' in configlines:
- return False
- # This MIPS uClibc toolchain fails to build the weston package
- if 'BR2_PACKAGE_WESTON=y\n' in configlines and \
- 'BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/mipsel-ctng-linux-uclibc.tar.xz"\n' in configlines:
- return False
- # The cs nios2 2017.02 toolchain is affected by binutils PR19405
- if 'BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_NIOSII=y\n' in configlines and \
- 'BR2_PACKAGE_BOOST=y\n' in configlines:
- return False
- # The cs nios2 2017.02 toolchain is affected by binutils PR19405
- if 'BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_NIOSII=y\n' in configlines and \
- 'BR2_PACKAGE_QT5BASE_GUI=y\n' in configlines:
- return False
- # The cs nios2 2017.02 toolchain is affected by binutils PR19405
- if 'BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_NIOSII=y\n' in configlines and \
- 'BR2_PACKAGE_QT_GUI_MODULE=y\n' in configlines:
- return False
- # The cs nios2 2017.02 toolchain is affected by binutils PR19405
- if 'BR2_TOOLCHAIN_EXTERNAL_CODESOURCERY_NIOSII=y\n' in configlines and \
- 'BR2_PACKAGE_FLANN=y\n' in configlines:
- return False
- # or1k affected by binutils PR21464
- if 'BR2_or1k=y\n' in configlines and \
- 'BR2_PACKAGE_QT_GUI_MODULE=y\n' in configlines:
- return False
-
- with open(os.path.join(outputdir, ".config"), "w+") as configf:
- configf.writelines(configlines)
-
- return True
-
def gen_config(**kwargs):
- """Generate a new random configuration
-
- This function generates the configuration, by choosing a random
- toolchain configuration and then generating a random selection of
- packages.
- """
-
+ """Generate a new random configuration."""
idir = "instance-%d" % kwargs['instance']
log = kwargs['log']
-
- # We need the absolute path to use with O=, because the relative
- # path to the output directory here is not relative to the
- # Buildroot sources, but to the location of the autobuilder
- # script.
- outputdir = os.path.abspath(os.path.join(idir, "output"))
srcdir = os.path.join(idir, "buildroot")
- log_write(log, "INFO: generate the configuration")
-
- # Select a random toolchain configuration
- try:
- configs = get_toolchain_configs(**kwargs)
- except:
- return -1
-
- i = randint(0, len(configs) - 1)
- config = configs[i]
-
- configlines = config
-
- # Amend the configuration with a few things.
- configlines.append("BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y\n")
- configlines.append("# BR2_TARGET_ROOTFS_TAR is not set\n")
- configlines.append("BR2_COMPILER_PARANOID_UNSAFE_PATH=y\n")
- if randint(0, 20) == 0:
- configlines.append("BR2_ENABLE_DEBUG=y\n")
- if randint(0, 30) == 0:
- configlines.append("BR2_INIT_SYSTEMD=y\n")
- elif randint(0, 20) == 0:
- configlines.append("BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV=y\n")
- if randint(0, 20) == 0:
- configlines.append("BR2_STATIC_LIBS=y\n")
- if randint(0, 20) == 0:
- configlines.append("BR2_PACKAGE_PYTHON_PY_ONLY=y\n")
-
- # Write out the configuration file
- with open(os.path.join(outputdir, ".config"), "w+") as configf:
- configf.writelines(configlines)
-
- devnull = open(os.devnull, "w")
-
- ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir, "olddefconfig"],
- stdout=devnull, stderr=devnull)
- if ret != 0:
- log_write(log, "ERROR: cannot oldconfig")
- return -1
-
- if not is_toolchain_usable(config=config, **kwargs):
- return -1
-
- # Now, generate the random selection of packages, and fixup
- # things if needed.
- # Safe-guard, in case we can not quickly come to a valid
- # configuration: allow at most 100 (arbitrary) iterations.
- bounded_loop = 100
- while True:
- if bounded_loop == 0:
- log_write(log, "ERROR: cannot generate random configuration after 100 iterations")
- return -1
- bounded_loop -= 1
- ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,
- "KCONFIG_PROBABILITY=%d" % randint(1,30), "randpackageconfig"],
- stdout=devnull, stderr=devnull)
- if ret != 0:
- log_write(log, "ERROR: cannot generate random configuration")
- return -1
- if fixup_config(**kwargs):
- break
-
- ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir, "olddefconfig"],
- stdout=devnull, stderr=devnull)
- if ret != 0:
- log_write(log, "ERROR: cannot oldconfig")
- return -1
-
- ret = subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir, "savedefconfig"],
- stdout=devnull, stderr=devnull)
- if ret != 0:
- log_write(log, "ERROR: cannot savedefconfig")
- return -1
-
- return 0
+ ret = subprocess.call([os.path.join(srcdir, "utils/genrandconfig"),
+ "-i", str(kwargs['instance']),
+ "--toolchains-url", kwargs['toolchains_url']],
+ stdout=log, stderr=log)
+ return ret
def do_build(**kwargs):
"""Run the build itself"""
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v3 2/4] autobuild-run: adapt to genrandconfig without log_write
2017-07-21 1:06 ` [Buildroot] [PATCH v3 1/4] autobuild-run: use in-tree genrandconfig script (initial version) Arnout Vandecappelle
@ 2017-07-21 1:06 ` Arnout Vandecappelle
2017-07-21 1:06 ` [Buildroot] [PATCH v3 3/4] autobuild-run: adapt to genrandconfig with outputdir and buildrootdir options Arnout Vandecappelle
` (2 subsequent siblings)
3 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:06 UTC (permalink / raw)
To: buildroot
We now should catch the stderr output from genrandconfig and discard
stdout. However, for debugging, stdout is still useful, so keep that
if the -d option is given.
Also log an explicit message when genrandconfig failed. genrandconfig
itself should already have printed some error message, but:
- it's possible that it didn't, for some reason;
- it's missing a timestamp.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
scripts/autobuild-run | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/scripts/autobuild-run b/scripts/autobuild-run
index 879c9c3..85d82d8 100755
--- a/scripts/autobuild-run
+++ b/scripts/autobuild-run
@@ -307,10 +307,16 @@ def gen_config(**kwargs):
log = kwargs['log']
srcdir = os.path.join(idir, "buildroot")
+ log_write(log, "INFO: generate the configuration")
+
+ if kwargs['debug']:
+ devnull = log
+ else:
+ devnull = open(os.devnull, "w")
ret = subprocess.call([os.path.join(srcdir, "utils/genrandconfig"),
"-i", str(kwargs['instance']),
"--toolchains-url", kwargs['toolchains_url']],
- stdout=log, stderr=log)
+ stdout=devnull, stderr=log)
return ret
def do_build(**kwargs):
@@ -546,6 +552,7 @@ def run_instance(**kwargs):
ret = gen_config(**kwargs)
if ret != 0:
+ log_write(kwargs['log'], "WARN: failed to generate configuration")
continue
ret = do_build(**kwargs)
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v3 3/4] autobuild-run: adapt to genrandconfig with outputdir and buildrootdir options
2017-07-21 1:06 ` [Buildroot] [PATCH v3 1/4] autobuild-run: use in-tree genrandconfig script (initial version) Arnout Vandecappelle
2017-07-21 1:06 ` [Buildroot] [PATCH v3 2/4] autobuild-run: adapt to genrandconfig without log_write Arnout Vandecappelle
@ 2017-07-21 1:06 ` Arnout Vandecappelle
2017-07-21 1:06 ` [Buildroot] [PATCH v3 4/4] autobuild-run: adapt to genrandconfig with toolchains-csv option Arnout Vandecappelle
2017-07-25 21:24 ` [Buildroot] [PATCH v3 1/4] autobuild-run: use in-tree genrandconfig script (initial version) Thomas Petazzoni
3 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:06 UTC (permalink / raw)
To: buildroot
The -i option has been removed from genrandconfig, it has been replaced
with an explicit -o OUTPUTDIR and -b BUILDROOTDIR.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
scripts/autobuild-run | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/scripts/autobuild-run b/scripts/autobuild-run
index 85d82d8..0f9d046 100755
--- a/scripts/autobuild-run
+++ b/scripts/autobuild-run
@@ -305,6 +305,7 @@ def gen_config(**kwargs):
"""Generate a new random configuration."""
idir = "instance-%d" % kwargs['instance']
log = kwargs['log']
+ outputdir = os.path.abspath(os.path.join(idir, "output"))
srcdir = os.path.join(idir, "buildroot")
log_write(log, "INFO: generate the configuration")
@@ -314,7 +315,7 @@ def gen_config(**kwargs):
else:
devnull = open(os.devnull, "w")
ret = subprocess.call([os.path.join(srcdir, "utils/genrandconfig"),
- "-i", str(kwargs['instance']),
+ "-o", outputdir, "-b", srcdir,
"--toolchains-url", kwargs['toolchains_url']],
stdout=devnull, stderr=log)
return ret
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v3 4/4] autobuild-run: adapt to genrandconfig with toolchains-csv option
2017-07-21 1:06 ` [Buildroot] [PATCH v3 1/4] autobuild-run: use in-tree genrandconfig script (initial version) Arnout Vandecappelle
2017-07-21 1:06 ` [Buildroot] [PATCH v3 2/4] autobuild-run: adapt to genrandconfig without log_write Arnout Vandecappelle
2017-07-21 1:06 ` [Buildroot] [PATCH v3 3/4] autobuild-run: adapt to genrandconfig with outputdir and buildrootdir options Arnout Vandecappelle
@ 2017-07-21 1:06 ` Arnout Vandecappelle
2017-07-25 21:24 ` [Buildroot] [PATCH v3 1/4] autobuild-run: use in-tree genrandconfig script (initial version) Thomas Petazzoni
3 siblings, 0 replies; 34+ messages in thread
From: Arnout Vandecappelle @ 2017-07-21 1:06 UTC (permalink / raw)
To: buildroot
The --toolchains-url option has been removed from genrandconfig, it has
been replaced with --toolchains-csv.
We don't actually need to pass this option if not set, so that the
in-tree default will be used. This makes sure that the autobuild-run
doesn't need to be updated if we change the location of the CSV file.
The -i option has been removed from genrandconfig, it has been replaced
with an explicit -o OUTPUTDIR and -b BUILDROOTDIR.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
scripts/autobuild-run | 21 ++++++++++++++-------
1 file changed, 14 insertions(+), 7 deletions(-)
diff --git a/scripts/autobuild-run b/scripts/autobuild-run
index 0f9d046..f1dade4 100755
--- a/scripts/autobuild-run
+++ b/scripts/autobuild-run
@@ -67,7 +67,7 @@ defaults = {
'--nice': 0,
'--pid-file': '/tmp/buildroot-autobuild.pid',
'--http-url': 'http://autobuild.buildroot.org/submit/',
- '--toolchains-url': 'http://autobuild.buildroot.org/toolchains/configs/toolchain-configs.csv',
+ '--toolchains-csv': 'support/config-fragments/autobuild/toolchain-configs.csv',
}
doc = """autobuild-run - run Buildroot autobuilder
@@ -102,7 +102,7 @@ Options:
-c, --config CONFIG path to configuration file
Not set by default.
-d, --debug Send log output to stdout instead of log file
- --toolchains-url URL URL of toolchain configuration file
+ --toolchains-csv CSVFILE Toolchain configuration file
Format of the configuration file:
@@ -314,10 +314,17 @@ def gen_config(**kwargs):
devnull = log
else:
devnull = open(os.devnull, "w")
- ret = subprocess.call([os.path.join(srcdir, "utils/genrandconfig"),
- "-o", outputdir, "-b", srcdir,
- "--toolchains-url", kwargs['toolchains_url']],
- stdout=devnull, stderr=log)
+
+ args = [os.path.join(srcdir, "utils/genrandconfig"),
+ "-o", outputdir, "-b", srcdir]
+
+ toolchains_csv = kwargs['toolchains_csv']
+ if toolchains_csv:
+ if not os.path.isabs(toolchains_csv):
+ toolchains_csv = os.path.join(srcdir, toolchains_csv)
+ args.extend(["--toolchains-csv", toolchains_csv])
+
+ ret = subprocess.call(args, stdout=devnull, stderr=log)
return ret
def do_build(**kwargs):
@@ -666,7 +673,7 @@ def main():
submitter = args['--submitter'],
make_opts = (args['--make-opts'] or ''),
nice = (args['--nice'] or 0),
- toolchains_url = args['--toolchains-url'],
+ toolchains_csv = args['--toolchains-csv'],
upload = upload,
buildpid = buildpid,
debug = args['--debug']
--
2.13.2
^ permalink raw reply related [flat|nested] 34+ messages in thread* [Buildroot] [PATCH v3 1/4] autobuild-run: use in-tree genrandconfig script (initial version)
2017-07-21 1:06 ` [Buildroot] [PATCH v3 1/4] autobuild-run: use in-tree genrandconfig script (initial version) Arnout Vandecappelle
` (2 preceding siblings ...)
2017-07-21 1:06 ` [Buildroot] [PATCH v3 4/4] autobuild-run: adapt to genrandconfig with toolchains-csv option Arnout Vandecappelle
@ 2017-07-25 21:24 ` Thomas Petazzoni
3 siblings, 0 replies; 34+ messages in thread
From: Thomas Petazzoni @ 2017-07-25 21:24 UTC (permalink / raw)
To: buildroot
Hello,
On Fri, 21 Jul 2017 03:06:18 +0200, Arnout Vandecappelle
(Essensium/Mind) wrote:
> The Buildroot tree now has a genrandconfig script that replaces a
> large part of the autobuild-run script. Call this script rather than
> having it in autobuild-run.
>
> The genrandconfig script inside the Buildroot tree has the advantage
> that it can be adapted to the changes in the source, without the
> need to update the autobuild-run script all the time. Also, it makes
> it feasible to support autobuilding multiple branches.
>
> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> ---
> scripts/autobuild-run | 301 +-------------------------------------------------
> 1 file changed, 6 insertions(+), 295 deletions(-)
All four patches applied to buildroot-test. Thanks!
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 34+ messages in thread
* [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree
2017-07-21 1:05 [Buildroot] [PATCH v7 00/23] Move toolchain configs and autobuild logic in-tree Arnout Vandecappelle
` (23 preceding siblings ...)
2017-07-21 1:06 ` [Buildroot] [PATCH v3 1/4] autobuild-run: use in-tree genrandconfig script (initial version) Arnout Vandecappelle
@ 2017-07-25 21:10 ` Thomas Petazzoni
24 siblings, 0 replies; 34+ messages in thread
From: Thomas Petazzoni @ 2017-07-25 21:10 UTC (permalink / raw)
To: buildroot
Hello,
On Fri, 21 Jul 2017 03:05:07 +0200, Arnout Vandecappelle
(Essensium/Mind) wrote:
> Arnout Vandecappelle (23):
> utils/genrandconfig: new script
> genrandconfig: use subprocess.check_output instead of Popen
> genrandconfig: fix (some) pep8 warnings
> genrandconfig: replace kwargs with explicit arguments
> genrandconfig: move instantiation of SystemInfo down
> genrandconfig: verbose output and use stderr
> genrandconfig: calculate outputdir in __main__
> genrandconfig: calculate buildrootdir in __main__
> genrandconfig: pass outputdir and buildrootdir as arguments
> genrandconfig: calculate configfile only once
> genrandconfig: fix the case when outputdir is 'output'
> support/test-pkg: move minimal.config into a separate file
> minimal.config: add BR2_COMPILER_PARANOID_UNSAFE_PATH=y
> minimal.config: add BR2_PACKAGE_BUSYBOX_SHOW_OTHERS=y
> genrandconfig: use minimal.config
> support: add autobuild toolchain config fragments
> test-pkg: get configs from in-tree toolchain-configs.csv
> support/test-pkg: add option to use an alternate toolchains CSV file
> genrandconfig: get configs from in-tree toolchain-configs.csv
> toolchain-configs.csv: remove unused libc column
> Makefile: refactor *config targets
> Makefile: add alldefconfig target
> test-pkg: use merge_config.sh to merge the fragments
Thanks, I've applied all patches, except "support/test-pkg: add option
to use an alternate toolchains CSV file", and replied to this one with
the explanation.
I only did a quick review, because the general direction is definitely
the right one, and I trust you to be there to fix issues :-)
Thanks for doing this, it was long overdue, and will definitely ease
the maintenance of autobuilder exceptions, and opens the path to
testing multiple branches.
Thanks!
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 34+ messages in thread