From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D95E8362127 for ; Tue, 10 Mar 2026 07:51:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.43 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773129063; cv=none; b=DrBQsDWKP3rjf7n/i6txgYmpDR79OtUOsHKgTIRdeHbkSTh7R6dW027yPfI2wa6P1du0RhgZWHYrJuXLipQBi3IvE7ZP7J/jMJ0l+RcW1YWZxq3H5IP+lEMeF/M5fAJ0bB0Mzdt5xLbQp5OMyWiSS11CSc5HcodvzR6CCeyLfvo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773129063; c=relaxed/simple; bh=71wvaGeTES5Wz8n2xDwOO3cq7GaeyIzQYOKs3oKK4as=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=JKLQE0R3PgwBQviomHLEyphNYjAmISVFNJRLOfd5SxeaG5g8F6kd+OaZIO51X3aUhddjZJ5oFpHSCg6Uf3zRwaWyDlauz+XMypPgBgxwLfnjBK+QYVDMf5iUQOFQHenbfduWNdu1PSwJ11jvvyh+nn1hSDxIN7Hxpb74w2SMdow= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=sXO9MaUz; arc=none smtp.client-ip=209.85.167.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="sXO9MaUz" Received: by mail-lf1-f43.google.com with SMTP id 2adb3069b0e04-59de8155501so7017335e87.3 for ; Tue, 10 Mar 2026 00:51:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1773129060; x=1773733860; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=bnz3PCpJQ2w/9VMb2f/lqQClCVYil90P6vL7y20LWw8=; b=sXO9MaUz77SCoSnJaqlfUC9lyBCBEIAZqJPeAW5SpW95Yo14xUuG7i3nMYufjRH8aE 6Tl6ot4Rd8JD+21tJ9/U5VNenx35YhZHdahAY8tqodOEeQkcFuiBeP8WLvW0M2MKNAK/ N2ZAphpjDQQAXIH7CG/XI8daoxBeJVy5ohSDD457xeKXqPtwo/SfnaqC72s6GVAUvoom TcCptRZn9Fxg2nxBf118qH4+Bm9r5QTDOItZByVEKjJhpXfqgg9coRAdxo9oOw2zJotb xIB63NZFyyuMqDLslsxOlKXVbJKfih8gIbmzsAoLbeGLFAyOvUbcioSz0a9xiHL44dNH Hv9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1773129060; x=1773733860; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=bnz3PCpJQ2w/9VMb2f/lqQClCVYil90P6vL7y20LWw8=; b=JMlpjH1Sx3czMxn+1KSlzRnpzFDc6r9G+D9cop/zTjlvw9Van+9cudNfZyhqEeqZ9c JeuusuAQUNKvB2iJg9/Odk6U8r89iuqloPqj/sKrgauk2gvz23pfOfSs68E+mTEu1Dd+ jwWOcYhfQ93/NRQLshdjxUygQTHrlImcm+ReV8mIz7NceqU4mSWc+OrSiXopdB4g6wQV 1mVUoY8CxoHM9jd6DvRbyoU+yH89KCxkyayWgF8uKugbcWCjNjJvLZVXtNrIBSJJ1v9V QtCBk0YF2AkW6E7Lpo7kcoi/SMdRDf2i627lpanoQ/CahZL2cT07NFYVBThtS2KlDoIu jxfg== X-Forwarded-Encrypted: i=1; AJvYcCVunZFLMiJBc5k4QHX3XP0o7gs4TaEITGQsEDlonokSqqsOBGeVsRiJC6wQvXBQHmOyO2qkg3fPnh78lfo=@vger.kernel.org X-Gm-Message-State: AOJu0YyJ2QgI4MfDPazErFCOHBYujeTvpggiM3+alD4g4DmIPNu0z+l2 +pvJ8P1XaJazemODfEkGr9q9IIl6zAzbD5MUu1kH9CHTEC1nIhyyXwygUB2K7iuxRIY= X-Gm-Gg: ATEYQzxppoFYvSOW25sqO6opd2/BAPIKMBAJ3SNv59dXBe1BOOW7ysAP/C37uOr93iT 4Tjp52qb3eQK7lSwuYJzJs14oB+JNWbrx0A1HkMAuXpSLjgrQx+QgVqpEmzVzSqvjQJXSpRcHRW ITjOjr2Rn0WnM6BF64ZPJ0fxJ4g8uPmdCk64RZUNgPAKyxETJkBepxC0pTl/hG1WohQn0xepIVL CMig+Qhe6FNp1qthGfNRmBW/tUezTVQFooIISx27S6KBKX331roEeMOJneFI0I8gJ6fsbbp93Ts RNl9cCY5wJtWd4IHY2q6DzQPYjYJ8qFzPCVxC/kmDHoOFw1n5Xiecpt782IVuQRDFlNhS+XEj4i 3Mg3xO8xmBmav9qJ3+RW9ji4N8fMVmFC3/AhOkRxTK3FplfxanbmduH4j6N8y7501UU7faZBG/D xD9NsIJ3LsmVIJGbVMhzA8mwnZQp2bzKdzPiFHmsklw6Dz8pd9uHH03tWU X-Received: by 2002:a05:6512:12d0:b0:5a1:1914:2bac with SMTP id 2adb3069b0e04-5a13ccf7f2cmr4359767e87.25.1773129059912; Tue, 10 Mar 2026 00:50:59 -0700 (PDT) Received: from nuoska (87-100-249-247.bb.dnainternet.fi. [87.100.249.247]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-5a13d01cd7fsm2617542e87.19.2026.03.10.00.50.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Mar 2026 00:50:59 -0700 (PDT) Date: Tue, 10 Mar 2026 09:50:49 +0200 From: Mikko Rapeli To: Nathan Chancellor Cc: Andreas Larsson , Nicolas Schier , Anders Roxell , linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 1/3] scripts: kconfig: merge_config.sh: refactor from shell/sed/grep to awk Message-ID: References: <20260122105751.2186609-1-mikko.rapeli@linaro.org> <20260309170904.GA2779008@ax162> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20260309170904.GA2779008@ax162> Hi, On Mon, Mar 09, 2026 at 10:09:04AM -0700, Nathan Chancellor wrote: > On Mon, Mar 09, 2026 at 05:38:58PM +0100, Andreas Larsson wrote: > > On 2026-01-22 11:57, Mikko Rapeli wrote: > > > From: Anders Roxell > > > > > > merge_config.sh shell/sed/grep loop scales poorly and is slow. > > > With Yocto genericarm64 kernel and around 190 config fragments > > > the script takes more than 20 minutes to run on a fast build machine. > > > Re-implementation with awk does the same job in 10 seconds. > > > Using awk since it is likely available in the build environments > > > and using perl, python etc would introduce more complex runtime > > > dependencies. awk is good enough and lot better than shell/sed/grep. > > > > > > Output stays the same but changed execution time means that > > > parallel job output may be ordered differently. > > > > > > Signed-off-by: Anders Roxell > > > Signed-off-by: Mikko Rapeli > > > --- > > > scripts/kconfig/merge_config.sh | 168 ++++++++++++++++++++++++-------- > > > 1 file changed, 128 insertions(+), 40 deletions(-) > > > > Hi, > > > > Commit 5fa9b82cbcfc ("scripts: kconfig: merge_config.sh: refactor from > > shell/sed/grep to awk") breaks merge_config.sh for me: > > > > ---------->%---------- > > $ make tiny.config > > HOSTCC scripts/basic/fixdep > > HOSTCC scripts/kconfig/conf.o > > HOSTCC scripts/kconfig/confdata.o > > HOSTCC scripts/kconfig/expr.o > > LEX scripts/kconfig/lexer.lex.c > > YACC scripts/kconfig/parser.tab.[ch] > > HOSTCC scripts/kconfig/lexer.lex.o > > HOSTCC scripts/kconfig/menu.o > > HOSTCC scripts/kconfig/parser.tab.o > > HOSTCC scripts/kconfig/preprocess.o > > HOSTCC scripts/kconfig/symbol.o > > HOSTCC scripts/kconfig/util.o > > HOSTLD scripts/kconfig/conf > > The base file '.config' does not exist. Creating one... > > Using .config as base > > Merging ./kernel/configs/tiny.config > > awk: cannot open ./.tmp.config.U9SROCKTBj.new (No such file or directory) > > mv: cannot stat './.tmp.config.U9SROCKTBj.new': No such file or directory > > make[2]: *** [scripts/kconfig/Makefile:112: tiny.config] Error 1 > > make[1]: *** [/Makefile:744: tiny.config] Error 2 > > make: *** [Makefile:248: __sub-make] Error 2 > > ---------->%---------- > > > > with this underlying call to merge_config.sh, an empty .config and where > > my awk is GNU Awk 5.1.0: > > > > ---------->%---------- > > $ make tiny.config V=1 > > make --no-print-directory -C \ > > -f /Makefile tiny.config > > make -f ./scripts/Makefile.build obj=scripts/basic > > make -f ./scripts/Makefile.build obj=scripts/kconfig tiny.config > > # cmd_merge_fragments tiny.config > > ./scripts/kconfig/merge_config.sh -m .config ./kernel/configs/tiny.config ./arch/x86/configs/tiny.config > > ... > > > > $ awk --version > > GNU Awk 5.1.0, API: 3.0 (GNU MPFR 4.1.0, GNU MP 6.2.1) > > ---------->%---------- > > > > At 5fa9b82cbcfc~ things works as expected. Hmm, all these steps work for me with GNU awk 5.1.0 so something else is different here. On line 267 awk is given input file args "$ORIG_MERGE_FILE" "$TMP_FILE" "$TMP_FILE.new" and it fails to create the output file "$TMP_FILE.new" with >> and plain print statement. > Hmmm, not sure how I have not seen this error myself since I test > tiny.config but I can reproduce with a clean output directory. Something > like this seems like a simple fix but the only instances of ARGV[3] in > the awk script that I can see use '>>', which should create the file if > it does not exist. Mikko, any ideas? > > diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh > index 735e1de450c6..070ecae87a1c 100755 > --- a/scripts/kconfig/merge_config.sh > +++ b/scripts/kconfig/merge_config.sh > @@ -123,6 +123,7 @@ fi > MERGE_LIST=$* > > TMP_FILE=$(mktemp ./.tmp.config.XXXXXXXXXX) > +touch "$TMP_FILE.new" > > echo "Using $INITFILE as base" This could help but I fail to understand why this would be needed. Why is awk not able to create this file on line 256? awk manual says: https://www.gnu.org/software/gawk/manual/html_node/Redirection.html print items >> output-file This redirection prints the items into the preexisting output file named output-file. The difference between this and the single-‘>’ redirection is that the old contents (if any) of output-file are not erased. Instead, the awk output is appended to the file. If output-file does not exist, then it is created. And in all my testing this works. Which distro is this? Is something else like file system setup in some way unusual? Cheers, -Mikko