Re: [OE-core] [PATCH] autoconf: Raise line length limit for sed/awk

Re: [OE-core] [PATCH] autoconf: Raise line length limit for sed/awk

[Yoann Congal]

[-- Attachment #1: Type: text/plain, Size: 5025 bytes --]

Le jeu. 27 nov. 2025 à 17:44, Khem Raj via lists.openembedded.org <raj.khem=
gmail.com@lists.openembedded.org> a écrit :

> Latest autoconf is splitting long cmd lines in config.status
> it generates using sed/awk to have compatibility with
> platform like HPUX,Solaris,AIX which is understandable given
> autotools try to achieve wider portability, however, this does
> mean that in OE builds we might get these newlines added in
> places dividing commands including buildpaths in them and the
> methods deployed to scrub them from output artifacts will fail
> since the string is now on two lines resulting in build QA to
> pass but the strings do sneak in affecting reproducibility
> one such example is e2tools recipe in meta-filesystems and there
> perhaps are more
>
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
>

For the impact on e2tools:
Tested-by: Yoann Congal <yoann.congal@smile.fr>
=> It does indeed make it reproducible by letting the sed replacing non
reproducible paths work.


> ---
>  ...-command-limit-to-prevent-line-wrapp.patch | 41 +++++++++++++++++++
>  .../autoconf/autoconf_2.72.bb                 |  3 +-
>  2 files changed, 43 insertions(+), 1 deletion(-)
>  create mode 100644
> meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
>
> diff --git
> a/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
> b/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
> new file mode 100644
> index 00000000000..41468ba316c
> --- /dev/null
> +++
> b/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
> @@ -0,0 +1,41 @@
> +From 139f757a6244d3b1e07f7fddc6d07d9c17f45313 Mon Sep 17 00:00:00 2001
> +From: Khem Raj <raj.khem@gmail.com>
> +Date: Tue, 25 Nov 2025 21:03:38 -0800
> +Subject: [PATCH] Increase awk/sed command limit to prevent line wrapping
> +
> +Autoconf 2.70+ wraps long variable assignments in config.status
> +using backslash-newline continuations. This breaks when LDFLAGS
> +or other variables contain special characters.
> +
> +Increase _AC_SED_CMD_LIMIT from 99 and _AC_AWK_LITERAL_LIMIT
> +from 148 to 4000 to prevent wrapping.
> +
> +Upstream-Status: Inappropriate [OE-Specific]
> +
> +Signed-off-by: Khem Raj <raj.khem@gmail.com>
> +---
> + lib/autoconf/status.m4 | 4 ++--
> + 1 file changed, 2 insertions(+), 2 deletions(-)
> +
> +diff --git a/lib/autoconf/status.m4 b/lib/autoconf/status.m4
> +index 2bfaf32..be444b2 100644
> +--- a/lib/autoconf/status.m4
> ++++ b/lib/autoconf/status.m4
> +@@ -297,7 +297,7 @@ AC_DEFUN([AC_CONFIG_FILES], [_AC_CONFIG_FOOS([FILES],
> $@)])
> + # Some seds have small command number limits, like on Digital OSF/1 and
> HP-UX.
> + m4_define([_AC_SED_CMD_LIMIT],
> + dnl One cannot portably go further than 99 commands because of HP-UX.
> +-[99])
> ++[4000])
> +
> +
> + # _AC_AWK_LITERAL_LIMIT
> +@@ -307,7 +307,7 @@ dnl One cannot portably go further than 99 commands
> because of HP-UX.
> + #
> + # Some awk's have small limits, such as Solaris and AIX awk.
> + m4_define([_AC_AWK_LITERAL_LIMIT],
> +-[148])
> ++[4000])
> +
> +
> + # _AC_OUTPUT_FILES_PREPARE
> diff --git a/meta/recipes-devtools/autoconf/autoconf_2.72.bb
> b/meta/recipes-devtools/autoconf/autoconf_2.72.bb
> index b599f270c79..20197d6f793 100644
> --- a/meta/recipes-devtools/autoconf/autoconf_2.72.bb
> +++ b/meta/recipes-devtools/autoconf/autoconf_2.72.bb
> @@ -1,5 +1,5 @@
>  SUMMARY = "A GNU tool that produce shell scripts to automatically
> configure software"
> -DESCRIPTION = "Autoconf is an extensible package of M4 macros that
> produce shell scripts to automatically \
> +DESCRIPTION = "Autoconf is an extensible package of M4 macros that
> produce shell scripts to automatically \
>  configure software source code packages. Autoconf creates a configuration
> script for a package from a template \
>  file that lists the operating system features that the package can use,
> in the form of M4 macro calls."
>  LICENSE = "GPL-3.0-or-later"
> @@ -19,6 +19,7 @@ SRC_URI = " \
>             file://preferbash.patch \
>             file://autotest-automake-result-format.patch \
>             file://man-host-perl.patch \
> +
> file://0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch \
>  "
>  SRC_URI:append:class-native = " file://no-man.patch"
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#226850):
> https://lists.openembedded.org/g/openembedded-core/message/226850
> Mute This Topic: https://lists.openembedded.org/mt/116503258/4316185
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> yoann.congal@smile.fr]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
Yoann Congal
Smile ECS

[-- Attachment #2: Type: text/html, Size: 7135 bytes --]

Re: [OE-core] [PATCH] autoconf: Raise line length limit for sed/awk

[Mathieu Dubois-Briand]

On Thu Nov 27, 2025 at 5:44 PM CET, Khem Raj via lists.openembedded.org wrote:
> Latest autoconf is splitting long cmd lines in config.status
> it generates using sed/awk to have compatibility with
> platform like HPUX,Solaris,AIX which is understandable given
> autotools try to achieve wider portability, however, this does
> mean that in OE builds we might get these newlines added in
> places dividing commands including buildpaths in them and the
> methods deployed to scrub them from output artifacts will fail
> since the string is now on two lines resulting in build QA to
> pass but the strings do sneak in affecting reproducibility
> one such example is e2tools recipe in meta-filesystems and there
> perhaps are more
>
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---

Hi Khem,

Thanks for your patch.

I've had some issues on the autobuilder for a few days, and it looks
like it was triggered by this patch. While there was no specific error,
the -tc build jobs had been running for several days and never ending.

One example:
https://autobuilder.yoctoproject.org/valkyrie/?#/builders/66/builds/2740

I've had a look on the worker: it looks like the host nfs server, used
to share files with the tests running in qemu, had been failing:

pokybui+ 4115014  1.1  0.0      0     0 ?        Z    14:23   4:32 [unfsd] <defunct>

I clearly can't see how this can be linked to this patch, yet, git
bisect points to it, and after its removal I don't have the issue
anymore.

So I'm not sure there is anything you can do, but I will accept any clue
about what might going on here.

Thanks,
Mathieu


-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH] autoconf: Raise line length limit for sed/awk

[Yoann Congal]

Le mer. 3 déc. 2025 à 08:23, Mathieu Dubois-Briand via
lists.openembedded.org
<mathieu.dubois-briand=bootlin.com@lists.openembedded.org> a écrit :
>
> On Thu Nov 27, 2025 at 5:44 PM CET, Khem Raj via lists.openembedded.org wrote:
> > Latest autoconf is splitting long cmd lines in config.status
> > it generates using sed/awk to have compatibility with
> > platform like HPUX,Solaris,AIX which is understandable given
> > autotools try to achieve wider portability, however, this does
> > mean that in OE builds we might get these newlines added in
> > places dividing commands including buildpaths in them and the
> > methods deployed to scrub them from output artifacts will fail
> > since the string is now on two lines resulting in build QA to
> > pass but the strings do sneak in affecting reproducibility
> > one such example is e2tools recipe in meta-filesystems and there
> > perhaps are more
> >
> > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > ---
>
> Hi Khem,
>
> Thanks for your patch.
>
> I've had some issues on the autobuilder for a few days, and it looks
> like it was triggered by this patch. While there was no specific error,
> the -tc build jobs had been running for several days and never ending.
>
> One example:
> https://autobuilder.yoctoproject.org/valkyrie/?#/builders/66/builds/2740
>
> I've had a look on the worker: it looks like the host nfs server, used
> to share files with the tests running in qemu, had been failing:
>
> pokybui+ 4115014  1.1  0.0      0     0 ?        Z    14:23   4:32 [unfsd] <defunct>
>
> I clearly can't see how this can be linked to this patch, yet, git
> bisect points to it, and after its removal I don't have the issue
> anymore.
>
> So I'm not sure there is anything you can do, but I will accept any clue
> about what might going on here.

Well, that's unfortunate... I've created a ticket related to this to
avoid losing this in our mailboxes:
16093 – Compatibility with old sed/awk interferes with reproducibility
https://bugzilla.yoctoproject.org/show_bug.cgi?id=16093

>
> Thanks,
> Mathieu
>
>
> --
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#227203): https://lists.openembedded.org/g/openembedded-core/message/227203
> Mute This Topic: https://lists.openembedded.org/mt/116503258/4316185
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [yoann.congal@smile.fr]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
Yoann Congal
Smile ECS

[PATCH] autoconf: Raise line length limit for sed/awk

[Khem Raj]

From: Khem Raj <raj.khem@gmail.com>

Latest autoconf is splitting long cmd lines in config.status
it generates using sed/awk to have compatibility with
platform like HPUX,Solaris,AIX which is understandable given
autotools try to achieve wider portability, however, this does
mean that in OE builds we might get these newlines added in
places dividing commands including buildpaths in them and the
methods deployed to scrub them from output artifacts will fail
since the string is now on two lines resulting in build QA to
pass but the strings do sneak in affecting reproducibility
one such example is e2tools recipe in meta-filesystems and there
perhaps are more

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 ...-command-limit-to-prevent-line-wrapp.patch | 41 +++++++++++++++++++
 .../autoconf/autoconf_2.72.bb                 |  1 +
 2 files changed, 42 insertions(+)
 create mode 100644 meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch

diff --git a/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch b/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
new file mode 100644
index 0000000000..0713af6d7d
--- /dev/null
+++ b/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
@@ -0,0 +1,41 @@
+From 139f757a6244d3b1e07f7fddc6d07d9c17f45313 Mon Sep 17 00:00:00 2001
+From: Khem Raj <raj.khem@gmail.com>
+Date: Tue, 25 Nov 2025 21:03:38 -0800
+Subject: [PATCH] Increase awk/sed command limit to prevent line wrapping
+
+Autoconf 2.70+ wraps long variable assignments in config.status
+using backslash-newline continuations. This breaks when LDFLAGS
+or other variables contain special characters.
+
+Increase _AC_SED_CMD_LIMIT from 99 and _AC_AWK_LITERAL_LIMIT
+from 148 to 4000 to prevent wrapping.
+
+Upstream-Status: Inappropriate [OE-Specific]
+
+Signed-off-by: Khem Raj <raj.khem@gmail.com>
+---
+ lib/autoconf/status.m4 | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/lib/autoconf/status.m4 b/lib/autoconf/status.m4
+index 2bfaf32..be444b2 100644
+--- a/lib/autoconf/status.m4
++++ b/lib/autoconf/status.m4
+@@ -297,7 +297,7 @@ AC_DEFUN([AC_CONFIG_FILES], [_AC_CONFIG_FOOS([FILES], $@)])
+ # Some seds have small command number limits, like on Digital OSF/1 and HP-UX.
+ m4_define([_AC_SED_CMD_LIMIT],
+ dnl One cannot portably go further than 99 commands because of HP-UX.
+-[99])
++[4000])
+
+
+ # _AC_AWK_LITERAL_LIMIT
+@@ -307,7 +307,7 @@ dnl One cannot portably go further than 99 commands because of HP-UX.
+ #
+ # Some awk's have small limits, such as Solaris and AIX awk.
+ m4_define([_AC_AWK_LITERAL_LIMIT],
+-[148])
++[4000])
+
+
+ # _AC_OUTPUT_FILES_PREPARE
diff --git a/meta/recipes-devtools/autoconf/autoconf_2.72.bb b/meta/recipes-devtools/autoconf/autoconf_2.72.bb
index b599f270c7..605d287e9a 100644
--- a/meta/recipes-devtools/autoconf/autoconf_2.72.bb
+++ b/meta/recipes-devtools/autoconf/autoconf_2.72.bb
@@ -19,6 +19,7 @@ SRC_URI = " \
            file://preferbash.patch \
            file://autotest-automake-result-format.patch \
            file://man-host-perl.patch \
+           file://0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch \
 "
 SRC_URI:append:class-native = " file://no-man.patch"
 

Re: [OE-core] [PATCH] autoconf: Raise line length limit for sed/awk

[Richard Purdie]

On Thu, 2026-04-02 at 19:43 -0700, Khem Raj via lists.openembedded.org wrote:
> From: Khem Raj <raj.khem@gmail.com>
> 
> Latest autoconf is splitting long cmd lines in config.status
> it generates using sed/awk to have compatibility with
> platform like HPUX,Solaris,AIX which is understandable given
> autotools try to achieve wider portability, however, this does
> mean that in OE builds we might get these newlines added in
> places dividing commands including buildpaths in them and the
> methods deployed to scrub them from output artifacts will fail
> since the string is now on two lines resulting in build QA to
> pass but the strings do sneak in affecting reproducibility
> one such example is e2tools recipe in meta-filesystems and there
> perhaps are more
> 
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---
>  ...-command-limit-to-prevent-line-wrapp.patch | 41 +++++++++++++++++++
>  .../autoconf/autoconf_2.72.bb                 |  1 +
>  2 files changed, 42 insertions(+)
>  create mode 100644 meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
> 
> diff --git a/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch b/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
> new file mode 100644
> index 0000000000..0713af6d7d
> --- /dev/null
> +++ b/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
> @@ -0,0 +1,41 @@
> +From 139f757a6244d3b1e07f7fddc6d07d9c17f45313 Mon Sep 17 00:00:00 2001
> +From: Khem Raj <raj.khem@gmail.com>
> +Date: Tue, 25 Nov 2025 21:03:38 -0800
> +Subject: [PATCH] Increase awk/sed command limit to prevent line wrapping
> +
> +Autoconf 2.70+ wraps long variable assignments in config.status
> +using backslash-newline continuations. This breaks when LDFLAGS
> +or other variables contain special characters.
> +
> +Increase _AC_SED_CMD_LIMIT from 99 and _AC_AWK_LITERAL_LIMIT
> +from 148 to 4000 to prevent wrapping.

The commit message explains why this is needed but the patch needs to
as well (buildpaths being linewrapped and then not being correctly
updated as they're over multiple lines).

Cheers,

Richard

Re: [OE-core] [PATCH] autoconf: Raise line length limit for sed/awk

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 2558 bytes --]

On Fri, Apr 3, 2026 at 1:53 AM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Thu, 2026-04-02 at 19:43 -0700, Khem Raj via lists.openembedded.org
> wrote:
> > From: Khem Raj <raj.khem@gmail.com>
> >
> > Latest autoconf is splitting long cmd lines in config.status
> > it generates using sed/awk to have compatibility with
> > platform like HPUX,Solaris,AIX which is understandable given
> > autotools try to achieve wider portability, however, this does
> > mean that in OE builds we might get these newlines added in
> > places dividing commands including buildpaths in them and the
> > methods deployed to scrub them from output artifacts will fail
> > since the string is now on two lines resulting in build QA to
> > pass but the strings do sneak in affecting reproducibility
> > one such example is e2tools recipe in meta-filesystems and there
> > perhaps are more
> >
> > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > ---
> >  ...-command-limit-to-prevent-line-wrapp.patch | 41 +++++++++++++++++++
> >  .../autoconf/autoconf_2.72.bb                 |  1 +
> >  2 files changed, 42 insertions(+)
> >  create mode 100644
> meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
> >
> > diff --git
> a/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
> b/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
> > new file mode 100644
> > index 0000000000..0713af6d7d
> > --- /dev/null
> > +++
> b/meta/recipes-devtools/autoconf/autoconf/0001-Increase-awk-sed-command-limit-to-prevent-line-wrapp.patch
> > @@ -0,0 +1,41 @@
> > +From 139f757a6244d3b1e07f7fddc6d07d9c17f45313 Mon Sep 17 00:00:00 2001
> > +From: Khem Raj <raj.khem@gmail.com>
> > +Date: Tue, 25 Nov 2025 21:03:38 -0800
> > +Subject: [PATCH] Increase awk/sed command limit to prevent line wrapping
> > +
> > +Autoconf 2.70+ wraps long variable assignments in config.status
> > +using backslash-newline continuations. This breaks when LDFLAGS
> > +or other variables contain special characters.
> > +
> > +Increase _AC_SED_CMD_LIMIT from 99 and _AC_AWK_LITERAL_LIMIT
> > +from 148 to 4000 to prevent wrapping.
>
> The commit message explains why this is needed but the patch needs to
> as well (buildpaths being linewrapped and then not being correctly
> updated as they're over multiple lines).
>

makes sense, I have sent v2


>
> Cheers,
>
> Richard
>

[-- Attachment #2: Type: text/html, Size: 3614 bytes --]