Broken /usr/bin/env (invalid option)

Broken /usr/bin/env (invalid option)

[Krzysztof Kozlowski]

Hi Dominique,

Your commit ece075366294 ("ld-version: use /usr/bin/env awk for
shebank") breaks current linux-next on Ubuntu 16.04:

    $ make defconfig

    make[1]: Entering directory '/mnt/data2/linux/linux-upstream/out'
      GEN     Makefile
    *** Default configuration is based on 'multi_v7_defconfig'
    /usr/bin/env: invalid option -- 'S'
    Try '/usr/bin/env --help' for more information.
    init/Kconfig:39: syntax error
    init/Kconfig:38: invalid statement
    ../scripts/kconfig/Makefile:80: recipe for target 'defconfig' failed
    make[2]: *** [defconfig] Error 1
    /mnt/data2/linux/linux-upstream/Makefile:602: recipe for target 'defconfig' failed
    make[1]: *** [defconfig] Error 2
    make[1]: Leaving directory '/mnt/data2/linux/linux-upstream/out'
    Makefile:185: recipe for target '__sub-make' failed
    make: *** [__sub-make] Error 2

https://krzk.eu/#/builders/12/builds/1170/steps/4/logs/stdio

It's not possible to build anything...

Best regards,
Krzysztof

Re: Broken /usr/bin/env (invalid option)

[Dominique Martinet]

Krzysztof Kozlowski wrote on Wed, Dec 09, 2020:
> Your commit ece075366294 ("ld-version: use /usr/bin/env awk for
> shebank") breaks current linux-next on Ubuntu 16.04:
> 
>     $ make defconfig
> 
>     make[1]: Entering directory '/mnt/data2/linux/linux-upstream/out'
>       GEN     Makefile
>     *** Default configuration is based on 'multi_v7_defconfig'
>     /usr/bin/env: invalid option -- 'S'
>     Try '/usr/bin/env --help' for more information.
>     init/Kconfig:39: syntax error
>     init/Kconfig:38: invalid statement
>     ../scripts/kconfig/Makefile:80: recipe for target 'defconfig' failed
>     make[2]: *** [defconfig] Error 1
>     /mnt/data2/linux/linux-upstream/Makefile:602: recipe for target 'defconfig' failed
>     make[1]: *** [defconfig] Error 2
>     make[1]: Leaving directory '/mnt/data2/linux/linux-upstream/out'
>     Makefile:185: recipe for target '__sub-make' failed
>     make: *** [__sub-make] Error 2
> 
> https://krzk.eu/#/builders/12/builds/1170/steps/4/logs/stdio
> 
> It's not possible to build anything...

Hm, what version of env is that?
I only checked on a couple of systems that env -S exists, but it might
not be available (e.g. now I'm looking, at least busybox env does not
provide -S)

OTOH, we need -S to split arguments because the kernel splits the
shebang as [binary, single argument], so without it awk wouldn't work.

Apparently gawk can work without an explicit -f is invoked as gawk (so
could be used without -S), but that won't be much better wrt
compatibility in my opinion.


I don't see any way to make this work out of the box for me (awk not in
/usr/bin) and you (env not supporting -S), but I guess I can keep that
patch around locally...

Sorry for the trouble.
-- 
Dominique

Re: Broken /usr/bin/env (invalid option)

[Krzysztof Kozlowski]

On Wed, 9 Dec 2020 at 14:13, Dominique Martinet <asmadeus@codewreck.org> wrote:
>
> Krzysztof Kozlowski wrote on Wed, Dec 09, 2020:
> > Your commit ece075366294 ("ld-version: use /usr/bin/env awk for
> > shebank") breaks current linux-next on Ubuntu 16.04:
> >
> >     $ make defconfig
> >
> >     make[1]: Entering directory '/mnt/data2/linux/linux-upstream/out'
> >       GEN     Makefile
> >     *** Default configuration is based on 'multi_v7_defconfig'
> >     /usr/bin/env: invalid option -- 'S'
> >     Try '/usr/bin/env --help' for more information.
> >     init/Kconfig:39: syntax error
> >     init/Kconfig:38: invalid statement
> >     ../scripts/kconfig/Makefile:80: recipe for target 'defconfig' failed
> >     make[2]: *** [defconfig] Error 1
> >     /mnt/data2/linux/linux-upstream/Makefile:602: recipe for target 'defconfig' failed
> >     make[1]: *** [defconfig] Error 2
> >     make[1]: Leaving directory '/mnt/data2/linux/linux-upstream/out'
> >     Makefile:185: recipe for target '__sub-make' failed
> >     make: *** [__sub-make] Error 2
> >
> > https://krzk.eu/#/builders/12/builds/1170/steps/4/logs/stdio
> >
> > It's not possible to build anything...
>
> Hm, what version of env is that?
> I only checked on a couple of systems that env -S exists, but it might
> not be available (e.g. now I'm looking, at least busybox env does not
> provide -S)

$ env --version
env (GNU coreutils) 8.25

> OTOH, we need -S to split arguments because the kernel splits the
> shebang as [binary, single argument], so without it awk wouldn't work.

None of the other scripts use 'env -S', maybe for the reason.

> Apparently gawk can work without an explicit -f is invoked as gawk (so
> could be used without -S), but that won't be much better wrt
> compatibility in my opinion.
>
>
> I don't see any way to make this work out of the box for me (awk not in
> /usr/bin) and you (env not supporting -S), but I guess I can keep that
> patch around locally...

You could add a wrapper which uses PATH or searches for the awk...
kind of reimplement other tools. Another solution is to clearly
document the requirements for the build system and explicitly say that
older systems like four-year Ubuntu are not supported.

Best regards,
Krzysztof

Re: Broken /usr/bin/env (invalid option)

[Dominique Martinet]

Krzysztof Kozlowski wrote on Wed, Dec 09, 2020:
> > Hm, what version of env is that?
> > I only checked on a couple of systems that env -S exists, but it might
> > not be available (e.g. now I'm looking, at least busybox env does not
> > provide -S)
> 
> $ env --version
> env (GNU coreutils) 8.25

Ok, so this is a much newer option than I thought it was, sorry for my
lack of checking.
I know we no longer support older gccs but it's probably best to keep
supporting this.

> > I don't see any way to make this work out of the box for me (awk not in
> > /usr/bin) and you (env not supporting -S), but I guess I can keep that
> > patch around locally...
> 
> You could add a wrapper which uses PATH or searches for the awk...
> kind of reimplement other tools. Another solution is to clearly
> document the requirements for the build system and explicitly say that
> older systems like four-year Ubuntu are not supported.

Well there definitely are ways, but I'm not sure it's worth it...
For example another idea is just to use /bin/sh (which is mandated to
exist) and have it invoke awk, the script is short enough to just quote
it.


Yamada-san, how about this instead?
But to be honest I'm fine with dropping my patch altogether, I'll work
around it locally if it's too much.
---
From d53ef3b4c55aa2ea5f9ae887b3e1ace368f30f66 Mon Sep 17 00:00:00 2001
From: Dominique Martinet <asmadeus@codewreck.org>
Date: Wed, 15 Jul 2020 16:00:13 +0200
Subject: [PATCH] ld-version: use /bin/sh then awk for shebank

/usr/bin/awk is not garanteed to exist (and doesn't on e.g. nixos),
using /bin/sh and invoking awk to have it look in PATH is more robust.

Signed-off-by: Dominique Martinet <asmadeus@codewreck.org>

diff --git a/scripts/ld-version.sh b/scripts/ld-version.sh
index f2be0ff9a738..02dbad7b5613 100755
--- a/scripts/ld-version.sh
+++ b/scripts/ld-version.sh
@@ -1,11 +1,11 @@
-#!/usr/bin/awk -f
+#!/bin/sh
 # SPDX-License-Identifier: GPL-2.0
 # extract linker version number from stdin and turn into single number
-       {
+awk '{
        gsub(".*\\)", "");
        gsub(".*version ", "");
        gsub("-.*", "");
        split($1,a, ".");
        print a[1]*100000000 + a[2]*1000000 + a[3]*10000;
        exit
-       }
+}'