[Buildroot] [PATCH 1/1] package/dhcp: specify host-gawk path for dhcp build environment

[Buildroot] [PATCH 1/1] package/dhcp: specify host-gawk path for dhcp build environment

[Sergey Matyukevich]

DHCP package silently fails to install its binaries if gawk tool
is not installed on build server. This is still the case even
if host-gawk package is already available: dhcp Makefiles detect
its presence but still attempt to use gawk from host system.

Add host-gawk to the list of dependencies for dhcp and explicitely
specify host-gawk location.

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
---
 package/dhcp/dhcp.mk | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/package/dhcp/dhcp.mk b/package/dhcp/dhcp.mk
index ad59804d3b..779d56801f 100644
--- a/package/dhcp/dhcp.mk
+++ b/package/dhcp/dhcp.mk
@@ -9,7 +9,7 @@ DHCP_SITE = http://ftp.isc.org/isc/dhcp/$(DHCP_VERSION)
 DHCP_INSTALL_STAGING = YES
 DHCP_LICENSE = MPL-2.0
 DHCP_LICENSE_FILES = LICENSE
-DHCP_DEPENDENCIES = bind
+DHCP_DEPENDENCIES = bind host-gawk
 
 # use libtool-enabled configure.ac
 define DHCP_LIBTOOL_AUTORECONF
@@ -21,6 +21,8 @@ DHCP_CONF_ENV = \
 		-D_PATH_DHCLIENT_CONF=\"/etc/dhcp/dhclient.conf\"' \
 	CFLAGS='$(TARGET_CFLAGS) -DISC_CHECK_NONE=1'
 
+DHCP_CONF_ENV += ac_cv_prog_AWK=$(HOST_DIR)/bin/gawk
+
 DHCP_CONF_OPTS = \
 	--with-libbind=$(STAGING_DIR)/usr \
 	--with-randomdev=/dev/random \
-- 
2.27.0

[Buildroot] [PATCH 1/1] package/dhcp: specify host-gawk path for dhcp build environment

[Thomas Petazzoni]

Hello Sergey,

On Tue, 23 Jun 2020 10:39:18 +0300
Sergey Matyukevich <geomatsi@gmail.com> wrote:

> DHCP package silently fails to install its binaries if gawk tool
> is not installed on build server. This is still the case even
> if host-gawk package is already available: dhcp Makefiles detect
> its presence but still attempt to use gawk from host system.
> 
> Add host-gawk to the list of dependencies for dhcp and explicitely
> specify host-gawk location.
> 
> Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>

Thanks for the patch. However, could you be more specific about what's
missing on the target?

Indeed, I just did two builds of the DHCP package, will all sub-options
enabled, one inside a Docker container that only has the "mawk"
implementation of awk, and another on my machine, which as "gawk"
installed as awk.

And in both cases, the set of files installed to target/ was exactly
identical.

Could you provide more details ?

Thanks a lot!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 1/1] package/dhcp: specify host-gawk path for dhcp build environment

[Sergey Matyukevich]

Hello Thomas and all,

> Hello Sergey,
> 
> On Tue, 23 Jun 2020 10:39:18 +0300
> Sergey Matyukevich <geomatsi@gmail.com> wrote:
> 
> > DHCP package silently fails to install its binaries if gawk tool
> > is not installed on build server. This is still the case even
> > if host-gawk package is already available: dhcp Makefiles detect
> > its presence but still attempt to use gawk from host system.
> > 
> > Add host-gawk to the list of dependencies for dhcp and explicitely
> > specify host-gawk location.
> > 
> > Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
> 
> Thanks for the patch. However, could you be more specific about what's
> missing on the target?
> 
> Indeed, I just did two builds of the DHCP package, will all sub-options
> enabled, one inside a Docker container that only has the "mawk"
> implementation of awk, and another on my machine, which as "gawk"
> installed as awk.
> 
> And in both cases, the set of files installed to target/ was exactly
> identical.
> 
> Could you provide more details ?

But what were the files installed in your tests? Did you see dhcpd and
dhclient amount installed files in output/target ?

I still can reproduce the issue. The procedure is similar to yours. I
enable both client and server for isc-dhcp in the board config. Finally,
I do not have both dhcpd and dhclient in output/target if I do not have
gawk installed on my build server. Moreover, in this case the following
warnings appear in the end of dhcp build log:

bin/sh: line 19: gawk: command not found

Such warning are emitted by DHCP_INSTALL_SERVER and DHCP_INSTALL_CLIENT
commands from dhcp.mk. The same result can be reproduced manually:

$ make -C /home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server DESTDIR=/home/matsi/devel/code/buildroot/buildroot-release/output/target install-sbinPROGRAMS         
make: Entering directory '/home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server'
 /usr/bin/mkdir -p  '/home/matsi/devel/code/buildroot/buildroot-release/output/target/usr/sbin'
/bin/sh: line 19: gawk: command not found
make: Leaving directory '/home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server'

What exactly goes on can be figured out using '-n' make option.
Makefiles for isc-dhcp components use $(AWK) which is properly
detected by configure script. It can any of gawk, mawk, nawk, awk.
However in generated Makefiles that tool is used without specific 
path. As a result, *awk tool can be available in buildroot build
environment, but missing on the build server.

Regards,
Sergey

[Buildroot] [PATCH 1/1] package/dhcp: specify host-gawk path for dhcp build environment

[Thomas Petazzoni]

Hello Sergey,

On Sat, 12 Sep 2020 23:01:14 +0300
Sergey Matyukevich <geomatsi@gmail.com> wrote:

> But what were the files installed in your tests? Did you see dhcpd and
> dhclient amount installed files in output/target ?

Yes, both dhcpd and dhclient are installed in output/target. I'm
attaching to this e-mail the list of files that were installed when
building with awk==mawk and awk==gawk.

> I still can reproduce the issue. The procedure is similar to yours. I
> enable both client and server for isc-dhcp in the board config. Finally,
> I do not have both dhcpd and dhclient in output/target if I do not have
> gawk installed on my build server. Moreover, in this case the following
> warnings appear in the end of dhcp build log:
> 
> bin/sh: line 19: gawk: command not found
> 
> Such warning are emitted by DHCP_INSTALL_SERVER and DHCP_INSTALL_CLIENT
> commands from dhcp.mk. The same result can be reproduced manually:
> 
> $ make -C /home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server DESTDIR=/home/matsi/devel/code/buildroot/buildroot-release/output/target install-sbinPROGRAMS         
> make: Entering directory '/home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server'
>  /usr/bin/mkdir -p  '/home/matsi/devel/code/buildroot/buildroot-release/output/target/usr/sbin'
> /bin/sh: line 19: gawk: command not found
> make: Leaving directory '/home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server'
> 
> What exactly goes on can be figured out using '-n' make option.
> Makefiles for isc-dhcp components use $(AWK) which is properly
> detected by configure script. It can any of gawk, mawk, nawk, awk.
> However in generated Makefiles that tool is used without specific 
> path. As a result, *awk tool can be available in buildroot build
> environment, but missing on the build server.

I've posted at https://paste.ack.tf/52f197 at raw the full build of my
Buildroot configuration, which has just BR2_PACKAGE_DHCP=y + client +
server. awk is really mawk:

checking for gawk... no
checking for mawk... mawk

Both dhpcd and dhclient are installed:

$ find target/ -name 'dhcpd' -o -name 'dhclient'
target/usr/sbin/dhcpd
target/sbin/dhclient

I have this version of mawk installed:

ii  mawk                       1.3.3-17+b3                    amd64        a pattern scanning and text processing language

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 1/1] package/dhcp: specify host-gawk path for dhcp build environment

[Sergey Matyukevich]

Hello Thomas,

> Hello Sergey,
> 
> On Sat, 12 Sep 2020 23:01:14 +0300
> Sergey Matyukevich <geomatsi@gmail.com> wrote:
> 
> > But what were the files installed in your tests? Did you see dhcpd and
> > dhclient amount installed files in output/target ?
> 
> Yes, both dhcpd and dhclient are installed in output/target. I'm
> attaching to this e-mail the list of files that were installed when
> building with awk==mawk and awk==gawk.
> 
> > I still can reproduce the issue. The procedure is similar to yours. I
> > enable both client and server for isc-dhcp in the board config. Finally,
> > I do not have both dhcpd and dhclient in output/target if I do not have
> > gawk installed on my build server. Moreover, in this case the following
> > warnings appear in the end of dhcp build log:
> > 
> > bin/sh: line 19: gawk: command not found
> > 
> > Such warning are emitted by DHCP_INSTALL_SERVER and DHCP_INSTALL_CLIENT
> > commands from dhcp.mk. The same result can be reproduced manually:
> > 
> > $ make -C /home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server DESTDIR=/home/matsi/devel/code/buildroot/buildroot-release/output/target install-sbinPROGRAMS         
> > make: Entering directory '/home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server'
> >  /usr/bin/mkdir -p  '/home/matsi/devel/code/buildroot/buildroot-release/output/target/usr/sbin'
> > /bin/sh: line 19: gawk: command not found
> > make: Leaving directory '/home/matsi/devel/code/buildroot/buildroot-release/output/build/dhcp-4.4.1/server'
> > 
> > What exactly goes on can be figured out using '-n' make option.
> > Makefiles for isc-dhcp components use $(AWK) which is properly
> > detected by configure script. It can any of gawk, mawk, nawk, awk.
> > However in generated Makefiles that tool is used without specific 
> > path. As a result, *awk tool can be available in buildroot build
> > environment, but missing on the build server.
> 
> I've posted at https://paste.ack.tf/52f197 at raw the full build of my
> Buildroot configuration, which has just BR2_PACKAGE_DHCP=y + client +
> server. awk is really mawk:
> 
> checking for gawk... no
> checking for mawk... mawk
> 
> Both dhpcd and dhclient are installed:
> 
> $ find target/ -name 'dhcpd' -o -name 'dhclient'
> target/usr/sbin/dhcpd
> target/sbin/dhclient
> 
> I have this version of mawk installed:
> 
> ii  mawk                       1.3.3-17+b3                    amd64        a pattern scanning and text processing language

Let me clarify a possible root cause. IIUC, the problem occurs when
buildroot configuration and build host environment provide _different_
flavors of awk. For instance, mawk is available on host and  gawk is
available in buildroot output/host. In this case isc-dhcp configure
script detects gawk in output/host and generates Makefiles specifying
gawk without explicit path. Finally, during Buildroot installation
phase, those Makefiles are used to install dhcp binaries. They attempt
to use gawk without specific path, but host does not provide gawk.

Could you please take another look at your tests and check whether you
have different awk flavors in buildroot config and on build host ?
If yes, then my understanding is not correct. In this case I would
suggest to mark this patch as rejected for the time being.

Regards,
Sergey