[PATCH] dhcp: use included bind version

[PATCH] dhcp: use included bind version

[Vyacheslav Yurkov]

From: Vyacheslav Yurkov <Vyacheslav.Yurkov@bruker.com>

ISC DHCP should link against BIND9 libraries, which are tailored
specifically for DHCP. BIND9 package in Yocto core layer has different
configuraiton, in particular it has threads and epoll enabled.

ISC DHCP isn't a multi-threaded application, running it with bind9
libraries compiled in with threading enabled is not something ISC can
vouch for.

BIND9 libraries support a lot of options specifically geared towards
optimizing DNS operations, many of which do not play nicely with
ISC DHCP's "architecture".  It isn't necessarily practical to build
those libraries for both purposes.

If threading is enabled, then DHCP sporadically fails/crashes with
messages like:

lib/isc/unix/socket.c:1054: epoll_ctl(DEL), 6: Bad file descriptor
lib/isc/unix/socket.c:3332: INSIST(!sock->pending_send) failed.

Signed-off-by: Vyacheslav Yurkov <Vyacheslav.Yurkov@bruker.com>
---
 meta/recipes-connectivity/dhcp/dhcp.inc       |  6 ++---
 .../0014-Use-static-version-of-libbind.patch  | 23 +++++++++++++++++++
 meta/recipes-connectivity/dhcp/dhcp_4.4.2.bb  |  1 +
 3 files changed, 27 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-connectivity/dhcp/dhcp/0014-Use-static-version-of-libbind.patch

diff --git a/meta/recipes-connectivity/dhcp/dhcp.inc b/meta/recipes-connectivity/dhcp/dhcp.inc
index d46130d49b..466cdf921a 100644
--- a/meta/recipes-connectivity/dhcp/dhcp.inc
+++ b/meta/recipes-connectivity/dhcp/dhcp.inc
@@ -10,7 +10,7 @@ HOMEPAGE = "http://www.isc.org/"
 LICENSE = "ISC"
 LIC_FILES_CHKSUM = "file://LICENSE;beginline=4;md5=004a4db50a1e20972e924a8618747c01"
 
-DEPENDS = "openssl bind"
+DEPENDS = "openssl zlib"
 
 SRC_URI = "http://ftp.isc.org/isc/dhcp/${PV}/dhcp-${PV}.tar.gz \
            file://init-relay file://default-relay \
@@ -50,9 +50,9 @@ EXTRA_OECONF = "--with-srv-lease-file=${localstatedir}/lib/dhcp/dhcpd.leases \
                 --with-cli6-lease-file=${localstatedir}/lib/dhcp/dhclient6.leases \
                 --enable-paranoia --disable-static \
                 --with-randomdev=/dev/random \
-                --with-libbind=${STAGING_DIR_HOST} \
-		--enable-libtool \
+                --enable-libtool \
                "
+PARALLEL_MAKE = ""
 
 #Enable shared libs per dhcp README
 do_configure_prepend () {
diff --git a/meta/recipes-connectivity/dhcp/dhcp/0014-Use-static-version-of-libbind.patch b/meta/recipes-connectivity/dhcp/dhcp/0014-Use-static-version-of-libbind.patch
new file mode 100644
index 0000000000..63c74b25a5
--- /dev/null
+++ b/meta/recipes-connectivity/dhcp/dhcp/0014-Use-static-version-of-libbind.patch
@@ -0,0 +1,23 @@
+From a96f65117d63bea4287f19e80595f7aab282410a Mon Sep 17 00:00:00 2001
+From: Vyacheslav Yurkov <Vyacheslav.Yurkov@bruker.com>
+Date: Wed, 10 Jun 2020 07:56:20 +0200
+Subject: [PATCH] Use static version of libbind
+
+Signed-off-by: Vyacheslav Yurkov <Vyacheslav.Yurkov@bruker.com>
+---
+ bind/Makefile.in | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/bind/Makefile.in b/bind/Makefile.in
+index 8fe8883..902fc3d 100644
+--- a/bind/Makefile.in
++++ b/bind/Makefile.in
+@@ -22,7 +22,7 @@ prefix = @prefix@
+ exec_prefix = @exec_prefix@
+ 
+ bindconfig = --without-openssl --without-libxml2 --without-libjson \
+-	--without-gssapi --disable-threads --without-lmdb \
++	--without-gssapi --disable-threads --without-lmdb --disable-shared \
+ 	--includedir=@includedir@ --libdir=@libdir@  --without-python\
+ 	@BINDLT@ @BINDIOMUX@ @BINDCONFIG@ --enable-full-report
+ 
diff --git a/meta/recipes-connectivity/dhcp/dhcp_4.4.2.bb b/meta/recipes-connectivity/dhcp/dhcp_4.4.2.bb
index b56a204821..a322794e60 100644
--- a/meta/recipes-connectivity/dhcp/dhcp_4.4.2.bb
+++ b/meta/recipes-connectivity/dhcp/dhcp_4.4.2.bb
@@ -10,6 +10,7 @@ SRC_URI += "file://0001-define-macro-_PATH_DHCPD_CONF-and-_PATH_DHCLIENT_CON.pat
             file://0012-dhcp-correct-the-intention-for-xml2-lib-search.patch \
             file://0013-fixup_use_libbind.patch \
             file://0001-workaround-busybox-limitation-in-linux-dhclient-script.patch \
+            file://0014-Use-static-version-of-libbind.patch \
 "
 
 SRC_URI[md5sum] = "2afdaf8498dc1edaf3012efdd589b3e1"
-- 
2.25.1

✗ patchtest: failure for dhcp: use included bind version

[Patchwork]

== Series Details ==

Series: dhcp: use included bind version
Revision: 1
URL   : https://patchwork.openembedded.org/series/24576/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Added patch file is missing Upstream-Status in the header [test_upstream_status_presence_format] 
  Suggested fix    Add Upstream-Status: <Valid status> to the header of meta/recipes-connectivity/dhcp/dhcp/0014-Use-static-version-of-libbind.patch
  Standard format  Upstream-Status: <Valid status>
  Valid status     Pending, Accepted, Backport, Denied, Inappropriate [reason], Submitted [where]



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

Re: [OE-core] [PATCH] dhcp: use included bind version

[Richard Purdie]

On Wed, 2020-06-10 at 16:35 +0200, Vyacheslav Yurkov wrote:
> From: Vyacheslav Yurkov <Vyacheslav.Yurkov@bruker.com>
> 
> ISC DHCP should link against BIND9 libraries, which are tailored
> specifically for DHCP. BIND9 package in Yocto core layer has
> different
> configuraiton, in particular it has threads and epoll enabled.
> 
> ISC DHCP isn't a multi-threaded application, running it with bind9
> libraries compiled in with threading enabled is not something ISC can
> vouch for.
> 
> BIND9 libraries support a lot of options specifically geared towards
> optimizing DNS operations, many of which do not play nicely with
> ISC DHCP's "architecture".  It isn't necessarily practical to build
> those libraries for both purposes.
> 
> If threading is enabled, then DHCP sporadically fails/crashes with
> messages like:
> 
> lib/isc/unix/socket.c:1054: epoll_ctl(DEL), 6: Bad file descriptor
> lib/isc/unix/socket.c:3332: INSIST(!sock->pending_send) failed.

You seem to disable parallel make yet no mention of that in the log?

Why doesn't dhcp build and link against a static libbind by default if
these issues are present the the dhcp authors don't want to work with
such libbinds?

Cheers,

Richard

Re: [OE-core] [PATCH] dhcp: use included bind version

[Vyacheslav Yurkov]

On 10.06.2020 17:05, Richard Purdie wrote:
> You seem to disable parallel make yet no mention of that in the log?
Right, I forgot to mention that in the commit message. Building bind at 
'compile' stage breaks parallel make. I assume that can be addressed, 
but I'd need to take a look at how to set that dependency with autotools.
> Why doesn't dhcp build and link against a static libbind by default if
> these issues are present the the dhcp authors don't want to work with
> such libbinds?
That's a good question. I assume the alternative is disable threads in 
bind recipes then a shared library can be used by both, dhcp and named 
(DNS). Hopefully Thomas from ISC (in CC) could clarify this better than me.
>
> Cheers,
>
> Richard
>

Regards,
Vyacheslav

Re: [OE-core] [PATCH] dhcp: use included bind version

[Khem Raj]

On Wednesday, June 10, 2020 9:36:52 AM PDT Vyacheslav Yurkov wrote:
> On 10.06.2020 17:05, Richard Purdie wrote:
> > You seem to disable parallel make yet no mention of that in the log?
> 
> Right, I forgot to mention that in the commit message. Building bind at
> 'compile' stage breaks parallel make. I assume that can be addressed,
> but I'd need to take a look at how to set that dependency with autotools.
> 
> > Why doesn't dhcp build and link against a static libbind by default if
> > these issues are present the the dhcp authors don't want to work with
> > such libbinds?
> 
> That's a good question. I assume the alternative is disable threads in
> bind recipes then a shared library can be used by both, dhcp and named
> (DNS). Hopefully Thomas from ISC (in CC) could clarify this better than me.
> 

I think it would be good to share if we can. Shipping two versions of bind 
libraries is less than ideal, have you explored disabling threading in the 
bind recipe and see if named is happy with that ?

> > Cheers,
> > 
> > Richard
> 
> Regards,
> Vyacheslav

Re: [OE-core] [PATCH] dhcp: use included bind version

[Vyacheslav Yurkov]

On 13.06.2020 02:41, Khem Raj wrote:
> I think it would be good to share if we can. Shipping two versions of bind
> libraries is less than ideal, have you explored disabling threading in the
> bind recipe and see if named is happy with that ?
>
Yes, I have. It partially solves the issue. At least DHCP doesn't 
crash/fail anymore, but warnings from epoll enabled are still seen in 
the logs. The other question is do we have to strip down BIND 
functionality in order to make DHCP work?

I understand that maintenance effort is at stake and we are looking for 
the best possible solution here, but IMHO having one set of bind 
libraries for both BIND and DHCP servers is not one of them.

We could also go Ubuntu way here, by having two bind-9.11.x for DHCP and 
upgrading BIND server to the latest stable (9.16.x if I'm not mistaken). 
Then security support could be only done for 9.11.x version, the latest 
stable should have most of security patches included already.

Vyacheslav

Re: [OE-core] [PATCH] dhcp: use included bind version

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 1228 bytes --]

This brings me to a question. Should we replace dhcp with kea?

https://www.isc.org/kea/

Alex

On Sat, 13 Jun 2020 at 10:31, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:

> On 13.06.2020 02:41, Khem Raj wrote:
> > I think it would be good to share if we can. Shipping two versions of
> bind
> > libraries is less than ideal, have you explored disabling threading in
> the
> > bind recipe and see if named is happy with that ?
> >
> Yes, I have. It partially solves the issue. At least DHCP doesn't
> crash/fail anymore, but warnings from epoll enabled are still seen in
> the logs. The other question is do we have to strip down BIND
> functionality in order to make DHCP work?
>
> I understand that maintenance effort is at stake and we are looking for
> the best possible solution here, but IMHO having one set of bind
> libraries for both BIND and DHCP servers is not one of them.
>
> We could also go Ubuntu way here, by having two bind-9.11.x for DHCP and
> upgrading BIND server to the latest stable (9.16.x if I'm not mistaken).
> Then security support could be only done for 9.11.x version, the latest
> stable should have most of security patches included already.
>
> Vyacheslav
> 
>

[-- Attachment #2: Type: text/html, Size: 1664 bytes --]

Re: [OE-core] [PATCH] dhcp: use included bind version

[Adrian Bunk]

On Wed, Jun 10, 2020 at 04:35:20PM +0200, Vyacheslav Yurkov wrote:
>...
> BIND9 libraries support a lot of options specifically geared towards
> optimizing DNS operations, many of which do not play nicely with
> ISC DHCP's "architecture".  It isn't necessarily practical to build
> those libraries for both purposes.
>...

Providing security support for several different copies of the BIND
code would be a nightmare.

cu
Adrian

Re: [OE-core] [PATCH] dhcp: use included bind version

[Vyacheslav Yurkov]

On 10.06.2020 18:07, Adrian Bunk wrote:
> On Wed, Jun 10, 2020 at 04:35:20PM +0200, Vyacheslav Yurkov wrote:
>> ...
>> BIND9 libraries support a lot of options specifically geared towards
>> optimizing DNS operations, many of which do not play nicely with
>> ISC DHCP's "architecture".  It isn't necessarily practical to build
>> those libraries for both purposes.
>> ...
> Providing security support for several different copies of the BIND
> code would be a nightmare.
>
> cu
> Adrian

Right, but I don't think there's a way to build two configurations out 
of one recipe, isn't it?

Vyacheslav

Re: [OE-core] [PATCH] dhcp: use included bind version

[Adrian Bunk]

On Wed, Jun 10, 2020 at 06:31:55PM +0200, Vyacheslav Yurkov wrote:
> On 10.06.2020 18:07, Adrian Bunk wrote:
> > On Wed, Jun 10, 2020 at 04:35:20PM +0200, Vyacheslav Yurkov wrote:
> > > ...
> > > BIND9 libraries support a lot of options specifically geared towards
> > > optimizing DNS operations, many of which do not play nicely with
> > > ISC DHCP's "architecture".  It isn't necessarily practical to build
> > > those libraries for both purposes.
> > > ...
> > Providing security support for several different copies of the BIND
> > code would be a nightmare.
> > 
> > cu
> > Adrian
> 
> Right, but I don't think there's a way to build two configurations out of
> one recipe, isn't it?

It is possible.
More common is sharing the sources between two (or more) recipes
with an .inc file.

Unrelated to that, I am a bit suprised why the current Yocto setup
is suddenly a problem. Other distributions like Debian/Ubuntu are
also sharing one build of the BIND libraries between BIND and dhcpd.

> Vyacheslav

cu
Adrian

Re: [OE-core] [PATCH] dhcp: use included bind version

[Vyacheslav Yurkov]

On 10.06.2020 19:42, Adrian Bunk wrote:
>> Right, but I don't think there's a way to build two configurations out of
>> one recipe, isn't it?
> It is possible.
> More common is sharing the sources between two (or more) recipes
> with an .inc file.
That's not really what I had in mind. That would result in two set of 
recipes anyway, perhaps with the same SRC_URI.
>
> Unrelated to that, I am a bit suprised why the current Yocto setup
> is suddenly a problem. Other distributions like Debian/Ubuntu are
> also sharing one build of the BIND libraries between BIND and dhcpd.
>
The thing is, they don't. This is the 'depends' output from my Ubuntu 
20.04 installation:
% sudo apt-cache depends isc-dhcp-server
...
   Depends: libdns-export1109
   Depends: libirs-export161
   Depends: libisc-export1105

% sudo apt-cache depends bind9
bind9
...
   Depends: bind9-libs

bind9-libs is bind-9.16.1, but export libraries are bind-9.11.16.

So answering Richard's question, having shared linking is also possible, 
but that would result in two sets of bind libraries.

> cu
> Adrian

Regards,
Vyacheslav