[Buildroot] [PATCH 1/3] package/python-iptables: add explicit dependency on dynamic libs

[Buildroot] [PATCH 1/3] package/python-iptables: add explicit dependency on dynamic libs

[Frank Vanbever]

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
---
 package/python-iptables/Config.in | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/package/python-iptables/Config.in b/package/python-iptables/Config.in
index e55359963e..a35577bad3 100644
--- a/package/python-iptables/Config.in
+++ b/package/python-iptables/Config.in
@@ -1,7 +1,11 @@
 config BR2_PACKAGE_PYTHON_IPTABLES
 	bool "python-iptables"
+	depends on !BR2_STATIC_LIBS
 	select BR2_PACKAGE_IPTABLES # runtime dependency
 	help
 	  Python bindings for iptables.
 
 	  https://github.com/ldx/python-iptables
+
+comment "python-iptables needs a toolchain w/ dynamic library"
+	depends on BR2_STATIC_LIBS
-- 
2.20.1

[Buildroot] [PATCH 2/3] package/python-iptables: try known libc instead of find_library()

[Frank Vanbever]

ctypes.uitl.find_library() depends on gcc and friends to detect the location of
a given shared library. Given that these are not available on the target and
that python-iptables depends on this functionality we need to work around this.
The SONAMEs of the libc are well known so we try the known ones for glibc,
uClibc and musl.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=12271

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
---
 ...-Add-separate-mechanism-to-load-libc.patch | 90 +++++++++++++++++++
 1 file changed, 90 insertions(+)
 create mode 100644 package/python-iptables/0001-Add-separate-mechanism-to-load-libc.patch

diff --git a/package/python-iptables/0001-Add-separate-mechanism-to-load-libc.patch b/package/python-iptables/0001-Add-separate-mechanism-to-load-libc.patch
new file mode 100644
index 0000000000..1a928b4235
--- /dev/null
+++ b/package/python-iptables/0001-Add-separate-mechanism-to-load-libc.patch
@@ -0,0 +1,90 @@
+From e3557528d7cdcdc2c579212be8837bc9b54635a4 Mon Sep 17 00:00:00 2001
+From: Frank Vanbever <frank.vanbever@essensium.com>
+Date: Thu, 20 Feb 2020 12:14:08 +0100
+Subject: [PATCH] Add separate mechanism to load libc
+
+ctypes.util.find_library() always returns None for systems which do not have the
+tools installed to determine the location of a given shared library (i.e.
+ldconfig, gcc, objdump). If find_libary() fails attempt to load known libc by
+SONAME.
+
+Upstream: https://github.com/ldx/python-iptables/pull/300
+
+Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
+---
+ iptc/ip4tc.py   |  4 ++--
+ iptc/util.py    | 16 ++++++++++++++++
+ iptc/xtables.py |  4 ++--
+ 3 files changed, 20 insertions(+), 4 deletions(-)
+
+diff --git a/iptc/ip4tc.py b/iptc/ip4tc.py
+index 4c5d690..4ddd2dc 100644
+--- a/iptc/ip4tc.py
++++ b/iptc/ip4tc.py
+@@ -9,7 +9,7 @@ import socket
+ import struct
+ import weakref
+ 
+-from .util import find_library, load_kernel
++from .util import find_library, load_kernel, find_libc
+ from .xtables import (XT_INV_PROTO, NFPROTO_IPV4, XTablesError, xtables,
+                       xt_align, xt_counters, xt_entry_target, xt_entry_match)
+ 
+@@ -26,7 +26,7 @@ if not hasattr(socket, 'IPPROTO_SCTP'):
+ 
+ _IFNAMSIZ = 16
+ 
+-_libc = ct.CDLL("libc.so.6")
++_libc = find_libc()
+ _get_errno_loc = _libc.__errno_location
+ _get_errno_loc.restype = ct.POINTER(ct.c_int)
+ _malloc = _libc.malloc
+diff --git a/iptc/util.py b/iptc/util.py
+index ae5fb9b..e6b1649 100644
+--- a/iptc/util.py
++++ b/iptc/util.py
+@@ -109,3 +109,19 @@ def find_library(*names):
+             major = int(m.group(1))
+         return lib, major
+     return None, None
++
++
++def find_libc():
++    lib = ctypes.util.find_library('c')
++    if lib is not None:
++        return ctypes.CDLL(lib, mode=ctypes.RTLD_GLOBAL)
++
++    libnames = ['libc.so.6', 'libc.so.0', 'libc.so']
++    for name in libnames:
++        try:
++            lib = ctypes.CDLL(name, mode=ctypes.RTLD_GLOBAL)
++            return lib
++        except:
++            pass
++
++    return None
+diff --git a/iptc/xtables.py b/iptc/xtables.py
+index 93bc080..cf21029 100644
+--- a/iptc/xtables.py
++++ b/iptc/xtables.py
+@@ -6,7 +6,7 @@ import sys
+ import weakref
+ 
+ from . import version
+-from .util import find_library
++from .util import find_library, find_libc
+ from .errors import *
+ 
+ XT_INV_PROTO = 0x40  # invert the sense of PROTO
+@@ -792,7 +792,7 @@ class xtables_target(ct.Union):
+                 ("v12", _xtables_target_v12)]
+ 
+ 
+-_libc, _ = find_library("c")
++_libc = find_libc()
+ _optind = ct.c_long.in_dll(_libc, "optind")
+ _optarg = ct.c_char_p.in_dll(_libc, "optarg")
+ 
+-- 
+2.20.1
+
-- 
2.20.1

[Buildroot] [PATCH 2/3] package/python-iptables: try known libc instead of find_library()

[Thomas Petazzoni]

Hello Frank,

On Wed, 26 Feb 2020 15:26:16 +0100
Frank Vanbever <frank.vanbever@essensium.com> wrote:

> ctypes.uitl.find_library() depends on gcc and friends to detect the location of

Typo: uitl -> util.

> a given shared library. Given that these are not available on the target and
> that python-iptables depends on this functionality we need to work around this.
> The SONAMEs of the libc are well known so we try the known ones for glibc,
> uClibc and musl.
> 
> Fixes: https://bugs.busybox.net/show_bug.cgi?id=12271
> 
> Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>

I think the solution is nice and good, well done. I remember thinking
about the issue a while ago, but couldn't come up with a simple way of
detecting the C library file name from the target. Testing this limited
hardcoded set of SONAMEs seems like a reasonable approach.

> +Upstream: https://github.com/ldx/python-iptables/pull/300

It has even been merged upstream, so this could even be replaced by a
link to the upstream commit now.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 3/3] package/python-iptables: use installed iptables by default

[Frank Vanbever]

python-iptables depends on ctypes.util.find_library() which does not work due to
the absence of gcc and friends on target. The location of the xtables library
and the iptables modules can be configured through environment variables. Within
the scope of buildroot we can determine what these should be at build time and
replace the calls to os.getenv() with the correct value.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=12271

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
---
 package/python-iptables/Config.in          | 2 +-
 package/python-iptables/python-iptables.mk | 9 +++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/package/python-iptables/Config.in b/package/python-iptables/Config.in
index a35577bad3..0e24283d06 100644
--- a/package/python-iptables/Config.in
+++ b/package/python-iptables/Config.in
@@ -1,7 +1,7 @@
 config BR2_PACKAGE_PYTHON_IPTABLES
 	bool "python-iptables"
 	depends on !BR2_STATIC_LIBS
-	select BR2_PACKAGE_IPTABLES # runtime dependency
+	select BR2_PACKAGE_IPTABLES
 	help
 	  Python bindings for iptables.
 
diff --git a/package/python-iptables/python-iptables.mk b/package/python-iptables/python-iptables.mk
index 66e478a89a..086d5b2457 100644
--- a/package/python-iptables/python-iptables.mk
+++ b/package/python-iptables/python-iptables.mk
@@ -9,5 +9,14 @@ PYTHON_IPTABLES_SITE = https://files.pythonhosted.org/packages/08/5e/16a5ca35c42
 PYTHON_IPTABLES_SETUP_TYPE = setuptools
 PYTHON_IPTABLES_LICENSE = Apache-2.0
 PYTHON_IPTABLES_LICENSE_FILES = NOTICE
+PYTHON_IPTABLES_DEPENDENCIES = iptables
+
+define PYTHON_IPTABLES_SET_XTABLES_ENV_VARS
+	XTABLES_VERSION=`awk '/XTABLES_VERSION_CODE/ {print $$NF}' $(STAGING_DIR)/usr/include/xtables-version.h`; \
+	sed -i "s/os.getenv(\"PYTHON_IPTABLES_XTABLES_VERSION\")/$$XTABLES_VERSION/" $(@D)/iptc/xtables.py; \
+	sed -i "s/os.getenv(\"XTABLES_LIBDIR\")/\"\/usr\/lib\/xtables\"/" $(@D)/iptc/xtables.py
+endef
+
+PYTHON_IPTABLES_PRE_BUILD_HOOKS += PYTHON_IPTABLES_SET_XTABLES_ENV_VARS
 
 $(eval $(python-package))
-- 
2.20.1

[Buildroot] [PATCH 3/3] package/python-iptables: use installed iptables by default

[Thomas Petazzoni]

Hello Frank,

On Wed, 26 Feb 2020 15:26:17 +0100
Frank Vanbever <frank.vanbever@essensium.com> wrote:

> diff --git a/package/python-iptables/python-iptables.mk b/package/python-iptables/python-iptables.mk
> index 66e478a89a..086d5b2457 100644
> --- a/package/python-iptables/python-iptables.mk
> +++ b/package/python-iptables/python-iptables.mk
> @@ -9,5 +9,14 @@ PYTHON_IPTABLES_SITE = https://files.pythonhosted.org/packages/08/5e/16a5ca35c42
>  PYTHON_IPTABLES_SETUP_TYPE = setuptools
>  PYTHON_IPTABLES_LICENSE = Apache-2.0
>  PYTHON_IPTABLES_LICENSE_FILES = NOTICE
> +PYTHON_IPTABLES_DEPENDENCIES = iptables
> +
> +define PYTHON_IPTABLES_SET_XTABLES_ENV_VARS
> +	XTABLES_VERSION=`awk '/XTABLES_VERSION_CODE/ {print $$NF}' $(STAGING_DIR)/usr/include/xtables-version.h`; \
> +	sed -i "s/os.getenv(\"PYTHON_IPTABLES_XTABLES_VERSION\")/$$XTABLES_VERSION/" $(@D)/iptc/xtables.py; \
> +	sed -i "s/os.getenv(\"XTABLES_LIBDIR\")/\"\/usr\/lib\/xtables\"/" $(@D)/iptc/xtables.py

This last line is not using the XTABLES_VERSION variable, so it does
not need to be a continuation line from the previous commands.

Also, use % as a sed separator instead of / so that you don't have to
escape all the slashes.

But overall, is there a better way ? Like some official way to pass
these values at build time as setup.py options, with the environment
variable taking precedence if available ? That would make this
hopefully acceptable by upstream. Perhaps the setup.py logic could even
check the xtables-version.h by itself ?

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 3/3] package/python-iptables: use installed iptables by default

[Frank Vanbever]

Hi Thomas,

Thank you for the comments.

 > But overall, is there a better way ? Like some official way to pass
> these values at build time as setup.py options, with the environment
> variable taking precedence if available ? That would make this
> hopefully acceptable by upstream. Perhaps the setup.py logic could even
> check the xtables-version.h by itself ?

For installations from source this would work. [1] However Python also has the 
option of distributing Wheel binary packages, where this logic would not be 
executed. [2] This would mean that we put the burden of handling the difference 
between installations on the upstream.

I took a second look and actually I think I can propose a workaround to the 
upstream. There's 2 problems that need to be tackled.

The first one is loading libxtables.so. iptables installs a libxtables.so 
symlink without any version. CDLL has no issues with this so it can just try 
and load that if all the other attempts fail.

The second problem is the location of the XTABLES_LIBDIR. Right now it tries 
all paths that are mentioned in the output of ldconfig -N -v. I guess it 
wouldn't be that much of a stretch to test some sensible locations (/usr/
local/lib/xtables, /usr/lib/xtables etc) if that should fail, in the same vein 
as the libc solution. It's not clean but it gets the job done.

I'll see if I can whip something up and submit it upstream.

Best regards,
Frank

[1] https://stackoverflow.com/questions/20288711/post-install-script-with-python-setuptools
[2] https://github.com/pypa/setuptools/issues/1782

[Buildroot] [PATCH 3/3] package/python-iptables: use installed iptables by default

[Frank Vanbever]

Hi Thomas,

> I'll see if I can whip something up and submit it upstream.

I gave it another try but it seems that I was a bit too optimistic yesterday. 
The xtables module requires the ABI  version to cast the return values for 
matches and targets to something it can access.

I have a final last ditch theory about how this could be fixed, but to be honest 
it's really starting to look like more trouble than it's worth.

The ABI version information is embedded into struct xtables_match and 
 struct xtables_target. You should be able to get to these just from a handle 
to the libxtables.so and the location of the iptables extensions (i.e. 
xtables_libdir), for which you could use the same solution as for libc and 
test a number of sensible locations if ldconfig is unavailable.
The xtables_{match,target} structs always have the version field as the first 
field in the returned struct so I think you could do an 
xtables_find_{target,match} and do a ctypes.cast to a dummy ctypes.struct class 
with  just the version field.

The way the module is written though requires the ABI  version global variable 
to instantiate the xtables class which contains the logic that would allow you 
to find a match/target. So you get into this chicken or the egg type situation 
where you need to know the ABI version to get the workaround way to get the 
ABI version to work.


Best regards,
Frank

[Buildroot] [PATCH 1/3] package/python-iptables: add explicit dependency on dynamic libs

[Thomas Petazzoni]

On Wed, 26 Feb 2020 15:26:15 +0100
Frank Vanbever <frank.vanbever@essensium.com> wrote:

> Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
> ---
>  package/python-iptables/Config.in | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/package/python-iptables/Config.in b/package/python-iptables/Config.in
> index e55359963e..a35577bad3 100644
> --- a/package/python-iptables/Config.in
> +++ b/package/python-iptables/Config.in
> @@ -1,7 +1,11 @@
>  config BR2_PACKAGE_PYTHON_IPTABLES
>  	bool "python-iptables"
> +	depends on !BR2_STATIC_LIBS

This needs a bit more explanation, at least a short comment above. Even
your commit log is empty.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH v2 1/3] package/python-iptables: add explicit dependency on dynamic libs

[Frank Vanbever]

The package uses ctypes.CDLL extensively which only makes sense when dynamic
libraries are available.

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
---
v1 - > v2:
 - Add some additional explanation in commit message.
---
 package/python-iptables/Config.in | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/package/python-iptables/Config.in b/package/python-iptables/Config.in
index e55359963e..a35577bad3 100644
--- a/package/python-iptables/Config.in
+++ b/package/python-iptables/Config.in
@@ -1,7 +1,11 @@
 config BR2_PACKAGE_PYTHON_IPTABLES
 	bool "python-iptables"
+	depends on !BR2_STATIC_LIBS
 	select BR2_PACKAGE_IPTABLES # runtime dependency
 	help
 	  Python bindings for iptables.
 
 	  https://github.com/ldx/python-iptables
+
+comment "python-iptables needs a toolchain w/ dynamic library"
+	depends on BR2_STATIC_LIBS
-- 
2.20.1

[Buildroot] [PATCH v2 2/3] package/python-iptables: try known libc instead of find_library()

[Frank Vanbever]

ctypes.util.find_library() depends on gcc and friends to detect the location of
a given shared library. Given that these are not available on the target and
that python-iptables depends on this functionality we need to work around this.
The SONAMEs of the libc are well known so we try the known ones for glibc,
uClibc and musl.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=12271

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
----
v1 -> v2:
  - Change URL in patch from pull request to commit
---
 ...-Add-separate-mechanism-to-load-libc.patch | 90 +++++++++++++++++++
 1 file changed, 90 insertions(+)
 create mode 100644 package/python-iptables/0001-Add-separate-mechanism-to-load-libc.patch

diff --git a/package/python-iptables/0001-Add-separate-mechanism-to-load-libc.patch b/package/python-iptables/0001-Add-separate-mechanism-to-load-libc.patch
new file mode 100644
index 0000000000..2a344c2f55
--- /dev/null
+++ b/package/python-iptables/0001-Add-separate-mechanism-to-load-libc.patch
@@ -0,0 +1,90 @@
+From e3557528d7cdcdc2c579212be8837bc9b54635a4 Mon Sep 17 00:00:00 2001
+From: Frank Vanbever <frank.vanbever@essensium.com>
+Date: Thu, 20 Feb 2020 12:14:08 +0100
+Subject: [PATCH] Add separate mechanism to load libc
+
+ctypes.util.find_library() always returns None for systems which do not have the
+tools installed to determine the location of a given shared library (i.e.
+ldconfig, gcc, objdump). If find_libary() fails attempt to load known libc by
+SONAME.
+
+Upstream: https://github.com/ldx/python-iptables/commit/e3557528d7cdcdc2c579212be8837bc9b54635a4
+
+Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
+---
+ iptc/ip4tc.py   |  4 ++--
+ iptc/util.py    | 16 ++++++++++++++++
+ iptc/xtables.py |  4 ++--
+ 3 files changed, 20 insertions(+), 4 deletions(-)
+
+diff --git a/iptc/ip4tc.py b/iptc/ip4tc.py
+index 4c5d690..4ddd2dc 100644
+--- a/iptc/ip4tc.py
++++ b/iptc/ip4tc.py
+@@ -9,7 +9,7 @@ import socket
+ import struct
+ import weakref
+ 
+-from .util import find_library, load_kernel
++from .util import find_library, load_kernel, find_libc
+ from .xtables import (XT_INV_PROTO, NFPROTO_IPV4, XTablesError, xtables,
+                       xt_align, xt_counters, xt_entry_target, xt_entry_match)
+ 
+@@ -26,7 +26,7 @@ if not hasattr(socket, 'IPPROTO_SCTP'):
+ 
+ _IFNAMSIZ = 16
+ 
+-_libc = ct.CDLL("libc.so.6")
++_libc = find_libc()
+ _get_errno_loc = _libc.__errno_location
+ _get_errno_loc.restype = ct.POINTER(ct.c_int)
+ _malloc = _libc.malloc
+diff --git a/iptc/util.py b/iptc/util.py
+index ae5fb9b..e6b1649 100644
+--- a/iptc/util.py
++++ b/iptc/util.py
+@@ -109,3 +109,19 @@ def find_library(*names):
+             major = int(m.group(1))
+         return lib, major
+     return None, None
++
++
++def find_libc():
++    lib = ctypes.util.find_library('c')
++    if lib is not None:
++        return ctypes.CDLL(lib, mode=ctypes.RTLD_GLOBAL)
++
++    libnames = ['libc.so.6', 'libc.so.0', 'libc.so']
++    for name in libnames:
++        try:
++            lib = ctypes.CDLL(name, mode=ctypes.RTLD_GLOBAL)
++            return lib
++        except:
++            pass
++
++    return None
+diff --git a/iptc/xtables.py b/iptc/xtables.py
+index 93bc080..cf21029 100644
+--- a/iptc/xtables.py
++++ b/iptc/xtables.py
+@@ -6,7 +6,7 @@ import sys
+ import weakref
+ 
+ from . import version
+-from .util import find_library
++from .util import find_library, find_libc
+ from .errors import *
+ 
+ XT_INV_PROTO = 0x40  # invert the sense of PROTO
+@@ -792,7 +792,7 @@ class xtables_target(ct.Union):
+                 ("v12", _xtables_target_v12)]
+ 
+ 
+-_libc, _ = find_library("c")
++_libc = find_libc()
+ _optind = ct.c_long.in_dll(_libc, "optind")
+ _optarg = ct.c_char_p.in_dll(_libc, "optarg")
+ 
+-- 
+2.20.1
+
-- 
2.20.1

[Buildroot] [PATCH v2 3/3] package/python-iptables: use installed iptables by default

[Frank Vanbever]

python-iptables depends on ctypes.util.find_library() which does not work due to
the absence of gcc and friends on target. The location of the xtables library
and the iptables modules can be configured through environment variables. Within
the scope of buildroot we can determine what these should be at build time and
replace the calls to os.getenv() with the correct value.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=12271

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
---
v1 -> v2:
 - Remove unnecessary continuation in pre-build hook
 - Change sed separator to %

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
---
 package/python-iptables/Config.in          | 2 +-
 package/python-iptables/python-iptables.mk | 9 +++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/package/python-iptables/Config.in b/package/python-iptables/Config.in
index a35577bad3..0e24283d06 100644
--- a/package/python-iptables/Config.in
+++ b/package/python-iptables/Config.in
@@ -1,7 +1,7 @@
 config BR2_PACKAGE_PYTHON_IPTABLES
 	bool "python-iptables"
 	depends on !BR2_STATIC_LIBS
-	select BR2_PACKAGE_IPTABLES # runtime dependency
+	select BR2_PACKAGE_IPTABLES
 	help
 	  Python bindings for iptables.
 
diff --git a/package/python-iptables/python-iptables.mk b/package/python-iptables/python-iptables.mk
index 66e478a89a..38c6ee3950 100644
--- a/package/python-iptables/python-iptables.mk
+++ b/package/python-iptables/python-iptables.mk
@@ -9,5 +9,14 @@ PYTHON_IPTABLES_SITE = https://files.pythonhosted.org/packages/08/5e/16a5ca35c42
 PYTHON_IPTABLES_SETUP_TYPE = setuptools
 PYTHON_IPTABLES_LICENSE = Apache-2.0
 PYTHON_IPTABLES_LICENSE_FILES = NOTICE
+PYTHON_IPTABLES_DEPENDENCIES = iptables
+
+define PYTHON_IPTABLES_SET_XTABLES_ENV_VARS
+	XTABLES_VERSION=`awk '/XTABLES_VERSION_CODE/ {print $$NF}' $(STAGING_DIR)/usr/include/xtables-version.h`; \
+	sed -i "s%os.getenv(\"PYTHON_IPTABLES_XTABLES_VERSION\")%$$XTABLES_VERSION%" $(@D)/iptc/xtables.py
+	sed -i "s%os.getenv(\"XTABLES_LIBDIR\")%\"/usr/lib/xtables\"%" $(@D)/iptc/xtables.py
+endef
+
+PYTHON_IPTABLES_PRE_BUILD_HOOKS += PYTHON_IPTABLES_SET_XTABLES_ENV_VARS
 
 $(eval $(python-package))
-- 
2.20.1

[Buildroot] [PATCH v3 1/4] package/python-iptables: add explicit dependency on dynamic libs

[Frank Vanbever]

The package uses ctypes.CDLL extensively which only makes sense when dynamic
libraries are available.

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
---
v2 -> v3:
 - None
v1 -> v2:
 - Add some additional explanation in commit message.

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
---
 package/python-iptables/Config.in | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/package/python-iptables/Config.in b/package/python-iptables/Config.in
index e55359963e..a35577bad3 100644
--- a/package/python-iptables/Config.in
+++ b/package/python-iptables/Config.in
@@ -1,7 +1,11 @@
 config BR2_PACKAGE_PYTHON_IPTABLES
 	bool "python-iptables"
+	depends on !BR2_STATIC_LIBS
 	select BR2_PACKAGE_IPTABLES # runtime dependency
 	help
 	  Python bindings for iptables.
 
 	  https://github.com/ldx/python-iptables
+
+comment "python-iptables needs a toolchain w/ dynamic library"
+	depends on BR2_STATIC_LIBS
-- 
2.20.1

[Buildroot] [PATCH v3 2/4] package/python-iptables: try known libc instead of find_library()

[Frank Vanbever]

ctypes.util.find_library() depends on gcc and friends to detect the location of
a given shared library. Given that these are not available on the target and
that python-iptables depends on this functionality we need to work around this.
The SONAMEs of the libc are well known so we try the known ones for glibc,
uClibc and musl.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=12271

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
----
v2 -> v3:
  - None
v1 -> v2:
  - Change URL in patch from pull request to commit

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
---
 ...-Add-separate-mechanism-to-load-libc.patch | 90 +++++++++++++++++++
 1 file changed, 90 insertions(+)
 create mode 100644 package/python-iptables/0001-Add-separate-mechanism-to-load-libc.patch

diff --git a/package/python-iptables/0001-Add-separate-mechanism-to-load-libc.patch b/package/python-iptables/0001-Add-separate-mechanism-to-load-libc.patch
new file mode 100644
index 0000000000..2a344c2f55
--- /dev/null
+++ b/package/python-iptables/0001-Add-separate-mechanism-to-load-libc.patch
@@ -0,0 +1,90 @@
+From e3557528d7cdcdc2c579212be8837bc9b54635a4 Mon Sep 17 00:00:00 2001
+From: Frank Vanbever <frank.vanbever@essensium.com>
+Date: Thu, 20 Feb 2020 12:14:08 +0100
+Subject: [PATCH] Add separate mechanism to load libc
+
+ctypes.util.find_library() always returns None for systems which do not have the
+tools installed to determine the location of a given shared library (i.e.
+ldconfig, gcc, objdump). If find_libary() fails attempt to load known libc by
+SONAME.
+
+Upstream: https://github.com/ldx/python-iptables/commit/e3557528d7cdcdc2c579212be8837bc9b54635a4
+
+Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
+---
+ iptc/ip4tc.py   |  4 ++--
+ iptc/util.py    | 16 ++++++++++++++++
+ iptc/xtables.py |  4 ++--
+ 3 files changed, 20 insertions(+), 4 deletions(-)
+
+diff --git a/iptc/ip4tc.py b/iptc/ip4tc.py
+index 4c5d690..4ddd2dc 100644
+--- a/iptc/ip4tc.py
++++ b/iptc/ip4tc.py
+@@ -9,7 +9,7 @@ import socket
+ import struct
+ import weakref
+ 
+-from .util import find_library, load_kernel
++from .util import find_library, load_kernel, find_libc
+ from .xtables import (XT_INV_PROTO, NFPROTO_IPV4, XTablesError, xtables,
+                       xt_align, xt_counters, xt_entry_target, xt_entry_match)
+ 
+@@ -26,7 +26,7 @@ if not hasattr(socket, 'IPPROTO_SCTP'):
+ 
+ _IFNAMSIZ = 16
+ 
+-_libc = ct.CDLL("libc.so.6")
++_libc = find_libc()
+ _get_errno_loc = _libc.__errno_location
+ _get_errno_loc.restype = ct.POINTER(ct.c_int)
+ _malloc = _libc.malloc
+diff --git a/iptc/util.py b/iptc/util.py
+index ae5fb9b..e6b1649 100644
+--- a/iptc/util.py
++++ b/iptc/util.py
+@@ -109,3 +109,19 @@ def find_library(*names):
+             major = int(m.group(1))
+         return lib, major
+     return None, None
++
++
++def find_libc():
++    lib = ctypes.util.find_library('c')
++    if lib is not None:
++        return ctypes.CDLL(lib, mode=ctypes.RTLD_GLOBAL)
++
++    libnames = ['libc.so.6', 'libc.so.0', 'libc.so']
++    for name in libnames:
++        try:
++            lib = ctypes.CDLL(name, mode=ctypes.RTLD_GLOBAL)
++            return lib
++        except:
++            pass
++
++    return None
+diff --git a/iptc/xtables.py b/iptc/xtables.py
+index 93bc080..cf21029 100644
+--- a/iptc/xtables.py
++++ b/iptc/xtables.py
+@@ -6,7 +6,7 @@ import sys
+ import weakref
+ 
+ from . import version
+-from .util import find_library
++from .util import find_library, find_libc
+ from .errors import *
+ 
+ XT_INV_PROTO = 0x40  # invert the sense of PROTO
+@@ -792,7 +792,7 @@ class xtables_target(ct.Union):
+                 ("v12", _xtables_target_v12)]
+ 
+ 
+-_libc, _ = find_library("c")
++_libc = find_libc()
+ _optind = ct.c_long.in_dll(_libc, "optind")
+ _optarg = ct.c_char_p.in_dll(_libc, "optarg")
+ 
+-- 
+2.20.1
+
-- 
2.20.1

[Buildroot] [PATCH v3 3/4] package/python-iptables: use installed iptables by default

[Frank Vanbever]

python-iptables depends on ctypes.util.find_library() which does not work due to
the absence of gcc and friends on target. The location of the xtables library
and the iptables modules can be configured through environment variables. Within
the scope of buildroot we can determine what these should be at build time and
replace the calls to os.getenv() with the correct value.

Fixes: https://bugs.busybox.net/show_bug.cgi?id=12271

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
---
v2 -> v3:
 - None
v1 -> v2:
 - Remove unnecessary continuation in pre-build hook
 - Change sed separator to %

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
---
 package/python-iptables/Config.in          | 2 +-
 package/python-iptables/python-iptables.mk | 9 +++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/package/python-iptables/Config.in b/package/python-iptables/Config.in
index a35577bad3..0e24283d06 100644
--- a/package/python-iptables/Config.in
+++ b/package/python-iptables/Config.in
@@ -1,7 +1,7 @@
 config BR2_PACKAGE_PYTHON_IPTABLES
 	bool "python-iptables"
 	depends on !BR2_STATIC_LIBS
-	select BR2_PACKAGE_IPTABLES # runtime dependency
+	select BR2_PACKAGE_IPTABLES
 	help
 	  Python bindings for iptables.
 
diff --git a/package/python-iptables/python-iptables.mk b/package/python-iptables/python-iptables.mk
index 66e478a89a..38c6ee3950 100644
--- a/package/python-iptables/python-iptables.mk
+++ b/package/python-iptables/python-iptables.mk
@@ -9,5 +9,14 @@ PYTHON_IPTABLES_SITE = https://files.pythonhosted.org/packages/08/5e/16a5ca35c42
 PYTHON_IPTABLES_SETUP_TYPE = setuptools
 PYTHON_IPTABLES_LICENSE = Apache-2.0
 PYTHON_IPTABLES_LICENSE_FILES = NOTICE
+PYTHON_IPTABLES_DEPENDENCIES = iptables
+
+define PYTHON_IPTABLES_SET_XTABLES_ENV_VARS
+	XTABLES_VERSION=`awk '/XTABLES_VERSION_CODE/ {print $$NF}' $(STAGING_DIR)/usr/include/xtables-version.h`; \
+	sed -i "s%os.getenv(\"PYTHON_IPTABLES_XTABLES_VERSION\")%$$XTABLES_VERSION%" $(@D)/iptc/xtables.py
+	sed -i "s%os.getenv(\"XTABLES_LIBDIR\")%\"/usr/lib/xtables\"%" $(@D)/iptc/xtables.py
+endef
+
+PYTHON_IPTABLES_PRE_BUILD_HOOKS += PYTHON_IPTABLES_SET_XTABLES_ENV_VARS
 
 $(eval $(python-package))
-- 
2.20.1

Re: [Buildroot] [PATCH v3 3/4] package/python-iptables: use installed iptables by default

[Arnout Vandecappelle]

On 11/03/2020 11:54, Frank Vanbever wrote:
> python-iptables depends on ctypes.util.find_library() which does not work due to
> the absence of gcc and friends on target. The location of the xtables library
> and the iptables modules can be configured through environment variables. Within
> the scope of buildroot we can determine what these should be at build time and
> replace the calls to os.getenv() with the correct value.
> 
> Fixes: https://bugs.busybox.net/show_bug.cgi?id=12271
> 
> Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>

  This one had fallen through the cracks. I finally applied it to master.

  To verify that it really solves a bug, I also added a runtime test which 
indeed fails without this patch.

  Regards,
  Arnout

> ---
> v2 -> v3:
>   - None
> v1 -> v2:
>   - Remove unnecessary continuation in pre-build hook
>   - Change sed separator to %
> 
> Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
> ---
>   package/python-iptables/Config.in          | 2 +-
>   package/python-iptables/python-iptables.mk | 9 +++++++++
>   2 files changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/package/python-iptables/Config.in b/package/python-iptables/Config.in
> index a35577bad3..0e24283d06 100644
> --- a/package/python-iptables/Config.in
> +++ b/package/python-iptables/Config.in
> @@ -1,7 +1,7 @@
>   config BR2_PACKAGE_PYTHON_IPTABLES
>   	bool "python-iptables"
>   	depends on !BR2_STATIC_LIBS
> -	select BR2_PACKAGE_IPTABLES # runtime dependency
> +	select BR2_PACKAGE_IPTABLES
>   	help
>   	  Python bindings for iptables.
>   
> diff --git a/package/python-iptables/python-iptables.mk b/package/python-iptables/python-iptables.mk
> index 66e478a89a..38c6ee3950 100644
> --- a/package/python-iptables/python-iptables.mk
> +++ b/package/python-iptables/python-iptables.mk
> @@ -9,5 +9,14 @@ PYTHON_IPTABLES_SITE = https://files.pythonhosted.org/packages/08/5e/16a5ca35c42
>   PYTHON_IPTABLES_SETUP_TYPE = setuptools
>   PYTHON_IPTABLES_LICENSE = Apache-2.0
>   PYTHON_IPTABLES_LICENSE_FILES = NOTICE
> +PYTHON_IPTABLES_DEPENDENCIES = iptables
> +
> +define PYTHON_IPTABLES_SET_XTABLES_ENV_VARS
> +	XTABLES_VERSION=`awk '/XTABLES_VERSION_CODE/ {print $$NF}' $(STAGING_DIR)/usr/include/xtables-version.h`; \
> +	sed -i "s%os.getenv(\"PYTHON_IPTABLES_XTABLES_VERSION\")%$$XTABLES_VERSION%" $(@D)/iptc/xtables.py
> +	sed -i "s%os.getenv(\"XTABLES_LIBDIR\")%\"/usr/lib/xtables\"%" $(@D)/iptc/xtables.py
> +endef
> +
> +PYTHON_IPTABLES_PRE_BUILD_HOOKS += PYTHON_IPTABLES_SET_XTABLES_ENV_VARS
>   
>   $(eval $(python-package))
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 4/4] package/python-iptables: account for platform tag in extensions

[Frank Vanbever]

EXT_SUFFIX in Python versions > 3.5 contains a platform tag which only applies
to cpython extensions. Given that ctypes.util.find_library does not work on the
target due to the absence of the underlying tools '.so' needs to be added as a
possible suffix for libraries to enable python-iptables to find the iptables
shared libraries.

Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
---
v2 -> v3:
   - Add additional patch to fix python-iptables for python > 3.5
---
 ...o-as-additional-shared-object-suffix.patch | 57 +++++++++++++++++++
 1 file changed, 57 insertions(+)
 create mode 100644 package/python-iptables/0002-Add-.so-as-additional-shared-object-suffix.patch

diff --git a/package/python-iptables/0002-Add-.so-as-additional-shared-object-suffix.patch b/package/python-iptables/0002-Add-.so-as-additional-shared-object-suffix.patch
new file mode 100644
index 0000000000..41d49fc1e6
--- /dev/null
+++ b/package/python-iptables/0002-Add-.so-as-additional-shared-object-suffix.patch
@@ -0,0 +1,57 @@
+From 899d25c511c6ce779b7153e9ae2e41055b30b9c5 Mon Sep 17 00:00:00 2001
+From: Frank Vanbever <frank.vanbever@essensium.com>
+Date: Mon, 9 Mar 2020 12:36:47 +0100
+Subject: [PATCH] Add '.so' as additional shared object suffix
+
+EXT_SUFFIX includes a platform information tag starting from Python 3.5 [0]
+For example:
+
+    >>> sysconfig.get_config_var("EXT_SUFFIX")
+    '.cpython-38-aarch64-linux-gnu.so'
+
+This suffix only applies to cpython extensions i.e. not to the iptables shared
+objects.
+
+Adding '.so' as an additional suffix for shared objects fixes the issue.
+
+Fixes: Issue #301
+
+Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
+
+Backported from: 899d25c511c6ce779b7153e9ae2e41055b30b9c5
+
+[0]: https://docs.python.org/3/whatsnew/3.5.html#build-and-c-api-changes
+---
+ iptc/util.py | 13 ++++++++++---
+ 1 file changed, 10 insertions(+), 3 deletions(-)
+
+diff --git a/iptc/util.py b/iptc/util.py
+index e6b1649..04fe905 100644
+--- a/iptc/util.py
++++ b/iptc/util.py
+@@ -80,12 +80,19 @@ def _do_find_library(name):
+ 
+ 
+ def _find_library(*names):
++    exts = []
+     if version_info >= (3, 3):
+-        ext = get_config_var("EXT_SUFFIX")
++        exts.append(get_config_var("EXT_SUFFIX"))
+     else:
+-        ext = get_config_var('SO')
++        exts.append(get_config_var('SO'))
++
++    if version_info >= (3, 5):
++        exts.append('.so')
++
+     for name in names:
+-        libnames = [name, "lib" + name, name + ext, "lib" + name + ext]
++        libnames = [name, "lib" + name]
++        for ext in exts:
++            libnames += [name + ext, "lib" + name + ext]
+         libdir = os.environ.get('IPTABLES_LIBDIR', None)
+         if libdir is not None:
+             libdirs = libdir.split(':')
+-- 
+2.20.1
+
-- 
2.20.1

[Buildroot] [PATCH v3 1/4] package/python-iptables: add explicit dependency on dynamic libs

[Yann E. MORIN]

Frank, All,

On 2020-03-11 11:54 +0100, Frank Vanbever spake thusly:
> The package uses ctypes.CDLL extensively which only makes sense when dynamic
> libraries are available.
> 
> Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>

Patches 1, 2, and 4, which are all backports of upstream commits,
applied to master, thanks.

I'm still pondering how to improve patch 3, though, as I concur with
Thomas that it would be nice if we could do better... I'm not gonna
sweat it, though...

Regards,
Yann E. MORIN.

> ---
> v2 -> v3:
>  - None
> v1 -> v2:
>  - Add some additional explanation in commit message.
> 
> Signed-off-by: Frank Vanbever <frank.vanbever@essensium.com>
> ---
>  package/python-iptables/Config.in | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/package/python-iptables/Config.in b/package/python-iptables/Config.in
> index e55359963e..a35577bad3 100644
> --- a/package/python-iptables/Config.in
> +++ b/package/python-iptables/Config.in
> @@ -1,7 +1,11 @@
>  config BR2_PACKAGE_PYTHON_IPTABLES
>  	bool "python-iptables"
> +	depends on !BR2_STATIC_LIBS
>  	select BR2_PACKAGE_IPTABLES # runtime dependency
>  	help
>  	  Python bindings for iptables.
>  
>  	  https://github.com/ldx/python-iptables
> +
> +comment "python-iptables needs a toolchain w/ dynamic library"
> +	depends on BR2_STATIC_LIBS
> -- 
> 2.20.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'