iptables: response to modern linkers, and symbol maps

iptables: response to modern linkers, and symbol maps

[Jan Engelhardt]

The following changes since commit 57ccd9b9b02d93091fa162c854d57073d7ad64d4:
  Merge branch 'stable' (2011-12-30 02:37:31 +0100) [temp]

are available FOR REVIEW in the git repository at:
  git://dev.medozas.de/iptables master
and proposed for:
  master

Description:
  Changes to accomodate contemporary tool chain and distro-level build
  systems. Not a bugfix, and since it emits e.g. new rpm symbols for
  distros, this is not for the stable branch.


Jan Engelhardt (3):
      libiptc: remove libiptc.so
      libiptc: add symbol maps to libiptc
      libipq: add symbol map to libipq

 configure.ac         |    5 ----
 libipq/Makefile.am   |    8 ++++++-
 libipq/libipq.map    |   28 ++++++++++++++++++++++++++
 libiptc/Makefile.am  |   15 ++++++++-----
 libiptc/libip4tc.map |   52 +++++++++++++++++++++++++++++++++++++++++++++++++
 libiptc/libip6tc.map |   53 ++++++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 149 insertions(+), 12 deletions(-)
 create mode 100644 libipq/libipq.map
 create mode 100644 libiptc/libip4tc.map
 create mode 100644 libiptc/libip6tc.map

[PATCH 1/3] libiptc: remove libiptc.so

[Jan Engelhardt]

When Linux distributions started to ship binutils-ld defaulting to
-Wl,--as-needed, we had to add a hack (-Wl,--no-as-needed) to the
iptables build such that libiptc.so continued to link to libip4tc.so
and libip6tc.so despite not requesting any symbols from there.

Given that distros start shipping binutils-ld defaulting to
-Wl,--no-copy-dt-needed-entries, 3rd-party programs linking
to -liptc will ignore libiptc.so's DT_NEEDED entries by default and
not search for symbols there. So libiptc.so has finally become
useless, so remove it. Dependent programs should have been using the
pkg-config infrastructure, where quering for "iptc" returned "-lip4tc
-lip6tc" for long enough.

Signed-off-by: Jan Engelhardt <jengelh@medozas.de>
---
 configure.ac        |    5 -----
 libiptc/Makefile.am |    7 ++-----
 2 files changed, 2 insertions(+), 10 deletions(-)

diff --git a/configure.ac b/configure.ac
index 0f85b83..8eb7572 100644
--- a/configure.ac
+++ b/configure.ac
@@ -53,11 +53,6 @@ AC_ARG_WITH([pkgconfigdir], AS_HELP_STRING([--with-pkgconfigdir=PATH],
 	[Path to the pkgconfig directory [[LIBDIR/pkgconfig]]]),
 	[pkgconfigdir="$withval"], [pkgconfigdir='${libdir}/pkgconfig'])
 
-libiptc_LDFLAGS2="";
-AX_CHECK_LINKER_FLAGS([-Wl,--no-as-needed],
-	[libiptc_LDFLAGS2="-Wl,--no-as-needed"])
-AC_SUBST([libiptc_LDFLAGS2])
-
 AC_MSG_CHECKING([whether $LD knows -Wl,--no-undefined])
 saved_LDFLAGS="$LDFLAGS";
 LDFLAGS="-Wl,--no-undefined";
diff --git a/libiptc/Makefile.am b/libiptc/Makefile.am
index f789d34..3803fe7 100644
--- a/libiptc/Makefile.am
+++ b/libiptc/Makefile.am
@@ -5,11 +5,8 @@ AM_CPPFLAGS      = ${regular_CPPFLAGS} -I${top_builddir}/include -I${top_srcdir}
 
 pkgconfig_DATA      = libiptc.pc libip4tc.pc libip6tc.pc
 
-lib_LTLIBRARIES     = libip4tc.la libip6tc.la libiptc.la
-libiptc_la_SOURCES  =
-libiptc_la_LIBADD   = libip4tc.la libip6tc.la
-libiptc_la_LDFLAGS  = -version-info 0:0:0 ${libiptc_LDFLAGS2}
+lib_LTLIBRARIES     = libip4tc.la libip6tc.la
 libip4tc_la_SOURCES = libip4tc.c
 libip4tc_la_LDFLAGS = -version-info 1:0:1
 libip6tc_la_SOURCES = libip6tc.c
-libip6tc_la_LDFLAGS = -version-info 1:0:1 ${libiptc_LDFLAGS2}
+libip6tc_la_LDFLAGS = -version-info 1:0:1
-- 
1.7.3.4

[PATCH 2/3] libiptc: add symbol maps to libiptc

[Jan Engelhardt]

This is mostly cosmetic, as using a libxtables.so with a too old
libiptc should now say something about "version symbol LIBIP4TC_1.4.13
required" (or so) instead of just "unknown symbol: iptc_ops".

Reported-by: Pablo Neira Ayuso
Signed-off-by: Jan Engelhardt <jengelh@medozas.de>
---
 libiptc/Makefile.am  |   10 +++++++-
 libiptc/libip4tc.map |   52 +++++++++++++++++++++++++++++++++++++++++++++++++
 libiptc/libip6tc.map |   53 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 113 insertions(+), 2 deletions(-)
 create mode 100644 libiptc/libip4tc.map
 create mode 100644 libiptc/libip6tc.map

diff --git a/libiptc/Makefile.am b/libiptc/Makefile.am
index 3803fe7..4c22812 100644
--- a/libiptc/Makefile.am
+++ b/libiptc/Makefile.am
@@ -6,7 +6,13 @@ AM_CPPFLAGS      = ${regular_CPPFLAGS} -I${top_builddir}/include -I${top_srcdir}
 pkgconfig_DATA      = libiptc.pc libip4tc.pc libip6tc.pc
 
 lib_LTLIBRARIES     = libip4tc.la libip6tc.la
+
 libip4tc_la_SOURCES = libip4tc.c
-libip4tc_la_LDFLAGS = -version-info 1:0:1
+libip4tc_la_LDFLAGS = -version-info 0:0:0 -Wl,--version-script -Wl,libip4tc.map
+libip4tc_la_DEPENDENCIES = libip4tc.map
+
 libip6tc_la_SOURCES = libip6tc.c
-libip6tc_la_LDFLAGS = -version-info 1:0:1
+libip6tc_la_LDFLAGS = -version-info 0:0:0 -Wl,--version-script -Wl,libip6tc.map
+libip6tc_la_DEPENDENCIES = libip6tc.map
+
+EXTRA_DIST = libip4tc.map libip6tc.map
diff --git a/libiptc/libip4tc.map b/libiptc/libip4tc.map
new file mode 100644
index 0000000..f978b50
--- /dev/null
+++ b/libiptc/libip4tc.map
@@ -0,0 +1,52 @@
+LIBIP4TC_1.0.0 {
+global:
+	iptc_builtin;
+	iptc_init;
+	iptc_is_chain;
+	iptc_strerror;
+local:
+	*;
+};
+
+LIBIP4TC_1.4.1 {
+global:
+	dump_entries;
+};
+
+LIBIP4TC_1.4.3 {
+global:
+	iptc_append_entry;
+	iptc_check_packet;
+	iptc_commit;
+	iptc_create_chain;
+	iptc_delete_chain;
+	iptc_delete_entry;
+	iptc_delete_num_entry;
+	iptc_first_chain;
+	iptc_first_rule;
+	iptc_flush_entries;
+	iptc_free;
+	iptc_get_policy;
+	iptc_get_references;
+	iptc_get_target;
+	iptc_insert_entry;
+	iptc_next_chain;
+	iptc_next_rule;
+	iptc_read_counter;
+	iptc_rename_chain;
+	iptc_replace_entry;
+	iptc_set_counter;
+	iptc_set_policy;
+	iptc_zero_counter;
+	iptc_zero_entries;
+} LIBIP4TC_1.4.1;
+
+LIBIP4TC_1.4.11 {
+global:
+	iptc_check_entry;
+} LIBIP4TC_1.4.3;
+
+LIBIP4TC_1.4.13 {
+global:
+	iptc_ops;
+} LIBIP4TC_1.4.11;
diff --git a/libiptc/libip6tc.map b/libiptc/libip6tc.map
new file mode 100644
index 0000000..e58695f
--- /dev/null
+++ b/libiptc/libip6tc.map
@@ -0,0 +1,53 @@
+LIBIP6TC_1.0.0 {
+global:
+	ip6tc_builtin;
+	ip6tc_init;
+	ip6tc_is_chain;
+	ip6tc_strerror;
+	ipv6_prefix_length;
+local:
+	*;
+};
+
+LIBIP6TC_1.4.1 {
+global:
+	dump_entries6;
+};
+
+LIBIP6TC_1.4.3 {
+global:
+	ip6tc_append_entry;
+	ip6tc_check_packet;
+	ip6tc_commit;
+	ip6tc_create_chain;
+	ip6tc_delete_chain;
+	ip6tc_delete_entry;
+	ip6tc_delete_num_entry;
+	ip6tc_first_chain;
+	ip6tc_first_rule;
+	ip6tc_flush_entries;
+	ip6tc_free;
+	ip6tc_get_policy;
+	ip6tc_get_references;
+	ip6tc_get_target;
+	ip6tc_insert_entry;
+	ip6tc_next_chain;
+	ip6tc_next_rule;
+	ip6tc_read_counter;
+	ip6tc_rename_chain;
+	ip6tc_replace_entry;
+	ip6tc_set_counter;
+	ip6tc_set_policy;
+	ip6tc_zero_counter;
+	ip6tc_zero_entries;
+} LIBIP6TC_1.4.1;
+
+LIBIP6TC_1.4.11 {
+global:
+	ip6tc_check_entry;
+} LIBIP6TC_1.4.3;
+
+LIBIP6TC_1.4.13 {
+global:
+	ip6tc_ops;
+} LIBIP6TC_1.4.11;
-- 
1.7.3.4

[PATCH 3/3] libipq: add symbol map to libipq

[Jan Engelhardt]

Signed-off-by: Jan Engelhardt <jengelh@medozas.de>
---
 libipq/Makefile.am |    8 +++++++-
 libipq/libipq.map  |   28 ++++++++++++++++++++++++++++
 2 files changed, 35 insertions(+), 1 deletions(-)
 create mode 100644 libipq/libipq.map

diff --git a/libipq/Makefile.am b/libipq/Makefile.am
index 9e3a2ca..116ba9a 100644
--- a/libipq/Makefile.am
+++ b/libipq/Makefile.am
@@ -3,11 +3,17 @@
 AM_CFLAGS = ${regular_CFLAGS}
 AM_CPPFLAGS = ${regular_CPPFLAGS} -I${top_builddir}/include -I${top_srcdir}/include
 
-libipq_la_SOURCES = libipq.c
 lib_LTLIBRARIES   = libipq.la
+
+libipq_la_SOURCES = libipq.c
+libipq_la_LDFLAGS = -version-info 0:0:0 -Wl,--version-script -Wl,libipq.map
+libipq_la_DEPENDENCIES = libipq.map
+
 man_MANS         = ipq_create_handle.3 ipq_destroy_handle.3 ipq_errstr.3 \
                    ipq_get_msgerr.3 ipq_get_packet.3 ipq_message_type.3 \
                    ipq_perror.3 ipq_read.3 ipq_set_mode.3 ipq_set_verdict.3 \
                    libipq.3
 
 pkgconfig_DATA = libipq.pc
+
+EXTRA_DIST = libipq.map
diff --git a/libipq/libipq.map b/libipq/libipq.map
new file mode 100644
index 0000000..3ac82bd
--- /dev/null
+++ b/libipq/libipq.map
@@ -0,0 +1,28 @@
+LIBIPQ_1.0.0 {
+global:
+	ipq_destroy_handle;
+	ipq_read;
+	ipq_set_mode;
+	ipq_get_packet;
+	ipq_message_type;
+	ipq_get_msgerr;
+	ipq_ctl;
+	ipq_perror;
+local:
+	*;
+};
+
+LIBIPQ_1.2.1 {
+global:
+	ipq_errstr;
+} LIBIPQ_1.0.0;
+
+LIBIPQ_1.2.5 {
+global:
+	ipq_create_handle;
+} LIBIPQ_1.2.1;
+
+LIBIPQ_1.2.9 {
+global:
+	ipq_set_verdict;
+} LIBIPQ_1.2.5;
-- 
1.7.3.4

Re: [PATCH 2/3] libiptc: add symbol maps to libiptc

[Pablo Neira Ayuso]

Hi Jan,

On Fri, Dec 30, 2011 at 02:46:33AM +0100, Jan Engelhardt wrote:
> This is mostly cosmetic, as using a libxtables.so with a too old
> libiptc should now say something about "version symbol LIBIP4TC_1.4.13
> required" (or so) instead of just "unknown symbol: iptc_ops".
> 
> Reported-by: Pablo Neira Ayuso
> Signed-off-by: Jan Engelhardt <jengelh@medozas.de>
> ---
>  libiptc/Makefile.am  |   10 +++++++-
>  libiptc/libip4tc.map |   52 +++++++++++++++++++++++++++++++++++++++++++++++++
>  libiptc/libip6tc.map |   53 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 113 insertions(+), 2 deletions(-)
>  create mode 100644 libiptc/libip4tc.map
>  create mode 100644 libiptc/libip6tc.map
> 
> diff --git a/libiptc/libip4tc.map b/libiptc/libip4tc.map
> new file mode 100644
> index 0000000..f978b50
> --- /dev/null
> +++ b/libiptc/libip4tc.map
> @@ -0,0 +1,52 @@
> +LIBIP4TC_1.0.0 {
> +global:
> +	iptc_builtin;
> +	iptc_init;
> +	iptc_is_chain;
> +	iptc_strerror;
> +local:
> +	*;
> +};
> +
> +LIBIP4TC_1.4.1 {
> +global:
> +	dump_entries;
> +};

I think this can be done much simpler. You can with the current symbol
list by the iptables version that includes the first symbol map, e.g:

LIBIP4TC_1.4.13 {
... all exported symbols here ...
}

Then, if you add new symbols in the hypothetical 1.4.14 release, add:

LIBIP4TC_1.4.14 {
... new symbols here ...
} LIBIPT4TC_1.4.13

and so on.

No old binaries will refer to old symbols that you have defined. See
`nm' utility, e.g.:

$ nm .libs/conntrack_dump
[...]
0000000000400930 T main
                 U nfct_callback_register
                 U nfct_close
                 U nfct_open
                 U nfct_query
                 U nfct_snprintf

No symbol maps in libnetfilter_conntrack. Thus, old binaries refer to
them without requesting any specific version.

In libmnl, we have symbol maps:

$ nm .libs/genl-family-get
[...]
                 U mnl_attr_get_payload@@LIBMNL_1.0
                 U mnl_attr_get_payload_len@@LIBMNL_1.0
                 U mnl_attr_get_str@@LIBMNL_1.0
                 U mnl_attr_get_type@@LIBMNL_1.0
                 U mnl_attr_get_u16@@LIBMNL_1.0
                 U mnl_attr_get_u32@@LIBMNL_1.0
                 U mnl_attr_next@@LIBMNL_1.0
                 U mnl_attr_ok@@LIBMNL_1.0

So, binaries ask for specific version.

Old iptables binaries will not benefit for such fine definition that
you're proposing.

Moreover, I remember to have read that once you start using symbol
maps, you only bump the revision field of the c:r:a library version
numbers.

I looked into this time ago (and I'm telling you by heart), so please
correct me if I'm wrong.

Re: [PATCH 1/3] libiptc: remove libiptc.so

[Pablo Neira Ayuso]

On Fri, Dec 30, 2011 at 02:46:32AM +0100, Jan Engelhardt wrote:
> When Linux distributions started to ship binutils-ld defaulting to
> -Wl,--as-needed, we had to add a hack (-Wl,--no-as-needed) to the
> iptables build such that libiptc.so continued to link to libip4tc.so
> and libip6tc.so despite not requesting any symbols from there.
> 
> Given that distros start shipping binutils-ld defaulting to
> -Wl,--no-copy-dt-needed-entries, 3rd-party programs linking
> to -liptc will ignore libiptc.so's DT_NEEDED entries by default and
> not search for symbols there. So libiptc.so has finally become
> useless, so remove it. Dependent programs should have been using the
> pkg-config infrastructure, where quering for "iptc" returned "-lip4tc
> -lip6tc" for long enough.

My only concern here, as usual, is if people with old distros (I know
a handful of them that use it in productive environments as firewalls,
with very basic package set, that only update necessary things
like iptables and the Linux kernel) may experience problems.

So, my question is since how long do distros include this mechanism?
Which replies to how many people will experience problems on using
iptables since these changes are applied.

Re: [PATCH 2/3] libiptc: add symbol maps to libiptc

[Jan Engelhardt]

On Friday 2011-12-30 12:48, Pablo Neira Ayuso wrote:
>On Fri, Dec 30, 2011 at 02:46:33AM +0100, Jan Engelhardt wrote:
>> This is mostly cosmetic, as using a libxtables.so with a too old
>> libiptc should now say something about "version symbol LIBIP4TC_1.4.13
>> required" (or so) instead of just "unknown symbol: iptc_ops".
>
>I think this can be done much simpler. You can with the current symbol
>list by the iptables version that includes the first symbol map.
>[...]
>Old iptables binaries will not benefit for such fine definition that
>you're proposing.

I know, though it seemed like having it differentiated out is an
interesting piece of documentation of history :)

>Moreover, I remember to have read that once you start using symbol
>maps, you only bump the revision field of the c:r:a library version
>numbers.
>I looked into this time ago (and I'm telling you by heart), so please
>correct me if I'm wrong.

This is a two-part consideration.

1.
I do not see a big practical value in the R field and therefore do
not care to change it for my projects. It perhaps tells you that
something changed, but distro tools will of course clean up
.so.12.3.4 if you install a new libfoo when it ships .so.12.3.5,
and all the symlinks point to .so.12.3.5 then as well.

Consider so.12.3.5 misbehaving. Changing the .so.12/.so.12.3 symlink
has no effect, since ldconfig will usually pick the newest version,
which is still so.12.3.5. Therefore, people will do a distro-assisted
downgrade or just remove 12.3.5 from their libdir rather than trying
to fiddle export LD_PRELOAD=libfoo.so.12.3.4 into their environment.

So I find R rather useless. libc.so also does not care about it
either :)

>Moreover, I remember to have read that once you start using symbol
>maps, you only bump the revision field of the c:r:a library version
>numbers.

2.
Symbol maps are a means to have more than one {function
implementation with the same name} to coexist, thereby making it
possible to avoid having to delete an ABI revision that would
otherwise be necessary, provided that you can keep the ABI for all
existing symbols.

But one still needs to do C::A housekeeping, because deletion of old
ABIs - if you do get around to do it someday - still demands
adjusting C,A.

Re: [PATCH 2/3] libiptc: add symbol maps to libiptc

[Pablo Neira Ayuso]

On Fri, Dec 30, 2011 at 02:49:49PM +0100, Jan Engelhardt wrote:
> 
> On Friday 2011-12-30 12:48, Pablo Neira Ayuso wrote:
> >On Fri, Dec 30, 2011 at 02:46:33AM +0100, Jan Engelhardt wrote:
> >> This is mostly cosmetic, as using a libxtables.so with a too old
> >> libiptc should now say something about "version symbol LIBIP4TC_1.4.13
> >> required" (or so) instead of just "unknown symbol: iptc_ops".
> >
> >I think this can be done much simpler. You can with the current symbol
> >list by the iptables version that includes the first symbol map.
> >[...]
> >Old iptables binaries will not benefit for such fine definition that
> >you're proposing.
> 
> I know, though it seemed like having it differentiated out is an
> interesting piece of documentation of history :)

I prefer to start from scratch not to confuse anyone. Add some note
stating something like: "prior to version 1.4.x, there was no symbol
mapping".

> >Moreover, I remember to have read that once you start using symbol
> >maps, you only bump the revision field of the c:r:a library version
> >numbers.
> >I looked into this time ago (and I'm telling you by heart), so please
> >correct me if I'm wrong.
> 
> This is a two-part consideration.
>
> 1.
> I do not see a big practical value in the R field and therefore do
> not care to change it for my projects. It perhaps tells you that
> something changed, but distro tools will of course clean up
> .so.12.3.4 if you install a new libfoo when it ships .so.12.3.5,
> and all the symlinks point to .so.12.3.5 then as well.
> 
> Consider so.12.3.5 misbehaving. Changing the .so.12/.so.12.3 symlink
> has no effect, since ldconfig will usually pick the newest version,
> which is still so.12.3.5. Therefore, people will do a distro-assisted
> downgrade or just remove 12.3.5 from their libdir rather than trying
> to fiddle export LD_PRELOAD=libfoo.so.12.3.4 into their environment.
> 
> So I find R rather useless. libc.so also does not care about it
> either :)

Not very useful, but for historical reason, it can be.

> >Moreover, I remember to have read that once you start using symbol
> >maps, you only bump the revision field of the c:r:a library version
> >numbers.
> 
> 2.
> Symbol maps are a means to have more than one {function
> implementation with the same name} to coexist, thereby making it
> possible to avoid having to delete an ABI revision that would
> otherwise be necessary, provided that you can keep the ABI for all
> existing symbols.
> 
> But one still needs to do C::A housekeeping, because deletion of old
> ABIs - if you do get around to do it someday - still demands
> adjusting C,A.

Indeed, make sense.

Re: [PATCH 1/3] libiptc: remove libiptc.so

[Jan Engelhardt]

On Friday 2011-12-30 12:54, Pablo Neira Ayuso wrote:

>On Fri, Dec 30, 2011 at 02:46:32AM +0100, Jan Engelhardt wrote:
>> When Linux distributions started to ship binutils-ld defaulting to
>> -Wl,--as-needed, we had to add a hack (-Wl,--no-as-needed) to the
>> iptables build such that libiptc.so continued to link to libip4tc.so
>> and libip6tc.so despite not requesting any symbols from there.
>> 
>> Given that distros start shipping binutils-ld defaulting to
>> -Wl,--no-copy-dt-needed-entries, 3rd-party programs linking
>> to -liptc will ignore libiptc.so's DT_NEEDED entries by default and
>> not search for symbols there. So libiptc.so has finally become
>> useless, so remove it. Dependent programs should have been using the
>> pkg-config infrastructure, where quering for "iptc" returned "-lip4tc
>> -lip6tc" for long enough.
>
>My only concern here, as usual, is if people with old distros (I know
>a handful of them that use it in productive environments as firewalls,
>with very basic package set, that only update necessary things
>like iptables and the Linux kernel) may experience problems.
>
>So, my question is since how long do distros include this mechanism?

Observed it with binutils 2.19+.

>Which replies to how many people will experience problems on using
>iptables since these changes are applied.

Only third party packages using -liptc directly (instead of properly 
using pkgconfig), which are not that many. And only during 
compilation.