possible issues with blowing up struct ipt_log_info

possible issues with blowing up struct ipt_log_info

[Roberto Nibali]

Hello,

For our central logging infrastructure we prefix a LOG rule with quite some
information which is not directly available in the ipt_LOG.c module. Plus this
allows our maintenance team to improve reaction time. For that I blew up the
ipt_log_info struct as follows:

-- linux-2.4.31-orig/include/linux/netfilter_ipv4/ipt_LOG.h    2000-03-17 19:56
:20 +0100
+++ linux-2.4.31-pab2/include/linux/netfilter_ipv4/ipt_LOG.h    2005-06-29 14:52
:03 +0200
@@ -9,7 +9,7 @@
 struct ipt_log_info {
        unsigned char level;
        unsigned char logflags;
-       char prefix[30];
+       char prefix[126];
 };

 #endif /*_IPT_LOG_H*/

My question is, if anyone sees any problems with this, regarding performance
degradation on 32bit boxes or with caching problems? Does anyone know? A typical
prefix entry for example looks as follows (just in case you'd ask yourself why
we need such a big entry):

`tfx3: fw-tcp [1004] a:ACCEPT s:NEW f:PREROUTING F=NOTRACK '

Where ...

   tfx3          : is the internal firewall version (changes depending on the
                   kernel booted, support from 2.0.x to 2.6.x),
   [1004]        : is the rule number of the meta rule
   a:<aaa>       : is the action taken
   s:<sss>       : is the state
   <f,m,n>:<fmn> : is the table and the chain
   F=<FFF>       : are reserved for the flags passed by the meta fw

Best regards,
Roberto Nibali, ratz
-- 
-------------------------------------------------------------
addr://Rathausgasse 31, CH-5001 Aarau  tel://++41 62 823 9355
http://www.terreactive.com             fax://++41 62 823 9356
-------------------------------------------------------------
terreActive AG                       Wir sichern Ihren Erfolg
-------------------------------------------------------------

Re: possible issues with blowing up struct ipt_log_info

[Patrick Schaaf]

> My question is, if anyone sees any problems with this, regarding performance
> degradation on 32bit boxes or with caching problems?

The problem is compatibility with older versions of the LOG target,
both Userlevel (iptables .so module), and Kernel.

You can either make this a new target, or look how the versioning
infrastructure discussed here some months ago, worked out. (I don't
know; maybe there's already a HOWTO, or you want to write one :)

best regards
  Patrick

Re: possible issues with blowing up struct ipt_log_info

[Roberto Nibali]

> The problem is compatibility with older versions of the LOG target,
> both Userlevel (iptables .so module), and Kernel.

Fair enough.

> You can either make this a new target, or look how the versioning
> infrastructure discussed here some months ago, worked out. (I don't
> know; maybe there's already a HOWTO, or you want to write one :)

The only reference so far I found is the one about Jamal barking because it hit
his tc baby ;). Are you referring to the sublevel or nested TLVs approach from
netlink?

Cheers,
Roberto Nibali, ratz
-- 
-------------------------------------------------------------
addr://Rathausgasse 31, CH-5001 Aarau  tel://++41 62 823 9355
http://www.terreactive.com             fax://++41 62 823 9356
-------------------------------------------------------------
terreActive AG                       Wir sichern Ihren Erfolg
-------------------------------------------------------------

Re: possible issues with blowing up struct ipt_log_info

[Herve Eychenne]

On Wed, Jun 29, 2005 at 05:40:49PM +0200, Patrick Schaaf wrote:

> > My question is, if anyone sees any problems with this, regarding performance
> > degradation on 32bit boxes or with caching problems?

> The problem is compatibility with older versions of the LOG target,
> both Userlevel (iptables .so module), and Kernel.

> You can either make this a new target, or look how the versioning
> infrastructure discussed here some months ago, worked out. (I don't
> know; maybe there's already a HOWTO, or you want to write one :)

I remember saying here that it would be cool to have the log prefix length
tunable at kernel LOG module insertion, have it exported (read-only) to some
/proc entry, and have userspace (iptables, or any other upper-level
application) rely on the real-time extracted value.
What do you think?

 Herve

-- 
 _
(°=  Hervé Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

Re: possible issues with blowing up struct ipt_log_info

[Roberto Nibali]

> I remember saying here that it would be cool to have the log prefix length
> tunable at kernel LOG module insertion, have it exported (read-only) to some
> /proc entry, and have userspace (iptables, or any other upper-level
> application) rely on the real-time extracted value.
> What do you think?

Could work, although I'm not sure why you'd want to change the prefix 
lenght that often, however flexibility is nice.

Another issue I see is that iptables, when sending the data from user to 
kernel space, must be made aware of setting the correct target{info}size 
in case the ipt_*LOG module is not loaded yet. Parsing proc-fs would be 
useless. This opens a possible "DoS", which can be addressed by having a 
max log prefix length accepted as being passed to the lkm.

Was there any consensus regarding this feature among the core netfilter 
developers or has anyone posted a POC patch? Granted, there are a 
billion more important things to address, but still ... ;)

Regards,
Roberto Nibali, ratz
-- 
echo 
'[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq' | dc

Re: possible issues with blowing up struct ipt_log_info

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 1483 bytes --]

On Wed, Jun 29, 2005 at 06:09:23PM +0200, Herve Eychenne wrote:
> On Wed, Jun 29, 2005 at 05:40:49PM +0200, Patrick Schaaf wrote:
> 
> > > My question is, if anyone sees any problems with this, regarding performance
> > > degradation on 32bit boxes or with caching problems?
> 
> > The problem is compatibility with older versions of the LOG target,
> > both Userlevel (iptables .so module), and Kernel.
> 
> > You can either make this a new target, or look how the versioning
> > infrastructure discussed here some months ago, worked out. (I don't
> > know; maybe there's already a HOWTO, or you want to write one :)
> 
> I remember saying here that it would be cool to have the log prefix length
> tunable at kernel LOG module insertion, have it exported (read-only) to some
> /proc entry, and have userspace (iptables, or any other upper-level
> application) rely on the real-time extracted value.

I really don't think that this is worth the complexity... I also don't
really like using out-of-band interfaces such as /proc to determine how
to communicate rules via {get,set}sockopt()

-- 
- Harald Welte <laforge@netfilter.org>                 http://netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: possible issues with blowing up struct ipt_log_info

[Herve Eychenne]

On Sun, Jul 03, 2005 at 02:36:50PM +0200, Harald Welte wrote:

> On Wed, Jun 29, 2005 at 06:09:23PM +0200, Herve Eychenne wrote:
> > On Wed, Jun 29, 2005 at 05:40:49PM +0200, Patrick Schaaf wrote:
> > 
> > > > My question is, if anyone sees any problems with this, regarding performance
> > > > degradation on 32bit boxes or with caching problems?
> > 
> > > The problem is compatibility with older versions of the LOG target,
> > > both Userlevel (iptables .so module), and Kernel.
> > 
> > > You can either make this a new target, or look how the versioning
> > > infrastructure discussed here some months ago, worked out. (I don't
> > > know; maybe there's already a HOWTO, or you want to write one :)
> > 
> > I remember saying here that it would be cool to have the log prefix length
> > tunable at kernel LOG module insertion, have it exported (read-only) to some
> > /proc entry, and have userspace (iptables, or any other upper-level
> > application) rely on the real-time extracted value.

> I really don't think that this is worth the complexity...

Trying to insert potentially many LOG rules to empirically determine
the maximum prefix length is far less complex, that's for sure. ;-)
For now, that's the only way an upper-level tool will produce valid
LOG rules in all cases.
Probably many of you won't care, but for this kind of upper-layers
(which I'm particularly concerned with) netfilter/iptables sometimes
makes it hard to turn "should work" into "will work".
That LOG thing is only one example, though probably not the best.

> I also don't really like using out-of-band interfaces such as /proc
> to determine how to communicate rules via {get,set}sockopt()

Maybe you don't like /proc, but the kernel still has to provide a way
to inform userspace about the fixed size of some of its data (as
kernelspace doesn't like dynamic things very much...). If you see a
better way, that's ok, but the need is still there.

 Herve

-- 
 _
(°=  Hervé Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

Re: possible issues with blowing up struct ipt_log_info

[Patrick Schaaf]

> Trying to insert potentially many LOG rules to empirically determine
> the maximum prefix length is far less complex, that's for sure. ;-)

Do we already have indeterminate log prefix length? I thought we had
one, fixed size, no probing neccessary. Am I wrong?

> For now, that's the only way an upper-level tool will produce valid
> LOG rules in all cases.

The way to produce valid log rules in all cases, is to keep this
fancy long long stuff in user level, and set pointer / index values
or whatever small token can be used, for a log prefix in the kernel.

Any scheme which probes the possible length, will also need some
fallback when kernel length is smaller than user desire. If your
goal is 'valid in all cases'. Right? Then you'll need such a
token/pointer approach anyway!

> Probably many of you won't care, but for this kind of upper-layers
> (which I'm particularly concerned with) netfilter/iptables sometimes
> makes it hard to turn "should work" into "will work".
> That LOG thing is only one example, though probably not the best.

Then maybe you could provide a better one?

> > I also don't really like using out-of-band interfaces such as /proc
> > to determine how to communicate rules via {get,set}sockopt()
> 
> Maybe you don't like /proc, but the kernel still has to provide a way
> to inform userspace about the fixed size of some of its data (as
> kernelspace doesn't like dynamic things very much...). If you see a
> better way, that's ok, but the need is still there.

If getsockopt is good to communicate iptables rules and other stuff,
it must be good enough to also communicate 'length of X is Y'...

best regards
  Patrick

Re: possible issues with blowing up struct ipt_log_info

[Roberto Nibali]

>>Trying to insert potentially many LOG rules to empirically determine
>>the maximum prefix length is far less complex, that's for sure. ;-)
> 
> Do we already have indeterminate log prefix length? I thought we had
> one, fixed size, no probing neccessary. Am I wrong?

No, it's fix. And "probing" can easily be done with a large sized prefix,
although I'm a bit astonished as to why someone wouldn't know the prefix size
when loading the packet filter ruleset.

>>For now, that's the only way an upper-level tool will produce valid
>>LOG rules in all cases.
>  
> The way to produce valid log rules in all cases, is to keep this
> fancy long long stuff in user level, and set pointer / index values
> or whatever small token can be used, for a log prefix in the kernel.

This does not work in environments where you need high speed logging since
having a pointer, or hashmap value, would require an intermediate process to
translate it into parsable strings again. This is from an architectural point of
view and from a performance point of view not possible, at least not in our
environment.

> Any scheme which probes the possible length, will also need some
> fallback when kernel length is smaller than user desire. If your
> goal is 'valid in all cases'. Right? Then you'll need such a
> token/pointer approach anyway!

The fallback is as it always was, exit with return value 2. Nothing changes. The
token/pointer approach is not something the netfilter architecture can provide
for you. That's something each packet filter software would have to provide by
itself. Example:

# A filter rule could expand into a ruleset where on of the rules could be
iptables -t filter -A INPUT -j LOG -i eth0 --log-level info --log-prefix
'ptr_666' -p icmp --icmp-type 8 -s 172.23.120.30/32 -d 172.23.120.120 -m state
--state NEW

# The ptr_666 is a pointer which can be translated again into human readable
# or log parsable strings.
ptr_666="pf rule set, ABI/55: icmp_from_fw [666] a:ACCEPT s:NEW f:INPUT TRACK
sub referenced as group"

>>Maybe you don't like /proc, but the kernel still has to provide a way
>>to inform userspace about the fixed size of some of its data (as
>>kernelspace doesn't like dynamic things very much...). If you see a
>>better way, that's ok, but the need is still there.
>  
> If getsockopt is good to communicate iptables rules and other stuff,
> it must be good enough to also communicate 'length of X is Y'...

Well, good enough is a relative term :), but the netfilter core team is working
extra hours on the netlink integration which could help a lot in this
perspective. It would be a lot more fun to get the connection tracking table
filled with thousands of entries in less than 2 Minutes, so the monitoring
system does not have so many spikes. Customers often ask if the system is
malfunctioning or down every 5-15 Minutes :).

Regards,
Roberto Nibali, ratz
-- 
-------------------------------------------------------------
addr://Rathausgasse 31, CH-5001 Aarau  tel://++41 62 823 9355
http://www.terreactive.com             fax://++41 62 823 9356
-------------------------------------------------------------
terreActive AG                       Wir sichern Ihren Erfolg
-------------------------------------------------------------

Re: possible issues with blowing up struct ipt_log_info

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 2395 bytes --]

On Mon, Jul 04, 2005 at 10:20:33AM +0200, Roberto Nibali wrote:
> > Do we already have indeterminate log prefix length? I thought we had
> > one, fixed size, no probing neccessary. Am I wrong?
> 
> No, it's fix. And "probing" can easily be done with a large sized prefix,
> although I'm a bit astonished as to why someone wouldn't know the prefix size
> when loading the packet filter ruleset.

I also can't remember that we ever changed the length of that field.

> >>For now, that's the only way an upper-level tool will produce valid
> >>LOG rules in all cases.
> >  
> > The way to produce valid log rules in all cases, is to keep this
> > fancy long long stuff in user level, and set pointer / index values
> > or whatever small token can be used, for a log prefix in the kernel.
> 
> This does not work in environments where you need high speed logging since
> having a pointer, or hashmap value, would require an intermediate process to
> translate it into parsable strings again. This is from an
> architectural point of view and from a performance point of view not
> possible, at least not in our environment.

If you care about performance, tou don't do hgh speed logging via
syslog.  I have some numbers comparing LOG with ULOG in multiple setup,
and it's in some cases a factor of 100 ...

OTOTH, ULOG also has a prefix, so the argument remains.  However, I'm
happy if somebody wanted to contribute an ulogd plugin to do that kind
of translation.

And anyways, you can defer such processing until the logs are actually
being analyzed.  It doesn't have to happen at the time where you have
your load spike and just want to get the data to disk.

> It would be a lot more fun to get the connection tracking table filled
> with thousands of entries in less than 2 Minutes, so the monitoring
> system does not have so many spikes.  Customers often ask if the
> system is malfunctioning or down every 5-15 Minutes :).

Sorry, I don't understand what you want to convey....

-- 
- Harald Welte <laforge@netfilter.org>                 http://netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: possible issues with blowing up struct ipt_log_info

[Roberto Nibali]

>>This does not work in environments where you need high speed logging since
>>having a pointer, or hashmap value, would require an intermediate process to
>>translate it into parsable strings again. This is from an
>>architectural point of view and from a performance point of view not
>>possible, at least not in our environment.
> 
> If you care about performance, tou don't do hgh speed logging via
> syslog.  I have some numbers comparing LOG with ULOG in multiple setup,
> and it's in some cases a factor of 100 ...

We are also using ULOG but not everywhere, because not all the meta rules have
been converted to using it yet.

> OTOTH, ULOG also has a prefix, so the argument remains.  However, I'm
> happy if somebody wanted to contribute an ulogd plugin to do that kind
> of translation.

Interesting idea, haven't thought that far (as always on mondays).

> And anyways, you can defer such processing until the logs are actually
> being analyzed.  It doesn't have to happen at the time where you have
> your load spike and just want to get the data to disk.

True to a certain extent.

The problem is that a lot of different software systems have to be integrated
and we cannot afford or don't have access to all code to change all backend
logic to do this. And then imagine having a central logging system doing the log
evaluation for thousands of systems, among those, the packet filters running
netfilter. Now let's say we have 10 packet filter in location A, 15 in location
B. Location A has 3 sysadmins who work together and maintain a single mapping
file for all log prefixes, Location B has 5 sysadmins but 2 groups of them which
work completely orthogonal to each other when it comes to defining log prefixes.
The central logging infrastructure is maintained by a third party company. Due
to many facets of security and the complexity involved I'd say that it's close
to impossible to defer the analysis. I hope I don't have to give you a more
detailed example, because I'd be writing a whole paper to explain this situation
(which is real, we're deploying such information systems and central logging
infrastructures).

>>It would be a lot more fun to get the connection tracking table filled
>>with thousands of entries in less than 2 Minutes, so the monitoring
>>system does not have so many spikes.  Customers often ask if the
>>system is malfunctioning or down every 5-15 Minutes :).
> 
> Sorry, I don't understand what you want to convey....

My bad, it was a rather unfortunate and even wrong description. What I wanted to
do is to quote an internal bug report regarding the cat'ing of
/proc/net/ip_conntrack while having a heavily populated with entries:

"Our team reported periodic (minutely) problems with connections through the
firewall. During 2 to 3 seconds e.g. pings round trip times through the firewall
went up from several ms to more than 1 second. During the time, also working on
the packet filter was shortly interrupted. The error count on the respective
interfaces (where the ping went through) increased by about 300 to 500,
sometimes even more. If the ping was not running, the timeout were still there,
but error number did not increase on the interface."

This observation is a result of snmp polling each minute the connection tracking
table (as dumb as this may sound, but it was a requirement of a bank) and in
2.4.x the only way to do it is to basically cat the proc-fs entry which is ...
slow and error prone.

That's all, nothing to worry about for the netfilter people. We have a lot of
workarounds :).

Best regards,
Roberto Nibali, ratz
-- 
-------------------------------------------------------------
addr://Rathausgasse 31, CH-5001 Aarau  tel://++41 62 823 9355
http://www.terreactive.com             fax://++41 62 823 9356
-------------------------------------------------------------
terreActive AG                       Wir sichern Ihren Erfolg
-------------------------------------------------------------

Re: possible issues with blowing up struct ipt_log_info

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 1675 bytes --]

On Mon, Jul 04, 2005 at 11:26:50AM +0200, Roberto Nibali wrote:
> > And anyways, you can defer such processing until the logs are actually
> > being analyzed.  It doesn't have to happen at the time where you have
> > your load spike and just want to get the data to disk.
> 
> True to a certain extent.
> 
> The problem is that a lot of different software systems have to be integrated
> and we cannot afford or don't have access to all code to change all backend
> logic to do this. 

Well, to be honest I'm happy that I can say - to a certain extend: I
don't care abou your particular setup and it's political/organizational
erstrictions ;)  [don't get me wrong, please].

As always, I think we should provide some infrastructure and let people
use it how they want.  We can try to be flexible, but we cannot optimize
ever possible setup.

Your particular setup would need something like a proxy that does the
interpretation before it ends up at the centralized logging server.
Alternatively you could store the hostname of the firewall and make the
interpretation of such a prefix local to that host.  Now you certainly
can come up with a reason why you can't do that.  And I can frankly
repeat: well, that's your problem ;)  Technically it is possible...

Cheers,
	Harald
-- 
- Harald Welte <laforge@netfilter.org>                 http://netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: possible issues with blowing up struct ipt_log_info

[Roberto Nibali]

> Well, to be honest I'm happy that I can say - to a certain extend: I
> don't care abou your particular setup and it's political/organizational
> erstrictions ;)  [don't get me wrong, please].

No, I understand and agree with you. You do the technical correctness of what
you implement and we deal with the correct architectural issues.

> As always, I think we should provide some infrastructure and let people
> use it how they want.  We can try to be flexible, but we cannot optimize
> ever possible setup.

Never asked for that ;).

> Your particular setup would need something like a proxy that does the
> interpretation before it ends up at the centralized logging server.

We also have that (a proxy), but still, it's not that easy ... _but_ also
completely off-topic for this list.

> Alternatively you could store the hostname of the firewall and make the
> interpretation of such a prefix local to that host.  Now you certainly
> can come up with a reason why you can't do that.  And I can frankly
> repeat: well, that's your problem ;)  Technically it is possible...

Yes, let's kill this part of the thread.

Cheers,
Roberto Nibali, ratz
-- 
-------------------------------------------------------------
addr://Rathausgasse 31, CH-5001 Aarau  tel://++41 62 823 9355
http://www.terreactive.com             fax://++41 62 823 9356
-------------------------------------------------------------
terreActive AG                       Wir sichern Ihren Erfolg
-------------------------------------------------------------

Re: possible issues with blowing up struct ipt_log_info

[Herve Eychenne]

On Mon, Jul 04, 2005 at 10:20:33AM +0200, Roberto Nibali wrote:

> >>Trying to insert potentially many LOG rules to empirically determine
> >>the maximum prefix length is far less complex, that's for sure. ;-)
> > 
> > Do we already have indeterminate log prefix length? I thought we had
> > one, fixed size, no probing neccessary. Am I wrong?

> No, it's fix. And "probing" can easily be done with a large sized prefix,

As I said, if you want to know the exact value, you have to use an
algorithm that tries to insert several LOG rules.
If you want to provide here the best algorithm that does the least
possible probes in the average case, you're welcome. ;-)

While this is an amusing exercise, I would prefer knowing the value
directly. :-)

> although I'm a bit astonished as to why someone wouldn't know the prefix size
> when loading the packet filter ruleset.

This is a perfect kernel guy assertion. ;-) Can you figure out that 90% of
Linux users in the world are meant to set up a firewall without even
knowing what a kernel is? ;-)
More seriously, I am reguarly asked to install a netfilter-based firewall
on machines I didn't install myself. And most people are not even
aware there's a limit for LOG prefix length until they discover
the "too long (must be under xx chars)" message, believe me.

 Herve

-- 
 _
(°=  Hervé Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

Re: possible issues with blowing up struct ipt_log_info

[Roberto Nibali]

>>No, it's fix. And "probing" can easily be done with a large sized prefix,
>  
> As I said, if you want to know the exact value, you have to use an
> algorithm that tries to insert several LOG rules.
> If you want to provide here the best algorithm that does the least
> possible probes in the average case, you're welcome. ;-)

It's a matter either of the Pareto principle or if you prefer best practises.

> While this is an amusing exercise, I would prefer knowing the value
> directly. :-)

Fiddle around with the preprocessor in ipt_LOG.c and have yourself the value
printed out using a new MODULE_PARM_DESC entry :)

>>although I'm a bit astonished as to why someone wouldn't know the prefix size
>>when loading the packet filter ruleset.
>  
> This is a perfect kernel guy assertion. ;-)

No, a rather practical approach, as a matter of fact. See below.

> Can you figure out that 90% of
> Linux users in the world are meant to set up a firewall without even
> knowing what a kernel is? ;-)

Where do you have these numbers from? But this is besides the point. If so,
those users will certainly not use iptables by hand, but a preconfigued script
or even one of the nice GUIs for setting up the rules. The backend can handle
such failures easily, no need to know the size :). And how many of those 90% do
not use standard Linux distributions? Because I bet you 10 bucks that none of
the well-known Linux Distributions is changing the ipt_log_info struct compared
to plain vanilla sources.

> More seriously, I am reguarly asked to install a netfilter-based firewall
> on machines I didn't install myself. And most people are not even
> aware there's a limit for LOG prefix length until they discover
> the "too long (must be under xx chars)" message, believe me.

I believe you.

Best regards,
Roberto Nibali, ratz
-- 
-------------------------------------------------------------
addr://Rathausgasse 31, CH-5001 Aarau  tel://++41 62 823 9355
http://www.terreactive.com             fax://++41 62 823 9356
-------------------------------------------------------------
terreActive AG                       Wir sichern Ihren Erfolg
-------------------------------------------------------------

Re: possible issues with blowing up struct ipt_log_info

[Herve Eychenne]

On Mon, Jul 04, 2005 at 12:48:32PM +0200, Roberto Nibali wrote:

> Fiddle around with the preprocessor in ipt_LOG.c and have yourself the value
> printed out using a new MODULE_PARM_DESC entry :)

That was what we were talking about earlier in this thread. Ok, but this
would then require some support/knowledge from userspace. I proposed /proc,
but Harald didn't seem very prone to use it. My question about the
reasons why is still unanswered.

> > Can you figure out that 90% of
> > Linux users in the world are meant to set up a firewall without even
> > knowing what a kernel is? ;-)

> Where do you have these numbers from? But this is besides the point. If so,
> those users will certainly not use iptables by hand, but a preconfigued script
> or even one of the nice GUIs for setting up the rules. The backend can handle
> such failures easily, no need to know the size :). And how many of those 90% do
> not use standard Linux distributions? Because I bet you 10 bucks that none of
> the well-known Linux Distributions is changing the ipt_log_info struct compared
> to plain vanilla sources.

Sorry, I thought the sententious tune and the smiley would have made this
affirmation appear as an impish kind-of-joke, or anyway one who didn't deserve
any rationnal answer (with which I can only agree though).

> > More seriously, I am reguarly asked to install a netfilter-based firewall
> > on machines I didn't install myself.

> > And most people are not even
> > aware there's a limit for LOG prefix length until they discover
> > the "too long (must be under xx chars)" message, believe me.

And it's the case for almost every static size...

 Herve

-- 
 _
(°=  Hervé Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/

Re: possible issues with blowing up struct ipt_log_info

[Herve Eychenne]

On Mon, Jul 04, 2005 at 07:55:42AM +0200, Patrick Schaaf wrote:

> > Trying to insert potentially many LOG rules to empirically determine
> > the maximum prefix length is far less complex, that's for sure. ;-)

> Do we already have indeterminate log prefix length? I thought we had
> one, fixed size, no probing neccessary. Am I wrong?

The max LOG prefix length is determined by the static length of the
kernel structure. So this size has to be changed in the kernel source
before compilation. That's rare, obviously.
However, I already saw a few setups where this value had been changed
by hand.
So we have two options:
- consider that those who do that should deal themselves with the
  consequences of their action
- take as much cases as possible into account

I guess many kernel guys here will vote for the first one.
I'm more user-oriented ("should work in any case, if possible").
So that LOG length thing is not so bad here, but the philosophy behind
it bothers me.

Parallel:
- static sizes are good
- static sizes are to be avoided every time this is possible,
  especially for "human" entered/oriented data
This leads to some Unix hall-of-shame relics like static command line
length and co.

> > For now, that's the only way an upper-level tool will produce valid
> > LOG rules in all cases.

> The way to produce valid log rules in all cases, is to keep this
> fancy long long stuff in user level, and set pointer / index values
> or whatever small token can be used, for a log prefix in the kernel.

> Any scheme which probes the possible length, will also need some
> fallback when kernel length is smaller than user desire. If your
> goal is 'valid in all cases'. Right? Then you'll need such a
> token/pointer approach anyway!

You're right, but the token has a drawback: it's not readable by a
human being.  Ok, you'll say that the token has been generated by a
higher-level application, so why not use another higher-level
application to read the logs?  I'll answer: yes, but then writing logs
to syslog (human oriented, right?) becomes all the more irrelevant, and
sometimes it's good not to require a special tool for simple tasks.
For the first argument you'll point me to ULOG, and the second is a
matter of taste (KISS or not KISS, that's the question).
End of the discussion? ;-)

The very same thing applies to chainname length.  Imagine you split
your ruleset into several "zone1-zone2 traffic" (zone1 to zone2)
chains, which is wise for complex setups.
Imagine you let the user choose zone names (fair enough).
This limits the name of a zone to 14 chars (2*14<30).
So it becomes almost practically impossible to permit zone names like
word1_word2.  You'll say: so don't do this.  I'll say: why
couldn't I if I want to?
But this won't lead us anywhere, I fear.

Maybe you'll even point me to the same token trick.  But the ruleset
(dumped by iptables-save, for example) will become really obfuscated, and
all the more hard to understand for a human being (bad, IMHO) and to
debug.

> > Probably many of you won't care, but for this kind of upper-layers
> > (which I'm particularly concerned with) netfilter/iptables sometimes
> > makes it hard to turn "should work" into "will work".
> > That LOG thing is only one example, though probably not the best.

> Then maybe you could provide a better one?

Are you ready to endure a really long discussion? ;-)
There are plenty of them. That would require a whole separated thread
and quite a lot of time (I somewhat lack it today).
Probably I'll start several threads later about this.
Be prepared... ;-)

> > > I also don't really like using out-of-band interfaces such as /proc
> > > to determine how to communicate rules via {get,set}sockopt()

> > Maybe you don't like /proc, but the kernel still has to provide a way
> > to inform userspace about the fixed size of some of its data (as
> > kernelspace doesn't like dynamic things very much...). If you see a
> > better way, that's ok, but the need is still there.

> If getsockopt is good to communicate iptables rules and other stuff,
> it must be good enough to also communicate 'length of X is Y'...

So some values would be obtained via /proc, some via [sg]etsockopt.
This lacks coherence... :-(
I don't care that much about the way it can be obtained, except that
with [sg]etsockopt, the user would need yet another tool to get the
value...
What's so wrong with /proc, BTW?

 Herve

-- 
 _
(°=  Hervé Eychenne
//)
v_/_ WallFire project:  http://www.wallfire.org/