Re: Problem with new --physdev-out style

Re: Problem with new --physdev-out style

[Patrick McHardy]

Volker Sauer wrote:
> with recent kernels, I have this problem:

Bugreports to netfilter-devel please.

> kernel: physdev match: using --physdev-out in the OUTPUT, FORWARD and
> POSTROUTING chains for non-bridged traffic is not supported anymore.
> 
> What does "non-bridged" in this context mean?? If it means rules (or
> traffic) that goes over the INPUT our OUTPUT chain, I do not understand,
> why my rule set causes this message to appers a thousand times.
> 
> Here's all my rules with --physdev-out:
> 
> arthur: ~ # grep physdev-out /etc/init.d/firewall
> $IPTABLES -A FORWARD -o $BR_INT -m physdev --physdev-out $IF_INT -i $IF_EXT -d $localnet -s $Any -m state --state ESTABLISHED,RELATED -j ACCEPT
> $IPTABLES -A FORWARD -i $BR_INT -m physdev --physdev-in $IF_INT --physdev-out $IF_DMZ -s $ZAPHOD -j ACCEPT
> $IPTABLES -A FORWARD -i $BR_INT -m physdev --physdev-in $IF_INT --physdev-out $IF_DMZ -s $localnet -p tcp -d $MARVIN --dport 3389 -j ACCEPT
> $IPTABLES -A FORWARD -i $BR_INT -m physdev --physdev-in $IF_DMZ --physdev-out $IF_INT -d $ZAPHOD -p tcp --dport 135:139 -j ACCEPT
> $IPTABLES -A FORWARD -i $BR_INT -m physdev --physdev-in $IF_DMZ --physdev-out $IF_INT -d $ZAPHOD -p udp --dport 135:139 -j ACCEPT
> $IPTABLES -A FORWARD -i $BR_INT -m physdev --physdev-in $IF_DMZ --physdev-out $IF_INT -d $ZAPHOD -p tcp --dport 445 -j ACCEPT
> $IPTABLES -A FORWARD -i $BR_INT -m physdev --physdev-in $IF_DMZ --physdev-out $IF_INT -d $ZAPHOD -p udp --dport 445 -j ACCEPT
> $IPTABLES -A FORWARD -i $BR_INT -m physdev --physdev-in $IF_INT --physdev-out $IF_DMZ -s $ZAPHOD -m state --state ESTABLISHED,RELATED -j ACCEPT
> $IPTABLES -A FORWARD -o $BR_INT -m physdev --physdev-out $IF_INT -i $BR_GUEST -m state --state ESTABLISHED,RELATED -j ACCEPT
> $IPTABLES -A FORWARD -i $BR_GUEST -o $BR_INT -m physdev --physdev-out $IF_DMZ -p tcp --dport 3389 -j ACCEPT
> $IPTABLES -A FORWARD -i $BR_GUEST -o $BR_INT -m physdev --physdev-out $IF_INT -p tcp --dport ssh -j ACCEPT
> $IPTABLES -A FORWARD -i $BR_GUEST -o $BR_INT -m physdev --physdev-out $IF_INT -p tcp --dport 30022 -j ACCEPT
> $IPTABLES -A FORWARD -i $IF_EXT -o $BR_INT -m physdev --physdev-out $IF_DMZ -p tcp --sport 22 -m state --state ESTABLISHED,RELATED -j ACCEPT
> $IPTABLES -A FORWARD -i $IF_EXT -o $BR_INT -m physdev --physdev-out $IF_DMZ -s $i -p udp --sport 53 -m state --state ESTABLISHED,RELATED -j ACCEPT
> 
> 
> Where: 
> 
> BR_INT="br-intern"
> BR_GUEST="br-guest"
> IF_EXT="eth0"
> IF_INT="eth1"
> IF_DMZ="vlan3"
> 
> You see, I use --physdev-out only in the FORWARD with bridged traffic, 
> because the Interfaces given with -i or -i in these rules are always 
> bridges (br-intern or br-guest).
> 
> Why do I get thousands of these error messages?

Try adding "--physdev-is-bridged" to your rules. Without that the kernel
is not able to tell whether they apply only to bridged packets or also
to forwarded or locally generated ones.

Suggestions how to improve this are welcome.

Re: Problem with new --physdev-out style

[Philip Craig]

Patrick McHardy wrote:
>> $IPTABLES -A FORWARD -i $BR_GUEST -o $BR_INT -m physdev --physdev-out $IF_DMZ -p tcp --dport 3389 -j ACCEPT
> 
> Try adding "--physdev-is-bridged" to your rules. Without that the kernel
> is not able to tell whether they apply only to bridged packets or also
> to forwarded or locally generated ones.

That won't work for the above rule, for example, since the packet is
being forwarded between two different bridges, so it is not bridged.

The only solution I am aware of is to stop bridging and use routing
and arp proxy.

Re: Problem with new --physdev-out style

[Patrick McHardy]

Philip Craig wrote:
> Patrick McHardy wrote:
>>> $IPTABLES -A FORWARD -i $BR_GUEST -o $BR_INT -m physdev --physdev-out $IF_DMZ -p tcp --dport 3389 -j ACCEPT
>> Try adding "--physdev-is-bridged" to your rules. Without that the kernel
>> is not able to tell whether they apply only to bridged packets or also
>> to forwarded or locally generated ones.
> 
> That won't work for the above rule, for example, since the packet is
> being forwarded between two different bridges, so it is not bridged.


I see nothing indicating that it is being forwarded. bridge-netfilter
passes packets though the iptables hooks by default.

Re: Problem with new --physdev-out style

[Pascal Hambourg]

Patrick McHardy a écrit :
> Philip Craig wrote:
> 
>> Patrick McHardy wrote:
>>
>>>> $IPTABLES -A FORWARD -i $BR_GUEST -o $BR_INT -m physdev 
>>>> --physdev-out $IF_DMZ -p tcp --dport 3389 -j ACCEPT
>>>
>>> Try adding "--physdev-is-bridged" to your rules. Without that the kernel
>>> is not able to tell whether they apply only to bridged packets or also
>>> to forwarded or locally generated ones.
>>
>> That won't work for the above rule, for example, since the packet is
>> being forwarded between two different bridges, so it is not bridged.

Do you mean that the "bridged" flag (or whatever way this is 
implemented) is not reset when the packet is routed ? Doesn't this sound 
like a bug ?

> I see nothing indicating that it is being forwarded

"-A FORWARD -i $BR_GUEST -o $BR_INT"

Re: Problem with new --physdev-out style

[Philip Craig]

Pascal Hambourg wrote:
> Do you mean that the "bridged" flag (or whatever way this is 
> implemented) is not reset when the packet is routed ? Doesn't this sound 
> like a bug ?

It's not set at all for routed packets, which is correct behaviour.

Re: Problem with new --physdev-out style

[Pascal Hambourg]

Philip Craig a écrit :
> Pascal Hambourg wrote:
> 
>>Do you mean that the "bridged" flag (or whatever way this is 
>>implemented) is not reset when the packet is routed ? Doesn't this sound 
>>like a bug ?
> 
> It's not set at all for routed packets, which is correct behaviour.

Sure, but what about packets that enter a bridged interface and then are 
routed ?
-
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Problem with new --physdev-out style

[Philip Craig]

Pascal Hambourg wrote:
> Sure, but what about packets that enter a bridged interface and then are 
> routed ?

It's not set for them either, they are still just routed packets.
Not sure I understand your question.  My original comment was that
because the packets are routed (even though they arrived on a bridge),
the bridged flag is not set, and so the --physdev-is-bridged option
would never match for that particular rule, and it simply isn't
possible to fix that rule to get the --physdev-out to work.

Re: Problem with new --physdev-out style

[Pascal Hambourg]

Philip Craig a écrit :
> Pascal Hambourg wrote:
> 
>>Sure, but what about packets that enter a bridged interface and then are 
>>routed ?
> 
> It's not set for them either, they are still just routed packets.
> Not sure I understand your question.  My original comment was that
> because the packets are routed (even though they arrived on a bridge),
> the bridged flag is not set, and so the --physdev-is-bridged option
> would never match for that particular rule, and it simply isn't
> possible to fix that rule to get the --physdev-out to work.

I misunderstood your original comment. I thought you meant that 
--physdev-is-bridged would not help distinguish between routed and 
bridged packets when the input and output interfaces are bridges. Thanks 
for taking the time to clarify.

Re: Problem with new --physdev-out style

[Philip Craig]

Patrick McHardy wrote:
> Philip Craig wrote:
>> Patrick McHardy wrote:
>>>> $IPTABLES -A FORWARD -i $BR_GUEST -o $BR_INT -m physdev --physdev-out $IF_DMZ -p tcp --dport 3389 -j ACCEPT
>>> Try adding "--physdev-is-bridged" to your rules. Without that the kernel
>>> is not able to tell whether they apply only to bridged packets or also
>>> to forwarded or locally generated ones.
>> That won't work for the above rule, for example, since the packet is
>> being forwarded between two different bridges, so it is not bridged.
> 
> 
> I see nothing indicating that it is being forwarded.

They are different bridges, BR_GUEST and BR_INT, doesn't that mean it must
be forwarded?

> bridge-netfilter
> passes packets though the iptables hooks by default.

Yes, but if the destination is a different bridge then isn't it passed up
to the IP layer without going through NF_BR_FORWARD, and so BRNF_BRIDGED
is never set?  And more importantly, nf_bridge->physoutdev is never set
until the output bridge processes it.

Re: Problem with new --physdev-out style

[Patrick McHardy]

Philip Craig wrote:
> Patrick McHardy wrote:
>> Philip Craig wrote:
>>> Patrick McHardy wrote:
>>>>> $IPTABLES -A FORWARD -i $BR_GUEST -o $BR_INT -m physdev --physdev-out $IF_DMZ -p tcp --dport 3389 -j ACCEPT
>>>> Try adding "--physdev-is-bridged" to your rules. Without that the kernel
>>>> is not able to tell whether they apply only to bridged packets or also
>>>> to forwarded or locally generated ones.
>>> That won't work for the above rule, for example, since the packet is
>>> being forwarded between two different bridges, so it is not bridged.
>>
>> I see nothing indicating that it is being forwarded.
> 
> They are different bridges, BR_GUEST and BR_INT, doesn't that mean it must
> be forwarded?


You're right.

Re: Problem with new --physdev-out style

[Volker Sauer]

[-- Attachment #1: Type: text/plain, Size: 2985 bytes --]

On Mi, 24 Okt 2007, Patrick McHardy <kaber@trash.net> wrote:

> You're right.

Yes, he's right but only for 5 of my rules which indeed bridge between
bridges (-i $BR_GUEST -o $BR_INT).

Let me summarize the discussion so far:

The warning message means, that --physdev-out can not be used, if the
packet is actually forwarded (instead of bridged) between two interfaces
even if both interfaces are bridges. In this case you either need
proxy-arp or you need to filter by other things than physdev or yo need
some magic with marking the packets.
Okay, so far so good. I can live with that, since bridging between two
bridges is only an exemption. Usually I have only one bridge inside the
firewall with physin and physout rules. This is the case for all my
firewalls except the one I took the example from and this can easily be
fixed by removing --physdev-out and using -s or -d or something like
this. It's just the firewall of my testing site.

Coming to the real point:

99% of my rules on all my firewalls are like that:

$IPTABLES -A FORWARD -i $BR_INT -m physdev --physdev-in $IF_INT
--physdev-out $IF_DMZ -s $ZAPHOD -j ACCEPT

IF_INT (eth1) and IF_DMZ (vlan3) are both members of BR_INT (br-intern):

fw1: ~ # brctl show
br-intern               8000.000d88cd28c1       yes     eth1
                                                        vlan3

This means, that all rules like that are valid even with the new concept
of netfilter, right?? But why do I get error messages like quoted in my
first mail for these rules - it *is* bridged traffic inside *one*
bridge!
And: I don't see how --physdev-is-bridged should help, since it's a
match and not a command to the kernel saying: "this *is* bridged
traffic". It the kernel does not see this by itself,
--physdev-is-bridged doesn't help.


From all your answers, I still do not get, why this rule is supposed
not to work anymore!!


If my arguments are correct, I suggest the following improvement:

In case someone is using physdev in OUTPUT, display the message like it
is now: "using --physdev-out in the OUTPUT chains for non-bridged traffic
is not supported anymore".

In case it is used inside FORWARD, check if all physdev interfaces are
members of the same bridge. If yes, accept the rule, because then it is
allowed to use it!!!  (Which is the case all the thousands of rules in
my firewalls except the 5 that I sent to this list :-().
If no, display a message like this:

"physdev match: using --physdev-out in the FORWARD chains is only 
allowed if all physical interfaces are members of the same bridge."


What do thing about that?

Regards
Volker


-- 
  Volker Sauer  *  Poststrasse 1/601   *   64293 Darmstadt  *   Germany
  E-Mail/Jabber: volker(at)volker-sauer.de * http://www.volker-sauer.de
  PGPKey-Fingerprint: DB26 11C7 B12E 0B27 3999 2E4F 7E35 4E4D 5DD5 D0E0
  http://wwwkeys.de.pgp.net/pks/lookup?op=get&search=0x7E354E4D5DD5D0E0 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Problem with new --physdev-out style

[Patrick McHardy]

Volker Sauer wrote:
> 99% of my rules on all my firewalls are like that:
> 
> $IPTABLES -A FORWARD -i $BR_INT -m physdev --physdev-in $IF_INT
> --physdev-out $IF_DMZ -s $ZAPHOD -j ACCEPT
> 
> IF_INT (eth1) and IF_DMZ (vlan3) are both members of BR_INT (br-intern):
> 
> fw1: ~ # brctl show
> br-intern               8000.000d88cd28c1       yes     eth1
>                                                         vlan3
> 
> This means, that all rules like that are valid even with the new concept
> of netfilter, right?? But why do I get error messages like quoted in my
> first mail for these rules - it *is* bridged traffic inside *one*
> bridge!
> And: I don't see how --physdev-is-bridged should help, since it's a
> match and not a command to the kernel saying: "this *is* bridged
> traffic". It the kernel does not see this by itself,
> --physdev-is-bridged doesn't help.


Whether you believe it or not, this is the only way to tell
the physdev match that the rule only affects purely bridged
traffic.

> If my arguments are correct, I suggest the following improvement:
> 
> In case someone is using physdev in OUTPUT, display the message like it
> is now: "using --physdev-out in the OUTPUT chains for non-bridged traffic
> is not supported anymore".
> 
> In case it is used inside FORWARD, check if all physdev interfaces are
> members of the same bridge. If yes, accept the rule, because then it is
> allowed to use it!!!  (Which is the case all the thousands of rules in
> my firewalls except the 5 that I sent to this list :-().


Does not work since one of the devices might be put in a different
bridge after you loaded the rules.

> If no, display a message like this:
> 
> "physdev match: using --physdev-out in the FORWARD chains is only 
> allowed if all physical interfaces are members of the same bridge."


Feel free to send a patch to improve the error messages.

Re: Problem with new --physdev-out style

[Volker Sauer]

[-- Attachment #1: Type: text/plain, Size: 1611 bytes --]

On Mi, 24 Okt 2007, Patrick McHardy <kaber@trash.net> wrote:
> >This means, that all rules like that are valid even with the new concept
> >of netfilter, right?? But why do I get error messages like quoted in my
> >first mail for these rules - it *is* bridged traffic inside *one*
> >bridge!
> >And: I don't see how --physdev-is-bridged should help, since it's a
> >match and not a command to the kernel saying: "this *is* bridged
> >traffic". It the kernel does not see this by itself,
> >--physdev-is-bridged doesn't help.
> 
> Whether you believe it or not, this is the only way to tell
> the physdev match that the rule only affects purely bridged
> traffic.

Aah, now I get it! This way I can avoid the error message....
Great, I'll test it as soon as I'm back to the lab. 

> Does not work since one of the devices might be put in a different
> bridge after you loaded the rules.

Yes, you're right. 
Then the only solution is to improve the error messages.

> >If no, display a message like this:
> >
> >"physdev match: using --physdev-out in the FORWARD chains is only 
> >allowed if all physical interfaces are members of the same bridge."
> 
> 
> Feel free to send a patch to improve the error messages.

Oh, I'll see if I can patch kernel 2.6.23.1.

Regards
Volker

-- 
  Volker Sauer  *  Poststrasse 1/601   *   64293 Darmstadt  *   Germany
  E-Mail/Jabber: volker(at)volker-sauer.de * http://www.volker-sauer.de
  PGPKey-Fingerprint: DB26 11C7 B12E 0B27 3999 2E4F 7E35 4E4D 5DD5 D0E0
  http://wwwkeys.de.pgp.net/pks/lookup?op=get&search=0x7E354E4D5DD5D0E0 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Problem with new --physdev-out style

[Pascal Hambourg]

Volker Sauer a écrit :
> 
> In case someone is using physdev in OUTPUT, display the message like it
> is now: "using --physdev-out in the OUTPUT chains for non-bridged traffic
> is not supported anymore".

Ok.

> In case it is used inside FORWARD, check if all physdev interfaces are
> members of the same bridge.

As Patrick said, that condition may change over time. I like to have all 
my ruleset loaded before the network is configured, even before some 
interfaces exist. Your proposed change would prevent it. Besides, my 
opinion is that it is not the job of iptables to do such checks.

> If yes, accept the rule, because then it is
> allowed to use it!!!  (Which is the case all the thousands of rules in
> my firewalls except the 5 that I sent to this list :-().
> If no, display a message like this:
> 
> "physdev match: using --physdev-out in the FORWARD chains is only 
> allowed if all physical interfaces are members of the same bridge."

This is wrong and inacurate. Using --physdev-out in the FORWARD and 
POSTROUTING chains is supported for *bridged* traffic only, period. All 
physical interfaces being members of the same bridge is not a sufficient 
condition to make sure that only bridged traffic will be matched. 
Traffic can still be routed from a bridge to itself.

Re: Problem with new --physdev-out style

[Volker Sauer]

[-- Attachment #1: Type: text/plain, Size: 1693 bytes --]

On Mi, 24 Okt 2007, Pascal Hambourg <pascal.mail@plouf.fr.eu.org> wrote:
> As Patrick said, that condition may change over time. I like to have all my 
> ruleset loaded before the network is configured, even before some interfaces 
> exist. Your proposed change would prevent it. Besides, my opinion is that it 
> is not the job of iptables to do such checks.

Agreed.

>
>> If yes, accept the rule, because then it is
>> allowed to use it!!!  (Which is the case all the thousands of rules in
>> my firewalls except the 5 that I sent to this list :-().
>> If no, display a message like this:
>> "physdev match: using --physdev-out in the FORWARD chains is only allowed 
>> if all physical interfaces are members of the same bridge."
>
> This is wrong and inacurate. Using --physdev-out in the FORWARD and 
> POSTROUTING chains is supported for *bridged* traffic only, period. All 
> physical interfaces being members of the same bridge is not a sufficient 
> condition to make sure that only bridged traffic will be matched. Traffic 
> can still be routed from a bridge to itself.

Yes, it is inacurate.
But I think one needs a better explenation. I'm a power-user but still a
user, not a developer. Users think in different terms and speak another
language. 
Maybe an advice like "look for the option "--physdev-is-bridged" - it
may help you" or so would be good.

-- 
  Volker Sauer  *  Poststrasse 1/601   *   64293 Darmstadt  *   Germany
  E-Mail/Jabber: volker(at)volker-sauer.de * http://www.volker-sauer.de
  PGPKey-Fingerprint: DB26 11C7 B12E 0B27 3999 2E4F 7E35 4E4D 5DD5 D0E0
  http://wwwkeys.de.pgp.net/pks/lookup?op=get&search=0x7E354E4D5DD5D0E0 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Problem with new --physdev-out style

[Philip Craig]

Philip Craig wrote:
> The only solution I am aware of is to stop bridging and use routing
> and arp proxy.

Sorry, the above isn't the only solution.  Other possibilities:

1.  Filter on IP address instead of output interface.

2.  Delay the decision by setting a mark in iptables and filtering again
in ebtables.  eg the mark could simply encode which port the packet is
allowed for, and ebtables drops if it is bridged out a different port.