Re: [Bugme-new] [Bug 9719] New: when a system is configured as a bridge, and at the same time configured to have multipath weighted route, with one leg goes thru NAT and another without NAT, the nat path will intermittently get packets leaking out using internal IP without being SNAT-ted

Re: [Bugme-new] [Bug 9719] New: when a system is configured as a bridge, and at the same time configured to have multipath weighted route, with one leg goes thru NAT and another without NAT, the nat path will intermittently get packets leaking out using internal IP without being SNAT-ted

[Andrew Morton]

(switched to email.  Please respond via emailed reply-to-all, not via the
bugzilla web interface).

On Wed,  9 Jan 2008 11:55:50 -0800 (PST)
bugme-daemon@bugzilla.kernel.org wrote:

> http://bugzilla.kernel.org/show_bug.cgi?id=9719
> 
>            Summary: when a system is configured as a bridge, and at the same
>                     time configured to have multipath weighted route, with
>                     one leg goes thru NAT and another without NAT, the nat
>                     path will intermittently get packets leaking out using
>                     internal IP without being SNAT-ted
>            Product: Networking
>            Version: 2.5
>      KernelVersion: 2.6.22.15 and 2.6.23
>           Platform: All
>         OS/Version: Linux
>               Tree: Mainline
>             Status: NEW
>           Severity: normal
>           Priority: P1
>          Component: Netfilter/Iptables
>         AssignedTo: networking_netfilter-iptables@kernel-bugs.osdl.org
>         ReportedBy: mingching.tiew@redtone.com
> 
> 
> Latest working kernel version: 2.6.23
> Earliest failing kernel version: 2.6.22.15

This doesn't make sense.  What we're trying to ask here (and we've been
unable to find a pair of questions which 100% of reporters can successfully
answer) is whether this is a regression, and in which kernel release did we
regress?

In other words: did we break it, and if so, when did we break it?

> Distribution: iptables 1.4.0 was used with kernel 2.6.23 and iptables 1.3.8
> with 2.6.22.15
> Hardware Environment: 3 interfaces, 2 interfaces bridged to form br0, and
> another connects to internet using pppoe.
> Software Environment: bridge, multipath routing
> Problem Description: when a system is configured as a bridge with IP assigned
> to br0 interface, and at the same time it is configured to have multipath
> weighted default route, and one of the default route is NAT-ed and another of
> the default route is not NAT-ed, then it is NAT-ed interface will occasionally
> get packets leaking out to it with packets with private IPs.
> 
> Steps to reproduce: 
> 1) setup the bridge interface and assign an IP to it 
> 2) setup an default gateway on side B of the bridge ( without NAT ) and default
> route the bridge to this gateway. 
> 3) Setup a client on side A of the bridge and default route to the bridge br0
> interface.
> 4) Start ping'ing an internet site, for example www.google.com from the client.
>    Run the ping continuously, for example :-
>          while true
>          do
>             ping -c 1 www.google.com
>             sleep 1
>          done
> 5) after successfully and consistently getting a ping response from the
> www.google.com, on the bridge system start up another uplink to the internet,
> but this uplink is SNAT-ed 
> 
>        ( eg iptables -t nat -A POSTROUTING -o eth2 -j MASQUERADE ) 
> 
> 6) verify and make sure that the second uplink is working.
> 7) change the default route on the bridge to multipath weighted route with
> equal weight on both the uplinks.
> 8) sniff the NAT-ed inteface for packets coming in from the LAN client.
> Occasionallly packets with private IP leaks to the NAT-ed interface.
> 

Re: [Bugme-new] [Bug 9719] New: when a system is configured as a bridge, and at the same time configured to have multipath weighted route, with one leg goes thru NAT and another without NAT, the nat path will intermittently get packets leaking out using internal IP without being SNAT-ted

[Ming-Ching Tiew]

Andrew Morton wrote:
> (switched to email.  Please respond via emailed reply-to-all, not via the
> bugzilla web interface).
>
> On Wed,  9 Jan 2008 11:55:50 -0800 (PST)
> bugme-daemon@bugzilla.kernel.org wrote:
>
>   
>> http://bugzilla.kernel.org/show_bug.cgi?id=9719
>>
>>            Summary: when a system is configured as a bridge, and at the same
>>                     time configured to have multipath weighted route, with
>>                     one leg goes thru NAT and another without NAT, the nat
>>                     path will intermittently get packets leaking out using
>>                     internal IP without being SNAT-ted
>>            Product: Networking
>>            Version: 2.5
>>      KernelVersion: 2.6.22.15 and 2.6.23
>>           Platform: All
>>         OS/Version: Linux
>>               Tree: Mainline
>>             Status: NEW
>>           Severity: normal
>>           Priority: P1
>>          Component: Netfilter/Iptables
>>         AssignedTo: networking_netfilter-iptables@kernel-bugs.osdl.org
>>         ReportedBy: mingching.tiew@redtone.com
>>
>>
>> Latest working kernel version: 2.6.23
>> Earliest failing kernel version: 2.6.22.15
>>     
>
> This doesn't make sense.  What we're trying to ask here (and we've been
> unable to find a pair of questions which 100% of reporters can successfully
> answer) is whether this is a regression, and in which kernel release did we
> regress?
>
> In other words: did we break it, and if so, when did we break it?
>   

Sorry for the confusion and for such a lousy first time bug reporter.

I realized that mistake immediately after I posted it on the web 
interface. However, the web interface does not seem to allow me to 
correct that.

What I meant was that it failed on both the kernel versions I tested. I 
am afraid it  is a problem which exists all a long. Perhaps it has been 
broken quite sometime already. I need to go back to try some older 
kernel version and see if I could repeat the problem.

Regards.

Re: [Bugme-new] [Bug 9719] New: when a system is configured as a bridge, and at the same time configured to have multipath weighted route, with one leg goes thru NAT and another without NAT, the nat path will intermittently get packets leaking out using internal IP without being SNAT-ted

[Ming-Ching Tiew]

Ming-Ching Tiew wrote:
>
> What I meant was that it failed on both the kernel versions I tested. 
> I am afraid it  is a problem which exists all a long. Perhaps it has 
> been broken quite sometime already. I need to go back to try some 
> older kernel version and see if I could repeat the problem.

OK based on the I repeat the problem, so far I could not find such 
misbehaviour on kernel 2.6.18. I will do more tests to make it more 
conclusive.

Re: [Bugme-new] [Bug 9719] New: when a system is configured as a bridge, and at the same time configured to have multipath weighted route, with one leg goes thru NAT and another without NAT, the nat path will intermittently get packets leaking out using internal IP without being SNAT-ted

[Ming-Ching Tiew]

Ming-Ching Tiew wrote:
> Ming-Ching Tiew wrote:
>>
>> What I meant was that it failed on both the kernel versions I tested. 
>> I am afraid it  is a problem which exists all a long. Perhaps it has 
>> been broken quite sometime already. I need to go back to try some 
>> older kernel version and see if I could repeat the problem.
>
> OK based on the I repeat the problem, so far I could not find such 
> misbehaviour on kernel 2.6.18. I will do more tests to make it more 
> conclusive.
>

Sorry for jumping the gun. Kernel 2.6.18 has the same problem too.
I think from now on, I will refrain from early posting until conclusive 
results.

Re: [Bugme-new] [Bug 9719] New: when a system is configured as a bridge, and at the same time configured to have multipath weighted route, with one leg goes thru NAT and another without NAT, the nat path will intermittently get packets leaking out using internal IP without being SNAT-ted

[Patrick McHardy]

Andrew Morton wrote:
>> Distribution: iptables 1.4.0 was used with kernel 2.6.23 and iptables 1.3.8
>> with 2.6.22.15
>> Hardware Environment: 3 interfaces, 2 interfaces bridged to form br0, and
>> another connects to internet using pppoe.
>> Software Environment: bridge, multipath routing
>> Problem Description: when a system is configured as a bridge with IP assigned
>> to br0 interface, and at the same time it is configured to have multipath
>> weighted default route, and one of the default route is NAT-ed and another of
>> the default route is not NAT-ed, then it is NAT-ed interface will occasionally
>> get packets leaking out to it with packets with private IPs.


That is most likely because the route changes over time (when the cache
is flushed) and the NAT mappings for the connection have been set up on
a different interface. The way to properly do this is to add routing
rules based on fwmark and use CONNMARK to bind a connection to one of
the interfaces after the initial multipath routing decision.

Re: [Bugme-new] [Bug 9719] New: when a system is configured as a bridge, and at the same time configured to have multipath weighted route, with one leg goes thru NAT and another without NAT, the nat path will intermittently get packets leaking out using internal IP without being SNAT-ted

[Ming-Ching Tiew]

Patrick McHardy wrote:
> Andrew Morton wrote:
>>> Distribution: iptables 1.4.0 was used with kernel 2.6.23 and 
>>> iptables 1.3.8
>>> with 2.6.22.15
>>> Hardware Environment: 3 interfaces, 2 interfaces bridged to form 
>>> br0, and
>>> another connects to internet using pppoe.
>>> Software Environment: bridge, multipath routing
>>> Problem Description: when a system is configured as a bridge with IP 
>>> assigned
>>> to br0 interface, and at the same time it is configured to have 
>>> multipath
>>> weighted default route, and one of the default route is NAT-ed and 
>>> another of
>>> the default route is not NAT-ed, then it is NAT-ed interface will 
>>> occasionally
>>> get packets leaking out to it with packets with private IPs.
>
>
> That is most likely because the route changes over time (when the cache
> is flushed) and the NAT mappings for the connection have been set up on
> a different interface. The way to properly do this is to add routing
> rules based on fwmark and use CONNMARK to bind a connection to one of
> the interfaces after the initial multipath routing decision.
>

First of all, I would like to say a big thank you to all of you takes 
interest in replying my post/email. I have altered the distribution 
slightly and the kernel bug list is removed.

It seems from your reply, what is implied is that I cannot change route 
within a connection, and whatever things I do, I must make sure that the 
route remains the same for a particular netfilter connection ?

Regards.