Troubleshooting SNAT

Troubleshooting SNAT

[Steve Brueckner]

I have an FC5 (2.6.16.13-xen kernel) box with 2 interfaces:
eth0 is 192.168.1.221 (external network)
eth1 is 192.168.10.1 (internal network)

I've got to nat traffic through this box from host 192.168.10.2 
to host 192.168.1.12.  So I enabled ip forwarding and source nat 
on the multi-homed box:
# sysctl -w net.ipv4.ip_forward=1
# iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to 192.168.1.221

That didn't work; the packets were indeed forwarded but their 
source address was unchanged (still 192.168.10.2):
# tcpdump -n -i eth0
18:14:12.425317 IP 192.168.10.2 > 192.168.1.12: ICMP echo 
request, id 2617, seq 9, length 64

I also tried plain old Masquerading:
# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
This also does not change the packets' source address, but 
it does forward them from eth1 to eth0 again.

This similar command has a different but still incorrect effect:
# iptables -t nat -A POSTROUTING -j MASQUERADE
It changes the source address of the packets on eth1 but 
of course does not forward them to eth0.

Nothing seems to work.  Packets are either forwarded but without 
new source IPs or they get new source IPs but aren't forwarded.  
My filter table is wide open (no rules).

The same kernel can do SNAT just fine using Debian.  I'm starting 
to think FC5 is missing something.  However, I seem to have the following 
modules, which appear sufficient to me:
# lsmod | grep ip
ipt_MASQUERADE          3776  0
iptable_filter          3104  1
iptable_nat             8836  1
ip_nat                 18092  2 ipt_MASQUERADE,iptable_nat
ip_conntrack           55800  4 xt_state,ipt_MASQUERADE,iptable_nat,ip_nat
nfnetlink               6520  2 ip_nat,ip_conntrack
ip_tables              13636  2 iptable_filter,iptable_nat
x_tables               13188  6
xt_state,ipt_MASQUERADE,xt_tcpudp,xt_physdev,iptable_nat,ip_tables
ipv6                  269056  14

Any ideas on how to proceed with troubleshooting this?

Thanks,

Steve Brueckner, ATC-NY

RE: Troubleshooting SNAT

[Steve Brueckner]

Thanks, but using the --to-source switch seems to have the same effect 
as just using --to.  And my attempt to use Masquerading failed as well.

I'm new to iptables, but it doesn't seem too complex as a user to try 
to do this, so I really think the problem isn't with my usage of 
iptables but that something is either broken or missing in my kernel.

I think what we need to do is some debugging, but I was hoping for some
ideas on how to do that from this list.

Thanks

Steve Brueckner, ATC-NY

James Shewey wrote:
> did you try "iptables -t nat -A POSTROUTING -o eth0 -j SNAT
> --to-source 192.168.1.221" 
> 
> Perhaps this will yeild better results.
> 
> You should also be able to do what you want with _all_ traffic that
> flows through the router too using the masquerade table. This may not
> work for you solution though.  
> 
> 
> On 2/12/07, Steve Brueckner <steve@atc-nycorp.com> wrote:
>> I have an FC5 (2.6.16.13-xen kernel) box with 2 interfaces:
>> eth0 is 192.168.1.221 (external network)
>> eth1 is 192.168.10.1 (internal network)
>> 
>> I've got to nat traffic through this box from host 192.168.10.2 to
>> host 192.168.1.12.  So I enabled ip forwarding and source nat on the
>> multi-homed box: # sysctl -w net.ipv4.ip_forward=1
>> # iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to 192.168.1.221
>> 
>> That didn't work; the packets were indeed forwarded but their source
>> address was unchanged (still 192.168.10.2):
>> # tcpdump -n -i eth0
>> 18:14:12.425317 IP 192.168.10.2 > 192.168.1.12: ICMP echo request,
>> id 2617, seq 9, length 64 
>> 
>> I also tried plain old Masquerading:
>> # iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE This also does
>> not change the packets' source address, but it does forward them
>> from eth1 to eth0 again. 
>> 
>> This similar command has a different but still incorrect effect:
>> # iptables -t nat -A POSTROUTING -j MASQUERADE It changes the source
>> address of the packets on eth1 but of course does not forward them
>> to eth0. 
>> 
>> Nothing seems to work.  Packets are either forwarded but without new
>> source IPs or they get new source IPs but aren't forwarded.
>> My filter table is wide open (no rules).
>> 
>> The same kernel can do SNAT just fine using Debian.  I'm starting to
>> think FC5 is missing something.  However, I seem to have the
>> following modules, which appear sufficient to me:
>> # lsmod | grep ip
>> ipt_MASQUERADE          3776  0
>> iptable_filter          3104  1
>> iptable_nat             8836  1
>> ip_nat                 18092  2 ipt_MASQUERADE,iptable_nat
>> ip_conntrack           55800  4
>> xt_state,ipt_MASQUERADE,iptable_nat,ip_nat nfnetlink              
>> 6520  2 ip_nat,ip_conntrack 
>> ip_tables              13636  2 iptable_filter,iptable_nat
>> x_tables               13188  6
>> xt_state,ipt_MASQUERADE,xt_tcpudp,xt_physdev,iptable_nat,ip_tables
>> ipv6                  269056  14
>> 
>> Any ideas on how to proceed with troubleshooting this?
>> 
>> Thanks,
>> 
>> Steve Brueckner, ATC-NY

Re: Troubleshooting SNAT

[Matt Richards]

[-- Attachment #1: Type: text/plain, Size: 3533 bytes --]

Hello  :) 

donno if this will help much but have you tried inserting the rule and
not appending it ?
-I POSTROUTING -t nat -o eth0 -j SNAT --to

I have been a little stumped by rules jumping packets to other chains
before they hit my newly entered rule before.

huh,
Matty.



Steve Brueckner wrote:
> Thanks, but using the --to-source switch seems to have the same effect 
> as just using --to.  And my attempt to use Masquerading failed as well.
>
> I'm new to iptables, but it doesn't seem too complex as a user to try 
> to do this, so I really think the problem isn't with my usage of 
> iptables but that something is either broken or missing in my kernel.
>
> I think what we need to do is some debugging, but I was hoping for some
> ideas on how to do that from this list.
>
> Thanks
>
> Steve Brueckner, ATC-NY
>
> James Shewey wrote:
>   
>> did you try "iptables -t nat -A POSTROUTING -o eth0 -j SNAT
>> --to-source 192.168.1.221" 
>>
>> Perhaps this will yeild better results.
>>
>> You should also be able to do what you want with _all_ traffic that
>> flows through the router too using the masquerade table. This may not
>> work for you solution though.  
>>
>>
>> On 2/12/07, Steve Brueckner <steve@atc-nycorp.com> wrote:
>>     
>>> I have an FC5 (2.6.16.13-xen kernel) box with 2 interfaces:
>>> eth0 is 192.168.1.221 (external network)
>>> eth1 is 192.168.10.1 (internal network)
>>>
>>> I've got to nat traffic through this box from host 192.168.10.2 to
>>> host 192.168.1.12.  So I enabled ip forwarding and source nat on the
>>> multi-homed box: # sysctl -w net.ipv4.ip_forward=1
>>> # iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to 192.168.1.221
>>>
>>> That didn't work; the packets were indeed forwarded but their source
>>> address was unchanged (still 192.168.10.2):
>>> # tcpdump -n -i eth0
>>> 18:14:12.425317 IP 192.168.10.2 > 192.168.1.12: ICMP echo request,
>>> id 2617, seq 9, length 64 
>>>
>>> I also tried plain old Masquerading:
>>> # iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE This also does
>>> not change the packets' source address, but it does forward them
>>> from eth1 to eth0 again. 
>>>
>>> This similar command has a different but still incorrect effect:
>>> # iptables -t nat -A POSTROUTING -j MASQUERADE It changes the source
>>> address of the packets on eth1 but of course does not forward them
>>> to eth0. 
>>>
>>> Nothing seems to work.  Packets are either forwarded but without new
>>> source IPs or they get new source IPs but aren't forwarded.
>>> My filter table is wide open (no rules).
>>>
>>> The same kernel can do SNAT just fine using Debian.  I'm starting to
>>> think FC5 is missing something.  However, I seem to have the
>>> following modules, which appear sufficient to me:
>>> # lsmod | grep ip
>>> ipt_MASQUERADE          3776  0
>>> iptable_filter          3104  1
>>> iptable_nat             8836  1
>>> ip_nat                 18092  2 ipt_MASQUERADE,iptable_nat
>>> ip_conntrack           55800  4
>>> xt_state,ipt_MASQUERADE,iptable_nat,ip_nat nfnetlink              
>>> 6520  2 ip_nat,ip_conntrack 
>>> ip_tables              13636  2 iptable_filter,iptable_nat
>>> x_tables               13188  6
>>> xt_state,ipt_MASQUERADE,xt_tcpudp,xt_physdev,iptable_nat,ip_tables
>>> ipv6                  269056  14
>>>
>>> Any ideas on how to proceed with troubleshooting this?
>>>
>>> Thanks,
>>>
>>> Steve Brueckner, ATC-NY
>>>       
>
>
>
>   



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

RE: Troubleshooting SNAT

[Steve Brueckner]

Good idea, but this is the only rule I have!  So there's no other rules
to get in the way.

I think what I need is some sort of debug mode for iptables.

 - Steve

Matt Richards wrote:
> Hello  :)
> 
> donno if this will help much but have you tried inserting the rule
> and not appending it ? -I POSTROUTING -t nat -o eth0 -j SNAT --to
> 
> I have been a little stumped by rules jumping packets to other chains
> before they hit my newly entered rule before. 
> 
> huh,
> Matty.
> On 2/12/07, Steve Brueckner <steve@atc-nycorp.com> wrote:
> 
>>>> I have an FC5 (2.6.16.13-xen kernel) box with 2 interfaces:
>>>> eth0 is 192.168.1.221 (external network)
>>>> eth1 is 192.168.10.1 (internal network)
>>>> 
>>>> I've got to nat traffic through this box from host 192.168.10.2 to
>>>> host 192.168.1.12.  So I enabled ip forwarding and source nat on
>>>> the multi-homed box: # sysctl -w net.ipv4.ip_forward=1 # iptables
>>>> -t nat -A POSTROUTING -o eth0 -j SNAT --to 192.168.1.221
>>>> 
>>>> That didn't work; the packets were indeed forwarded but their
>>>> source address was unchanged (still 192.168.10.2):
>>>> # tcpdump -n -i eth0
>>>> 18:14:12.425317 IP 192.168.10.2 > 192.168.1.12: ICMP echo request,
>>>> id 2617, seq 9, length 64 
>>>> 
>>>> I also tried plain old Masquerading:
>>>> # iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE This also
>>>> does not change the packets' source address, but it does forward
>>>> them from eth1 to eth0 again. 
>>>> 
>>>> This similar command has a different but still incorrect effect:
>>>> # iptables -t nat -A POSTROUTING -j MASQUERADE It changes the
>>>> source address of the packets on eth1 but of course does not
>>>> forward them to eth0. 
>>>> 
>>>> Nothing seems to work.  Packets are either forwarded but without
>>>> new source IPs or they get new source IPs but aren't forwarded.
>>>> My filter table is wide open (no rules).
>>>> 
>>>> The same kernel can do SNAT just fine using Debian.  I'm starting
>>>> to think FC5 is missing something.  However, I seem to have the
>>>> following modules, which appear sufficient to me:
>>>> # lsmod | grep ip
>>>> ipt_MASQUERADE          3776  0
>>>> iptable_filter          3104  1
>>>> iptable_nat             8836  1
>>>> ip_nat                 18092  2 ipt_MASQUERADE,iptable_nat
>>>> ip_conntrack           55800  4
>>>> xt_state,ipt_MASQUERADE,iptable_nat,ip_nat nfnetlink 6520  2
>>>> ip_nat,ip_conntrack ip_tables              13636  2
>>>> iptable_filter,iptable_nat 
>>>> x_tables               13188  6
>>>> xt_state,ipt_MASQUERADE,xt_tcpudp,xt_physdev,iptable_nat,ip_tables
>>>> ipv6                  269056  14
>>>> 
>>>> Any ideas on how to proceed with troubleshooting this?
>>>> 
>>>> Thanks,
>>>> 
>>>> Steve Brueckner, ATC-NY

RE: Troubleshooting SNAT

[Steve Brueckner]

The FC5 I referred to in my original message is my distro; it stands for 
"Fedora Core 5."  I will ask on that list next but I figured they were
generalists and this list had the iptables experts.

I'm not really a visual person, and prefer to understand what's going on 
under the hood, but thanks for the GUI idea.  My syntax seems to be
correct; as I said in my original message this works fine on another 
machine.

This is the only rule I have, so there is no script of other rules to
show.  The rule in my original message is the only one in play.  I 
should have mentioned that earlier.

I included the partial output from lsmod in my original message in case 
anyone on this list could point out a missing module.

 - Steve

James Shewey wrote:
> What distro are you using? Perhaps someone on that distro can offer
> some advice. Also, if you are a more visual person, you might think
> about kmyfirewall which is still gives you the conceptual overview
> while alleviating questions about whether you go the syntax right.
> Also, you might consider sending the script of your iptables rules to
> the list so that they can make sure there aren't any problems there. 
> Also, you can make sure you have all your kernel modules loaded using
> lsmod. 
> 
> On 2/12/07, Steve Brueckner <steve@atc-nycorp.com> wrote:
>> Thanks, but using the --to-source switch seems to have the same
>> effect as just using --to.  And my attempt to use Masquerading
>> failed as well. 
>> 
>> I'm new to iptables, but it doesn't seem too complex as a user to try
>> to do this, so I really think the problem isn't with my usage of
>> iptables but that something is either broken or missing in my kernel.
>> 
>> I think what we need to do is some debugging, but I was hoping for
>> some ideas on how to do that from this list.
>> 
>> Thanks
>> 
>> Steve Brueckner, ATC-NY
>> 
>> James Shewey wrote:
>>> did you try "iptables -t nat -A POSTROUTING -o eth0 -j SNAT
>>> --to-source 192.168.1.221" 
>>> 
>>> Perhaps this will yeild better results.
>>> 
>>> You should also be able to do what you want with _all_ traffic that
>>> flows through the router too using the masquerade table. This may
>>> not work for you solution though.
>>> 
>>> 
>>> On 2/12/07, Steve Brueckner <steve@atc-nycorp.com> wrote:
>>>> I have an FC5 (2.6.16.13-xen kernel) box with 2 interfaces:
>>>> eth0 is 192.168.1.221 (external network)
>>>> eth1 is 192.168.10.1 (internal network)
>>>> 
>>>> I've got to nat traffic through this box from host 192.168.10.2 to
>>>> host 192.168.1.12.  So I enabled ip forwarding and source nat on
>>>> the multi-homed box: # sysctl -w net.ipv4.ip_forward=1 # iptables
>>>> -t nat -A POSTROUTING -o eth0 -j SNAT --to 192.168.1.221
>>>> 
>>>> That didn't work; the packets were indeed forwarded but their
>>>> source address was unchanged (still 192.168.10.2):
>>>> # tcpdump -n -i eth0
>>>> 18:14:12.425317 IP 192.168.10.2 > 192.168.1.12: ICMP echo request,
>>>> id 2617, seq 9, length 64 
>>>> 
>>>> I also tried plain old Masquerading:
>>>> # iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE This also
>>>> does not change the packets' source address, but it does forward
>>>> them from eth1 to eth0 again. 
>>>> 
>>>> This similar command has a different but still incorrect effect:
>>>> # iptables -t nat -A POSTROUTING -j MASQUERADE It changes the
>>>> source address of the packets on eth1 but of course does not
>>>> forward them to eth0. 
>>>> 
>>>> Nothing seems to work.  Packets are either forwarded but without
>>>> new source IPs or they get new source IPs but aren't forwarded.
>>>> My filter table is wide open (no rules).
>>>> 
>>>> The same kernel can do SNAT just fine using Debian.  I'm starting
>>>> to think FC5 is missing something.  However, I seem to have the
>>>> following modules, which appear sufficient to me:
>>>> # lsmod | grep ip
>>>> ipt_MASQUERADE          3776  0
>>>> iptable_filter          3104  1
>>>> iptable_nat             8836  1
>>>> ip_nat                 18092  2 ipt_MASQUERADE,iptable_nat
>>>> ip_conntrack           55800  4
>>>> xt_state,ipt_MASQUERADE,iptable_nat,ip_nat nfnetlink 6520  2
>>>> ip_nat,ip_conntrack ip_tables              13636  2
>>>> iptable_filter,iptable_nat x_tables               13188  6
>>>> xt_state,ipt_MASQUERADE,xt_tcpudp,xt_physdev,iptable_nat,ip_tables
>>>> ipv6                  269056  14
>>>> 
>>>> Any ideas on how to proceed with troubleshooting this?
>>>> 
>>>> Thanks,
>>>> 
>>>> Steve Brueckner, ATC-NY

RE: Troubleshooting SNAT

[Steve Brueckner]

Well naturally I shot myself in the foot by not giving everyone the full
info.  Careful readers will notice that I'm executing a Xen kernel, and
indeed if I was in a Xen domU (guest VM) then my problem would have been
truly mystifying.  However, I am executing in dom0 (the VMM domain), which
has a somewhat more complicated view of system networking.  Specifically,
there are bridges and virtual interfaces.  I think my problem is arising
because bridges operate at the MAC level and not the IP level.  So right now
I'm looking into ebtables to fix my problem.  I'll report back if/when I
figure it out entirely.

Steve Brueckner, ATC-NY