NAT through Dom0 on unstable branch

NAT through Dom0 on unstable branch

[Eitan Isaacson]

Hello,
I might be the first to encounter this issue, so I think it is worth posting...

I am trying to set up a configuration in which a DomU and Dom0 are on
their own subnet, and DomU accesses the real network through Dom0's
NAT.

These are the steps that I take (note, these steps worked fine in
"testing" and "stable" branches):

I first set up a bridge with a private IP:

brctl addbr mybr0
ip addr add 192.168.0.1/24 dev mybr0
ip link set mybr0 up

These are my network scripts in the xend config file:

(network-script     network-route)
(vif-script         vif-bridge)
(vif-bridge         mybr0)

I start a DomU with an IP of 192.168.0.101, and I am able to ping Dom0
(192.186.0.1).

I enable IP forwarding, and I set up NAT:

sysctl -w net.ipv4.ip_forward=1  # (if not allready done by xend)
iptables -t nat -A POSTROUTING -j MASQUERADE \
  -o eth0 -s 192.168.0.1/24

I am able to ping the outside world by IP, but name resolution, or any
other TCP/IP traffic does not work.
The very strange part is that tcpdump seems to show packets arriving
at their destination on the outside world hosts. but besides the ACKs,
the outside hosts do not respond.
I am not a networking expert, so I am sorry if my diagnoses is not full.

Here is a bit more info (on Dom0):
# iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  192.168.0.0/24       anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0F:1F:8C:17:D6
          inet addr: 10.0.20.10  Bcast:10.0.255.255  Mask:255.255.0.0
          UP BROADCAST NOTRAILERS RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:30868 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1121 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:8172500 (7.7 Mb)  TX bytes:158965 (155.2 Kb)
          Base address:0xdf40 Memory:fcfe0000-fd000000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:5 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:288 (288.0 b)  TX bytes:288 (288.0 b)

mybr0     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet addr:192.168.0.1  Bcast:0.0.0.0  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:12 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:834 (834.0 b)  TX bytes:938 (938.0 b)

vif1.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:13 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1044 (1.0 Kb)  TX bytes:812 (812.0 b)

# brctl show
bridge name     bridge id               STP enabled     interfaces
mybr0           8000.feffffffffff       no              vif1.0


Thanks!

Re: NAT through Dom0 on unstable branch

[Nils Toedtmann]

Am Montag, den 20.06.2005, 11:40 -0700 schrieb Eitan Isaacson:
> Hello,
> I might be the first to encounter this issue, so I think it is worth posting...
> 
> I am trying to set up a configuration in which a DomU and Dom0 are on
> their own subnet, and DomU accesses the real network through Dom0's
> NAT.
> 
> These are the steps that I take (note, these steps worked fine in
> "testing" and "stable" branches):
> 
> I first set up a bridge with a private IP:
> 
> brctl addbr mybr0
> ip addr add 192.168.0.1/24 dev mybr0
> ip link set mybr0 up
> 
> These are my network scripts in the xend config file:
> 
> (network-script     network-route)
> (vif-script         vif-bridge)
> (vif-bridge         mybr0)
> 
> I start a DomU with an IP of 192.168.0.101, and I am able to ping Dom0
> (192.186.0.1).
> 
> I enable IP forwarding, and I set up NAT:
> 
> sysctl -w net.ipv4.ip_forward=1  # (if not allready done by xend)
> iptables -t nat -A POSTROUTING -j MASQUERADE \
>   -o eth0 -s 192.168.0.1/24
> 
> I am able to ping the outside world by IP, but name resolution, or any
> other TCP/IP traffic does not work.
> The very strange part is that tcpdump seems to show packets arriving
> at their destination on the outside world hosts. but besides the ACKs,
> the outside hosts do not respond.
> I am not a networking expert, so I am sorry if my diagnoses is not full.

I cannot see any error (which does not mean that there is no one). Make
sure ("iptables -nL") that the filter chains have all "ACCEPT".

So TCP-SYN hits the target, it answers with TCP-SYN/ACK and the last
step in the TCP handshake (TCP-ACK domU-->target) is missing, right?
Sniff on vif1.0 (or better on domU's eth0) if the targets answer
(TCP-SYN/ACK) arrives at the domU and if the third packet (TCP-ACK)
leaves it. Check ("tcpdump -e") if the frames have the correct
destination MACs.

/nils.

> Here is a bit more info (on Dom0):
> # iptables -t nat -L
> Chain PREROUTING (policy ACCEPT)
> target     prot opt source               destination
> 
> Chain POSTROUTING (policy ACCEPT)
> target     prot opt source               destination
> MASQUERADE  all  --  192.168.0.0/24       anywhere
> 
> Chain OUTPUT (policy ACCEPT)
> target     prot opt source               destination
> 
> # ifconfig
> eth0      Link encap:Ethernet  HWaddr 00:0F:1F:8C:17:D6
>           inet addr: 10.0.20.10  Bcast:10.0.255.255  Mask:255.255.0.0
>           UP BROADCAST NOTRAILERS RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:30868 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:1121 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:8172500 (7.7 Mb)  TX bytes:158965 (155.2 Kb)
>           Base address:0xdf40 Memory:fcfe0000-fd000000
> 
> lo        Link encap:Local Loopback
>           inet addr:127.0.0.1  Mask:255.0.0.0
>           UP LOOPBACK RUNNING  MTU:16436  Metric:1
>           RX packets:5 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:288 (288.0 b)  TX bytes:288 (288.0 b)
> 
> mybr0     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>           inet addr:192.168.0.1  Bcast:0.0.0.0  Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:12 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:834 (834.0 b)  TX bytes:938 (938.0 b)
> 
> vif1.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:13 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:1044 (1.0 Kb)  TX bytes:812 (812.0 b)
> 
> # brctl show
> bridge name     bridge id               STP enabled     interfaces
> mybr0           8000.feffffffffff       no              vif1.0
-- 
there is no sig

Re: NAT through Dom0 on unstable branch

[Grzegorz Milos]

There are some scripts for setting up the NAT already in Xen (although they do 
not do exactly the same thing as you want to, as no bridge is set up. At 
least in the original version that I came up with).
It would be interesting to see if they work.

Cheers
Gregor

> Hello,
> I might be the first to encounter this issue, so I think it is worth
> posting...
>
> I am trying to set up a configuration in which a DomU and Dom0 are on
> their own subnet, and DomU accesses the real network through Dom0's
> NAT.
>
> These are the steps that I take (note, these steps worked fine in
> "testing" and "stable" branches):
>
> I first set up a bridge with a private IP:
>
> brctl addbr mybr0
> ip addr add 192.168.0.1/24 dev mybr0
> ip link set mybr0 up
>
> These are my network scripts in the xend config file:
>
> (network-script     network-route)
> (vif-script         vif-bridge)
> (vif-bridge         mybr0)
>
> I start a DomU with an IP of 192.168.0.101, and I am able to ping Dom0
> (192.186.0.1).
>
> I enable IP forwarding, and I set up NAT:
>
> sysctl -w net.ipv4.ip_forward=1  # (if not allready done by xend)
> iptables -t nat -A POSTROUTING -j MASQUERADE \
>   -o eth0 -s 192.168.0.1/24
>
> I am able to ping the outside world by IP, but name resolution, or any
> other TCP/IP traffic does not work.
> The very strange part is that tcpdump seems to show packets arriving
> at their destination on the outside world hosts. but besides the ACKs,
> the outside hosts do not respond.
> I am not a networking expert, so I am sorry if my diagnoses is not full.
>
> Here is a bit more info (on Dom0):
> # iptables -t nat -L
> Chain PREROUTING (policy ACCEPT)
> target     prot opt source               destination
>
> Chain POSTROUTING (policy ACCEPT)
> target     prot opt source               destination
> MASQUERADE  all  --  192.168.0.0/24       anywhere
>
> Chain OUTPUT (policy ACCEPT)
> target     prot opt source               destination
>
> # ifconfig
> eth0      Link encap:Ethernet  HWaddr 00:0F:1F:8C:17:D6
>           inet addr: 10.0.20.10  Bcast:10.0.255.255  Mask:255.255.0.0
>           UP BROADCAST NOTRAILERS RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:30868 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:1121 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000
>           RX bytes:8172500 (7.7 Mb)  TX bytes:158965 (155.2 Kb)
>           Base address:0xdf40 Memory:fcfe0000-fd000000
>
> lo        Link encap:Local Loopback
>           inet addr:127.0.0.1  Mask:255.0.0.0
>           UP LOOPBACK RUNNING  MTU:16436  Metric:1
>           RX packets:5 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:288 (288.0 b)  TX bytes:288 (288.0 b)
>
> mybr0     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>           inet addr:192.168.0.1  Bcast:0.0.0.0  Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:12 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:834 (834.0 b)  TX bytes:938 (938.0 b)
>
> vif1.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:13 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:10 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0
>           RX bytes:1044 (1.0 Kb)  TX bytes:812 (812.0 b)
>
> # brctl show
> bridge name     bridge id               STP enabled     interfaces
> mybr0           8000.feffffffffff       no              vif1.0
>
>
> Thanks!
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

-- 
Quidquid latine dictum sit, altum viditur --- Anon

Re: NAT through Dom0 on unstable branch

[Bernhard Schmidt]

On 2005-06-20, Eitan Isaacson <ee.jay.eye@gmail.com> wrote:

Hi,

> I am trying to set up a configuration in which a DomU and Dom0 are on
> their own subnet, and DomU accesses the real network through Dom0's
> NAT.
>
> These are the steps that I take (note, these steps worked fine in
> "testing" and "stable" branches):

I've seen a similar problem, which might or might not be related...

Although not being a developer, I'm running XEN unstable because I need
ACPI for my network interface chip. My network setup looked like the
following

                ,----------------------------------------------------,
                |                                          ,-------- |
                |  xen0       ,--------,  ,------,         |       | |
                |           ,-+ vlan10 +--+ br10 +-vif0.0 --- eth0 | |
                | ,------,  | `--------´  `------´         |       | |
,--------,      | |      |  | ,--------,  ,------,         |       | |
| Switch +--------+ eth0 +--+-+ vlan11 +--+ br11 |         |  xenU | |
`--------´      | |      |  | `--------´  `------´         |       | |
                | `------,  | ,--------,  ,------,         |       | |
                |           `-+ vlan20 +--+ br20 |         `-------´ |
                |             `--------´  `------`                   |
                |                                                    |
                `----------------------------------------------------´
                      

vlan10 is my internal network connecting all clients, vlan11 was unused, 
vlan20 contains the PPPoE modem to connect to my ISP. Besides having this
PPPoE connection xen0 also runs an IPv6-in-IPv4 tunnel for IPv6 connectivity
of the whole network.

xen0 only had an RFC1918 IP address and an IPv6 address assigned on br10, a
public IPv4 address on ppp0 and another IPv6 address on sixxs (the
IPv6-tunnel). xen0 has NAT compiled in and enabled with

iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE

Connectivity between other clients, xen0 and xenU worked flawless in both
IPv4 and IPv6. Other clients could also access the internet being NATed in
xen0 for IPv4 and routed in IPv6. xen0 had no connection problems as well.

xenU could connect to IPv6 hosts in the internet without a single flaw. But,
as soon as a single IPv4 packet was sent from xenU to the internet I could
not transfer one single byte through the DSL line anymore, even from xen0
itself. tcpdump on ppp0 showed only incoming packets from some idiots
hammering my dynamic IP for peer2peer applications, but not a single byte
going out. Routing-table looked good. As soon as I killed pppd (closing
ppp0) and restarting it again it worked, until sending IPv4 from xenU again.

The box is an Athlon64 in i386-mode on an nVidia nForce3 chipset, the NIC is
a Marvell GigE. Both xen0 and xenU are Ubuntu hoary, the xen release was
downloaded the day after the patch from 2.6.11.11 to 2.6.11.12 was included
in xen-unstable. xen's own networking scripts creating the bridge at startup
have been disabled in favour of manually setting up the vlans and the
bridges in the Debian/Ubuntu way. When bridging vlan20 to xenU as well and
running PPPoE (and of course masquerading) there everything works like a
charm (this is my current setup).

Any ideas? 

Bernhard

Re: Re: NAT through Dom0 on unstable branch

[Sheng S Lu]

[-- Attachment #1.1: Type: text/plain, Size: 4241 bytes --]

I've met a similar case with my desktop and laptop computer connected with 
a switcher, not in XEN.

The reason of my problem is that I set a conflict IP address in one of my 
computer with the maintenance IP address of DSL modem.
And when the conflicting computer send out packets, the switcher puzzled 
and frozen. (Many DSL modems use IP address like 192.168.0.1, 
192.168.0.11, 10.0.0.1 and so on)

It may not be the reason of your case. Just for you information.




Bernhard Schmidt <berni@birkenwald.de> 
Sent by: xen-devel-bounces@lists.xensource.com
2005-06-22 01:51

To
xen-devel@lists.xensource.com
cc

Subject
[Xen-devel] Re: NAT through Dom0 on unstable branch






On 2005-06-20, Eitan Isaacson <ee.jay.eye@gmail.com> wrote:

Hi,

> I am trying to set up a configuration in which a DomU and Dom0 are on
> their own subnet, and DomU accesses the real network through Dom0's
> NAT.
>
> These are the steps that I take (note, these steps worked fine in
> "testing" and "stable" branches):

I've seen a similar problem, which might or might not be related...

Although not being a developer, I'm running XEN unstable because I need
ACPI for my network interface chip. My network setup looked like the
following

                ,----------------------------------------------------,
                |                                          ,-------- |
                |  xen0       ,--------,  ,------,         |       | |
                |           ,-+ vlan10 +--+ br10 +-vif0.0 --- eth0 | |
                | ,------,  | `--------´  `------´         |       | |
,--------,      | |      |  | ,--------,  ,------,         |       | |
| Switch +--------+ eth0 +--+-+ vlan11 +--+ br11 |         |  xenU | |
`--------´      | |      |  | `--------´  `------´         |       | |
                | `------,  | ,--------,  ,------,         |       | |
                |           `-+ vlan20 +--+ br20 |         `-------´ |
                |             `--------´  `------`                   |
                |                                                    |
                `----------------------------------------------------´
 

vlan10 is my internal network connecting all clients, vlan11 was unused, 
vlan20 contains the PPPoE modem to connect to my ISP. Besides having this
PPPoE connection xen0 also runs an IPv6-in-IPv4 tunnel for IPv6 
connectivity
of the whole network.

xen0 only had an RFC1918 IP address and an IPv6 address assigned on br10, 
a
public IPv4 address on ppp0 and another IPv6 address on sixxs (the
IPv6-tunnel). xen0 has NAT compiled in and enabled with

iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE

Connectivity between other clients, xen0 and xenU worked flawless in both
IPv4 and IPv6. Other clients could also access the internet being NATed in
xen0 for IPv4 and routed in IPv6. xen0 had no connection problems as well.

xenU could connect to IPv6 hosts in the internet without a single flaw. 
But,
as soon as a single IPv4 packet was sent from xenU to the internet I could
not transfer one single byte through the DSL line anymore, even from xen0
itself. tcpdump on ppp0 showed only incoming packets from some idiots
hammering my dynamic IP for peer2peer applications, but not a single byte
going out. Routing-table looked good. As soon as I killed pppd (closing
ppp0) and restarting it again it worked, until sending IPv4 from xenU 
again.

The box is an Athlon64 in i386-mode on an nVidia nForce3 chipset, the NIC 
is
a Marvell GigE. Both xen0 and xenU are Ubuntu hoary, the xen release was
downloaded the day after the patch from 2.6.11.11 to 2.6.11.12 was 
included
in xen-unstable. xen's own networking scripts creating the bridge at 
startup
have been disabled in favour of manually setting up the vlans and the
bridges in the Debian/Ubuntu way. When bridging vlan20 to xenU as well and
running PPPoE (and of course masquerading) there everything works like a
charm (this is my current setup).

Any ideas? 

Bernhard


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel


[-- Attachment #1.2: Type: text/html, Size: 6642 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: NAT through Dom0 on unstable branch

[Toens Bueker]

Eitan Isaacson <ee.jay.eye@gmail.com> wrote:

> These are the steps that I take (note, these steps worked fine in
> "testing" and "stable" branches):
> 
> I first set up a bridge with a private IP:
> 
> brctl addbr mybr0
> ip addr add 192.168.0.1/24 dev mybr0
> ip link set mybr0 up
> 
> These are my network scripts in the xend config file:
> 
> (network-script     network-route)
> (vif-script         vif-bridge)
> (vif-bridge         mybr0)
> 
> I start a DomU with an IP of 192.168.0.101, and I am able to ping Dom0
> (192.186.0.1).
> 
> I enable IP forwarding, and I set up NAT:
> 
> sysctl -w net.ipv4.ip_forward=1  # (if not allready done by xend)
> iptables -t nat -A POSTROUTING -j MASQUERADE \
>   -o eth0 -s 192.168.0.1/24

An alternative solution would be:

iptables -t nat -F POSTROUTING
iptables -t nat -F PREROUTING
iptables -t nat -A POSTROUTING -o eth0 -s 192.168.2.0/24 -j SNAT \ 
         --to-source <your.public.ip.address>

for incoming connections you could do some portforwarding:

iptables -t nat -A PREROUTING -i eth0 -d <your.public.ip.address> \ 
         -p tcp --dport 80 -j DNAT --to-dest <ip.address.of.domU>

by
Töns
-- 
There is no safe distance.

Re: NAT through Dom0 on unstable branch

[Rusty Russell]

On Tue, 2005-06-21 at 21:15 +0200, Toens Bueker wrote:
> Eitan Isaacson <ee.jay.eye@gmail.com> wrote:
> > sysctl -w net.ipv4.ip_forward=1  # (if not allready done by xend)
> > iptables -t nat -A POSTROUTING -j MASQUERADE \
> >   -o eth0 -s 192.168.0.1/24
> 
> An alternative solution would be:
> 
> iptables -t nat -F POSTROUTING
> iptables -t nat -F PREROUTING
> iptables -t nat -A POSTROUTING -o eth0 -s 192.168.2.0/24 -j SNAT \ 
>          --to-source <your.public.ip.address>

Yes.  Unless the IP address of eth0 is dynamic, the latter form is
preferable.

Cheers,
Rusty.
-- 
A bad analogy is like a leaky screwdriver -- Richard Braakman