Re: Adding support for Stateless Static NAT for TAP devices

Re: Adding support for Stateless Static NAT for TAP devices

[Stefan Hajnoczi]

On Thu, Aug 30, 2012 at 10:27 AM, John Basila <jbasila@checkpoint.com> wrote:
> I have tried NAT and this is why I came up with this feature.

QEMU's net/tap.c is the wrong place to add NAT code.  The point of tap
is to use the host network stack.  If you want userspace networking,
use -netdev user or -netdev socket.

Please look into iptables more.  I have CCed the netfilter mailing
list.  The question is:

The host has several tap interfaces (tap0, tap1, ...) and the machine
on the other end of each tap interface uses IP address 10.0.0.2.  So
we have:

tap0 <-> virtual machine #0 (10.0.0.2)
tap1 <-> virtual machine #1 (10.0.0.2)
tap2 <-> virtual machine #2 (10.0.0.2)

Because the virtual machines all use the same static IP address, they
cannot communicate with each other or the outside world (they fight
over ARP).  We'd like to NAT the tap interfaces:

tap0 <-> virtual machine #0 (10.0.0.2 NAT to 192.168.0.2)
tap1 <-> virtual machine #1 (10.0.0.2 NAT to 192.168.0.3)
tap2 <-> virtual machine #2 (10.0.0.2 NAT to 192.168.0.4)

This would allow the virtual machines to communicate even though each
believes it is 10.0.0.2.

How can this be done using iptables and friends?

Thanks,
Stefan

RE: Adding support for Stateless Static NAT for TAP devices

[John Basila]

Please allow me to add a few comments:

The problem here is related to the fact that QEMU is executed with multiple instances and all instances start from the same snapshot, thus if they all send a UDP DNS query, they will all create a packet - for example - 10.0.0.2:2345 -> DNSERVER:53. The source port is the same. The first packet that reaches the ipfilter will result in going over the iptables rules and get NATed properly, the second QEMU instance that will send the same UDP packet will not get to run over the iptables rules as the ipfilter already saw this packet and the packet should be "RELATED" to a different connection and thus will cause the response packets of machine B to be received via machine A as the NAT rule will de-NAT the return packet to to the relevant connection which is related to machine A.

John

-----Original Message-----
From: Stefan Hajnoczi [mailto:stefanha@gmail.com] 
Sent: Thursday, August 30, 2012 1:44 PM
To: John Basila
Cc: qemu-devel@nongnu.org; Anthony Liguori; Rusty Russell; netfilter@vger.kernel.org
Subject: Re: Adding support for Stateless Static NAT for TAP devices

On Thu, Aug 30, 2012 at 10:27 AM, John Basila <jbasila@checkpoint.com> wrote:
> I have tried NAT and this is why I came up with this feature.

QEMU's net/tap.c is the wrong place to add NAT code.  The point of tap is to use the host network stack.  If you want userspace networking, use -netdev user or -netdev socket.

Please look into iptables more.  I have CCed the netfilter mailing list.  The question is:

The host has several tap interfaces (tap0, tap1, ...) and the machine on the other end of each tap interface uses IP address 10.0.0.2.  So we have:

tap0 <-> virtual machine #0 (10.0.0.2)
tap1 <-> virtual machine #1 (10.0.0.2)
tap2 <-> virtual machine #2 (10.0.0.2)

Because the virtual machines all use the same static IP address, they cannot communicate with each other or the outside world (they fight over ARP).  We'd like to NAT the tap interfaces:

tap0 <-> virtual machine #0 (10.0.0.2 NAT to 192.168.0.2)
tap1 <-> virtual machine #1 (10.0.0.2 NAT to 192.168.0.3)
tap2 <-> virtual machine #2 (10.0.0.2 NAT to 192.168.0.4)

This would allow the virtual machines to communicate even though each believes it is 10.0.0.2.

How can this be done using iptables and friends?

Thanks,
Stefan

Scanned by Check Point Total Security Gateway.

Re: Adding support for Stateless Static NAT for TAP devices

[Ivan Shmakov]

>>>>> John Basila <jbasila@checkpoint.com> writes:

[…]

 > The problem here is related to the fact that QEMU is executed with
 > multiple instances and all instances start from the same snapshot,

	Isn't it possible to resolve such an issue using, e. g., DHCPv6
	or DHCP?  All the QEMU instances will (AIUI) have random MAC
	addresses by default, but a static “instance to MAC” mapping is
	also possible, as is the respective “MAC to IP” mapping.

[…]

-- 
FSF associate member #7257	http://sfd.am-1.org/

Re: Adding support for Stateless Static NAT for TAP devices

[Dennis Jacobfeuerborn]

On 08/30/2012 12:58 PM, John Basila wrote:
> Please allow me to add a few comments:
> 
> The problem here is related to the fact that QEMU is executed with multiple instances and all instances start from the same snapshot, thus if they all send a UDP DNS query, they will all create a packet - for example - 10.0.0.2:2345 -> DNSERVER:53. The source port is the same. The first packet that reaches the ipfilter will result in going over the iptables rules and get NATed properly, the second QEMU instance that will send the same UDP packet will not get to run over the iptables rules as the ipfilter already saw this packet and the packet should be "RELATED" to a different connection and thus will cause the response packets of machine B to be received via machine A as the NAT rule will de-NAT the return packet to to the relevant connection which is related to machine A.
> 
> John
> 
> -----Original Message-----
> From: Stefan Hajnoczi [mailto:stefanha@gmail.com] 
> Sent: Thursday, August 30, 2012 1:44 PM
> To: John Basila
> Cc: qemu-devel@nongnu.org; Anthony Liguori; Rusty Russell; netfilter@vger.kernel.org
> Subject: Re: Adding support for Stateless Static NAT for TAP devices
> 
> On Thu, Aug 30, 2012 at 10:27 AM, John Basila <jbasila@checkpoint.com> wrote:
>> I have tried NAT and this is why I came up with this feature.
> 
> QEMU's net/tap.c is the wrong place to add NAT code.  The point of tap is to use the host network stack.  If you want userspace networking, use -netdev user or -netdev socket.
> 
> Please look into iptables more.  I have CCed the netfilter mailing list.  The question is:
> 
> The host has several tap interfaces (tap0, tap1, ...) and the machine on the other end of each tap interface uses IP address 10.0.0.2.  So we have:
> 
> tap0 <-> virtual machine #0 (10.0.0.2)
> tap1 <-> virtual machine #1 (10.0.0.2)
> tap2 <-> virtual machine #2 (10.0.0.2)
> 
> Because the virtual machines all use the same static IP address, they cannot communicate with each other or the outside world (they fight over ARP).  We'd like to NAT the tap interfaces:
> 
> tap0 <-> virtual machine #0 (10.0.0.2 NAT to 192.168.0.2)
> tap1 <-> virtual machine #1 (10.0.0.2 NAT to 192.168.0.3)
> tap2 <-> virtual machine #2 (10.0.0.2 NAT to 192.168.0.4)
> 
> This would allow the virtual machines to communicate even though each believes it is 10.0.0.2.
> 
> How can this be done using iptables and friends?

Why do the systems have the same IP? That seems like a broken network
config to me.

Regards,
  Dennis

RE: Adding support for Stateless Static NAT for TAP devices

[John Basila]

I have a setup that requires to run virtual machines using QEMU. All these machines will be executed from the same snapshot thus giving them the save same state when they come to life, this is why they all will have the same source IP, the only difference between them is that each one is connected to a different TAP device.

I have tried using iptables to NAT the connections based on the TAP interface and make it change the source IP of the connection to an IP that upon return will go back to the correct TAP device, but alas, the problem of state fullness of the iptables caused the problem with the second instance of the virtual machine that sent the same packet that was did pass the rule base as it was matched on an already opened connection that thus was NATed to the first virtual machine source IP.

-----Original Message-----
From: Dennis Jacobfeuerborn [mailto:dennisml@conversis.de] 
Sent: Thursday, August 30, 2012 3:38 PM
To: John Basila
Cc: Stefan Hajnoczi; qemu-devel@nongnu.org; Anthony Liguori; Rusty Russell; netfilter@vger.kernel.org
Subject: Re: Adding support for Stateless Static NAT for TAP devices

On 08/30/2012 12:58 PM, John Basila wrote:
> Please allow me to add a few comments:
> 
> The problem here is related to the fact that QEMU is executed with multiple instances and all instances start from the same snapshot, thus if they all send a UDP DNS query, they will all create a packet - for example - 10.0.0.2:2345 -> DNSERVER:53. The source port is the same. The first packet that reaches the ipfilter will result in going over the iptables rules and get NATed properly, the second QEMU instance that will send the same UDP packet will not get to run over the iptables rules as the ipfilter already saw this packet and the packet should be "RELATED" to a different connection and thus will cause the response packets of machine B to be received via machine A as the NAT rule will de-NAT the return packet to to the relevant connection which is related to machine A.
> 
> John
> 
> -----Original Message-----
> From: Stefan Hajnoczi [mailto:stefanha@gmail.com]
> Sent: Thursday, August 30, 2012 1:44 PM
> To: John Basila
> Cc: qemu-devel@nongnu.org; Anthony Liguori; Rusty Russell; 
> netfilter@vger.kernel.org
> Subject: Re: Adding support for Stateless Static NAT for TAP devices
> 
> On Thu, Aug 30, 2012 at 10:27 AM, John Basila <jbasila@checkpoint.com> wrote:
>> I have tried NAT and this is why I came up with this feature.
> 
> QEMU's net/tap.c is the wrong place to add NAT code.  The point of tap is to use the host network stack.  If you want userspace networking, use -netdev user or -netdev socket.
> 
> Please look into iptables more.  I have CCed the netfilter mailing list.  The question is:
> 
> The host has several tap interfaces (tap0, tap1, ...) and the machine on the other end of each tap interface uses IP address 10.0.0.2.  So we have:
> 
> tap0 <-> virtual machine #0 (10.0.0.2)
> tap1 <-> virtual machine #1 (10.0.0.2)
> tap2 <-> virtual machine #2 (10.0.0.2)
> 
> Because the virtual machines all use the same static IP address, they cannot communicate with each other or the outside world (they fight over ARP).  We'd like to NAT the tap interfaces:
> 
> tap0 <-> virtual machine #0 (10.0.0.2 NAT to 192.168.0.2)
> tap1 <-> virtual machine #1 (10.0.0.2 NAT to 192.168.0.3)
> tap2 <-> virtual machine #2 (10.0.0.2 NAT to 192.168.0.4)
> 
> This would allow the virtual machines to communicate even though each believes it is 10.0.0.2.
> 
> How can this be done using iptables and friends?

Why do the systems have the same IP? That seems like a broken network config to me.

Regards,
  Dennis


Scanned by Check Point Total Security Gateway.