ClusterIP and MAC NAT

ClusterIP and MAC NAT

[Michele Codutti]

Hello, I'm trying to fix a problem with ClusterIP and Cisco (and also
other brands) routers. The problem is the multicast MAC address that
these routers doesn't "like". They discard any incoming packet with MAC
multicast address to be compliant with RFC1812.
The only documented (by Cisco) workaround is to put a fixed arp entry
with the multicast address that maps the clustered IP in the router.
This method is deprecated here by the network staff so i must find
another solution.
In my opinion a possible solution is to use the existing bridge in front
of the pool of clustered IP hosts with some ebtable rules that
substitute the multicast MAC address with a forged unicast MAC address
for the outgoing packets and substitute the forged unicast MAC address
with the multicast one for the incoming packets.
Suppose that the multicast MAC address is: 01:02:03:04:05:06
and the ClusterIP address is: 10.0.0.100
Now I forge a unicast MAC address for the ClusterIP: 00:02:03:04:05:06
So the rule for the incoming packets is (taken from
http://ebtables.sourceforge.net/examples/basic.html#ex_nat):
ebtables -t nat -A PREROUTING -d 00:02:03:04:05:06 -i incoming-eth1 -j
dnat --to-destination 01:02:03:04:05:06
And similarly the rule for the outgoing packets is:
ebtables -t nat -A POSTROUTING -s 01:02:03:04:05:06 -o outgoing-eth0 -j
snat --to-source 00:02:03:04:05:06
Now the problem is with the arp queries. In need to "NAT" also the
queries substituting the mac address also in the payload of the packet
not only in the header. Can i do that?



Michele Codutti
Centro Servizi Informatici e Telematici (CSIT)
Universita' degli Studi di Udine
via Delle Scienze, 208 - 33100 UDINE
tel +39 0432 558928
fax +39 0432 558911
e-mail: michele.codutti at uniud.it

Re: ClusterIP and MAC NAT

[Grant Taylor]

On 10/26/10 09:30, Michele Codutti wrote:
> In my opinion a possible solution is to use the existing bridge in 
> front of the pool of clustered IP hosts with some ebtable rules that 
> substitute the multicast MAC address with a forged unicast MAC 
> address for the outgoing packets and substitute the forged unicast 
> MAC address with the multicast one for the incoming packets.

This will work.

The only down side that I'm aware of is the possible single point of 
failure that the bridge creates.

Other than that (and possible performance issues if the bridge isn't 
scaled properly) things should work as you want.

> Suppose that the multicast MAC address is: 01:02:03:04:05:06 and the 
> ClusterIP address is: 10.0.0.100 Now I forge a unicast MAC address 
> for the ClusterIP: 00:02:03:04:05:06 So the rule for the incoming 
> packets is (taken from 
> http://ebtables.sourceforge.net/examples/basic.html#ex_nat):

Agreed.

I have an install that is dealing with a cranky switch that can't see 
the same MAC addresses on multiple VLANs where I am doing almost exactly 
this for 30(ish) VLAN interfaces.  It has been in production for five 
years and working great.  (Recently I upgraded the system, carrying the 
old ARPTables / EBTables / IPTables scripts / configs forward.)

> Now the problem is with the arp queries. In need to "NAT" also the 
> queries substituting the mac address also in the payload of the 
> packet not only in the header. Can i do that?

You will need to use ARPTables to help EBTables with the ARP problem.  I 
will go through my backups and see if I can't find an example set of 
rules for you to gander at.

Here's a +1 on what you are wanting to do can be done and does work. 
You just need to look at ARPTables to assist with the ARP specific problem.



Grant. . . .

[SOLVED] Re: ClusterIP and MAC NAT

[Michele Codutti]

> I have an install that is dealing with a cranky switch that can't see 
> the same MAC addresses on multiple VLANs where I am doing almost exactly 
> this for 30(ish) VLAN interfaces.  It has been in production for five 
> years and working great.  (Recently I upgraded the system, carrying the 
> old ARPTables / EBTables / IPTables scripts / configs forward.)
> 
> > Now the problem is with the arp queries. In need to "NAT" also the 
> > queries substituting the mac address also in the payload of the 
> > packet not only in the header. Can i do that?
> 
> You will need to use ARPTables to help EBTables with the ARP problem.  I 
> will go through my backups and see if I can't find an example set of 
> rules for you to gander at.
> 
> Here's a +1 on what you are wanting to do can be done and does work. 
> You just need to look at ARPTables to assist with the ARP specific problem.

Hello everyone today I managed to nat a multicast address of a clustered
ip. So I'm writing to the ML to keep track of the solution.
The servers and the bridges all are Debian Lenny with only packaged
software, the router is a Cisco 7200 VXR.
There are only 3 ebtables rules to do the trick:
   ebtables -t nat -A PREROUTING \
      --in-interface "$OUTERFACE" \
      --protocol arp \
      --arp-opcode Request \
      --arp-ip-dst "$ip" \
      --jump arpreply \
      --arpreply-mac "$UMAC_OUI:" \
      --arpreply-target DROP
   ebtables -t nat -A PREROUTING \
      --in-interface "$OUTERFACE" \
      --destination "$UMAC_OUI:$MAC_EUI" \
      --jump dnat 
      --to-destination "$MMAC_OUI:$MAC_EUI" \
      --dnat-target ACCEPT
   ebtables -t nat -A POSTROUTING \
      --out-interface "$OUTERFACE" \
      --protocol arp --arp-opcode Request \
      --arp-ip-src "$ip" \
      --jump snat \
      --snat-arp \
      --to-source "$UMAC_OUI:$MAC_EUI" \
      --snat-target ACCEPT
Where:
 - $ip is the cluster ip shared by servers;
 - $OUTERFACE is the interface of the bridge connected
   on router ethernet segment;
 - $MMAC_OUI is the multicast OUI part of the MAC address;
 - $MMAC_OUI is the unicast OUI part of the MAC address;
 - $MAC_EUI is the final part of the MAC address.

Special thanks to Grant Taylor.

Re: [SOLVED] Re: ClusterIP and MAC NAT

[Grant Taylor]

On 11/08/10 10:36, Michele Codutti wrote:
> Hello everyone today I managed to nat a multicast address of a clustered
> ip. So I'm writing to the ML to keep track of the solution.

I'm glad that you got things working the way that you wanted them to.

Thank you for replying to the mailing list so that others can search the 
archives in the future.

> The servers and the bridges all are Debian Lenny with only packaged
> software, the router is a Cisco 7200 VXR.

*nod*

> There are only 3 ebtables rules to do the trick:
>    ebtables -t nat -A PREROUTING \
>       --in-interface "$OUTERFACE" \
>       --protocol arp \
>       --arp-opcode Request \
>       --arp-ip-dst "$ip" \
>       --jump arpreply \
>       --arpreply-mac "$UMAC_OUI:" \
>       --arpreply-target DROP
>    ebtables -t nat -A PREROUTING \
>       --in-interface "$OUTERFACE" \
>       --destination "$UMAC_OUI:$MAC_EUI" \
>       --jump dnat 
>       --to-destination "$MMAC_OUI:$MAC_EUI" \
>       --dnat-target ACCEPT
>    ebtables -t nat -A POSTROUTING \
>       --out-interface "$OUTERFACE" \
>       --protocol arp \
>       --arp-opcode Request \
>       --arp-ip-src "$ip" \
>       --jump snat \
>       --snat-arp \
>       --to-source "$UMAC_OUI:$MAC_EUI" \
>       --snat-target ACCEPT
> Where:
>  - $ip is the cluster ip shared by servers;
>  - $OUTERFACE is the interface of the bridge connected
>    on router ethernet segment;
>  - $MMAC_OUI is the multicast OUI part of the MAC address;
>  - $MMAC_OUI is the unicast OUI part of the MAC address;
>  - $MAC_EUI is the final part of the MAC address.

Did you mean $UMAC_OUI for the unicast?

> Special thanks to Grant Taylor.

Id's say you are welcome, but I didn't do any thing other than point in 
a direction and say yes it could be done.



Grant. . . .