anyone know how to use clusterip

anyone know how to use clusterip

[Jason Joines]

    I recently noticed the clusterip target extension in my iptables man 
page.  It looks really interesting but I have no idea how how to go 
about trying  it out as I'm not exactly sure what it is supposed to do 
and don't understand exactly what all these options mean.  I've 
consoulted the netfilter website and googled far and wide but haven't 
had any luck finding any examples or more documentation.  Is anyone 
using this thing?

~> man iptables
<--- snip --->
   CLUSTERIP
       This  module allows you to configure a simple cluster of nodes 
that share a certain IP and MAC address without an explicit load
       balancer in front of them.  Connections are statically 
distributed between the nodes in this cluster.

       --new  Create a new ClusterIP.  You always have to set this on 
the first rule for a given ClusterIP.

       --hashmode mode
              Specify the hashing mode.  Has to be one of sourceip, 
sourceip-sourceport, sourceip-sourceport-destport

       --clustermac mac
              Specify the ClusterIP MAC address.  Has to be a link-layer 
multicast address

       --total-nodes num
              Number of total nodes within this cluster.

       --local-node num
              Local node number within this cluster.

       --hash-init rnd
              Specify the random seed used for hash initialization
<--- snip --->
~>


Jason Joines
=================================

Re: anyone know how to use clusterip

[Bryan Waters]

Jason...i've asked the same question already.  It seems that this is new 
enough that absolutely nobody has any experience with it...at least 
nobody on this list.  I'm setting up machines to experiment with this 
now...i'll post my test results to the list.

-bryanw

Jason Joines wrote:
>    I recently noticed the clusterip target extension in my iptables man 
> page.  It looks really interesting but I have no idea how how to go 
> about trying  it out as I'm not exactly sure what it is supposed to do 
> and don't understand exactly what all these options mean.  I've 
> consoulted the netfilter website and googled far and wide but haven't 
> had any luck finding any examples or more documentation.  Is anyone 
> using this thing?
> 
> ~> man iptables
> <--- snip --->
>   CLUSTERIP
>       This  module allows you to configure a simple cluster of nodes 
> that share a certain IP and MAC address without an explicit load
>       balancer in front of them.  Connections are statically distributed 
> between the nodes in this cluster.
> 
>       --new  Create a new ClusterIP.  You always have to set this on the 
> first rule for a given ClusterIP.
> 
>       --hashmode mode
>              Specify the hashing mode.  Has to be one of sourceip, 
> sourceip-sourceport, sourceip-sourceport-destport
> 
>       --clustermac mac
>              Specify the ClusterIP MAC address.  Has to be a link-layer 
> multicast address
> 
>       --total-nodes num
>              Number of total nodes within this cluster.
> 
>       --local-node num
>              Local node number within this cluster.
> 
>       --hash-init rnd
>              Specify the random seed used for hash initialization
> <--- snip --->
> ~>
> 
> 
> Jason Joines
> =================================
> 
> 
> 
> 

Re: anyone know how to use clusterip

[Alexander Stieglecker]

Hi Bryan,

I experimented with CLUSTERIP using 2 nodes.
The traffic is distributed using Multicast-MAC-addresses over N-Nodes (e.g.
webservers). One node decides to ACCEPT the incoming packet. All others DROP
the packet (based on the srcip of the packet and the local node number).
There is also a failover mechanism if one or more nodes die.

My starting commands were:

Node1:
iptables -A INPUT -d 10.0.0.1 -p tcp --dport 80 -j CLUSTERIP --new
--hashmode sourceip --clustermac 01:23:45:67:89:AB --total-nodes 2
--local-node 1

Node2:
iptables -A INPUT -d 10.0.0.1 -p tcp --dport 80 -j CLUSTERIP --new
--hashmode sourceip --clustermac 01:23:45:67:89:AB --total-nodes 2
--local-node 2

It's important that the clustermac parameter is a MULTICAST-MAC-address.
CLUSTERIP replies with this multicast mac to ARP-requests containing ip
10.0.0.1 . This makes the switch to flood the packets on all ports. Each
node decides itself if it wants to drop the incoming packet based in the
hash of the srcip (respectively srcip-srcport/srcip-srcport-destport).
Be sure to enter at least "-d 10.0.0.1" as a standard match.
After you've entered the commands take a look at the proc-fs using

cat /proc/net/ipt_CLUSTERIP/10.0.0.1

This shows you for how many node numbers the local node feels responsible.
You can add a node using the command:

echo "+1" > /proc/net/ipt_CLUSTERIP/10.0.0.1

or remove one using the same command but with "-1"

A daemon like heartbeat could be used to do these commands when one node
fails.

I also discovered some problems:

*Each node replies to an ARP-request with the clustermac but after a while
it sends an ARR-request itself containing its NIC-MAC which destroys the
IP-Clustermac mapping on the clients. (Fixed thos using ARP-Tables)
*Problems using FTP (two TCP-connections)