[PATCH] Runtime interception method switch

[PATCH] Runtime interception method switch

[Raphael Vallazza]

Hi,

i've just finished the patch for changing the connection interception  
method at runtime (in /proc/sys/net/ipv4/vs/input_hook), it works  
pretty well :) I've tested it with all the seetings and it worked  
without any problems.

It has been made for net-2.6.25 branch. I've also made some minor  
corrections to the "connection interception choice patch". These  
patches have to be appliet together...

Thanks,
Raphael

P.S. i hope my mailer doesn't mess up the patches this time :-/

-- 
:: e n d i a n
:: open source - open minds


#### 0001-IPVS-Add-choice-for-connection-interception-method.patch ####

[PATCH] [IPVS]: Add choice for connection interception method

This patch adds an option to set the position at which IPVS intercepts
incoming connections from Netfilter.

The options are:

1. INPUT (default)
Intercept incoming connections after they have traveled through
the INPUT table, only connections that have the director as
destination address will be processed.

2. FORWARD
Intercept incoming connections after they have traveled through
the INPUT or the FORWARD table. It has the same functionlity of
the "INPUT method", but also processes connections that are
routed through the director, supporting VIP-less setups.

3. PREROUTING
Intercept incoming connections before DNAT and input filtering
has been applied, this enables transparent proxying on realnodes
and localnode.

Signed-off-by: Raphael Vallazza <raphael@endian.com>
---
  net/ipv4/ipvs/Kconfig      |   44 +++++++++++++++++++++++++++++++++++ 
+++++++++
  net/ipv4/ipvs/ip_vs_core.c |   30 +++++++++++++++++++++++++++++-
  2 files changed, 73 insertions(+), 1 deletions(-)

diff --git a/net/ipv4/ipvs/Kconfig b/net/ipv4/ipvs/Kconfig
index 09d0c3f..319f3e8 100644
--- a/net/ipv4/ipvs/Kconfig
+++ b/net/ipv4/ipvs/Kconfig
@@ -24,6 +24,50 @@ menuconfig IP_VS

  if IP_VS

+choice
+	prompt "IPVS connection interception method"
+	default IP_VS_INPUT_LOCAL_IN
+	help
+	  This option sets the position at which IPVS intercepts incoming
+	  connections from Netfilter. If in doubt select 'LOCAL_IN'.
+
+config IP_VS_INPUT_LOCAL_IN
+	bool "INPUT"
+	---help---
+	  Intercept incoming connections after they have traveled through
+	  the INPUT table, only connections that have the director as
+	  destination address will be processed.
+
+	  This method allows to apply packet filtering in the INPUT table
+	  before the connection is intercepted by IPVS.
+
+config IP_VS_INPUT_FORWARD
+	bool "FORWARD"
+	---help---
+	  Intercept incoming connections after they have traveled through
+	  the INPUT or the FORWARD table. It has the same functionlity of
+	  the "INPUT method", but also processes connections that are
+	  routed through the director, supporting VIP-less setups.
+
+	  This method allows to apply packet filtering in the INPUT or
+	  FORWARD table, before the connection is intercepted by IPVS.
+
+config IP_VS_INPUT_PRE_ROUTING
+	bool "PREROUTING"
+	---help---
+	  Intercept incoming connections before DNAT and input filtering
+	  has been applied, this allows transparent proxying on realnodes
+	  and localnode. Incoming connections are intercepted right after
+	  the mangle PREROUTING table and before the nat PREROUTING table,
+	  supporting VIP-less setups.
+
+	  WARNING: This method doesn't apply any packet filtering before
+	  packets are intercepted by IPVS. To filter the connections that
+	  should be intercepted, you have to mark the traffic in the
+	  mangle PREROUTING table.
+
+endchoice
+
  config	IP_VS_DEBUG
  	bool "IP virtual server debugging"
  	---help---
diff --git a/net/ipv4/ipvs/ip_vs_core.c b/net/ipv4/ipvs/ip_vs_core.c
index 963981a..0da4ef6 100644
--- a/net/ipv4/ipvs/ip_vs_core.c
+++ b/net/ipv4/ipvs/ip_vs_core.c
@@ -1026,6 +1026,7 @@ ip_vs_forward_icmp(unsigned int hooknum, struct  
sk_buff *skb,


  static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
+#if defined(CONFIG_IP_VS_INPUT_LOCAL_IN) ||  
defined(CONFIG_IP_VS_INPUT_FORWARD)
  	/* After packet filtering, forward packet through VS/DR, VS/TUN,
  	 * or VS/NAT(change destination), so that filtering rules can be
  	 * applied to IPVS. */
@@ -1036,6 +1037,33 @@ static struct nf_hook_ops ip_vs_ops[]  
__read_mostly = {
  		.hooknum        = NF_INET_LOCAL_IN,
  		.priority       = 100,
  	},
+#endif
+#ifdef CONFIG_IP_VS_INPUT_FORWARD
+	/* Intercept incoming connections after they have traveled through
+	 * the INPUT or the FORWARD table. It has the same functionlity of
+	 * the "INPUT method", but also processes connections that are
+	 * routed through the director, supporting VIP-less setups. */
+	{
+		.hook		= ip_vs_in,
+		.owner		= THIS_MODULE,
+		.pf		= PF_INET,
+		.hooknum        = NF_INET_FORWARD,
+		.priority       = 98,
+	},
+#endif
+#ifdef CONFIG_IP_VS_INPUT_PRE_ROUTING
+	/* Intercept incoming connections before DNAT and input filtering
+	 * has been applied, this enables ransparent proxying on realnodes
+	 * and localnode. Hook right after MANGLE and before NAT_DST.
+	 */
+	{
+		.hook           = ip_vs_in,
+		.owner          = THIS_MODULE,
+		.pf             = PF_INET,
+		.hooknum        = NF_INET_PRE_ROUTING,
+		.priority       = NF_IP_PRI_NAT_DST - 1,
+	},
+#endif
  	/* After packet filtering, change source only for VS/NAT */
  	{
  		.hook		= ip_vs_out,
@@ -1059,7 +1087,7 @@ static struct nf_hook_ops ip_vs_ops[]  
__read_mostly = {
  		.owner		= THIS_MODULE,
  		.pf		= PF_INET,
  		.hooknum        = NF_INET_POST_ROUTING,
-		.priority       = NF_IP_PRI_NAT_SRC-1,
+		.priority       = NF_IP_PRI_NAT_SRC - 1,
  	},
  };

-- 
1.5.3.7


#### 0002-IPVS-Runtime-interception-method-switch.patch ####

[IPVS]: Runtime interception method switch

This patch allows to switch interception method at runtime by changing
the value of /proc/sys/net/ipv4/vs/input_hook with one of the following
values:
0 = INPUT
1 = FORWARD
2 = PREROUTING

Signed-off-by: Raphael Vallazza <raphael@endian.com>
---
  include/net/ip_vs.h        |   15 +++++
  net/ipv4/ipvs/Kconfig      |   10 +++-
  net/ipv4/ipvs/ip_vs_core.c |  141 +++++++++++++++++++++++++++++++++++ 
++-------
  net/ipv4/ipvs/ip_vs_ctl.c  |   43 +++++++++++++
  4 files changed, 186 insertions(+), 23 deletions(-)

diff --git a/include/net/ip_vs.h b/include/net/ip_vs.h
index 56f3c94..6b71e31 100644
--- a/include/net/ip_vs.h
+++ b/include/net/ip_vs.h
@@ -681,6 +681,21 @@ extern void ip_vs_init_hash_table(struct  
list_head *table, int rows);
  #define IP_VS_APP_TYPE_FTP	1

  /*
+ *	IPVS input hook functions
+ */
+enum {
+	IP_VS_INPUT_HOOK_FIRST = -1,
+	IP_VS_INPUT_HOOK_LOCAL_IN,
+	IP_VS_INPUT_HOOK_FORWARD,
+	IP_VS_INPUT_HOOK_PRE_ROUTING,
+	IP_VS_INPUT_HOOK_LAST,
+};
+
+extern int ip_vs_get_input_hook(void);
+extern int ip_vs_register_hooks(int input_hook);
+extern int ip_vs_unregister_hooks(int input_hook);
+
+/*
   *     ip_vs_conn handling functions
   *     (from ip_vs_conn.c)
   */
diff --git a/net/ipv4/ipvs/Kconfig b/net/ipv4/ipvs/Kconfig
index 319f3e8..19217d7 100644
--- a/net/ipv4/ipvs/Kconfig
+++ b/net/ipv4/ipvs/Kconfig
@@ -28,8 +28,14 @@ choice
  	prompt "IPVS connection interception method"
  	default IP_VS_INPUT_LOCAL_IN
  	help
-	  This option sets the position at which IPVS intercepts incoming
-	  connections from Netfilter. If in doubt select 'LOCAL_IN'.
+	  This option selects the default position at which IPVS intercepts
+	  incoming connections from Netfilter. If in doubt select 'INPUT'.
+	
+	  The interception method can be switched at runtime in
+	  /proc/sys/net/ipv4/vs/input_hook with the following values:
+	    0 = INPUT
+	    1 = FORWARD
+	    2 = PREROUTING	

  config IP_VS_INPUT_LOCAL_IN
  	bool "INPUT"
diff --git a/net/ipv4/ipvs/ip_vs_core.c b/net/ipv4/ipvs/ip_vs_core.c
index 0da4ef6..6acfbbd 100644
--- a/net/ipv4/ipvs/ip_vs_core.c
+++ b/net/ipv4/ipvs/ip_vs_core.c
@@ -1024,12 +1024,111 @@ ip_vs_forward_icmp(unsigned int hooknum,  
struct sk_buff *skb,
  	return ip_vs_in_icmp(skb, &r, hooknum);
  }

+/*
+ * Register netfilter hook based on input_hook type
+ */
+
+int ip_vs_register_hooks(int input_hook)
+{
+	int ret;
+	char *hookstr;
+	struct nf_hook_ops *in_hooks;
+	int count;
+
+	IP_VS_DBG(5, "Registering input hooks: %i\n", input_hook);
+
+	switch (input_hook) {
+	case IP_VS_INPUT_HOOK_LOCAL_IN:
+		hookstr = "INPUT";
+		in_hooks = ip_vs_ops_local_in;
+		count = ARRAY_SIZE(ip_vs_ops_local_in);
+		break;
+	case IP_VS_INPUT_HOOK_FORWARD:
+		hookstr = "FORWARD";
+		in_hooks = ip_vs_ops_forward;
+		count = ARRAY_SIZE(ip_vs_ops_forward);
+		break;
+	case IP_VS_INPUT_HOOK_PRE_ROUTING:
+		hookstr = "PREROUTING";
+		in_hooks = ip_vs_ops_pre_routing;
+		count = ARRAY_SIZE(ip_vs_ops_pre_routing);
+		break;
+	default:
+		return -1;
+	}
+
+	ret = nf_register_hooks(in_hooks, count);
+	if (ret < 0) {
+		IP_VS_ERR("Can't register %s hooks.\n", hookstr);
+		return -1;
+	}
+
+	ret = nf_register_hooks(ip_vs_ops_generic,
+				ARRAY_SIZE(ip_vs_ops_generic));
+	if (ret < 0) {
+		nf_unregister_hooks(in_hooks, count);
+		IP_VS_ERR("Can't register generic hooks.\n");
+		return -1;
+	}
+
+	IP_VS_INFO("Registered interception method: %s\n", hookstr);
+	return 0;
+}
+
+/*
+ * Unregister netfilter hook based on input_hook type
+ */

-static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
-#if defined(CONFIG_IP_VS_INPUT_LOCAL_IN) ||  
defined(CONFIG_IP_VS_INPUT_FORWARD)
-	/* After packet filtering, forward packet through VS/DR, VS/TUN,
-	 * or VS/NAT(change destination), so that filtering rules can be
-	 * applied to IPVS. */
+int ip_vs_unregister_hooks(int input_hook)
+{
+	struct nf_hook_ops *in_hooks;
+	int count;
+
+	IP_VS_DBG(5, "Unregistering input hooks: %i\n", input_hook);
+
+	switch (input_hook) {
+	case IP_VS_INPUT_HOOK_LOCAL_IN:
+		in_hooks = ip_vs_ops_local_in;
+		count = ARRAY_SIZE(ip_vs_ops_local_in);
+		break;
+	case IP_VS_INPUT_HOOK_FORWARD:
+		in_hooks = ip_vs_ops_forward;
+		count = ARRAY_SIZE(ip_vs_ops_forward);
+		break;
+	case IP_VS_INPUT_HOOK_PRE_ROUTING:
+		in_hooks = ip_vs_ops_pre_routing;
+		count = ARRAY_SIZE(ip_vs_ops_pre_routing);
+		break;
+	default:
+		return -1;
+	}
+
+	nf_unregister_hooks(in_hooks, count);
+	nf_unregister_hooks(ip_vs_ops_generic, ARRAY_SIZE(ip_vs_ops_generic));
+
+	IP_VS_DBG(5, "Unregistered input hooks.\n");
+	return 0;
+}
+
+
+/* After packet filtering, forward packet through VS/DR, VS/TUN,
+ * or VS/NAT(change destination), so that filtering rules can be
+ * applied to IPVS. */
+static struct nf_hook_ops ip_vs_ops_local_in[] __read_mostly = {
+	{
+		.hook		= ip_vs_in,
+		.owner		= THIS_MODULE,
+		.pf		= PF_INET,
+		.hooknum        = NF_INET_LOCAL_IN,
+		.priority       = 100,
+	},
+};
+
+/* Intercept incoming connections after they have traveled through
+ * the INPUT or the FORWARD table. It has the same functionlity of
+ * the "INPUT method", but also processes connections that are
+ * routed through the director, supporting VIP-less setups. */
+static struct nf_hook_ops ip_vs_ops_forward[] __read_mostly = {
  	{
  		.hook		= ip_vs_in,
  		.owner		= THIS_MODULE,
@@ -1037,12 +1136,6 @@ static struct nf_hook_ops ip_vs_ops[]  
__read_mostly = {
  		.hooknum        = NF_INET_LOCAL_IN,
  		.priority       = 100,
  	},
-#endif
-#ifdef CONFIG_IP_VS_INPUT_FORWARD
-	/* Intercept incoming connections after they have traveled through
-	 * the INPUT or the FORWARD table. It has the same functionlity of
-	 * the "INPUT method", but also processes connections that are
-	 * routed through the director, supporting VIP-less setups. */
  	{
  		.hook		= ip_vs_in,
  		.owner		= THIS_MODULE,
@@ -1050,12 +1143,13 @@ static struct nf_hook_ops ip_vs_ops[]  
__read_mostly = {
  		.hooknum        = NF_INET_FORWARD,
  		.priority       = 98,
  	},
-#endif
-#ifdef CONFIG_IP_VS_INPUT_PRE_ROUTING
-	/* Intercept incoming connections before DNAT and input filtering
-	 * has been applied, this enables ransparent proxying on realnodes
-	 * and localnode. Hook right after MANGLE and before NAT_DST.
-	 */
+};
+
+/* Intercept incoming connections before DNAT and input filtering
+ * has been applied, this enables ransparent proxying on realnodes
+ * and localnode. Hook right after MANGLE and before NAT_DST.
+ */
+static struct nf_hook_ops ip_vs_ops_pre_routing[] __read_mostly = {
  	{
  		.hook           = ip_vs_in,
  		.owner          = THIS_MODULE,
@@ -1063,7 +1157,13 @@ static struct nf_hook_ops ip_vs_ops[]  
__read_mostly = {
  		.hooknum        = NF_INET_PRE_ROUTING,
  		.priority       = NF_IP_PRI_NAT_DST - 1,
  	},
-#endif
+};
+
+/*
+ * Generic Netfilter hooks required for all the input methods
+ */
+
+static struct nf_hook_ops ip_vs_ops_generic[] __read_mostly = {
  	/* After packet filtering, change source only for VS/NAT */
  	{
  		.hook		= ip_vs_out,
@@ -1119,9 +1219,8 @@ static int __init ip_vs_init(void)
  		goto cleanup_app;
  	}

-	ret = nf_register_hooks(ip_vs_ops, ARRAY_SIZE(ip_vs_ops));
+	ret = ip_vs_register_hooks(ip_vs_get_input_hook());
  	if (ret < 0) {
-		IP_VS_ERR("can't register hooks.\n");
  		goto cleanup_conn;
  	}

@@ -1141,7 +1240,7 @@ static int __init ip_vs_init(void)

  static void __exit ip_vs_cleanup(void)
  {
-	nf_unregister_hooks(ip_vs_ops, ARRAY_SIZE(ip_vs_ops));
+	ip_vs_unregister_hooks(ip_vs_get_input_hook());
  	ip_vs_conn_cleanup();
  	ip_vs_app_cleanup();
  	ip_vs_protocol_cleanup();
diff --git a/net/ipv4/ipvs/ip_vs_ctl.c b/net/ipv4/ipvs/ip_vs_ctl.c
index 94c5767..1e05c54 100644
--- a/net/ipv4/ipvs/ip_vs_ctl.c
+++ b/net/ipv4/ipvs/ip_vs_ctl.c
@@ -82,6 +82,15 @@ int sysctl_ip_vs_expire_quiescent_template = 0;
  int sysctl_ip_vs_sync_threshold[2] = { 3, 50 };
  int sysctl_ip_vs_nat_icmp_send = 0;

+#ifdef CONFIG_IP_VS_INPUT_LOCAL_IN
+static int sysctl_ip_vs_input_hook = IP_VS_INPUT_HOOK_LOCAL_IN;
+#endif
+#ifdef CONFIG_IP_VS_INPUT_FORWARD
+static int sysctl_ip_vs_input_hook = IP_VS_INPUT_HOOK_FORWARD;
+#endif
+#ifdef CONFIG_IP_VS_INPUT_PRE_ROUTING
+static int sysctl_ip_vs_input_hook = IP_VS_INPUT_HOOK_PRE_ROUTING;
+#endif

  #ifdef CONFIG_IP_VS_DEBUG
  static int sysctl_ip_vs_debug_level = 0;
@@ -92,6 +101,11 @@ int ip_vs_get_debug_level(void)
  }
  #endif

+int ip_vs_get_input_hook(void)
+{
+	return sysctl_ip_vs_input_hook;
+}
+
  /*
   *	update_defense_level is called from keventd and from sysctl,
   *	so it needs to protect itself from softirqs
@@ -1376,6 +1390,28 @@ static int ip_vs_zero_all(void)
  	return 0;
  }

+static int
+proc_do_input_hook(struct ctl_table *table, int write, struct file  
*filp,
+		   void __user *buffer, size_t *lenp, loff_t *ppos)
+{
+	char *valp = table->data;
+	int oldval = *valp;
+	int rc;
+
+	rc = proc_dointvec(table, write, filp, buffer, lenp, ppos);
+	if (write && (*valp != oldval)) {
+		if ((*valp <= IP_VS_INPUT_HOOK_FIRST) ||
+		    (*valp >= IP_VS_INPUT_HOOK_LAST)) {
+			IP_VS_ERR("Invalid input hook value: %i\n", *valp);
+			*valp = oldval;
+		} else {
+			/* unregister old and register new input hooks */
+			ip_vs_unregister_hooks(oldval);
+			ip_vs_register_hooks(*valp);
+		}
+	}
+	return rc;
+}

  static int
  proc_do_defense_mode(ctl_table *table, int write, struct file * filp,
@@ -1430,6 +1466,13 @@ static struct ctl_table vs_vars[] = {
  		.mode		= 0644,
  		.proc_handler	= &proc_dointvec,
  	},
+	{
+		.procname	= "input_hook",
+		.data		= &sysctl_ip_vs_input_hook,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= &proc_do_input_hook,
+	},
  #ifdef CONFIG_IP_VS_DEBUG
  	{
  		.procname	= "debug_level",
-- 
1.5.3.7

Re: [PATCH] Runtime interception method switch

[Joseph Mack NA3T]

On Sun, 13 Jan 2008, Raphael Vallazza wrote:

> 3. PREROUTING Intercept incoming connections before DNAT 
> and input filtering has been applied, this enables 
> transparent proxying on realnodes and localnode.

Raphael,

 	What's the best way of implementing F5-SNAT? All 
packets must arrive at the realservers with src_addr=DIP. 
Where should ipvs be hooked and where should the iptables 
rules be to NAT the packets?

client: CIP->VIP:80

ipvs on LVS-NAT director: CIP->RIP:80

iptables rules on director (in POSTROUTING?) DIP->RIP:80

realserver: RIP:80->DIP

iptables rules on director RIP:80->CIP

ipvs on LVS-NAT director: VIP:80->CIP

client: gets packet VIP:80->CIP

Thanks 
Joe
-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

Re: [PATCH] Runtime interception method switch

[Raphael Vallazza]

I think this has nothing to with the input method, it's more a problem  
of the *xmit* function. Packets for realservers don't seem to flow  
through the SNAT chain, this way it's not possible to change the  
source IP.

This could probably be implemented either by letting the packets flow  
through the iptables/SNAT (it seems that the patch on http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.non-modified_realservers.html 
  does this), or to implement SNAT in the IPVS/NAT method.

Raphael

Am 13.01.2008 um 18:59 schrieb Joseph Mack NA3T:

> On Sun, 13 Jan 2008, Raphael Vallazza wrote:
>
>> 3. PREROUTING Intercept incoming connections before DNAT and input  
>> filtering has been applied, this enables transparent proxying on  
>> realnodes and localnode.
>
> Raphael,
>
> 	What's the best way of implementing F5-SNAT? All packets must  
> arrive at the realservers with src_addr=DIP. Where should ipvs be  
> hooked and where should the iptables rules be to NAT the packets?
>
> client: CIP->VIP:80
>
> ipvs on LVS-NAT director: CIP->RIP:80
>
> iptables rules on director (in POSTROUTING?) DIP->RIP:80
>
> realserver: RIP:80->DIP
>
> iptables rules on director RIP:80->CIP
>
> ipvs on LVS-NAT director: VIP:80->CIP
>
> client: gets packet VIP:80->CIP
>
> Thanks Joe
> -- 
> Joseph Mack NA3T EME(B,D), FM05lw North Carolina
> jmack (at) wm7d (dot) net - azimuthal equidistant map
> generator at http://www.wm7d.net/azproj.shtml
> Homepage http://www.austintek.com/ It's GNU/Linux!
> -
> To unsubscribe from this list: send the line "unsubscribe lvs-devel"  
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 

:: e n d i a n
:: open source - open minds

:: raphael vallazza
:: phone +39 0471 631763  :: fax +39 0471 631764
:: http://www.endian.com  :: raphael (AT) endian.com

Re: [PATCH] Runtime interception method switch

[Joseph Mack NA3T]

On Mon, 14 Jan 2008, Raphael Vallazza wrote:

> I think this has nothing to with the input method,

I'd assumed that it would be fixed by hooking the FORWARD 
chain because then the packets wouldn't bypass netfilter.

> it's more a problem of the *xmit* function. Packets for 
> realservers don't seem to flow through the SNAT chain, 
> this way it's not possible to change the source IP.

OK

> This could probably be implemented either by letting the 
> packets flow through the iptables/SNAT (it seems that the 
> patch on 
> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.non-modified_realservers.html 
> does this),

can this be put into the standard ipvs?

> or to implement SNAT in the IPVS/NAT method.

can you do either of these?

Thanks Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

Re: [PATCH] Runtime interception method switch

[Simon Horman]

On Sun, Jan 13, 2008 at 04:12:38PM +0100, Raphael Vallazza wrote:
> Hi,
>
> i've just finished the patch for changing the connection interception  
> method at runtime (in /proc/sys/net/ipv4/vs/input_hook), it works pretty 
> well :) I've tested it with all the seetings and it worked without any 
> problems.
>
> It has been made for net-2.6.25 branch. I've also made some minor  
> corrections to the "connection interception choice patch". These patches 
> have to be appliet together...
>
> Thanks,
> Raphael
>
> P.S. i hope my mailer doesn't mess up the patches this time :-/

Unfortunately it did :-(

That asside, the patches look basically good.
I rediffed them and they seem to work fine.
Let me know if you want my version of the patches,
though I have not fixed all the issues that I saw.

Comments in-line.

> -- 
> :: e n d i a n
> :: open source - open minds
>
>
> #### 0001-IPVS-Add-choice-for-connection-interception-method.patch ####
>
> [PATCH] [IPVS]: Add choice for connection interception method
>
> This patch adds an option to set the position at which IPVS intercepts
> incoming connections from Netfilter.
>
> The options are:
>
> 1. INPUT (default)
> Intercept incoming connections after they have traveled through
> the INPUT table, only connections that have the director as
> destination address will be processed.
>
> 2. FORWARD
> Intercept incoming connections after they have traveled through
> the INPUT or the FORWARD table. It has the same functionlity of
> the "INPUT method", but also processes connections that are
> routed through the director, supporting VIP-less setups.
>
> 3. PREROUTING
> Intercept incoming connections before DNAT and input filtering
> has been applied, this enables transparent proxying on realnodes
> and localnode.
>
> Signed-off-by: Raphael Vallazza <raphael@endian.com>
> ---
>  net/ipv4/ipvs/Kconfig      |   44 +++++++++++++++++++++++++++++++++++ 
> +++++++++
>  net/ipv4/ipvs/ip_vs_core.c |   30 +++++++++++++++++++++++++++++-
>  2 files changed, 73 insertions(+), 1 deletions(-)
>
> diff --git a/net/ipv4/ipvs/Kconfig b/net/ipv4/ipvs/Kconfig
> index 09d0c3f..319f3e8 100644
> --- a/net/ipv4/ipvs/Kconfig
> +++ b/net/ipv4/ipvs/Kconfig
> @@ -24,6 +24,50 @@ menuconfig IP_VS
>
>  if IP_VS
>
> +choice
> +	prompt "IPVS connection interception method"
> +	default IP_VS_INPUT_LOCAL_IN
> +	help
> +	  This option sets the position at which IPVS intercepts incoming
> +	  connections from Netfilter. If in doubt select 'LOCAL_IN'.
> +
> +config IP_VS_INPUT_LOCAL_IN
> +	bool "INPUT"
> +	---help---
> +	  Intercept incoming connections after they have traveled through
> +	  the INPUT table, only connections that have the director as
> +	  destination address will be processed.
> +
> +	  This method allows to apply packet filtering in the INPUT table
> +	  before the connection is intercepted by IPVS.
> +
> +config IP_VS_INPUT_FORWARD
> +	bool "FORWARD"
> +	---help---
> +	  Intercept incoming connections after they have traveled through
> +	  the INPUT or the FORWARD table. It has the same functionlity of
> +	  the "INPUT method", but also processes connections that are
> +	  routed through the director, supporting VIP-less setups.
> +
> +	  This method allows to apply packet filtering in the INPUT or
> +	  FORWARD table, before the connection is intercepted by IPVS.
> +
> +config IP_VS_INPUT_PRE_ROUTING
> +	bool "PREROUTING"
> +	---help---
> +	  Intercept incoming connections before DNAT and input filtering
> +	  has been applied, this allows transparent proxying on realnodes
> +	  and localnode. Incoming connections are intercepted right after
> +	  the mangle PREROUTING table and before the nat PREROUTING table,
> +	  supporting VIP-less setups.
> +
> +	  WARNING: This method doesn't apply any packet filtering before
> +	  packets are intercepted by IPVS. To filter the connections that
> +	  should be intercepted, you have to mark the traffic in the
> +	  mangle PREROUTING table.
> +
> +endchoice
> +
>  config	IP_VS_DEBUG
>  	bool "IP virtual server debugging"
>  	---help---
> diff --git a/net/ipv4/ipvs/ip_vs_core.c b/net/ipv4/ipvs/ip_vs_core.c
> index 963981a..0da4ef6 100644
> --- a/net/ipv4/ipvs/ip_vs_core.c
> +++ b/net/ipv4/ipvs/ip_vs_core.c
> @@ -1026,6 +1026,7 @@ ip_vs_forward_icmp(unsigned int hooknum, struct  
> sk_buff *skb,
>
>
>  static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
> +#if defined(CONFIG_IP_VS_INPUT_LOCAL_IN) || defined(CONFIG_IP_VS_INPUT_FORWARD)

Shouldn't this just be

#if defined(CONFIG_IP_VS_INPUT_LOCAL_IN)

>  	/* After packet filtering, forward packet through VS/DR, VS/TUN,
>  	 * or VS/NAT(change destination), so that filtering rules can be
>  	 * applied to IPVS. */
> @@ -1036,6 +1037,33 @@ static struct nf_hook_ops ip_vs_ops[]  
> __read_mostly = {
>  		.hooknum        = NF_INET_LOCAL_IN,
>  		.priority       = 100,
>  	},
> +#endif
> +#ifdef CONFIG_IP_VS_INPUT_FORWARD
> +	/* Intercept incoming connections after they have traveled through
> +	 * the INPUT or the FORWARD table. It has the same functionlity of
> +	 * the "INPUT method", but also processes connections that are
> +	 * routed through the director, supporting VIP-less setups. */
> +	{
> +		.hook		= ip_vs_in,
> +		.owner		= THIS_MODULE,
> +		.pf		= PF_INET,
> +		.hooknum        = NF_INET_FORWARD,
> +		.priority       = 98,
> +	},
> +#endif
> +#ifdef CONFIG_IP_VS_INPUT_PRE_ROUTING
> +	/* Intercept incoming connections before DNAT and input filtering
> +	 * has been applied, this enables ransparent proxying on realnodes
> +	 * and localnode. Hook right after MANGLE and before NAT_DST.
> +	 */
> +	{
> +		.hook           = ip_vs_in,
> +		.owner          = THIS_MODULE,
> +		.pf             = PF_INET,
> +		.hooknum        = NF_INET_PRE_ROUTING,
> +		.priority       = NF_IP_PRI_NAT_DST - 1,
> +	},
> +#endif
>  	/* After packet filtering, change source only for VS/NAT */
>  	{
>  		.hook		= ip_vs_out,
> @@ -1059,7 +1087,7 @@ static struct nf_hook_ops ip_vs_ops[]  
> __read_mostly = {
>  		.owner		= THIS_MODULE,
>  		.pf		= PF_INET,
>  		.hooknum        = NF_INET_POST_ROUTING,
> -		.priority       = NF_IP_PRI_NAT_SRC-1,
> +		.priority       = NF_IP_PRI_NAT_SRC - 1,
>  	},
>  };
>
> -- 
> 1.5.3.7
>
>
> #### 0002-IPVS-Runtime-interception-method-switch.patch ####
>
> [IPVS]: Runtime interception method switch
>
> This patch allows to switch interception method at runtime by changing
> the value of /proc/sys/net/ipv4/vs/input_hook with one of the following
> values:
> 0 = INPUT
> 1 = FORWARD
> 2 = PREROUTING

Could you ass documentation of this to
Documentation/networking/ipvs-sysctl.txt ?

>
> Signed-off-by: Raphael Vallazza <raphael@endian.com>
> ---
>  include/net/ip_vs.h        |   15 +++++
>  net/ipv4/ipvs/Kconfig      |   10 +++-
>  net/ipv4/ipvs/ip_vs_core.c |  141 +++++++++++++++++++++++++++++++++++ 
> ++-------
>  net/ipv4/ipvs/ip_vs_ctl.c  |   43 +++++++++++++
>  4 files changed, 186 insertions(+), 23 deletions(-)
>
> diff --git a/include/net/ip_vs.h b/include/net/ip_vs.h
> index 56f3c94..6b71e31 100644
> --- a/include/net/ip_vs.h
> +++ b/include/net/ip_vs.h
> @@ -681,6 +681,21 @@ extern void ip_vs_init_hash_table(struct list_head 
> *table, int rows);
>  #define IP_VS_APP_TYPE_FTP	1
>
>  /*
> + *	IPVS input hook functions
> + */
> +enum {
> +	IP_VS_INPUT_HOOK_FIRST = -1,
> +	IP_VS_INPUT_HOOK_LOCAL_IN,
> +	IP_VS_INPUT_HOOK_FORWARD,
> +	IP_VS_INPUT_HOOK_PRE_ROUTING,
> +	IP_VS_INPUT_HOOK_LAST,
> +};
> +
> +extern int ip_vs_get_input_hook(void);
> +extern int ip_vs_register_hooks(int input_hook);
> +extern int ip_vs_unregister_hooks(int input_hook);
> +
> +/*
>   *     ip_vs_conn handling functions
>   *     (from ip_vs_conn.c)
>   */
> diff --git a/net/ipv4/ipvs/Kconfig b/net/ipv4/ipvs/Kconfig
> index 319f3e8..19217d7 100644
> --- a/net/ipv4/ipvs/Kconfig
> +++ b/net/ipv4/ipvs/Kconfig
> @@ -28,8 +28,14 @@ choice
>  	prompt "IPVS connection interception method"
>  	default IP_VS_INPUT_LOCAL_IN
>  	help
> -	  This option sets the position at which IPVS intercepts incoming
> -	  connections from Netfilter. If in doubt select 'LOCAL_IN'.
> +	  This option selects the default position at which IPVS intercepts
> +	  incoming connections from Netfilter. If in doubt select 'INPUT'.
> +	
> +	  The interception method can be switched at runtime in
> +	  /proc/sys/net/ipv4/vs/input_hook with the following values:
> +	    0 = INPUT
> +	    1 = FORWARD
> +	    2 = PREROUTING	
>
>  config IP_VS_INPUT_LOCAL_IN
>  	bool "INPUT"
> diff --git a/net/ipv4/ipvs/ip_vs_core.c b/net/ipv4/ipvs/ip_vs_core.c
> index 0da4ef6..6acfbbd 100644
> --- a/net/ipv4/ipvs/ip_vs_core.c
> +++ b/net/ipv4/ipvs/ip_vs_core.c
> @@ -1024,12 +1024,111 @@ ip_vs_forward_icmp(unsigned int hooknum, struct 
> sk_buff *skb,
>  	return ip_vs_in_icmp(skb, &r, hooknum);
>  }
>
> +/*
> + * Register netfilter hook based on input_hook type
> + */
> +
> +int ip_vs_register_hooks(int input_hook)
> +{
> +	int ret;
> +	char *hookstr;
> +	struct nf_hook_ops *in_hooks;

	Just me, but in_hook seems like a better name than in_hooks.

> +	int count;
> +
> +	IP_VS_DBG(5, "Registering input hooks: %i\n", input_hook);
> +
> +	switch (input_hook) {
> +	case IP_VS_INPUT_HOOK_LOCAL_IN:
> +		hookstr = "INPUT";
> +		in_hooks = ip_vs_ops_local_in;
> +		count = ARRAY_SIZE(ip_vs_ops_local_in);
> +		break;
> +	case IP_VS_INPUT_HOOK_FORWARD:
> +		hookstr = "FORWARD";
> +		in_hooks = ip_vs_ops_forward;
> +		count = ARRAY_SIZE(ip_vs_ops_forward);
> +		break;
> +	case IP_VS_INPUT_HOOK_PRE_ROUTING:
> +		hookstr = "PREROUTING";
> +		in_hooks = ip_vs_ops_pre_routing;
> +		count = ARRAY_SIZE(ip_vs_ops_pre_routing);
> +		break;
> +	default:
> +		return -1;
> +	}

Could the calculation of count be moved out of the switch to here
as the following?

	count = ARRAY_SIZE(in_hooks);
> +
> +	ret = nf_register_hooks(in_hooks, count);
> +	if (ret < 0) {
> +		IP_VS_ERR("Can't register %s hooks.\n", hookstr);
> +		return -1;
> +	}
> +
> +	ret = nf_register_hooks(ip_vs_ops_generic,
> +				ARRAY_SIZE(ip_vs_ops_generic));
> +	if (ret < 0) {
> +		nf_unregister_hooks(in_hooks, count);
> +		IP_VS_ERR("Can't register generic hooks.\n");
> +		return -1;
> +	}
> +
> +	IP_VS_INFO("Registered interception method: %s\n", hookstr);
> +	return 0;
> +}
> +
> +/*
> + * Unregister netfilter hook based on input_hook type
> + */
>
> -static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
> -#if defined(CONFIG_IP_VS_INPUT_LOCAL_IN) ||  
> defined(CONFIG_IP_VS_INPUT_FORWARD)
> -	/* After packet filtering, forward packet through VS/DR, VS/TUN,
> -	 * or VS/NAT(change destination), so that filtering rules can be
> -	 * applied to IPVS. */
> +int ip_vs_unregister_hooks(int input_hook)
> +{
> +	struct nf_hook_ops *in_hooks;
> +	int count;
> +
> +	IP_VS_DBG(5, "Unregistering input hooks: %i\n", input_hook);
> +
> +	switch (input_hook) {
> +	case IP_VS_INPUT_HOOK_LOCAL_IN:
> +		in_hooks = ip_vs_ops_local_in;
> +		count = ARRAY_SIZE(ip_vs_ops_local_in);
> +		break;
> +	case IP_VS_INPUT_HOOK_FORWARD:
> +		in_hooks = ip_vs_ops_forward;
> +		count = ARRAY_SIZE(ip_vs_ops_forward);
> +		break;
> +	case IP_VS_INPUT_HOOK_PRE_ROUTING:
> +		in_hooks = ip_vs_ops_pre_routing;
> +		count = ARRAY_SIZE(ip_vs_ops_pre_routing);
> +		break;
> +	default:
> +		return -1;
> +	}

Again, could the calculation of count be moved out of the switch
statement? Perhaps it could just be put directly in the
call to nf_unregister_hooks() ?

	nf_unregister_hooks(in_hooks, ARRAY_SIZE(in_hooks));

> +
> +	nf_unregister_hooks(in_hooks, count);
> +	nf_unregister_hooks(ip_vs_ops_generic, ARRAY_SIZE(ip_vs_ops_generic));
> +
> +	IP_VS_DBG(5, "Unregistered input hooks.\n");
> +	return 0;
> +}

I found that I needed to move the definition of ip_ve_register_hooks()
and ip_vs_unregister_hooks() to below the definition of all
the struct nf_hook_ops, as they are used in these functions.

The hunks below seem a bit messed up
and ip_vs_ips_pre_forward ends up with a duplicate
of the NF_INET_LOCAL_IN fragment from ip_vs_ops_local_in.

> +
> +
> +/* After packet filtering, forward packet through VS/DR, VS/TUN,
> + * or VS/NAT(change destination), so that filtering rules can be
> + * applied to IPVS. */
> +static struct nf_hook_ops ip_vs_ops_local_in[] __read_mostly = {
> +	{
> +		.hook		= ip_vs_in,
> +		.owner		= THIS_MODULE,
> +		.pf		= PF_INET,
> +		.hooknum        = NF_INET_LOCAL_IN,
> +		.priority       = 100,
> +	},
> +};
> +
> +/* Intercept incoming connections after they have traveled through
> + * the INPUT or the FORWARD table. It has the same functionlity of
> + * the "INPUT method", but also processes connections that are
> + * routed through the director, supporting VIP-less setups. */
> +static struct nf_hook_ops ip_vs_ops_forward[] __read_mostly = {
>  	{
>  		.hook		= ip_vs_in,
>  		.owner		= THIS_MODULE,
> @@ -1037,12 +1136,6 @@ static struct nf_hook_ops ip_vs_ops[]  
> __read_mostly = {
>  		.hooknum        = NF_INET_LOCAL_IN,
>  		.priority       = 100,
>  	},
> -#endif
> -#ifdef CONFIG_IP_VS_INPUT_FORWARD
> -	/* Intercept incoming connections after they have traveled through
> -	 * the INPUT or the FORWARD table. It has the same functionlity of
> -	 * the "INPUT method", but also processes connections that are
> -	 * routed through the director, supporting VIP-less setups. */
>  	{
>  		.hook		= ip_vs_in,
>  		.owner		= THIS_MODULE,
> @@ -1050,12 +1143,13 @@ static struct nf_hook_ops ip_vs_ops[]  
> __read_mostly = {
>  		.hooknum        = NF_INET_FORWARD,
>  		.priority       = 98,
>  	},
> -#endif
> -#ifdef CONFIG_IP_VS_INPUT_PRE_ROUTING
> -	/* Intercept incoming connections before DNAT and input filtering
> -	 * has been applied, this enables ransparent proxying on realnodes
> -	 * and localnode. Hook right after MANGLE and before NAT_DST.
> -	 */
> +};
> +
> +/* Intercept incoming connections before DNAT and input filtering
> + * has been applied, this enables ransparent proxying on realnodes
> + * and localnode. Hook right after MANGLE and before NAT_DST.
> + */
> +static struct nf_hook_ops ip_vs_ops_pre_routing[] __read_mostly = {
>  	{
>  		.hook           = ip_vs_in,
>  		.owner          = THIS_MODULE,
> @@ -1063,7 +1157,13 @@ static struct nf_hook_ops ip_vs_ops[]  
> __read_mostly = {
>  		.hooknum        = NF_INET_PRE_ROUTING,
>  		.priority       = NF_IP_PRI_NAT_DST - 1,
>  	},
> -#endif
> +};
> +
> +/*
> + * Generic Netfilter hooks required for all the input methods
> + */
> +
> +static struct nf_hook_ops ip_vs_ops_generic[] __read_mostly = {
>  	/* After packet filtering, change source only for VS/NAT */
>  	{
>  		.hook		= ip_vs_out,
> @@ -1119,9 +1219,8 @@ static int __init ip_vs_init(void)
>  		goto cleanup_app;
>  	}
>
> -	ret = nf_register_hooks(ip_vs_ops, ARRAY_SIZE(ip_vs_ops));
> +	ret = ip_vs_register_hooks(ip_vs_get_input_hook());
>  	if (ret < 0) {
> -		IP_VS_ERR("can't register hooks.\n");
>  		goto cleanup_conn;
>  	}
>
> @@ -1141,7 +1240,7 @@ static int __init ip_vs_init(void)
>
>  static void __exit ip_vs_cleanup(void)
>  {
> -	nf_unregister_hooks(ip_vs_ops, ARRAY_SIZE(ip_vs_ops));
> +	ip_vs_unregister_hooks(ip_vs_get_input_hook());
>  	ip_vs_conn_cleanup();
>  	ip_vs_app_cleanup();
>  	ip_vs_protocol_cleanup();
> diff --git a/net/ipv4/ipvs/ip_vs_ctl.c b/net/ipv4/ipvs/ip_vs_ctl.c
> index 94c5767..1e05c54 100644
> --- a/net/ipv4/ipvs/ip_vs_ctl.c
> +++ b/net/ipv4/ipvs/ip_vs_ctl.c
> @@ -82,6 +82,15 @@ int sysctl_ip_vs_expire_quiescent_template = 0;
>  int sysctl_ip_vs_sync_threshold[2] = { 3, 50 };
>  int sysctl_ip_vs_nat_icmp_send = 0;
>
> +#ifdef CONFIG_IP_VS_INPUT_LOCAL_IN
> +static int sysctl_ip_vs_input_hook = IP_VS_INPUT_HOOK_LOCAL_IN;
> +#endif
> +#ifdef CONFIG_IP_VS_INPUT_FORWARD
> +static int sysctl_ip_vs_input_hook = IP_VS_INPUT_HOOK_FORWARD;
> +#endif
> +#ifdef CONFIG_IP_VS_INPUT_PRE_ROUTING
> +static int sysctl_ip_vs_input_hook = IP_VS_INPUT_HOOK_PRE_ROUTING;
> +#endif
>
>  #ifdef CONFIG_IP_VS_DEBUG
>  static int sysctl_ip_vs_debug_level = 0;
> @@ -92,6 +101,11 @@ int ip_vs_get_debug_level(void)
>  }
>  #endif
>
> +int ip_vs_get_input_hook(void)
> +{
> +	return sysctl_ip_vs_input_hook;
> +}
> +
>  /*
>   *	update_defense_level is called from keventd and from sysctl,
>   *	so it needs to protect itself from softirqs
> @@ -1376,6 +1390,28 @@ static int ip_vs_zero_all(void)
>  	return 0;
>  }
>
> +static int
> +proc_do_input_hook(struct ctl_table *table, int write, struct file  
> *filp,
> +		   void __user *buffer, size_t *lenp, loff_t *ppos)
> +{
> +	char *valp = table->data;
> +	int oldval = *valp;
> +	int rc;
> +
> +	rc = proc_dointvec(table, write, filp, buffer, lenp, ppos);
> +	if (write && (*valp != oldval)) {
> +		if ((*valp <= IP_VS_INPUT_HOOK_FIRST) ||
> +		    (*valp >= IP_VS_INPUT_HOOK_LAST)) {
> +			IP_VS_ERR("Invalid input hook value: %i\n", *valp);
> +			*valp = oldval;
> +		} else {
> +			/* unregister old and register new input hooks */
> +			ip_vs_unregister_hooks(oldval);
> +			ip_vs_register_hooks(*valp);
> +		}
> +	}
> +	return rc;
> +}
>
>  static int
>  proc_do_defense_mode(ctl_table *table, int write, struct file * filp,
> @@ -1430,6 +1466,13 @@ static struct ctl_table vs_vars[] = {
>  		.mode		= 0644,
>  		.proc_handler	= &proc_dointvec,
>  	},
> +	{
> +		.procname	= "input_hook",
> +		.data		= &sysctl_ip_vs_input_hook,
> +		.maxlen		= sizeof(int),
> +		.mode		= 0644,
> +		.proc_handler	= &proc_do_input_hook,
> +	},
>  #ifdef CONFIG_IP_VS_DEBUG
>  	{
>  		.procname	= "debug_level",
> -- 
> 1.5.3.7
>
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe lvs-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Horms

Re: [PATCH] Runtime interception method switch

[Simon Horman]

On Mon, Jan 14, 2008 at 01:42:00AM -0800, Joseph Mack NA3T wrote:
> On Mon, 14 Jan 2008, Raphael Vallazza wrote:
>
>> I think this has nothing to with the input method,
>
> I'd assumed that it would be fixed by hooking the FORWARD chain because 
> then the packets wouldn't bypass netfilter.
>
>> it's more a problem of the *xmit* function. Packets for realservers 
>> don't seem to flow through the SNAT chain, this way it's not possible 
>> to change the source IP.
>
> OK
>
>> This could probably be implemented either by letting the packets flow 
>> through the iptables/SNAT (it seems that the patch on  
>> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.non-modified_realservers.html 
>> does this),
>
> can this be put into the standard ipvs?

In principle yes, though I am a bit wary about what
side-effects it might have.

>> or to implement SNAT in the IPVS/NAT method.
>
> can you do either of these?

-- 
Horms

Re: [PATCH] Runtime interception method switch

[Joseph Mack NA3T]

On Tue, 15 Jan 2008, Simon Horman wrote:

>> can this be put into the standard ipvs?

("this" == F5-SNAT

http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.non-modified_realservers.html#F5_snat

)

> In principle yes, though I am a bit wary about what
> side-effects it might have.
>
>>> or to implement SNAT in the IPVS/NAT method.
>>
>> can you do either of these?

I know I'm not the one doing the coding but since

o we're having a bit of a redesign

o people reasonably expect iptables rules to be able to NAT 
packets coming out of the director going to the realservers 
and it would be nice to restore that functionality for ipvs

o F5-SNAT is useful

this would be a good time to look at adding F5-SNAT.

What side effects might there be? Are they worse than not 
being able to NAT packets emerging from a director?

Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

Re: [PATCH] Runtime interception method switch

[Simon Horman]

On Tue, Jan 15, 2008 at 05:13:14AM -0800, Joseph Mack NA3T wrote:
> On Tue, 15 Jan 2008, Simon Horman wrote:
>
>>> can this be put into the standard ipvs?
>
> ("this" == F5-SNAT
>
> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.non-modified_realservers.html#F5_snat
>
> )
>
>> In principle yes, though I am a bit wary about what
>> side-effects it might have.
>>
>>>> or to implement SNAT in the IPVS/NAT method.
>>>
>>> can you do either of these?
>
> I know I'm not the one doing the coding but since
>
> o we're having a bit of a redesign
>
> o people reasonably expect iptables rules to be able to NAT packets 
> coming out of the director going to the realservers and it would be nice 
> to restore that functionality for ipvs
>
> o F5-SNAT is useful
>
> this would be a good time to look at adding F5-SNAT.
>
> What side effects might there be? Are they worse than not being able to 
> NAT packets emerging from a director?

I'm not sure, and thats what concerns me.

For starters could we clarify that the patch in question is the
following one by Janusz Krzysztofik?

Also can I clarify that the aim is to be able to SNAT LVS-DR
connections (and if possible LVS-NAT and LVS-TUN)?
Or is the aim to add a new method, LVS-FULL-NAT?

--- linux-source-2.6.17-2-e49_9.200610211740/net/ipv4/ipvs/ip_vs_core.c.orig	2006-06-18 03:49:35.000000000 +0200
+++ linux-source-2.6.17-2-e49_9.200610211740/net/ipv4/ipvs/ip_vs_core.c	2006-10-21 21:38:20.000000000 +0200
@@ -672,6 +672,9 @@ static int ip_vs_out_icmp(struct sk_buff
 	if (!cp)
 		return NF_ACCEPT;
 
+	if (IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_DROUTE)
+		return NF_ACCEPT;
+
 	verdict = NF_DROP;
 
 	if (IP_VS_FWD_METHOD(cp) != 0) {
@@ -801,6 +804,9 @@ ip_vs_out(unsigned int hooknum, struct s
 		return NF_ACCEPT;
 	}
 
+	if (IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_DROUTE)
+		return NF_ACCEPT;
+
 	IP_VS_DBG_PKT(11, pp, skb, 0, "Outgoing packet");
 
 	if (!ip_vs_make_skb_writable(pskb, ihl))
--- linux-source-2.6.17-2-e49_9.200610211740/net/ipv4/ipvs/ip_vs_xmit.c.orig	2006-06-18 03:49:35.000000000 +0200
+++ linux-source-2.6.17-2-e49_9.200610211740/net/ipv4/ipvs/ip_vs_xmit.c	2006-10-21 21:22:56.000000000 +0200
@@ -127,7 +127,6 @@ ip_vs_dst_reset(struct ip_vs_dest *dest)
 
 #define IP_VS_XMIT(skb, rt)				\
 do {							\
-	(skb)->ipvs_property = 1;			\
 	(skb)->ip_summed = CHECKSUM_NONE;		\
 	NF_HOOK(PF_INET, NF_IP_LOCAL_OUT, (skb), NULL,	\
 		(rt)->u.dst.dev, dst_output);		\
@@ -278,6 +277,7 @@ ip_vs_nat_xmit(struct sk_buff *skb, stru
 	/* Another hack: avoid icmp_send in ip_fragment */
 	skb->local_df = 1;
 
+	skb->ipvs_property = 1;
 	IP_VS_XMIT(skb, rt);
 
 	LeaveFunction(10);
@@ -411,6 +411,7 @@ ip_vs_tunnel_xmit(struct sk_buff *skb, s
 	/* Another hack: avoid icmp_send in ip_fragment */
 	skb->local_df = 1;
 
+	skb->ipvs_property = 1;
 	IP_VS_XMIT(skb, rt);
 
 	LeaveFunction(10);
@@ -542,6 +543,7 @@ ip_vs_icmp_xmit(struct sk_buff *skb, str
 	/* Another hack: avoid icmp_send in ip_fragment */
 	skb->local_df = 1;
 
+	skb->ipvs_property = 1;
 	IP_VS_XMIT(skb, rt);
 
 	rc = NF_STOLEN;

Re: [PATCH] Runtime interception method switch

[Janusz Krzysztofik]

Simon Horman wrote:
> On Tue, Jan 15, 2008 at 05:13:14AM -0800, Joseph Mack NA3T wrote:
>> What side effects might there be? Are they worse than not being able to 
>> NAT packets emerging from a director?
> 
> I'm not sure, and thats what concerns me.
> 
> For starters could we clarify that the patch in question is the
> following one by Janusz Krzysztofik?
> 
> Also can I clarify that the aim is to be able to SNAT LVS-DR
> connections ...

Hi,

I can confirm what I have already said before: this patch works for me 
(now for over two years) without any unexpected side effects. The only 
side effect I can see is that all LVS-DR incoming packets go through 
conntrack, even if you do not intend to SNAT them. That could present 
excessive load on ancient hardware, but one can just unload conntrack 
modules, or turn connection tracking off for specific traffic with 
PREROUTING raw hook rules.
However, I think this patch should be considered, if at all, as a 
temporary solution.

Thanks,
Janusz

PS. I am still busy with a different project, but subscribed to lvs-devel.

Re: [PATCH] Runtime interception method switch

[Raphael Vallazza]

[-- Attachment #1: Type: text/plain, Size: 172 bytes --]

I've added the documentation for the input_hook switch, and attached  
both patches for net-2.6.25 (i hope the mailer doesn't mess up things  
this time :).

Bye,
Raphael


[-- Attachment #2: 0001-IPVS-Add-choice-for-connection-interception-method.patch --]
[-- Type: application/octet-stream, Size: 5226 bytes --]

From 31be5b549c946579e322b20d2c388fae0c221bb9 Mon Sep 17 00:00:00 2001
From: Raphael Vallazza <raphael@endian.com>
Date: Sun, 13 Jan 2008 09:29:53 +0100
Subject: [PATCH] [IPVS] Add choice for connection interception method

This patch adds an option to set the position at which IPVS intercepts
incoming connections from Netfilter.

The options are:

1. INPUT (default)
Intercept incoming connections after they have traveled through
the INPUT table, only connections that have the director as
destination address will be processed.

2. FORWARD
Intercept incoming connections after they have traveled through
the INPUT or the FORWARD table. It has the same functionlity of
the "INPUT method", but also processes connections that are
routed through the director, supporting VIP-less setups.

3. PREROUTING
Intercept incoming connections before DNAT and input filtering
has been applied, this enables transparent proxying on realnodes
and localnode.

Signed-off-by: Raphael Vallazza <raphael@endian.com>
---
 net/ipv4/ipvs/Kconfig      |   44 ++++++++++++++++++++++++++++++++++++++++++++
 net/ipv4/ipvs/ip_vs_core.c |   30 +++++++++++++++++++++++++++++-
 2 files changed, 73 insertions(+), 1 deletions(-)

diff --git a/net/ipv4/ipvs/Kconfig b/net/ipv4/ipvs/Kconfig
index 09d0c3f..319f3e8 100644
--- a/net/ipv4/ipvs/Kconfig
+++ b/net/ipv4/ipvs/Kconfig
@@ -24,6 +24,50 @@ menuconfig IP_VS
 
 if IP_VS
 
+choice
+	prompt "IPVS connection interception method"
+	default IP_VS_INPUT_LOCAL_IN
+	help
+	  This option sets the position at which IPVS intercepts incoming
+	  connections from Netfilter. If in doubt select 'LOCAL_IN'.
+
+config IP_VS_INPUT_LOCAL_IN
+	bool "INPUT"
+	---help---
+	  Intercept incoming connections after they have traveled through
+	  the INPUT table, only connections that have the director as
+	  destination address will be processed.
+
+	  This method allows to apply packet filtering in the INPUT table
+	  before the connection is intercepted by IPVS.
+
+config IP_VS_INPUT_FORWARD
+	bool "FORWARD"
+	---help---
+	  Intercept incoming connections after they have traveled through
+	  the INPUT or the FORWARD table. It has the same functionlity of
+	  the "INPUT method", but also processes connections that are
+	  routed through the director, supporting VIP-less setups.
+
+	  This method allows to apply packet filtering in the INPUT or
+	  FORWARD table, before the connection is intercepted by IPVS.
+
+config IP_VS_INPUT_PRE_ROUTING
+	bool "PREROUTING"
+	---help---
+	  Intercept incoming connections before DNAT and input filtering
+	  has been applied, this allows transparent proxying on realnodes
+	  and localnode. Incoming connections are intercepted right after
+	  the mangle PREROUTING table and before the nat PREROUTING table,
+	  supporting VIP-less setups.
+
+	  WARNING: This method doesn't apply any packet filtering before
+	  packets are intercepted by IPVS. To filter the connections that
+	  should be intercepted, you have to mark the traffic in the
+	  mangle PREROUTING table.
+
+endchoice
+
 config	IP_VS_DEBUG
 	bool "IP virtual server debugging"
 	---help---
diff --git a/net/ipv4/ipvs/ip_vs_core.c b/net/ipv4/ipvs/ip_vs_core.c
index 963981a..0da4ef6 100644
--- a/net/ipv4/ipvs/ip_vs_core.c
+++ b/net/ipv4/ipvs/ip_vs_core.c
@@ -1026,6 +1026,7 @@ ip_vs_forward_icmp(unsigned int hooknum, struct sk_buff *skb,
 
 
 static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
+#if defined(CONFIG_IP_VS_INPUT_LOCAL_IN) || defined(CONFIG_IP_VS_INPUT_FORWARD)
 	/* After packet filtering, forward packet through VS/DR, VS/TUN,
 	 * or VS/NAT(change destination), so that filtering rules can be
 	 * applied to IPVS. */
@@ -1036,6 +1037,33 @@ static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
 		.hooknum        = NF_INET_LOCAL_IN,
 		.priority       = 100,
 	},
+#endif
+#ifdef CONFIG_IP_VS_INPUT_FORWARD
+	/* Intercept incoming connections after they have traveled through
+	 * the INPUT or the FORWARD table. It has the same functionlity of
+	 * the "INPUT method", but also processes connections that are
+	 * routed through the director, supporting VIP-less setups. */
+	{
+		.hook		= ip_vs_in,
+		.owner		= THIS_MODULE,
+		.pf		= PF_INET,
+		.hooknum        = NF_INET_FORWARD,
+		.priority       = 98,
+	},
+#endif
+#ifdef CONFIG_IP_VS_INPUT_PRE_ROUTING
+	/* Intercept incoming connections before DNAT and input filtering
+	 * has been applied, this enables ransparent proxying on realnodes
+	 * and localnode. Hook right after MANGLE and before NAT_DST.
+	 */
+	{
+		.hook           = ip_vs_in,
+		.owner          = THIS_MODULE,
+		.pf             = PF_INET,
+		.hooknum        = NF_INET_PRE_ROUTING,
+		.priority       = NF_IP_PRI_NAT_DST - 1,
+	},
+#endif
 	/* After packet filtering, change source only for VS/NAT */
 	{
 		.hook		= ip_vs_out,
@@ -1059,7 +1087,7 @@ static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
 		.owner		= THIS_MODULE,
 		.pf		= PF_INET,
 		.hooknum        = NF_INET_POST_ROUTING,
-		.priority       = NF_IP_PRI_NAT_SRC-1,
+		.priority       = NF_IP_PRI_NAT_SRC - 1,
 	},
 };
 
-- 
1.5.3.7


[-- Attachment #3: 0002-IPVS-Runtime-interception-method-switch.patch --]
[-- Type: application/octet-stream, Size: 11981 bytes --]

From 918801f6b9233ed26ffe8cd0a4ea1ebdb4fc588b Mon Sep 17 00:00:00 2001
From: Raphael Vallazza <raphael@endian.com>
Date: Wed, 16 Jan 2008 12:23:59 +0100
Subject: [PATCH] [IPVS]: Runtime interception method switch

    This patch allows to switch interception method at runtime by changing
    the value of /proc/sys/net/ipv4/vs/input_hook with one of the following
    values:
    0 	- INPUT (default)
    1	- FORWARD
    2	- PREROUTING

Signed-off-by: Raphael Vallazza <raphael@endian.com>
---
 Documentation/networking/ipvs-sysctl.txt |   27 ++++++
 include/net/ip_vs.h                      |   15 +++
 net/ipv4/ipvs/Kconfig                    |   10 ++-
 net/ipv4/ipvs/ip_vs_core.c               |  141 +++++++++++++++++++++++++-----
 net/ipv4/ipvs/ip_vs_ctl.c                |   43 +++++++++
 5 files changed, 213 insertions(+), 23 deletions(-)

diff --git a/Documentation/networking/ipvs-sysctl.txt b/Documentation/networking/ipvs-sysctl.txt
index 4ccdbca..3f15b8f 100644
--- a/Documentation/networking/ipvs-sysctl.txt
+++ b/Documentation/networking/ipvs-sysctl.txt
@@ -112,6 +112,33 @@ expire_quiescent_template - BOOLEAN
 	persistence template if it is to be used to schedule a new
 	connection and the destination server is quiescent.
 
+input_hook - INTEGER
+	0 	- INPUT (default)
+	1	- FORWARD
+	2	- PREROUTING
+	
+	This switch sets the interception method used by IPVS for
+	intercepting incoming connections.
+
+	INPUT - Intercept incoming connections after they have traveled
+	through the INPUT table, only connections that have the director as
+	destination address will be processed.
+
+	FORWARD - Intercept incoming connections after they have traveled
+	through the INPUT or the FORWARD table. It has the same functionlity
+	of the "INPUT method", but also processes connections that are
+	routed through the director, supporting VIP-less setups.
+
+	PREROUTING - Intercept incoming connections before DNAT and input
+	filtering has been applied, this allows transparent proxying on
+	realnodes and localnode. Incoming connections are intercepted right
+	after the mangle PREROUTING table and before the nat PREROUTING
+	table, supporting VIP-less setups.
+	WARNING: This method doesn't apply any packet filtering before
+	packets are intercepted by IPVS. To filter the connections that
+	should be intercepted, you have to mark the traffic in the
+	mangle PREROUTING table.
+
 nat_icmp_send - BOOLEAN
         0 - disabled (default)
         not 0 - enabled
diff --git a/include/net/ip_vs.h b/include/net/ip_vs.h
index 56f3c94..6b71e31 100644
--- a/include/net/ip_vs.h
+++ b/include/net/ip_vs.h
@@ -681,6 +681,21 @@ extern void ip_vs_init_hash_table(struct list_head *table, int rows);
 #define IP_VS_APP_TYPE_FTP	1
 
 /*
+ *	IPVS input hook functions
+ */
+enum {
+	IP_VS_INPUT_HOOK_FIRST = -1,
+	IP_VS_INPUT_HOOK_LOCAL_IN,
+	IP_VS_INPUT_HOOK_FORWARD,
+	IP_VS_INPUT_HOOK_PRE_ROUTING,
+	IP_VS_INPUT_HOOK_LAST,
+};
+
+extern int ip_vs_get_input_hook(void);
+extern int ip_vs_register_hooks(int input_hook);
+extern int ip_vs_unregister_hooks(int input_hook);
+
+/*
  *     ip_vs_conn handling functions
  *     (from ip_vs_conn.c)
  */
diff --git a/net/ipv4/ipvs/Kconfig b/net/ipv4/ipvs/Kconfig
index 319f3e8..19217d7 100644
--- a/net/ipv4/ipvs/Kconfig
+++ b/net/ipv4/ipvs/Kconfig
@@ -28,8 +28,14 @@ choice
 	prompt "IPVS connection interception method"
 	default IP_VS_INPUT_LOCAL_IN
 	help
-	  This option sets the position at which IPVS intercepts incoming
-	  connections from Netfilter. If in doubt select 'LOCAL_IN'.
+	  This option selects the default position at which IPVS intercepts
+	  incoming connections from Netfilter. If in doubt select 'INPUT'.
+	
+	  The interception method can be switched at runtime in 
+	  /proc/sys/net/ipv4/vs/input_hook with the following values:
+	    0 = INPUT
+	    1 = FORWARD
+	    2 = PREROUTING	  
 
 config IP_VS_INPUT_LOCAL_IN
 	bool "INPUT"
diff --git a/net/ipv4/ipvs/ip_vs_core.c b/net/ipv4/ipvs/ip_vs_core.c
index 0da4ef6..6acfbbd 100644
--- a/net/ipv4/ipvs/ip_vs_core.c
+++ b/net/ipv4/ipvs/ip_vs_core.c
@@ -1024,12 +1024,111 @@ ip_vs_forward_icmp(unsigned int hooknum, struct sk_buff *skb,
 	return ip_vs_in_icmp(skb, &r, hooknum);
 }
 
+/*
+ * Register netfilter hook based on input_hook type
+ */
+
+int ip_vs_register_hooks(int input_hook)
+{
+	int ret;
+	char *hookstr;
+	struct nf_hook_ops *in_hooks;
+	int count;
+
+	IP_VS_DBG(5, "Registering input hooks: %i\n", input_hook);
+
+	switch (input_hook) {
+	case IP_VS_INPUT_HOOK_LOCAL_IN:
+		hookstr = "INPUT";
+		in_hooks = ip_vs_ops_local_in;
+		count = ARRAY_SIZE(ip_vs_ops_local_in);
+		break;
+	case IP_VS_INPUT_HOOK_FORWARD:
+		hookstr = "FORWARD";
+		in_hooks = ip_vs_ops_forward;
+		count = ARRAY_SIZE(ip_vs_ops_forward);
+		break;
+	case IP_VS_INPUT_HOOK_PRE_ROUTING:
+		hookstr = "PREROUTING";
+		in_hooks = ip_vs_ops_pre_routing;
+		count = ARRAY_SIZE(ip_vs_ops_pre_routing);
+		break;
+	default:
+		return -1;
+	}
+
+	ret = nf_register_hooks(in_hooks, count);
+	if (ret < 0) {
+		IP_VS_ERR("Can't register %s hooks.\n", hookstr);
+		return -1;
+	}
+
+	ret = nf_register_hooks(ip_vs_ops_generic,
+				ARRAY_SIZE(ip_vs_ops_generic));
+	if (ret < 0) {
+		nf_unregister_hooks(in_hooks, count);
+		IP_VS_ERR("Can't register generic hooks.\n");
+		return -1;
+	}
+
+	IP_VS_INFO("Registered interception method: %s\n", hookstr);
+	return 0;
+}
+
+/*
+ * Unregister netfilter hook based on input_hook type
+ */
 
-static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
-#if defined(CONFIG_IP_VS_INPUT_LOCAL_IN) || defined(CONFIG_IP_VS_INPUT_FORWARD)
-	/* After packet filtering, forward packet through VS/DR, VS/TUN,
-	 * or VS/NAT(change destination), so that filtering rules can be
-	 * applied to IPVS. */
+int ip_vs_unregister_hooks(int input_hook)
+{
+	struct nf_hook_ops *in_hooks;
+	int count;
+
+	IP_VS_DBG(5, "Unregistering input hooks: %i\n", input_hook);
+
+	switch (input_hook) {
+	case IP_VS_INPUT_HOOK_LOCAL_IN:
+		in_hooks = ip_vs_ops_local_in;
+		count = ARRAY_SIZE(ip_vs_ops_local_in);
+		break;
+	case IP_VS_INPUT_HOOK_FORWARD:
+		in_hooks = ip_vs_ops_forward;
+		count = ARRAY_SIZE(ip_vs_ops_forward);
+		break;
+	case IP_VS_INPUT_HOOK_PRE_ROUTING:
+		in_hooks = ip_vs_ops_pre_routing;
+		count = ARRAY_SIZE(ip_vs_ops_pre_routing);
+		break;
+	default:
+		return -1;
+	}
+
+	nf_unregister_hooks(in_hooks, count);
+	nf_unregister_hooks(ip_vs_ops_generic, ARRAY_SIZE(ip_vs_ops_generic));
+
+	IP_VS_DBG(5, "Unregistered input hooks.\n");
+	return 0;
+}
+
+
+/* After packet filtering, forward packet through VS/DR, VS/TUN,
+ * or VS/NAT(change destination), so that filtering rules can be
+ * applied to IPVS. */
+static struct nf_hook_ops ip_vs_ops_local_in[] __read_mostly = {
+	{
+		.hook		= ip_vs_in,
+		.owner		= THIS_MODULE,
+		.pf		= PF_INET,
+		.hooknum        = NF_INET_LOCAL_IN,
+		.priority       = 100,
+	},
+};
+
+/* Intercept incoming connections after they have traveled through
+ * the INPUT or the FORWARD table. It has the same functionlity of
+ * the "INPUT method", but also processes connections that are
+ * routed through the director, supporting VIP-less setups. */
+static struct nf_hook_ops ip_vs_ops_forward[] __read_mostly = {
 	{
 		.hook		= ip_vs_in,
 		.owner		= THIS_MODULE,
@@ -1037,12 +1136,6 @@ static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
 		.hooknum        = NF_INET_LOCAL_IN,
 		.priority       = 100,
 	},
-#endif
-#ifdef CONFIG_IP_VS_INPUT_FORWARD
-	/* Intercept incoming connections after they have traveled through
-	 * the INPUT or the FORWARD table. It has the same functionlity of
-	 * the "INPUT method", but also processes connections that are
-	 * routed through the director, supporting VIP-less setups. */
 	{
 		.hook		= ip_vs_in,
 		.owner		= THIS_MODULE,
@@ -1050,12 +1143,13 @@ static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
 		.hooknum        = NF_INET_FORWARD,
 		.priority       = 98,
 	},
-#endif
-#ifdef CONFIG_IP_VS_INPUT_PRE_ROUTING
-	/* Intercept incoming connections before DNAT and input filtering
-	 * has been applied, this enables ransparent proxying on realnodes
-	 * and localnode. Hook right after MANGLE and before NAT_DST.
-	 */
+};
+
+/* Intercept incoming connections before DNAT and input filtering
+ * has been applied, this enables ransparent proxying on realnodes
+ * and localnode. Hook right after MANGLE and before NAT_DST.
+ */
+static struct nf_hook_ops ip_vs_ops_pre_routing[] __read_mostly = {
 	{
 		.hook           = ip_vs_in,
 		.owner          = THIS_MODULE,
@@ -1063,7 +1157,13 @@ static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
 		.hooknum        = NF_INET_PRE_ROUTING,
 		.priority       = NF_IP_PRI_NAT_DST - 1,
 	},
-#endif
+};
+
+/*
+ * Generic Netfilter hooks required for all the input methods
+ */
+
+static struct nf_hook_ops ip_vs_ops_generic[] __read_mostly = {
 	/* After packet filtering, change source only for VS/NAT */
 	{
 		.hook		= ip_vs_out,
@@ -1119,9 +1219,8 @@ static int __init ip_vs_init(void)
 		goto cleanup_app;
 	}
 
-	ret = nf_register_hooks(ip_vs_ops, ARRAY_SIZE(ip_vs_ops));
+	ret = ip_vs_register_hooks(ip_vs_get_input_hook());
 	if (ret < 0) {
-		IP_VS_ERR("can't register hooks.\n");
 		goto cleanup_conn;
 	}
 
@@ -1141,7 +1240,7 @@ static int __init ip_vs_init(void)
 
 static void __exit ip_vs_cleanup(void)
 {
-	nf_unregister_hooks(ip_vs_ops, ARRAY_SIZE(ip_vs_ops));
+	ip_vs_unregister_hooks(ip_vs_get_input_hook());
 	ip_vs_conn_cleanup();
 	ip_vs_app_cleanup();
 	ip_vs_protocol_cleanup();
diff --git a/net/ipv4/ipvs/ip_vs_ctl.c b/net/ipv4/ipvs/ip_vs_ctl.c
index 94c5767..1e05c54 100644
--- a/net/ipv4/ipvs/ip_vs_ctl.c
+++ b/net/ipv4/ipvs/ip_vs_ctl.c
@@ -82,6 +82,15 @@ int sysctl_ip_vs_expire_quiescent_template = 0;
 int sysctl_ip_vs_sync_threshold[2] = { 3, 50 };
 int sysctl_ip_vs_nat_icmp_send = 0;
 
+#ifdef CONFIG_IP_VS_INPUT_LOCAL_IN
+static int sysctl_ip_vs_input_hook = IP_VS_INPUT_HOOK_LOCAL_IN;
+#endif
+#ifdef CONFIG_IP_VS_INPUT_FORWARD
+static int sysctl_ip_vs_input_hook = IP_VS_INPUT_HOOK_FORWARD;
+#endif
+#ifdef CONFIG_IP_VS_INPUT_PRE_ROUTING
+static int sysctl_ip_vs_input_hook = IP_VS_INPUT_HOOK_PRE_ROUTING;
+#endif
 
 #ifdef CONFIG_IP_VS_DEBUG
 static int sysctl_ip_vs_debug_level = 0;
@@ -92,6 +101,11 @@ int ip_vs_get_debug_level(void)
 }
 #endif
 
+int ip_vs_get_input_hook(void)
+{
+	return sysctl_ip_vs_input_hook;
+}
+
 /*
  *	update_defense_level is called from keventd and from sysctl,
  *	so it needs to protect itself from softirqs
@@ -1376,6 +1390,28 @@ static int ip_vs_zero_all(void)
 	return 0;
 }
 
+static int
+proc_do_input_hook(struct ctl_table *table, int write, struct file *filp,
+		   void __user *buffer, size_t *lenp, loff_t *ppos)
+{
+	char *valp = table->data;
+	int oldval = *valp;
+	int rc;
+
+	rc = proc_dointvec(table, write, filp, buffer, lenp, ppos);
+	if (write && (*valp != oldval)) {
+		if ((*valp <= IP_VS_INPUT_HOOK_FIRST) ||
+		    (*valp >= IP_VS_INPUT_HOOK_LAST)) {
+			IP_VS_ERR("Invalid input hook value: %i\n", *valp);
+			*valp = oldval;
+		} else {
+			/* unregister old and register new input hooks */
+			ip_vs_unregister_hooks(oldval);
+			ip_vs_register_hooks(*valp);
+		}
+	}
+	return rc;
+}
 
 static int
 proc_do_defense_mode(ctl_table *table, int write, struct file * filp,
@@ -1430,6 +1466,13 @@ static struct ctl_table vs_vars[] = {
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
+	{
+		.procname	= "input_hook",
+		.data		= &sysctl_ip_vs_input_hook,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= &proc_do_input_hook,
+	},
 #ifdef CONFIG_IP_VS_DEBUG
 	{
 		.procname	= "debug_level",
-- 
1.5.3.7


[-- Attachment #4: Type: text/plain, Size: 1553 bytes --]


Am 16.01.2008 um 12:16 schrieb Janusz Krzysztofik:

> Simon Horman wrote:
>> On Tue, Jan 15, 2008 at 05:13:14AM -0800, Joseph Mack NA3T wrote:
>>> What side effects might there be? Are they worse than not being  
>>> able to NAT packets emerging from a director?
>> I'm not sure, and thats what concerns me.
>> For starters could we clarify that the patch in question is the
>> following one by Janusz Krzysztofik?
>> Also can I clarify that the aim is to be able to SNAT LVS-DR
>> connections ...
>
> Hi,
>
> I can confirm what I have already said before: this patch works for  
> me (now for over two years) without any unexpected side effects. The  
> only side effect I can see is that all LVS-DR incoming packets go  
> through conntrack, even if you do not intend to SNAT them. That  
> could present excessive load on ancient hardware, but one can just  
> unload conntrack modules, or turn connection tracking off for  
> specific traffic with PREROUTING raw hook rules.
> However, I think this patch should be considered, if at all, as a  
> temporary solution.
>
> Thanks,
> Janusz
>
> PS. I am still busy with a different project, but subscribed to lvs- 
> devel.
> -
> To unsubscribe from this list: send the line "unsubscribe lvs-devel"  
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 

:: e n d i a n
:: open source - open minds

:: raphael vallazza
:: phone +39 0471 631763  :: fax +39 0471 631764
:: http://www.endian.com  :: raphael (AT) endian.com

Re: [PATCH] Runtime interception method switch

[Joseph Mack NA3T]

On Wed, 16 Jan 2008, Simon Horman wrote:

> For starters could we clarify that the patch in question 
> is the following one by Janusz Krzysztofik?

I see Janusz has replied to this.

I had assumed that if Raphael could output the packets to 
the right spot (before POSTROUTING on the inbound 
direction?) that iptables could handle the NAT'ing and no 
extra ipvs code would be neccessary.

What I didn't know was the original reason the packets were 
output to a place where iptables couldn't manipulate them. 
Was this for speed? to get ipvs to work at all? If for 
speed, the director has always been limited by wirespeed, 
not by anything in ipvs, so any increase in latency through 
ipvs may not be seen.

> Also can I clarify that the aim is to be able to SNAT LVS-DR
> connections

I didn't realise Janusz was SNAT'ing LVS-DR.

> (and if possible LVS-NAT and LVS-TUN)?
> Or is the aim to add a new method, LVS-FULL-NAT?

What the users want is to be able to put unmodified servers 
behind a director - they can't even change the default gw. 
The only thing they can change is the RIP. So the servers 
would have to be realservers behind an LVS-NAT director 
which is outputting packets with src_addr=DIP, ie the 
realservers see connect requests only from the DIP. I'd 
assumed the director would be running a new version of 
standard LVS-NAT, with iptables doing the SNAT in 
POSTROUTING.

Joe
-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

Re: [PATCH] Runtime interception method switch

[Simon Horman]

On Wed, Jan 16, 2008 at 12:28:21PM +0100, Raphael Vallazza wrote:
> I've added the documentation for the input_hook switch, and attached  
> both patches for net-2.6.25 (i hope the mailer doesn't mess up things  
> this time :).

Indeed, this seems a lot better :-)


These patches are looking quite good. I only have two minor issues
at this stage.

1) The last fragment of the first patch seems spurious as its
   just a whitespace change. But I'm happy to send it on to DaveM
   and netdev as is.

@@ -1059,7 +1087,7 @@ static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
 		.owner		= THIS_MODULE,
 		.pf		= PF_INET,
 		.hooknum        = NF_INET_POST_ROUTING,
-		.priority       = NF_IP_PRI_NAT_SRC-1,
+		.priority       = NF_IP_PRI_NAT_SRC - 1,
 	},
 };

2) I am still getting errors with the second patch which
   I can resolve by moving the hook definitions to below
   ip_vs_register_hooks() and ip_vs_unregister_hooks()

   The gcc version and errors are bellow.
   As is a patch to do the re-ordering - that is all it does.

   If you are happy with this change I can fold it into your
   patch and send it on to DaveM and netdev, or you can send me
   a fresh patch if you would prefer. I do belive this error is real,
   perhaps your gcc is a different (older?) version and is ignoring
   this?

# gcc --version
i686-unknown-linux-gnu-gcc (GCC) 3.4.5
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is
NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.

# make
  CHK     include/linux/version.h
  CHK     include/linux/utsrelease.h
  CALL    scripts/checksyscalls.sh
  CHK     include/linux/compile.h
  CC      net/ipv4/ipvs/ip_vs_core.o
net/ipv4/ipvs/ip_vs_core.c: In function `ip_vs_register_hooks':
net/ipv4/ipvs/ip_vs_core.c:1043: error: `ip_vs_ops_local_in' undeclared (first use in this function)
net/ipv4/ipvs/ip_vs_core.c:1043: error: (Each undeclared identifier is reported only once
net/ipv4/ipvs/ip_vs_core.c:1043: error: for each function it appears in.)
net/ipv4/ipvs/ip_vs_core.c:1044: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1044: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1044: error: size of array `type name' is negative
net/ipv4/ipvs/ip_vs_core.c:1048: error: `ip_vs_ops_forward' undeclared (first use in this function)
net/ipv4/ipvs/ip_vs_core.c:1049: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1049: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1049: error: size of array `type name' is negative
net/ipv4/ipvs/ip_vs_core.c:1053: error: `ip_vs_ops_pre_routing' undeclared (first use in this function)
net/ipv4/ipvs/ip_vs_core.c:1054: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1054: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1054: error: size of array `type name' is negative
net/ipv4/ipvs/ip_vs_core.c:1066: error: `ip_vs_ops_generic' undeclared (first use in this function)
net/ipv4/ipvs/ip_vs_core.c:1067: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1067: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1067: error: size of array `type name' is negative
net/ipv4/ipvs/ip_vs_core.c: In function `ip_vs_unregister_hooks':
net/ipv4/ipvs/ip_vs_core.c:1091: error: `ip_vs_ops_local_in' undeclared (first use in this function)
net/ipv4/ipvs/ip_vs_core.c:1092: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1092: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1092: error: size of array `type name' is negative
net/ipv4/ipvs/ip_vs_core.c:1095: error: `ip_vs_ops_forward' undeclared (first use in this function)
net/ipv4/ipvs/ip_vs_core.c:1096: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1096: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1096: error: size of array `type name' is negative
net/ipv4/ipvs/ip_vs_core.c:1099: error: `ip_vs_ops_pre_routing' undeclared (first use in this function)
net/ipv4/ipvs/ip_vs_core.c:1100: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1100: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1100: error: size of array `type name' is negative
net/ipv4/ipvs/ip_vs_core.c:1107: error: `ip_vs_ops_generic' undeclared (first use in this function)
net/ipv4/ipvs/ip_vs_core.c:1107: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1107: warning: type defaults to `int' in declaration of `type name'
net/ipv4/ipvs/ip_vs_core.c:1107: error: size of array `type name' is negative
net/ipv4/ipvs/ip_vs_core.c: At top level:
net/ipv4/ipvs/ip_vs_core.c:1117: warning: 'ip_vs_ops_local_in' defined but not used
net/ipv4/ipvs/ip_vs_core.c:1131: warning: 'ip_vs_ops_forward' defined but not used
net/ipv4/ipvs/ip_vs_core.c:1152: warning: 'ip_vs_ops_pre_routing' defined but not used
net/ipv4/ipvs/ip_vs_core.c:1166: warning: 'ip_vs_ops_generic' defined but not used
make[3]: *** [net/ipv4/ipvs/ip_vs_core.o] エラー 1
make[2]: *** [net/ipv4/ipvs] エラー 2
make[1]: *** [net/ipv4] エラー 2
make: *** [net] エラー 2


Index: net-2.6.25/net/ipv4/ipvs/ip_vs_core.c
===================================================================
--- net-2.6.25.orig/net/ipv4/ipvs/ip_vs_core.c	2008-01-18 17:34:23.000000000 +0900
+++ net-2.6.25/net/ipv4/ipvs/ip_vs_core.c	2008-01-18 17:35:07.000000000 +0900
@@ -1024,6 +1024,87 @@ ip_vs_forward_icmp(unsigned int hooknum,
 	return ip_vs_in_icmp(skb, &r, hooknum);
 }
 
+
+/* After packet filtering, forward packet through VS/DR, VS/TUN,
+ * or VS/NAT(change destination), so that filtering rules can be
+ * applied to IPVS. */
+static struct nf_hook_ops ip_vs_ops_local_in[] __read_mostly = {
+	{
+		.hook		= ip_vs_in,
+		.owner		= THIS_MODULE,
+		.pf		= PF_INET,
+		.hooknum        = NF_INET_LOCAL_IN,
+		.priority       = 100,
+	},
+};
+
+/* Intercept incoming connections after they have traveled through
+ * the INPUT or the FORWARD table. It has the same functionlity of
+ * the "INPUT method", but also processes connections that are
+ * routed through the director, supporting VIP-less setups. */
+static struct nf_hook_ops ip_vs_ops_forward[] __read_mostly = {
+	{
+		.hook		= ip_vs_in,
+		.owner		= THIS_MODULE,
+		.pf		= PF_INET,
+		.hooknum        = NF_INET_LOCAL_IN,
+		.priority       = 100,
+	},
+	{
+		.hook		= ip_vs_in,
+		.owner		= THIS_MODULE,
+		.pf		= PF_INET,
+		.hooknum        = NF_INET_FORWARD,
+		.priority       = 98,
+	},
+};
+
+/* Intercept incoming connections before DNAT and input filtering
+ * has been applied, this enables ransparent proxying on realnodes
+ * and localnode. Hook right after MANGLE and before NAT_DST.
+ */
+static struct nf_hook_ops ip_vs_ops_pre_routing[] __read_mostly = {
+	{
+		.hook           = ip_vs_in,
+		.owner          = THIS_MODULE,
+		.pf             = PF_INET,
+		.hooknum        = NF_INET_PRE_ROUTING,
+		.priority       = NF_IP_PRI_NAT_DST - 1,
+	},
+};
+
+/*
+ * Generic Netfilter hooks required for all the input methods
+ */
+
+static struct nf_hook_ops ip_vs_ops_generic[] __read_mostly = {
+	/* After packet filtering, change source only for VS/NAT */
+	{
+		.hook		= ip_vs_out,
+		.owner		= THIS_MODULE,
+		.pf		= PF_INET,
+		.hooknum        = NF_INET_FORWARD,
+		.priority       = 100,
+	},
+	/* After packet filtering (but before ip_vs_out_icmp), catch icmp
+	 * destined for 0.0.0.0/0, which is for incoming IPVS connections */
+	{
+		.hook		= ip_vs_forward_icmp,
+		.owner		= THIS_MODULE,
+		.pf		= PF_INET,
+		.hooknum        = NF_INET_FORWARD,
+		.priority       = 99,
+	},
+	/* Before the netfilter connection tracking, exit from POST_ROUTING */
+	{
+		.hook		= ip_vs_post_routing,
+		.owner		= THIS_MODULE,
+		.pf		= PF_INET,
+		.hooknum        = NF_INET_POST_ROUTING,
+		.priority       = NF_IP_PRI_NAT_SRC - 1,
+	},
+};
+
 /*
  * Register netfilter hook based on input_hook type
  */
@@ -1111,87 +1192,6 @@ int ip_vs_unregister_hooks(int input_hoo
 }
 
 
-/* After packet filtering, forward packet through VS/DR, VS/TUN,
- * or VS/NAT(change destination), so that filtering rules can be
- * applied to IPVS. */
-static struct nf_hook_ops ip_vs_ops_local_in[] __read_mostly = {
-	{
-		.hook		= ip_vs_in,
-		.owner		= THIS_MODULE,
-		.pf		= PF_INET,
-		.hooknum        = NF_INET_LOCAL_IN,
-		.priority       = 100,
-	},
-};
-
-/* Intercept incoming connections after they have traveled through
- * the INPUT or the FORWARD table. It has the same functionlity of
- * the "INPUT method", but also processes connections that are
- * routed through the director, supporting VIP-less setups. */
-static struct nf_hook_ops ip_vs_ops_forward[] __read_mostly = {
-	{
-		.hook		= ip_vs_in,
-		.owner		= THIS_MODULE,
-		.pf		= PF_INET,
-		.hooknum        = NF_INET_LOCAL_IN,
-		.priority       = 100,
-	},
-	{
-		.hook		= ip_vs_in,
-		.owner		= THIS_MODULE,
-		.pf		= PF_INET,
-		.hooknum        = NF_INET_FORWARD,
-		.priority       = 98,
-	},
-};
-
-/* Intercept incoming connections before DNAT and input filtering
- * has been applied, this enables ransparent proxying on realnodes
- * and localnode. Hook right after MANGLE and before NAT_DST.
- */
-static struct nf_hook_ops ip_vs_ops_pre_routing[] __read_mostly = {
-	{
-		.hook           = ip_vs_in,
-		.owner          = THIS_MODULE,
-		.pf             = PF_INET,
-		.hooknum        = NF_INET_PRE_ROUTING,
-		.priority       = NF_IP_PRI_NAT_DST - 1,
-	},
-};
-
-/*
- * Generic Netfilter hooks required for all the input methods
- */
-
-static struct nf_hook_ops ip_vs_ops_generic[] __read_mostly = {
-	/* After packet filtering, change source only for VS/NAT */
-	{
-		.hook		= ip_vs_out,
-		.owner		= THIS_MODULE,
-		.pf		= PF_INET,
-		.hooknum        = NF_INET_FORWARD,
-		.priority       = 100,
-	},
-	/* After packet filtering (but before ip_vs_out_icmp), catch icmp
-	 * destined for 0.0.0.0/0, which is for incoming IPVS connections */
-	{
-		.hook		= ip_vs_forward_icmp,
-		.owner		= THIS_MODULE,
-		.pf		= PF_INET,
-		.hooknum        = NF_INET_FORWARD,
-		.priority       = 99,
-	},
-	/* Before the netfilter connection tracking, exit from POST_ROUTING */
-	{
-		.hook		= ip_vs_post_routing,
-		.owner		= THIS_MODULE,
-		.pf		= PF_INET,
-		.hooknum        = NF_INET_POST_ROUTING,
-		.priority       = NF_IP_PRI_NAT_SRC - 1,
-	},
-};
-
-
 /*
  *	Initialize IP Virtual Server
  */

Re: [PATCH] Runtime interception method switch

[Simon Horman]

On Wed, Jan 16, 2008 at 07:09:26AM -0800, Joseph Mack NA3T wrote:
> On Wed, 16 Jan 2008, Simon Horman wrote:
>
>> For starters could we clarify that the patch in question is the 
>> following one by Janusz Krzysztofik?
>
> I see Janusz has replied to this.

Yes I see. I'll take a look over the code a bit more.
But if he says its working then that is certainly a plus.

For the record, I am in favour of this change.

> I had assumed that if Raphael could output the packets to the right spot 
> (before POSTROUTING on the inbound direction?) that iptables could handle 
> the NAT'ing and no extra ipvs code would be neccessary.
>
> What I didn't know was the original reason the packets were output to a 
> place where iptables couldn't manipulate them. Was this for speed? to get 
> ipvs to work at all? If for speed, the director has always been limited 
> by wirespeed, not by anything in ipvs, so any increase in latency through 
> ipvs may not be seen.

I don't know the answer to that. But I guess speed. And you are right,
speed has never been much of a problem. Flexibilty on the other hand
and in particular interaction with contrack has always been problematic.

>> Also can I clarify that the aim is to be able to SNAT LVS-DR
>> connections
>
> I didn't realise Janusz was SNAT'ing LVS-DR.
>
>> (and if possible LVS-NAT and LVS-TUN)?
>> Or is the aim to add a new method, LVS-FULL-NAT?
>
> What the users want is to be able to put unmodified servers behind a 
> director - they can't even change the default gw. The only thing they can 
> change is the RIP. So the servers would have to be realservers behind an 
> LVS-NAT director which is outputting packets with src_addr=DIP, ie the  
> realservers see connect requests only from the DIP. I'd assumed the 
> director would be running a new version of standard LVS-NAT, with 
> iptables doing the SNAT in POSTROUTING.

Sorry to be picky. It seems to me that Janusz does achive the goal in
mind, in a fairly simple way. I will review ASAP.

-- 
Horms

Re: [PATCH] Runtime interception method switch

[Joseph Mack NA3T]

On Fri, 18 Jan 2008, Simon Horman wrote:

>> What the users want is to be able to put unmodified servers behind a
>> director - they can't even change the default gw. The only thing they can
>> change is the RIP. So the servers would have to be realservers behind an
>> LVS-NAT director which is outputting packets with src_addr=DIP, ie the
>> realservers see connect requests only from the DIP. I'd assumed the
>> director would be running a new version of standard LVS-NAT, with
>> iptables doing the SNAT in POSTROUTING.
>
> Sorry to be picky. It seems to me that Janusz does achive the goal in
> mind, in a fairly simple way. I will review ASAP.

not picky at all. I'm happy for anything that achieves the 
functionality.

Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!