* [patch] ipvs: Keep track of backlog connections
@ 2010-09-26 13:31 Simon Horman
2010-09-26 14:31 ` Sven Wegener
0 siblings, 1 reply; 5+ messages in thread
From: Simon Horman @ 2010-09-26 13:31 UTC (permalink / raw)
To: Sven Wegener
Cc: lvs-devel, netdev, Wensong Zhang, Julian Anastasov,
Venkata Mohan Reddy Koppula
From: Sven Wegener <sven.wegener@stealer.net>
A backlog connection is a connection that is on its way from inactive to
active. Speaking in TCP language, a connection from which we've seen the
initial SYN packet, but the three-way handshake hasn't finished yet.
These connections are expected to move to active soon. When a
destination is overloaded or isn't able to successfully establish
connections for various reasons, this count increases quickly and is an
indication for a problem.
Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
Signed-off-by: Julian Anastasov <ja@ssi.bg>
---
Hi Sven,
here is an updated though as yet untested version of your patch from Julian
to take into account recent changes. In particualr, the ip_vs_sync.c
portion is no longer needed as only the flags in the first 16 bits are
synced now. It applies against Patrick McHardy's nf-next-2.6 tree.
You mentioned in your original post that you would work on an ipvsadm
patch for this feature. Have you had time to do so?
Also, are you in a position to test this? If not I can do so.
Hi Mohan,
do you have any thoughts on an appropriate to add this feature to SCTP?
diff -urp net-next-2.6-e548833-nfct/linux/include/linux/ip_vs.h linux/include/linux/ip_vs.h
--- net-next-2.6-e548833-nfct/linux/include/linux/ip_vs.h 2010-09-15 11:28:02.000000000 +0300
+++ linux/include/linux/ip_vs.h 2010-09-26 15:21:53.507865229 +0300
@@ -91,6 +91,7 @@
/* Flags that are not sent to backup server start from bit 16 */
#define IP_VS_CONN_F_NFCT (1 << 16) /* use netfilter conntrack */
+#define IP_VS_CONN_F_BACKLOG (1 << 17) /* backlog connection */
/* Connection flags from destination that can be changed by user space */
#define IP_VS_CONN_F_DEST_MASK (IP_VS_CONN_F_FWD_MASK | \
@@ -360,6 +361,7 @@ enum {
IPVS_DEST_ATTR_PERSIST_CONNS, /* persistent connections */
IPVS_DEST_ATTR_STATS, /* nested attribute for dest stats */
+ IPVS_DEST_ATTR_BACKLOG_CONNS, /* backlog connections */
__IPVS_DEST_ATTR_MAX,
};
diff -urp net-next-2.6-e548833-nfct/linux/include/net/ip_vs.h linux/include/net/ip_vs.h
--- net-next-2.6-e548833-nfct/linux/include/net/ip_vs.h 2010-09-16 08:53:04.000000000 +0300
+++ linux/include/net/ip_vs.h 2010-09-26 15:31:27.369865994 +0300
@@ -501,6 +501,7 @@ struct ip_vs_dest {
/* connection counters and thresholds */
atomic_t activeconns; /* active connections */
atomic_t inactconns; /* inactive connections */
+ atomic_t backlogconns; /* backlog connections */
atomic_t persistconns; /* persistent connections */
__u32 u_threshold; /* upper threshold */
__u32 l_threshold; /* lower threshold */
diff -urp net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_conn.c linux/net/netfilter/ipvs/ip_vs_conn.c
--- net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_conn.c 2010-09-15 11:14:13.000000000 +0300
+++ linux/net/netfilter/ipvs/ip_vs_conn.c 2010-09-26 15:24:48.292865793 +0300
@@ -611,6 +611,8 @@ static inline void ip_vs_unbind_dest(str
} else {
atomic_dec(&dest->activeconns);
}
+ if (cp->flags & IP_VS_CONN_F_BACKLOG)
+ atomic_dec(&dest->backlogconns);
} else {
/* It is a persistent connection/template, so decrease
the peristent connection counter */
diff -urp net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_ctl.c linux/net/netfilter/ipvs/ip_vs_ctl.c
--- net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_ctl.c 2010-09-16 08:56:34.000000000 +0300
+++ linux/net/netfilter/ipvs/ip_vs_ctl.c 2010-09-26 15:26:45.407867200 +0300
@@ -2593,6 +2593,7 @@ static const struct nla_policy ip_vs_des
[IPVS_DEST_ATTR_INACT_CONNS] = { .type = NLA_U32 },
[IPVS_DEST_ATTR_PERSIST_CONNS] = { .type = NLA_U32 },
[IPVS_DEST_ATTR_STATS] = { .type = NLA_NESTED },
+ [IPVS_DEST_ATTR_BACKLOG_CONNS] = { .type = NLA_U32 },
};
static int ip_vs_genl_fill_stats(struct sk_buff *skb, int container_type,
@@ -2840,6 +2841,8 @@ static int ip_vs_genl_fill_dest(struct s
atomic_read(&dest->activeconns));
NLA_PUT_U32(skb, IPVS_DEST_ATTR_INACT_CONNS,
atomic_read(&dest->inactconns));
+ NLA_PUT_U32(skb, IPVS_DEST_ATTR_BACKLOG_CONNS,
+ atomic_read(&dest->backlogconns));
NLA_PUT_U32(skb, IPVS_DEST_ATTR_PERSIST_CONNS,
atomic_read(&dest->persistconns));
diff -urp net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_proto_tcp.c linux/net/netfilter/ipvs/ip_vs_proto_tcp.c
--- net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_proto_tcp.c 2010-09-10 08:27:33.000000000 +0300
+++ linux/net/netfilter/ipvs/ip_vs_proto_tcp.c 2010-09-26 15:29:18.425865407 +0300
@@ -510,6 +510,15 @@ set_tcp_state(struct ip_vs_protocol *pp,
atomic_dec(&dest->inactconns);
cp->flags &= ~IP_VS_CONN_F_INACTIVE;
}
+ if (new_state == IP_VS_TCP_S_SYN_RECV &&
+ !(cp->flags & IP_VS_CONN_F_BACKLOG)) {
+ atomic_inc(&dest->backlogconns);
+ cp->flags |= IP_VS_CONN_F_BACKLOG;
+ } else if (new_state == IP_VS_TCP_S_ESTABLISHED &&
+ cp->flags & IP_VS_CONN_F_BACKLOG) {
+ atomic_dec(&dest->backlogconns);
+ cp->flags &= ~IP_VS_CONN_F_BACKLOG;
+ }
}
}
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [patch] ipvs: Keep track of backlog connections
2010-09-26 13:31 [patch] ipvs: Keep track of backlog connections Simon Horman
@ 2010-09-26 14:31 ` Sven Wegener
2010-09-26 14:59 ` Sven Wegener
2010-09-27 11:06 ` Simon Horman
0 siblings, 2 replies; 5+ messages in thread
From: Sven Wegener @ 2010-09-26 14:31 UTC (permalink / raw)
To: Simon Horman
Cc: lvs-devel, netdev, Wensong Zhang, Julian Anastasov,
Venkata Mohan Reddy Koppula
On Sun, 26 Sep 2010, Simon Horman wrote:
> here is an updated though as yet untested version of your patch from Julian
> to take into account recent changes. In particualr, the ip_vs_sync.c
> portion is no longer needed as only the flags in the first 16 bits are
> synced now. It applies against Patrick McHardy's nf-next-2.6 tree.
>
> You mentioned in your original post that you would work on an ipvsadm
> patch for this feature. Have you had time to do so?
>
> Also, are you in a position to test this? If not I can do so.
Hi,
I've tested your patch with the below addition to ipvsadm and it works
correctly.
Sven
From: Sven Wegener <sven.wegener@stealer.net>
Subject: [PATCH] Show backlog connections
Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
---
ipvsadm.c | 51 ++++++++++++++++++++++++++++++--------------
libipvs/ip_vs.h | 4 +++
libipvs/ip_vs_nl_policy.c | 1 +
libipvs/libipvs.c | 3 ++
4 files changed, 43 insertions(+), 16 deletions(-)
diff --git a/ipvsadm.c b/ipvsadm.c
index 8557d21..54ae110 100644
--- a/ipvsadm.c
+++ b/ipvsadm.c
@@ -181,7 +181,8 @@ static const char* cmdnames[] = {
#define OPT_SYNCID 0x080000
#define OPT_EXACT 0x100000
#define OPT_ONEPACKET 0x200000
-#define NUMBER_OF_OPT 22
+#define OPT_BACKLOGCONN 0x400000
+#define NUMBER_OF_OPT 23
static const char* optnames[] = {
"numeric",
@@ -206,6 +207,7 @@ static const char* optnames[] = {
"syncid",
"exact",
"ops",
+ "backlog-conn",
};
/*
@@ -218,21 +220,21 @@ static const char* optnames[] = {
*/
static const char commands_v_options[NUMBER_OF_CMD][NUMBER_OF_OPT] =
{
- /* -n -c svc -s -p -M -r fwd -w -x -y -mc tot dmn -st -rt thr -pc srt sid -ex ops */
-/*ADD*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' '},
-/*EDIT*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' '},
-/*DEL*/ {'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
-/*FLUSH*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
-/*LIST*/ {' ', '1', '1', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', '1', '1', ' ', ' ', ' ', ' ', ' ', ' ', ' ', 'x'},
-/*ADDSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
-/*DELSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
-/*EDITSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
-/*TIMEOUT*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
-/*STARTD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x'},
-/*STOPD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x'},
-/*RESTORE*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
-/*SAVE*/ {' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
-/*ZERO*/ {'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
+ /* -n -c svc -s -p -M -r fwd -w -x -y -mc tot dmn -st -rt thr -pc srt sid -ex ops, blc */
+/*ADD*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x'},
+/*EDIT*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x'},
+/*DEL*/ {'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
+/*FLUSH*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
+/*LIST*/ {' ', '1', '1', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', '1', '1', ' ', ' ', ' ', ' ', ' ', ' ', ' ', 'x', ' '},
+/*ADDSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
+/*DELSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
+/*EDITSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
+/*TIMEOUT*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
+/*STARTD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x'},
+/*STOPD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x'},
+/*RESTORE*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
+/*SAVE*/ {' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
+/*ZERO*/ {'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
};
/* printing format flags */
@@ -245,6 +247,7 @@ static const char commands_v_options[NUMBER_OF_CMD][NUMBER_OF_OPT] =
#define FMT_PERSISTENTCONN 0x0020
#define FMT_NOSORT 0x0040
#define FMT_EXACT 0x0080
+#define FMT_BACKLOGCONN 0x0100
#define SERVICE_NONE 0x0000
#define SERVICE_ADDR 0x0001
@@ -282,6 +285,7 @@ enum {
TAG_PERSISTENTCONN,
TAG_SORT,
TAG_NO_SORT,
+ TAG_BACKLOGCONN,
};
/* various parsing helpers & parsing functions */
@@ -421,6 +425,8 @@ parse_options(int argc, char **argv, struct ipvs_command_entry *ce,
{ "exact", 'X', POPT_ARG_NONE, NULL, 'X', NULL, NULL },
{ "ipv6", '6', POPT_ARG_NONE, NULL, '6', NULL, NULL },
{ "ops", 'o', POPT_ARG_NONE, NULL, 'o', NULL, NULL },
+ { "backlog-conn", '\0', POPT_ARG_NONE, NULL,
+ TAG_BACKLOGCONN, NULL, NULL },
{ NULL, 0, 0, NULL, 0, NULL, NULL }
};
@@ -647,6 +653,10 @@ parse_options(int argc, char **argv, struct ipvs_command_entry *ce,
set_option(options, OPT_ONEPACKET);
ce->svc.flags |= IP_VS_SVC_F_ONEPACKET;
break;
+ case TAG_BACKLOGCONN:
+ set_option(options, OPT_BACKLOGCONN);
+ *format |= FMT_BACKLOGCONN;
+ break;
default:
fail(2, "invalid option `%s'",
poptBadOption(context, POPT_BADOPTION_NOALIAS));
@@ -1396,6 +1406,11 @@ static void print_title(unsigned int format)
" -> RemoteAddress:Port\n",
"Prot LocalAddress:Port",
"Weight", "PersistConn", "ActiveConn", "InActConn");
+ else if (format & FMT_BACKLOGCONN)
+ printf("%-33s %-9s %-11s %-10s %-10s\n"
+ " -> RemoteAddress:Port\n",
+ "Prot LocalAddress:Port",
+ "Weight", "BacklogConn", "ActiveConn", "InActConn");
else if (!(format & FMT_RULE))
printf("Prot LocalAddress:Port Scheduler Flags\n"
" -> RemoteAddress:Port Forward Weight ActiveConn InActConn\n");
@@ -1539,6 +1554,10 @@ print_service_entry(ipvs_service_entry_t *se, unsigned int format)
printf(" -> %-28s %-9u %-11u %-10u %-10u\n", dname,
e->weight, e->persistconns,
e->activeconns, e->inactconns);
+ } else if (format & FMT_BACKLOGCONN) {
+ printf(" -> %-28s %-9u %-11u %-10u %-10u\n", dname,
+ e->weight, e->backlogconns,
+ e->activeconns, e->inactconns);
} else
printf(" -> %-28s %-7s %-6d %-10u %-10u\n",
dname, fwd_name(e->conn_flags),
diff --git a/libipvs/ip_vs.h b/libipvs/ip_vs.h
index 843c51a..4c2c265 100644
--- a/libipvs/ip_vs.h
+++ b/libipvs/ip_vs.h
@@ -277,6 +277,8 @@ struct ip_vs_dest_entry {
struct ip_vs_stats_user stats;
u_int16_t af;
union nf_inet_addr addr;
+
+ u_int32_t backlogconns; /* backlog connections */
};
/* The argument to IP_VS_SO_GET_DESTS */
@@ -455,6 +457,8 @@ enum {
IPVS_DEST_ATTR_PERSIST_CONNS, /* persistent connections */
IPVS_DEST_ATTR_STATS, /* nested attribute for dest stats */
+
+ IPVS_DEST_ATTR_BACKLOG_CONNS, /* backlog connections */
__IPVS_DEST_ATTR_MAX,
};
diff --git a/libipvs/ip_vs_nl_policy.c b/libipvs/ip_vs_nl_policy.c
index c80083e..d06a490 100644
--- a/libipvs/ip_vs_nl_policy.c
+++ b/libipvs/ip_vs_nl_policy.c
@@ -40,6 +40,7 @@ struct nla_policy ipvs_dest_policy[IPVS_DEST_ATTR_MAX + 1] = {
[IPVS_DEST_ATTR_INACT_CONNS] = { .type = NLA_U32 },
[IPVS_DEST_ATTR_PERSIST_CONNS] = { .type = NLA_U32 },
[IPVS_DEST_ATTR_STATS] = { .type = NLA_NESTED },
+ [IPVS_DEST_ATTR_BACKLOG_CONNS] = { .type = NLA_U32 },
};
struct nla_policy ipvs_stats_policy[IPVS_STATS_ATTR_MAX + 1] = {
diff --git a/libipvs/libipvs.c b/libipvs/libipvs.c
index 979d5bd..e06f9fa 100644
--- a/libipvs/libipvs.c
+++ b/libipvs/libipvs.c
@@ -748,6 +748,8 @@ static int ipvs_dests_parse_cb(struct nl_msg *msg, void *arg)
d->entrytable[i].l_threshold = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_L_THRESH]);
d->entrytable[i].activeconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_ACTIVE_CONNS]);
d->entrytable[i].inactconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_INACT_CONNS]);
+ if (dest_attrs[IPVS_DEST_ATTR_BACKLOG_CONNS])
+ d->entrytable[i].backlogconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_BACKLOG_CONNS]);
d->entrytable[i].persistconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_PERSIST_CONNS]);
d->entrytable[i].af = d->af;
@@ -853,6 +855,7 @@ ipvs_nl_dest_failure:
sizeof(struct ip_vs_dest_entry_kern));
d->entrytable[i].af = AF_INET;
d->entrytable[i].addr.ip = d->entrytable[i].__addr_v4;
+ d->entrytable[i].backlogconns = 0;
}
free(dk);
return d;
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [patch] ipvs: Keep track of backlog connections
2010-09-26 14:31 ` Sven Wegener
@ 2010-09-26 14:59 ` Sven Wegener
2010-09-27 11:06 ` Simon Horman
1 sibling, 0 replies; 5+ messages in thread
From: Sven Wegener @ 2010-09-26 14:59 UTC (permalink / raw)
To: Simon Horman
Cc: lvs-devel, netdev, Wensong Zhang, Julian Anastasov,
Venkata Mohan Reddy Koppula
On Sun, 26 Sep 2010, Sven Wegener wrote:
> On Sun, 26 Sep 2010, Simon Horman wrote:
>
> > here is an updated though as yet untested version of your patch from Julian
> > to take into account recent changes. In particualr, the ip_vs_sync.c
> > portion is no longer needed as only the flags in the first 16 bits are
> > synced now. It applies against Patrick McHardy's nf-next-2.6 tree.
> >
> > You mentioned in your original post that you would work on an ipvsadm
> > patch for this feature. Have you had time to do so?
> >
> > Also, are you in a position to test this? If not I can do so.
>
> I've tested your patch with the below addition to ipvsadm and it works
> correctly.
And here is a small patch for the manpage:
diff --git a/ipvsadm.8 b/ipvsadm.8
index 6874c0e..78d67ef 100644
--- a/ipvsadm.8
+++ b/ipvsadm.8
@@ -392,6 +392,15 @@ information of each server in service listing. The persistent
connection is used to forward the actual connections from the same
client/network to the same server.
.TP
+.B --backlog-conn
+Output of backlog connection information. The \fIlist\fP command
+with this option will display the backlog connection counter
+information of each server in service listing. A backlog connection
+is a connection that is inactive, but is expected to move to active
+soon. Speaking in TCP language, a connection from which we have seen
+the initial SYN packet, but the three-way handshake has not finished
+yet.
+.TP
.B --sort
Sort the list of virtual services and real servers. The virtual
service entries are sorted in ascending order by <protocol, address,
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [patch] ipvs: Keep track of backlog connections
2010-09-26 14:31 ` Sven Wegener
2010-09-26 14:59 ` Sven Wegener
@ 2010-09-27 11:06 ` Simon Horman
1 sibling, 0 replies; 5+ messages in thread
From: Simon Horman @ 2010-09-27 11:06 UTC (permalink / raw)
To: Sven Wegener
Cc: lvs-devel, netdev, Wensong Zhang, Julian Anastasov,
Venkata Mohan Reddy Koppula
On Sun, Sep 26, 2010 at 04:31:49PM +0200, Sven Wegener wrote:
> On Sun, 26 Sep 2010, Simon Horman wrote:
>
> > here is an updated though as yet untested version of your patch from Julian
> > to take into account recent changes. In particualr, the ip_vs_sync.c
> > portion is no longer needed as only the flags in the first 16 bits are
> > synced now. It applies against Patrick McHardy's nf-next-2.6 tree.
> >
> > You mentioned in your original post that you would work on an ipvsadm
> > patch for this feature. Have you had time to do so?
> >
> > Also, are you in a position to test this? If not I can do so.
>
> Hi,
>
> I've tested your patch with the below addition to ipvsadm and it works
> correctly.
Thanks Sven,
I'll push the kernel side to Patrick McHardy and assuming
Wensong doesn't object, I'll push the ipvsadm side myself.
>
> Sven
>
> From: Sven Wegener <sven.wegener@stealer.net>
> Subject: [PATCH] Show backlog connections
>
> Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
> ---
> ipvsadm.c | 51 ++++++++++++++++++++++++++++++--------------
> libipvs/ip_vs.h | 4 +++
> libipvs/ip_vs_nl_policy.c | 1 +
> libipvs/libipvs.c | 3 ++
> 4 files changed, 43 insertions(+), 16 deletions(-)
>
> diff --git a/ipvsadm.c b/ipvsadm.c
> index 8557d21..54ae110 100644
> --- a/ipvsadm.c
> +++ b/ipvsadm.c
> @@ -181,7 +181,8 @@ static const char* cmdnames[] = {
> #define OPT_SYNCID 0x080000
> #define OPT_EXACT 0x100000
> #define OPT_ONEPACKET 0x200000
> -#define NUMBER_OF_OPT 22
> +#define OPT_BACKLOGCONN 0x400000
> +#define NUMBER_OF_OPT 23
>
> static const char* optnames[] = {
> "numeric",
> @@ -206,6 +207,7 @@ static const char* optnames[] = {
> "syncid",
> "exact",
> "ops",
> + "backlog-conn",
> };
>
> /*
> @@ -218,21 +220,21 @@ static const char* optnames[] = {
> */
> static const char commands_v_options[NUMBER_OF_CMD][NUMBER_OF_OPT] =
> {
> - /* -n -c svc -s -p -M -r fwd -w -x -y -mc tot dmn -st -rt thr -pc srt sid -ex ops */
> -/*ADD*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' '},
> -/*EDIT*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' '},
> -/*DEL*/ {'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> -/*FLUSH*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> -/*LIST*/ {' ', '1', '1', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', '1', '1', ' ', ' ', ' ', ' ', ' ', ' ', ' ', 'x'},
> -/*ADDSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> -/*DELSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> -/*EDITSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> -/*TIMEOUT*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> -/*STARTD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x'},
> -/*STOPD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x'},
> -/*RESTORE*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> -/*SAVE*/ {' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> -/*ZERO*/ {'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> + /* -n -c svc -s -p -M -r fwd -w -x -y -mc tot dmn -st -rt thr -pc srt sid -ex ops, blc */
> +/*ADD*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x'},
> +/*EDIT*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x'},
> +/*DEL*/ {'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> +/*FLUSH*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> +/*LIST*/ {' ', '1', '1', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', '1', '1', ' ', ' ', ' ', ' ', ' ', ' ', ' ', 'x', ' '},
> +/*ADDSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> +/*DELSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> +/*EDITSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> +/*TIMEOUT*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> +/*STARTD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x'},
> +/*STOPD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x'},
> +/*RESTORE*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> +/*SAVE*/ {' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> +/*ZERO*/ {'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'},
> };
>
> /* printing format flags */
> @@ -245,6 +247,7 @@ static const char commands_v_options[NUMBER_OF_CMD][NUMBER_OF_OPT] =
> #define FMT_PERSISTENTCONN 0x0020
> #define FMT_NOSORT 0x0040
> #define FMT_EXACT 0x0080
> +#define FMT_BACKLOGCONN 0x0100
>
> #define SERVICE_NONE 0x0000
> #define SERVICE_ADDR 0x0001
> @@ -282,6 +285,7 @@ enum {
> TAG_PERSISTENTCONN,
> TAG_SORT,
> TAG_NO_SORT,
> + TAG_BACKLOGCONN,
> };
>
> /* various parsing helpers & parsing functions */
> @@ -421,6 +425,8 @@ parse_options(int argc, char **argv, struct ipvs_command_entry *ce,
> { "exact", 'X', POPT_ARG_NONE, NULL, 'X', NULL, NULL },
> { "ipv6", '6', POPT_ARG_NONE, NULL, '6', NULL, NULL },
> { "ops", 'o', POPT_ARG_NONE, NULL, 'o', NULL, NULL },
> + { "backlog-conn", '\0', POPT_ARG_NONE, NULL,
> + TAG_BACKLOGCONN, NULL, NULL },
> { NULL, 0, 0, NULL, 0, NULL, NULL }
> };
>
> @@ -647,6 +653,10 @@ parse_options(int argc, char **argv, struct ipvs_command_entry *ce,
> set_option(options, OPT_ONEPACKET);
> ce->svc.flags |= IP_VS_SVC_F_ONEPACKET;
> break;
> + case TAG_BACKLOGCONN:
> + set_option(options, OPT_BACKLOGCONN);
> + *format |= FMT_BACKLOGCONN;
> + break;
> default:
> fail(2, "invalid option `%s'",
> poptBadOption(context, POPT_BADOPTION_NOALIAS));
> @@ -1396,6 +1406,11 @@ static void print_title(unsigned int format)
> " -> RemoteAddress:Port\n",
> "Prot LocalAddress:Port",
> "Weight", "PersistConn", "ActiveConn", "InActConn");
> + else if (format & FMT_BACKLOGCONN)
> + printf("%-33s %-9s %-11s %-10s %-10s\n"
> + " -> RemoteAddress:Port\n",
> + "Prot LocalAddress:Port",
> + "Weight", "BacklogConn", "ActiveConn", "InActConn");
> else if (!(format & FMT_RULE))
> printf("Prot LocalAddress:Port Scheduler Flags\n"
> " -> RemoteAddress:Port Forward Weight ActiveConn InActConn\n");
> @@ -1539,6 +1554,10 @@ print_service_entry(ipvs_service_entry_t *se, unsigned int format)
> printf(" -> %-28s %-9u %-11u %-10u %-10u\n", dname,
> e->weight, e->persistconns,
> e->activeconns, e->inactconns);
> + } else if (format & FMT_BACKLOGCONN) {
> + printf(" -> %-28s %-9u %-11u %-10u %-10u\n", dname,
> + e->weight, e->backlogconns,
> + e->activeconns, e->inactconns);
> } else
> printf(" -> %-28s %-7s %-6d %-10u %-10u\n",
> dname, fwd_name(e->conn_flags),
> diff --git a/libipvs/ip_vs.h b/libipvs/ip_vs.h
> index 843c51a..4c2c265 100644
> --- a/libipvs/ip_vs.h
> +++ b/libipvs/ip_vs.h
> @@ -277,6 +277,8 @@ struct ip_vs_dest_entry {
> struct ip_vs_stats_user stats;
> u_int16_t af;
> union nf_inet_addr addr;
> +
> + u_int32_t backlogconns; /* backlog connections */
> };
>
> /* The argument to IP_VS_SO_GET_DESTS */
> @@ -455,6 +457,8 @@ enum {
> IPVS_DEST_ATTR_PERSIST_CONNS, /* persistent connections */
>
> IPVS_DEST_ATTR_STATS, /* nested attribute for dest stats */
> +
> + IPVS_DEST_ATTR_BACKLOG_CONNS, /* backlog connections */
> __IPVS_DEST_ATTR_MAX,
> };
>
> diff --git a/libipvs/ip_vs_nl_policy.c b/libipvs/ip_vs_nl_policy.c
> index c80083e..d06a490 100644
> --- a/libipvs/ip_vs_nl_policy.c
> +++ b/libipvs/ip_vs_nl_policy.c
> @@ -40,6 +40,7 @@ struct nla_policy ipvs_dest_policy[IPVS_DEST_ATTR_MAX + 1] = {
> [IPVS_DEST_ATTR_INACT_CONNS] = { .type = NLA_U32 },
> [IPVS_DEST_ATTR_PERSIST_CONNS] = { .type = NLA_U32 },
> [IPVS_DEST_ATTR_STATS] = { .type = NLA_NESTED },
> + [IPVS_DEST_ATTR_BACKLOG_CONNS] = { .type = NLA_U32 },
> };
>
> struct nla_policy ipvs_stats_policy[IPVS_STATS_ATTR_MAX + 1] = {
> diff --git a/libipvs/libipvs.c b/libipvs/libipvs.c
> index 979d5bd..e06f9fa 100644
> --- a/libipvs/libipvs.c
> +++ b/libipvs/libipvs.c
> @@ -748,6 +748,8 @@ static int ipvs_dests_parse_cb(struct nl_msg *msg, void *arg)
> d->entrytable[i].l_threshold = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_L_THRESH]);
> d->entrytable[i].activeconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_ACTIVE_CONNS]);
> d->entrytable[i].inactconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_INACT_CONNS]);
> + if (dest_attrs[IPVS_DEST_ATTR_BACKLOG_CONNS])
> + d->entrytable[i].backlogconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_BACKLOG_CONNS]);
> d->entrytable[i].persistconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_PERSIST_CONNS]);
> d->entrytable[i].af = d->af;
>
> @@ -853,6 +855,7 @@ ipvs_nl_dest_failure:
> sizeof(struct ip_vs_dest_entry_kern));
> d->entrytable[i].af = AF_INET;
> d->entrytable[i].addr.ip = d->entrytable[i].__addr_v4;
> + d->entrytable[i].backlogconns = 0;
> }
> free(dk);
> return d;
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* [patch] ipvs: Keep track of backlog connections
@ 2010-09-27 14:05 Simon Horman
0 siblings, 0 replies; 5+ messages in thread
From: Simon Horman @ 2010-09-27 14:05 UTC (permalink / raw)
To: lvs-devel, netfilter-devel, netdev
Cc: Sven Wegener, Wensong Zhang, Julian Anastasov,
Venkata Mohan Reddy Koppula, Patrick McHardy
From: Sven Wegener <sven.wegener@stealer.net>
A backlog connection is a connection that is on its way from inactive to
active. Speaking in TCP language, a connection from which we've seen the
initial SYN packet, but the three-way handshake hasn't finished yet.
These connections are expected to move to active soon. When a
destination is overloaded or isn't able to successfully establish
connections for various reasons, this count increases quickly and is an
indication for a problem.
Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
Signed-off-by: Julian Anastasov <ja@ssi.bg>
Acked-by: Simon Horman <horms@verge.net.au>
---
Patrick, please consider this for nf-next
diff -urp net-next-2.6-e548833-nfct/linux/include/linux/ip_vs.h linux/include/linux/ip_vs.h
--- net-next-2.6-e548833-nfct/linux/include/linux/ip_vs.h 2010-09-15 11:28:02.000000000 +0300
+++ linux/include/linux/ip_vs.h 2010-09-26 15:21:53.507865229 +0300
@@ -91,6 +91,7 @@
/* Flags that are not sent to backup server start from bit 16 */
#define IP_VS_CONN_F_NFCT (1 << 16) /* use netfilter conntrack */
+#define IP_VS_CONN_F_BACKLOG (1 << 17) /* backlog connection */
/* Connection flags from destination that can be changed by user space */
#define IP_VS_CONN_F_DEST_MASK (IP_VS_CONN_F_FWD_MASK | \
@@ -360,6 +361,7 @@ enum {
IPVS_DEST_ATTR_PERSIST_CONNS, /* persistent connections */
IPVS_DEST_ATTR_STATS, /* nested attribute for dest stats */
+ IPVS_DEST_ATTR_BACKLOG_CONNS, /* backlog connections */
__IPVS_DEST_ATTR_MAX,
};
diff -urp net-next-2.6-e548833-nfct/linux/include/net/ip_vs.h linux/include/net/ip_vs.h
--- net-next-2.6-e548833-nfct/linux/include/net/ip_vs.h 2010-09-16 08:53:04.000000000 +0300
+++ linux/include/net/ip_vs.h 2010-09-26 15:31:27.369865994 +0300
@@ -501,6 +501,7 @@ struct ip_vs_dest {
/* connection counters and thresholds */
atomic_t activeconns; /* active connections */
atomic_t inactconns; /* inactive connections */
+ atomic_t backlogconns; /* backlog connections */
atomic_t persistconns; /* persistent connections */
__u32 u_threshold; /* upper threshold */
__u32 l_threshold; /* lower threshold */
diff -urp net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_conn.c linux/net/netfilter/ipvs/ip_vs_conn.c
--- net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_conn.c 2010-09-15 11:14:13.000000000 +0300
+++ linux/net/netfilter/ipvs/ip_vs_conn.c 2010-09-26 15:24:48.292865793 +0300
@@ -611,6 +611,8 @@ static inline void ip_vs_unbind_dest(str
} else {
atomic_dec(&dest->activeconns);
}
+ if (cp->flags & IP_VS_CONN_F_BACKLOG)
+ atomic_dec(&dest->backlogconns);
} else {
/* It is a persistent connection/template, so decrease
the peristent connection counter */
diff -urp net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_ctl.c linux/net/netfilter/ipvs/ip_vs_ctl.c
--- net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_ctl.c 2010-09-16 08:56:34.000000000 +0300
+++ linux/net/netfilter/ipvs/ip_vs_ctl.c 2010-09-26 15:26:45.407867200 +0300
@@ -2593,6 +2593,7 @@ static const struct nla_policy ip_vs_des
[IPVS_DEST_ATTR_INACT_CONNS] = { .type = NLA_U32 },
[IPVS_DEST_ATTR_PERSIST_CONNS] = { .type = NLA_U32 },
[IPVS_DEST_ATTR_STATS] = { .type = NLA_NESTED },
+ [IPVS_DEST_ATTR_BACKLOG_CONNS] = { .type = NLA_U32 },
};
static int ip_vs_genl_fill_stats(struct sk_buff *skb, int container_type,
@@ -2840,6 +2841,8 @@ static int ip_vs_genl_fill_dest(struct s
atomic_read(&dest->activeconns));
NLA_PUT_U32(skb, IPVS_DEST_ATTR_INACT_CONNS,
atomic_read(&dest->inactconns));
+ NLA_PUT_U32(skb, IPVS_DEST_ATTR_BACKLOG_CONNS,
+ atomic_read(&dest->backlogconns));
NLA_PUT_U32(skb, IPVS_DEST_ATTR_PERSIST_CONNS,
atomic_read(&dest->persistconns));
diff -urp net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_proto_tcp.c linux/net/netfilter/ipvs/ip_vs_proto_tcp.c
--- net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_proto_tcp.c 2010-09-10 08:27:33.000000000 +0300
+++ linux/net/netfilter/ipvs/ip_vs_proto_tcp.c 2010-09-26 15:29:18.425865407 +0300
@@ -510,6 +510,15 @@ set_tcp_state(struct ip_vs_protocol *pp,
atomic_dec(&dest->inactconns);
cp->flags &= ~IP_VS_CONN_F_INACTIVE;
}
+ if (new_state == IP_VS_TCP_S_SYN_RECV &&
+ !(cp->flags & IP_VS_CONN_F_BACKLOG)) {
+ atomic_inc(&dest->backlogconns);
+ cp->flags |= IP_VS_CONN_F_BACKLOG;
+ } else if (new_state == IP_VS_TCP_S_ESTABLISHED &&
+ cp->flags & IP_VS_CONN_F_BACKLOG) {
+ atomic_dec(&dest->backlogconns);
+ cp->flags &= ~IP_VS_CONN_F_BACKLOG;
+ }
}
}
--
To unsubscribe from this list: send the line "unsubscribe lvs-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2010-09-27 14:05 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-09-26 13:31 [patch] ipvs: Keep track of backlog connections Simon Horman
2010-09-26 14:31 ` Sven Wegener
2010-09-26 14:59 ` Sven Wegener
2010-09-27 11:06 ` Simon Horman
-- strict thread matches above, loose matches on Subject: below --
2010-09-27 14:05 Simon Horman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).