* [patch] ipvs: Keep track of backlog connections
@ 2010-09-26 13:31 Simon Horman
2010-09-26 14:31 ` Sven Wegener
0 siblings, 1 reply; 5+ messages in thread
From: Simon Horman @ 2010-09-26 13:31 UTC (permalink / raw)
To: Sven Wegener
Cc: lvs-devel, netdev, Wensong Zhang, Julian Anastasov,
Venkata Mohan Reddy Koppula
From: Sven Wegener <sven.wegener@stealer.net>
A backlog connection is a connection that is on its way from inactive to
active. Speaking in TCP language, a connection from which we've seen the
initial SYN packet, but the three-way handshake hasn't finished yet.
These connections are expected to move to active soon. When a
destination is overloaded or isn't able to successfully establish
connections for various reasons, this count increases quickly and is an
indication for a problem.
Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
Signed-off-by: Julian Anastasov <ja@ssi.bg>
---
Hi Sven,
here is an updated though as yet untested version of your patch from Julian
to take into account recent changes. In particualr, the ip_vs_sync.c
portion is no longer needed as only the flags in the first 16 bits are
synced now. It applies against Patrick McHardy's nf-next-2.6 tree.
You mentioned in your original post that you would work on an ipvsadm
patch for this feature. Have you had time to do so?
Also, are you in a position to test this? If not I can do so.
Hi Mohan,
do you have any thoughts on an appropriate to add this feature to SCTP?
diff -urp net-next-2.6-e548833-nfct/linux/include/linux/ip_vs.h linux/include/linux/ip_vs.h
--- net-next-2.6-e548833-nfct/linux/include/linux/ip_vs.h 2010-09-15 11:28:02.000000000 +0300
+++ linux/include/linux/ip_vs.h 2010-09-26 15:21:53.507865229 +0300
@@ -91,6 +91,7 @@
/* Flags that are not sent to backup server start from bit 16 */
#define IP_VS_CONN_F_NFCT (1 << 16) /* use netfilter conntrack */
+#define IP_VS_CONN_F_BACKLOG (1 << 17) /* backlog connection */
/* Connection flags from destination that can be changed by user space */
#define IP_VS_CONN_F_DEST_MASK (IP_VS_CONN_F_FWD_MASK | \
@@ -360,6 +361,7 @@ enum {
IPVS_DEST_ATTR_PERSIST_CONNS, /* persistent connections */
IPVS_DEST_ATTR_STATS, /* nested attribute for dest stats */
+ IPVS_DEST_ATTR_BACKLOG_CONNS, /* backlog connections */
__IPVS_DEST_ATTR_MAX,
};
diff -urp net-next-2.6-e548833-nfct/linux/include/net/ip_vs.h linux/include/net/ip_vs.h
--- net-next-2.6-e548833-nfct/linux/include/net/ip_vs.h 2010-09-16 08:53:04.000000000 +0300
+++ linux/include/net/ip_vs.h 2010-09-26 15:31:27.369865994 +0300
@@ -501,6 +501,7 @@ struct ip_vs_dest {
/* connection counters and thresholds */
atomic_t activeconns; /* active connections */
atomic_t inactconns; /* inactive connections */
+ atomic_t backlogconns; /* backlog connections */
atomic_t persistconns; /* persistent connections */
__u32 u_threshold; /* upper threshold */
__u32 l_threshold; /* lower threshold */
diff -urp net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_conn.c linux/net/netfilter/ipvs/ip_vs_conn.c
--- net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_conn.c 2010-09-15 11:14:13.000000000 +0300
+++ linux/net/netfilter/ipvs/ip_vs_conn.c 2010-09-26 15:24:48.292865793 +0300
@@ -611,6 +611,8 @@ static inline void ip_vs_unbind_dest(str
} else {
atomic_dec(&dest->activeconns);
}
+ if (cp->flags & IP_VS_CONN_F_BACKLOG)
+ atomic_dec(&dest->backlogconns);
} else {
/* It is a persistent connection/template, so decrease
the peristent connection counter */
diff -urp net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_ctl.c linux/net/netfilter/ipvs/ip_vs_ctl.c
--- net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_ctl.c 2010-09-16 08:56:34.000000000 +0300
+++ linux/net/netfilter/ipvs/ip_vs_ctl.c 2010-09-26 15:26:45.407867200 +0300
@@ -2593,6 +2593,7 @@ static const struct nla_policy ip_vs_des
[IPVS_DEST_ATTR_INACT_CONNS] = { .type = NLA_U32 },
[IPVS_DEST_ATTR_PERSIST_CONNS] = { .type = NLA_U32 },
[IPVS_DEST_ATTR_STATS] = { .type = NLA_NESTED },
+ [IPVS_DEST_ATTR_BACKLOG_CONNS] = { .type = NLA_U32 },
};
static int ip_vs_genl_fill_stats(struct sk_buff *skb, int container_type,
@@ -2840,6 +2841,8 @@ static int ip_vs_genl_fill_dest(struct s
atomic_read(&dest->activeconns));
NLA_PUT_U32(skb, IPVS_DEST_ATTR_INACT_CONNS,
atomic_read(&dest->inactconns));
+ NLA_PUT_U32(skb, IPVS_DEST_ATTR_BACKLOG_CONNS,
+ atomic_read(&dest->backlogconns));
NLA_PUT_U32(skb, IPVS_DEST_ATTR_PERSIST_CONNS,
atomic_read(&dest->persistconns));
diff -urp net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_proto_tcp.c linux/net/netfilter/ipvs/ip_vs_proto_tcp.c
--- net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_proto_tcp.c 2010-09-10 08:27:33.000000000 +0300
+++ linux/net/netfilter/ipvs/ip_vs_proto_tcp.c 2010-09-26 15:29:18.425865407 +0300
@@ -510,6 +510,15 @@ set_tcp_state(struct ip_vs_protocol *pp,
atomic_dec(&dest->inactconns);
cp->flags &= ~IP_VS_CONN_F_INACTIVE;
}
+ if (new_state == IP_VS_TCP_S_SYN_RECV &&
+ !(cp->flags & IP_VS_CONN_F_BACKLOG)) {
+ atomic_inc(&dest->backlogconns);
+ cp->flags |= IP_VS_CONN_F_BACKLOG;
+ } else if (new_state == IP_VS_TCP_S_ESTABLISHED &&
+ cp->flags & IP_VS_CONN_F_BACKLOG) {
+ atomic_dec(&dest->backlogconns);
+ cp->flags &= ~IP_VS_CONN_F_BACKLOG;
+ }
}
}
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [patch] ipvs: Keep track of backlog connections 2010-09-26 13:31 [patch] ipvs: Keep track of backlog connections Simon Horman @ 2010-09-26 14:31 ` Sven Wegener 2010-09-26 14:59 ` Sven Wegener 2010-09-27 11:06 ` Simon Horman 0 siblings, 2 replies; 5+ messages in thread From: Sven Wegener @ 2010-09-26 14:31 UTC (permalink / raw) To: Simon Horman Cc: lvs-devel, netdev, Wensong Zhang, Julian Anastasov, Venkata Mohan Reddy Koppula On Sun, 26 Sep 2010, Simon Horman wrote: > here is an updated though as yet untested version of your patch from Julian > to take into account recent changes. In particualr, the ip_vs_sync.c > portion is no longer needed as only the flags in the first 16 bits are > synced now. It applies against Patrick McHardy's nf-next-2.6 tree. > > You mentioned in your original post that you would work on an ipvsadm > patch for this feature. Have you had time to do so? > > Also, are you in a position to test this? If not I can do so. Hi, I've tested your patch with the below addition to ipvsadm and it works correctly. Sven From: Sven Wegener <sven.wegener@stealer.net> Subject: [PATCH] Show backlog connections Signed-off-by: Sven Wegener <sven.wegener@stealer.net> --- ipvsadm.c | 51 ++++++++++++++++++++++++++++++-------------- libipvs/ip_vs.h | 4 +++ libipvs/ip_vs_nl_policy.c | 1 + libipvs/libipvs.c | 3 ++ 4 files changed, 43 insertions(+), 16 deletions(-) diff --git a/ipvsadm.c b/ipvsadm.c index 8557d21..54ae110 100644 --- a/ipvsadm.c +++ b/ipvsadm.c @@ -181,7 +181,8 @@ static const char* cmdnames[] = { #define OPT_SYNCID 0x080000 #define OPT_EXACT 0x100000 #define OPT_ONEPACKET 0x200000 -#define NUMBER_OF_OPT 22 +#define OPT_BACKLOGCONN 0x400000 +#define NUMBER_OF_OPT 23 static const char* optnames[] = { "numeric", @@ -206,6 +207,7 @@ static const char* optnames[] = { "syncid", "exact", "ops", + "backlog-conn", }; /* @@ -218,21 +220,21 @@ static const char* optnames[] = { */ static const char commands_v_options[NUMBER_OF_CMD][NUMBER_OF_OPT] = { - /* -n -c svc -s -p -M -r fwd -w -x -y -mc tot dmn -st -rt thr -pc srt sid -ex ops */ -/*ADD*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' '}, -/*EDIT*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' '}, -/*DEL*/ {'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, -/*FLUSH*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, -/*LIST*/ {' ', '1', '1', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', '1', '1', ' ', ' ', ' ', ' ', ' ', ' ', ' ', 'x'}, -/*ADDSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, -/*DELSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, -/*EDITSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, -/*TIMEOUT*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, -/*STARTD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x'}, -/*STOPD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x'}, -/*RESTORE*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, -/*SAVE*/ {' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, -/*ZERO*/ {'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, + /* -n -c svc -s -p -M -r fwd -w -x -y -mc tot dmn -st -rt thr -pc srt sid -ex ops, blc */ +/*ADD*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x'}, +/*EDIT*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x'}, +/*DEL*/ {'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, +/*FLUSH*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, +/*LIST*/ {' ', '1', '1', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', '1', '1', ' ', ' ', ' ', ' ', ' ', ' ', ' ', 'x', ' '}, +/*ADDSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, +/*DELSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, +/*EDITSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, +/*TIMEOUT*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, +/*STARTD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x'}, +/*STOPD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x'}, +/*RESTORE*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, +/*SAVE*/ {' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, +/*ZERO*/ {'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, }; /* printing format flags */ @@ -245,6 +247,7 @@ static const char commands_v_options[NUMBER_OF_CMD][NUMBER_OF_OPT] = #define FMT_PERSISTENTCONN 0x0020 #define FMT_NOSORT 0x0040 #define FMT_EXACT 0x0080 +#define FMT_BACKLOGCONN 0x0100 #define SERVICE_NONE 0x0000 #define SERVICE_ADDR 0x0001 @@ -282,6 +285,7 @@ enum { TAG_PERSISTENTCONN, TAG_SORT, TAG_NO_SORT, + TAG_BACKLOGCONN, }; /* various parsing helpers & parsing functions */ @@ -421,6 +425,8 @@ parse_options(int argc, char **argv, struct ipvs_command_entry *ce, { "exact", 'X', POPT_ARG_NONE, NULL, 'X', NULL, NULL }, { "ipv6", '6', POPT_ARG_NONE, NULL, '6', NULL, NULL }, { "ops", 'o', POPT_ARG_NONE, NULL, 'o', NULL, NULL }, + { "backlog-conn", '\0', POPT_ARG_NONE, NULL, + TAG_BACKLOGCONN, NULL, NULL }, { NULL, 0, 0, NULL, 0, NULL, NULL } }; @@ -647,6 +653,10 @@ parse_options(int argc, char **argv, struct ipvs_command_entry *ce, set_option(options, OPT_ONEPACKET); ce->svc.flags |= IP_VS_SVC_F_ONEPACKET; break; + case TAG_BACKLOGCONN: + set_option(options, OPT_BACKLOGCONN); + *format |= FMT_BACKLOGCONN; + break; default: fail(2, "invalid option `%s'", poptBadOption(context, POPT_BADOPTION_NOALIAS)); @@ -1396,6 +1406,11 @@ static void print_title(unsigned int format) " -> RemoteAddress:Port\n", "Prot LocalAddress:Port", "Weight", "PersistConn", "ActiveConn", "InActConn"); + else if (format & FMT_BACKLOGCONN) + printf("%-33s %-9s %-11s %-10s %-10s\n" + " -> RemoteAddress:Port\n", + "Prot LocalAddress:Port", + "Weight", "BacklogConn", "ActiveConn", "InActConn"); else if (!(format & FMT_RULE)) printf("Prot LocalAddress:Port Scheduler Flags\n" " -> RemoteAddress:Port Forward Weight ActiveConn InActConn\n"); @@ -1539,6 +1554,10 @@ print_service_entry(ipvs_service_entry_t *se, unsigned int format) printf(" -> %-28s %-9u %-11u %-10u %-10u\n", dname, e->weight, e->persistconns, e->activeconns, e->inactconns); + } else if (format & FMT_BACKLOGCONN) { + printf(" -> %-28s %-9u %-11u %-10u %-10u\n", dname, + e->weight, e->backlogconns, + e->activeconns, e->inactconns); } else printf(" -> %-28s %-7s %-6d %-10u %-10u\n", dname, fwd_name(e->conn_flags), diff --git a/libipvs/ip_vs.h b/libipvs/ip_vs.h index 843c51a..4c2c265 100644 --- a/libipvs/ip_vs.h +++ b/libipvs/ip_vs.h @@ -277,6 +277,8 @@ struct ip_vs_dest_entry { struct ip_vs_stats_user stats; u_int16_t af; union nf_inet_addr addr; + + u_int32_t backlogconns; /* backlog connections */ }; /* The argument to IP_VS_SO_GET_DESTS */ @@ -455,6 +457,8 @@ enum { IPVS_DEST_ATTR_PERSIST_CONNS, /* persistent connections */ IPVS_DEST_ATTR_STATS, /* nested attribute for dest stats */ + + IPVS_DEST_ATTR_BACKLOG_CONNS, /* backlog connections */ __IPVS_DEST_ATTR_MAX, }; diff --git a/libipvs/ip_vs_nl_policy.c b/libipvs/ip_vs_nl_policy.c index c80083e..d06a490 100644 --- a/libipvs/ip_vs_nl_policy.c +++ b/libipvs/ip_vs_nl_policy.c @@ -40,6 +40,7 @@ struct nla_policy ipvs_dest_policy[IPVS_DEST_ATTR_MAX + 1] = { [IPVS_DEST_ATTR_INACT_CONNS] = { .type = NLA_U32 }, [IPVS_DEST_ATTR_PERSIST_CONNS] = { .type = NLA_U32 }, [IPVS_DEST_ATTR_STATS] = { .type = NLA_NESTED }, + [IPVS_DEST_ATTR_BACKLOG_CONNS] = { .type = NLA_U32 }, }; struct nla_policy ipvs_stats_policy[IPVS_STATS_ATTR_MAX + 1] = { diff --git a/libipvs/libipvs.c b/libipvs/libipvs.c index 979d5bd..e06f9fa 100644 --- a/libipvs/libipvs.c +++ b/libipvs/libipvs.c @@ -748,6 +748,8 @@ static int ipvs_dests_parse_cb(struct nl_msg *msg, void *arg) d->entrytable[i].l_threshold = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_L_THRESH]); d->entrytable[i].activeconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_ACTIVE_CONNS]); d->entrytable[i].inactconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_INACT_CONNS]); + if (dest_attrs[IPVS_DEST_ATTR_BACKLOG_CONNS]) + d->entrytable[i].backlogconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_BACKLOG_CONNS]); d->entrytable[i].persistconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_PERSIST_CONNS]); d->entrytable[i].af = d->af; @@ -853,6 +855,7 @@ ipvs_nl_dest_failure: sizeof(struct ip_vs_dest_entry_kern)); d->entrytable[i].af = AF_INET; d->entrytable[i].addr.ip = d->entrytable[i].__addr_v4; + d->entrytable[i].backlogconns = 0; } free(dk); return d; ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [patch] ipvs: Keep track of backlog connections 2010-09-26 14:31 ` Sven Wegener @ 2010-09-26 14:59 ` Sven Wegener 2010-09-27 11:06 ` Simon Horman 1 sibling, 0 replies; 5+ messages in thread From: Sven Wegener @ 2010-09-26 14:59 UTC (permalink / raw) To: Simon Horman Cc: lvs-devel, netdev, Wensong Zhang, Julian Anastasov, Venkata Mohan Reddy Koppula On Sun, 26 Sep 2010, Sven Wegener wrote: > On Sun, 26 Sep 2010, Simon Horman wrote: > > > here is an updated though as yet untested version of your patch from Julian > > to take into account recent changes. In particualr, the ip_vs_sync.c > > portion is no longer needed as only the flags in the first 16 bits are > > synced now. It applies against Patrick McHardy's nf-next-2.6 tree. > > > > You mentioned in your original post that you would work on an ipvsadm > > patch for this feature. Have you had time to do so? > > > > Also, are you in a position to test this? If not I can do so. > > I've tested your patch with the below addition to ipvsadm and it works > correctly. And here is a small patch for the manpage: diff --git a/ipvsadm.8 b/ipvsadm.8 index 6874c0e..78d67ef 100644 --- a/ipvsadm.8 +++ b/ipvsadm.8 @@ -392,6 +392,15 @@ information of each server in service listing. The persistent connection is used to forward the actual connections from the same client/network to the same server. .TP +.B --backlog-conn +Output of backlog connection information. The \fIlist\fP command +with this option will display the backlog connection counter +information of each server in service listing. A backlog connection +is a connection that is inactive, but is expected to move to active +soon. Speaking in TCP language, a connection from which we have seen +the initial SYN packet, but the three-way handshake has not finished +yet. +.TP .B --sort Sort the list of virtual services and real servers. The virtual service entries are sorted in ascending order by <protocol, address, ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [patch] ipvs: Keep track of backlog connections 2010-09-26 14:31 ` Sven Wegener 2010-09-26 14:59 ` Sven Wegener @ 2010-09-27 11:06 ` Simon Horman 1 sibling, 0 replies; 5+ messages in thread From: Simon Horman @ 2010-09-27 11:06 UTC (permalink / raw) To: Sven Wegener Cc: lvs-devel, netdev, Wensong Zhang, Julian Anastasov, Venkata Mohan Reddy Koppula On Sun, Sep 26, 2010 at 04:31:49PM +0200, Sven Wegener wrote: > On Sun, 26 Sep 2010, Simon Horman wrote: > > > here is an updated though as yet untested version of your patch from Julian > > to take into account recent changes. In particualr, the ip_vs_sync.c > > portion is no longer needed as only the flags in the first 16 bits are > > synced now. It applies against Patrick McHardy's nf-next-2.6 tree. > > > > You mentioned in your original post that you would work on an ipvsadm > > patch for this feature. Have you had time to do so? > > > > Also, are you in a position to test this? If not I can do so. > > Hi, > > I've tested your patch with the below addition to ipvsadm and it works > correctly. Thanks Sven, I'll push the kernel side to Patrick McHardy and assuming Wensong doesn't object, I'll push the ipvsadm side myself. > > Sven > > From: Sven Wegener <sven.wegener@stealer.net> > Subject: [PATCH] Show backlog connections > > Signed-off-by: Sven Wegener <sven.wegener@stealer.net> > --- > ipvsadm.c | 51 ++++++++++++++++++++++++++++++-------------- > libipvs/ip_vs.h | 4 +++ > libipvs/ip_vs_nl_policy.c | 1 + > libipvs/libipvs.c | 3 ++ > 4 files changed, 43 insertions(+), 16 deletions(-) > > diff --git a/ipvsadm.c b/ipvsadm.c > index 8557d21..54ae110 100644 > --- a/ipvsadm.c > +++ b/ipvsadm.c > @@ -181,7 +181,8 @@ static const char* cmdnames[] = { > #define OPT_SYNCID 0x080000 > #define OPT_EXACT 0x100000 > #define OPT_ONEPACKET 0x200000 > -#define NUMBER_OF_OPT 22 > +#define OPT_BACKLOGCONN 0x400000 > +#define NUMBER_OF_OPT 23 > > static const char* optnames[] = { > "numeric", > @@ -206,6 +207,7 @@ static const char* optnames[] = { > "syncid", > "exact", > "ops", > + "backlog-conn", > }; > > /* > @@ -218,21 +220,21 @@ static const char* optnames[] = { > */ > static const char commands_v_options[NUMBER_OF_CMD][NUMBER_OF_OPT] = > { > - /* -n -c svc -s -p -M -r fwd -w -x -y -mc tot dmn -st -rt thr -pc srt sid -ex ops */ > -/*ADD*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' '}, > -/*EDIT*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' '}, > -/*DEL*/ {'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > -/*FLUSH*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > -/*LIST*/ {' ', '1', '1', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', '1', '1', ' ', ' ', ' ', ' ', ' ', ' ', ' ', 'x'}, > -/*ADDSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > -/*DELSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > -/*EDITSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > -/*TIMEOUT*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > -/*STARTD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x'}, > -/*STOPD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x'}, > -/*RESTORE*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > -/*SAVE*/ {' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > -/*ZERO*/ {'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > + /* -n -c svc -s -p -M -r fwd -w -x -y -mc tot dmn -st -rt thr -pc srt sid -ex ops, blc */ > +/*ADD*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x'}, > +/*EDIT*/ {'x', 'x', '+', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x'}, > +/*DEL*/ {'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > +/*FLUSH*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > +/*LIST*/ {' ', '1', '1', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', '1', '1', ' ', ' ', ' ', ' ', ' ', ' ', ' ', 'x', ' '}, > +/*ADDSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > +/*DELSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > +/*EDITSRV*/ {'x', 'x', '+', 'x', 'x', 'x', '+', ' ', ' ', ' ', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > +/*TIMEOUT*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > +/*STARTD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x'}, > +/*STOPD*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', ' ', 'x', 'x', 'x'}, > +/*RESTORE*/ {'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > +/*SAVE*/ {' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > +/*ZERO*/ {'x', 'x', ' ', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x', 'x'}, > }; > > /* printing format flags */ > @@ -245,6 +247,7 @@ static const char commands_v_options[NUMBER_OF_CMD][NUMBER_OF_OPT] = > #define FMT_PERSISTENTCONN 0x0020 > #define FMT_NOSORT 0x0040 > #define FMT_EXACT 0x0080 > +#define FMT_BACKLOGCONN 0x0100 > > #define SERVICE_NONE 0x0000 > #define SERVICE_ADDR 0x0001 > @@ -282,6 +285,7 @@ enum { > TAG_PERSISTENTCONN, > TAG_SORT, > TAG_NO_SORT, > + TAG_BACKLOGCONN, > }; > > /* various parsing helpers & parsing functions */ > @@ -421,6 +425,8 @@ parse_options(int argc, char **argv, struct ipvs_command_entry *ce, > { "exact", 'X', POPT_ARG_NONE, NULL, 'X', NULL, NULL }, > { "ipv6", '6', POPT_ARG_NONE, NULL, '6', NULL, NULL }, > { "ops", 'o', POPT_ARG_NONE, NULL, 'o', NULL, NULL }, > + { "backlog-conn", '\0', POPT_ARG_NONE, NULL, > + TAG_BACKLOGCONN, NULL, NULL }, > { NULL, 0, 0, NULL, 0, NULL, NULL } > }; > > @@ -647,6 +653,10 @@ parse_options(int argc, char **argv, struct ipvs_command_entry *ce, > set_option(options, OPT_ONEPACKET); > ce->svc.flags |= IP_VS_SVC_F_ONEPACKET; > break; > + case TAG_BACKLOGCONN: > + set_option(options, OPT_BACKLOGCONN); > + *format |= FMT_BACKLOGCONN; > + break; > default: > fail(2, "invalid option `%s'", > poptBadOption(context, POPT_BADOPTION_NOALIAS)); > @@ -1396,6 +1406,11 @@ static void print_title(unsigned int format) > " -> RemoteAddress:Port\n", > "Prot LocalAddress:Port", > "Weight", "PersistConn", "ActiveConn", "InActConn"); > + else if (format & FMT_BACKLOGCONN) > + printf("%-33s %-9s %-11s %-10s %-10s\n" > + " -> RemoteAddress:Port\n", > + "Prot LocalAddress:Port", > + "Weight", "BacklogConn", "ActiveConn", "InActConn"); > else if (!(format & FMT_RULE)) > printf("Prot LocalAddress:Port Scheduler Flags\n" > " -> RemoteAddress:Port Forward Weight ActiveConn InActConn\n"); > @@ -1539,6 +1554,10 @@ print_service_entry(ipvs_service_entry_t *se, unsigned int format) > printf(" -> %-28s %-9u %-11u %-10u %-10u\n", dname, > e->weight, e->persistconns, > e->activeconns, e->inactconns); > + } else if (format & FMT_BACKLOGCONN) { > + printf(" -> %-28s %-9u %-11u %-10u %-10u\n", dname, > + e->weight, e->backlogconns, > + e->activeconns, e->inactconns); > } else > printf(" -> %-28s %-7s %-6d %-10u %-10u\n", > dname, fwd_name(e->conn_flags), > diff --git a/libipvs/ip_vs.h b/libipvs/ip_vs.h > index 843c51a..4c2c265 100644 > --- a/libipvs/ip_vs.h > +++ b/libipvs/ip_vs.h > @@ -277,6 +277,8 @@ struct ip_vs_dest_entry { > struct ip_vs_stats_user stats; > u_int16_t af; > union nf_inet_addr addr; > + > + u_int32_t backlogconns; /* backlog connections */ > }; > > /* The argument to IP_VS_SO_GET_DESTS */ > @@ -455,6 +457,8 @@ enum { > IPVS_DEST_ATTR_PERSIST_CONNS, /* persistent connections */ > > IPVS_DEST_ATTR_STATS, /* nested attribute for dest stats */ > + > + IPVS_DEST_ATTR_BACKLOG_CONNS, /* backlog connections */ > __IPVS_DEST_ATTR_MAX, > }; > > diff --git a/libipvs/ip_vs_nl_policy.c b/libipvs/ip_vs_nl_policy.c > index c80083e..d06a490 100644 > --- a/libipvs/ip_vs_nl_policy.c > +++ b/libipvs/ip_vs_nl_policy.c > @@ -40,6 +40,7 @@ struct nla_policy ipvs_dest_policy[IPVS_DEST_ATTR_MAX + 1] = { > [IPVS_DEST_ATTR_INACT_CONNS] = { .type = NLA_U32 }, > [IPVS_DEST_ATTR_PERSIST_CONNS] = { .type = NLA_U32 }, > [IPVS_DEST_ATTR_STATS] = { .type = NLA_NESTED }, > + [IPVS_DEST_ATTR_BACKLOG_CONNS] = { .type = NLA_U32 }, > }; > > struct nla_policy ipvs_stats_policy[IPVS_STATS_ATTR_MAX + 1] = { > diff --git a/libipvs/libipvs.c b/libipvs/libipvs.c > index 979d5bd..e06f9fa 100644 > --- a/libipvs/libipvs.c > +++ b/libipvs/libipvs.c > @@ -748,6 +748,8 @@ static int ipvs_dests_parse_cb(struct nl_msg *msg, void *arg) > d->entrytable[i].l_threshold = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_L_THRESH]); > d->entrytable[i].activeconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_ACTIVE_CONNS]); > d->entrytable[i].inactconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_INACT_CONNS]); > + if (dest_attrs[IPVS_DEST_ATTR_BACKLOG_CONNS]) > + d->entrytable[i].backlogconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_BACKLOG_CONNS]); > d->entrytable[i].persistconns = nla_get_u32(dest_attrs[IPVS_DEST_ATTR_PERSIST_CONNS]); > d->entrytable[i].af = d->af; > > @@ -853,6 +855,7 @@ ipvs_nl_dest_failure: > sizeof(struct ip_vs_dest_entry_kern)); > d->entrytable[i].af = AF_INET; > d->entrytable[i].addr.ip = d->entrytable[i].__addr_v4; > + d->entrytable[i].backlogconns = 0; > } > free(dk); > return d; > ^ permalink raw reply [flat|nested] 5+ messages in thread
* [patch] ipvs: Keep track of backlog connections
@ 2010-09-27 14:05 Simon Horman
0 siblings, 0 replies; 5+ messages in thread
From: Simon Horman @ 2010-09-27 14:05 UTC (permalink / raw)
To: lvs-devel, netfilter-devel, netdev
Cc: Sven Wegener, Wensong Zhang, Julian Anastasov,
Venkata Mohan Reddy Koppula, Patrick McHardy
From: Sven Wegener <sven.wegener@stealer.net>
A backlog connection is a connection that is on its way from inactive to
active. Speaking in TCP language, a connection from which we've seen the
initial SYN packet, but the three-way handshake hasn't finished yet.
These connections are expected to move to active soon. When a
destination is overloaded or isn't able to successfully establish
connections for various reasons, this count increases quickly and is an
indication for a problem.
Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
Signed-off-by: Julian Anastasov <ja@ssi.bg>
Acked-by: Simon Horman <horms@verge.net.au>
---
Patrick, please consider this for nf-next
diff -urp net-next-2.6-e548833-nfct/linux/include/linux/ip_vs.h linux/include/linux/ip_vs.h
--- net-next-2.6-e548833-nfct/linux/include/linux/ip_vs.h 2010-09-15 11:28:02.000000000 +0300
+++ linux/include/linux/ip_vs.h 2010-09-26 15:21:53.507865229 +0300
@@ -91,6 +91,7 @@
/* Flags that are not sent to backup server start from bit 16 */
#define IP_VS_CONN_F_NFCT (1 << 16) /* use netfilter conntrack */
+#define IP_VS_CONN_F_BACKLOG (1 << 17) /* backlog connection */
/* Connection flags from destination that can be changed by user space */
#define IP_VS_CONN_F_DEST_MASK (IP_VS_CONN_F_FWD_MASK | \
@@ -360,6 +361,7 @@ enum {
IPVS_DEST_ATTR_PERSIST_CONNS, /* persistent connections */
IPVS_DEST_ATTR_STATS, /* nested attribute for dest stats */
+ IPVS_DEST_ATTR_BACKLOG_CONNS, /* backlog connections */
__IPVS_DEST_ATTR_MAX,
};
diff -urp net-next-2.6-e548833-nfct/linux/include/net/ip_vs.h linux/include/net/ip_vs.h
--- net-next-2.6-e548833-nfct/linux/include/net/ip_vs.h 2010-09-16 08:53:04.000000000 +0300
+++ linux/include/net/ip_vs.h 2010-09-26 15:31:27.369865994 +0300
@@ -501,6 +501,7 @@ struct ip_vs_dest {
/* connection counters and thresholds */
atomic_t activeconns; /* active connections */
atomic_t inactconns; /* inactive connections */
+ atomic_t backlogconns; /* backlog connections */
atomic_t persistconns; /* persistent connections */
__u32 u_threshold; /* upper threshold */
__u32 l_threshold; /* lower threshold */
diff -urp net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_conn.c linux/net/netfilter/ipvs/ip_vs_conn.c
--- net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_conn.c 2010-09-15 11:14:13.000000000 +0300
+++ linux/net/netfilter/ipvs/ip_vs_conn.c 2010-09-26 15:24:48.292865793 +0300
@@ -611,6 +611,8 @@ static inline void ip_vs_unbind_dest(str
} else {
atomic_dec(&dest->activeconns);
}
+ if (cp->flags & IP_VS_CONN_F_BACKLOG)
+ atomic_dec(&dest->backlogconns);
} else {
/* It is a persistent connection/template, so decrease
the peristent connection counter */
diff -urp net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_ctl.c linux/net/netfilter/ipvs/ip_vs_ctl.c
--- net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_ctl.c 2010-09-16 08:56:34.000000000 +0300
+++ linux/net/netfilter/ipvs/ip_vs_ctl.c 2010-09-26 15:26:45.407867200 +0300
@@ -2593,6 +2593,7 @@ static const struct nla_policy ip_vs_des
[IPVS_DEST_ATTR_INACT_CONNS] = { .type = NLA_U32 },
[IPVS_DEST_ATTR_PERSIST_CONNS] = { .type = NLA_U32 },
[IPVS_DEST_ATTR_STATS] = { .type = NLA_NESTED },
+ [IPVS_DEST_ATTR_BACKLOG_CONNS] = { .type = NLA_U32 },
};
static int ip_vs_genl_fill_stats(struct sk_buff *skb, int container_type,
@@ -2840,6 +2841,8 @@ static int ip_vs_genl_fill_dest(struct s
atomic_read(&dest->activeconns));
NLA_PUT_U32(skb, IPVS_DEST_ATTR_INACT_CONNS,
atomic_read(&dest->inactconns));
+ NLA_PUT_U32(skb, IPVS_DEST_ATTR_BACKLOG_CONNS,
+ atomic_read(&dest->backlogconns));
NLA_PUT_U32(skb, IPVS_DEST_ATTR_PERSIST_CONNS,
atomic_read(&dest->persistconns));
diff -urp net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_proto_tcp.c linux/net/netfilter/ipvs/ip_vs_proto_tcp.c
--- net-next-2.6-e548833-nfct/linux/net/netfilter/ipvs/ip_vs_proto_tcp.c 2010-09-10 08:27:33.000000000 +0300
+++ linux/net/netfilter/ipvs/ip_vs_proto_tcp.c 2010-09-26 15:29:18.425865407 +0300
@@ -510,6 +510,15 @@ set_tcp_state(struct ip_vs_protocol *pp,
atomic_dec(&dest->inactconns);
cp->flags &= ~IP_VS_CONN_F_INACTIVE;
}
+ if (new_state == IP_VS_TCP_S_SYN_RECV &&
+ !(cp->flags & IP_VS_CONN_F_BACKLOG)) {
+ atomic_inc(&dest->backlogconns);
+ cp->flags |= IP_VS_CONN_F_BACKLOG;
+ } else if (new_state == IP_VS_TCP_S_ESTABLISHED &&
+ cp->flags & IP_VS_CONN_F_BACKLOG) {
+ atomic_dec(&dest->backlogconns);
+ cp->flags &= ~IP_VS_CONN_F_BACKLOG;
+ }
}
}
--
To unsubscribe from this list: send the line "unsubscribe lvs-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in threadend of thread, other threads:[~2010-09-27 14:05 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-09-26 13:31 [patch] ipvs: Keep track of backlog connections Simon Horman 2010-09-26 14:31 ` Sven Wegener 2010-09-26 14:59 ` Sven Wegener 2010-09-27 11:06 ` Simon Horman -- strict thread matches above, loose matches on Subject: below -- 2010-09-27 14:05 Simon Horman
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).