* Re: [RFC-PATCH] libiscsi dhcp handler
From: Dan Carpenter @ 2009-11-20 13:29 UTC (permalink / raw)
To: Rakesh Ranjan
Cc: David Miller, michaelc, open-iscsi, netdev, linux-kernel,
James.Bottomley, kxie
In-Reply-To: <4B01577E.2020003@chelsio.com>
On 11/16/09, Rakesh Ranjan <rakesh@chelsio.com> wrote:
> David Miller wrote:
>> From: Rakesh Ranjan <rakesh@chelsio.com>
>> Date: Mon, 16 Nov 2009 18:41:49 +0530
>>
>>> Herein attached patches to support dhcp based provisioning for iSCSI
>>> offload capable cards. I have made dhcp code as generic as possible,
>>> please go through the code. Based on the feedback I will submit final
>>> version of these patches.
>>
>> You can't really add objects to the build before the patch that
>> adds the source for that object.
>>
>
> Hi david,
>
> Fixed patch attached.
>
> Regards
> Rakesh Ranjan
>
The header file changes cause warnings in allmodconfig.
CC [M] drivers/scsi/be2iscsi/be_iscsi.o
+In file included from drivers/scsi/be2iscsi/be_iscsi.c:21:
+include/scsi/libiscsi.h:445: warning: ‘struct net_device’ declared
inside parameter list
+include/scsi/libiscsi.h:445: warning: its scope is only this
definition or declaration, which is probably not what you want
+include/scsi/libiscsi.h:447: warning: ‘struct sk_buff’ declared
inside parameter list
+include/scsi/libiscsi.h:447: warning: ‘struct net_device’ declared
inside parameter list
regards,
dan carpenter
^ permalink raw reply
* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Jarek Poplawski @ 2009-11-20 13:32 UTC (permalink / raw)
To: Eric Dumazet; +Cc: David S. Miller, Tom Herbert, Linux Netdev List
In-Reply-To: <4B05D8DC.7020907@gmail.com>
On 20-11-2009 00:46, Eric Dumazet wrote:
> Here is first version of XPS.
>
> Goal of XPS is to free TX completed skbs by the cpu that submitted the transmit.
But why?... OK, you write in another message about sock_wfree(). Then
how about users, who don't sock_wfree (routers)? Will there be any way
to disable it?
>
> Because I chose to union skb->iif with skb->sending_cpu, I chose
> to introduce a new xps_consume_skb(skb), and not generalize consume_skb() itself.
>
> This means that selected drivers must use new function to benefit from XPS
>
> Preliminary tests are quite good, especially on NUMA machines.
>
> Only NAPI drivers can use this new infrastructure (xps_consume_skb() cannot
> be called from hardirq context, only from softirq)
>
> I converted tg3 and pktgen for my tests
>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
> ---
...
> diff --git a/net/core/xps.c b/net/core/xps.c
> index e69de29..e580159 100644
> --- a/net/core/xps.c
> +++ b/net/core/xps.c
> @@ -0,0 +1,145 @@
> +/*
> + * XPS : Xmit Packet Steering
> + *
> + * TX completion packet freeing is performed on cpu that sent packet.
> + */
> +#if defined(CONFIG_SMP)
Shouldn't it be in the Makefile?
...
> +/*
> + * called at end of net_rx_action()
> + * preemption (and cpu migration/offline/online) disabled
> + */
> +void xps_flush(void)
> +{
> + int cpu, prevlen;
> + struct sk_buff_head *head = per_cpu_ptr(xps_array, smp_processor_id());
> + struct xps_pcpu_queue *q;
> + struct sk_buff *skb;
> +
> + for_each_cpu_mask_nr(cpu, __get_cpu_var(xps_cpus)) {
> + q = &per_cpu(xps_pcpu_queue, cpu);
> + if (cpu_online(cpu)) {
> + spin_lock(&q->list.lock);
This lock probably needs irq disabling: let's say 2 cpus run this at
the same time and both are interrupted with these (previously
scheduled) IPIs?
> + prevlen = skb_queue_len(&q->list);
> + skb_queue_splice_init(&head[cpu], &q->list);
> + spin_unlock(&q->list.lock);
> + /*
> + * We hope remote cpu will be fast enough to transfert
> + * this list to its completion queue before our
> + * next xps_flush() call
> + */
> + if (!prevlen)
> + __smp_call_function_single(cpu, &q->csd, 0);
> + continue;
> + }
> + /*
> + * ok, we must free these skbs, even if we tried to avoid it :)
> + */
> + while ((skb = __skb_dequeue(&head[cpu])) != NULL)
> + __kfree_skb(skb);
> + }
> + cpus_clear(__get_cpu_var(xps_cpus));
> +}
> +
> +/*
> + * called from hardirq (IPI) context
> + */
> +static void remote_free_skb_list(void *arg)
> +{
> + struct sk_buff *last;
> + struct softnet_data *sd;
> + struct xps_pcpu_queue *q = arg; /* &__get_cpu_var(xps_pcpu_queue); */
> +
> + spin_lock(&q->list.lock);
> +
> + last = q->list.prev;
Is q->list handled in case this cpu goes down before this IPI is
triggered?
Jarek P.
> + sd = &__get_cpu_var(softnet_data);
> + last->next = sd->completion_queue;
> + sd->completion_queue = q->list.next;
> + __skb_queue_head_init(&q->list);
> +
> + spin_unlock(&q->list.lock);
> +
> + raise_softirq_irqoff(NET_TX_SOFTIRQ);
> +}
...
^ permalink raw reply
* [PATCH] ixgbe: move tc variable to CONFIG_IXGBE_DCB
From: Jaswinder Singh Rajput @ 2009-11-20 14:02 UTC (permalink / raw)
To: David Miller, Yi Zou, Peter P Waskiewicz Jr, Jeff Kirsher, netdev
tc is required by CONFIG_IXGBE_DCB.
This also fixes compilation warning:
drivers/net/ixgbe/ixgbe_main.c: In function ‘ixgbe_tx_is_paused’:
drivers/net/ixgbe/ixgbe_main.c:245: warning: unused variable ‘tc’
Signed-off-by: Jaswinder Singh Rajput <jaswinderrajput@gmail.com>
---
drivers/net/ixgbe/ixgbe_main.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ixgbe/ixgbe_main.c b/drivers/net/ixgbe/ixgbe_main.c
index ebcec30..61a02ce 100644
--- a/drivers/net/ixgbe/ixgbe_main.c
+++ b/drivers/net/ixgbe/ixgbe_main.c
@@ -242,11 +242,11 @@ static void ixgbe_unmap_and_free_tx_resource(struct ixgbe_adapter *adapter,
static inline bool ixgbe_tx_is_paused(struct ixgbe_adapter *adapter,
struct ixgbe_ring *tx_ring)
{
- int tc;
u32 txoff = IXGBE_TFCS_TXOFF;
#ifdef CONFIG_IXGBE_DCB
if (adapter->flags & IXGBE_FLAG_DCB_ENABLED) {
+ int tc;
int reg_idx = tx_ring->reg_idx;
int dcb_i = adapter->ring_feature[RING_F_DCB].indices;
--
1.6.5.3
^ permalink raw reply related
* [net-next-2.6 PATCH v7 0/7 RFC] TCPCT part 1: cookie option exchange
From: William Allen Simpson @ 2009-11-20 14:04 UTC (permalink / raw)
To: Linux Kernel Network Developers
Initiator Responder
========= =========
<SYN> ->
Cookie
<- <SYN,ACK(SYN)>
Cookie
optional data
V7:
Updated to patch cleanly with recent large commits.
Compiled and lightly tested. Awaiting performance testing results.
Revised RCU spinlocks, based on review by Eric and Joe.
Trivial savings of 2 if tests, based on review by Joe.
Cosmetic changes to comment blocks, based on review by Eric.
Processing optional SYNACK data removed until later.
V6:
The new TCP_MSS_* symbols are now in an entirely separate patch. Applied.
Updated and reorganized to split the code base into several pieces, as
requested by Eric. No significant changes.
...
^ permalink raw reply
* [net-next-2.6 PATCH v7 1/7 RFC] TCPCT part 1a: add request_values parameter for sending SYNACK
From: William Allen Simpson @ 2009-11-20 14:12 UTC (permalink / raw)
To: Linux Kernel Network Developers
In-Reply-To: <4B06A1FF.8000202@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]
Add optional function parameters associated with sending SYNACK.
These parameters are not needed after sending SYNACK, and are not
used for retransmission. Avoids extending struct tcp_request_sock,
and avoids allocating kernel memory.
Also affects DCCP as it uses common struct request_sock_ops,
but this parameter is currently reserved for future use.
Signed-off-by: William.Allen.Simpson@gmail.com
Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
---
include/net/request_sock.h | 8 +++++++-
include/net/tcp.h | 3 ++-
net/dccp/ipv4.c | 5 +++--
net/dccp/ipv6.c | 5 +++--
net/dccp/minisocks.c | 2 +-
net/ipv4/inet_connection_sock.c | 2 +-
net/ipv4/tcp_ipv4.c | 19 +++++++++++--------
net/ipv4/tcp_minisocks.c | 2 +-
net/ipv4/tcp_output.c | 3 ++-
net/ipv6/tcp_ipv6.c | 28 +++++++++++++---------------
10 files changed, 44 insertions(+), 33 deletions(-)
[-- Attachment #2: TCPCT+1a7.patch --]
[-- Type: text/plain, Size: 8878 bytes --]
diff --git a/include/net/request_sock.h b/include/net/request_sock.h
index c719084..c9b50eb 100644
--- a/include/net/request_sock.h
+++ b/include/net/request_sock.h
@@ -27,13 +27,19 @@ struct sk_buff;
struct dst_entry;
struct proto;
+/* empty to "strongly type" an otherwise void parameter.
+ */
+struct request_values {
+};
+
struct request_sock_ops {
int family;
int obj_size;
struct kmem_cache *slab;
char *slab_name;
int (*rtx_syn_ack)(struct sock *sk,
- struct request_sock *req);
+ struct request_sock *req,
+ struct request_values *rvp);
void (*send_ack)(struct sock *sk, struct sk_buff *skb,
struct request_sock *req);
void (*send_reset)(struct sock *sk,
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 325bfcf..ec183fd 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -440,7 +440,8 @@ extern int tcp_connect(struct sock *sk);
extern struct sk_buff * tcp_make_synack(struct sock *sk,
struct dst_entry *dst,
- struct request_sock *req);
+ struct request_sock *req,
+ struct request_values *rvp);
extern int tcp_disconnect(struct sock *sk, int flags);
diff --git a/net/dccp/ipv4.c b/net/dccp/ipv4.c
index 2423a08..efbcfdc 100644
--- a/net/dccp/ipv4.c
+++ b/net/dccp/ipv4.c
@@ -477,7 +477,8 @@ static struct dst_entry* dccp_v4_route_skb(struct net *net, struct sock *sk,
return &rt->u.dst;
}
-static int dccp_v4_send_response(struct sock *sk, struct request_sock *req)
+static int dccp_v4_send_response(struct sock *sk, struct request_sock *req,
+ struct request_values *rv_unused)
{
int err = -1;
struct sk_buff *skb;
@@ -626,7 +627,7 @@ int dccp_v4_conn_request(struct sock *sk, struct sk_buff *skb)
dreq->dreq_iss = dccp_v4_init_sequence(skb);
dreq->dreq_service = service;
- if (dccp_v4_send_response(sk, req))
+ if (dccp_v4_send_response(sk, req, NULL))
goto drop_and_free;
inet_csk_reqsk_queue_hash_add(sk, req, DCCP_TIMEOUT_INIT);
diff --git a/net/dccp/ipv6.c b/net/dccp/ipv6.c
index 50ea91a..6574215 100644
--- a/net/dccp/ipv6.c
+++ b/net/dccp/ipv6.c
@@ -241,7 +241,8 @@ out:
}
-static int dccp_v6_send_response(struct sock *sk, struct request_sock *req)
+static int dccp_v6_send_response(struct sock *sk, struct request_sock *req,
+ struct request_values *rv_unused)
{
struct inet6_request_sock *ireq6 = inet6_rsk(req);
struct ipv6_pinfo *np = inet6_sk(sk);
@@ -468,7 +469,7 @@ static int dccp_v6_conn_request(struct sock *sk, struct sk_buff *skb)
dreq->dreq_iss = dccp_v6_init_sequence(skb);
dreq->dreq_service = service;
- if (dccp_v6_send_response(sk, req))
+ if (dccp_v6_send_response(sk, req, NULL))
goto drop_and_free;
inet6_csk_reqsk_queue_hash_add(sk, req, DCCP_TIMEOUT_INIT);
diff --git a/net/dccp/minisocks.c b/net/dccp/minisocks.c
index 5ca49ce..af226a0 100644
--- a/net/dccp/minisocks.c
+++ b/net/dccp/minisocks.c
@@ -184,7 +184,7 @@ struct sock *dccp_check_req(struct sock *sk, struct sk_buff *skb,
* counter (backoff, monitored by dccp_response_timer).
*/
req->retrans++;
- req->rsk_ops->rtx_syn_ack(sk, req);
+ req->rsk_ops->rtx_syn_ack(sk, req, NULL);
}
/* Network Duplicate, discard packet */
return NULL;
diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c
index 26fb50e..ad098d6 100644
--- a/net/ipv4/inet_connection_sock.c
+++ b/net/ipv4/inet_connection_sock.c
@@ -531,7 +531,7 @@ void inet_csk_reqsk_queue_prune(struct sock *parent,
&expire, &resend);
if (!expire &&
(!resend ||
- !req->rsk_ops->rtx_syn_ack(parent, req) ||
+ !req->rsk_ops->rtx_syn_ack(parent, req, NULL) ||
inet_rsk(req)->acked)) {
unsigned long timeo;
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index df18ce0..397ab8f 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -742,8 +742,9 @@ static void tcp_v4_reqsk_send_ack(struct sock *sk, struct sk_buff *skb,
* This still operates on a request_sock only, not on a big
* socket.
*/
-static int __tcp_v4_send_synack(struct sock *sk, struct request_sock *req,
- struct dst_entry *dst)
+static int __tcp_v4_send_synack(struct sock *sk, struct dst_entry *dst,
+ struct request_sock *req,
+ struct request_values *rvp)
{
const struct inet_request_sock *ireq = inet_rsk(req);
int err = -1;
@@ -753,7 +754,7 @@ static int __tcp_v4_send_synack(struct sock *sk, struct request_sock *req,
if (!dst && (dst = inet_csk_route_req(sk, req)) == NULL)
return -1;
- skb = tcp_make_synack(sk, dst, req);
+ skb = tcp_make_synack(sk, dst, req, rvp);
if (skb) {
struct tcphdr *th = tcp_hdr(skb);
@@ -774,9 +775,10 @@ static int __tcp_v4_send_synack(struct sock *sk, struct request_sock *req,
return err;
}
-static int tcp_v4_send_synack(struct sock *sk, struct request_sock *req)
+static int tcp_v4_send_synack(struct sock *sk, struct request_sock *req,
+ struct request_values *rvp)
{
- return __tcp_v4_send_synack(sk, req, NULL);
+ return __tcp_v4_send_synack(sk, NULL, req, rvp);
}
/*
@@ -1211,13 +1213,13 @@ static struct timewait_sock_ops tcp_timewait_sock_ops = {
int tcp_v4_conn_request(struct sock *sk, struct sk_buff *skb)
{
- struct inet_request_sock *ireq;
struct tcp_options_received tmp_opt;
struct request_sock *req;
+ struct inet_request_sock *ireq;
+ struct dst_entry *dst = NULL;
__be32 saddr = ip_hdr(skb)->saddr;
__be32 daddr = ip_hdr(skb)->daddr;
__u32 isn = TCP_SKB_CB(skb)->when;
- struct dst_entry *dst = NULL;
#ifdef CONFIG_SYN_COOKIES
int want_cookie = 0;
#else
@@ -1337,7 +1339,8 @@ int tcp_v4_conn_request(struct sock *sk, struct sk_buff *skb)
}
tcp_rsk(req)->snt_isn = isn;
- if (__tcp_v4_send_synack(sk, req, dst) || want_cookie)
+ if (__tcp_v4_send_synack(sk, dst, req, NULL)
+ || want_cookie)
goto drop_and_free;
inet_csk_reqsk_queue_hash_add(sk, req, TCP_TIMEOUT_INIT);
diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c
index 4be2228..7a42990 100644
--- a/net/ipv4/tcp_minisocks.c
+++ b/net/ipv4/tcp_minisocks.c
@@ -537,7 +537,7 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
* Enforce "SYN-ACK" according to figure 8, figure 6
* of RFC793, fixed by RFC1122.
*/
- req->rsk_ops->rtx_syn_ack(sk, req);
+ req->rsk_ops->rtx_syn_ack(sk, req, NULL);
return NULL;
}
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 616c686..1151cb8 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -2224,7 +2224,8 @@ int tcp_send_synack(struct sock *sk)
/* Prepare a SYN-ACK. */
struct sk_buff *tcp_make_synack(struct sock *sk, struct dst_entry *dst,
- struct request_sock *req)
+ struct request_sock *req,
+ struct request_values *rvp)
{
struct inet_request_sock *ireq = inet_rsk(req);
struct tcp_sock *tp = tcp_sk(sk);
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index de70909..3e327bc 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -461,7 +461,8 @@ out:
}
-static int tcp_v6_send_synack(struct sock *sk, struct request_sock *req)
+static int tcp_v6_send_synack(struct sock *sk, struct request_sock *req,
+ struct request_values *rvp)
{
struct inet6_request_sock *treq = inet6_rsk(req);
struct ipv6_pinfo *np = inet6_sk(sk);
@@ -499,7 +500,7 @@ static int tcp_v6_send_synack(struct sock *sk, struct request_sock *req)
if ((err = xfrm_lookup(sock_net(sk), &dst, &fl, sk, 0)) < 0)
goto done;
- skb = tcp_make_synack(sk, dst, req);
+ skb = tcp_make_synack(sk, dst, req, rvp);
if (skb) {
struct tcphdr *th = tcp_hdr(skb);
@@ -1161,13 +1162,13 @@ static struct sock *tcp_v6_hnd_req(struct sock *sk,struct sk_buff *skb)
*/
static int tcp_v6_conn_request(struct sock *sk, struct sk_buff *skb)
{
+ struct tcp_options_received tmp_opt;
+ struct request_sock *req;
struct inet6_request_sock *treq;
struct ipv6_pinfo *np = inet6_sk(sk);
- struct tcp_options_received tmp_opt;
struct tcp_sock *tp = tcp_sk(sk);
- struct request_sock *req = NULL;
- __u32 isn = TCP_SKB_CB(skb)->when;
struct dst_entry *dst = __sk_dst_get(sk);
+ __u32 isn = TCP_SKB_CB(skb)->when;
#ifdef CONFIG_SYN_COOKIES
int want_cookie = 0;
#else
@@ -1239,23 +1240,20 @@ static int tcp_v6_conn_request(struct sock *sk, struct sk_buff *skb)
isn = tcp_v6_init_sequence(skb);
}
-
tcp_rsk(req)->snt_isn = isn;
security_inet_conn_request(sk, skb, req);
- if (tcp_v6_send_synack(sk, req))
- goto drop;
+ if (tcp_v6_send_synack(sk, req, NULL)
+ || want_cookie)
+ goto drop_and_free;
- if (!want_cookie) {
- inet6_csk_reqsk_queue_hash_add(sk, req, TCP_TIMEOUT_INIT);
- return 0;
- }
+ inet6_csk_reqsk_queue_hash_add(sk, req, TCP_TIMEOUT_INIT);
+ return 0;
+drop_and_free:
+ reqsk_free(req);
drop:
- if (req)
- reqsk_free(req);
-
return 0; /* don't send reset */
}
--
1.6.3.3
^ permalink raw reply related
* [net-next-2.6 PATCH v7 2/7 RFC] TCPCT part 1b: generate Responder Cookie secret
From: William Allen Simpson @ 2009-11-20 14:23 UTC (permalink / raw)
To: Linux Kernel Network Developers
In-Reply-To: <4B06A1FF.8000202@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 876 bytes --]
Define (missing) hash message size for SHA1.
Define hashing size constants specific to TCP cookies.
Add new function: tcp_cookie_generator().
Maintain global secret values for tcp_cookie_generator().
This is a significantly revised implementation of earlier (15-year-old)
Photuris [RFC-2522] code for the KA9Q cooperative multitasking platform.
Linux RCU technique appears to be well-suited to this application, though
neither of the circular queue items are freed.
These functions will also be used in subsequent patches that implement
additional features.
Signed-off-by: William.Allen.Simpson@gmail.com
Acked-by: Eric Dumazet <eric.dumazet@gmail.com>
---
include/linux/cryptohash.h | 1 +
include/net/tcp.h | 8 +++
net/ipv4/tcp.c | 140 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 149 insertions(+), 0 deletions(-)
[-- Attachment #2: TCPCT+1b7.patch --]
[-- Type: text/plain, Size: 6811 bytes --]
diff --git a/include/linux/cryptohash.h b/include/linux/cryptohash.h
index c118b2a..ec78a4b 100644
--- a/include/linux/cryptohash.h
+++ b/include/linux/cryptohash.h
@@ -2,6 +2,7 @@
#define __CRYPTOHASH_H
#define SHA_DIGEST_WORDS 5
+#define SHA_MESSAGE_BYTES (512 /*bits*/ / 8)
#define SHA_WORKSPACE_WORDS 80
void sha_init(__u32 *buf);
diff --git a/include/net/tcp.h b/include/net/tcp.h
index ec183fd..4a99a8e 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -1478,6 +1478,14 @@ struct tcp_request_sock_ops {
#endif
};
+/* Using SHA1 for now, define some constants.
+ */
+#define COOKIE_DIGEST_WORDS (SHA_DIGEST_WORDS)
+#define COOKIE_MESSAGE_WORDS (SHA_MESSAGE_BYTES / 4)
+#define COOKIE_WORKSPACE_WORDS (COOKIE_DIGEST_WORDS + COOKIE_MESSAGE_WORDS)
+
+extern int tcp_cookie_generator(u32 *bakery);
+
extern void tcp_v4_init(void);
extern void tcp_init(void);
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 524f976..1356e3d 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -264,6 +264,7 @@
#include <linux/cache.h>
#include <linux/err.h>
#include <linux/crypto.h>
+#include <linux/time.h>
#include <net/icmp.h>
#include <net/tcp.h>
@@ -2847,6 +2848,135 @@ EXPORT_SYMBOL(tcp_md5_hash_key);
#endif
+/**
+ * Each Responder maintains up to two secret values concurrently for
+ * efficient secret rollover. Each secret value has 4 states:
+ *
+ * Generating. (tcp_secret_generating != tcp_secret_primary)
+ * Generates new Responder-Cookies, but not yet used for primary
+ * verification. This is a short-term state, typically lasting only
+ * one round trip time (RTT).
+ *
+ * Primary. (tcp_secret_generating == tcp_secret_primary)
+ * Used both for generation and primary verification.
+ *
+ * Retiring. (tcp_secret_retiring != tcp_secret_secondary)
+ * Used for verification, until the first failure that can be
+ * verified by the newer Generating secret. At that time, this
+ * cookie's state is changed to Secondary, and the Generating
+ * cookie's state is changed to Primary. This is a short-term state,
+ * typically lasting only one round trip time (RTT).
+ *
+ * Secondary. (tcp_secret_retiring == tcp_secret_secondary)
+ * Used for secondary verification, after primary verification
+ * failures. This state lasts no more than twice the Maximum Segment
+ * Lifetime (2MSL). Then, the secret is discarded.
+ */
+struct tcp_cookie_secret {
+ /* The secret is divided into two parts. The digest part is the
+ * equivalent of previously hashing a secret and saving the state,
+ * and serves as an initialization vector (IV). The message part
+ * serves as the trailing secret.
+ */
+ u32 secrets[COOKIE_WORKSPACE_WORDS];
+ unsigned long expires;
+};
+
+#define TCP_SECRET_1MSL (HZ * TCP_PAWS_MSL)
+#define TCP_SECRET_2MSL (HZ * TCP_PAWS_MSL * 2)
+#define TCP_SECRET_LIFE (HZ * 600)
+
+static struct tcp_cookie_secret tcp_secret_one;
+static struct tcp_cookie_secret tcp_secret_two;
+
+/* Essentially a circular list, without dynamic allocation. */
+static struct tcp_cookie_secret *tcp_secret_generating;
+static struct tcp_cookie_secret *tcp_secret_primary;
+static struct tcp_cookie_secret *tcp_secret_retiring;
+static struct tcp_cookie_secret *tcp_secret_secondary;
+
+static DEFINE_SPINLOCK(tcp_secret_locker);
+
+/* Select a pseudo-random word in the cookie workspace.
+ */
+static inline u32 tcp_cookie_work(const u32 *ws, const int n)
+{
+ return ws[COOKIE_DIGEST_WORDS + ((COOKIE_MESSAGE_WORDS-1) & ws[n])];
+}
+
+/* Fill bakery[COOKIE_WORKSPACE_WORDS] with generator, updating as needed.
+ * Called in softirq context.
+ * Returns: 0 for success.
+ */
+int tcp_cookie_generator(u32 *bakery)
+{
+ unsigned long jiffy = jiffies;
+
+ if (unlikely(time_after_eq(jiffy, tcp_secret_generating->expires))) {
+ spin_lock_bh(&tcp_secret_locker);
+ if (!time_after_eq(jiffy, tcp_secret_generating->expires)) {
+ /* refreshed by another */
+ memcpy(bakery,
+ &tcp_secret_generating->secrets[0],
+ COOKIE_WORKSPACE_WORDS);
+ } else {
+ /* still needs refreshing */
+ get_random_bytes(bakery, COOKIE_WORKSPACE_WORDS);
+
+ /* The first time, paranoia assumes that the
+ * randomization function isn't as strong. But,
+ * this secret initialization is delayed until
+ * the last possible moment (packet arrival).
+ * Although that time is observable, it is
+ * unpredictably variable. Mash in the most
+ * volatile clock bits available, and expire the
+ * secret extra quickly.
+ */
+ if (unlikely(tcp_secret_primary->expires ==
+ tcp_secret_secondary->expires)) {
+ struct timespec tv;
+
+ getnstimeofday(&tv);
+ bakery[COOKIE_DIGEST_WORDS+0] ^=
+ (u32)tv.tv_nsec;
+
+ tcp_secret_secondary->expires = jiffy
+ + TCP_SECRET_1MSL
+ + (0x0f & tcp_cookie_work(bakery, 0));
+ } else {
+ tcp_secret_secondary->expires = jiffy
+ + TCP_SECRET_LIFE
+ + (0xff & tcp_cookie_work(bakery, 1));
+ tcp_secret_primary->expires = jiffy
+ + TCP_SECRET_2MSL
+ + (0x1f & tcp_cookie_work(bakery, 2));
+ }
+ memcpy(&tcp_secret_secondary->secrets[0],
+ bakery, COOKIE_WORKSPACE_WORDS);
+
+ rcu_assign_pointer(tcp_secret_generating,
+ tcp_secret_secondary);
+ rcu_assign_pointer(tcp_secret_retiring,
+ tcp_secret_primary);
+ /*
+ * Neither call_rcu() nor synchronize_rcu() needed.
+ * Retiring data is not freed. It is replaced after
+ * further (locked) pointer updates, and a quiet time
+ * (minimum 1MSL, maximum LIFE - 2MSL).
+ */
+ }
+ spin_unlock_bh(&tcp_secret_locker);
+ } else {
+ rcu_read_lock_bh();
+ memcpy(bakery,
+ &rcu_dereference(tcp_secret_generating)->secrets[0],
+ COOKIE_WORKSPACE_WORDS);
+ rcu_read_unlock_bh();
+ }
+ return 0;
+}
+EXPORT_SYMBOL(tcp_cookie_generator);
+
void tcp_done(struct sock *sk)
{
if (sk->sk_state == TCP_SYN_SENT || sk->sk_state == TCP_SYN_RECV)
@@ -2881,6 +3011,7 @@ void __init tcp_init(void)
struct sk_buff *skb = NULL;
unsigned long nr_pages, limit;
int order, i, max_share;
+ unsigned long jiffy = jiffies;
BUILD_BUG_ON(sizeof(struct tcp_skb_cb) > sizeof(skb->cb));
@@ -2974,6 +3105,15 @@ void __init tcp_init(void)
tcp_hashinfo.ehash_mask + 1, tcp_hashinfo.bhash_size);
tcp_register_congestion_control(&tcp_reno);
+
+ memset(&tcp_secret_one.secrets[0], 0, sizeof(tcp_secret_one.secrets));
+ memset(&tcp_secret_two.secrets[0], 0, sizeof(tcp_secret_two.secrets));
+ tcp_secret_one.expires = jiffy; /* past due */
+ tcp_secret_two.expires = jiffy; /* past due */
+ tcp_secret_generating = &tcp_secret_one;
+ tcp_secret_primary = &tcp_secret_one;
+ tcp_secret_retiring = &tcp_secret_two;
+ tcp_secret_secondary = &tcp_secret_two;
}
EXPORT_SYMBOL(tcp_close);
--
1.6.3.3
^ permalink raw reply related
* Re: [PATCH] ixgbe: move tc variable to CONFIG_IXGBE_DCB
From: Jeff Kirsher @ 2009-11-20 14:24 UTC (permalink / raw)
To: Jaswinder Singh Rajput
Cc: David Miller, Yi Zou, Peter P Waskiewicz Jr, netdev, LKML
In-Reply-To: <1258725747.3562.5.camel@ht.satnam>
On Fri, Nov 20, 2009 at 06:02, Jaswinder Singh Rajput
<jaswinder@kernel.org> wrote:
>
> tc is required by CONFIG_IXGBE_DCB.
> This also fixes compilation warning:
>
> drivers/net/ixgbe/ixgbe_main.c: In function ‘ixgbe_tx_is_paused’:
> drivers/net/ixgbe/ixgbe_main.c:245: warning: unused variable ‘tc’
>
> Signed-off-by: Jaswinder Singh Rajput <jaswinderrajput@gmail.com>
> ---
> drivers/net/ixgbe/ixgbe_main.c | 2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/drivers/net/ixgbe/ixgbe_main.c b/drivers/net/ixgbe/ixgbe_main.c
> index ebcec30..61a02ce 100644
> --- a/drivers/net/ixgbe/ixgbe_main.c
> +++ b/drivers/net/ixgbe/ixgbe_main.c
> @@ -242,11 +242,11 @@ static void ixgbe_unmap_and_free_tx_resource(struct ixgbe_adapter *adapter,
> static inline bool ixgbe_tx_is_paused(struct ixgbe_adapter *adapter,
> struct ixgbe_ring *tx_ring)
> {
> - int tc;
> u32 txoff = IXGBE_TFCS_TXOFF;
>
> #ifdef CONFIG_IXGBE_DCB
> if (adapter->flags & IXGBE_FLAG_DCB_ENABLED) {
> + int tc;
> int reg_idx = tx_ring->reg_idx;
> int dcb_i = adapter->ring_feature[RING_F_DCB].indices;
>
> --
> 1.6.5.3
>
>
> --
I just submitted a patch to fix this. So NAK.
--
Cheers,
Jeff
^ permalink raw reply
* [net-next-2.6 PATCH v7 3/7 RFC] TCPCT part 1c: sysctl_tcp_cookie_size, socket option TCP_COOKIE_TRANSACTIONS
From: William Allen Simpson @ 2009-11-20 14:33 UTC (permalink / raw)
To: Linux Kernel Network Developers
In-Reply-To: <4B06A1FF.8000202@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1190 bytes --]
Define sysctl (tcp_cookie_size) to turn on and off the cookie option
default globally, instead of a compiled configuration option.
Define per socket option (TCP_COOKIE_TRANSACTIONS) for setting constant
data values, retrieving variable cookie values, and other facilities.
Move inline tcp_clear_options() unchanged from net/tcp.h to linux/tcp.h,
near its corresponding struct tcp_options_received (prior to changes).
This is a straightforward re-implementation of an earlier (year-old)
patch that no longer applies cleanly, with permission of the original
author (Adam Langley):
http://thread.gmane.org/gmane.linux.network/102586
These functions will also be used in subsequent patches that implement
additional features.
Requires:
net: TCP_MSS_DEFAULT, TCP_MSS_DESIRED
Signed-off-by: William.Allen.Simpson@gmail.com
---
Documentation/networking/ip-sysctl.txt | 8 +++++++
include/linux/tcp.h | 33 +++++++++++++++++++++++++++++++-
include/net/tcp.h | 6 +----
net/ipv4/sysctl_net_ipv4.c | 8 +++++++
net/ipv4/tcp_output.c | 8 +++++++
5 files changed, 57 insertions(+), 6 deletions(-)
[-- Attachment #2: TCPCT+1c7.patch --]
[-- Type: text/plain, Size: 5141 bytes --]
diff --git a/Documentation/networking/ip-sysctl.txt b/Documentation/networking/ip-sysctl.txt
index a0e134d..820dd4b 100644
--- a/Documentation/networking/ip-sysctl.txt
+++ b/Documentation/networking/ip-sysctl.txt
@@ -164,6 +164,14 @@ tcp_congestion_control - STRING
additional choices may be available based on kernel configuration.
Default is set as part of kernel configuration.
+tcp_cookie_size - INTEGER
+ Default size of TCP Cookie Transactions (TCPCT) option, that may be
+ overridden on a per socket basis by the TCPCT socket option.
+ Values greater than the maximum (16) are interpreted as the maximum.
+ Values greater than zero and less than the minimum (8) are interpreted
+ as the minimum. Odd values are interpreted as the next even value.
+ Default: 0 (off).
+
tcp_dsack - BOOLEAN
Allows TCP to send "duplicate" SACKs.
diff --git a/include/linux/tcp.h b/include/linux/tcp.h
index 32d7d77..eaa3113 100644
--- a/include/linux/tcp.h
+++ b/include/linux/tcp.h
@@ -102,7 +102,9 @@ enum {
#define TCP_QUICKACK 12 /* Block/reenable quick acks */
#define TCP_CONGESTION 13 /* Congestion control algorithm */
#define TCP_MD5SIG 14 /* TCP MD5 Signature (RFC2385) */
+#define TCP_COOKIE_TRANSACTIONS 15 /* TCP Cookie Transactions */
+/* for TCP_INFO socket option */
#define TCPI_OPT_TIMESTAMPS 1
#define TCPI_OPT_SACK 2
#define TCPI_OPT_WSCALE 4
@@ -174,6 +176,30 @@ struct tcp_md5sig {
__u8 tcpm_key[TCP_MD5SIG_MAXKEYLEN]; /* key (binary) */
};
+/* for TCP_COOKIE_TRANSACTIONS (TCPCT) socket option */
+#define TCP_COOKIE_MIN 8 /* 64-bits */
+#define TCP_COOKIE_MAX 16 /* 128-bits */
+#define TCP_COOKIE_PAIR_SIZE (2*TCP_COOKIE_MAX)
+
+/* Flags for both getsockopt and setsockopt */
+#define TCP_COOKIE_IN_ALWAYS (1 << 0) /* Discard SYN without cookie */
+#define TCP_COOKIE_OUT_NEVER (1 << 1) /* Prohibit outgoing cookies,
+ * supercedes everything. */
+
+/* Flags for getsockopt */
+#define TCP_S_DATA_IN (1 << 2) /* Was data received? */
+#define TCP_S_DATA_OUT (1 << 3) /* Was data sent? */
+
+/* TCP_COOKIE_TRANSACTIONS data */
+struct tcp_cookie_transactions {
+ __u16 tcpct_flags; /* see above */
+ __u8 __tcpct_pad1; /* zero */
+ __u8 tcpct_cookie_desired; /* bytes */
+ __u16 tcpct_s_data_desired; /* bytes of variable data */
+ __u16 tcpct_used; /* bytes in value */
+ __u8 tcpct_value[TCP_MSS_DEFAULT];
+};
+
#ifdef __KERNEL__
#include <linux/skbuff.h>
@@ -227,6 +253,11 @@ struct tcp_options_received {
u16 mss_clamp; /* Maximal mss, negotiated at connection setup */
};
+static inline void tcp_clear_options(struct tcp_options_received *rx_opt)
+{
+ rx_opt->tstamp_ok = rx_opt->sack_ok = rx_opt->wscale_ok = rx_opt->snd_wscale = 0;
+}
+
/* This is the max number of SACKS that we'll generate and process. It's safe
* to increse this, although since:
* size = TCPOLEN_SACK_BASE_ALIGNED (4) + n * TCPOLEN_SACK_PERBLOCK (8)
@@ -435,6 +466,6 @@ static inline struct tcp_timewait_sock *tcp_twsk(const struct sock *sk)
return (struct tcp_timewait_sock *)sk;
}
-#endif
+#endif /* __KERNEL__ */
#endif /* _LINUX_TCP_H */
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 4a99a8e..738b65f 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -234,6 +234,7 @@ extern int sysctl_tcp_base_mss;
extern int sysctl_tcp_workaround_signed_windows;
extern int sysctl_tcp_slow_start_after_idle;
extern int sysctl_tcp_max_ssthresh;
+extern int sysctl_tcp_cookie_size;
extern atomic_t tcp_memory_allocated;
extern struct percpu_counter tcp_sockets_allocated;
@@ -340,11 +341,6 @@ static inline void tcp_dec_quickack_mode(struct sock *sk,
extern void tcp_enter_quickack_mode(struct sock *sk);
-static inline void tcp_clear_options(struct tcp_options_received *rx_opt)
-{
- rx_opt->tstamp_ok = rx_opt->sack_ok = rx_opt->wscale_ok = rx_opt->snd_wscale = 0;
-}
-
#define TCP_ECN_OK 1
#define TCP_ECN_QUEUE_CWR 2
#define TCP_ECN_DEMAND_CWR 4
diff --git a/net/ipv4/sysctl_net_ipv4.c b/net/ipv4/sysctl_net_ipv4.c
index 2dcf04d..3422c54 100644
--- a/net/ipv4/sysctl_net_ipv4.c
+++ b/net/ipv4/sysctl_net_ipv4.c
@@ -714,6 +714,14 @@ static struct ctl_table ipv4_table[] = {
},
{
.ctl_name = CTL_UNNUMBERED,
+ .procname = "tcp_cookie_size",
+ .data = &sysctl_tcp_cookie_size,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec
+ },
+ {
+ .ctl_name = CTL_UNNUMBERED,
.procname = "udp_mem",
.data = &sysctl_udp_mem,
.maxlen = sizeof(sysctl_udp_mem),
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 1151cb8..e59fa5a 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -59,6 +59,14 @@ int sysctl_tcp_base_mss __read_mostly = 512;
/* By default, RFC2861 behavior. */
int sysctl_tcp_slow_start_after_idle __read_mostly = 1;
+#ifdef CONFIG_SYSCTL
+/* By default, let the user enable it. */
+int sysctl_tcp_cookie_size __read_mostly = 0;
+#else
+int sysctl_tcp_cookie_size __read_mostly = TCP_COOKIE_MAX;
+#endif
+
+
/* Account for new data that has been sent to the network. */
static void tcp_event_new_data_sent(struct sock *sk, struct sk_buff *skb)
{
--
1.6.3.3
^ permalink raw reply related
* [net-next-2.6 PATCH v7 4/7 RFC] TCPCT part 1d: define TCP cookie option, extend existing struct's
From: William Allen Simpson @ 2009-11-20 14:38 UTC (permalink / raw)
To: Linux Kernel Network Developers
In-Reply-To: <4B06A1FF.8000202@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1860 bytes --]
Data structures are carefully composed to require minimal additions.
For example, the struct tcp_options_received cookie_plus variable fits
between existing 16-bit and 8-bit variables, requiring no additional
space (taking alignment into consideration). There are no additions to
tcp_request_sock, and only 1 pointer in tcp_sock.
This is a significantly revised implementation of an earlier (year-old)
patch that no longer applies cleanly, with permission of the original
author (Adam Langley):
http://thread.gmane.org/gmane.linux.network/102586
The principle difference is using a TCP option to carry the cookie nonce,
instead of a user configured offset in the data. This is more flexible and
less subject to user configuration error. Such a cookie option has been
suggested for many years, and is also useful without SYN data, allowing
several related concepts to use the same extension option.
"Re: SYN floods (was: does history repeat itself?)", September 9, 1996.
http://www.merit.net/mail.archives/nanog/1996-09/msg00235.html
"Re: what a new TCP header might look like", May 12, 1998.
ftp://ftp.isi.edu/end2end/end2end-interest-1998.mail
These functions will also be used in subsequent patches that implement
additional features.
Requires:
TCPCT part 1a: add request_values parameter for sending SYNACK
TCPCT part 1b: generate Responder Cookie secret
TCPCT part 1c: sysctl_tcp_cookie_size, socket option TCP_COOKIE_TRANSACTIONS
Signed-off-by: William.Allen.Simpson@gmail.com
---
include/linux/tcp.h | 29 +++++++++++++---
include/net/tcp.h | 83 ++++++++++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_ipv4.c | 20 +++++++++++
net/ipv4/tcp_minisocks.c | 46 +++++++++++++++++++++----
net/ipv6/tcp_ipv6.c | 13 +++++++
5 files changed, 177 insertions(+), 14 deletions(-)
[-- Attachment #2: TCPCT+1d7.patch --]
[-- Type: text/plain, Size: 10961 bytes --]
diff --git a/include/linux/tcp.h b/include/linux/tcp.h
index eaa3113..7fee8a4 100644
--- a/include/linux/tcp.h
+++ b/include/linux/tcp.h
@@ -247,31 +247,38 @@ struct tcp_options_received {
sack_ok : 4, /* SACK seen on SYN packet */
snd_wscale : 4, /* Window scaling received from sender */
rcv_wscale : 4; /* Window scaling to send to receiver */
-/* SACKs data */
+ u8 cookie_plus:6, /* bytes in authenticator/cookie option */
+ cookie_out_never:1,
+ cookie_in_always:1;
u8 num_sacks; /* Number of SACK blocks */
- u16 user_mss; /* mss requested by user in ioctl */
+ u16 user_mss; /* mss requested by user in ioctl */
u16 mss_clamp; /* Maximal mss, negotiated at connection setup */
};
static inline void tcp_clear_options(struct tcp_options_received *rx_opt)
{
- rx_opt->tstamp_ok = rx_opt->sack_ok = rx_opt->wscale_ok = rx_opt->snd_wscale = 0;
+ rx_opt->tstamp_ok = rx_opt->sack_ok = 0;
+ rx_opt->wscale_ok = rx_opt->snd_wscale = 0;
+ rx_opt->cookie_plus = 0;
}
/* This is the max number of SACKS that we'll generate and process. It's safe
- * to increse this, although since:
+ * to increase this, although since:
* size = TCPOLEN_SACK_BASE_ALIGNED (4) + n * TCPOLEN_SACK_PERBLOCK (8)
* only four options will fit in a standard TCP header */
#define TCP_NUM_SACKS 4
+struct tcp_cookie_values;
+struct tcp_request_sock_ops;
+
struct tcp_request_sock {
struct inet_request_sock req;
#ifdef CONFIG_TCP_MD5SIG
/* Only used by TCP MD5 Signature so far. */
const struct tcp_request_sock_ops *af_specific;
#endif
- u32 rcv_isn;
- u32 snt_isn;
+ u32 rcv_isn;
+ u32 snt_isn;
};
static inline struct tcp_request_sock *tcp_rsk(const struct request_sock *req)
@@ -441,6 +448,12 @@ struct tcp_sock {
/* TCP MD5 Signature Option information */
struct tcp_md5sig_info *md5sig_info;
#endif
+
+ /* When the cookie options are generated and exchanged, then this
+ * object holds a reference to them (cookie_values->kref). Also
+ * contains related tcp_cookie_transactions fields.
+ */
+ struct tcp_cookie_values *cookie_values;
};
static inline struct tcp_sock *tcp_sk(const struct sock *sk)
@@ -459,6 +472,10 @@ struct tcp_timewait_sock {
u16 tw_md5_keylen;
u8 tw_md5_key[TCP_MD5SIG_MAXKEYLEN];
#endif
+ /* Few sockets in timewait have cookies; in that case, then this
+ * object holds a reference to them (tw_cookie_values->kref).
+ */
+ struct tcp_cookie_values *tw_cookie_values;
};
static inline struct tcp_timewait_sock *tcp_twsk(const struct sock *sk)
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 738b65f..f9abd9b 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -30,6 +30,7 @@
#include <linux/dmaengine.h>
#include <linux/crypto.h>
#include <linux/cryptohash.h>
+#include <linux/kref.h>
#include <net/inet_connection_sock.h>
#include <net/inet_timewait_sock.h>
@@ -164,6 +165,7 @@ extern void tcp_time_wait(struct sock *sk, int state, int timeo);
#define TCPOPT_SACK 5 /* SACK Block */
#define TCPOPT_TIMESTAMP 8 /* Better RTT estimations/PAWS */
#define TCPOPT_MD5SIG 19 /* MD5 Signature (RFC2385) */
+#define TCPOPT_COOKIE 253 /* Cookie extension (experimental) */
/*
* TCP option lengths
@@ -174,6 +176,10 @@ extern void tcp_time_wait(struct sock *sk, int state, int timeo);
#define TCPOLEN_SACK_PERM 2
#define TCPOLEN_TIMESTAMP 10
#define TCPOLEN_MD5SIG 18
+#define TCPOLEN_COOKIE_BASE 2 /* Cookie-less header extension */
+#define TCPOLEN_COOKIE_PAIR 3 /* Cookie pair header extension */
+#define TCPOLEN_COOKIE_MIN (TCPOLEN_COOKIE_BASE+TCP_COOKIE_MIN)
+#define TCPOLEN_COOKIE_MAX (TCPOLEN_COOKIE_BASE+TCP_COOKIE_MAX)
/* But this is what stacks really send out. */
#define TCPOLEN_TSTAMP_ALIGNED 12
@@ -1482,6 +1488,83 @@ struct tcp_request_sock_ops {
extern int tcp_cookie_generator(u32 *bakery);
+/**
+ * struct tcp_cookie_values - each socket needs extra space for the
+ * cookies, together with (optional) space for any SYN data.
+ *
+ * A tcp_sock contains a pointer to the current value, and this is
+ * cloned to the tcp_timewait_sock.
+ *
+ * @cookie_pair: variable data from the option exchange.
+ *
+ * @cookie_desired: user specified tcpct_cookie_desired. Zero
+ * indicates default (sysctl_tcp_cookie_size).
+ * After cookie sent, remembers size of cookie.
+ * Range 0, TCP_COOKIE_MIN to TCP_COOKIE_MAX.
+ *
+ * @s_data_desired: user specified tcpct_s_data_desired. When the
+ * constant payload is specified (@s_data_constant),
+ * holds its length instead.
+ * Range 0 to TCP_MSS_DESIRED.
+ *
+ * @s_data_payload: constant data that is to be included in the
+ * payload of SYN or SYNACK segments when the
+ * cookie option is present.
+ */
+struct tcp_cookie_values {
+ struct kref kref;
+ u8 cookie_pair[TCP_COOKIE_PAIR_SIZE];
+ u8 cookie_pair_size;
+ u8 cookie_desired;
+ u16 s_data_desired:11,
+ s_data_constant:1,
+ s_data_in:1,
+ s_data_out:1,
+ s_data_unused:2;
+ u8 s_data_payload[0];
+};
+
+static inline void tcp_cookie_values_release(struct kref *kref)
+{
+ kfree(container_of(kref, struct tcp_cookie_values, kref));
+}
+
+/* The length of constant payload data. Note that s_data_desired is
+ * overloaded, depending on s_data_constant: either the length of constant
+ * data (returned here) or the limit on variable data.
+ */
+static inline int tcp_s_data_size(const struct tcp_sock *tp)
+{
+ return (tp->cookie_values != NULL && tp->cookie_values->s_data_constant)
+ ? tp->cookie_values->s_data_desired
+ : 0;
+}
+
+/**
+ * struct tcp_extend_values - tcp_ipv?.c to tcp_output.c workspace.
+ *
+ * As tcp_request_sock has already been extended in other places, the
+ * only remaining method is to pass stack values along as function
+ * parameters. These parameters are not needed after sending SYNACK.
+ *
+ * @cookie_bakery: cryptographic secret and message workspace.
+ *
+ * @cookie_plus: bytes in authenticator/cookie option, copied from
+ * struct tcp_options_received (above).
+ */
+struct tcp_extend_values {
+ struct request_values rv;
+ u32 cookie_bakery[COOKIE_WORKSPACE_WORDS];
+ u8 cookie_plus:6,
+ cookie_out_never:1,
+ cookie_in_always:1;
+};
+
+static inline struct tcp_extend_values *tcp_xv(struct request_values *rvp)
+{
+ return (struct tcp_extend_values *)rvp;
+}
+
extern void tcp_v4_init(void);
extern void tcp_init(void);
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 397ab8f..2bb7864 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1834,6 +1834,19 @@ static int tcp_v4_init_sock(struct sock *sk)
tp->af_specific = &tcp_sock_ipv4_specific;
#endif
+ /* TCP Cookie Transactions */
+ if (sysctl_tcp_cookie_size > 0) {
+ /* Default, cookies without s_data_payload. */
+ tp->cookie_values =
+ kzalloc(sizeof(*tp->cookie_values),
+ sk->sk_allocation);
+ if (tp->cookie_values != NULL)
+ kref_init(&tp->cookie_values->kref);
+ }
+ /* Presumed zeroed, in order of appearance:
+ * cookie_in_always, cookie_out_never,
+ * s_data_constant, s_data_in, s_data_out
+ */
sk->sk_sndbuf = sysctl_tcp_wmem[1];
sk->sk_rcvbuf = sysctl_tcp_rmem[1];
@@ -1887,6 +1900,13 @@ void tcp_v4_destroy_sock(struct sock *sk)
sk->sk_sndmsg_page = NULL;
}
+ /* TCP Cookie Transactions */
+ if (tp->cookie_values != NULL) {
+ kref_put(&tp->cookie_values->kref,
+ tcp_cookie_values_release);
+ tp->cookie_values = NULL;
+ }
+
percpu_counter_dec(&tcp_sockets_allocated);
}
diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c
index 7a42990..53ef6d8 100644
--- a/net/ipv4/tcp_minisocks.c
+++ b/net/ipv4/tcp_minisocks.c
@@ -389,14 +389,43 @@ struct sock *tcp_create_openreq_child(struct sock *sk, struct request_sock *req,
const struct inet_request_sock *ireq = inet_rsk(req);
struct tcp_request_sock *treq = tcp_rsk(req);
struct inet_connection_sock *newicsk = inet_csk(newsk);
- struct tcp_sock *newtp;
+ struct tcp_sock *newtp = tcp_sk(newsk);
+ struct tcp_sock *oldtp = tcp_sk(sk);
+ struct tcp_cookie_values *oldcvp = oldtp->cookie_values;
+
+ /* TCP Cookie Transactions require space for the cookie pair,
+ * as it differs for each connection. There is no need to
+ * copy any s_data_payload stored at the original socket.
+ * Failure will prevent resuming the connection.
+ *
+ * Presumed copied, in order of appearance:
+ * cookie_in_always, cookie_out_never
+ */
+ if (oldcvp != NULL) {
+ struct tcp_cookie_values *newcvp =
+ kzalloc(sizeof(*newtp->cookie_values),
+ GFP_ATOMIC);
+
+ if (newcvp != NULL) {
+ kref_init(&newcvp->kref);
+ newcvp->cookie_desired =
+ oldcvp->cookie_desired;
+ newtp->cookie_values = newcvp;
+ } else {
+ /* Not Yet Implemented */
+ newtp->cookie_values = NULL;
+ }
+ }
/* Now setup tcp_sock */
- newtp = tcp_sk(newsk);
newtp->pred_flags = 0;
- newtp->rcv_wup = newtp->copied_seq = newtp->rcv_nxt = treq->rcv_isn + 1;
- newtp->snd_sml = newtp->snd_una = newtp->snd_nxt = treq->snt_isn + 1;
- newtp->snd_up = treq->snt_isn + 1;
+
+ newtp->rcv_wup = newtp->copied_seq =
+ newtp->rcv_nxt = treq->rcv_isn + 1;
+
+ newtp->snd_sml = newtp->snd_una =
+ newtp->snd_nxt = newtp->snd_up =
+ treq->snt_isn + 1 + tcp_s_data_size(oldtp);
tcp_prequeue_init(newtp);
@@ -429,8 +458,8 @@ struct sock *tcp_create_openreq_child(struct sock *sk, struct request_sock *req,
tcp_set_ca_state(newsk, TCP_CA_Open);
tcp_init_xmit_timers(newsk);
skb_queue_head_init(&newtp->out_of_order_queue);
- newtp->write_seq = treq->snt_isn + 1;
- newtp->pushed_seq = newtp->write_seq;
+ newtp->write_seq = newtp->pushed_seq =
+ treq->snt_isn + 1 + tcp_s_data_size(oldtp);
newtp->rx_opt.saw_tstamp = 0;
@@ -596,7 +625,8 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
* Invalid ACK: reset will be sent by listening socket
*/
if ((flg & TCP_FLAG_ACK) &&
- (TCP_SKB_CB(skb)->ack_seq != tcp_rsk(req)->snt_isn + 1))
+ (TCP_SKB_CB(skb)->ack_seq !=
+ tcp_rsk(req)->snt_isn + 1 + tcp_s_data_size(tcp_sk(sk))))
return sk;
/* Also, it would be not so bad idea to check rcv_tsecr, which
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 3e327bc..973096a 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -1865,6 +1865,19 @@ static int tcp_v6_init_sock(struct sock *sk)
tp->af_specific = &tcp_sock_ipv6_specific;
#endif
+ /* TCP Cookie Transactions */
+ if (sysctl_tcp_cookie_size > 0) {
+ /* Default, cookies without s_data_payload. */
+ tp->cookie_values =
+ kzalloc(sizeof(*tp->cookie_values),
+ sk->sk_allocation);
+ if (tp->cookie_values != NULL)
+ kref_init(&tp->cookie_values->kref);
+ }
+ /* Presumed zeroed, in order of appearance:
+ * cookie_in_always, cookie_out_never,
+ * s_data_constant, s_data_in, s_data_out
+ */
sk->sk_sndbuf = sysctl_tcp_wmem[1];
sk->sk_rcvbuf = sysctl_tcp_rmem[1];
--
1.6.3.3
^ permalink raw reply related
* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Eric Dumazet @ 2009-11-20 14:45 UTC (permalink / raw)
To: Jarek Poplawski; +Cc: David S. Miller, Tom Herbert, Linux Netdev List
In-Reply-To: <20091120133245.GA9038@ff.dom.local>
Jarek Poplawski a écrit :
> On 20-11-2009 00:46, Eric Dumazet wrote:
>> Here is first version of XPS.
>>
>> Goal of XPS is to free TX completed skbs by the cpu that submitted the transmit.
>
> But why?... OK, you write in another message about sock_wfree(). Then
> how about users, who don't sock_wfree (routers)? Will there be any way
> to disable it?
This is open for discussion, but I saw no problem with routing workloads.
sock_wfree() is not that expensive for tcp anyway.
You also have a cost of kfreeing() two blocks of memory per skb, if allocation was done by another cpu.
If this happens to be a problem, we can immediately free packet if it
has no destructors :
At xmit time, initialize skb->sending_cpu like that
skb->sending_cpu = (skb->destructor) ? smp_processor_id() : 0xFFFF;
to make sure we dont touch too many cache lines at tx completion time.
>> +/*
>> + * XPS : Xmit Packet Steering
>> + *
>> + * TX completion packet freeing is performed on cpu that sent packet.
>> + */
>> +#if defined(CONFIG_SMP)
>
> Shouldn't it be in the Makefile?
It is in Makefile too, I let it in prelim code to make it clear this was CONFIG_SMP only.
>
> ...
>> +/*
>> + * called at end of net_rx_action()
>> + * preemption (and cpu migration/offline/online) disabled
>> + */
>> +void xps_flush(void)
>> +{
>> + int cpu, prevlen;
>> + struct sk_buff_head *head = per_cpu_ptr(xps_array, smp_processor_id());
>> + struct xps_pcpu_queue *q;
>> + struct sk_buff *skb;
>> +
>> + for_each_cpu_mask_nr(cpu, __get_cpu_var(xps_cpus)) {
>> + q = &per_cpu(xps_pcpu_queue, cpu);
>> + if (cpu_online(cpu)) {
>> + spin_lock(&q->list.lock);
>
> This lock probably needs irq disabling: let's say 2 cpus run this at
> the same time and both are interrupted with these (previously
> scheduled) IPIs?
Repeat after me :
lockdep is my friend, lockdep is my friend, lockdep is my friend... :)
Seriously, I must think again on this locking schem.
>> +static void remote_free_skb_list(void *arg)
>> +{
>> + struct sk_buff *last;
>> + struct softnet_data *sd;
>> + struct xps_pcpu_queue *q = arg; /* &__get_cpu_var(xps_pcpu_queue); */
>> +
>> + spin_lock(&q->list.lock);
>> +
>> + last = q->list.prev;
>
> Is q->list handled in case this cpu goes down before this IPI is
> triggered?
[block migration] (how ? this is the question)
if (cpu_online(cpu)) {
give_work_to_cpu(cpu);
trigger IPI
} else {
handle_work_ourself()
}
[unblock migration]
General problem is : what guards cpu going off line between the if (cpu_online(cpu))
and the IPI.
I dont know yet, but it seems that disabling preemption is enough to get this
guarantee. This seems strange.
We can add a notifier (or better call a function from existing one : dev_cpu_callback()) to
flush this queue when necessary.
Thanks
^ permalink raw reply
* [net-next-2.6 PATCH v7 5/7 RFC] TCPCT part 1e: implement socket option TCP_COOKIE_TRANSACTIONS
From: William Allen Simpson @ 2009-11-20 14:48 UTC (permalink / raw)
To: Linux Kernel Network Developers
In-Reply-To: <4B06A1FF.8000202@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 868 bytes --]
Provide per socket control of the TCP cookie option and SYN/SYNACK data.
This is a straightforward re-implementation of an earlier (year-old)
patch that no longer applies cleanly, with permission of the original
author (Adam Langley):
http://thread.gmane.org/gmane.linux.network/102586
The principle difference is using a TCP option to carry the cookie nonce,
instead of a user configured offset in the data.
Allocations have been rearranged to avoid requiring GFP_ATOMIC.
Requires:
net: TCP_MSS_DEFAULT, TCP_MSS_DESIRED
TCPCT part 1c: sysctl_tcp_cookie_size, socket option TCP_COOKIE_TRANSACTIONS
TCPCT part 1d: define TCP cookie option, extend existing struct's
Signed-off-by: William.Allen.Simpson@gmail.com
---
net/ipv4/tcp.c | 133 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 131 insertions(+), 2 deletions(-)
[-- Attachment #2: TCPCT+1e7.patch --]
[-- Type: text/plain, Size: 4406 bytes --]
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 1356e3d..5b14e2b 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2084,8 +2084,9 @@ static int do_tcp_setsockopt(struct sock *sk, int level,
int val;
int err = 0;
- /* This is a string value all the others are int's */
- if (optname == TCP_CONGESTION) {
+ /* These are data/string values, all the others are ints */
+ switch (optname) {
+ case TCP_CONGESTION: {
char name[TCP_CA_NAME_MAX];
if (optlen < 1)
@@ -2102,6 +2103,93 @@ static int do_tcp_setsockopt(struct sock *sk, int level,
release_sock(sk);
return err;
}
+ case TCP_COOKIE_TRANSACTIONS: {
+ struct tcp_cookie_transactions ctd;
+ struct tcp_cookie_values *cvp = NULL;
+
+ if (sizeof(ctd) > optlen)
+ return -EINVAL;
+ if (copy_from_user(&ctd, optval, sizeof(ctd)))
+ return -EFAULT;
+
+ if (ctd.tcpct_used > sizeof(ctd.tcpct_value)
+ || ctd.tcpct_s_data_desired > TCP_MSS_DESIRED)
+ return -EINVAL;
+
+ if (ctd.tcpct_cookie_desired == 0) {
+ /* default to global value */
+ } else if ((0x1 & ctd.tcpct_cookie_desired)
+ || ctd.tcpct_cookie_desired > TCP_COOKIE_MAX
+ || ctd.tcpct_cookie_desired < TCP_COOKIE_MIN) {
+ return -EINVAL;
+ }
+
+ if (TCP_COOKIE_OUT_NEVER & ctd.tcpct_flags) {
+ /* Supercedes all other values */
+ lock_sock(sk);
+ if (tp->cookie_values != NULL) {
+ kref_put(&tp->cookie_values->kref,
+ tcp_cookie_values_release);
+ tp->cookie_values = NULL;
+ }
+ tp->rx_opt.cookie_in_always = 0; /* false */
+ tp->rx_opt.cookie_out_never = 1; /* true */
+ release_sock(sk);
+ return err;
+ }
+
+ /* Allocate ancillary memory before locking.
+ */
+ if (ctd.tcpct_used > 0
+ || (tp->cookie_values == NULL
+ && (sysctl_tcp_cookie_size > 0
+ || ctd.tcpct_cookie_desired > 0
+ || ctd.tcpct_s_data_desired > 0))) {
+ cvp = kzalloc(sizeof(*cvp) + ctd.tcpct_used,
+ GFP_KERNEL);
+ if (cvp == NULL)
+ return -ENOMEM;
+ }
+ lock_sock(sk);
+ tp->rx_opt.cookie_in_always =
+ (TCP_COOKIE_IN_ALWAYS & ctd.tcpct_flags);
+ tp->rx_opt.cookie_out_never = 0; /* false */
+
+ if (tp->cookie_values != NULL) {
+ if (cvp != NULL) {
+ /* Changed values are recorded by a changed
+ * pointer, ensuring the cookie will differ,
+ * without separately hashing each value later.
+ */
+ kref_put(&tp->cookie_values->kref,
+ tcp_cookie_values_release);
+ kref_init(&cvp->kref);
+ tp->cookie_values = cvp;
+ } else {
+ cvp = tp->cookie_values;
+ }
+ }
+ if (cvp != NULL) {
+ cvp->cookie_desired = ctd.tcpct_cookie_desired;
+
+ if (ctd.tcpct_used > 0) {
+ memcpy(cvp->s_data_payload, ctd.tcpct_value,
+ ctd.tcpct_used);
+ cvp->s_data_desired = ctd.tcpct_used;
+ cvp->s_data_constant = 1; /* true */
+ } else {
+ /* No constant payload data. */
+ cvp->s_data_desired = ctd.tcpct_s_data_desired;
+ cvp->s_data_constant = 0; /* false */
+ }
+ }
+ release_sock(sk);
+ return err;
+ }
+ default:
+ /* fallthru */
+ break;
+ };
if (optlen < sizeof(int))
return -EINVAL;
@@ -2426,6 +2514,47 @@ static int do_tcp_getsockopt(struct sock *sk, int level,
if (copy_to_user(optval, icsk->icsk_ca_ops->name, len))
return -EFAULT;
return 0;
+
+ case TCP_COOKIE_TRANSACTIONS: {
+ struct tcp_cookie_transactions ctd;
+ struct tcp_cookie_values *cvp = tp->cookie_values;
+
+ if (get_user(len, optlen))
+ return -EFAULT;
+ if (len < sizeof(ctd))
+ return -EINVAL;
+
+ memset(&ctd, 0, sizeof(ctd));
+ ctd.tcpct_flags = (tp->rx_opt.cookie_in_always
+ ? TCP_COOKIE_IN_ALWAYS : 0)
+ | (tp->rx_opt.cookie_out_never
+ ? TCP_COOKIE_OUT_NEVER : 0);
+
+ if (cvp != NULL) {
+ ctd.tcpct_flags |= (cvp->s_data_in
+ ? TCP_S_DATA_IN : 0)
+ | (cvp->s_data_out
+ ? TCP_S_DATA_OUT : 0);
+
+ ctd.tcpct_cookie_desired = cvp->cookie_desired;
+ ctd.tcpct_s_data_desired = cvp->s_data_desired;
+
+ /* Cookie(s) saved, return as nonce */
+ if (sizeof(ctd.tcpct_value) < cvp->cookie_pair_size) {
+ /* impossible? */
+ return -EINVAL;
+ }
+ memcpy(&ctd.tcpct_value[0], &cvp->cookie_pair[0],
+ cvp->cookie_pair_size);
+ ctd.tcpct_used = cvp->cookie_pair_size;
+ }
+
+ if (put_user(sizeof(ctd), optlen))
+ return -EFAULT;
+ if (copy_to_user(optval, &ctd, sizeof(ctd)))
+ return -EFAULT;
+ return 0;
+ }
default:
return -ENOPROTOOPT;
}
--
1.6.3.3
^ permalink raw reply related
* [net-next-2.6 PATCH v7 6/7 RFC] TCPCT part 1f: Initiator Cookie => Responder
From: William Allen Simpson @ 2009-11-20 14:55 UTC (permalink / raw)
To: Linux Kernel Network Developers
In-Reply-To: <4B06A1FF.8000202@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 615 bytes --]
Calculate and format <SYN> TCP_COOKIE option.
This is a significantly revised implementation of an earlier (year-old)
patch that no longer applies cleanly, with permission of the original
author (Adam Langley):
http://thread.gmane.org/gmane.linux.network/102586
Requires:
TCPCT part 1c: sysctl_tcp_cookie_size, socket option TCP_COOKIE_TRANSACTIONS
TCPCT part 1d: define TCP cookie option, extend existing struct's
Signed-off-by: William.Allen.Simpson@gmail.com
---
net/ipv4/tcp_output.c | 182 ++++++++++++++++++++++++++++++++++++++++++-------
1 files changed, 158 insertions(+), 24 deletions(-)
[-- Attachment #2: TCPCT+1f7.patch --]
[-- Type: text/plain, Size: 9484 bytes --]
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index e59fa5a..76dab84 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -370,15 +370,45 @@ static inline int tcp_urg_mode(const struct tcp_sock *tp)
#define OPTION_TS (1 << 1)
#define OPTION_MD5 (1 << 2)
#define OPTION_WSCALE (1 << 3)
+#define OPTION_COOKIE_EXTENSION (1 << 4)
struct tcp_out_options {
u8 options; /* bit field of OPTION_* */
u8 ws; /* window scale, 0 to disable */
u8 num_sack_blocks; /* number of SACK blocks to include */
+ u8 hash_size; /* bytes in hash_location */
u16 mss; /* 0 to disable */
__u32 tsval, tsecr; /* need to include OPTION_TS */
+ __u8 *hash_location; /* temporary pointer, overloaded */
};
+/* The sysctl int routines are generic, so check consistency here.
+ */
+static u8 tcp_cookie_size_check(u8 desired)
+{
+ if (desired > 0) {
+ /* previously specified */
+ return desired;
+ }
+ if (sysctl_tcp_cookie_size <= 0) {
+ /* no default specified */
+ return 0;
+ }
+ if (sysctl_tcp_cookie_size <= TCP_COOKIE_MIN) {
+ /* value too small, specify minimum */
+ return TCP_COOKIE_MIN;
+ }
+ if (sysctl_tcp_cookie_size >= TCP_COOKIE_MAX) {
+ /* value too large, specify maximum */
+ return TCP_COOKIE_MAX;
+ }
+ if (0x1 & sysctl_tcp_cookie_size) {
+ /* 8-bit multiple, illegal, fix it */
+ return (u8)(sysctl_tcp_cookie_size + 0x1);
+ }
+ return (u8)sysctl_tcp_cookie_size;
+}
+
/* Write previously computed TCP options to the packet.
*
* Beware: Something in the Internet is very sensitive to the ordering of
@@ -393,17 +423,34 @@ struct tcp_out_options {
* (but it may well be that other scenarios fail similarly).
*/
static void tcp_options_write(__be32 *ptr, struct tcp_sock *tp,
- const struct tcp_out_options *opts,
- __u8 **md5_hash) {
- if (unlikely(OPTION_MD5 & opts->options)) {
- *ptr++ = htonl((TCPOPT_NOP << 24) |
- (TCPOPT_NOP << 16) |
- (TCPOPT_MD5SIG << 8) |
- TCPOLEN_MD5SIG);
- *md5_hash = (__u8 *)ptr;
+ struct tcp_out_options *opts)
+{
+ u8 options = opts->options; /* mungable copy */
+
+ /* Having both authentication and cookies for security is redundant,
+ * and there's certainly not enough room. Instead, the cookie-less
+ * extension variant is proposed.
+ *
+ * Consider the pessimal case with authentication. The options
+ * could look like:
+ * COOKIE|MD5(20) + MSS(4) + SACK|TS(12) + WSCALE(4) == 40
+ */
+ if (unlikely(OPTION_MD5 & options)) {
+ if (unlikely(OPTION_COOKIE_EXTENSION & options)) {
+ *ptr++ = htonl((TCPOPT_COOKIE << 24) |
+ (TCPOLEN_COOKIE_BASE << 16) |
+ (TCPOPT_MD5SIG << 8) |
+ TCPOLEN_MD5SIG);
+ } else {
+ *ptr++ = htonl((TCPOPT_NOP << 24) |
+ (TCPOPT_NOP << 16) |
+ (TCPOPT_MD5SIG << 8) |
+ TCPOLEN_MD5SIG);
+ }
+ options &= ~OPTION_COOKIE_EXTENSION;
+ /* overload cookie hash location */
+ opts->hash_location = (__u8 *)ptr;
ptr += 4;
- } else {
- *md5_hash = NULL;
}
if (unlikely(opts->mss)) {
@@ -412,12 +459,13 @@ static void tcp_options_write(__be32 *ptr, struct tcp_sock *tp,
opts->mss);
}
- if (likely(OPTION_TS & opts->options)) {
- if (unlikely(OPTION_SACK_ADVERTISE & opts->options)) {
+ if (likely(OPTION_TS & options)) {
+ if (unlikely(OPTION_SACK_ADVERTISE & options)) {
*ptr++ = htonl((TCPOPT_SACK_PERM << 24) |
(TCPOLEN_SACK_PERM << 16) |
(TCPOPT_TIMESTAMP << 8) |
TCPOLEN_TIMESTAMP);
+ options &= ~OPTION_SACK_ADVERTISE;
} else {
*ptr++ = htonl((TCPOPT_NOP << 24) |
(TCPOPT_NOP << 16) |
@@ -428,15 +476,52 @@ static void tcp_options_write(__be32 *ptr, struct tcp_sock *tp,
*ptr++ = htonl(opts->tsecr);
}
- if (unlikely(OPTION_SACK_ADVERTISE & opts->options &&
- !(OPTION_TS & opts->options))) {
+ /* Specification requires after timestamp, so do it now.
+ *
+ * Consider the pessimal case without authentication. The options
+ * could look like:
+ * MSS(4) + SACK|TS(12) + COOKIE(20) + WSCALE(4) == 40
+ */
+ if (unlikely(OPTION_COOKIE_EXTENSION & options)) {
+ __u8 *cookie_copy = opts->hash_location;
+ u8 cookie_size = opts->hash_size;
+
+ /* 8-bit multiple handled in tcp_cookie_size_check() above,
+ * and elsewhere.
+ */
+ if (0x2 & cookie_size) {
+ __u8 *p = (__u8 *)ptr;
+
+ /* 16-bit multiple */
+ *p++ = TCPOPT_COOKIE;
+ *p++ = TCPOLEN_COOKIE_BASE + cookie_size;
+ *p++ = *cookie_copy++;
+ *p++ = *cookie_copy++;
+ ptr++;
+ cookie_size -= 2;
+ } else {
+ /* 32-bit multiple */
+ *ptr++ = htonl(((TCPOPT_NOP << 24) |
+ (TCPOPT_NOP << 16) |
+ (TCPOPT_COOKIE << 8) |
+ TCPOLEN_COOKIE_BASE) +
+ cookie_size);
+ }
+
+ if (cookie_size > 0) {
+ memcpy(ptr, cookie_copy, cookie_size);
+ ptr += (cookie_size / 4);
+ }
+ }
+
+ if (unlikely(OPTION_SACK_ADVERTISE & options)) {
*ptr++ = htonl((TCPOPT_NOP << 24) |
(TCPOPT_NOP << 16) |
(TCPOPT_SACK_PERM << 8) |
TCPOLEN_SACK_PERM);
}
- if (unlikely(OPTION_WSCALE & opts->options)) {
+ if (unlikely(OPTION_WSCALE & options)) {
*ptr++ = htonl((TCPOPT_NOP << 24) |
(TCPOPT_WINDOW << 16) |
(TCPOLEN_WINDOW << 8) |
@@ -471,8 +556,12 @@ static unsigned tcp_syn_options(struct sock *sk, struct sk_buff *skb,
struct tcp_out_options *opts,
struct tcp_md5sig_key **md5) {
struct tcp_sock *tp = tcp_sk(sk);
- unsigned size = 0;
+ struct tcp_cookie_values *cvp = tp->cookie_values;
struct dst_entry *dst = __sk_dst_get(sk);
+ unsigned size = 0;
+ u8 cookie_size = (!tp->rx_opt.cookie_out_never && cvp != NULL)
+ ? tcp_cookie_size_check(cvp->cookie_desired)
+ : 0;
#ifdef CONFIG_TCP_MD5SIG
*md5 = tp->af_specific->md5_lookup(sk, sk);
@@ -517,6 +606,53 @@ static unsigned tcp_syn_options(struct sock *sk, struct sk_buff *skb,
size += TCPOLEN_SACKPERM_ALIGNED;
}
+ /* Note that timestamps are required by the specification.
+ *
+ * Odd numbers of bytes are prohibited by the specification, ensuring
+ * that the cookie is 16-bit aligned, and the resulting cookie pair is
+ * 32-bit aligned.
+ */
+ if (*md5 == NULL
+ && (OPTION_TS & opts->options)
+ && cookie_size > 0) {
+ int need = TCPOLEN_COOKIE_BASE + cookie_size;
+ int remaining = MAX_TCP_OPTION_SPACE - size;
+
+ if (0x2 & need) {
+ /* 32-bit multiple */
+ need += 2; /* NOPs */
+
+ if (need > remaining) {
+ /* try shrinking cookie to fit */
+ cookie_size -= 2;
+ need -= 4;
+ }
+ }
+ while (need > remaining && TCP_COOKIE_MIN <= cookie_size) {
+ cookie_size -= 4;
+ need -= 4;
+ }
+ if (TCP_COOKIE_MIN <= cookie_size) {
+ opts->options |= OPTION_COOKIE_EXTENSION;
+ opts->hash_location = (__u8 *)&cvp->cookie_pair[0];
+ opts->hash_size = cookie_size;
+
+ /* Remember for future incarnations. */
+ cvp->cookie_desired = cookie_size;
+
+ if (cvp->cookie_desired != cvp->cookie_pair_size) {
+ /* Currently use random bytes as a nonce,
+ * assuming these are completely unpredictable
+ * by hostile users of the same system.
+ */
+ get_random_bytes(&cvp->cookie_pair[0],
+ cookie_size);
+ cvp->cookie_pair_size = cookie_size;
+ }
+
+ size += need;
+ }
+ }
return size;
}
@@ -632,7 +768,6 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
struct tcp_out_options opts;
unsigned tcp_options_size, tcp_header_size;
struct tcp_md5sig_key *md5;
- __u8 *md5_hash_location;
struct tcphdr *th;
int err;
@@ -703,7 +838,7 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
}
}
- tcp_options_write((__be32 *)(th + 1), tp, &opts, &md5_hash_location);
+ tcp_options_write((__be32 *)(th + 1), tp, &opts);
if (likely((tcb->flags & TCPCB_FLAG_SYN) == 0))
TCP_ECN_send(sk, skb, tcp_header_size);
@@ -711,7 +846,7 @@ static int tcp_transmit_skb(struct sock *sk, struct sk_buff *skb, int clone_it,
/* Calculate the MD5 hash, as we have all we need now */
if (md5) {
sk->sk_route_caps &= ~NETIF_F_GSO_MASK;
- tp->af_specific->calc_md5_hash(md5_hash_location,
+ tp->af_specific->calc_md5_hash(opts.hash_location,
md5, sk, NULL, skb);
}
#endif
@@ -2235,14 +2370,13 @@ struct sk_buff *tcp_make_synack(struct sock *sk, struct dst_entry *dst,
struct request_sock *req,
struct request_values *rvp)
{
+ struct tcp_out_options opts;
struct inet_request_sock *ireq = inet_rsk(req);
struct tcp_sock *tp = tcp_sk(sk);
struct tcphdr *th;
- int tcp_header_size;
- struct tcp_out_options opts;
struct sk_buff *skb;
struct tcp_md5sig_key *md5;
- __u8 *md5_hash_location;
+ int tcp_header_size;
int mss;
skb = sock_wmalloc(sk, MAX_TCP_HEADER + 15, 1, GFP_ATOMIC);
@@ -2303,14 +2437,14 @@ struct sk_buff *tcp_make_synack(struct sock *sk, struct dst_entry *dst,
/* RFC1323: The window in SYN & SYN/ACK segments is never scaled. */
th->window = htons(min(req->rcv_wnd, 65535U));
- tcp_options_write((__be32 *)(th + 1), tp, &opts, &md5_hash_location);
+ tcp_options_write((__be32 *)(th + 1), tp, &opts);
th->doff = (tcp_header_size >> 2);
TCP_INC_STATS(sock_net(sk), TCP_MIB_OUTSEGS);
#ifdef CONFIG_TCP_MD5SIG
/* Okay, we have all we need - do the md5 hash if needed */
if (md5) {
- tcp_rsk(req)->af_specific->calc_md5_hash(md5_hash_location,
+ tcp_rsk(req)->af_specific->calc_md5_hash(opts.hash_location,
md5, NULL, req, skb);
}
#endif
--
1.6.3.3
^ permalink raw reply related
* [net-next-2.6 PATCH v7 7/7 RFC] TCPCT part 1g: Responder Cookie => Initiator
From: William Allen Simpson @ 2009-11-20 15:06 UTC (permalink / raw)
To: Linux Kernel Network Developers
In-Reply-To: <4B06A1FF.8000202@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1269 bytes --]
Parse incoming TCP_COOKIE option(s).
Calculate <SYN,ACK> TCP_COOKIE option.
Send optional <SYN,ACK> data.
This is a significantly revised implementation of an earlier (year-old)
patch that no longer applies cleanly, with permission of the original
author (Adam Langley):
http://thread.gmane.org/gmane.linux.network/102586
Requires:
TCPCT part 1a: add request_values parameter for sending SYNACK
TCPCT part 1b: generate Responder Cookie secret
TCPCT part 1c: sysctl_tcp_cookie_size, socket option TCP_COOKIE_TRANSACTIONS
TCPCT part 1d: define TCP cookie option, extend existing struct's
TCPCT part 1e: implement socket option TCP_COOKIE_TRANSACTIONS
TCPCT part 1f: Initiator Cookie => Responder
Signed-off-by: William.Allen.Simpson@gmail.com
---
include/net/tcp.h | 1 +
net/ipv4/syncookies.c | 5 ++-
net/ipv4/tcp_input.c | 77 +++++++++++++++++++++++++++++++++++++++------
net/ipv4/tcp_ipv4.c | 46 +++++++++++++++++++++++++--
net/ipv4/tcp_minisocks.c | 14 +++++---
net/ipv4/tcp_output.c | 76 +++++++++++++++++++++++++++++++++++++++++++--
net/ipv6/syncookies.c | 5 ++-
net/ipv6/tcp_ipv6.c | 51 +++++++++++++++++++++++++++++-
8 files changed, 244 insertions(+), 31 deletions(-)
[-- Attachment #2: TCPCT+1g7.patch --]
[-- Type: text/plain, Size: 17346 bytes --]
diff --git a/include/net/tcp.h b/include/net/tcp.h
index f9abd9b..28b04ff 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -407,6 +407,7 @@ extern int tcp_recvmsg(struct kiocb *iocb, struct sock *sk,
extern void tcp_parse_options(struct sk_buff *skb,
struct tcp_options_received *opt_rx,
+ u8 **hvpp,
int estab,
struct dst_entry *dst);
diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c
index 3146cc4..26399ad 100644
--- a/net/ipv4/syncookies.c
+++ b/net/ipv4/syncookies.c
@@ -253,6 +253,8 @@ EXPORT_SYMBOL(cookie_check_timestamp);
struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb,
struct ip_options *opt)
{
+ struct tcp_options_received tcp_opt;
+ u8 *hash_location;
struct inet_request_sock *ireq;
struct tcp_request_sock *treq;
struct tcp_sock *tp = tcp_sk(sk);
@@ -263,7 +265,6 @@ struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb,
int mss;
struct rtable *rt;
__u8 rcv_wscale;
- struct tcp_options_received tcp_opt;
if (!sysctl_tcp_syncookies || !th->ack)
goto out;
@@ -341,7 +342,7 @@ struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb,
/* check for timestamp cookie support */
memset(&tcp_opt, 0, sizeof(tcp_opt));
- tcp_parse_options(skb, &tcp_opt, 0, &rt->u.dst);
+ tcp_parse_options(skb, &tcp_opt, &hash_location, 0, &rt->u.dst);
if (tcp_opt.saw_tstamp)
cookie_check_timestamp(&tcp_opt);
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index cc306ac..1470615 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -3698,7 +3698,7 @@ old_ack:
* the fast version below fails.
*/
void tcp_parse_options(struct sk_buff *skb, struct tcp_options_received *opt_rx,
- int estab, struct dst_entry *dst)
+ u8 **hvpp, int estab, struct dst_entry *dst)
{
unsigned char *ptr;
struct tcphdr *th = tcp_hdr(skb);
@@ -3785,7 +3785,30 @@ void tcp_parse_options(struct sk_buff *skb, struct tcp_options_received *opt_rx,
*/
break;
#endif
- }
+ case TCPOPT_COOKIE:
+ /* This option is variable length.
+ */
+ switch (opsize) {
+ case TCPOLEN_COOKIE_BASE:
+ /* not yet implemented */
+ break;
+ case TCPOLEN_COOKIE_PAIR:
+ /* not yet implemented */
+ break;
+ case TCPOLEN_COOKIE_MIN+0:
+ case TCPOLEN_COOKIE_MIN+2:
+ case TCPOLEN_COOKIE_MIN+4:
+ case TCPOLEN_COOKIE_MIN+6:
+ case TCPOLEN_COOKIE_MAX:
+ /* 16-bit multiple */
+ opt_rx->cookie_plus = opsize;
+ *hvpp = ptr;
+ default:
+ /* ignore option */
+ break;
+ };
+ break;
+ };
ptr += opsize-2;
length -= opsize;
@@ -3813,17 +3836,20 @@ static int tcp_parse_aligned_timestamp(struct tcp_sock *tp, struct tcphdr *th)
* If it is wrong it falls back on tcp_parse_options().
*/
static int tcp_fast_parse_options(struct sk_buff *skb, struct tcphdr *th,
- struct tcp_sock *tp)
+ struct tcp_sock *tp, u8 **hvpp)
{
- if (th->doff == sizeof(struct tcphdr) >> 2) {
+ /* In the spirit of fast parsing, compare doff directly to constant
+ * values. Because equality is used, short doff can be ignored here.
+ */
+ if (th->doff == (sizeof(*th) / 4)) {
tp->rx_opt.saw_tstamp = 0;
return 0;
} else if (tp->rx_opt.tstamp_ok &&
- th->doff == (sizeof(struct tcphdr)>>2)+(TCPOLEN_TSTAMP_ALIGNED>>2)) {
+ th->doff == ((sizeof(*th) + TCPOLEN_TSTAMP_ALIGNED) / 4)) {
if (tcp_parse_aligned_timestamp(tp, th))
return 1;
}
- tcp_parse_options(skb, &tp->rx_opt, 1, NULL);
+ tcp_parse_options(skb, &tp->rx_opt, hvpp, 1, NULL);
return 1;
}
@@ -5077,11 +5103,13 @@ out:
static int tcp_validate_incoming(struct sock *sk, struct sk_buff *skb,
struct tcphdr *th, int syn_inerr)
{
+ u8 *hash_location;
struct tcp_sock *tp = tcp_sk(sk);
/* RFC1323: H1. Apply PAWS check first. */
- if (tcp_fast_parse_options(skb, th, tp) && tp->rx_opt.saw_tstamp &&
- tcp_paws_discard(sk, skb)) {
+ if (tcp_fast_parse_options(skb, th, tp, &hash_location)
+ && tp->rx_opt.saw_tstamp
+ && tcp_paws_discard(sk, skb)) {
if (!th->rst) {
NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_PAWSESTABREJECTED);
tcp_send_dupack(sk, skb);
@@ -5368,12 +5396,14 @@ discard:
static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
struct tcphdr *th, unsigned len)
{
- struct tcp_sock *tp = tcp_sk(sk);
+ u8 *hash_location;
struct inet_connection_sock *icsk = inet_csk(sk);
- int saved_clamp = tp->rx_opt.mss_clamp;
+ struct tcp_sock *tp = tcp_sk(sk);
struct dst_entry *dst = __sk_dst_get(sk);
+ struct tcp_cookie_values *cvp = tp->cookie_values;
+ int saved_clamp = tp->rx_opt.mss_clamp;
- tcp_parse_options(skb, &tp->rx_opt, 0, dst);
+ tcp_parse_options(skb, &tp->rx_opt, &hash_location, 0, dst);
if (th->ack) {
/* rfc793:
@@ -5470,6 +5500,31 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
* Change state from SYN-SENT only after copied_seq
* is initialized. */
tp->copied_seq = tp->rcv_nxt;
+
+ if (cvp != NULL
+ && cvp->cookie_pair_size > 0
+ && tp->rx_opt.cookie_plus > 0) {
+ int cookie_size = tp->rx_opt.cookie_plus
+ - TCPOLEN_COOKIE_BASE;
+ int cookie_pair_size = cookie_size
+ + cvp->cookie_desired;
+
+ /* A cookie extension option was sent and returned.
+ * Note that each incoming SYNACK replaces the
+ * Responder cookie. The initial exchange is most
+ * fragile, as protection against spoofing relies
+ * entirely upon the sequence and timestamp (above).
+ * This replacement strategy allows the correct pair to
+ * pass through, while any others will be filtered via
+ * Responder verification later.
+ */
+ if (sizeof(cvp->cookie_pair) >= cookie_pair_size) {
+ memcpy(&cvp->cookie_pair[cvp->cookie_desired],
+ hash_location, cookie_size);
+ cvp->cookie_pair_size = cookie_pair_size;
+ }
+ }
+
smp_mb();
tcp_set_state(sk, TCP_ESTABLISHED);
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 2bb7864..15bc4c5 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1213,9 +1213,12 @@ static struct timewait_sock_ops tcp_timewait_sock_ops = {
int tcp_v4_conn_request(struct sock *sk, struct sk_buff *skb)
{
+ struct tcp_extend_values tmp_ext;
struct tcp_options_received tmp_opt;
+ u8 *hash_location;
struct request_sock *req;
struct inet_request_sock *ireq;
+ struct tcp_sock *tp = tcp_sk(sk);
struct dst_entry *dst = NULL;
__be32 saddr = ip_hdr(skb)->saddr;
__be32 daddr = ip_hdr(skb)->daddr;
@@ -1271,15 +1274,49 @@ int tcp_v4_conn_request(struct sock *sk, struct sk_buff *skb)
tcp_clear_options(&tmp_opt);
tmp_opt.mss_clamp = TCP_MSS_DEFAULT;
- tmp_opt.user_mss = tcp_sk(sk)->rx_opt.user_mss;
+ tmp_opt.user_mss = tp->rx_opt.user_mss;
+ tcp_parse_options(skb, &tmp_opt, &hash_location, 0, dst);
+
+ if (tmp_opt.cookie_plus > 0
+ && tmp_opt.saw_tstamp
+ && !tp->rx_opt.cookie_out_never
+ && (sysctl_tcp_cookie_size > 0
+ || (tp->cookie_values != NULL
+ && tp->cookie_values->cookie_desired > 0))) {
+ u8 *c;
+ u32 *mess = &tmp_ext.cookie_bakery[COOKIE_DIGEST_WORDS];
+ int l = tmp_opt.cookie_plus - TCPOLEN_COOKIE_BASE;
+
+ if (tcp_cookie_generator(&tmp_ext.cookie_bakery[0]) != 0)
+ goto drop_and_release;
+
+ /* Secret recipe starts with IP addresses */
+ *mess++ ^= daddr;
+ *mess++ ^= saddr;
- tcp_parse_options(skb, &tmp_opt, 0, dst);
+ /* plus variable length Initiator Cookie */
+ c = (u8 *)mess;
+ while (l-- > 0)
+ *c++ ^= *hash_location++;
+
+#ifdef CONFIG_SYN_COOKIES
+ want_cookie = 0; /* not our kind of cookie */
+#endif
+ tmp_ext.cookie_out_never = 0; /* false */
+ tmp_ext.cookie_plus = tmp_opt.cookie_plus;
+ } else if (!tp->rx_opt.cookie_in_always) {
+ /* redundant indications, but ensure initialization. */
+ tmp_ext.cookie_out_never = 1; /* true */
+ tmp_ext.cookie_plus = 0;
+ } else {
+ goto drop_and_release;
+ }
+ tmp_ext.cookie_in_always = tp->rx_opt.cookie_in_always;
if (want_cookie && !tmp_opt.saw_tstamp)
tcp_clear_options(&tmp_opt);
tmp_opt.tstamp_ok = tmp_opt.saw_tstamp;
-
tcp_openreq_init(req, &tmp_opt, skb);
if (security_inet_conn_request(sk, skb, req))
@@ -1339,7 +1376,8 @@ int tcp_v4_conn_request(struct sock *sk, struct sk_buff *skb)
}
tcp_rsk(req)->snt_isn = isn;
- if (__tcp_v4_send_synack(sk, dst, req, NULL)
+ if (__tcp_v4_send_synack(sk, dst, req,
+ (struct request_values *)&tmp_ext)
|| want_cookie)
goto drop_and_free;
diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c
index 53ef6d8..16a9040 100644
--- a/net/ipv4/tcp_minisocks.c
+++ b/net/ipv4/tcp_minisocks.c
@@ -96,13 +96,14 @@ enum tcp_tw_status
tcp_timewait_state_process(struct inet_timewait_sock *tw, struct sk_buff *skb,
const struct tcphdr *th)
{
- struct tcp_timewait_sock *tcptw = tcp_twsk((struct sock *)tw);
struct tcp_options_received tmp_opt;
+ u8 *hash_location;
+ struct tcp_timewait_sock *tcptw = tcp_twsk((struct sock *)tw);
int paws_reject = 0;
if (th->doff > (sizeof(*th) >> 2) && tcptw->tw_ts_recent_stamp) {
tmp_opt.tstamp_ok = 1;
- tcp_parse_options(skb, &tmp_opt, 1, NULL);
+ tcp_parse_options(skb, &tmp_opt, &hash_location, 1, NULL);
if (tmp_opt.saw_tstamp) {
tmp_opt.ts_recent = tcptw->tw_ts_recent;
@@ -524,15 +525,16 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
struct request_sock *req,
struct request_sock **prev)
{
+ struct tcp_options_received tmp_opt;
+ u8 *hash_location;
+ struct sock *child;
const struct tcphdr *th = tcp_hdr(skb);
__be32 flg = tcp_flag_word(th) & (TCP_FLAG_RST|TCP_FLAG_SYN|TCP_FLAG_ACK);
int paws_reject = 0;
- struct tcp_options_received tmp_opt;
- struct sock *child;
- if ((th->doff > (sizeof(struct tcphdr)>>2)) && (req->ts_recent)) {
+ if ((th->doff > (sizeof(*th) >> 2)) && (req->ts_recent)) {
tmp_opt.tstamp_ok = 1;
- tcp_parse_options(skb, &tmp_opt, 1, NULL);
+ tcp_parse_options(skb, &tmp_opt, &hash_location, 1, NULL);
if (tmp_opt.saw_tstamp) {
tmp_opt.ts_recent = req->ts_recent;
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 76dab84..ac7ab9b 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -661,9 +661,14 @@ static unsigned tcp_synack_options(struct sock *sk,
struct request_sock *req,
unsigned mss, struct sk_buff *skb,
struct tcp_out_options *opts,
- struct tcp_md5sig_key **md5) {
- unsigned size = 0;
+ struct tcp_md5sig_key **md5,
+ struct tcp_extend_values *xvp)
+{
struct inet_request_sock *ireq = inet_rsk(req);
+ unsigned size = 0;
+ u8 cookie_plus = (xvp != NULL && !xvp->cookie_out_never)
+ ? xvp->cookie_plus
+ : 0;
char doing_ts;
#ifdef CONFIG_TCP_MD5SIG
@@ -702,6 +707,29 @@ static unsigned tcp_synack_options(struct sock *sk,
size += TCPOLEN_SACKPERM_ALIGNED;
}
+ /* Similar rationale to tcp_syn_options() applies here, too.
+ * If the <SYN> options fit, the same options should fit now!
+ */
+ if (*md5 == NULL
+ && doing_ts
+ && cookie_plus > TCPOLEN_COOKIE_BASE) {
+ int need = cookie_plus; /* has TCPOLEN_COOKIE_BASE */
+ int remaining = MAX_TCP_OPTION_SPACE - size;
+
+ if (0x2 & need) {
+ /* 32-bit multiple */
+ need += 2; /* NOPs */
+ }
+ if (need <= remaining) {
+ opts->options |= OPTION_COOKIE_EXTENSION;
+ opts->hash_size = cookie_plus - TCPOLEN_COOKIE_BASE;
+ size += need;
+ } else {
+ /* There's no error return, so flag it. */
+ xvp->cookie_out_never = 1; /* true */
+ opts->hash_size = 0;
+ }
+ }
return size;
}
@@ -2371,6 +2399,7 @@ struct sk_buff *tcp_make_synack(struct sock *sk, struct dst_entry *dst,
struct request_values *rvp)
{
struct tcp_out_options opts;
+ struct tcp_extend_values *xvp = tcp_xv(rvp);
struct inet_request_sock *ireq = inet_rsk(req);
struct tcp_sock *tp = tcp_sk(sk);
struct tcphdr *th;
@@ -2414,8 +2443,8 @@ struct sk_buff *tcp_make_synack(struct sock *sk, struct dst_entry *dst,
#endif
TCP_SKB_CB(skb)->when = tcp_time_stamp;
tcp_header_size = tcp_synack_options(sk, req, mss,
- skb, &opts, &md5) +
- sizeof(struct tcphdr);
+ skb, &opts, &md5, xvp)
+ + sizeof(*th);
skb_push(skb, tcp_header_size);
skb_reset_transport_header(skb);
@@ -2432,6 +2461,45 @@ struct sk_buff *tcp_make_synack(struct sock *sk, struct dst_entry *dst,
*/
tcp_init_nondata_skb(skb, tcp_rsk(req)->snt_isn,
TCPCB_FLAG_SYN | TCPCB_FLAG_ACK);
+
+ if (OPTION_COOKIE_EXTENSION & opts.options) {
+ const struct tcp_cookie_values *cvp = tp->cookie_values;
+
+ if (cvp != NULL
+ && cvp->s_data_constant
+ && cvp->s_data_desired > 0) {
+ u8 *buf = skb_put(skb, cvp->s_data_desired);
+
+ /* copy data directly from the listening socket. */
+ memcpy(buf, cvp->s_data_payload, cvp->s_data_desired);
+ TCP_SKB_CB(skb)->end_seq += cvp->s_data_desired;
+ }
+
+ if (opts.hash_size > 0) {
+ __u32 workspace[SHA_WORKSPACE_WORDS];
+ u32 *mess = &xvp->cookie_bakery[COOKIE_DIGEST_WORDS];
+ u32 *tail = &mess[COOKIE_MESSAGE_WORDS-1];
+
+ /* Secret recipe depends on the Timestamp, (future)
+ * Sequence and Acknowledgment Numbers, Initiator
+ * Cookie, and others handled by IP variant caller.
+ */
+ *tail-- ^= opts.tsval;
+ *tail-- ^= tcp_rsk(req)->rcv_isn + 1;
+ *tail-- ^= TCP_SKB_CB(skb)->seq + 1;
+
+ /* recommended */
+ *tail-- ^= ((th->dest << 16) | th->source);
+ *tail-- ^= (u32)cvp; /* per sockopt */
+
+ sha_transform((__u32 *)&xvp->cookie_bakery[0],
+ (char *)mess,
+ &workspace[0]);
+ opts.hash_location =
+ (__u8 *)&xvp->cookie_bakery[0];
+ }
+ }
+
th->seq = htonl(TCP_SKB_CB(skb)->seq);
th->ack_seq = htonl(tcp_rsk(req)->rcv_isn + 1);
diff --git a/net/ipv6/syncookies.c b/net/ipv6/syncookies.c
index 612fc53..5b9af50 100644
--- a/net/ipv6/syncookies.c
+++ b/net/ipv6/syncookies.c
@@ -159,6 +159,8 @@ static inline int cookie_check(struct sk_buff *skb, __u32 cookie)
struct sock *cookie_v6_check(struct sock *sk, struct sk_buff *skb)
{
+ struct tcp_options_received tcp_opt;
+ u8 *hash_location;
struct inet_request_sock *ireq;
struct inet6_request_sock *ireq6;
struct tcp_request_sock *treq;
@@ -171,7 +173,6 @@ struct sock *cookie_v6_check(struct sock *sk, struct sk_buff *skb)
int mss;
struct dst_entry *dst;
__u8 rcv_wscale;
- struct tcp_options_received tcp_opt;
if (!sysctl_tcp_syncookies || !th->ack)
goto out;
@@ -254,7 +255,7 @@ struct sock *cookie_v6_check(struct sock *sk, struct sk_buff *skb)
/* check for timestamp cookie support */
memset(&tcp_opt, 0, sizeof(tcp_opt));
- tcp_parse_options(skb, &tcp_opt, 0, dst);
+ tcp_parse_options(skb, &tcp_opt, &hash_location, 0, dst);
if (tcp_opt.saw_tstamp)
cookie_check_timestamp(&tcp_opt);
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 973096a..a37ce5d 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -1162,7 +1162,9 @@ static struct sock *tcp_v6_hnd_req(struct sock *sk,struct sk_buff *skb)
*/
static int tcp_v6_conn_request(struct sock *sk, struct sk_buff *skb)
{
+ struct tcp_extend_values tmp_ext;
struct tcp_options_received tmp_opt;
+ u8 *hash_location;
struct request_sock *req;
struct inet6_request_sock *treq;
struct ipv6_pinfo *np = inet6_sk(sk);
@@ -1206,8 +1208,52 @@ static int tcp_v6_conn_request(struct sock *sk, struct sk_buff *skb)
tcp_clear_options(&tmp_opt);
tmp_opt.mss_clamp = IPV6_MIN_MTU - sizeof(struct tcphdr) - sizeof(struct ipv6hdr);
tmp_opt.user_mss = tp->rx_opt.user_mss;
+ tcp_parse_options(skb, &tmp_opt, &hash_location, 0, dst);
+
+ if (tmp_opt.cookie_plus > 0
+ && tmp_opt.saw_tstamp
+ && !tp->rx_opt.cookie_out_never
+ && (sysctl_tcp_cookie_size > 0
+ || (tp->cookie_values != NULL
+ && tp->cookie_values->cookie_desired > 0))) {
+ u8 *c;
+ u32 *d;
+ u32 *mess = &tmp_ext.cookie_bakery[COOKIE_DIGEST_WORDS];
+ int l = tmp_opt.cookie_plus - TCPOLEN_COOKIE_BASE;
+
+ if (tcp_cookie_generator(&tmp_ext.cookie_bakery[0]) != 0)
+ goto drop_and_free;
+
+ /* Secret recipe starts with IP addresses */
+ d = &ipv6_hdr(skb)->daddr.s6_addr32[0];
+ *mess++ ^= *d++;
+ *mess++ ^= *d++;
+ *mess++ ^= *d++;
+ *mess++ ^= *d++;
+ d = &ipv6_hdr(skb)->saddr.s6_addr32[0];
+ *mess++ ^= *d++;
+ *mess++ ^= *d++;
+ *mess++ ^= *d++;
+ *mess++ ^= *d++;
+
+ /* plus variable length Initiator Cookie */
+ c = (u8 *)mess;
+ while (l-- > 0)
+ *c++ ^= *hash_location++;
- tcp_parse_options(skb, &tmp_opt, 0, dst);
+#ifdef CONFIG_SYN_COOKIES
+ want_cookie = 0; /* not our kind of cookie */
+#endif
+ tmp_ext.cookie_out_never = 0; /* false */
+ tmp_ext.cookie_plus = tmp_opt.cookie_plus;
+ } else if (!tp->rx_opt.cookie_in_always) {
+ /* redundant indications, but ensure initialization. */
+ tmp_ext.cookie_out_never = 1; /* true */
+ tmp_ext.cookie_plus = 0;
+ } else {
+ goto drop_and_free;
+ }
+ tmp_ext.cookie_in_always = tp->rx_opt.cookie_in_always;
if (want_cookie && !tmp_opt.saw_tstamp)
tcp_clear_options(&tmp_opt);
@@ -1244,7 +1290,8 @@ static int tcp_v6_conn_request(struct sock *sk, struct sk_buff *skb)
security_inet_conn_request(sk, skb, req);
- if (tcp_v6_send_synack(sk, req, NULL)
+ if (tcp_v6_send_synack(sk, req,
+ (struct request_values *)&tmp_ext)
|| want_cookie)
goto drop_and_free;
--
1.6.3.3
^ permalink raw reply related
* [PATCH 0/7] via-velocity performance fixes
From: Simon Kagstrom @ 2009-11-20 15:06 UTC (permalink / raw)
To: netdev, davem, davej, shemminger, romieu
Hi everyone!
I've been fighting with the via-velocity driver for a while,
suffered a few bad blows, but finally managed to land a few patches on
it. I'm sending them together with this mail.
The main reason for the work is to get performance for the mainline
driver back on par with the out-of-tree VIA driver. Most of it are
backports from the VIA driver although there is some original work as
well. The series comes with a RFC tag, and I'd like feedback and
(preferably) testing of the patches since I'm not that familiar with
the driver and Linux networking.
The patches are:
1. Correct setting of skipped checksums (unsure about this). The
mainline driver sets CHECKSUM_UNNECESSARY if this is an IP packet
except if the TCP checksum is NOT ok.
The VIA driver sets CHECKSUM_UNNECESSARY if this is an UDP/TCP
packet except if the TCP checksum is not OK. The patch selects the
VIA behavior.
2. See to it that data is 64-byte aligned (as required by the
hardware). Again different behavior than the VIA driver, and from
looking at the code, it seems to me that VIA handles it correct here.
3. Enable support for adaptive interrupt supression. The velocity
hardware is able to supress interrupts during bursts. This (together
with the next patch) improves behavior quite a bit in my tests.
4. Add NAPI support for via velocity. Also takes in a change in the
interrupt handler from upstream VIA (run rx/tx handlers twice) which
improves performance.
5. Change the DMA_LENGTH_DEF to that of the VIA driver. Large
performance improvement together with the last two patches.
6. Take back the transmit scatter-gather support. A few months after
Dave removed it, it gets back in a fixed manner again :-). I'm
unsure about this one since it doesn't improve performance in my
netperf tests (rather decreases it!).
It might be that I need other tests to benefit from this, or that
it's simply not improving things, but obviously I'm unsure if this
should be added at all.
7. Bump the version number.
The tests I run are basic (quite arbitrary I must say) netperf tests:
#!/bin/sh
netperf -H $1 -c -C -l 20 -t UDP_STREAM
netperf -H $1 -c -C -l 20 -t TCP_STREAM
netperf -H $1 -c -C -l 20 -t TCP_SENDFILE
and I have two identical 1.4GHz Pentium M boards with VIA velocities
that the traffic goes between. The remote board has all patches
applied. The numbers are below:
2.6.32-rc8 without patches
---------------------------
UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to pl-ncaa (169.254.1.33) port 0 AF_INET : demo
Socket Message Elapsed Messages CPU Service
Size Size Time Okay Errors Throughput Util Demand
bytes bytes secs # # 10^6bits/sec % SS us/KB
107520 65507 20.00 20680 0 541.8 41.10 6.214
108544 20.00 20680 541.8 16.96 2.564
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to pl-ncaa (169.254.1.33) port 0 AF_INET : demo
Recv Send Send Utilization Service Demand
Socket Socket Message Elapsed Send Recv Send Recv
Size Size Size Time Throughput local remote local remote
bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
87380 16384 16384 20.02 505.25 60.54 29.52 9.817 4.787
TCP SENDFILE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to pl-ncaa (169.254.1.33) port 0 AF_INET : demo
Recv Send Send Utilization Service Demand
Socket Socket Message Elapsed Send Recv Send Recv
Size Size Size Time Throughput local remote local remote
bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
87380 16384 16384 20.02 507.64 60.45 27.63 9.754 4.458
# cat /proc/interrupts
CPU0
0: 22153 IO-APIC-edge timer
[...]
16: 2673939 IO-APIC-fasteoi uhci_hcd:usb1, eth-swa
2.6.32-rc8 with NAPI + adaptive
--------------------------------
UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to pl-ncaa (169.254.1.33) port 0 AF_INET : demo
Socket Message Elapsed Messages CPU Service
Size Size Time Okay Errors Throughput Util Demand
bytes bytes secs # # 10^6bits/sec % SS us/KB
107520 65507 20.00 26615 0 697.3 17.61 2.069
108544 20.00 26613 697.2 23.55 2.767
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to pl-ncaa (169.254.1.33) port 0 AF_INET : demo
Recv Send Send Utilization Service Demand
Socket Socket Message Elapsed Send Recv Send Recv
Size Size Size Time Throughput local remote local remote
bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
87380 16384 16384 20.02 641.77 41.62 35.61 5.312 4.546
TCP SENDFILE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to pl-ncaa (169.254.1.33) port 0 AF_INET : demo
Recv Send Send Utilization Service Demand
Socket Socket Message Elapsed Send Recv Send Recv
Size Size Size Time Throughput local remote local remote
bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
87380 16384 16384 20.02 641.98 43.76 36.50 5.584 4.658
# cat /proc/interrupts
CPU0
0: 22605 IO-APIC-edge timer
[...]
16: 321020 IO-APIC-fasteoi uhci_hcd:usb1, eth-swa
2.6.32-rc8 with all patches
---------------------------
UDP UNIDIRECTIONAL SEND TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to pl-ncaa (169.254.1.33) port 0 AF_INET : demo
Socket Message Elapsed Messages CPU Service
Size Size Time Okay Errors Throughput Util Demand
bytes bytes secs # # 10^6bits/sec % SS us/KB
107520 65507 20.00 26606 0 697.1 17.60 2.068
108544 20.00 26605 697.1 24.95 2.932
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to pl-ncaa (169.254.1.33) port 0 AF_INET : demo
Recv Send Send Utilization Service Demand
Socket Socket Message Elapsed Send Recv Send Recv
Size Size Size Time Throughput local remote local remote
bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
87380 16384 16384 20.02 563.36 25.58 31.23 3.720 4.542
TCP SENDFILE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to pl-ncaa (169.254.1.33) port 0 AF_INET : demo
Recv Send Send Utilization Service Demand
Socket Socket Message Elapsed Send Recv Send Recv
Size Size Size Time Throughput local remote local remote
bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
87380 16384 16384 20.03 562.54 22.12 30.77 3.221 4.480
# cat /proc/interrupts
CPU0
0: 23652 IO-APIC-edge timer
[...]
16: 341394 IO-APIC-fasteoi uhci_hcd:usb1, eth-swa
As you can see, the best results for this particular test are without
the transmit scatter-gather stuff. Also note the difference in
CPU-utilization and interrupt count between the first and second case,
which is fairly nice. With the patches, the performance is again on par
with the VIA driver.
// Simon
^ permalink raw reply
* [PATCH 1/7] via-velocity: Correct setting of skipped checksums
From: Simon Kagstrom @ 2009-11-20 15:12 UTC (permalink / raw)
To: netdev; +Cc: davem, davej, shemminger, romieu
In-Reply-To: <20091120160633.77b7aee0@marrow.netinsight.se>
(Taken from the VIA driver). Check that this is actually an UDP or TCP
packet before claiming that the checksum is unnecessary.
Signed-off-by: Simon Kagstrom <simon.kagstrom@netinsight.net>
---
drivers/net/via-velocity.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/via-velocity.c b/drivers/net/via-velocity.c
index e04e5be..af79969 100644
--- a/drivers/net/via-velocity.c
+++ b/drivers/net/via-velocity.c
@@ -1924,8 +1924,8 @@ static inline void velocity_rx_csum(struct rx_desc *rd, struct sk_buff *skb)
(rd->rdesc1.CSM & CSM_UDPKT)) {
if (!(rd->rdesc1.CSM & CSM_TUPOK))
return;
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
}
- skb->ip_summed = CHECKSUM_UNNECESSARY;
}
}
}
--
1.6.0.4
^ permalink raw reply related
* [PATCH 2/7] via-velocity: Correct 64-byte alignment for rx buffers
From: Simon Kagstrom @ 2009-11-20 15:12 UTC (permalink / raw)
To: netdev; +Cc: davem, davej, shemminger, romieu
In-Reply-To: <20091120160633.77b7aee0@marrow.netinsight.se>
(From the VIA driver). The current code does not guarantee 64-byte
alignment since it simply does
int add = skb->data & 63;
skb->data += add;
(via skb_reserve). So for example, if add would be 0x10, this
would result in 32-byte alignment.
Correct by adding
64 - (skb->data & 63)
instead.
Signed-off-by: Simon Kagstrom <simon.kagstrom@netinsight.net>
---
drivers/net/via-velocity.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/drivers/net/via-velocity.c b/drivers/net/via-velocity.c
index af79969..944a678 100644
--- a/drivers/net/via-velocity.c
+++ b/drivers/net/via-velocity.c
@@ -1483,7 +1483,8 @@ static int velocity_alloc_rx_buf(struct velocity_info *vptr, int idx)
* Do the gymnastics to get the buffer head for data at
* 64byte alignment.
*/
- skb_reserve(rd_info->skb, (unsigned long) rd_info->skb->data & 63);
+ skb_reserve(rd_info->skb,
+ 64 - ((unsigned long) rd_info->skb->data & 63));
rd_info->skb_dma = pci_map_single(vptr->pdev, rd_info->skb->data,
vptr->rx.buf_sz, PCI_DMA_FROMDEVICE);
--
1.6.0.4
^ permalink raw reply related
* [PATCH 3/7] via-velocity: Enable support for adaptive interrupt supression
From: Simon Kagstrom @ 2009-11-20 15:12 UTC (permalink / raw)
To: netdev; +Cc: davem, davej, shemminger, romieu
In-Reply-To: <20091120160633.77b7aee0@marrow.netinsight.se>
(From the upstream VIA driver). This reduces the amount of interrupts
under load and improves performance by quite a bit by reducing the
amount of work for the CPU.
Signed-off-by: Simon Kagstrom <simon.kagstrom@netinsight.net>
---
drivers/net/via-velocity.c | 70 +++++++++++++++++++++++++++++++++++++++++++-
drivers/net/via-velocity.h | 7 ++++-
2 files changed, 75 insertions(+), 2 deletions(-)
diff --git a/drivers/net/via-velocity.c b/drivers/net/via-velocity.c
index 944a678..932339b 100644
--- a/drivers/net/via-velocity.c
+++ b/drivers/net/via-velocity.c
@@ -364,6 +364,26 @@ static int rx_copybreak = 200;
module_param(rx_copybreak, int, 0644);
MODULE_PARM_DESC(rx_copybreak, "Copy breakpoint for copy-only-tiny-frames");
+#define TXQUEUE_TIMER_DEF 0x59
+#define TXQUEUE_TIMER_MIN 0x00
+#define TXQUEUE_TIMER_MAX 0xFF
+VELOCITY_PARAM(txqueue_timer, "Tx Queue Empty defer timer");
+
+#define RXQUEUE_TIMER_DEF 0x14
+#define RXQUEUE_TIMER_MIN 0x00
+#define RXQUEUE_TIMER_MAX 0xFF
+VELOCITY_PARAM(rxqueue_timer, "Rx Queue Empty defer timer");
+
+#define TX_INTSUP_DEF 0x1F
+#define TX_INTSUP_MIN 0x01
+#define TX_INTSUP_MAX 0x3F
+VELOCITY_PARAM(tx_intsup, "Tx Interrupt Suppression Threshold");
+
+#define RX_INTSUP_DEF 0x1F
+#define RX_INTSUP_MIN 0x01
+#define RX_INTSUP_MAX 0x3F
+VELOCITY_PARAM(rx_intsup, "Rx Interrupt Suppression Threshold");
+
#ifdef CONFIG_PM
static DEFINE_SPINLOCK(velocity_dev_list_lock);
static LIST_HEAD(velocity_dev_list);
@@ -517,6 +537,10 @@ static void __devinit velocity_get_options(struct velocity_opt *opts, int index,
velocity_set_int_opt((int *) &opts->spd_dpx, speed_duplex[index], MED_LNK_MIN, MED_LNK_MAX, MED_LNK_DEF, "Media link mode", devname);
velocity_set_int_opt((int *) &opts->wol_opts, wol_opts[index], WOL_OPT_MIN, WOL_OPT_MAX, WOL_OPT_DEF, "Wake On Lan options", devname);
velocity_set_int_opt((int *) &opts->int_works, int_works[index], INT_WORKS_MIN, INT_WORKS_MAX, INT_WORKS_DEF, "Interrupt service works", devname);
+ velocity_set_int_opt((int *) &opts->txqueue_timer, txqueue_timer[index], TXQUEUE_TIMER_MIN, TXQUEUE_TIMER_MAX, TXQUEUE_TIMER_DEF, "TX queue empty defer timer", devname);
+ velocity_set_int_opt((int *) &opts->rxqueue_timer, rxqueue_timer[index], RXQUEUE_TIMER_MIN, RXQUEUE_TIMER_MAX, RXQUEUE_TIMER_DEF, "RX queue empty defer timer", devname);
+ velocity_set_int_opt((int *) &opts->tx_intsup, tx_intsup[index], TX_INTSUP_MIN, TX_INTSUP_MAX, TX_INTSUP_DEF, "TX interrupt suppression threshold", devname);
+ velocity_set_int_opt((int *) &opts->rx_intsup, rx_intsup[index], RX_INTSUP_MIN, RX_INTSUP_MAX, RX_INTSUP_DEF, "RX interrupt suppression threshold", devname);
opts->numrx = (opts->numrx & ~3);
}
@@ -1259,6 +1283,35 @@ static void mii_init(struct velocity_info *vptr, u32 mii_status)
}
}
+/**
+ * enable_adaptive_interrupts - Turn on interrupt suppression
+ *
+ * @vptr velocity adapter
+ *
+ * The velocity is able to supress interrupt during high interrupt load.
+ * This function turns on that feature.
+ */
+static void enable_adaptive_interrupts(struct velocity_info *vptr)
+{
+ struct mac_regs __iomem *regs = vptr->mac_regs;
+
+ vptr->int_mask = INT_MASK_DEF;
+
+ /* Set Tx Interrupt Suppression Threshold */
+ writeb(CAMCR_PS0, ®s->CAMCR);
+ writew(vptr->options.tx_intsup, ®s->ISRCTL);
+
+ /* Set Rx Interrupt Suppression Threshold */
+ writeb(CAMCR_PS1, ®s->CAMCR);
+ writew(vptr->options.rx_intsup, ®s->ISRCTL);
+
+ /* Select page to interrupt hold timer */
+ writeb(0, ®s->CAMCR);
+
+ /* Modify int mask */
+ vptr->int_mask &= ~(ISR_PTXI | ISR_PTX0I | ISR_PTX1I | ISR_PTX2I |
+ ISR_PTX3I | ISR_PRXI);
+}
/**
* velocity_init_registers - initialise MAC registers
@@ -1345,7 +1398,7 @@ static void velocity_init_registers(struct velocity_info *vptr,
*/
enable_mii_autopoll(regs);
- vptr->int_mask = INT_MASK_DEF;
+ enable_adaptive_interrupts(vptr);
writel(vptr->rx.pool_dma, ®s->RDBaseLo);
writew(vptr->options.numrx - 1, ®s->RDCSize);
@@ -1802,6 +1855,21 @@ static void velocity_error(struct velocity_info *vptr, int status)
BYTE_REG_BITS_OFF(TESTCFG_HBDIS, ®s->TESTCFG);
else
BYTE_REG_BITS_ON(TESTCFG_HBDIS, ®s->TESTCFG);
+
+ /* Adaptive interrupts */
+ if (vptr->rev_id >= REV_ID_VT3216_A0) {
+ u8 txqueue_timer = 0;
+ u8 rxqueue_timer = 0;
+
+ if (vptr->mii_status & (VELOCITY_SPEED_1000 |
+ VELOCITY_SPEED_100)) {
+ txqueue_timer = vptr->options.txqueue_timer;
+ rxqueue_timer = vptr->options.rxqueue_timer;
+ }
+
+ writeb(txqueue_timer, ®s->TQETMR);
+ writeb(rxqueue_timer, ®s->RQETMR);
+ }
}
/*
* Get link status from PHYSR0
diff --git a/drivers/net/via-velocity.h b/drivers/net/via-velocity.h
index 2f00c13..6091946 100644
--- a/drivers/net/via-velocity.h
+++ b/drivers/net/via-velocity.h
@@ -1005,7 +1005,8 @@ struct mac_regs {
volatile __le32 RDBaseLo; /* 0x38 */
volatile __le16 RDIdx; /* 0x3C */
- volatile __le16 reserved_3E;
+ volatile u8 TQETMR; /* 0x3E, VT3216 and above only */
+ volatile u8 RQETMR; /* 0x3F, VT3216 and above only */
volatile __le32 TDBaseLo[4]; /* 0x40 */
@@ -1491,6 +1492,10 @@ struct velocity_opt {
int rx_bandwidth_hi;
int rx_bandwidth_lo;
int rx_bandwidth_en;
+ int rxqueue_timer;
+ int txqueue_timer;
+ int tx_intsup;
+ int rx_intsup;
u32 flags;
};
--
1.6.0.4
^ permalink raw reply related
* [PATCH 4/7] via-velocity: Implement NAPI support
From: Simon Kagstrom @ 2009-11-20 15:12 UTC (permalink / raw)
To: netdev; +Cc: davem, davej, shemminger, romieu
In-Reply-To: <20091120160633.77b7aee0@marrow.netinsight.se>
This patch adds NAPI support for VIA velocity. The new velocity_poll
function also pairs tx/rx handling twice which improves perforamance on
some workloads (e.g., netperf UDP_STREAM) significantly (that part is
from the VIA driver).
Signed-off-by: Simon Kagstrom <simon.kagstrom@netinsight.net>
---
drivers/net/via-velocity.c | 81 +++++++++++++++++++++++++-------------------
drivers/net/via-velocity.h | 3 ++
2 files changed, 49 insertions(+), 35 deletions(-)
diff --git a/drivers/net/via-velocity.c b/drivers/net/via-velocity.c
index 932339b..a59ea12 100644
--- a/drivers/net/via-velocity.c
+++ b/drivers/net/via-velocity.c
@@ -354,12 +354,6 @@ VELOCITY_PARAM(ValPktLen, "Receiving or Drop invalid 802.3 frame");
*/
VELOCITY_PARAM(wol_opts, "Wake On Lan options");
-#define INT_WORKS_DEF 20
-#define INT_WORKS_MIN 10
-#define INT_WORKS_MAX 64
-
-VELOCITY_PARAM(int_works, "Number of packets per interrupt services");
-
static int rx_copybreak = 200;
module_param(rx_copybreak, int, 0644);
MODULE_PARM_DESC(rx_copybreak, "Copy breakpoint for copy-only-tiny-frames");
@@ -536,7 +530,6 @@ static void __devinit velocity_get_options(struct velocity_opt *opts, int index,
velocity_set_bool_opt(&opts->flags, ValPktLen[index], VAL_PKT_LEN_DEF, VELOCITY_FLAGS_VAL_PKT_LEN, "ValPktLen", devname);
velocity_set_int_opt((int *) &opts->spd_dpx, speed_duplex[index], MED_LNK_MIN, MED_LNK_MAX, MED_LNK_DEF, "Media link mode", devname);
velocity_set_int_opt((int *) &opts->wol_opts, wol_opts[index], WOL_OPT_MIN, WOL_OPT_MAX, WOL_OPT_DEF, "Wake On Lan options", devname);
- velocity_set_int_opt((int *) &opts->int_works, int_works[index], INT_WORKS_MIN, INT_WORKS_MAX, INT_WORKS_DEF, "Interrupt service works", devname);
velocity_set_int_opt((int *) &opts->txqueue_timer, txqueue_timer[index], TXQUEUE_TIMER_MIN, TXQUEUE_TIMER_MAX, TXQUEUE_TIMER_DEF, "TX queue empty defer timer", devname);
velocity_set_int_opt((int *) &opts->rxqueue_timer, rxqueue_timer[index], RXQUEUE_TIMER_MIN, RXQUEUE_TIMER_MAX, RXQUEUE_TIMER_DEF, "RX queue empty defer timer", devname);
velocity_set_int_opt((int *) &opts->tx_intsup, tx_intsup[index], TX_INTSUP_MIN, TX_INTSUP_MAX, TX_INTSUP_DEF, "TX interrupt suppression threshold", devname);
@@ -2129,13 +2122,14 @@ static int velocity_receive_frame(struct velocity_info *vptr, int idx)
* any received packets from the receive queue. Hand the ring
* slots back to the adapter for reuse.
*/
-static int velocity_rx_srv(struct velocity_info *vptr, int status)
+static int velocity_rx_srv(struct velocity_info *vptr, int status,
+ int budget_left)
{
struct net_device_stats *stats = &vptr->dev->stats;
int rd_curr = vptr->rx.curr;
int works = 0;
- do {
+ while (works < budget_left) {
struct rx_desc *rd = vptr->rx.ring + rd_curr;
if (!vptr->rx.info[rd_curr].skb)
@@ -2166,7 +2160,8 @@ static int velocity_rx_srv(struct velocity_info *vptr, int status)
rd_curr++;
if (rd_curr >= vptr->options.numrx)
rd_curr = 0;
- } while (++works <= 15);
+ works++;
+ }
vptr->rx.curr = rd_curr;
@@ -2177,6 +2172,40 @@ static int velocity_rx_srv(struct velocity_info *vptr, int status)
return works;
}
+static int velocity_poll(struct napi_struct *napi, int budget)
+{
+ struct velocity_info *vptr = container_of(napi,
+ struct velocity_info, napi);
+ unsigned int rx_done;
+ u32 isr_status;
+
+ spin_lock(&vptr->lock);
+ isr_status = mac_read_isr(vptr->mac_regs);
+
+ /* Ack the interrupt */
+ mac_write_isr(vptr->mac_regs, isr_status);
+ if (isr_status & (~(ISR_PRXI | ISR_PPRXI | ISR_PTXI | ISR_PPTXI)))
+ velocity_error(vptr, isr_status);
+
+ /*
+ * Do rx and tx twice for performance (taken from the VIA
+ * out-of-tree driver).
+ */
+ rx_done = velocity_rx_srv(vptr, isr_status, budget / 2);
+ velocity_tx_srv(vptr, isr_status);
+ rx_done += velocity_rx_srv(vptr, isr_status, budget - rx_done);
+ velocity_tx_srv(vptr, isr_status);
+
+ spin_unlock(&vptr->lock);
+
+ /* If budget not fully consumed, exit the polling mode */
+ if (rx_done < budget) {
+ napi_complete(napi);
+ mac_enable_int(vptr->mac_regs);
+ }
+
+ return rx_done;
+}
/**
* velocity_intr - interrupt callback
@@ -2193,8 +2222,6 @@ static irqreturn_t velocity_intr(int irq, void *dev_instance)
struct net_device *dev = dev_instance;
struct velocity_info *vptr = netdev_priv(dev);
u32 isr_status;
- int max_count = 0;
-
spin_lock(&vptr->lock);
isr_status = mac_read_isr(vptr->mac_regs);
@@ -2205,32 +2232,13 @@ static irqreturn_t velocity_intr(int irq, void *dev_instance)
return IRQ_NONE;
}
- mac_disable_int(vptr->mac_regs);
-
- /*
- * Keep processing the ISR until we have completed
- * processing and the isr_status becomes zero
- */
-
- while (isr_status != 0) {
- mac_write_isr(vptr->mac_regs, isr_status);
- if (isr_status & (~(ISR_PRXI | ISR_PPRXI | ISR_PTXI | ISR_PPTXI)))
- velocity_error(vptr, isr_status);
- if (isr_status & (ISR_PRXI | ISR_PPRXI))
- max_count += velocity_rx_srv(vptr, isr_status);
- if (isr_status & (ISR_PTXI | ISR_PPTXI))
- max_count += velocity_tx_srv(vptr, isr_status);
- isr_status = mac_read_isr(vptr->mac_regs);
- if (max_count > vptr->options.int_works) {
- printk(KERN_WARNING "%s: excessive work at interrupt.\n",
- dev->name);
- max_count = 0;
- }
+ if (likely(napi_schedule_prep(&vptr->napi))) {
+ mac_disable_int(vptr->mac_regs);
+ __napi_schedule(&vptr->napi);
}
spin_unlock(&vptr->lock);
- mac_enable_int(vptr->mac_regs);
- return IRQ_HANDLED;
+ return IRQ_HANDLED;
}
/**
@@ -2270,6 +2278,7 @@ static int velocity_open(struct net_device *dev)
mac_enable_int(vptr->mac_regs);
netif_start_queue(dev);
+ napi_enable(&vptr->napi);
vptr->flags |= VELOCITY_FLAGS_OPENED;
out:
return ret;
@@ -2505,6 +2514,7 @@ static int velocity_close(struct net_device *dev)
{
struct velocity_info *vptr = netdev_priv(dev);
+ napi_disable(&vptr->napi);
netif_stop_queue(dev);
velocity_shutdown(vptr);
@@ -2824,6 +2834,7 @@ static int __devinit velocity_found1(struct pci_dev *pdev, const struct pci_devi
dev->irq = pdev->irq;
dev->netdev_ops = &velocity_netdev_ops;
dev->ethtool_ops = &velocity_ethtool_ops;
+ netif_napi_add(dev, &vptr->napi, velocity_poll, VELOCITY_NAPI_WEIGHT);
dev->features |= NETIF_F_HW_VLAN_TX | NETIF_F_HW_VLAN_FILTER |
NETIF_F_HW_VLAN_RX;
diff --git a/drivers/net/via-velocity.h b/drivers/net/via-velocity.h
index 6091946..22bfea4 100644
--- a/drivers/net/via-velocity.h
+++ b/drivers/net/via-velocity.h
@@ -32,6 +32,7 @@
#define VELOCITY_VERSION "1.14"
#define VELOCITY_IO_SIZE 256
+#define VELOCITY_NAPI_WEIGHT 64
#define PKT_BUF_SZ 1540
@@ -1564,6 +1565,8 @@ struct velocity_info {
u32 ticks;
u8 rev_id;
+
+ struct napi_struct napi;
};
/**
--
1.6.0.4
^ permalink raw reply related
* [PATCH 5/7] via-velocity: Change DMA_LENGTH_DEF (from the VIA driver)
From: Simon Kagstrom @ 2009-11-20 15:12 UTC (permalink / raw)
To: netdev; +Cc: davem, davej, shemminger, romieu
In-Reply-To: <20091120160633.77b7aee0@marrow.netinsight.se>
The VIA driver has changed the default for the DMA_LENGTH_DEF parameter.
Together with adaptive interrupt supression and NAPI support, this
improves performance quite a bit
Signed-off-by: Simon Kagstrom <simon.kagstrom@netinsight.net>
---
drivers/net/via-velocity.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/via-velocity.c b/drivers/net/via-velocity.c
index a59ea12..be75814 100644
--- a/drivers/net/via-velocity.c
+++ b/drivers/net/via-velocity.c
@@ -275,7 +275,7 @@ VELOCITY_PARAM(rx_thresh, "Receive fifo threshold");
#define DMA_LENGTH_MIN 0
#define DMA_LENGTH_MAX 7
-#define DMA_LENGTH_DEF 0
+#define DMA_LENGTH_DEF 6
/* DMA_length[] is used for controlling the DMA length
0: 8 DWORDs
--
1.6.0.4
^ permalink raw reply related
* [PATCH 6/7] via-velocity: Re-enable the transmit scatter-gather support
From: Simon Kagstrom @ 2009-11-20 15:12 UTC (permalink / raw)
To: netdev; +Cc: davem, davej, shemminger, romieu
In-Reply-To: <20091120160633.77b7aee0@marrow.netinsight.se>
The velocity hardware can handle up to 7 memory segments.
Signed-off-by: Simon Kagstrom <simon.kagstrom@netinsight.net>
---
drivers/net/via-velocity.c | 89 ++++++++++++++++++++++++++++---------------
1 files changed, 58 insertions(+), 31 deletions(-)
diff --git a/drivers/net/via-velocity.c b/drivers/net/via-velocity.c
index be75814..8cce9ea 100644
--- a/drivers/net/via-velocity.c
+++ b/drivers/net/via-velocity.c
@@ -9,7 +9,6 @@
*
* TODO
* rx_copybreak/alignment
- * Scatter gather
* More testing
*
* The changes are (c) Copyright 2004, Red Hat Inc. <alan@lxorguk.ukuu.org.uk>
@@ -1649,12 +1648,10 @@ out:
*/
static int velocity_init_td_ring(struct velocity_info *vptr)
{
- dma_addr_t curr;
int j;
/* Init the TD ring entries */
for (j = 0; j < vptr->tx.numq; j++) {
- curr = vptr->tx.pool_dma[j];
vptr->tx.infos[j] = kcalloc(vptr->options.numtx,
sizeof(struct velocity_td_info),
@@ -1720,21 +1717,27 @@ err_free_dma_rings_0:
* Release an transmit buffer. If the buffer was preallocated then
* recycle it, if not then unmap the buffer.
*/
-static void velocity_free_tx_buf(struct velocity_info *vptr, struct velocity_td_info *tdinfo)
+static void velocity_free_tx_buf(struct velocity_info *vptr,
+ struct velocity_td_info *tdinfo, struct tx_desc *td)
{
struct sk_buff *skb = tdinfo->skb;
- int i;
- int pktlen;
/*
* Don't unmap the pre-allocated tx_bufs
*/
if (tdinfo->skb_dma) {
+ int i;
- pktlen = max_t(unsigned int, skb->len, ETH_ZLEN);
for (i = 0; i < tdinfo->nskb_dma; i++) {
- pci_unmap_single(vptr->pdev, tdinfo->skb_dma[i], pktlen, PCI_DMA_TODEVICE);
- tdinfo->skb_dma[i] = 0;
+ size_t pktlen = max_t(size_t, skb->len, ETH_ZLEN);
+
+ /* For scatter-gather */
+ if (skb_shinfo(skb)->nr_frags > 0)
+ pktlen = max_t(size_t, pktlen,
+ td->td_buf[i].size & ~TD_QUEUE);
+
+ pci_unmap_single(vptr->pdev, tdinfo->skb_dma[i],
+ le16_to_cpu(pktlen), PCI_DMA_TODEVICE);
}
}
dev_kfree_skb_irq(skb);
@@ -1949,7 +1952,7 @@ static int velocity_tx_srv(struct velocity_info *vptr, u32 status)
stats->tx_packets++;
stats->tx_bytes += tdinfo->skb->len;
}
- velocity_free_tx_buf(vptr, tdinfo);
+ velocity_free_tx_buf(vptr, tdinfo, td);
vptr->tx.used[qnum]--;
}
vptr->tx.tail[qnum] = idx;
@@ -2549,14 +2552,27 @@ static netdev_tx_t velocity_xmit(struct sk_buff *skb,
struct velocity_td_info *tdinfo;
unsigned long flags;
int pktlen;
- __le16 len;
- int index;
+ int index, prev;
+ int i = 0;
if (skb_padto(skb, ETH_ZLEN))
goto out;
- pktlen = max_t(unsigned int, skb->len, ETH_ZLEN);
- len = cpu_to_le16(pktlen);
+ /* The hardware can handle at most 7 memory segments, so merge
+ * the skb if there are more */
+ if (skb_shinfo(skb)->nr_frags > 6 && __skb_linearize(skb)) {
+ kfree_skb(skb);
+ return 0;
+ }
+ /* If it's still above 6 we can't do anything */
+ if (skb_shinfo(skb)->nr_frags > 6) {
+ dev_err(&vptr->pdev->dev,
+ "via-velocity: more than 6 frags, can't send.\n");
+ return 0;
+ }
+ pktlen = skb_shinfo(skb)->nr_frags == 0 ?
+ max_t(unsigned int, skb->len, ETH_ZLEN) :
+ skb_headlen(skb);
spin_lock_irqsave(&vptr->lock, flags);
@@ -2573,11 +2589,24 @@ static netdev_tx_t velocity_xmit(struct sk_buff *skb,
*/
tdinfo->skb = skb;
tdinfo->skb_dma[0] = pci_map_single(vptr->pdev, skb->data, pktlen, PCI_DMA_TODEVICE);
- td_ptr->tdesc0.len = len;
+ td_ptr->tdesc0.len = cpu_to_le16(pktlen);
td_ptr->td_buf[0].pa_low = cpu_to_le32(tdinfo->skb_dma[0]);
td_ptr->td_buf[0].pa_high = 0;
- td_ptr->td_buf[0].size = len;
- tdinfo->nskb_dma = 1;
+ td_ptr->td_buf[0].size = cpu_to_le16(pktlen);
+
+ /* Handle fragments */
+ for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
+ skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
+
+ tdinfo->skb_dma[i + 1] = pci_map_page(vptr->pdev, frag->page,
+ frag->page_offset, frag->size,
+ PCI_DMA_TODEVICE);
+
+ td_ptr->td_buf[i + 1].pa_low = cpu_to_le32(tdinfo->skb_dma[i + 1]);
+ td_ptr->td_buf[i + 1].pa_high = 0;
+ td_ptr->td_buf[i + 1].size = cpu_to_le16(frag->size);
+ }
+ tdinfo->nskb_dma = i + 1;
td_ptr->tdesc1.cmd = TCPLS_NORMAL + (tdinfo->nskb_dma + 1) * 16;
@@ -2598,23 +2627,21 @@ static netdev_tx_t velocity_xmit(struct sk_buff *skb,
td_ptr->tdesc1.TCR |= (TCR0_UDPCK);
td_ptr->tdesc1.TCR |= TCR0_IPCK;
}
- {
- int prev = index - 1;
+ prev = index - 1;
+ if (prev < 0)
+ prev = vptr->options.numtx - 1;
+ td_ptr->tdesc0.len |= OWNED_BY_NIC;
+ vptr->tx.used[qnum]++;
+ vptr->tx.curr[qnum] = (index + 1) % vptr->options.numtx;
- if (prev < 0)
- prev = vptr->options.numtx - 1;
- td_ptr->tdesc0.len |= OWNED_BY_NIC;
- vptr->tx.used[qnum]++;
- vptr->tx.curr[qnum] = (index + 1) % vptr->options.numtx;
+ if (AVAIL_TD(vptr, qnum) < 1)
+ netif_stop_queue(dev);
- if (AVAIL_TD(vptr, qnum) < 1)
- netif_stop_queue(dev);
+ td_ptr = &(vptr->tx.rings[qnum][prev]);
+ td_ptr->td_buf[0].size |= TD_QUEUE;
+ mac_tx_queue_wake(vptr->mac_regs, qnum);
- td_ptr = &(vptr->tx.rings[qnum][prev]);
- td_ptr->td_buf[0].size |= TD_QUEUE;
- mac_tx_queue_wake(vptr->mac_regs, qnum);
- }
dev->trans_start = jiffies;
spin_unlock_irqrestore(&vptr->lock, flags);
out:
@@ -2837,7 +2864,7 @@ static int __devinit velocity_found1(struct pci_dev *pdev, const struct pci_devi
netif_napi_add(dev, &vptr->napi, velocity_poll, VELOCITY_NAPI_WEIGHT);
dev->features |= NETIF_F_HW_VLAN_TX | NETIF_F_HW_VLAN_FILTER |
- NETIF_F_HW_VLAN_RX;
+ NETIF_F_HW_VLAN_RX | NETIF_F_SG;
if (vptr->flags & VELOCITY_FLAGS_TX_CSUM)
dev->features |= NETIF_F_IP_CSUM;
--
1.6.0.4
^ permalink raw reply related
* [PATCH 7/7] via-velocity: Bump version
From: Simon Kagstrom @ 2009-11-20 15:12 UTC (permalink / raw)
To: netdev; +Cc: davem, davej, shemminger, romieu
In-Reply-To: <20091120160633.77b7aee0@marrow.netinsight.se>
Signed-off-by: Simon Kagstrom <simon.kagstrom@netinsight.net>
---
drivers/net/via-velocity.h | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/via-velocity.h b/drivers/net/via-velocity.h
index 22bfea4..083585b 100644
--- a/drivers/net/via-velocity.h
+++ b/drivers/net/via-velocity.h
@@ -29,7 +29,7 @@
#define VELOCITY_NAME "via-velocity"
#define VELOCITY_FULL_DRV_NAM "VIA Networking Velocity Family Gigabit Ethernet Adapter Driver"
-#define VELOCITY_VERSION "1.14"
+#define VELOCITY_VERSION "1.15"
#define VELOCITY_IO_SIZE 256
#define VELOCITY_NAPI_WEIGHT 64
--
1.6.0.4
^ permalink raw reply related
* Re: [PATCH 1/1] Defer skb allocation for both mergeable buffers and big packets in virtio_net
From: Shirley Ma @ 2009-11-20 16:08 UTC (permalink / raw)
To: Eric Dumazet
Cc: Michael S. Tsirkin, Avi Kivity, Rusty Russell, netdev, kvm,
linux-kernel
In-Reply-To: <4B063509.10006@gmail.com>
On Fri, 2009-11-20 at 07:19 +0100, Eric Dumazet wrote:
> > +void virtio_free_pages(void *buf)
> > +{
> > + struct page *page = (struct page *)buf;
> > +
> > + while (page) {
> > + __free_pages(page, 0);
> > + page = (struct page *)page->private;
> > + }
> > +}
> > +
>
> Interesting use after free :)
Good catch. This code was run when virtio_net removal. I run many times
of remove virtio_net, and didn't hit any panic :(. Fixed it as below:
void virtio_free_pages(void *buf)
{
struct page *page = (struct page *)buf;
struct page *npage;
while (page) {
npage = page->private;
__free_pages(page, 0);
page = npage;
}
}
Thanks
Shirley
^ permalink raw reply
* Re: [BUG] netxen: Stops working between 2.6.30 and 2.6.31-rc1
From: Jens Rosenboom @ 2009-11-20 16:11 UTC (permalink / raw)
To: Jens Rosenboom; +Cc: Dhananjay Phadke, netdev@vger.kernel.org, Amit Salecha
In-Reply-To: <20091120074903.GL14661@jayr.de>
On Fri, Nov 20, 2009 at 08:49:03AM +0100, Jens Rosenboom wrote:
> On Thu, Nov 19, 2009 at 02:11:33PM -0800, Dhananjay Phadke wrote:
> > > Sorry, I forgot to mention that all later kernels that I tested
> > > including 2.6.31 and the current net-2.6 also fail, so the badness
> > > comes in somewhere in between 2.6.30 and 2.6.31-rc1.
> > >
> > > I also noticed that the newer kernel allocate four interrupts for the
> > > card instead of only one, but none of them seem to get triggered, the
> > > /proc/interrupts counters all stay at zero.
> >
> > What firmware revision you have? Since you are saying nothing
> > transmitted as well, I doubt if you have a link. Otherwise I
> > would imagine kernel tries to send some neighbor solicitation
> > crap as soon as you bring up interface. What does your
> > "ethtool ethx" say about the link?
>
> ethtool says "Link detected: yes" , if I try to ping a different host on the
> LAN the MAC of the card appears in the FDB on the switch, so I'm pretty sure
> that packets do get sent even if the kernel doesn't get a report for that
> because of the broken interrupts. Firmware is 3.4.336, which is the only one
> I could find from IBM Japan, the original Netxen pages seem to have been dumped
> by Qlogic. :-( The firmware on the card itself is being rejected by the
> kernel as too old.
>
> > It's possible to bisect commits which applied in driver/net/netxen.
> > That way you have fewer commits to rewind and remains focused on
> > the driver rather than screwing scsi.
>
> I did restrict the bisect to net/ + driver/net and still ran into trouble,
> I can retry with your suggestion.
Sorry for following up to myself, but I made some progress. The bisect still
broke things, so I started to try to patch the latest kernel to use only a
single interrupt, but that didn't help either.
But I managed in finding another firmware which has version v3.4.250, which
is called "legacy" by the kernel. Loading this firmware results also in the
driver only using one interrupt, and the good news is: It Works. ;-)
Maybe this helps you to further narrow down the problem, I'm also ready to
take some testing/debugging patches or send you any other information that
might be helpful.
^ permalink raw reply
* Re: [PATCH 1/1] Defer skb allocation for both mergeable buffers and big packets in virtio_net
From: Shirley Ma @ 2009-11-20 16:21 UTC (permalink / raw)
To: Eric Dumazet
Cc: Michael S. Tsirkin, Avi Kivity, Rusty Russell, netdev, kvm,
linux-kernel
In-Reply-To: <4B063509.10006@gmail.com>
On Fri, 2009-11-20 at 07:19 +0100, Eric Dumazet wrote:
> Interesting use after free :)
Thanks for catching the stupid mistake. This is the updated patch for
review.
Signed-off-by: Shirley Ma (xma@us.ibm.com)
------
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index b9e002f..5699bd3 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -80,33 +80,50 @@ static inline struct skb_vnet_hdr *skb_vnet_hdr(struct sk_buff *skb)
return (struct skb_vnet_hdr *)skb->cb;
}
-static void give_a_page(struct virtnet_info *vi, struct page *page)
+static void give_pages(struct virtnet_info *vi, struct page *page)
{
- page->private = (unsigned long)vi->pages;
+ struct page *npage = (struct page *)page->private;
+
+ if (!npage)
+ page->private = (unsigned long)vi->pages;
+ else {
+ /* give a page list */
+ while (npage) {
+ if (npage->private == (unsigned long)0) {
+ npage->private = (unsigned long)vi->pages;
+ break;
+ }
+ npage = (struct page *)npage->private;
+ }
+ }
vi->pages = page;
}
-static void trim_pages(struct virtnet_info *vi, struct sk_buff *skb)
-{
- unsigned int i;
-
- for (i = 0; i < skb_shinfo(skb)->nr_frags; i++)
- give_a_page(vi, skb_shinfo(skb)->frags[i].page);
- skb_shinfo(skb)->nr_frags = 0;
- skb->data_len = 0;
-}
-
static struct page *get_a_page(struct virtnet_info *vi, gfp_t gfp_mask)
{
struct page *p = vi->pages;
- if (p)
+ if (p) {
vi->pages = (struct page *)p->private;
- else
+ /* use private to chain big packets */
+ p->private = (unsigned long)0;
+ } else
p = alloc_page(gfp_mask);
return p;
}
+void virtio_free_pages(void *buf)
+{
+ struct page *page = (struct page *)buf;
+ struct page *npage;
+
+ while (page) {
+ npage = (struct page *)page->private;
+ __free_pages(page, 0);
+ page = npage;
+ }
+}
+
static void skb_xmit_done(struct virtqueue *svq)
{
struct virtnet_info *vi = svq->vdev->priv;
@@ -118,12 +135,36 @@ static void skb_xmit_done(struct virtqueue *svq)
netif_wake_queue(vi->dev);
}
-static void receive_skb(struct net_device *dev, struct sk_buff *skb,
+static int set_skb_frags(struct sk_buff *skb, struct page *page,
+ int offset, int len)
+{
+ int i = skb_shinfo(skb)->nr_frags;
+ skb_frag_t *f;
+
+ i = skb_shinfo(skb)->nr_frags;
+ f = &skb_shinfo(skb)->frags[i];
+ f->page = page;
+ f->page_offset = offset;
+
+ if (len > (PAGE_SIZE - f->page_offset))
+ f->size = PAGE_SIZE - f->page_offset;
+ else
+ f->size = len;
+
+ skb_shinfo(skb)->nr_frags++;
+ skb->data_len += f->size;
+ skb->len += f->size;
+
+ len -= f->size;
+ return len;
+}
+
+static void receive_skb(struct net_device *dev, void *buf,
unsigned len)
{
struct virtnet_info *vi = netdev_priv(dev);
- struct skb_vnet_hdr *hdr = skb_vnet_hdr(skb);
- int err;
+ struct skb_vnet_hdr *hdr;
+ struct sk_buff *skb;
int i;
if (unlikely(len < sizeof(struct virtio_net_hdr) + ETH_HLEN)) {
@@ -132,39 +173,71 @@ static void receive_skb(struct net_device *dev, struct sk_buff *skb,
goto drop;
}
- if (vi->mergeable_rx_bufs) {
- unsigned int copy;
- char *p = page_address(skb_shinfo(skb)->frags[0].page);
+ if (!vi->mergeable_rx_bufs && !vi->big_packets) {
+ skb = (struct sk_buff *)buf;
+
+ __skb_unlink(skb, &vi->recv);
+
+ hdr = skb_vnet_hdr(skb);
+ len -= sizeof(hdr->hdr);
+ skb_trim(skb, len);
+ } else {
+ struct page *page = (struct page *)buf;
+ int copy, hdr_len, num_buf, offset;
+ char *p;
+
+ p = page_address(page);
- if (len > PAGE_SIZE)
- len = PAGE_SIZE;
- len -= sizeof(struct virtio_net_hdr_mrg_rxbuf);
+ skb = netdev_alloc_skb(vi->dev, GOOD_COPY_LEN + NET_IP_ALIGN);
+ if (unlikely(!skb)) {
+ dev->stats.rx_dropped++;
+ return;
+ }
+ skb_reserve(skb, NET_IP_ALIGN);
+ hdr = skb_vnet_hdr(skb);
- memcpy(&hdr->mhdr, p, sizeof(hdr->mhdr));
- p += sizeof(hdr->mhdr);
+ if (vi->mergeable_rx_bufs) {
+ hdr_len = sizeof(hdr->mhdr);
+ memcpy(&hdr->mhdr, p, hdr_len);
+ num_buf = hdr->mhdr.num_buffers;
+ offset = hdr_len;
+ if (len > PAGE_SIZE)
+ len = PAGE_SIZE;
+ } else {
+ /* big packtes 6 bytes alignment between virtio_net
+ * header and data */
+ hdr_len = sizeof(hdr->hdr);
+ memcpy(&hdr->hdr, p, hdr_len);
+ offset = hdr_len + 6;
+ }
+
+ p += offset;
+ len -= hdr_len;
copy = len;
if (copy > skb_tailroom(skb))
copy = skb_tailroom(skb);
-
memcpy(skb_put(skb, copy), p, copy);
len -= copy;
- if (!len) {
- give_a_page(vi, skb_shinfo(skb)->frags[0].page);
- skb_shinfo(skb)->nr_frags--;
- } else {
- skb_shinfo(skb)->frags[0].page_offset +=
- sizeof(hdr->mhdr) + copy;
- skb_shinfo(skb)->frags[0].size = len;
- skb->data_len += len;
- skb->len += len;
+ if (!len)
+ give_pages(vi, page);
+ else {
+ len = set_skb_frags(skb, page, copy + offset, len);
+ /* process big packets */
+ while (len > 0) {
+ page = (struct page *)page->private;
+ if (!page)
+ break;
+ len = set_skb_frags(skb, page, 0, len);
+ }
+ if (page && page->private)
+ give_pages(vi, (struct page *)page->private);
}
- while (--hdr->mhdr.num_buffers) {
- struct sk_buff *nskb;
-
+ /* process mergeable buffers */
+ while (vi->mergeable_rx_bufs && --num_buf) {
i = skb_shinfo(skb)->nr_frags;
if (i >= MAX_SKB_FRAGS) {
pr_debug("%s: packet too long %d\n", dev->name,
@@ -173,41 +246,20 @@ static void receive_skb(struct net_device *dev, struct sk_buff *skb,
goto drop;
}
- nskb = vi->rvq->vq_ops->get_buf(vi->rvq, &len);
- if (!nskb) {
+ page = vi->rvq->vq_ops->get_buf(vi->rvq, &len);
+ if (!page) {
pr_debug("%s: rx error: %d buffers missing\n",
dev->name, hdr->mhdr.num_buffers);
dev->stats.rx_length_errors++;
goto drop;
}
- __skb_unlink(nskb, &vi->recv);
- vi->num--;
-
- skb_shinfo(skb)->frags[i] = skb_shinfo(nskb)->frags[0];
- skb_shinfo(nskb)->nr_frags = 0;
- kfree_skb(nskb);
-
if (len > PAGE_SIZE)
len = PAGE_SIZE;
- skb_shinfo(skb)->frags[i].size = len;
- skb_shinfo(skb)->nr_frags++;
- skb->data_len += len;
- skb->len += len;
- }
- } else {
- len -= sizeof(hdr->hdr);
-
- if (len <= MAX_PACKET_LEN)
- trim_pages(vi, skb);
+ set_skb_frags(skb, page, 0, len);
- err = pskb_trim(skb, len);
- if (err) {
- pr_debug("%s: pskb_trim failed %i %d\n", dev->name,
- len, err);
- dev->stats.rx_dropped++;
- goto drop;
+ vi->num--;
}
}
@@ -271,107 +323,105 @@ drop:
dev_kfree_skb(skb);
}
-static bool try_fill_recv_maxbufs(struct virtnet_info *vi, gfp_t gfp)
+/* Returns false if we couldn't fill entirely (OOM). */
+static bool try_fill_recv(struct virtnet_info *vi, gfp_t gfp)
{
- struct sk_buff *skb;
struct scatterlist sg[2+MAX_SKB_FRAGS];
- int num, err, i;
+ int err = 0;
bool oom = false;
sg_init_table(sg, 2+MAX_SKB_FRAGS);
do {
- struct skb_vnet_hdr *hdr;
-
- skb = netdev_alloc_skb(vi->dev, MAX_PACKET_LEN + NET_IP_ALIGN);
- if (unlikely(!skb)) {
- oom = true;
- break;
- }
-
- skb_reserve(skb, NET_IP_ALIGN);
- skb_put(skb, MAX_PACKET_LEN);
-
- hdr = skb_vnet_hdr(skb);
- sg_set_buf(sg, &hdr->hdr, sizeof(hdr->hdr));
-
- if (vi->big_packets) {
- for (i = 0; i < MAX_SKB_FRAGS; i++) {
- skb_frag_t *f = &skb_shinfo(skb)->frags[i];
- f->page = get_a_page(vi, gfp);
- if (!f->page)
- break;
-
- f->page_offset = 0;
- f->size = PAGE_SIZE;
-
- skb->data_len += PAGE_SIZE;
- skb->len += PAGE_SIZE;
-
- skb_shinfo(skb)->nr_frags++;
+ /* allocate skb for MAX_PACKET_LEN len */
+ if (!vi->big_packets && !vi->mergeable_rx_bufs) {
+ struct skb_vnet_hdr *hdr;
+ struct sk_buff *skb;
+
+ skb = netdev_alloc_skb(vi->dev,
+ MAX_PACKET_LEN + NET_IP_ALIGN);
+ if (unlikely(!skb)) {
+ oom = true;
+ break;
}
- }
-
- num = skb_to_sgvec(skb, sg+1, 0, skb->len) + 1;
- skb_queue_head(&vi->recv, skb);
-
- err = vi->rvq->vq_ops->add_buf(vi->rvq, sg, 0, num, skb);
- if (err < 0) {
- skb_unlink(skb, &vi->recv);
- trim_pages(vi, skb);
- kfree_skb(skb);
- break;
- }
- vi->num++;
- } while (err >= num);
- if (unlikely(vi->num > vi->max))
- vi->max = vi->num;
- vi->rvq->vq_ops->kick(vi->rvq);
- return !oom;
-}
-
-/* Returns false if we couldn't fill entirely (OOM). */
-static bool try_fill_recv(struct virtnet_info *vi, gfp_t gfp)
-{
- struct sk_buff *skb;
- struct scatterlist sg[1];
- int err;
- bool oom = false;
- if (!vi->mergeable_rx_bufs)
- return try_fill_recv_maxbufs(vi, gfp);
+ skb_reserve(skb, NET_IP_ALIGN);
+ skb_put(skb, MAX_PACKET_LEN);
- do {
- skb_frag_t *f;
+ hdr = skb_vnet_hdr(skb);
+ sg_set_buf(sg, &hdr->hdr, sizeof(hdr->hdr));
- skb = netdev_alloc_skb(vi->dev, GOOD_COPY_LEN + NET_IP_ALIGN);
- if (unlikely(!skb)) {
- oom = true;
- break;
- }
-
- skb_reserve(skb, NET_IP_ALIGN);
-
- f = &skb_shinfo(skb)->frags[0];
- f->page = get_a_page(vi, gfp);
- if (!f->page) {
- oom = true;
- kfree_skb(skb);
- break;
- }
+ skb_to_sgvec(skb, sg+1, 0, skb->len);
+ skb_queue_head(&vi->recv, skb);
- f->page_offset = 0;
- f->size = PAGE_SIZE;
-
- skb_shinfo(skb)->nr_frags++;
+ err = vi->rvq->vq_ops->add_buf(vi->rvq, sg, 0, 2, skb);
+ if (err < 0) {
+ skb_unlink(skb, &vi->recv);
+ kfree_skb(skb);
+ break;
+ }
- sg_init_one(sg, page_address(f->page), PAGE_SIZE);
- skb_queue_head(&vi->recv, skb);
+ } else {
+ struct page *first_page = NULL;
+ struct page *page;
+ int i = MAX_SKB_FRAGS + 2;
+ char *p;
+
+ /*
+ * chain pages for big packets, allocate skb
+ * late for both big packets and mergeable
+ * buffers
+ */
+more: page = get_a_page(vi, gfp);
+ if (!page) {
+ if (first_page)
+ give_pages(vi, first_page);
+ oom = true;
+ break;
+ }
- err = vi->rvq->vq_ops->add_buf(vi->rvq, sg, 0, 1, skb);
- if (err < 0) {
- skb_unlink(skb, &vi->recv);
- kfree_skb(skb);
- break;
+ p = page_address(page);
+ if (vi->mergeable_rx_bufs) {
+ sg_init_one(sg, p, PAGE_SIZE);
+ err = vi->rvq->vq_ops->add_buf(vi->rvq, sg, 0,
+ 1, page);
+ if (err < 0) {
+ give_pages(vi, page);
+ break;
+ }
+ } else {
+ int hdr_len = sizeof(struct virtio_net_hdr);
+
+ /*
+ * allocate MAX_SKB_FRAGS + 1 pages for
+ * big packets
+ */
+ page->private = (unsigned long)first_page;
+ first_page = page;
+ if (--i == 1) {
+ int offset = hdr_len + 6;
+
+ /*
+ * share one page between virtio_net
+ * header and data, and reserve 6 bytes
+ * for alignment
+ */
+ sg_set_buf(sg, p, hdr_len);
+ sg_set_buf(sg+1, p + offset,
+ PAGE_SIZE - offset);
+ err = vi->rvq->vq_ops->add_buf(vi->rvq,
+ sg, 0,
+ MAX_SKB_FRAGS + 2,
+ first_page);
+ if (err < 0) {
+ give_pages(vi, first_page);
+ break;
+ }
+
+ } else {
+ sg_set_buf(&sg[i], p, PAGE_SIZE);
+ goto more;
+ }
+ }
}
vi->num++;
} while (err > 0);
@@ -411,14 +461,13 @@ static void refill_work(struct work_struct *work)
static int virtnet_poll(struct napi_struct *napi, int budget)
{
struct virtnet_info *vi = container_of(napi, struct virtnet_info, napi);
- struct sk_buff *skb = NULL;
+ void *buf = NULL;
unsigned int len, received = 0;
again:
while (received < budget &&
- (skb = vi->rvq->vq_ops->get_buf(vi->rvq, &len)) != NULL) {
- __skb_unlink(skb, &vi->recv);
- receive_skb(vi->dev, skb, len);
+ (buf = vi->rvq->vq_ops->get_buf(vi->rvq, &len)) != NULL) {
+ receive_skb(vi->dev, buf, len);
vi->num--;
received++;
}
@@ -959,6 +1008,7 @@ static void __devexit virtnet_remove(struct virtio_device *vdev)
{
struct virtnet_info *vi = vdev->priv;
struct sk_buff *skb;
+ int freed;
/* Stop all the virtqueues. */
vdev->config->reset(vdev);
@@ -970,11 +1020,17 @@ static void __devexit virtnet_remove(struct virtio_device *vdev)
}
__skb_queue_purge(&vi->send);
- BUG_ON(vi->num != 0);
-
unregister_netdev(vi->dev);
cancel_delayed_work_sync(&vi->refill);
+ if (vi->mergeable_rx_bufs || vi->big_packets) {
+ freed = vi->rvq->vq_ops->destroy_buf(vi->rvq,
+ virtio_free_pages);
+ vi->num -= freed;
+ }
+
+ BUG_ON(vi->num != 0);
+
vdev->config->del_vqs(vi->vdev);
while (vi->pages)
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index fbd2ecd..aec7fe7 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -334,6 +334,29 @@ static bool vring_enable_cb(struct virtqueue *_vq)
return true;
}
+static int vring_destroy_buf(struct virtqueue *_vq, void (*callback)(void *))
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+ void *ret;
+ unsigned int i;
+ int freed = 0;
+
+ START_USE(vq);
+
+ for (i = 0; i < vq->vring.num; i++) {
+ if (vq->data[i]) {
+ /* detach_buf clears data, so grab it now. */
+ ret = vq->data[i];
+ detach_buf(vq, i);
+ callback(ret);
+ freed++;
+ }
+ }
+
+ END_USE(vq);
+ return freed;
+}
+
irqreturn_t vring_interrupt(int irq, void *_vq)
{
struct vring_virtqueue *vq = to_vvq(_vq);
@@ -360,6 +383,7 @@ static struct virtqueue_ops vring_vq_ops = {
.kick = vring_kick,
.disable_cb = vring_disable_cb,
.enable_cb = vring_enable_cb,
+ .destroy_buf = vring_destroy_buf,
};
struct virtqueue *vring_new_virtqueue(unsigned int num,
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index 057a2e0..7b1e86c 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -71,6 +71,7 @@ struct virtqueue_ops {
void (*disable_cb)(struct virtqueue *vq);
bool (*enable_cb)(struct virtqueue *vq);
+ int (*destroy_buf)(struct virtqueue *vq, void (*callback)(void *));
};
/**
^ permalink raw reply related
* Re: [net-next-2.6 PATCH v2] allow access to sysfs_groups member
From: Eric W. Biederman @ 2009-11-20 16:32 UTC (permalink / raw)
To: Kurt Van Dijck; +Cc: David Miller, netdev
In-Reply-To: <20091120102113.GC282@e-circ.dyndns.org>
Kurt Van Dijck <kurt.van.dijck@eia.be> writes:
> On Thu, Nov 19, 2009 at 05:11:45PM -0800, Eric W. Biederman wrote:
>> Kurt Van Dijck <kurt.van.dijck@eia.be> writes:
>>
>> > On Wed, Nov 18, 2009 at 01:08:59PM -0800, David Miller wrote:
>> >>
>> >> Unfortunately, the code in this function is much different
>> >> in net-next-2.6 which is where I want to add this, and your
>> >> patch doesn't apply.
>> >>
>> >> Please rebase your patch on that tree, thank you.
>> > As I mentioned, I'm not experienced with using git yet.
>> > Is there a fine manual I can inspect? I get lost in the man-pages.
>>
>> git clone git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next-2.6.git
>>
>> Should get you started.
> $ git pull git://git.kernel.... master
> did the job today. It took me a while
An incremental pull typically much faster than the initial one.
>> In this case please look at how I modified the bonding driver to do
>> what you are trying to do. That is the conflict in net-next and
>> I think it was actually less code.
> It _is_ less code (does a little less too :-) ).
> It solves the problem anyway. I'll stop with my patch.
You were just worried about CAN devices and not all network devices
correct?
Eric
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox