* Re: 2.6.21 -> 2.6.22 & 2.6.23-rc8 performance regression
From: Eric Dumazet @ 2007-10-01 20:10 UTC (permalink / raw)
To: Denys, David S. Miller; +Cc: linux-kernel, Linux Netdev List
In-Reply-To: <47008CB0.7010808@cosmosbay.com>
[-- Attachment #1: Type: text/plain, Size: 673 bytes --]
So maybe the following patch is necessary...
I believe IPV6 & DCCP are immune to this problem.
Thanks again Denys for spotting this.
Eric
[PATCH] TCP : secure_tcp_sequence_number() should not use a too fast clock
TCP V4 sequence numbers are 32bits, and RFC 793 assumed a 250 KHz clock.
In order to follow network speed increase, we can use a faster clock, but
we should limit this clock so that the delay between two rollovers is
greater than MSL (TCP Maximum Segment Lifetime : 2 minutes)
Choosing a 64 nsec clock should be OK, since the rollovers occur every
274 seconds.
Problem spotted by Denys Fedoryshchenko
Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>
[-- Attachment #2: seq.patch --]
[-- Type: text/plain, Size: 990 bytes --]
--- linux-2.6.22/drivers/char/random.c 2007-10-01 10:18:42.000000000 +0200
+++ linux-2.6.22-ed/drivers/char/random.c 2007-10-01 21:47:58.000000000 +0200
@@ -1550,11 +1550,13 @@ __u32 secure_tcp_sequence_number(__be32
* As close as possible to RFC 793, which
* suggests using a 250 kHz clock.
* Further reading shows this assumes 2 Mb/s networks.
- * For 10 Gb/s Ethernet, a 1 GHz clock is appropriate.
- * That's funny, Linux has one built in! Use it!
- * (Networks are faster now - should this be increased?)
+ * For 10 Mb/s Ethernet, a 1 MHz clock is appropriate.
+ * For 10 Gb/s Ethernet, a 1 GHz clock should be ok, but
+ * we also need to limit the resolution so that the u32 seq
+ * overlaps less than one time per MSL (2 minutes).
+ * Choosing a clock of 64 ns period is OK. (period of 274 s)
*/
- seq += ktime_get_real().tv64;
+ seq += ktime_get_real().tv64 >> 6;
#if 0
printk("init_seq(%lx, %lx, %d, %d) = %d\n",
saddr, daddr, sport, dport, seq);
^ permalink raw reply
* sk98lin, jumbo frames, and memory fragmentation
From: Chris Friesen @ 2007-10-01 20:35 UTC (permalink / raw)
To: linux, netdev
Hi all,
We're considering some hardware that uses the sk98lin network hardware,
and we'll be using jumbo frames. Looking at the driver, when using a
9KB MTU it seems like it would end up trying to atomically allocate a
16KB buffer.
Has anyone heard of this been a problem? It would seem like trying to
atomically allocate four physically contiguous pages could become tricky
after the system has been running for a while.
The reason I ask is that we ran into this with the e1000. Before they
added the new jumbo frame code it was trying to atomically allocate 32KB
buffers and we would start getting allocation failures after a month or
so of uptime.
Any information anyone can provide would be appreciated.
Thanks,
Chris
^ permalink raw reply
* Re: sk98lin, jumbo frames, and memory fragmentation
From: John Heffner @ 2007-10-01 20:44 UTC (permalink / raw)
To: Chris Friesen; +Cc: linux, netdev
In-Reply-To: <47015A24.7000701@nortel.com>
Yes it has this problem. I've observed it in practice on a busy firewall.
-John
Chris Friesen wrote:
>
> Hi all,
>
> We're considering some hardware that uses the sk98lin network hardware,
> and we'll be using jumbo frames. Looking at the driver, when using a
> 9KB MTU it seems like it would end up trying to atomically allocate a
> 16KB buffer.
>
> Has anyone heard of this been a problem? It would seem like trying to
> atomically allocate four physically contiguous pages could become tricky
> after the system has been running for a while.
>
> The reason I ask is that we ran into this with the e1000. Before they
> added the new jumbo frame code it was trying to atomically allocate 32KB
> buffers and we would start getting allocation failures after a month or
> so of uptime.
>
> Any information anyone can provide would be appreciated.
>
>
> Thanks,
>
> Chris
> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] net-2.6.24: old ax25 driver fix
From: David Miller @ 2007-10-01 20:45 UTC (permalink / raw)
To: shemminger; +Cc: akpm, netdev
In-Reply-To: <20071001112417.10bf522d@oldman>
From: Stephen Hemminger <shemminger@linux-foundation.org>
Date: Mon, 1 Oct 2007 11:24:17 -0700
> Recent change in hard header broke build of these old drivers.
>
> Signed-off-by: Stephen Hemminger <shemminger@linux-foundation.org>
Applied, thanks Stephen.
^ permalink raw reply
* Re: [PATCH 01/10] Preparatory refactoring part 1.
From: Corey Hickey @ 2007-10-01 20:46 UTC (permalink / raw)
To: Patrick McHardy, Linux Netdev List
In-Reply-To: <4700F54A.6070408@trash.net>
Patrick McHardy wrote:
> Corey Hickey wrote:
>> Make a new function sfq_q_enqueue() that operates directly on the
>> queue data. This will be useful for implementing sfq_change() in
>> a later patch. A pleasant side-effect is reducing most of the
>> duplicate code in sfq_enqueue() and sfq_requeue().
>>
>> Similarly, make a new function sfq_q_dequeue().
>>
>> Signed-off-by: Corey Hickey <bugfood-ml@fatooh.org>
>> ---
>> net/sched/sch_sfq.c | 72 +++++++++++++++++++++++++++------------------------
>> 1 files changed, 38 insertions(+), 34 deletions(-)
>>
>> diff --git a/net/sched/sch_sfq.c b/net/sched/sch_sfq.c
>> index 3a23e30..57485ef 100644
>> --- a/net/sched/sch_sfq.c
>> +++ b/net/sched/sch_sfq.c
>>
>
>
> The sfq_q_enqueue part looks fine.
>
>>
>> - sch->qstats.drops++;
>
>
> A line in the changelog explaining that this was increased twice
> would have been nice.
Certainly; I think I didn't realize, when you originally pointed out the
duplicate incrementing, that it was a bug in the original version and
not in my patch. Otherwise, I would have sent it as a separate patch.
If a note in this patch will suffice, though, I'll definitely do so.
>> sfq_drop(sch);
>> return NET_XMIT_CN;
>> }
>>
>> -
>> -
>> -
>> -static struct sk_buff *
>> -sfq_dequeue(struct Qdisc* sch)
>> +static struct
>> +sk_buff *sfq_q_dequeue(struct sfq_sched_data *q)
>
>
> What is this function needed for?
It gets used in sfq_change for moving packets from the old queue into
the new one. In this case, we don't want to modify sch->q.qlen or
sch->qstats.backlog, since those don't actually change.
while ((skb = sfq_q_dequeue(q)) != NULL)
sfq_q_enqueue(skb, &tmp, SFQ_TAIL);
I'll improve the description of this patch to make that more clear.
-Corey
^ permalink raw reply
* Re: [PATCH 02/10] Preparatory refactoring part 2.
From: Corey Hickey @ 2007-10-01 20:46 UTC (permalink / raw)
To: Patrick McHardy, Linux Netdev List
In-Reply-To: <4700F715.9000200@trash.net>
Patrick McHardy wrote:
> Corey Hickey wrote:
>> The sfq_destroy() --> sfq_q_destroy() change looks pointless here,
>> but it's cleaner to split now and add code to sfq_q_destroy() in a
>> later patch.
>>
>> +static void sfq_destroy(struct Qdisc *sch)
>> +{
>> + struct sfq_sched_data *q = qdisc_priv(sch);
>> + sfq_q_destroy(q);
>> +}
>
>
> It does look pointless, after applying all patches sfq_destroy still
> remains a simply wrapper around sfq_q_destroy.
It does remain a wrapper, but both functions are used. It doesn't have
to be this way, but I wanted to avoid duplicating code and I didn't see
a better layout.
sfq_q_destroy is used in sfq_q_init if a kcalloc fails. sfq_q_init knows
nothing about "struct Qdisc *sch", so it can't call sfq_destroy.
sfq_destroy is still marked as the destroy function in sfq_qdisc_ops.
-Corey
^ permalink raw reply
* Re: [PATCH 03/10] Move two functions.
From: Corey Hickey @ 2007-10-01 20:46 UTC (permalink / raw)
To: Patrick McHardy, Linux Netdev List
In-Reply-To: <4700F5AC.9020609@trash.net>
Patrick McHardy wrote:
> Corey Hickey wrote:
>> Move sfq_q_destroy() to above sfq_q_init() so that it can be used
>> by an error case in a later patch.
>>
>> Move sfq_destroy() as well, for clarity.
>
>
> This patch looks pointless, just put them where you need them
> in the patch introducing them.
As you wish. I thought having a separate patch would ease reviewing.
-Corey
^ permalink raw reply
* Re: [PATCH 05/10] Add divisor.
From: Corey Hickey @ 2007-10-01 20:47 UTC (permalink / raw)
To: Patrick McHardy, Linux Netdev List
In-Reply-To: <4700F83F.9020300@trash.net>
Patrick McHardy wrote:
> Corey Hickey wrote:
>> Make hash divisor user-configurable.
>>
>
>> @@ -120,7 +121,7 @@ static __inline__ unsigned sfq_fold_hash(struct sfq_sched_data *q, u32 h, u32 h1
>> /* Have we any rotation primitives? If not, WHY? */
>> h ^= (h1<<pert) ^ (h1>>(0x1F - pert));
>> h ^= h>>10;
>> - return h & 0x3FF;
>> + return h & (q->hash_divisor-1);
>
>
> This assumes that hash_divisor is a power of two, but this is
> not enforced anywhere.
Ok. I'll move that part from userspace to the kernel. That should be
better anyway.
-Corey
^ permalink raw reply
* Re: [PATCH 06/10] Make qdisc changeable.
From: Corey Hickey @ 2007-10-01 20:47 UTC (permalink / raw)
To: Patrick McHardy, Linux Netdev List /s
In-Reply-To: <4700F980.3000701@trash.net>
Patrick McHardy wrote:
> Corey Hickey wrote:
>> Re-implement sfq_change() and enable Qdisc_opts.change so "tc qdisc
>> change" will work.
>>
>
>> +static int sfq_change(struct Qdisc *sch, struct rtattr *opt)
>> +{
>> + ...
>> +
>> + /* finish up */
>> + if (q->perturb_period) {
>> + q->perturb_timer.expires = jiffies + q->perturb_period;
>> + add_timer(&q->perturb_timer);
>> + } else {
>> + q->perturbation = 0;
>
>
> Seems counter-productive to explicitly set it to zero since
> it was still used during tranfering the packets with the
> old value. So I'd suggest to remove this or alternatively
> set it to the final value *before* transfering the packets.
I suppose so; you're right. I'll adapt that part to fit before
transferring the packets.
-Corey
^ permalink raw reply
* Re: [PATCH 09/10] Change perturb_period to unsigned.
From: Corey Hickey @ 2007-10-01 20:47 UTC (permalink / raw)
To: Patrick McHardy; +Cc: netdev
In-Reply-To: <4700FA02.8090800@trash.net>
Patrick McHardy wrote:
> Corey Hickey wrote:
>> perturb_period is currently a signed integer, but I can't see any good
>> reason why this is so--a negative perturbation period will add a timer
>> that expires in the past, causing constant perturbation, which makes
>> hashing useless.
>>
>> if (q->perturb_period) {
>> q->perturb_timer.expires = jiffies + q->perturb_period;
>> add_timer(&q->perturb_timer);
>> }
>>
>> Strictly speaking, this will break binary compatibility with older
>> versions of tc, but that ought not to be a problem because (a) there's
>> no valid use for a negative perturb_period, and (b) negative values
>> will be seen as high values (> INT_MAX), which don't work anyway.
>>
>> If perturb_period is too large, (perturb_period * HZ) will overflow the
>> size of an unsigned int and wrap around. So, check for thet and reject
>> values that are too high.
>
>
> Sounds reasonable.
>
>> --- a/net/sched/sch_sfq.c
>> +++ b/net/sched/sch_sfq.c
>> @@ -74,6 +74,9 @@
>> typedef unsigned int sfq_index;
>> #define SFQ_MAX_DEPTH (UINT_MAX / 2 - 1)
>>
>> +/* We don't want perturb_period * HZ to overflow an unsigned int. */
>> +#define SFQ_MAX_PERTURB (UINT_MAX / HZ)
>
>
> jiffies are unsigned long.
Hmm. You're right. It looks like my previous patch obviated the need for
this part. I'll remove it.
-Corey
^ permalink raw reply
* Re: [PATCH 10/10] Use nested compat attributes to pass parameters.
From: Corey Hickey @ 2007-10-01 20:47 UTC (permalink / raw)
To: Patrick McHardy, Linux Netdev List
In-Reply-To: <4700FC1E.9040309@trash.net>
Patrick McHardy wrote:
> Corey Hickey wrote:
>> +
>> +#define GET_PARAM(dst, nest, compat) do { \
>> + struct rtattr *rta = tb[(nest) - 1]; \
>> + if (rta) \
>> + (dst) = RTA_GET_U32(rta); \
>> + else if ((compat)) \
>> + (dst) = (compat); \
>> +} while (0)
>
>
> An inline function and a comment why this is done would increase
> readability.
Well, I had a reason for making a macro, but it probably wasn't a good
reason. Looking now, I don't see why not to make a function. I'll see
what I can do.
>> + nest = RTA_NEST_COMPAT(skb, TCA_OPTIONS, sizeof(opt), &opt);
>> +
>> + RTA_PUT_U32(skb, TCA_SFQ_QUANTUM, q->quantum);
>> + RTA_PUT_U32(skb, TCA_SFQ_PERTURB, q->perturb_period);
>> + RTA_PUT_U32(skb, TCA_SFQ_LIMIT, q->limit);
>> + RTA_PUT_U32(skb, TCA_SFQ_DIVISOR, q->hash_divisor);
>> + RTA_PUT_U32(skb, TCA_SFQ_FLOWS, q->depth);
>> RTA_PUT(skb, TCA_OPTIONS, sizeof(opt), &opt);
>
>
> This is wrong, RTA_NEST_COMPAT already dumps the structure.
You mean that last line (RTA_PUT) is superfluous, right? I can't see a
reason for it to be there, so I must have just forgotten to delete it
from the original code.
If I'm wrong, I might need a little hand-holding here. My understanding
of all the RTA stuff is a bit shaky.
Much thanks for the review. I'll make a new set of patches soon.
-Corey
^ permalink raw reply
* Re: sk98lin, jumbo frames, and memory fragmentation
From: Jeff Garzik @ 2007-10-01 20:47 UTC (permalink / raw)
To: Chris Friesen; +Cc: linux, netdev
In-Reply-To: <47015A24.7000701@nortel.com>
Chris Friesen wrote:
> We're considering some hardware that uses the sk98lin network hardware,
> and we'll be using jumbo frames. Looking at the driver, when using a
> 9KB MTU it seems like it would end up trying to atomically allocate a
> 16KB buffer.
The sk98lin driver is going away, please don't use it.
It's unmaintained and full of known bugs.
Jeff
^ permalink raw reply
* Re: 2.6.21 -> 2.6.22 & 2.6.23-rc8 performance regression
From: David Miller @ 2007-10-01 20:57 UTC (permalink / raw)
To: dada1; +Cc: nuclearcat, linux-kernel, netdev
In-Reply-To: <4701541B.70108@cosmosbay.com>
From: Eric Dumazet <dada1@cosmosbay.com>
Date: Mon, 01 Oct 2007 22:10:03 +0200
> So maybe the following patch is necessary...
>
> I believe IPV6 & DCCP are immune to this problem.
>
> Thanks again Denys for spotting this.
>
> Eric
>
> [PATCH] TCP : secure_tcp_sequence_number() should not use a too fast clock
>
> TCP V4 sequence numbers are 32bits, and RFC 793 assumed a 250 KHz clock.
> In order to follow network speed increase, we can use a faster clock, but
> we should limit this clock so that the delay between two rollovers is
> greater than MSL (TCP Maximum Segment Lifetime : 2 minutes)
>
> Choosing a 64 nsec clock should be OK, since the rollovers occur every
> 274 seconds.
>
> Problem spotted by Denys Fedoryshchenko
>
> Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>
Thanks a lot Eric for bringing closure to this.
I'll apply this and add a reference in the commit message to the
changeset that introduced this problem, since it might help
others who look at this.
^ permalink raw reply
* Re: sk98lin, jumbo frames, and memory fragmentation
From: Stephen Hemminger @ 2007-10-01 21:06 UTC (permalink / raw)
To: Chris Friesen; +Cc: linux, netdev
In-Reply-To: <47015A24.7000701@nortel.com>
On Mon, 01 Oct 2007 14:35:48 -0600
"Chris Friesen" <cfriesen@nortel.com> wrote:
>
> Hi all,
>
> We're considering some hardware that uses the sk98lin network hardware,
> and we'll be using jumbo frames. Looking at the driver, when using a
> 9KB MTU it seems like it would end up trying to atomically allocate a
> 16KB buffer.
>
> Has anyone heard of this been a problem? It would seem like trying to
> atomically allocate four physically contiguous pages could become tricky
> after the system has been running for a while.
>
> The reason I ask is that we ran into this with the e1000. Before they
> added the new jumbo frame code it was trying to atomically allocate 32KB
> buffers and we would start getting allocation failures after a month or
> so of uptime.
>
> Any information anyone can provide would be appreciated.
Adding fragmentation support to skge driver is on my list of
possible extensions. sky2 driver already supports it (yet one
more feature that the vendor sk98lin driver doesn't do).
--
Stephen Hemminger <shemminger@linux-foundation.org>
^ permalink raw reply
* Re: sk98lin, jumbo frames, and memory fragmentation
From: Chris Friesen @ 2007-10-01 21:15 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: linux, netdev
In-Reply-To: <20071001140627.1e2e665f@freepuppy.rosehill>
Stephen Hemminger wrote:
> Adding fragmentation support to skge driver is on my list of
> possible extensions. sky2 driver already supports it (yet one
> more feature that the vendor sk98lin driver doesn't do).
Thanks for speaking up. As I mentioned in my email to Jeff it looks
like the sky2 driver is what I need (Marvel Yukon 88E8062). However,
I'm on 2.6.14 and it doesn't exist there...do you anticipate any issues
if I were to backport it?
Thanks,
Chris
^ permalink raw reply
* Re: sk98lin, jumbo frames, and memory fragmentation
From: Chris Friesen @ 2007-10-01 21:08 UTC (permalink / raw)
To: Jeff Garzik; +Cc: linux, netdev
In-Reply-To: <47015CE0.3080706@garzik.org>
Jeff Garzik wrote:
> The sk98lin driver is going away, please don't use it.
>
> It's unmaintained and full of known bugs.
Okay...so it looks like the proper driver for the Marvell Yukon 88E8062
is the sky2 driver, and this one does avoid order>0 allocations. Am I
on track?
Chris
^ permalink raw reply
* Re: sk98lin, jumbo frames, and memory fragmentation
From: Stephen Hemminger @ 2007-10-01 21:57 UTC (permalink / raw)
To: Chris Friesen; +Cc: linux, netdev
In-Reply-To: <4701638F.2020903@nortel.com>
On Mon, 01 Oct 2007 15:15:59 -0600
"Chris Friesen" <cfriesen@nortel.com> wrote:
> Stephen Hemminger wrote:
>
> > Adding fragmentation support to skge driver is on my list of
> > possible extensions. sky2 driver already supports it (yet one
> > more feature that the vendor sk98lin driver doesn't do).
>
> Thanks for speaking up. As I mentioned in my email to Jeff it looks
> like the sky2 driver is what I need (Marvel Yukon 88E8062). However,
> I'm on 2.6.14 and it doesn't exist there...do you anticipate any issues
> if I were to backport it?
Nothing but usual annoying kernel API changes..
--
Stephen Hemminger <shemminger@linux-foundation.org>
^ permalink raw reply
* Re: [PATCH net-2.6.24 0/4]: TCP fixes
From: David Miller @ 2007-10-01 22:29 UTC (permalink / raw)
To: ilpo.jarvinen; +Cc: netdev
In-Reply-To: <11912417841620-git-send-email-ilpo.jarvinen@helsinki.fi>
From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi>
Date: Mon, 1 Oct 2007 15:29:40 +0300
> This fixes the newreno fackets_out case, which turned out to be
> not related to the Cedric's case being under investigation. Two
> trivial comment patches, and frto with high-speed seqno
> wrap-around protection. Compile tested. Please apply to
> net-2.6.24.
I've applied them all to net-2.6.24, thanks Ilpo!
^ permalink raw reply
* [PATCH] mv643xx_eth: Do not modify struct netdev tx_queue_len
From: Dale Farnsworth @ 2007-10-01 23:02 UTC (permalink / raw)
To: Jeff Garzik; +Cc: netdev
From: Dale Farnsworth <dale@farnsworth.org>
This driver erroneously zeros dev->tx_queue_len, since
mp->tx_ring_size has not yet been initialized. Actually,
the driver shouldn't modify tx_queue_len at all and should
leave the value set by alloc_etherdev(), currently 1000.
Signed-off-by: Dale Farnsworth <dale@farnsworth.org>
---
Jeff, this bug was just reported today, or I would have batched
it with the one I sent you last week. It's an obvious bugfix,
so I'm not going to hold it in my queue.
drivers/net/mv643xx_eth.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/net/mv643xx_eth.c b/drivers/net/mv643xx_eth.c
index 34288fe..3153356 100644
--- a/drivers/net/mv643xx_eth.c
+++ b/drivers/net/mv643xx_eth.c
@@ -1357,7 +1357,6 @@ static int mv643xx_eth_probe(struct platform_device *pdev)
#endif
dev->watchdog_timeo = 2 * HZ;
- dev->tx_queue_len = mp->tx_ring_size;
dev->base_addr = 0;
dev->change_mtu = mv643xx_eth_change_mtu;
dev->do_ioctl = mv643xx_eth_do_ioctl;
^ permalink raw reply related
* Re: [PATCH][TG3]Some cleanups
From: Michael Chan @ 2007-10-02 0:21 UTC (permalink / raw)
To: hadi; +Cc: Matt Carlson, netdev
In-Reply-To: <1191175890.6165.17.camel@localhost>
On Sun, 2007-09-30 at 14:11 -0400, jamal wrote:
> Here are some non-batching related changes that i have in my batching
> tree. Like the e1000e, they make the xmit code more readable.
> I wouldnt mind if you take them over.
>
Jamal, in tg3_enqueue_buggy(), we may have to call tg3_tso_bug() which
will recursively call tg3_start_xmit_dma_bug() after segmenting the TSO
packet into normal packets. We need to restore the VLAN tag so that the
GSO code will create the chain of segmented SKBs with the proper VLAN
tag.
^ permalink raw reply
* How do queue-less virtual devices wake higher level senders?
From: Ben Greear @ 2007-10-01 23:49 UTC (permalink / raw)
To: NetDev
Hello!
I am having some trouble figuring out how virtual interfaces
(such as mac-vlans) can wake up writers (such as udp sockets).
For 'real' hardware, it seems that the netif_stop_queue and
netif_wake_queue methods handle stopping and waking the
higher level senders, but for virtual devices with no
queues, how does this work?
In my case, I'm using a virtual Station interface that sits on
top of a wifi radio interface (hacked up madwifi). I notice
that UDP connections set up for high speed, unidirectional
sends are stalling after a few minutes. netstat -an shows
a write-buffer that is quite full, but nothing is transmitted.
If I ping or start any other type of traffic on these interfaces,
the udp recovers. It seems like the udp send logic is just
getting stuck and needs a kick.
I do not see any problems with TCP connections, and if I keep
a slow-speed tcp connection running, the UDP will not hang.
It's likely the bug is in my driver and/or code, so this is
not a bug report..just a question to hopefully help me debug
it further :)
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply
* Re: How do queue-less virtual devices wake higher level senders?
From: David Miller @ 2007-10-02 0:04 UTC (permalink / raw)
To: greearb; +Cc: netdev
In-Reply-To: <47018772.4060803@candelatech.com>
From: Ben Greear <greearb@candelatech.com>
Date: Mon, 01 Oct 2007 16:49:06 -0700
> For 'real' hardware, it seems that the netif_stop_queue and
> netif_wake_queue methods handle stopping and waking the
> higher level senders, but for virtual devices with no
> queues, how does this work?
They don't queue, there is nothing to stop or wakeup.
^ permalink raw reply
* Re: How do queue-less virtual devices wake higher level senders?
From: Ben Greear @ 2007-10-02 0:24 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <20071001.170410.40806513.davem@davemloft.net>
David Miller wrote:
> From: Ben Greear <greearb@candelatech.com>
> Date: Mon, 01 Oct 2007 16:49:06 -0700
>
>> For 'real' hardware, it seems that the netif_stop_queue and
>> netif_wake_queue methods handle stopping and waking the
>> higher level senders, but for virtual devices with no
>> queues, how does this work?
>
> They don't queue, there is nothing to stop or wakeup.
Ok, so if I have a UDP socket bound to an interface that has
no queue, and yet I see the send portion of the queue being
full in netstat, what does this mean?
Maybe the device I think has no queue somehow does?
I added some debugging to print out dev->state in sysfs, and
the state of the virtual is always 0x6, which appears right
to me. It's underlying device goes back and forth between 0x7 and 0x6,
which also seems right to me.
When the thing is in the hung state, phys and virtual interface have 0x6
state, and yet the udp tx queue remains full. The physical NIC also
prints out some errors about being low on buffers right before the
hang, but it seems to recover since just doing a ping or starting
a second udp connection brings everything back to life.
Other than IFF_UP and dev->state, are there other things that
can make the tx logic stop sending to a device?
Thanks,
Ben
> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply
* Re: tcp bw in 2.6
From: Larry McVoy @ 2007-10-02 0:59 UTC (permalink / raw)
To: Linus Torvalds; +Cc: davem, wscott, netdev
In-Reply-To: <alpine.LFD.0.999.0709291050200.3579@woody.linux-foundation.org>
On Sat, Sep 29, 2007 at 11:02:32AM -0700, Linus Torvalds wrote:
> On Sat, 29 Sep 2007, Larry McVoy wrote:
> > I haven't kept up on switch technology but in the past they were much
> > better than you are thinking. The Kalpana switch that I had modified
> > to support vlans (invented by yours truly), did not store and forward,
> > it was cut through and could handle any load that was theoretically
> > possible within about 1%.
>
> Hey, you may well be right. Maybe my assumptions about cutting corners are
> just cynical and pessimistic.
So I got a netgear switch and it works fine. But my tests are busted.
Catching netdev up, I'm trying to optimize traffic to a server that has
a gbit interface; I moved to a 24 port netgear that is all 10/100/1000
and I have a pile of clients to act as load generators.
I can do this on each of the clients
dd if=/dev/zero bs=1024000 | rsh work "dd of=/dev/null"
and that cranks up to about 47K packets/second which is about 70MB/sec.
One of my clients also has gigabit so I played around with just that
one and it (itanium running hpux w/ broadcom gigabit) can push the load
as well. One weird thing is that it is dependent on the direction the
data is flowing. If the hp is sending then I get 46MB/sec, if linux is
sending then I get 18MB/sec. Weird. Linux is debian, running
Linux work 2.6.18-5-k7 #1 SMP Thu Aug 30 02:52:31 UTC 2007 i686
and dual e1000 cards:
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
I wrote a tiny little program to try and emulate this and I can't get
it to do as well. I've tracked it down, I think, to the read side.
The server sources, the client sinks, the server looks like:
11689 accept(3, {sa_family=AF_INET, sin_port=htons(49376), sin_addr=inet_addr("10.3.1.38")}, [16]) = 4
11689 setsockopt(4, SOL_SOCKET, SO_RCVBUF, [1048576], 4) = 0
11689 setsockopt(4, SOL_SOCKET, SO_SNDBUF, [1048576], 4) = 0
11689 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb7ddf708) = 11694
11689 close(4) = 0
11689 accept(3, <unfinished ...>
11694 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 1048576
11694 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 1048576
11694 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 1048576
11694 write(4, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 1048576
...
but the client looks like
connect(3, {sa_family=AF_INET, sin_port=htons(31235), sin_addr=inet_addr("10.3.9.1")}, 16) = 0
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 2896
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 1448
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 2896
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 2896
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 2896
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 2896
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 2896
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 2896
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 2896
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 1448
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 2896
which I suspect may be the problem.
I played around with SO_RCVBUF/SO_SNDBUF and that didn't help. So any ideas why
a simple dd piped through rsh is kicking my ass? It must be something simple
but my test program is tiny and does nothing weird that I can see.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com
^ permalink raw reply
* Re: tcp bw in 2.6
From: Linus Torvalds @ 2007-10-02 2:14 UTC (permalink / raw)
To: Larry McVoy; +Cc: davem, wscott, netdev
In-Reply-To: <20071002005917.GB5480@bitmover.com>
On Mon, 1 Oct 2007, Larry McVoy wrote:
>
> but the client looks like
>
> connect(3, {sa_family=AF_INET, sin_port=htons(31235), sin_addr=inet_addr("10.3.9.1")}, 16) = 0
> read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 2896
> read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 1448
> read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 2896
..
This is exactly what I'd expect if the machine is *not* under excessive
load.
The system calls are fast enough that the latency for the TCP stack is
roughly on the same scale as the time it takes to receive one new packet,
so since a socket read will always return when it has any data (not until
it has filled the whole buffer), you get exactly that "one or two packets"
pattern.
If you'd be really CPU-limited or under load from other programs, you'd
have more packets come in while you're in the read path, and you'd get
bigger reads.
But do a tcpdump both ways, and see (for example) if the TCP window is
much bigger going the other way.
Linus
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox