From: Eric Dumazet <dada1@cosmosbay.com>
To: Ben Greear <greearb@candelatech.com>
Cc: Andi Kleen <andi@firstfloor.org>, netdev@vger.kernel.org, bcrl@kvack.org
Subject: Re: TCP connection stops after high load.
Date: Thu, 12 Apr 2007 22:41:46 +0200 [thread overview]
Message-ID: <461E998A.8080409@cosmosbay.com> (raw)
In-Reply-To: <461E8496.2050003@candelatech.com>
Ben Greear a écrit :
> Eric Dumazet wrote:
>
>> What "tc -s -d qdisc"
>> "ifconfig -a"
>> "cat /proc/interrupts" "cat /proc/net/sockstat" and
>> "cat /proc/net/softnet_stat" are telling ?
>
>
> In this test, eth2 is talking to eth3, using something similar to this
> send-to-self patch:
> http://www.candelatech.com/oss/sts.patch
>
>
> [root@lf1001-240 ipv4]# ifconfig -a
> eth0 Link encap:Ethernet HWaddr 00:30:48:89:74:60
> inet addr:192.168.100.187 Bcast:192.168.100.255
> Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:1672220 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1560305 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:151896589 (144.8 MiB) TX bytes:1375280163 (1.2 GiB)
> Interrupt:17
>
> eth1 Link encap:Ethernet HWaddr 00:30:48:89:74:61
> UP BROADCAST MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
> Interrupt:18
>
> eth2 Link encap:Ethernet HWaddr 00:07:E9:1F:CE:02
> inet addr:20.20.20.20 Bcast:20.20.20.255 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:2175144684 errors:0 dropped:2 overruns:0 frame:0
> TX packets:2196560123 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:1321380186 (1.2 GiB) TX bytes:2274982574 (2.1 GiB)
> Base address:0xd000 Memory:d0000000-d0020000
>
> eth3 Link encap:Ethernet HWaddr 00:07:E9:1F:CE:03
> inet addr:20.20.20.30 Bcast:20.20.20.255 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:2196315966 errors:0 dropped:0 overruns:0 frame:0
> TX packets:2174900538 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:2257901986 (2.1 GiB) TX bytes:1304493504 (1.2 GiB)
> Base address:0xd100 Memory:d0020000-d0040000
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:1159378 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1159378 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:1133646590 (1.0 GiB) TX bytes:1133646590 (1.0 GiB)
>
> [root@lf1001-240 ipv4]# tc -s -d qdisc
> qdisc pfifo_fast 0: dev eth0 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1
> 1 1 1
> Sent 1367521025 bytes 1324808 pkt (dropped 0, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 0b 0p requeues 0
> qdisc pfifo_fast 0: dev eth1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1
> 1 1 1
> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 0b 0p requeues 0
> qdisc pfifo_fast 0: dev eth2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1
> 1 1 1
> Sent 1815657070136 bytes 1536674488 pkt (dropped 0, overlimits 0
> requeues 1448094)
> rate 0bit 0pps backlog 0b 0p requeues 1448094
> qdisc pfifo_fast 0: dev eth3 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1
> 1 1 1
> Sent 1752033393324 bytes 1536906566 pkt (dropped 0, overlimits 0
> requeues 1063672)
> rate 0bit 0pps backlog 0b 0p requeues 1063672
>
> [root@lf1001-240 ipv4]# cat /proc/interrupts
> CPU0 CPU1
> 0: 46020594 44501954 IO-APIC-edge timer
> 1: 9 0 IO-APIC-edge i8042
> 7: 0 0 IO-APIC-edge parport0
> 8: 1 0 IO-APIC-edge rtc
> 9: 0 0 IO-APIC-level acpi
> 12: 96 0 IO-APIC-edge i8042
> 14: 394023 407282 IO-APIC-edge ide0
> 16: 0 0 IO-APIC-level uhci_hcd:usb4
> 17: 1134346 1034006 IO-APIC-level uhci_hcd:usb3, eth0
> 18: 81605 83739 IO-APIC-level libata, uhci_hcd:usb2, eth1
> 19: 0 0 IO-APIC-level uhci_hcd:usb1, ehci_hcd:usb5
> 20: 53056128 46598235 IO-APIC-level eth2
> 21: 47534577 52189674 IO-APIC-level eth3
> NMI: 0 0
> LOC: 90485383 90485382
> ERR: 0
> MIS: 0
>
> [root@lf1001-240 ipv4]# cat /proc/net/sockstat
> sockets: used 334
> TCP: inuse 27 orphan 0 tw 0 alloc 27 mem 360
> UDP: inuse 12
> RAW: inuse 0
> FRAG: inuse 0 memory 0
>
> [root@lf1001-240 ipv4]# cat /proc/net/softnet_stat
> d58236f1 00000000 023badc3 00000000 00000000 00000000 00000000 00000000
> 0004ef01
> 3a4354a1 00000000 01b57b4b 00000000 00000000 00000000 00000000 00000000
> 0005445f
>
Hum, could you try to bind nic irqs on separate cpus ?
eth2 -> CPU0 and eth3 -> CPU1
# echo 1 >/proc/irq/20/smp_affinity
# echo 2 >/proc/irq/21/smp_affinity
next prev parent reply other threads:[~2007-04-12 20:41 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-04-11 18:50 TCP connection stops after high load Ben Greear
2007-04-11 20:26 ` Ben Greear
2007-04-11 20:48 ` David Miller
2007-04-11 21:06 ` Ben Greear
2007-04-11 21:11 ` David Miller
2007-04-11 21:31 ` Ben Greear
2007-04-11 21:39 ` David Miller
2007-04-12 2:44 ` SANGTAE HA
2007-04-12 1:06 ` Benjamin LaHaise
2007-04-12 14:48 ` Andi Kleen
2007-04-12 17:59 ` Ben Greear
2007-04-12 18:19 ` Eric Dumazet
2007-04-12 19:12 ` Ben Greear
2007-04-12 20:41 ` Eric Dumazet [this message]
2007-04-12 21:36 ` Ben Greear
2007-04-13 7:09 ` Evgeniy Polyakov
2007-04-13 16:42 ` Ben Greear
2007-04-13 16:10 ` Daniel Schaffrath
2007-04-13 16:41 ` Eric Dumazet
2007-04-14 4:21 ` Herbert Xu
2007-04-14 4:25 ` David Miller
2007-04-14 5:31 ` Eric Dumazet
2007-04-14 5:37 ` David Miller
2007-04-11 20:41 ` David Miller
2007-04-12 6:12 ` Ilpo Järvinen
-- strict thread matches above, loose matches on Subject: below --
2007-04-12 21:11 Robert Iakobashvili
2007-04-12 21:15 ` David Miller
2007-04-15 12:14 ` Robert Iakobashvili
2007-04-15 15:31 ` John Heffner
2007-04-15 15:49 ` Robert Iakobashvili
2007-04-16 18:07 ` John Heffner
2007-04-16 18:51 ` Robert Iakobashvili
2007-04-16 19:11 ` John Heffner
2007-04-16 19:17 ` David Miller
2007-04-16 19:15 ` David Miller
2007-04-17 7:58 ` Robert Iakobashvili
2007-04-17 19:39 ` David Miller
2007-04-17 19:47 ` John Heffner
2007-04-17 19:51 ` David Miller
2007-04-17 19:58 ` Robert Iakobashvili
2007-04-15 13:52 ` Robert Iakobashvili
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=461E998A.8080409@cosmosbay.com \
--to=dada1@cosmosbay.com \
--cc=andi@firstfloor.org \
--cc=bcrl@kvack.org \
--cc=greearb@candelatech.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.