From: Eric Dumazet <dada1@cosmosbay.com>
To: Ben Greear <greearb@candelatech.com>
Cc: Andi Kleen <andi@firstfloor.org>, netdev@vger.kernel.org, bcrl@kvack.org
Subject: Re: TCP connection stops after high load.
Date: Thu, 12 Apr 2007 22:41:46 +0200 [thread overview]
Message-ID: <461E998A.8080409@cosmosbay.com> (raw)
In-Reply-To: <461E8496.2050003@candelatech.com>
Ben Greear a écrit :
> Eric Dumazet wrote:
>
>> What "tc -s -d qdisc"
>> "ifconfig -a"
>> "cat /proc/interrupts" "cat /proc/net/sockstat" and
>> "cat /proc/net/softnet_stat" are telling ?
>
>
> In this test, eth2 is talking to eth3, using something similar to this
> send-to-self patch:
> http://www.candelatech.com/oss/sts.patch
>
>
> [root@lf1001-240 ipv4]# ifconfig -a
> eth0 Link encap:Ethernet HWaddr 00:30:48:89:74:60
> inet addr:192.168.100.187 Bcast:192.168.100.255
> Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:1672220 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1560305 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:151896589 (144.8 MiB) TX bytes:1375280163 (1.2 GiB)
> Interrupt:17
>
> eth1 Link encap:Ethernet HWaddr 00:30:48:89:74:61
> UP BROADCAST MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
> Interrupt:18
>
> eth2 Link encap:Ethernet HWaddr 00:07:E9:1F:CE:02
> inet addr:20.20.20.20 Bcast:20.20.20.255 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:2175144684 errors:0 dropped:2 overruns:0 frame:0
> TX packets:2196560123 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:1321380186 (1.2 GiB) TX bytes:2274982574 (2.1 GiB)
> Base address:0xd000 Memory:d0000000-d0020000
>
> eth3 Link encap:Ethernet HWaddr 00:07:E9:1F:CE:03
> inet addr:20.20.20.30 Bcast:20.20.20.255 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:2196315966 errors:0 dropped:0 overruns:0 frame:0
> TX packets:2174900538 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:2257901986 (2.1 GiB) TX bytes:1304493504 (1.2 GiB)
> Base address:0xd100 Memory:d0020000-d0040000
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:1159378 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1159378 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:1133646590 (1.0 GiB) TX bytes:1133646590 (1.0 GiB)
>
> [root@lf1001-240 ipv4]# tc -s -d qdisc
> qdisc pfifo_fast 0: dev eth0 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1
> 1 1 1
> Sent 1367521025 bytes 1324808 pkt (dropped 0, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 0b 0p requeues 0
> qdisc pfifo_fast 0: dev eth1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1
> 1 1 1
> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
> rate 0bit 0pps backlog 0b 0p requeues 0
> qdisc pfifo_fast 0: dev eth2 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1
> 1 1 1
> Sent 1815657070136 bytes 1536674488 pkt (dropped 0, overlimits 0
> requeues 1448094)
> rate 0bit 0pps backlog 0b 0p requeues 1448094
> qdisc pfifo_fast 0: dev eth3 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1
> 1 1 1
> Sent 1752033393324 bytes 1536906566 pkt (dropped 0, overlimits 0
> requeues 1063672)
> rate 0bit 0pps backlog 0b 0p requeues 1063672
>
> [root@lf1001-240 ipv4]# cat /proc/interrupts
> CPU0 CPU1
> 0: 46020594 44501954 IO-APIC-edge timer
> 1: 9 0 IO-APIC-edge i8042
> 7: 0 0 IO-APIC-edge parport0
> 8: 1 0 IO-APIC-edge rtc
> 9: 0 0 IO-APIC-level acpi
> 12: 96 0 IO-APIC-edge i8042
> 14: 394023 407282 IO-APIC-edge ide0
> 16: 0 0 IO-APIC-level uhci_hcd:usb4
> 17: 1134346 1034006 IO-APIC-level uhci_hcd:usb3, eth0
> 18: 81605 83739 IO-APIC-level libata, uhci_hcd:usb2, eth1
> 19: 0 0 IO-APIC-level uhci_hcd:usb1, ehci_hcd:usb5
> 20: 53056128 46598235 IO-APIC-level eth2
> 21: 47534577 52189674 IO-APIC-level eth3
> NMI: 0 0
> LOC: 90485383 90485382
> ERR: 0
> MIS: 0
>
> [root@lf1001-240 ipv4]# cat /proc/net/sockstat
> sockets: used 334
> TCP: inuse 27 orphan 0 tw 0 alloc 27 mem 360
> UDP: inuse 12
> RAW: inuse 0
> FRAG: inuse 0 memory 0
>
> [root@lf1001-240 ipv4]# cat /proc/net/softnet_stat
> d58236f1 00000000 023badc3 00000000 00000000 00000000 00000000 00000000
> 0004ef01
> 3a4354a1 00000000 01b57b4b 00000000 00000000 00000000 00000000 00000000
> 0005445f
>
Hum, could you try to bind nic irqs on separate cpus ?
eth2 -> CPU0 and eth3 -> CPU1
# echo 1 >/proc/irq/20/smp_affinity
# echo 2 >/proc/irq/21/smp_affinity
next prev parent reply other threads:[~2007-04-12 20:41 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-04-11 18:50 TCP connection stops after high load Ben Greear
2007-04-11 20:26 ` Ben Greear
2007-04-11 20:48 ` David Miller
2007-04-11 21:06 ` Ben Greear
2007-04-11 21:11 ` David Miller
2007-04-11 21:31 ` Ben Greear
2007-04-11 21:39 ` David Miller
2007-04-12 2:44 ` SANGTAE HA
2007-04-12 1:06 ` Benjamin LaHaise
2007-04-12 14:48 ` Andi Kleen
2007-04-12 17:59 ` Ben Greear
2007-04-12 18:19 ` Eric Dumazet
2007-04-12 19:12 ` Ben Greear
2007-04-12 20:41 ` Eric Dumazet [this message]
2007-04-12 21:36 ` Ben Greear
2007-04-13 7:09 ` Evgeniy Polyakov
2007-04-13 16:42 ` Ben Greear
2007-04-13 16:10 ` Daniel Schaffrath
2007-04-13 16:41 ` Eric Dumazet
2007-04-14 4:21 ` Herbert Xu
2007-04-14 4:25 ` David Miller
2007-04-14 5:31 ` Eric Dumazet
2007-04-14 5:37 ` David Miller
2007-04-11 20:41 ` David Miller
2007-04-12 6:12 ` Ilpo Järvinen
-- strict thread matches above, loose matches on Subject: below --
2007-04-12 21:11 Robert Iakobashvili
2007-04-12 21:15 ` David Miller
2007-04-15 12:14 ` Robert Iakobashvili
2007-04-15 15:31 ` John Heffner
2007-04-15 15:49 ` Robert Iakobashvili
2007-04-16 18:07 ` John Heffner
2007-04-16 18:51 ` Robert Iakobashvili
2007-04-16 19:11 ` John Heffner
2007-04-16 19:17 ` David Miller
2007-04-16 19:15 ` David Miller
2007-04-17 7:58 ` Robert Iakobashvili
2007-04-17 19:39 ` David Miller
2007-04-17 19:47 ` John Heffner
2007-04-17 19:51 ` David Miller
2007-04-17 19:58 ` Robert Iakobashvili
2007-04-15 13:52 ` Robert Iakobashvili
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=461E998A.8080409@cosmosbay.com \
--to=dada1@cosmosbay.com \
--cc=andi@firstfloor.org \
--cc=bcrl@kvack.org \
--cc=greearb@candelatech.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).