* packetloss, on e1000e worse than r8169?
@ 2008-06-16 19:37 Denys Fedoryshchenko
2008-06-16 19:48 ` Eric Dumazet
` (3 more replies)
0 siblings, 4 replies; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 19:37 UTC (permalink / raw)
To: netdev; +Cc: linux.nics
Hi again
I moved from PCI Realtek 8169 to onboard PCI-Express e1000e, and situation
become worse. Strange? Yes.
Here is info
Load and error rate:
MegaRouter-KARAM /sys # ifconfig eth1;sleep 10;ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:91991642 errors:0 dropped:1803444 overruns:0 frame:0
TX packets:91914611 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:316112765 (301.4 MiB) TX bytes:138303107 (131.8 MiB)
Memory:90300000-90320000
eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:92812514 errors:0 dropped:1815490 overruns:0 frame:0
TX packets:92734865 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:739941322 (705.6 MiB) TX bytes:560547138 (534.5 MiB)
Memory:90300000-90320000
System load (there is no NAT, no shapers, just routing, around 600-700 routes):
MegaRouter-KARAM /sys # mpstat 1
Linux 2.6.26-rc6-git2-build-0029 (MegaRouter-KARAM) 06/16/08
22:36:36 CPU %user %nice %sys %iowait %irq %soft %steal
%idle intr/s
22:36:37 all 0.00 0.00 0.00 0.00 0.00 1.49 0.00
98.51 19764.00
22:36:38 all 0.00 0.00 0.50 0.00 0.00 0.50 0.00
99.01 19888.00
22:36:39 all 0.00 0.00 0.00 0.00 0.00 0.50 0.00
99.50 19642.00
22:36:40 all 0.00 0.00 0.00 0.00 1.01 9.05 0.00
89.95 19543.00
(system almost idle)
MegaRouter-KARAM /sys # ethtool -S eth1
NIC statistics:
rx_packets: 109977509
tx_packets: 109887692
rx_bytes: 57656749138
tx_bytes: 57536071746
rx_broadcast: 6497
tx_broadcast: 92
rx_multicast: 48995
tx_multicast: 1960
rx_errors: 0
tx_errors: 0
tx_dropped: 0
multicast: 48995
collisions: 0
rx_length_errors: 0
rx_over_errors: 0
rx_crc_errors: 0
rx_frame_errors: 0
rx_no_buffer_count: 1796
rx_missed_errors: 2182679
tx_aborted_errors: 0
tx_carrier_errors: 0
tx_fifo_errors: 0
tx_heartbeat_errors: 0
tx_window_errors: 0
tx_abort_late_coll: 0
tx_deferred_ok: 55617
tx_single_coll_ok: 0
tx_multi_coll_ok: 0
tx_timeout_count: 0
tx_restart_queue: 1626
rx_long_length_errors: 0
rx_short_length_errors: 0
rx_align_errors: 0
tx_tcp_seg_good: 0
tx_tcp_seg_failed: 0
rx_flow_control_xon: 55461
rx_flow_control_xoff: 57329
tx_flow_control_xon: 39114
tx_flow_control_xoff: 48341
rx_long_byte_count: 57656749138
rx_csum_offload_good: 104097306
rx_csum_offload_errors: 2209
rx_header_split: 0
alloc_rx_buff_failed: 0
tx_smbus: 22
rx_smbus: 2201
dropped_smbus: 0
rx_dma_failed: 0
tx_dma_failed: 0
MegaRouter-KARAM /sys # ethtool -S eth1
NIC statistics:
rx_packets: 110154571
tx_packets: 110064459
rx_bytes: 57753358981
tx_bytes: 57632419451
rx_broadcast: 6501
tx_broadcast: 92
rx_multicast: 49155
tx_multicast: 1960
rx_errors: 0
tx_errors: 0
tx_dropped: 0
multicast: 49155
collisions: 0
rx_length_errors: 0
rx_over_errors: 0
rx_crc_errors: 0
rx_frame_errors: 0
rx_no_buffer_count: 1796
rx_missed_errors: 2187703
tx_aborted_errors: 0
tx_carrier_errors: 0
tx_fifo_errors: 0
tx_heartbeat_errors: 0
tx_window_errors: 0
tx_abort_late_coll: 0
tx_deferred_ok: 55772
tx_single_coll_ok: 0
tx_multi_coll_ok: 0
tx_timeout_count: 0
tx_restart_queue: 1628
rx_long_length_errors: 0
rx_short_length_errors: 0
rx_align_errors: 0
tx_tcp_seg_good: 0
tx_tcp_seg_failed: 0
rx_flow_control_xon: 55615
rx_flow_control_xoff: 57488
tx_flow_control_xon: 39209
tx_flow_control_xoff: 48448
rx_long_byte_count: 57753358981
rx_csum_offload_good: 104264656
rx_csum_offload_errors: 2210
rx_header_split: 0
alloc_rx_buff_failed: 0
tx_smbus: 23
rx_smbus: 2207
dropped_smbus: 0
rx_dma_failed: 0
tx_dma_failed: 0
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 19:37 packetloss, on e1000e worse than r8169? Denys Fedoryshchenko
@ 2008-06-16 19:48 ` Eric Dumazet
2008-06-16 20:20 ` Denys Fedoryshchenko
2008-06-16 20:23 ` Denys Fedoryshchenko
2008-06-16 20:15 ` Auke Kok
` (2 subsequent siblings)
3 siblings, 2 replies; 33+ messages in thread
From: Eric Dumazet @ 2008-06-16 19:48 UTC (permalink / raw)
To: Denys Fedoryshchenko; +Cc: netdev, linux.nics
Denys Fedoryshchenko a e'crit :
> Hi again
>
> I moved from PCI Realtek 8169 to onboard PCI-Express e1000e, and situation
> become worse. Strange? Yes.
>
> Here is info
> Load and error rate:
>
> MegaRouter-KARAM /sys # ifconfig eth1;sleep 10;ifconfig eth1
> eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
> inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:91991642 errors:0 dropped:1803444 overruns:0 frame:0
> TX packets:91914611 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:316112765 (301.4 MiB) TX bytes:138303107 (131.8 MiB)
> Memory:90300000-90320000
>
> eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
> inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:92812514 errors:0 dropped:1815490 overruns:0 frame:0
> TX packets:92734865 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:739941322 (705.6 MiB) TX bytes:560547138 (534.5 MiB)
> Memory:90300000-90320000
>
> System load (there is no NAT, no shapers, just routing, around 600-700 routes):
> MegaRouter-KARAM /sys # mpstat 1
> Linux 2.6.26-rc6-git2-build-0029 (MegaRouter-KARAM) 06/16/08
>
> 22:36:36 CPU %user %nice %sys %iowait %irq %soft %steal
> %idle intr/s
> 22:36:37 all 0.00 0.00 0.00 0.00 0.00 1.49 0.00
> 98.51 19764.00
> 22:36:38 all 0.00 0.00 0.50 0.00 0.00 0.50 0.00
> 99.01 19888.00
> 22:36:39 all 0.00 0.00 0.00 0.00 0.00 0.50 0.00
> 99.50 19642.00
> 22:36:40 all 0.00 0.00 0.00 0.00 1.01 9.05 0.00
> 89.95 19543.00
>
> (system almost idle)
>
> MegaRouter-KARAM /sys # ethtool -S eth1
> NIC statistics:
> rx_packets: 109977509
> tx_packets: 109887692
> rx_bytes: 57656749138
> tx_bytes: 57536071746
> rx_broadcast: 6497
> tx_broadcast: 92
> rx_multicast: 48995
> tx_multicast: 1960
> rx_errors: 0
> tx_errors: 0
> tx_dropped: 0
> multicast: 48995
> collisions: 0
> rx_length_errors: 0
> rx_over_errors: 0
> rx_crc_errors: 0
> rx_frame_errors: 0
> rx_no_buffer_count: 1796
> rx_missed_errors: 2182679
> tx_aborted_errors: 0
> tx_carrier_errors: 0
> tx_fifo_errors: 0
> tx_heartbeat_errors: 0
> tx_window_errors: 0
> tx_abort_late_coll: 0
> tx_deferred_ok: 55617
> tx_single_coll_ok: 0
> tx_multi_coll_ok: 0
> tx_timeout_count: 0
> tx_restart_queue: 1626
> rx_long_length_errors: 0
> rx_short_length_errors: 0
> rx_align_errors: 0
> tx_tcp_seg_good: 0
> tx_tcp_seg_failed: 0
> rx_flow_control_xon: 55461
> rx_flow_control_xoff: 57329
> tx_flow_control_xon: 39114
> tx_flow_control_xoff: 48341
> rx_long_byte_count: 57656749138
> rx_csum_offload_good: 104097306
> rx_csum_offload_errors: 2209
> rx_header_split: 0
> alloc_rx_buff_failed: 0
> tx_smbus: 22
> rx_smbus: 2201
> dropped_smbus: 0
> rx_dma_failed: 0
> tx_dma_failed: 0
> MegaRouter-KARAM /sys # ethtool -S eth1
> NIC statistics:
> rx_packets: 110154571
> tx_packets: 110064459
> rx_bytes: 57753358981
> tx_bytes: 57632419451
> rx_broadcast: 6501
> tx_broadcast: 92
> rx_multicast: 49155
> tx_multicast: 1960
> rx_errors: 0
> tx_errors: 0
> tx_dropped: 0
> multicast: 49155
> collisions: 0
> rx_length_errors: 0
> rx_over_errors: 0
> rx_crc_errors: 0
> rx_frame_errors: 0
> rx_no_buffer_count: 1796
> rx_missed_errors: 2187703
> tx_aborted_errors: 0
> tx_carrier_errors: 0
> tx_fifo_errors: 0
> tx_heartbeat_errors: 0
> tx_window_errors: 0
> tx_abort_late_coll: 0
> tx_deferred_ok: 55772
> tx_single_coll_ok: 0
> tx_multi_coll_ok: 0
> tx_timeout_count: 0
> tx_restart_queue: 1628
> rx_long_length_errors: 0
> rx_short_length_errors: 0
> rx_align_errors: 0
> tx_tcp_seg_good: 0
> tx_tcp_seg_failed: 0
> rx_flow_control_xon: 55615
> rx_flow_control_xoff: 57488
> tx_flow_control_xon: 39209
> tx_flow_control_xoff: 48448
> rx_long_byte_count: 57753358981
> rx_csum_offload_good: 104264656
> rx_csum_offload_errors: 2210
> rx_header_split: 0
> alloc_rx_buff_failed: 0
> tx_smbus: 23
> rx_smbus: 2207
> dropped_smbus: 0
> rx_dma_failed: 0
> tx_dma_failed: 0
>
>
>
What RX & TX rings look like on e1000e ?
ethtool -g eth1
Also, please post "cat /proc/interrupts"
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 19:37 packetloss, on e1000e worse than r8169? Denys Fedoryshchenko
2008-06-16 19:48 ` Eric Dumazet
@ 2008-06-16 20:15 ` Auke Kok
2008-06-16 20:29 ` Denys Fedoryshchenko
2008-06-16 20:26 ` Francois Romieu
2008-06-16 20:37 ` Waskiewicz Jr, Peter P
3 siblings, 1 reply; 33+ messages in thread
From: Auke Kok @ 2008-06-16 20:15 UTC (permalink / raw)
To: Denys Fedoryshchenko; +Cc: netdev, e1000-list
Denys Fedoryshchenko wrote:
> Hi again
>
> I moved from PCI Realtek 8169 to onboard PCI-Express e1000e, and situation
> become worse. Strange? Yes.
>
> Here is info
> Load and error rate:
>
> MegaRouter-KARAM /sys # ifconfig eth1;sleep 10;ifconfig eth1
> eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
> inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:91991642 errors:0 dropped:1803444 overruns:0 frame:0
> TX packets:91914611 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:316112765 (301.4 MiB) TX bytes:138303107 (131.8 MiB)
> Memory:90300000-90320000
>
> eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
> inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:92812514 errors:0 dropped:1815490 overruns:0 frame:0
> TX packets:92734865 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:739941322 (705.6 MiB) TX bytes:560547138 (534.5 MiB)
> Memory:90300000-90320000
>
> System load (there is no NAT, no shapers, just routing, around 600-700 routes):
> MegaRouter-KARAM /sys # mpstat 1
> Linux 2.6.26-rc6-git2-build-0029 (MegaRouter-KARAM) 06/16/08
>
> 22:36:36 CPU %user %nice %sys %iowait %irq %soft %steal
> %idle intr/s
> 22:36:37 all 0.00 0.00 0.00 0.00 0.00 1.49 0.00
> 98.51 19764.00
> 22:36:38 all 0.00 0.00 0.50 0.00 0.00 0.50 0.00
> 99.01 19888.00
> 22:36:39 all 0.00 0.00 0.00 0.00 0.00 0.50 0.00
> 99.50 19642.00
> 22:36:40 all 0.00 0.00 0.00 0.00 1.01 9.05 0.00
> 89.95 19543.00
>
> (system almost idle)
but has 20000 interrupts per second! that's hardly idle by any account :)
what options did use to load e1000e? can you show the relevant dmesg parts? post
contents of /proc/interrupts?
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 19:48 ` Eric Dumazet
@ 2008-06-16 20:20 ` Denys Fedoryshchenko
2008-06-16 20:23 ` Denys Fedoryshchenko
1 sibling, 0 replies; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 20:20 UTC (permalink / raw)
To: Eric Dumazet; +Cc: netdev
On Mon, 16 Jun 2008 21:48:55 +0200, Eric Dumazet wrote
> Denys Fedoryshchenko a e'crit :
> > Hi again
> >
> > I moved from PCI Realtek 8169 to onboard PCI-Express e1000e, and situation
> > become worse. Strange? Yes.
> >
> > Here is info
> > Load and error rate:
> >
> > MegaRouter-KARAM /sys # ifconfig eth1;sleep 10;ifconfig eth1
> > eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
> > inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > RX packets:91991642 errors:0 dropped:1803444 overruns:0 frame:0
> > TX packets:91914611 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:1000
> > RX bytes:316112765 (301.4 MiB) TX bytes:138303107 (131.8 MiB)
> > Memory:90300000-90320000
> >
> > eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
> > inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > RX packets:92812514 errors:0 dropped:1815490 overruns:0 frame:0
> > TX packets:92734865 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:1000
> > RX bytes:739941322 (705.6 MiB) TX bytes:560547138 (534.5 MiB)
> > Memory:90300000-90320000
> >
> > System load (there is no NAT, no shapers, just routing, around 600-700
routes):
> > MegaRouter-KARAM /sys # mpstat 1
> > Linux 2.6.26-rc6-git2-build-0029 (MegaRouter-KARAM) 06/16/08
> >
> > 22:36:36 CPU %user %nice %sys %iowait %irq %soft %steal
> > %idle intr/s
> > 22:36:37 all 0.00 0.00 0.00 0.00 0.00 1.49 0.00
> > 98.51 19764.00
> > 22:36:38 all 0.00 0.00 0.50 0.00 0.00 0.50 0.00
> > 99.01 19888.00
> > 22:36:39 all 0.00 0.00 0.00 0.00 0.00 0.50 0.00
> > 99.50 19642.00
> > 22:36:40 all 0.00 0.00 0.00 0.00 1.01 9.05 0.00
> > 89.95 19543.00
> >
> > (system almost idle)
> >
> > MegaRouter-KARAM /sys # ethtool -S eth1
> > NIC statistics:
> > rx_packets: 109977509
> > tx_packets: 109887692
> > rx_bytes: 57656749138
> > tx_bytes: 57536071746
> > rx_broadcast: 6497
> > tx_broadcast: 92
> > rx_multicast: 48995
> > tx_multicast: 1960
> > rx_errors: 0
> > tx_errors: 0
> > tx_dropped: 0
> > multicast: 48995
> > collisions: 0
> > rx_length_errors: 0
> > rx_over_errors: 0
> > rx_crc_errors: 0
> > rx_frame_errors: 0
> > rx_no_buffer_count: 1796
> > rx_missed_errors: 2182679
> > tx_aborted_errors: 0
> > tx_carrier_errors: 0
> > tx_fifo_errors: 0
> > tx_heartbeat_errors: 0
> > tx_window_errors: 0
> > tx_abort_late_coll: 0
> > tx_deferred_ok: 55617
> > tx_single_coll_ok: 0
> > tx_multi_coll_ok: 0
> > tx_timeout_count: 0
> > tx_restart_queue: 1626
> > rx_long_length_errors: 0
> > rx_short_length_errors: 0
> > rx_align_errors: 0
> > tx_tcp_seg_good: 0
> > tx_tcp_seg_failed: 0
> > rx_flow_control_xon: 55461
> > rx_flow_control_xoff: 57329
> > tx_flow_control_xon: 39114
> > tx_flow_control_xoff: 48341
> > rx_long_byte_count: 57656749138
> > rx_csum_offload_good: 104097306
> > rx_csum_offload_errors: 2209
> > rx_header_split: 0
> > alloc_rx_buff_failed: 0
> > tx_smbus: 22
> > rx_smbus: 2201
> > dropped_smbus: 0
> > rx_dma_failed: 0
> > tx_dma_failed: 0
> > MegaRouter-KARAM /sys # ethtool -S eth1
> > NIC statistics:
> > rx_packets: 110154571
> > tx_packets: 110064459
> > rx_bytes: 57753358981
> > tx_bytes: 57632419451
> > rx_broadcast: 6501
> > tx_broadcast: 92
> > rx_multicast: 49155
> > tx_multicast: 1960
> > rx_errors: 0
> > tx_errors: 0
> > tx_dropped: 0
> > multicast: 49155
> > collisions: 0
> > rx_length_errors: 0
> > rx_over_errors: 0
> > rx_crc_errors: 0
> > rx_frame_errors: 0
> > rx_no_buffer_count: 1796
> > rx_missed_errors: 2187703
> > tx_aborted_errors: 0
> > tx_carrier_errors: 0
> > tx_fifo_errors: 0
> > tx_heartbeat_errors: 0
> > tx_window_errors: 0
> > tx_abort_late_coll: 0
> > tx_deferred_ok: 55772
> > tx_single_coll_ok: 0
> > tx_multi_coll_ok: 0
> > tx_timeout_count: 0
> > tx_restart_queue: 1628
> > rx_long_length_errors: 0
> > rx_short_length_errors: 0
> > rx_align_errors: 0
> > tx_tcp_seg_good: 0
> > tx_tcp_seg_failed: 0
> > rx_flow_control_xon: 55615
> > rx_flow_control_xoff: 57488
> > tx_flow_control_xon: 39209
> > tx_flow_control_xoff: 48448
> > rx_long_byte_count: 57753358981
> > rx_csum_offload_good: 104264656
> > rx_csum_offload_errors: 2210
> > rx_header_split: 0
> > alloc_rx_buff_failed: 0
> > tx_smbus: 23
> > rx_smbus: 2207
> > dropped_smbus: 0
> > rx_dma_failed: 0
> > tx_dma_failed: 0
> >
> >
> >
>
> What RX & TX rings look like on e1000e ?
>
> ethtool -g eth1
>
> Also, please post "cat /proc/interrupts"
>
MegaRouter-KARAM ~ # ethtool -g eth1
Ring parameters for eth1:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 256
RX Mini: 0
RX Jumbo: 0
TX: 256
After i tried to set ethtool -G eth1 4096 system becomes extremely slow, what
is very strange for me. I had to soft-reboot it (over kexec).
MegaRouter-KARAM ~ # cat /proc/interrupts;sleep 10;cat /proc/interrupts
CPU0 CPU1
0: 86148 0 IO-APIC-edge timer
1: 2 0 IO-APIC-edge i8042
9: 0 0 IO-APIC-fasteoi acpi
12: 5 0 IO-APIC-edge i8042
16: 0 0 IO-APIC-fasteoi uhci_hcd:usb3
18: 0 0 IO-APIC-fasteoi ehci_hcd:usb1, uhci_hcd:usb7
19: 0 0 IO-APIC-fasteoi uhci_hcd:usb6
21: 0 0 IO-APIC-fasteoi uhci_hcd:usb4
23: 968 0 IO-APIC-fasteoi ehci_hcd:usb2, uhci_hcd:usb5
217: 3 1460438 PCI-MSI-edge eth1
NMI: 86087 85892 Non-maskable interrupts
LOC: 0 85892 Local timer interrupts
RES: 44 81 Rescheduling interrupts
CAL: 88 56 function call interrupts
TLB: 91 78 TLB shootdowns
TRM: 0 0 Thermal event interrupts
SPU: 0 0 Spurious interrupts
ERR: 0
MIS: 0
CPU0 CPU1
0: 96150 0 IO-APIC-edge timer
1: 2 0 IO-APIC-edge i8042
9: 0 0 IO-APIC-fasteoi acpi
12: 5 0 IO-APIC-edge i8042
16: 0 0 IO-APIC-fasteoi uhci_hcd:usb3
18: 0 0 IO-APIC-fasteoi ehci_hcd:usb1, uhci_hcd:usb7
19: 0 0 IO-APIC-fasteoi uhci_hcd:usb6
21: 0 0 IO-APIC-fasteoi uhci_hcd:usb4
23: 968 0 IO-APIC-fasteoi ehci_hcd:usb2, uhci_hcd:usb5
217: 3 1646537 PCI-MSI-edge eth1
NMI: 96088 95893 Non-maskable interrupts
LOC: 0 95893 Local timer interrupts
RES: 44 81 Rescheduling interrupts
CAL: 89 56 function call interrupts
TLB: 91 78 TLB shootdowns
TRM: 0 0 Thermal event interrupts
SPU: 0 0 Spurious interrupts
ERR: 0
MIS: 0
About NMI interrupts - nmi_watchdog=1, if i disable it, it doesn't change
anything.
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 19:48 ` Eric Dumazet
2008-06-16 20:20 ` Denys Fedoryshchenko
@ 2008-06-16 20:23 ` Denys Fedoryshchenko
2008-06-16 20:36 ` Kok, Auke
1 sibling, 1 reply; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 20:23 UTC (permalink / raw)
To: Eric Dumazet; +Cc: netdev, linux.nics
Also if important
MegaRouter-KARAM ~ # dmesg|grep eth1
[ 3.955620] eth1: (PCI Express:2.5GB/s:Width x1) 00:19:d1:71:5f:33
[ 3.955622] eth1: Intel(R) PRO/1000 Network Connection
[ 3.955642] eth1: MAC: 4, PHY: 6, PBA No: ffffff-0ff
[ 5.358191] netconsole: interface eth1
[ 6.432345] eth1: Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
MegaRouter-KARAM ~ # ethtool eth1
Settings for eth1:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbag
Wake-on: g
Current message level: 0x00000001 (1)
Link detected: yes
MegaRouter-KARAM ~ # ethtool -c eth1
Coalesce parameters for eth1:
Adaptive RX: off TX: off
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0
rx-usecs: 3
I tried to change Intel's "magic" rx-usecs to 0,1,3,10000,100000, it doesn't
help at all. Attempt to change offloading parameters (disabling) doesn't make
sense too.
On Mon, 16 Jun 2008 21:48:55 +0200, Eric Dumazet wrote
> Denys Fedoryshchenko a e'crit :
> > Hi again
> >
> > I moved from PCI Realtek 8169 to onboard PCI-Express e1000e, and situation
> > become worse. Strange? Yes.
> >
> > Here is info
> > Load and error rate:
> >
> > MegaRouter-KARAM /sys # ifconfig eth1;sleep 10;ifconfig eth1
> > eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
> > inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > RX packets:91991642 errors:0 dropped:1803444 overruns:0 frame:0
> > TX packets:91914611 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:1000
> > RX bytes:316112765 (301.4 MiB) TX bytes:138303107 (131.8 MiB)
> > Memory:90300000-90320000
> >
> > eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
> > inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > RX packets:92812514 errors:0 dropped:1815490 overruns:0 frame:0
> > TX packets:92734865 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:1000
> > RX bytes:739941322 (705.6 MiB) TX bytes:560547138 (534.5 MiB)
> > Memory:90300000-90320000
> >
> > System load (there is no NAT, no shapers, just routing, around 600-700
routes):
> > MegaRouter-KARAM /sys # mpstat 1
> > Linux 2.6.26-rc6-git2-build-0029 (MegaRouter-KARAM) 06/16/08
> >
> > 22:36:36 CPU %user %nice %sys %iowait %irq %soft %steal
> > %idle intr/s
> > 22:36:37 all 0.00 0.00 0.00 0.00 0.00 1.49 0.00
> > 98.51 19764.00
> > 22:36:38 all 0.00 0.00 0.50 0.00 0.00 0.50 0.00
> > 99.01 19888.00
> > 22:36:39 all 0.00 0.00 0.00 0.00 0.00 0.50 0.00
> > 99.50 19642.00
> > 22:36:40 all 0.00 0.00 0.00 0.00 1.01 9.05 0.00
> > 89.95 19543.00
> >
> > (system almost idle)
> >
> > MegaRouter-KARAM /sys # ethtool -S eth1
> > NIC statistics:
> > rx_packets: 109977509
> > tx_packets: 109887692
> > rx_bytes: 57656749138
> > tx_bytes: 57536071746
> > rx_broadcast: 6497
> > tx_broadcast: 92
> > rx_multicast: 48995
> > tx_multicast: 1960
> > rx_errors: 0
> > tx_errors: 0
> > tx_dropped: 0
> > multicast: 48995
> > collisions: 0
> > rx_length_errors: 0
> > rx_over_errors: 0
> > rx_crc_errors: 0
> > rx_frame_errors: 0
> > rx_no_buffer_count: 1796
> > rx_missed_errors: 2182679
> > tx_aborted_errors: 0
> > tx_carrier_errors: 0
> > tx_fifo_errors: 0
> > tx_heartbeat_errors: 0
> > tx_window_errors: 0
> > tx_abort_late_coll: 0
> > tx_deferred_ok: 55617
> > tx_single_coll_ok: 0
> > tx_multi_coll_ok: 0
> > tx_timeout_count: 0
> > tx_restart_queue: 1626
> > rx_long_length_errors: 0
> > rx_short_length_errors: 0
> > rx_align_errors: 0
> > tx_tcp_seg_good: 0
> > tx_tcp_seg_failed: 0
> > rx_flow_control_xon: 55461
> > rx_flow_control_xoff: 57329
> > tx_flow_control_xon: 39114
> > tx_flow_control_xoff: 48341
> > rx_long_byte_count: 57656749138
> > rx_csum_offload_good: 104097306
> > rx_csum_offload_errors: 2209
> > rx_header_split: 0
> > alloc_rx_buff_failed: 0
> > tx_smbus: 22
> > rx_smbus: 2201
> > dropped_smbus: 0
> > rx_dma_failed: 0
> > tx_dma_failed: 0
> > MegaRouter-KARAM /sys # ethtool -S eth1
> > NIC statistics:
> > rx_packets: 110154571
> > tx_packets: 110064459
> > rx_bytes: 57753358981
> > tx_bytes: 57632419451
> > rx_broadcast: 6501
> > tx_broadcast: 92
> > rx_multicast: 49155
> > tx_multicast: 1960
> > rx_errors: 0
> > tx_errors: 0
> > tx_dropped: 0
> > multicast: 49155
> > collisions: 0
> > rx_length_errors: 0
> > rx_over_errors: 0
> > rx_crc_errors: 0
> > rx_frame_errors: 0
> > rx_no_buffer_count: 1796
> > rx_missed_errors: 2187703
> > tx_aborted_errors: 0
> > tx_carrier_errors: 0
> > tx_fifo_errors: 0
> > tx_heartbeat_errors: 0
> > tx_window_errors: 0
> > tx_abort_late_coll: 0
> > tx_deferred_ok: 55772
> > tx_single_coll_ok: 0
> > tx_multi_coll_ok: 0
> > tx_timeout_count: 0
> > tx_restart_queue: 1628
> > rx_long_length_errors: 0
> > rx_short_length_errors: 0
> > rx_align_errors: 0
> > tx_tcp_seg_good: 0
> > tx_tcp_seg_failed: 0
> > rx_flow_control_xon: 55615
> > rx_flow_control_xoff: 57488
> > tx_flow_control_xon: 39209
> > tx_flow_control_xoff: 48448
> > rx_long_byte_count: 57753358981
> > rx_csum_offload_good: 104264656
> > rx_csum_offload_errors: 2210
> > rx_header_split: 0
> > alloc_rx_buff_failed: 0
> > tx_smbus: 23
> > rx_smbus: 2207
> > dropped_smbus: 0
> > rx_dma_failed: 0
> > tx_dma_failed: 0
> >
> >
> >
>
> What RX & TX rings look like on e1000e ?
>
> ethtool -g eth1
>
> Also, please post "cat /proc/interrupts"
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 19:37 packetloss, on e1000e worse than r8169? Denys Fedoryshchenko
2008-06-16 19:48 ` Eric Dumazet
2008-06-16 20:15 ` Auke Kok
@ 2008-06-16 20:26 ` Francois Romieu
2008-06-16 20:37 ` Waskiewicz Jr, Peter P
3 siblings, 0 replies; 33+ messages in thread
From: Francois Romieu @ 2008-06-16 20:26 UTC (permalink / raw)
To: Denys Fedoryshchenko; +Cc: netdev, linux.nics
Denys Fedoryshchenko <denys@visp.net.lb> :
[...]
> I moved from PCI Realtek 8169 to onboard PCI-Express e1000e, and situation
> become worse. Strange ? Yes.
Could it be that the r8169 and the e1000e driver do not use the same locking
tricks at all ?
--
Ueimor
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 20:15 ` Auke Kok
@ 2008-06-16 20:29 ` Denys Fedoryshchenko
0 siblings, 0 replies; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 20:29 UTC (permalink / raw)
To: Auke Kok; +Cc: netdev, e1000-list
I did in previous message.
Here is more information about hardware.
Base Board Information
Manufacturer: Intel Corporation
Product Name: DQ965GF
Version: AAD41676-402
Serial Number: BQGF712003M6
MegaRouter-KARAM ~ # lspci -t
-[0000:00]-+-00.0
+-01.0-[0000:01]--
+-02.0
+-03.0
+-03.2
+-03.3
+-19.0
+-1a.0
+-1a.1
+-1a.7
+-1b.0
+-1c.0-[0000:02]--
+-1c.1-[0000:03]----00.0
+-1c.2-[0000:04]--
+-1c.3-[0000:05]--
+-1c.4-[0000:06]--
+-1d.0
+-1d.1
+-1d.2
+-1d.7
+-1e.0-[0000:07]----00.0
+-1f.0
+-1f.2
\-1f.3
MegaRouter-KARAM ~ # lspci -t -v
-[0000:00]-+-00.0 Intel Corporation 82Q963/Q965 Memory Controller Hub
+-01.0-[0000:01]--
+-02.0 Intel Corporation 82Q963/Q965 Integrated Graphics Controller
+-03.0 Intel Corporation 82Q963/Q965 HECI Controller
+-03.2 Intel Corporation 82Q963/Q965 PT IDER Controller
+-03.3 Intel Corporation 82Q963/Q965 KT Controller
+-19.0 Intel Corporation 82566DM Gigabit Network Connection
+-1a.0 Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #4
+-1a.1 Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #5
+-1a.7 Intel Corporation 82801H (ICH8 Family) USB2 EHCI Controller #2
+-1b.0 Intel Corporation 82801H (ICH8 Family) HD Audio Controller
+-1c.0-[0000:02]--
+-1c.1-[0000:03]----00.0 Marvell Technology Group Ltd. 88SE6101
single-port PATA133 interface
+-1c.2-[0000:04]--
+-1c.3-[0000:05]--
+-1c.4-[0000:06]--
+-1d.0 Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #1
+-1d.1 Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #2
+-1d.2 Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #3
+-1d.7 Intel Corporation 82801H (ICH8 Family) USB2 EHCI Controller #1
+-1e.0-[0000:07]----00.0 Linksys Gigabit Network Adapter
+-1f.0 Intel Corporation 82801HO (ICH8DO) LPC Interface Controller
+-1f.2 Intel Corporation 82801HR/HO/HH (ICH8R/DO/DH) 6 port SATA
AHCI Controller
\-1f.3 Intel Corporation 82801H (ICH8 Family) SMBus Controller
On Mon, 16 Jun 2008 13:15:41 -0700, Auke Kok wrote
> Denys Fedoryshchenko wrote:
> > Hi again
> >
> > I moved from PCI Realtek 8169 to onboard PCI-Express e1000e, and situation
> > become worse. Strange? Yes.
> >
> > Here is info
> > Load and error rate:
> >
> > MegaRouter-KARAM /sys # ifconfig eth1;sleep 10;ifconfig eth1
> > eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
> > inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > RX packets:91991642 errors:0 dropped:1803444 overruns:0 frame:0
> > TX packets:91914611 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:1000
> > RX bytes:316112765 (301.4 MiB) TX bytes:138303107 (131.8 MiB)
> > Memory:90300000-90320000
> >
> > eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
> > inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > RX packets:92812514 errors:0 dropped:1815490 overruns:0 frame:0
> > TX packets:92734865 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:1000
> > RX bytes:739941322 (705.6 MiB) TX bytes:560547138 (534.5 MiB)
> > Memory:90300000-90320000
> >
> > System load (there is no NAT, no shapers, just routing, around 600-700
routes):
> > MegaRouter-KARAM /sys # mpstat 1
> > Linux 2.6.26-rc6-git2-build-0029 (MegaRouter-KARAM) 06/16/08
> >
> > 22:36:36 CPU %user %nice %sys %iowait %irq %soft %steal
> > %idle intr/s
> > 22:36:37 all 0.00 0.00 0.00 0.00 0.00 1.49 0.00
> > 98.51 19764.00
> > 22:36:38 all 0.00 0.00 0.50 0.00 0.00 0.50 0.00
> > 99.01 19888.00
> > 22:36:39 all 0.00 0.00 0.00 0.00 0.00 0.50 0.00
> > 99.50 19642.00
> > 22:36:40 all 0.00 0.00 0.00 0.00 1.01 9.05 0.00
> > 89.95 19543.00
> >
> > (system almost idle)
>
> but has 20000 interrupts per second! that's hardly idle by any
> account :)
>
> what options did use to load e1000e? can you show the relevant dmesg
> parts? post contents of /proc/interrupts?
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 20:23 ` Denys Fedoryshchenko
@ 2008-06-16 20:36 ` Kok, Auke
2008-06-16 20:52 ` Denys Fedoryshchenko
[not found] ` <20080616204411.M52834@visp.net.lb>
0 siblings, 2 replies; 33+ messages in thread
From: Kok, Auke @ 2008-06-16 20:36 UTC (permalink / raw)
To: Denys Fedoryshchenko; +Cc: Eric Dumazet, netdev, linux.nics
Denys Fedoryshchenko wrote:
> Also if important
> MegaRouter-KARAM ~ # dmesg|grep eth1
> [ 3.955620] eth1: (PCI Express:2.5GB/s:Width x1) 00:19:d1:71:5f:33
> [ 3.955622] eth1: Intel(R) PRO/1000 Network Connection
> [ 3.955642] eth1: MAC: 4, PHY: 6, PBA No: ffffff-0ff
> [ 5.358191] netconsole: interface eth1
please disable netconsole on this device. You should not run netconsole over a
critical (and busy) line like this.
> [ 6.432345] eth1: Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
>
> MegaRouter-KARAM ~ # ethtool eth1
> Settings for eth1:
> Supported ports: [ TP ]
> Supported link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Full
> Supports auto-negotiation: Yes
> Advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Full
> Advertised auto-negotiation: Yes
> Speed: 1000Mb/s
> Duplex: Full
> Port: Twisted Pair
> PHYAD: 1
> Transceiver: internal
> Auto-negotiation: on
> Supports Wake-on: pumbag
> Wake-on: g
> Current message level: 0x00000001 (1)
> Link detected: yes
>
>
> MegaRouter-KARAM ~ # ethtool -c eth1
> Coalesce parameters for eth1:
> Adaptive RX: off TX: off
> stats-block-usecs: 0
> sample-interval: 0
> pkt-rate-low: 0
> pkt-rate-high: 0
>
> rx-usecs: 3
>
> I tried to change Intel's "magic" rx-usecs to 0,1,3,10000,100000, it doesn't
> help at all. Attempt to change offloading parameters (disabling) doesn't make
> sense too.
>
>
> On Mon, 16 Jun 2008 21:48:55 +0200, Eric Dumazet wrote
>> Denys Fedoryshchenko a e'crit :
>>> Hi again
>>>
>>> I moved from PCI Realtek 8169 to onboard PCI-Express e1000e, and situation
>>> become worse. Strange? Yes.
>>>
>>> Here is info
>>> Load and error rate:
>>>
>>> MegaRouter-KARAM /sys # ifconfig eth1;sleep 10;ifconfig eth1
>>> eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
>>> inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>>> RX packets:91991642 errors:0 dropped:1803444 overruns:0 frame:0
>>> TX packets:91914611 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:1000
>>> RX bytes:316112765 (301.4 MiB) TX bytes:138303107 (131.8 MiB)
>>> Memory:90300000-90320000
>>>
>>> eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
>>> inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>>> RX packets:92812514 errors:0 dropped:1815490 overruns:0 frame:0
>>> TX packets:92734865 errors:0 dropped:0 overruns:0 carrier:0
>>> collisions:0 txqueuelen:1000
>>> RX bytes:739941322 (705.6 MiB) TX bytes:560547138 (534.5 MiB)
>>> Memory:90300000-90320000
>>>
>>> System load (there is no NAT, no shapers, just routing, around 600-700
> routes):
>>> MegaRouter-KARAM /sys # mpstat 1
>>> Linux 2.6.26-rc6-git2-build-0029 (MegaRouter-KARAM) 06/16/08
>>>
>>> 22:36:36 CPU %user %nice %sys %iowait %irq %soft %steal
>>> %idle intr/s
>>> 22:36:37 all 0.00 0.00 0.00 0.00 0.00 1.49 0.00
>>> 98.51 19764.00
>>> 22:36:38 all 0.00 0.00 0.50 0.00 0.00 0.50 0.00
>>> 99.01 19888.00
>>> 22:36:39 all 0.00 0.00 0.00 0.00 0.00 0.50 0.00
>>> 99.50 19642.00
>>> 22:36:40 all 0.00 0.00 0.00 0.00 1.01 9.05 0.00
>>> 89.95 19543.00
>>>
>>> (system almost idle)
>>>
>>> MegaRouter-KARAM /sys # ethtool -S eth1
>>> NIC statistics:
>>> rx_packets: 109977509
>>> tx_packets: 109887692
>>> rx_bytes: 57656749138
>>> tx_bytes: 57536071746
>>> rx_broadcast: 6497
>>> tx_broadcast: 92
>>> rx_multicast: 48995
>>> tx_multicast: 1960
>>> rx_errors: 0
>>> tx_errors: 0
>>> tx_dropped: 0
>>> multicast: 48995
>>> collisions: 0
>>> rx_length_errors: 0
>>> rx_over_errors: 0
>>> rx_crc_errors: 0
>>> rx_frame_errors: 0
>>> rx_no_buffer_count: 1796
>>> rx_missed_errors: 2182679
>>> tx_aborted_errors: 0
>>> tx_carrier_errors: 0
>>> tx_fifo_errors: 0
>>> tx_heartbeat_errors: 0
>>> tx_window_errors: 0
>>> tx_abort_late_coll: 0
>>> tx_deferred_ok: 55617
>>> tx_single_coll_ok: 0
>>> tx_multi_coll_ok: 0
>>> tx_timeout_count: 0
>>> tx_restart_queue: 1626
>>> rx_long_length_errors: 0
>>> rx_short_length_errors: 0
>>> rx_align_errors: 0
>>> tx_tcp_seg_good: 0
>>> tx_tcp_seg_failed: 0
>>> rx_flow_control_xon: 55461
>>> rx_flow_control_xoff: 57329
>>> tx_flow_control_xon: 39114
>>> tx_flow_control_xoff: 48341
>>> rx_long_byte_count: 57656749138
>>> rx_csum_offload_good: 104097306
>>> rx_csum_offload_errors: 2209
>>> rx_header_split: 0
>>> alloc_rx_buff_failed: 0
>>> tx_smbus: 22
>>> rx_smbus: 2201
>>> dropped_smbus: 0
>>> rx_dma_failed: 0
>>> tx_dma_failed: 0
>>> MegaRouter-KARAM /sys # ethtool -S eth1
>>> NIC statistics:
>>> rx_packets: 110154571
>>> tx_packets: 110064459
>>> rx_bytes: 57753358981
>>> tx_bytes: 57632419451
>>> rx_broadcast: 6501
>>> tx_broadcast: 92
>>> rx_multicast: 49155
>>> tx_multicast: 1960
>>> rx_errors: 0
>>> tx_errors: 0
>>> tx_dropped: 0
>>> multicast: 49155
>>> collisions: 0
>>> rx_length_errors: 0
>>> rx_over_errors: 0
>>> rx_crc_errors: 0
>>> rx_frame_errors: 0
>>> rx_no_buffer_count: 1796
>>> rx_missed_errors: 2187703
>>> tx_aborted_errors: 0
>>> tx_carrier_errors: 0
>>> tx_fifo_errors: 0
>>> tx_heartbeat_errors: 0
>>> tx_window_errors: 0
>>> tx_abort_late_coll: 0
>>> tx_deferred_ok: 55772
>>> tx_single_coll_ok: 0
>>> tx_multi_coll_ok: 0
>>> tx_timeout_count: 0
>>> tx_restart_queue: 1628
>>> rx_long_length_errors: 0
>>> rx_short_length_errors: 0
>>> rx_align_errors: 0
>>> tx_tcp_seg_good: 0
>>> tx_tcp_seg_failed: 0
>>> rx_flow_control_xon: 55615
>>> rx_flow_control_xoff: 57488
>>> tx_flow_control_xon: 39209
>>> tx_flow_control_xoff: 48448
>>> rx_long_byte_count: 57753358981
>>> rx_csum_offload_good: 104264656
>>> rx_csum_offload_errors: 2210
>>> rx_header_split: 0
>>> alloc_rx_buff_failed: 0
>>> tx_smbus: 23
>>> rx_smbus: 2207
>>> dropped_smbus: 0
>>> rx_dma_failed: 0
>>> tx_dma_failed: 0
>>>
>>>
>>>
>> What RX & TX rings look like on e1000e ?
>>
>> ethtool -g eth1
>>
>> Also, please post "cat /proc/interrupts"
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
> --
> Denys Fedoryshchenko
> Technical Manager
> Virtual ISP S.A.L.
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 33+ messages in thread
* RE: packetloss, on e1000e worse than r8169?
2008-06-16 19:37 packetloss, on e1000e worse than r8169? Denys Fedoryshchenko
` (2 preceding siblings ...)
2008-06-16 20:26 ` Francois Romieu
@ 2008-06-16 20:37 ` Waskiewicz Jr, Peter P
2008-06-16 20:42 ` Denys Fedoryshchenko
3 siblings, 1 reply; 33+ messages in thread
From: Waskiewicz Jr, Peter P @ 2008-06-16 20:37 UTC (permalink / raw)
To: Denys Fedoryshchenko, netdev; +Cc: Linux NICS
>MegaRouter-KARAM /sys # ethtool -S eth1
>NIC statistics:
> rx_packets: 109977509
> tx_packets: 109887692
> rx_bytes: 57656749138
> tx_bytes: 57536071746
> rx_broadcast: 6497
> tx_broadcast: 92
> rx_multicast: 48995
> tx_multicast: 1960
> rx_errors: 0
> tx_errors: 0
> tx_dropped: 0
> multicast: 48995
> collisions: 0
> rx_length_errors: 0
> rx_over_errors: 0
> rx_crc_errors: 0
> rx_frame_errors: 0
> rx_no_buffer_count: 1796
> rx_missed_errors: 2182679
This is an indication here that your host isn't processing your Rx fast
enough, and your Rx ring is out of descriptors. Hence, your hardware is
needing to drop the packet. What's disturbing is that you actually do
have flow control packets being processed, so the NIC is trying to help
the host.
> tx_aborted_errors: 0
> tx_carrier_errors: 0
> tx_fifo_errors: 0
> tx_heartbeat_errors: 0
> tx_window_errors: 0
> tx_abort_late_coll: 0
> tx_deferred_ok: 55617
> tx_single_coll_ok: 0
> tx_multi_coll_ok: 0
> tx_timeout_count: 0
> tx_restart_queue: 1626
> rx_long_length_errors: 0
> rx_short_length_errors: 0
> rx_align_errors: 0
> tx_tcp_seg_good: 0
> tx_tcp_seg_failed: 0
> rx_flow_control_xon: 55461
> rx_flow_control_xoff: 57329
> tx_flow_control_xon: 39114
> tx_flow_control_xoff: 48341
> rx_long_byte_count: 57656749138
> rx_csum_offload_good: 104097306
> rx_csum_offload_errors: 2209
This is also a bit disturbing, that Rx CSUM offload is running into
issues. I think though this is due to the rx_no_buffer_count.
I see in a followup email you tried increasing your ring size to 4096
descriptors. I'd suggest trying 512 descriptors; something slow,
instead of going for 4096 out of the gate. However, if your host can't
keep up with 256 descriptors, I think you're just going to prolong your
problem by increasing your descriptor ring size. But I don't know what
the profile of your traffic is, so perhaps bumping up the descriptor
ring size to 512 or even 1024 descriptors might help.
Cheers,
-PJ Waskiewicz
<peter.p.waskiewicz.jr@intel.com>
^ permalink raw reply [flat|nested] 33+ messages in thread
* RE: packetloss, on e1000e worse than r8169?
2008-06-16 20:37 ` Waskiewicz Jr, Peter P
@ 2008-06-16 20:42 ` Denys Fedoryshchenko
0 siblings, 0 replies; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 20:42 UTC (permalink / raw)
To: Waskiewicz Jr, Peter P, netdev
On Mon, 16 Jun 2008 13:37:06 -0700, Waskiewicz Jr, Peter P wrote
> >MegaRouter-KARAM /sys # ethtool -S eth1
> >NIC statistics:
> > rx_packets: 109977509
> > tx_packets: 109887692
> > rx_bytes: 57656749138
> > tx_bytes: 57536071746
> > rx_broadcast: 6497
> > tx_broadcast: 92
> > rx_multicast: 48995
> > tx_multicast: 1960
> > rx_errors: 0
> > tx_errors: 0
> > tx_dropped: 0
> > multicast: 48995
> > collisions: 0
> > rx_length_errors: 0
> > rx_over_errors: 0
> > rx_crc_errors: 0
> > rx_frame_errors: 0
> > rx_no_buffer_count: 1796
> > rx_missed_errors: 2182679
>
> This is an indication here that your host isn't processing your Rx fast
> enough, and your Rx ring is out of descriptors. Hence, your
> hardware is needing to drop the packet. What's disturbing is that
> you actually do have flow control packets being processed, so the
> NIC is trying to help the host.
>
> > tx_aborted_errors: 0
> > tx_carrier_errors: 0
> > tx_fifo_errors: 0
> > tx_heartbeat_errors: 0
> > tx_window_errors: 0
> > tx_abort_late_coll: 0
> > tx_deferred_ok: 55617
> > tx_single_coll_ok: 0
> > tx_multi_coll_ok: 0
> > tx_timeout_count: 0
> > tx_restart_queue: 1626
> > rx_long_length_errors: 0
> > rx_short_length_errors: 0
> > rx_align_errors: 0
> > tx_tcp_seg_good: 0
> > tx_tcp_seg_failed: 0
> > rx_flow_control_xon: 55461
> > rx_flow_control_xoff: 57329
> > tx_flow_control_xon: 39114
> > tx_flow_control_xoff: 48341
> > rx_long_byte_count: 57656749138
> > rx_csum_offload_good: 104097306
> > rx_csum_offload_errors: 2209
>
> This is also a bit disturbing, that Rx CSUM offload is running into
> issues. I think though this is due to the rx_no_buffer_count.
>
> I see in a followup email you tried increasing your ring size to 4096
> descriptors. I'd suggest trying 512 descriptors; something slow,
> instead of going for 4096 out of the gate. However, if your host can't
> keep up with 256 descriptors, I think you're just going to prolong your
> problem by increasing your descriptor ring size. But I don't know what
> the profile of your traffic is, so perhaps bumping up the descriptor
> ring size to 512 or even 1024 descriptors might help.
>
If i am not wrong when situation is related to ring, i will have large amount
of errors in rx_no_buffer_count. I tried now 512 and 1024, it doesn't change
anything at all.
MegaRouter-KARAM ~ # ethtool -g eth1
Ring parameters for eth1:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 1024
RX Mini: 0
RX Jumbo: 0
TX: 256
MegaRouter-KARAM ~ # ifconfig eth1; sleep 10;ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:105760686 errors:0 dropped:1728264 overruns:0 frame:0
TX packets:105667743 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4291052222 (3.9 GiB) TX bytes:4081974720 (3.8 GiB)
Memory:90300000-90320000
eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:106541093 errors:0 dropped:1744393 overruns:0 frame:0
TX packets:106447803 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:413277601 (394.1 MiB) TX bytes:202824806 (193.4 MiB)
Memory:90300000-90320000
rx_no_buffer_count is not a big deal, i had this issue on Sun Fire (e1000 over
PCI-X 66 Mhz), and increasing ring solved the problem. But this case seems
different. My headache now is rx_missed_errors. It can be bus bandwidth hog
also, as i read in maillists, but it's supposed to be x1 PCI-Express with 2.5
GB/s throughput!
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
[not found] ` <20080616204411.M52834@visp.net.lb>
@ 2008-06-16 20:49 ` Kok, Auke
2008-06-16 20:59 ` Denys Fedoryshchenko
` (2 more replies)
2008-06-16 21:05 ` Eric Dumazet
1 sibling, 3 replies; 33+ messages in thread
From: Kok, Auke @ 2008-06-16 20:49 UTC (permalink / raw)
To: Denys Fedoryshchenko; +Cc: Eric Dumazet, netdev, linux.nics
Denys Fedoryshchenko wrote:
> [ 5.227679] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k2
> [ 5.227681] e1000e: Copyright (c) 1999-2008 Intel Corporation.
> [ 5.227724] e1000e 0000:00:19.0: enabling device (0000 -> 0003)
> [ 5.227731] ACPI: PCI Interrupt 0000:00:19.0[A] -> GSI 20 (level, low) ->
> IRQ 20
> [ 5.227742] PCI: Setting latency timer of device 0000:00:19.0 to 64
> [ 5.424362] eth1: (PCI Express:2.5GB/s:Width x1) 00:19:d1:71:5f:33
> [ 5.424364] eth1: Intel(R) PRO/1000 Network Connection
> [ 5.424385] eth1: MAC: 4, PHY: 6, PBA No: ffffff-0ff
> [ 5.985995] warning: `zebra' uses 32-bit capabilities (legacy support in use)
> [ 6.337111] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
disable (all of) netfilter if you don't need it. DEFINATELY do disable
nf_conntrack as it's known problematic for routing performance.
Auke
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 20:36 ` Kok, Auke
@ 2008-06-16 20:52 ` Denys Fedoryshchenko
[not found] ` <20080616204411.M52834@visp.net.lb>
1 sibling, 0 replies; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 20:52 UTC (permalink / raw)
To: netdev
On Mon, 16 Jun 2008 13:36:44 -0700, Kok, Auke wrote
> Denys Fedoryshchenko wrote:
> > Also if important
> > MegaRouter-KARAM ~ # dmesg|grep eth1
> > [ 3.955620] eth1: (PCI Express:2.5GB/s:Width x1) 00:19:d1:71:5f:33
> > [ 3.955622] eth1: Intel(R) PRO/1000 Network Connection
> > [ 3.955642] eth1: MAC: 4, PHY: 6, PBA No: ffffff-0ff
> > [ 5.358191] netconsole: interface eth1
>
> please disable netconsole on this device. You should not run
> netconsole over a critical (and busy) line like this.
>
> > [ 6.432345] eth1: Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
> >
> > MegaRouter-KARAM ~ # ethtool eth1
> > Settings for eth1:
> > Supported ports: [ TP ]
> > Supported link modes: 10baseT/Half 10baseT/Full
> > 100baseT/Half 100baseT/Full
> > 1000baseT/Full
> > Supports auto-negotiation: Yes
> > Advertised link modes: 10baseT/Half 10baseT/Full
> > 100baseT/Half 100baseT/Full
> > 1000baseT/Full
> > Advertised auto-negotiation: Yes
> > Speed: 1000Mb/s
> > Duplex: Full
> > Port: Twisted Pair
> > PHYAD: 1
> > Transceiver: internal
> > Auto-negotiation: on
> > Supports Wake-on: pumbag
> > Wake-on: g
> > Current message level: 0x00000001 (1)
> > Link detected: yes
> >
> >
> > MegaRouter-KARAM ~ # ethtool -c eth1
> > Coalesce parameters for eth1:
> > Adaptive RX: off TX: off
> > stats-block-usecs: 0
> > sample-interval: 0
> > pkt-rate-low: 0
> > pkt-rate-high: 0
> >
> > rx-usecs: 3
> >
> > I tried to change Intel's "magic" rx-usecs to 0,1,3,10000,100000, it doesn't
> > help at all. Attempt to change offloading parameters (disabling) doesn't make
> > sense too.
> >
> >
> > On Mon, 16 Jun 2008 21:48:55 +0200, Eric Dumazet wrote
> >> Denys Fedoryshchenko a e'crit :
> >>> Hi again
> >>>
> >>> I moved from PCI Realtek 8169 to onboard PCI-Express e1000e, and situation
> >>> become worse. Strange? Yes.
> >>>
> >>> Here is info
> >>> Load and error rate:
> >>>
> >>> MegaRouter-KARAM /sys # ifconfig eth1;sleep 10;ifconfig eth1
> >>> eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
> >>> inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
> >>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >>> RX packets:91991642 errors:0 dropped:1803444 overruns:0 frame:0
> >>> TX packets:91914611 errors:0 dropped:0 overruns:0 carrier:0
> >>> collisions:0 txqueuelen:1000
> >>> RX bytes:316112765 (301.4 MiB) TX bytes:138303107 (131.8 MiB)
> >>> Memory:90300000-90320000
> >>>
> >>> eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
> >>> inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
> >>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> >>> RX packets:92812514 errors:0 dropped:1815490 overruns:0 frame:0
> >>> TX packets:92734865 errors:0 dropped:0 overruns:0 carrier:0
> >>> collisions:0 txqueuelen:1000
> >>> RX bytes:739941322 (705.6 MiB) TX bytes:560547138 (534.5 MiB)
> >>> Memory:90300000-90320000
> >>>
> >>> System load (there is no NAT, no shapers, just routing, around 600-700
> > routes):
> >>> MegaRouter-KARAM /sys # mpstat 1
> >>> Linux 2.6.26-rc6-git2-build-0029 (MegaRouter-KARAM) 06/16/08
> >>>
> >>> 22:36:36 CPU %user %nice %sys %iowait %irq %soft %steal
> >>> %idle intr/s
> >>> 22:36:37 all 0.00 0.00 0.00 0.00 0.00 1.49 0.00
> >>> 98.51 19764.00
> >>> 22:36:38 all 0.00 0.00 0.50 0.00 0.00 0.50 0.00
> >>> 99.01 19888.00
> >>> 22:36:39 all 0.00 0.00 0.00 0.00 0.00 0.50 0.00
> >>> 99.50 19642.00
> >>> 22:36:40 all 0.00 0.00 0.00 0.00 1.01 9.05 0.00
> >>> 89.95 19543.00
> >>>
> >>> (system almost idle)
> >>>
> >>> MegaRouter-KARAM /sys # ethtool -S eth1
> >>> NIC statistics:
> >>> rx_packets: 109977509
> >>> tx_packets: 109887692
> >>> rx_bytes: 57656749138
> >>> tx_bytes: 57536071746
> >>> rx_broadcast: 6497
> >>> tx_broadcast: 92
> >>> rx_multicast: 48995
> >>> tx_multicast: 1960
> >>> rx_errors: 0
> >>> tx_errors: 0
> >>> tx_dropped: 0
> >>> multicast: 48995
> >>> collisions: 0
> >>> rx_length_errors: 0
> >>> rx_over_errors: 0
> >>> rx_crc_errors: 0
> >>> rx_frame_errors: 0
> >>> rx_no_buffer_count: 1796
> >>> rx_missed_errors: 2182679
> >>> tx_aborted_errors: 0
> >>> tx_carrier_errors: 0
> >>> tx_fifo_errors: 0
> >>> tx_heartbeat_errors: 0
> >>> tx_window_errors: 0
> >>> tx_abort_late_coll: 0
> >>> tx_deferred_ok: 55617
> >>> tx_single_coll_ok: 0
> >>> tx_multi_coll_ok: 0
> >>> tx_timeout_count: 0
> >>> tx_restart_queue: 1626
> >>> rx_long_length_errors: 0
> >>> rx_short_length_errors: 0
> >>> rx_align_errors: 0
> >>> tx_tcp_seg_good: 0
> >>> tx_tcp_seg_failed: 0
> >>> rx_flow_control_xon: 55461
> >>> rx_flow_control_xoff: 57329
> >>> tx_flow_control_xon: 39114
> >>> tx_flow_control_xoff: 48341
> >>> rx_long_byte_count: 57656749138
> >>> rx_csum_offload_good: 104097306
> >>> rx_csum_offload_errors: 2209
> >>> rx_header_split: 0
> >>> alloc_rx_buff_failed: 0
> >>> tx_smbus: 22
> >>> rx_smbus: 2201
> >>> dropped_smbus: 0
> >>> rx_dma_failed: 0
> >>> tx_dma_failed: 0
> >>> MegaRouter-KARAM /sys # ethtool -S eth1
> >>> NIC statistics:
> >>> rx_packets: 110154571
> >>> tx_packets: 110064459
> >>> rx_bytes: 57753358981
> >>> tx_bytes: 57632419451
> >>> rx_broadcast: 6501
> >>> tx_broadcast: 92
> >>> rx_multicast: 49155
> >>> tx_multicast: 1960
> >>> rx_errors: 0
> >>> tx_errors: 0
> >>> tx_dropped: 0
> >>> multicast: 49155
> >>> collisions: 0
> >>> rx_length_errors: 0
> >>> rx_over_errors: 0
> >>> rx_crc_errors: 0
> >>> rx_frame_errors: 0
> >>> rx_no_buffer_count: 1796
> >>> rx_missed_errors: 2187703
> >>> tx_aborted_errors: 0
> >>> tx_carrier_errors: 0
> >>> tx_fifo_errors: 0
> >>> tx_heartbeat_errors: 0
> >>> tx_window_errors: 0
> >>> tx_abort_late_coll: 0
> >>> tx_deferred_ok: 55772
> >>> tx_single_coll_ok: 0
> >>> tx_multi_coll_ok: 0
> >>> tx_timeout_count: 0
> >>> tx_restart_queue: 1628
> >>> rx_long_length_errors: 0
> >>> rx_short_length_errors: 0
> >>> rx_align_errors: 0
> >>> tx_tcp_seg_good: 0
> >>> tx_tcp_seg_failed: 0
> >>> rx_flow_control_xon: 55615
> >>> rx_flow_control_xoff: 57488
> >>> tx_flow_control_xon: 39209
> >>> tx_flow_control_xoff: 48448
> >>> rx_long_byte_count: 57753358981
> >>> rx_csum_offload_good: 104264656
> >>> rx_csum_offload_errors: 2210
> >>> rx_header_split: 0
> >>> alloc_rx_buff_failed: 0
> >>> tx_smbus: 23
> >>> rx_smbus: 2207
> >>> dropped_smbus: 0
> >>> rx_dma_failed: 0
> >>> tx_dma_failed: 0
> >>>
> >>>
> >>>
> >> What RX & TX rings look like on e1000e ?
> >>
> >> ethtool -g eth1
> >>
> >> Also, please post "cat /proc/interrupts"
> >>
Disabled and rebooted. It changes nothing. (By the way it is first thing i tried).
MegaRouter-KARAM ~ # ifconfig eth1;sleep 10;ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4507198 errors:0 dropped:12335 overruns:0 frame:0
TX packets:4494262 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1431300040 (1.3 GiB) TX bytes:1421772965 (1.3 GiB)
Memory:90300000-90320000
eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:5546385 errors:0 dropped:20298 overruns:0 frame:0
TX packets:5531599 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1849199364 (1.7 GiB) TX bytes:1837492827 (1.7 GiB)
Memory:90300000-90320000
Here is full dmesg, as proof, that it is off
] PCI: Setting latency timer of device 0000:00:1c.0 to 64
[ 0.235558] ACPI: PCI Interrupt 0000:00:1c.1[B] -> GSI 16 (level, low) ->
IRQ 16
[ 0.235562] PCI: Setting latency timer of device 0000:00:1c.1 to 64
[ 0.235576] ACPI: PCI Interrupt 0000:00:1c.2[C] -> GSI 18 (level, low) ->
IRQ 18
[ 0.235580] PCI: Setting latency timer of device 0000:00:1c.2 to 64
[ 0.235593] ACPI: PCI Interrupt 0000:00:1c.3[D] -> GSI 19 (level, low) ->
IRQ 19
[ 0.235597] PCI: Setting latency timer of device 0000:00:1c.3 to 64
[ 0.235611] ACPI: PCI Interrupt 0000:00:1c.4[A] -> GSI 17 (level, low) ->
IRQ 17
[ 0.235615] PCI: Setting latency timer of device 0000:00:1c.4 to 64
[ 0.235623] PCI: Setting latency timer of device 0000:00:1e.0 to 64
[ 0.235631] NET: Registered protocol family 2
[ 0.241017] IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
[ 0.241231] TCP established hash table entries: 131072 (order: 8, 1048576
bytes)
[ 0.241532] TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.241671] TCP: Hash tables configured (established 131072 bind 65536)
[ 0.241673] TCP reno registered
[ 0.242025] NET: Registered protocol family 1
[ 0.323486] Machine check exception polling timer started.
[ 0.323843] highmem bounce pool size: 64 pages
[ 0.325672] JFFS2 version 2.2. (SUMMARY) 2001-2006 Red Hat, Inc.
[ 0.325828] msgmni has been set to 1741
[ 0.325962] Block layer SCSI generic (bsg) driver version 0.4 loaded (major
254)
[ 0.325962] io scheduler noop registered
[ 0.325962] io scheduler cfq registered (default)
[ 0.325962] pci 0000:00:02.0: Boot video device
[ 0.326137] PCI: Setting latency timer of device 0000:00:01.0 to 64
[ 0.326161] assign_interrupt_mode Found MSI capability
[ 0.326181] Allocate Port Service[0000:00:01.0:pcie00]
[ 0.326212] Allocate Port Service[0000:00:01.0:pcie03]
[ 0.326272] PCI: Setting latency timer of device 0000:00:1c.0 to 64
[ 0.326299] assign_interrupt_mode Found MSI capability
[ 0.326325] Allocate Port Service[0000:00:1c.0:pcie00]
[ 0.326354] Allocate Port Service[0000:00:1c.0:pcie02]
[ 0.326381] Allocate Port Service[0000:00:1c.0:pcie03]
[ 0.326447] PCI: Setting latency timer of device 0000:00:1c.1 to 64
[ 0.326473] assign_interrupt_mode Found MSI capability
[ 0.326499] Allocate Port Service[0000:00:1c.1:pcie00]
[ 0.326528] Allocate Port Service[0000:00:1c.1:pcie02]
[ 0.326558] Allocate Port Service[0000:00:1c.1:pcie03]
[ 0.326624] PCI: Setting latency timer of device 0000:00:1c.2 to 64
[ 0.326650] assign_interrupt_mode Found MSI capability
[ 0.326676] Allocate Port Service[0000:00:1c.2:pcie00]
[ 0.326705] Allocate Port Service[0000:00:1c.2:pcie02]
[ 0.326734] Allocate Port Service[0000:00:1c.2:pcie03]
[ 0.326801] PCI: Setting latency timer of device 0000:00:1c.3 to 64
[ 0.326827] assign_interrupt_mode Found MSI capability
[ 0.326852] Allocate Port Service[0000:00:1c.3:pcie00]
[ 0.326881] Allocate Port Service[0000:00:1c.3:pcie02]
[ 0.326908] Allocate Port Service[0000:00:1c.3:pcie03]
[ 0.326963] PCI: Setting latency timer of device 0000:00:1c.4 to 64
[ 0.326963] assign_interrupt_mode Found MSI capability
[ 0.326992] Allocate Port Service[0000:00:1c.4:pcie00]
[ 0.327021] Allocate Port Service[0000:00:1c.4:pcie02]
[ 0.327048] Allocate Port Service[0000:00:1c.4:pcie03]
[ 0.341961] hpet_resources: 0xfed00000 is busy
[ 0.342028] Driver 'sd' needs updating - please use bus_type methods
[ 0.342159] PNP: No PS/2 controller found. Probing ports directly.
[ 0.345015] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.345019] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.345081] mice: PS/2 mouse device common for all mice
[ 0.345164] input: PC Speaker as /class/input/input0
[ 0.499965] Clockevents: could not switch to one-shot mode: lapic is not
functional.
[ 0.499969] Could not switch to high resolution mode on CPU 1
[ 0.500964] Clockevents: could not switch to one-shot mode: lapic is not
functional.
[ 0.500968] Could not switch to high resolution mode on CPU 0
[ 0.510859] device-mapper: uevent: version 1.0.3
[ 0.510949] device-mapper: ioctl: 4.13.0-ioctl (2007-10-18) initialised:
dm-devel@redhat.com
[ 0.510949] cpuidle: using governor ladder
[ 0.510949] cpuidle: using governor menu
[ 0.511209] TCP cubic registered
[ 0.511211] NET: Registered protocol family 17
[ 0.511220] Using IPI Shortcut mode
[ 0.513051] Freeing unused kernel memory: 5508k freed
[ 0.860439] usbcore: registered new interface driver usbfs
[ 0.860488] usbcore: registered new interface driver hub
[ 0.860534] usbcore: registered new device driver usb
[ 0.889476] ohci_hcd: 2006 August 04 USB 1.1 'Open' Host Controller (OHCI)
Driver
[ 0.918682] ACPI: PCI Interrupt 0000:00:1a.7[C] -> GSI 18 (level, low) ->
IRQ 18
[ 0.918694] PCI: Setting latency timer of device 0000:00:1a.7 to 64
[ 0.918697] ehci_hcd 0000:00:1a.7: EHCI Host Controller
[ 0.918876] ehci_hcd 0000:00:1a.7: new USB bus registered, assigned bus
number 1
[ 0.922777] ehci_hcd 0000:00:1a.7: debug port 1
[ 0.922783] PCI: cache line size of 32 is not supported by device 0000:00:1a.7
[ 0.922792] ehci_hcd 0000:00:1a.7: irq 18, io mem 0x90326c00
[ 0.931942] ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00, driver 10
Dec 2004
[ 0.932035] usb usb1: configuration #1 chosen from 1 choice
[ 0.932089] hub 1-0:1.0: USB hub found
[ 0.932096] hub 1-0:1.0: 4 ports detected
[ 1.033074] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[ 1.033078] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 1.033082] usb usb1: Product: EHCI Host Controller
[ 1.033084] usb usb1: Manufacturer: Linux 2.6.26-rc6-git2-build-0029 ehci_hcd
[ 1.033086] usb usb1: SerialNumber: 0000:00:1a.7
[ 1.033113] ACPI: PCI Interrupt 0000:00:1d.7[A] -> GSI 23 (level, low) ->
IRQ 23
[ 1.033122] PCI: Setting latency timer of device 0000:00:1d.7 to 64
[ 1.033126] ehci_hcd 0000:00:1d.7: EHCI Host Controller
[ 1.033177] ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus
number 2
[ 1.037079] ehci_hcd 0000:00:1d.7: debug port 1
[ 1.037085] PCI: cache line size of 32 is not supported by device 0000:00:1d.7
[ 1.037091] ehci_hcd 0000:00:1d.7: irq 23, io mem 0x90326800
[ 1.046922] ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10
Dec 2004
[ 1.047013] usb usb2: configuration #1 chosen from 1 choice
[ 1.047064] hub 2-0:1.0: USB hub found
[ 1.047071] hub 2-0:1.0: 6 ports detected
[ 1.148018] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
[ 1.148021] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 1.148025] usb usb2: Product: EHCI Host Controller
[ 1.148028] usb usb2: Manufacturer: Linux 2.6.26-rc6-git2-build-0029 ehci_hcd
[ 1.148030] usb usb2: SerialNumber: 0000:00:1d.7
[ 1.178131] USB Universal Host Controller Interface driver v3.0
[ 1.178162] ACPI: PCI Interrupt 0000:00:1a.0[A] -> GSI 16 (level, low) ->
IRQ 16
[ 1.178169] PCI: Setting latency timer of device 0000:00:1a.0 to 64
[ 1.178172] uhci_hcd 0000:00:1a.0: UHCI Host Controller
[ 1.178211] uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus
number 3
[ 1.178235] uhci_hcd 0000:00:1a.0: irq 16, io base 0x000030c0
[ 1.178309] usb usb3: configuration #1 chosen from 1 choice
[ 1.178347] hub 3-0:1.0: USB hub found
[ 1.178353] hub 3-0:1.0: 2 ports detected
[ 1.279009] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
[ 1.279013] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 1.279017] usb usb3: Product: UHCI Host Controller
[ 1.279020] usb usb3: Manufacturer: Linux 2.6.26-rc6-git2-build-0029 uhci_hcd
[ 1.279023] usb usb3: SerialNumber: 0000:00:1a.0
[ 1.279042] ACPI: PCI Interrupt 0000:00:1a.1[B] -> GSI 21 (level, low) ->
IRQ 21
[ 1.279050] PCI: Setting latency timer of device 0000:00:1a.1 to 64
[ 1.279054] uhci_hcd 0000:00:1a.1: UHCI Host Controller
[ 1.279101] uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus
number 4
[ 1.279127] uhci_hcd 0000:00:1a.1: irq 21, io base 0x000030a0
[ 1.279212] usb usb4: configuration #1 chosen from 1 choice
[ 1.279253] hub 4-0:1.0: USB hub found
[ 1.279257] hub 4-0:1.0: 2 ports detected
[ 1.353906] usb 2-4: new high speed USB device using ehci_hcd and address 2
[ 1.380002] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
[ 1.380006] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 1.380010] usb usb4: Product: UHCI Host Controller
[ 1.380013] usb usb4: Manufacturer: Linux 2.6.26-rc6-git2-build-0029 uhci_hcd
[ 1.380016] usb usb4: SerialNumber: 0000:00:1a.1
[ 1.380039] ACPI: PCI Interrupt 0000:00:1d.0[A] -> GSI 23 (level, low) ->
IRQ 23
[ 1.380047] PCI: Setting latency timer of device 0000:00:1d.0 to 64
[ 1.380051] uhci_hcd 0000:00:1d.0: UHCI Host Controller
[ 1.380099] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus
number 5
[ 1.380121] uhci_hcd 0000:00:1d.0: irq 23, io base 0x00003080
[ 1.380205] usb usb5: configuration #1 chosen from 1 choice
[ 1.380244] hub 5-0:1.0: USB hub found
[ 1.380249] hub 5-0:1.0: 2 ports detected
[ 1.470314] usb 2-4: configuration #1 chosen from 1 choice
[ 1.470643] usb 2-4: New USB device found, idVendor=0951, idProduct=1603
[ 1.470645] usb 2-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1.470647] usb 2-4: Product: DataTraveler 2.0
[ 1.470649] usb 2-4: Manufacturer: Kingston
[ 1.470650] usb 2-4: SerialNumber: 8990000000000000000000FB
[ 1.481001] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001
[ 1.481004] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 1.481008] usb usb5: Product: UHCI Host Controller
[ 1.481011] usb usb5: Manufacturer: Linux 2.6.26-rc6-git2-build-0029 uhci_hcd
[ 1.481015] usb usb5: SerialNumber: 0000:00:1d.0
[ 1.481033] ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 19 (level, low) ->
IRQ 19
[ 1.481041] PCI: Setting latency timer of device 0000:00:1d.1 to 64
[ 1.481044] uhci_hcd 0000:00:1d.1: UHCI Host Controller
[ 1.481094] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus
number 6
[ 1.481120] uhci_hcd 0000:00:1d.1: irq 19, io base 0x00003060
[ 1.481204] usb usb6: configuration #1 chosen from 1 choice
[ 1.481242] hub 6-0:1.0: USB hub found
[ 1.481248] hub 6-0:1.0: 2 ports detected
[ 1.581993] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001
[ 1.581996] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 1.582000] usb usb6: Product: UHCI Host Controller
[ 1.582003] usb usb6: Manufacturer: Linux 2.6.26-rc6-git2-build-0029 uhci_hcd
[ 1.582007] usb usb6: SerialNumber: 0000:00:1d.1
[ 1.582025] ACPI: PCI Interrupt 0000:00:1d.2[C] -> GSI 18 (level, low) ->
IRQ 18
[ 1.582033] PCI: Setting latency timer of device 0000:00:1d.2 to 64
[ 1.582037] uhci_hcd 0000:00:1d.2: UHCI Host Controller
[ 1.582083] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus
number 7
[ 1.582105] uhci_hcd 0000:00:1d.2: irq 18, io base 0x00003040
[ 1.582189] usb usb7: configuration #1 chosen from 1 choice
[ 1.582226] hub 7-0:1.0: USB hub found
[ 1.582231] hub 7-0:1.0: 2 ports detected
[ 1.682982] usb usb7: New USB device found, idVendor=1d6b, idProduct=0001
[ 1.682985] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 1.682989] usb usb7: Product: UHCI Host Controller
[ 1.682992] usb usb7: Manufacturer: Linux 2.6.26-rc6-git2-build-0029 uhci_hcd
[ 1.682996] usb usb7: SerialNumber: 0000:00:1d.2
[ 1.712358] usbcore: registered new interface driver usbhid
[ 1.712361] usbhid: v2.6:USB HID core driver
[ 1.801687] Initializing USB Mass Storage driver...
[ 1.801739] scsi0 : SCSI emulation for USB Mass Storage devices
[ 1.801784] usb-storage: device found at 2
[ 1.801852] usbcore: registered new interface driver usb-storage
[ 1.801856] USB Mass Storage support registered.
[ 1.802390] scsi 0:0:0:0: Direct-Access Kingston DataTraveler 2.0 1.00
PQ: 0 ANSI: 2
[ 1.830484] sd 0:0:0:0: [sda] 1965056 512-byte hardware sectors (1006 MB)
[ 1.831109] sd 0:0:0:0: [sda] Write Protect is off
[ 1.831111] sd 0:0:0:0: [sda] Mode Sense: 23 00 00 00
[ 1.831113] sd 0:0:0:0: [sda] Assuming drive cache: write through
[ 1.833734] sd 0:0:0:0: [sda] 1965056 512-byte hardware sectors (1006 MB)
[ 1.834360] sd 0:0:0:0: [sda] Write Protect is off
[ 1.834363] sd 0:0:0:0: [sda] Mode Sense: 23 00 00 00
[ 1.834364] sd 0:0:0:0: [sda] Assuming drive cache: write through
[ 1.834433] sda: sda1 sda2
[ 1.835240] sd 0:0:0:0: [sda] Attached SCSI removable disk
[ 1.835440] usb-storage: device scan complete
[ 2.912741] e100: Intel(R) PRO/100 Network Driver, 3.5.23-k4-NAPI
[ 2.912743] e100: Copyright(c) 1999-2006 Intel Corporation
[ 2.942563] Intel(R) PRO/1000 Network Driver - version 7.3.20-k2-NAPI
[ 2.942566] Copyright (c) 1999-2006 Intel Corporation.
[ 2.971978] 8139too Fast Ethernet driver 0.9.28
[ 3.088025] ne2k-pci.c:v1.03 9/22/2003 D. Becker/P. Gortmaker
[ 3.117402] sis900.c: v1.08.10 Apr. 2 2006
[ 3.146609] via-rhine.c:v1.10-LK1.4.3 2007-03-06 Written by Donald Becker
[ 3.235204] r8169 Gigabit Ethernet driver 2.2LK-NAPI loaded
[ 3.235223] ACPI: PCI Interrupt 0000:07:00.0[A] -> GSI 21 (level, low) ->
IRQ 21
[ 3.235828] eth0: RTL8110s at 0xf8894000, 00:18:f8:0b:46:a6, XID 04000000
IRQ 21
[ 3.267764] Linux Tulip driver version 1.1.15-NAPI (Feb 27, 2007)
[ 4.525631] kjournald starting. Commit interval 5 seconds
[ 4.525633] EXT3-fs warning: maximal mount count reached, running e2fsck is
recommended
[ 4.674095] EXT3 FS on dm-0, internal journal
[ 4.674098] EXT3-fs: recovery complete.
[ 4.678221] EXT3-fs: mounted filesystem with ordered data mode.
[ 5.064977] tun: Universal TUN/TAP device driver, 1.6
[ 5.064979] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[ 5.095188] 802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
[ 5.095191] All bugs added by David S. Miller <davem@redhat.com>
[ 5.156387] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 5.188039] IPv4 FIB: Using LC-trie version 0.408
[ 5.227679] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k2
[ 5.227681] e1000e: Copyright (c) 1999-2008 Intel Corporation.
[ 5.227724] e1000e 0000:00:19.0: enabling device (0000 -> 0003)
[ 5.227731] ACPI: PCI Interrupt 0000:00:19.0[A] -> GSI 20 (level, low) ->
IRQ 20
[ 5.227742] PCI: Setting latency timer of device 0000:00:19.0 to 64
[ 5.424362] eth1: (PCI Express:2.5GB/s:Width x1) 00:19:d1:71:5f:33
[ 5.424364] eth1: Intel(R) PRO/1000 Network Connection
[ 5.424385] eth1: MAC: 4, PHY: 6, PBA No: ffffff-0ff
[ 5.985995] warning: `zebra' uses 32-bit capabilities (legacy support in use)
[ 6.337111] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[ 6.580759] ACPI: PCI Interrupt 0000:00:1f.3[B] -> GSI 21 (level, low) ->
IRQ 21
[ 6.613496] coretemp coretemp.0: Using relative temperature scale!
[ 6.614373] coretemp coretemp.1: Using relative temperature scale!
[ 6.793450] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.03 (30-Apr-2008)
[ 6.794349] iTCO_wdt: Found a ICH8DO TCO device (Version=2, TCOBASE=0x0460)
[ 6.794831] iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)
[ 7.874067] eth1: Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 20:49 ` Kok, Auke
@ 2008-06-16 20:59 ` Denys Fedoryshchenko
2008-06-16 21:13 ` Denys Fedoryshchenko
2008-06-16 21:17 ` Denys Fedoryshchenko
2 siblings, 0 replies; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 20:59 UTC (permalink / raw)
To: Kok, Auke; +Cc: Eric Dumazet, netdev, linux.nics
nf_conntrack unloaded later (just it is not giving message about unloading).
MegaRouter-KARAM ~ # lsmod
Module Size Used by Not tainted
iTCO_wdt 10404 0
coretemp 5888 0
hwmon 3100 1 coretemp
i2c_i801 8592 0
i2c_core 17556 1 i2c_i801
e1000e 85924 0
iptable_filter 3456 1
ip_tables 10768 1 iptable_filter
x_tables 12292 1 ip_tables
8021q 17184 0
tun 9092 0
tulip 45216 0
r8169 24708 0
sky2 38660 0
via_velocity 27272 0
via_rhine 19848 0
sis900 18176 0
ne2k_pci 8928 0
8390 8704 1 ne2k_pci
skge 34448 0
tg3 102148 0
8139too 19968 0
e1000 103488 0
e100 30348 0
usb_storage 72896 1
mtdblock 4736 0
mtd_blkdevs 6400 1 mtdblock
usbhid 18340 0
uhci_hcd 19340 0
ehci_hcd 28940 0
ohci_hcd 19204 0
usbcore 112496 6 usb_storage,usbhid,uhci_hcd,ehci_hcd,ohci_hcd
MegaRouter-KARAM ~ # rmmod iptable_filter
MegaRouter-KARAM ~ # rmmod ip_tables
MegaRouter-KARAM ~ # ifconfig eth1;sleep 10;ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:59092374 errors:0 dropped:1223955 overruns:0 frame:0
TX packets:59034276 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1495703425 (1.3 GiB) TX bytes:1378135513 (1.2 GiB)
Memory:90300000-90320000
eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:59840912 errors:0 dropped:1237198 overruns:0 frame:0
TX packets:59782319 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1924123084 (1.7 GiB) TX bytes:1805153608 (1.6 GiB)
Memory:90300000-90320000
even
MegaRouter-KARAM ~ # rmmod i2c_i801 i2c_core
MegaRouter-KARAM ~ # ifconfig eth1;sleep 10;ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:61504491 errors:0 dropped:1277608 overruns:0 frame:0
TX packets:61444301 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2873416996 (2.6 GiB) TX bytes:2751062058 (2.5 GiB)
Memory:90300000-90320000
eth1 Link encap:Ethernet HWaddr 00:19:D1:71:5F:33
inet addr:192.168.20.10 Bcast:0.0.0.0 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:62252150 errors:0 dropped:1297292 overruns:0 frame:0
TX packets:62191551 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3296181383 (3.0 GiB) TX bytes:3172426660 (2.9 GiB)
Memory:90300000-90320000
removing iTCO_wdt and USB modules doesn't help too.
On Mon, 16 Jun 2008 13:49:26 -0700, Kok, Auke wrote
> Denys Fedoryshchenko wrote:
> > [ 5.227679] e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k2
> > [ 5.227681] e1000e: Copyright (c) 1999-2008 Intel Corporation.
> > [ 5.227724] e1000e 0000:00:19.0: enabling device (0000 -> 0003)
> > [ 5.227731] ACPI: PCI Interrupt 0000:00:19.0[A] -> GSI 20 (level, low) ->
> > IRQ 20
> > [ 5.227742] PCI: Setting latency timer of device 0000:00:19.0 to 64
> > [ 5.424362] eth1: (PCI Express:2.5GB/s:Width x1) 00:19:d1:71:5f:33
> > [ 5.424364] eth1: Intel(R) PRO/1000 Network Connection
> > [ 5.424385] eth1: MAC: 4, PHY: 6, PBA No: ffffff-0ff
> > [ 5.985995] warning: `zebra' uses 32-bit capabilities (legacy support
in use)
> > [ 6.337111] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
>
> disable (all of) netfilter if you don't need it. DEFINATELY do
> disable nf_conntrack as it's known problematic for routing performance.
>
> Auke
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
[not found] ` <20080616204411.M52834@visp.net.lb>
2008-06-16 20:49 ` Kok, Auke
@ 2008-06-16 21:05 ` Eric Dumazet
1 sibling, 0 replies; 33+ messages in thread
From: Eric Dumazet @ 2008-06-16 21:05 UTC (permalink / raw)
To: Denys Fedoryshchenko; +Cc: Kok, Auke, netdev, linux.nics
Denys Fedoryshchenko a e'crit :
> [ 0.241017] IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
> [ 6.337111] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
Are you sure nf_conntrack or ip route cache is not killing you ?
Filling 512 or 1024 RX ring on Gigabit link can be very fast, especially
if ip route cache is full.
rtstat -c10 -i1
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 20:49 ` Kok, Auke
2008-06-16 20:59 ` Denys Fedoryshchenko
@ 2008-06-16 21:13 ` Denys Fedoryshchenko
2008-06-16 21:29 ` Eric Dumazet
2008-06-16 21:17 ` Denys Fedoryshchenko
2 siblings, 1 reply; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 21:13 UTC (permalink / raw)
To: Eric Dumazet; +Cc: netdev
>Are you sure nf_conntrack or ip route cache is not killing you ?
>
>Filling 512 or 1024 RX ring on Gigabit link can be very fast, especially
>if ip route cache is full.
>
>rtstat -c10 -i1
conntrack disabled, it is just enabled for second on load and then unloaded.
MegaRouter-KARAM ~ # rtstat -c10 -i1
rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
entries| in_hit|in_slow_|in_slow_|in_no_ro| in_brd|in_marti|in_marti|
out_hit|out_slow|out_slow|gc_total|gc_ignor|gc_goal_|gc_dst_o|in_hlist|out_hlis|
| | tot| mc| ute| | an_dst| an_src|
| _tot| _mc| | ed| miss| verflow| _search|t_search|
63448|229209146|22225578| 13754| 12| 2822| 0| 104|
54647| 27606| 570|21782075|21776859| 74| 0|215222374| 61294|
66085| 141462| 5268| 8| 0| 4| 0| 0|
4| 2| 0| 5274| 5274| 0| 0| 254424| 10|
59947| 132660| 20570| 4| 0| 4| 0| 0|
8| 14| 0| 20584| 20584| 0| 0| 185738| 24|
56995| 132416| 16918| 12| 0| 2| 0| 0|
6| 4| 0| 16932| 16932| 0| 0| 68378| 8|
56422| 137058| 12336| 8| 0| 0| 0| 0|
8| 2| 0| 12344| 12344| 0| 0| 84022| 4|
56819| 140526| 9896| 10| 0| 0| 0| 0|
6| 0| 0| 9898| 9896| 0| 0| 99138| 4|
57580| 136936| 8370| 10| 0| 2| 0| 0|
8| 6| 2| 8378| 8378| 0| 0| 110834| 22|
51583| 120138| 26828| 20| 0| 0| 0| 0|
4| 8| 0| 26848| 26848| 0| 0| 99292| 24|
49354| 128076| 21606| 14| 0| 2| 0| 0|
0| 10| 0| 21626| 21626| 0| 0| 60546| 12|
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 20:49 ` Kok, Auke
2008-06-16 20:59 ` Denys Fedoryshchenko
2008-06-16 21:13 ` Denys Fedoryshchenko
@ 2008-06-16 21:17 ` Denys Fedoryshchenko
2 siblings, 0 replies; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 21:17 UTC (permalink / raw)
To: Kok, Auke; +Cc: Eric Dumazet, netdev
What i notice now, i am not sure if errors updated realtime (maybe with delay,
then my idea is wrong).
Error appearing each 1 or 2 seconds.
If i do watch -n 1 "ifconfig eth1", exactly each _2nd_ cycle (means one is ok,
one errors increased, one is ok, one is errors counter increased), errors is
increasing. Maybe it is some SMI interrupt happening (for example Intel AMT
maybe using card for something)?
Or some routing processing (it is 900 routes there), like garbage collection.
But as i know garbage collector doesn't run each 1-2 second.
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 21:13 ` Denys Fedoryshchenko
@ 2008-06-16 21:29 ` Eric Dumazet
2008-06-16 21:41 ` Denys Fedoryshchenko
` (2 more replies)
0 siblings, 3 replies; 33+ messages in thread
From: Eric Dumazet @ 2008-06-16 21:29 UTC (permalink / raw)
To: Denys Fedoryshchenko; +Cc: netdev
Denys Fedoryshchenko a écrit :
>> Are you sure nf_conntrack or ip route cache is not killing you ?
>>
>> Filling 512 or 1024 RX ring on Gigabit link can be very fast, especially
>> if ip route cache is full.
>>
>> rtstat -c10 -i1
>
> conntrack disabled, it is just enabled for second on load and then unloaded.
ok :)
>
> MegaRouter-KARAM ~ # rtstat -c10 -i1
> rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
> entries| in_hit|in_slow_|in_slow_|in_no_ro| in_brd|in_marti|in_marti|
> out_hit|out_slow|out_slow|gc_total|gc_ignor|gc_goal_|gc_dst_o|in_hlist|out_hlis|
> | | tot| mc| ute| | an_dst| an_src|
> | _tot| _mc| | ed| miss| verflow| _search|t_search|
> 63448|229209146|22225578| 13754| 12| 2822| 0| 104|
> 54647| 27606| 570|21782075|21776859| 74| 0|215222374| 61294|
> 66085| 141462| 5268| 8| 0| 4| 0| 0|
> 4| 2| 0| 5274| 5274| 0| 0| 254424| 10|
> 59947| 132660| 20570| 4| 0| 4| 0| 0|
> 8| 14| 0| 20584| 20584| 0| 0| 185738| 24|
> 56995| 132416| 16918| 12| 0| 2| 0| 0|
> 6| 4| 0| 16932| 16932| 0| 0| 68378| 8|
> 56422| 137058| 12336| 8| 0| 0| 0| 0|
> 8| 2| 0| 12344| 12344| 0| 0| 84022| 4|
> 56819| 140526| 9896| 10| 0| 0| 0| 0|
> 6| 0| 0| 9898| 9896| 0| 0| 99138| 4|
> 57580| 136936| 8370| 10| 0| 2| 0| 0|
> 8| 6| 2| 8378| 8378| 0| 0| 110834| 22|
> 51583| 120138| 26828| 20| 0| 0| 0| 0|
> 4| 8| 0| 26848| 26848| 0| 0| 99292| 24|
> 49354| 128076| 21606| 14| 0| 2| 0| 0|
> 0| 10| 0| 21626| 21626| 0| 0| 60546| 12|
>
>
>
Hum... typical IP route cache congestion ?
echo 1 >/proc/sys/net/ipv4/route/gc_interval
echo 2 >/proc/sys/net/ipv4/route/gc_elasticity
You might want to boot with rhash_entries=131071 to play with IP route cache size, but I am not sure your workload can fit.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 21:29 ` Eric Dumazet
@ 2008-06-16 21:41 ` Denys Fedoryshchenko
2008-06-16 21:41 ` Denys Fedoryshchenko
[not found] ` <20080616213419.M212@visp.net.lb>
2 siblings, 0 replies; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 21:41 UTC (permalink / raw)
To: Eric Dumazet; +Cc: netdev
On Mon, 16 Jun 2008 23:29:01 +0200, Eric Dumazet wrote
> >> Are you sure nf_conntrack or ip route cache is not killing you ?
> >>
> >> Filling 512 or 1024 RX ring on Gigabit link can be very fast, especially
> >> if ip route cache is full.
> >>
> >> rtstat -c10 -i1
> >
> > conntrack disabled, it is just enabled for second on load and then unloaded.
>
> ok :)
>
> >
> > MegaRouter-KARAM ~ # rtstat -c10 -i1
> >
rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
> > entries| in_hit|in_slow_|in_slow_|in_no_ro| in_brd|in_marti|in_marti|
> >
out_hit|out_slow|out_slow|gc_total|gc_ignor|gc_goal_|gc_dst_o|in_hlist|out_hlis|
> > | | tot| mc| ute| | an_dst| an_src|
> > | _tot| _mc| | ed| miss| verflow| _search|t_search|
> > 63448|229209146|22225578| 13754| 12| 2822| 0| 104|
> > 54647| 27606| 570|21782075|21776859| 74| 0|215222374|
61294|
> > 66085| 141462| 5268| 8| 0| 4| 0| 0|
> > 4| 2| 0| 5274| 5274| 0| 0| 254424| 10|
> > 59947| 132660| 20570| 4| 0| 4| 0| 0|
> > 8| 14| 0| 20584| 20584| 0| 0| 185738| 24|
> > 56995| 132416| 16918| 12| 0| 2| 0| 0|
> > 6| 4| 0| 16932| 16932| 0| 0| 68378| 8|
> > 56422| 137058| 12336| 8| 0| 0| 0| 0|
> > 8| 2| 0| 12344| 12344| 0| 0| 84022| 4|
> > 56819| 140526| 9896| 10| 0| 0| 0| 0|
> > 6| 0| 0| 9898| 9896| 0| 0| 99138| 4|
> > 57580| 136936| 8370| 10| 0| 2| 0| 0|
> > 8| 6| 2| 8378| 8378| 0| 0| 110834| 22|
> > 51583| 120138| 26828| 20| 0| 0| 0| 0|
> > 4| 8| 0| 26848| 26848| 0| 0| 99292| 24|
> > 49354| 128076| 21606| 14| 0| 2| 0| 0|
> > 0| 10| 0| 21626| 21626| 0| 0| 60546| 12|
> >
> >
> >
>
> Hum... typical IP route cache congestion ?
>
> echo 1 >/proc/sys/net/ipv4/route/gc_interval
> echo 2 >/proc/sys/net/ipv4/route/gc_elasticity
Doesn't help, nothing changed.
>
> You might want to boot with rhash_entries=131071 to play with IP
> route cache size, but I am not sure your workload can fit.
I will try it, but thats kind of difficult, i cannot reboot anymore near 30
minutes.
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 21:29 ` Eric Dumazet
2008-06-16 21:41 ` Denys Fedoryshchenko
@ 2008-06-16 21:41 ` Denys Fedoryshchenko
[not found] ` <20080616213419.M212@visp.net.lb>
2 siblings, 0 replies; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 21:41 UTC (permalink / raw)
To: netdev
On Mon, 16 Jun 2008 23:29:01 +0200, Eric Dumazet wrote
> >> Are you sure nf_conntrack or ip route cache is not killing you ?
> >>
> >> Filling 512 or 1024 RX ring on Gigabit link can be very fast, especially
> >> if ip route cache is full.
> >>
> >> rtstat -c10 -i1
> >
> > conntrack disabled, it is just enabled for second on load and then unloaded.
>
> ok :)
>
> >
> > MegaRouter-KARAM ~ # rtstat -c10 -i1
> >
rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
> > entries| in_hit|in_slow_|in_slow_|in_no_ro| in_brd|in_marti|in_marti|
> >
out_hit|out_slow|out_slow|gc_total|gc_ignor|gc_goal_|gc_dst_o|in_hlist|out_hlis|
> > | | tot| mc| ute| | an_dst| an_src|
> > | _tot| _mc| | ed| miss| verflow| _search|t_search|
> > 63448|229209146|22225578| 13754| 12| 2822| 0| 104|
> > 54647| 27606| 570|21782075|21776859| 74| 0|215222374|
61294|
> > 66085| 141462| 5268| 8| 0| 4| 0| 0|
> > 4| 2| 0| 5274| 5274| 0| 0| 254424| 10|
> > 59947| 132660| 20570| 4| 0| 4| 0| 0|
> > 8| 14| 0| 20584| 20584| 0| 0| 185738| 24|
> > 56995| 132416| 16918| 12| 0| 2| 0| 0|
> > 6| 4| 0| 16932| 16932| 0| 0| 68378| 8|
> > 56422| 137058| 12336| 8| 0| 0| 0| 0|
> > 8| 2| 0| 12344| 12344| 0| 0| 84022| 4|
> > 56819| 140526| 9896| 10| 0| 0| 0| 0|
> > 6| 0| 0| 9898| 9896| 0| 0| 99138| 4|
> > 57580| 136936| 8370| 10| 0| 2| 0| 0|
> > 8| 6| 2| 8378| 8378| 0| 0| 110834| 22|
> > 51583| 120138| 26828| 20| 0| 0| 0| 0|
> > 4| 8| 0| 26848| 26848| 0| 0| 99292| 24|
> > 49354| 128076| 21606| 14| 0| 2| 0| 0|
> > 0| 10| 0| 21626| 21626| 0| 0| 60546| 12|
> >
> >
> >
>
> Hum... typical IP route cache congestion ?
>
> echo 1 >/proc/sys/net/ipv4/route/gc_interval
> echo 2 >/proc/sys/net/ipv4/route/gc_elasticity
Doesn't help, nothing changed.
>
> You might want to boot with rhash_entries=131071 to play with IP
> route cache size, but I am not sure your workload can fit.
I will try it, but thats kind of difficult, i cannot reboot anymore near 30
minutes.
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
[not found] ` <20080616213419.M212@visp.net.lb>
@ 2008-06-16 21:41 ` Eric Dumazet
2008-06-16 21:44 ` Eric Dumazet
0 siblings, 1 reply; 33+ messages in thread
From: Eric Dumazet @ 2008-06-16 21:41 UTC (permalink / raw)
To: Denys Fedoryshchenko; +Cc: netdev
Denys Fedoryshchenko a e'crit :
> On Mon, 16 Jun 2008 23:29:01 +0200, Eric Dumazet wrote
>> Denys Fedoryshchenko a [UTF-8?]ц╘crit :
>>>> Are you sure nf_conntrack or ip route cache is not killing you ?
>>>>
>>>> Filling 512 or 1024 RX ring on Gigabit link can be very fast, especially
>>>> if ip route cache is full.
>>>>
>>>> rtstat -c10 -i1
>>> conntrack disabled, it is just enabled for second on load and then unloaded.
>> ok :)
>>
>>> MegaRouter-KARAM ~ # rtstat -c10 -i1
>>>
> rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
>>> entries| in_hit|in_slow_|in_slow_|in_no_ro| in_brd|in_marti|in_marti|
>>>
> out_hit|out_slow|out_slow|gc_total|gc_ignor|gc_goal_|gc_dst_o|in_hlist|out_hlis|
>>> | | tot| mc| ute| | an_dst| an_src|
>>> | _tot| _mc| | ed| miss| verflow| _search|t_search|
>>> 63448|229209146|22225578| 13754| 12| 2822| 0| 104|
>>> 54647| 27606| 570|21782075|21776859| 74| 0|215222374|
> 61294|
>>> 66085| 141462| 5268| 8| 0| 4| 0| 0|
>>> 4| 2| 0| 5274| 5274| 0| 0| 254424| 10|
>>> 59947| 132660| 20570| 4| 0| 4| 0| 0|
>>> 8| 14| 0| 20584| 20584| 0| 0| 185738| 24|
>>> 56995| 132416| 16918| 12| 0| 2| 0| 0|
>>> 6| 4| 0| 16932| 16932| 0| 0| 68378| 8|
>>> 56422| 137058| 12336| 8| 0| 0| 0| 0|
>>> 8| 2| 0| 12344| 12344| 0| 0| 84022| 4|
>>> 56819| 140526| 9896| 10| 0| 0| 0| 0|
>>> 6| 0| 0| 9898| 9896| 0| 0| 99138| 4|
>>> 57580| 136936| 8370| 10| 0| 2| 0| 0|
>>> 8| 6| 2| 8378| 8378| 0| 0| 110834| 22|
>>> 51583| 120138| 26828| 20| 0| 0| 0| 0|
>>> 4| 8| 0| 26848| 26848| 0| 0| 99292| 24|
>>> 49354| 128076| 21606| 14| 0| 2| 0| 0|
>>> 0| 10| 0| 21626| 21626| 0| 0| 60546| 12|
>>>
>>>
>>>
>> Hum... typical IP route cache congestion ?
>>
>> echo 1 >/proc/sys/net/ipv4/route/gc_interval
>> echo 2 >/proc/sys/net/ipv4/route/gc_elasticity
> Doesn't help, nothing changed.
Your change on gc_interval can be delayed up to 60 seconds, you need to be patient :)
>
>> You might want to boot with rhash_entries=131071 to play with IP
>> route cache size, but I am not sure your workload can fit.
> I will try it, but thats kind of difficult, i cannot reboot anymore near 30
> minutes.
Before rebooting, make sure you can oprofile your kernel, this is the next step :)
Also, try to cpu affine both eth1 interrupts and timer interrupts (same cpu handling both)
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 21:41 ` Eric Dumazet
@ 2008-06-16 21:44 ` Eric Dumazet
2008-06-16 22:08 ` Denys Fedoryshchenko
2008-06-16 22:08 ` Denys Fedoryshchenko
0 siblings, 2 replies; 33+ messages in thread
From: Eric Dumazet @ 2008-06-16 21:44 UTC (permalink / raw)
To: Denys Fedoryshchenko; +Cc: netdev
Eric Dumazet a e'crit :
> Denys Fedoryshchenko a e'crit :
>> On Mon, 16 Jun 2008 23:29:01 +0200, Eric Dumazet wrote
>>> Denys Fedoryshchenko a [UTF-8?]ц╘crit :
>>>>> Are you sure nf_conntrack or ip route cache is not killing you ?
>>>>>
>>>>> Filling 512 or 1024 RX ring on Gigabit link can be very fast,
>>>>> especially if ip route cache is full.
>>>>>
>>>>> rtstat -c10 -i1
>>>> conntrack disabled, it is just enabled for second on load and then
>>>> unloaded.
>>> ok :)
>>>
>>>> MegaRouter-KARAM ~ # rtstat -c10 -i1
>>>>
>> rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
>>
>>>> entries| in_hit|in_slow_|in_slow_|in_no_ro|
>>>> in_brd|in_marti|in_marti|
>>>>
>> out_hit|out_slow|out_slow|gc_total|gc_ignor|gc_goal_|gc_dst_o|in_hlist|out_hlis|
>>
>>>> | | tot| mc| ute| | an_dst|
>>>> an_src| | _tot| _mc| | ed| miss|
>>>> verflow| _search|t_search|
>>>> 63448|229209146|22225578| 13754| 12| 2822| 0|
>>>> 104| 54647| 27606| 570|21782075|21776859| 74|
>>>> 0|215222374|
>> 61294|
>>>> 66085| 141462| 5268| 8| 0| 4|
>>>> 0| 0| 4| 2| 0| 5274| 5274|
>>>> 0| 0| 254424| 10|
>>>> 59947| 132660| 20570| 4| 0| 4|
>>>> 0| 0| 8| 14| 0| 20584| 20584|
>>>> 0| 0| 185738| 24|
>>>> 56995| 132416| 16918| 12| 0| 2|
>>>> 0| 0| 6| 4| 0| 16932| 16932|
>>>> 0| 0| 68378| 8|
>>>> 56422| 137058| 12336| 8| 0| 0|
>>>> 0| 0| 8| 2| 0| 12344| 12344|
>>>> 0| 0| 84022| 4|
>>>> 56819| 140526| 9896| 10| 0| 0|
>>>> 0| 0| 6| 0| 0| 9898| 9896|
>>>> 0| 0| 99138| 4|
>>>> 57580| 136936| 8370| 10| 0| 2|
>>>> 0| 0| 8| 6| 2| 8378| 8378|
>>>> 0| 0| 110834| 22|
>>>> 51583| 120138| 26828| 20| 0| 0|
>>>> 0| 0| 4| 8| 0| 26848| 26848|
>>>> 0| 0| 99292| 24|
>>>> 49354| 128076| 21606| 14| 0| 2|
>>>> 0| 0| 0| 10| 0| 21626| 21626|
>>>> 0| 0| 60546| 12|
>>>>
>>>>
>>>>
>>> Hum... typical IP route cache congestion ?
>>>
>>> echo 1 >/proc/sys/net/ipv4/route/gc_interval
>>> echo 2 >/proc/sys/net/ipv4/route/gc_elasticity
>> Doesn't help, nothing changed.
>
> Your change on gc_interval can be delayed up to 60 seconds, you need to
> be patient :)
>
>>
>>> You might want to boot with rhash_entries=131071 to play with IP
>>> route cache size, but I am not sure your workload can fit.
>> I will try it, but thats kind of difficult, i cannot reboot anymore
>> near 30
>> minutes.
>
> Before rebooting, make sure you can oprofile your kernel, this is the
> next step :)
>
> Also, try to cpu affine both eth1 interrupts and timer interrupts (same
> cpu handling both)
>
ALso please try to lower the copybreak value of e1000e driver
(module parameter copybreak=20 for example)
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 21:44 ` Eric Dumazet
@ 2008-06-16 22:08 ` Denys Fedoryshchenko
2008-06-17 2:17 ` Eric Dumazet
2008-06-16 22:08 ` Denys Fedoryshchenko
1 sibling, 1 reply; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 22:08 UTC (permalink / raw)
To: Eric Dumazet; +Cc: netdev
On Mon, 16 Jun 2008 23:44:07 +0200, Eric Dumazet wrote
opreport -l /mnt/flash/vmlinux |more
Here is oprofile
CPU: Core 2, speed 2397.64 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit
mask of 0x00 (Unhalted core cycles) count 100000
samples % symbol name
25409 15.0068 mwait_idle
14330 8.4634 ip_route_input
7827 4.6227 irq_entries_start
6488 3.8319 dev_queue_xmit
5497 3.2466 __qdisc_run
5033 2.9725 __slab_free
3569 2.1079 native_read_tsc
3560 2.1026 ip_rcv
3520 2.0789 __alloc_skb
3435 2.0287 __kmalloc_track_caller
3381 1.9968 kmem_cache_alloc
3340 1.9726 ip_forward
3289 1.9425 kfree
MegaRouter-KARAM /lib # opreport
CPU: Core 2, speed 2397.64 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit
mask of 0x00 (Unhalted core cycles) count 100000
CPU_CLK_UNHALT...|
samples| %|
------------------
612990 64.5673 vmlinux
276274 29.1004 e1000e
31470 3.3148 ip_tables
9676 1.0192 libzebra.so.0.0.0
3763 0.3964 oprofiled
MegaRouter-KARAM ~ # opreport image:/e1000e -l
CPU: Core 2, speed 2397.64 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit
mask of 0x00 (Unhalted core cycles) count 100000
warning: could not check that the binary file /e1000e has not been modified
since the profile was taken. Results may be inaccurate.
samples % symbol name
144449 25.7521 e1000_irq_enable
97006 17.2941 e1000_intr_msi
95009 16.9380 e1000_clean_rx_irq
87833 15.6587 e1000_xmit_frame
53723 9.5776 e1000_clean_tx_irq
26546 4.7326 e1000_clean
19968 3.5599 e1000_alloc_rx_buffers
11941 2.1288 e1000_receive_skb
6578 1.1727 e1000_update_itr
5864 1.0454 e1000_desc_unused
4847 0.8641 e1000_maybe_stop_tx
3537 0.6306 e1000_put_txbuf
3490 0.6222 e1000_rx_checksum
107 0.0191 e1000e_update_stats
9 0.0016 e1000_acquire_swflag_ich8lan
7 0.0012 e1000_release_swflag_ich8lan
5 8.9e-04 e1000e_read_phy_reg_mdic
1 1.8e-04 e1000_get_settings
1 1.8e-04 e1000_ioctl
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 21:44 ` Eric Dumazet
2008-06-16 22:08 ` Denys Fedoryshchenko
@ 2008-06-16 22:08 ` Denys Fedoryshchenko
2008-06-16 23:24 ` Brandeburg, Jesse
1 sibling, 1 reply; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-16 22:08 UTC (permalink / raw)
To: Eric Dumazet; +Cc: netdev
On Mon, 16 Jun 2008 23:44:07 +0200, Eric Dumazet wrote
>
> ALso please try to lower the copybreak value of e1000e driver
> (module parameter copybreak=20 for example)
Already tried, but over sysfs
MegaRouter-KARAM ~ # echo 20 > /sys/module/e1000e/parameters/copybreak
Doesn't change anything. I am preparing profiling now.
I dont think it is CPU load issue. I have motherboard DP35DP working on
similar workload(less routes, but conntrack + ifb + htb without hysteresis)
perfectly, but it doesn't have Intel AMT, maybe that's the key.
Why CPU load not the issue? I don't think mpstat is lying. 98-99% CPU is idle.
But i will try oprofile, maybe it can help and will show something.
I will try also PREEMPT kernel now, maybe it is some locking issue, and
preemption can help with that. Because "bursty" and clearly periodic character
of errors means or counters updated each 1-2 seconds or it is something
happening each 1-2 seconds. I am not sure oprofile can catch that.
MegaRouter-KARAM ~ # mpstat 1
Linux 2.6.26-rc6-git2-build-0029 (MegaRouter-KARAM) 06/17/08
00:54:23 CPU %user %nice %sys %iowait %irq %soft %steal
%idle intr/s
00:54:24 all 0.51 0.00 0.00 0.00 0.00 1.02 0.00
98.48 18902.00
00:54:25 all 0.00 0.00 0.00 0.00 0.00 0.51 0.00
99.49 18735.00
00:54:26 all 0.00 0.00 0.00 0.00 0.00 0.48 0.00
99.52 19057.00
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* RE: packetloss, on e1000e worse than r8169?
2008-06-16 22:08 ` Denys Fedoryshchenko
@ 2008-06-16 23:24 ` Brandeburg, Jesse
2008-06-17 1:53 ` Denys Fedoryshchenko
` (2 more replies)
0 siblings, 3 replies; 33+ messages in thread
From: Brandeburg, Jesse @ 2008-06-16 23:24 UTC (permalink / raw)
To: Denys Fedoryshchenko, Eric Dumazet; +Cc: netdev
Denys Fedoryshchenko wrote:
> On Mon, 16 Jun 2008 23:44:07 +0200, Eric Dumazet wrote
>>
>> ALso please try to lower the copybreak value of e1000e driver
>> (module parameter copybreak=20 for example) Already tried, but over
>> sysfs
>
> MegaRouter-KARAM ~ # echo 20 > /sys/module/e1000e/parameters/copybreak
>
> Doesn't change anything. I am preparing profiling now.
>
> I dont think it is CPU load issue. I have motherboard DP35DP working
> on similar workload(less routes, but conntrack + ifb + htb without
> hysteresis) perfectly, but it doesn't have Intel AMT, maybe that's
> the key.
>
> Why CPU load not the issue? I don't think mpstat is lying. 98-99% CPU
> is idle. But i will try oprofile, maybe it can help and will show
> something.
> I will try also PREEMPT kernel now, maybe it is some locking issue,
> and preemption can help with that. Because "bursty" and clearly
> periodic character of errors means or counters updated each 1-2
> seconds or it is something happening each 1-2 seconds. I am not sure
> oprofile can catch that.
error stats update only every two seconds from e1000e watchdog timer.
Please try using ethtool -C ethX rx-usecs 10 (100,000 interrupts per
second possible)
The realtek may not have the same kind of interrupt moderation as the
e1000e part. Also, you mentioned elsewhere in this thread that PCIe
should have plenty of bandwith but the reality is that x1 PCIe has a lot
of latency so as you try to push a lot of small packets they may not be
processed fast enough.
Try turning off Flow Control using ethtool -A ethX rx off tx off autoneg
off
rx_missed_errors is because you're running out of RX FIFO in the
adapter, and we can try to change some code to increase the RX fifo to a
larger value, if you're using default 1500 MTU, you only need 4K TX
fifo, so can have a little more RX fifo than what you have by default
(it's the PBA register we need to change the lower 16 bits to 0x1c)
Jesse
^ permalink raw reply [flat|nested] 33+ messages in thread
* RE: packetloss, on e1000e worse than r8169?
2008-06-16 23:24 ` Brandeburg, Jesse
@ 2008-06-17 1:53 ` Denys Fedoryshchenko
2008-06-17 6:46 ` Denys Fedoryshchenko
2008-06-17 21:04 ` Denys Fedoryshchenko
2 siblings, 0 replies; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-17 1:53 UTC (permalink / raw)
To: Brandeburg, Jesse, Eric Dumazet; +Cc: netdev
On Mon, 16 Jun 2008 16:24:22 -0700, Brandeburg, Jesse wrote
>
> error stats update only every two seconds from e1000e watchdog timer.
>
> Please try using ethtool -C ethX rx-usecs 10 (100,000 interrupts per
> second possible)
>
> The realtek may not have the same kind of interrupt moderation as the
> e1000e part. Also, you mentioned elsewhere in this thread that PCIe
> should have plenty of bandwith but the reality is that x1 PCIe has a
> lot of latency so as you try to push a lot of small packets they may
> not be processed fast enough.
>
> Try turning off Flow Control using ethtool -A ethX rx off tx off autoneg
> off
>
> rx_missed_errors is because you're running out of RX FIFO in the
> adapter, and we can try to change some code to increase the RX fifo
> to a larger value, if you're using default 1500 MTU, you only need
> 4K TX fifo, so can have a little more RX fifo than what you have by default
> (it's the PBA register we need to change the lower 16 bits to 0x1c)
>
> Jesse
First of all thanks for all efforts and help you gave me guys. I wish i can
solve this problem and probably it will help someone in future. Sure i can
easily put server with PCI-X e1000, but probably it is good idea to find bug,
if there is any.
Now i changed motherboard, to another one (without AMT), same family and same
e1000e onboard.
Now it is DG965SS.
I will try to play with flow control and interrupt moderation on it.
By the way, i have ICH8, is there anything i can hit because of it?
i notice
/* Workaround for ICH8 bit corruption issue in FIFO memory */
if (hw->mac.type == e1000_ich8lan) {
/* Set Tx and Rx buffer allocation to 8k apiece. */
ew32(PBA, E1000_PBA_8K);
/* Set Packet Buffer Size to 16k. */
ew32(PBS, E1000_PBS_16K);
}
About PBA, is i need to change something here?
/*
* the Tx fifo also stores 16 bytes of information about the tx
* but don't include ethernet FCS because hardware appends it
*/
min_tx_space = (adapter->max_frame_size +
sizeof(struct e1000_tx_desc) -
ETH_FCS_LEN) * 2;
min_tx_space = ALIGN(min_tx_space, 1024);
min_tx_space >>= 10;
/* software strips receive CRC, so leave room for it */
min_rx_space = adapter->max_frame_size;
min_rx_space = ALIGN(min_rx_space, 1024);
min_rx_space >>= 10;
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 22:08 ` Denys Fedoryshchenko
@ 2008-06-17 2:17 ` Eric Dumazet
2008-06-17 2:47 ` Denys Fedoryshchenko
0 siblings, 1 reply; 33+ messages in thread
From: Eric Dumazet @ 2008-06-17 2:17 UTC (permalink / raw)
To: Denys Fedoryshchenko; +Cc: netdev
Denys Fedoryshchenko a e'crit :
> On Mon, 16 Jun 2008 23:44:07 +0200, Eric Dumazet wrote
>
> opreport -l /mnt/flash/vmlinux |more
> Here is oprofile
> CPU: Core 2, speed 2397.64 MHz (estimated)
> Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit
> mask of 0x00 (Unhalted core cycles) count 100000
> samples % symbol name
> 25409 15.0068 mwait_idle
> 14330 8.4634 ip_route_input
> 7827 4.6227 irq_entries_start
> 6488 3.8319 dev_queue_xmit
> 5497 3.2466 __qdisc_run
> 5033 2.9725 __slab_free
> 3569 2.1079 native_read_tsc
> 3560 2.1026 ip_rcv
> 3520 2.0789 __alloc_skb
> 3435 2.0287 __kmalloc_track_caller
> 3381 1.9968 kmem_cache_alloc
> 3340 1.9726 ip_forward
> 3289 1.9425 kfree
>
> MegaRouter-KARAM /lib # opreport
> CPU: Core 2, speed 2397.64 MHz (estimated)
> Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit
> mask of 0x00 (Unhalted core cycles) count 100000
> CPU_CLK_UNHALT...|
> samples| %|
> ------------------
> 612990 64.5673 vmlinux
> 276274 29.1004 e1000e
> 31470 3.3148 ip_tables
How often is iptables used on your machine ?
> 9676 1.0192 libzebra.so.0.0.0
> 3763 0.3964 oprofiled
>
> MegaRouter-KARAM ~ # opreport image:/e1000e -l
> CPU: Core 2, speed 2397.64 MHz (estimated)
> Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit
> mask of 0x00 (Unhalted core cycles) count 100000
> warning: could not check that the binary file /e1000e has not been modified
> since the profile was taken. Results may be inaccurate.
> samples % symbol name
> 144449 25.7521 e1000_irq_enable
I wonder why this one is called so often. If your
interface is really flooded with packets, we should not
exit from NAPI polling...
> 97006 17.2941 e1000_intr_msi
> 95009 16.9380 e1000_clean_rx_irq
> 87833 15.6587 e1000_xmit_frame
> 53723 9.5776 e1000_clean_tx_irq
> 26546 4.7326 e1000_clean
> 19968 3.5599 e1000_alloc_rx_buffers
> 11941 2.1288 e1000_receive_skb
> 6578 1.1727 e1000_update_itr
> 5864 1.0454 e1000_desc_unused
> 4847 0.8641 e1000_maybe_stop_tx
> 3537 0.6306 e1000_put_txbuf
> 3490 0.6222 e1000_rx_checksum
> 107 0.0191 e1000e_update_stats
> 9 0.0016 e1000_acquire_swflag_ich8lan
> 7 0.0012 e1000_release_swflag_ich8lan
> 5 8.9e-04 e1000e_read_phy_reg_mdic
> 1 1.8e-04 e1000_get_settings
> 1 1.8e-04 e1000_ioctl
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-17 2:17 ` Eric Dumazet
@ 2008-06-17 2:47 ` Denys Fedoryshchenko
0 siblings, 0 replies; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-17 2:47 UTC (permalink / raw)
To: Eric Dumazet; +Cc: netdev
On Tue, 17 Jun 2008 04:17:22 +0200, Eric Dumazet wrote
> How often is iptables used on your machine ?
On this machine, just 3-4 rules. As i mention before, unloading iptables
modules didn't help.
> > since the profile was taken. Results may be inaccurate.
> > samples % symbol name
> > 144449 25.7521 e1000_irq_enable
>
> I wonder why this one is called so often. If your
> interface is really flooded with packets, we should not
> exit from NAPI polling...
At that moments values RX 82Kpps, TX also around 82Kpps
339 Mbps RX, 337 Mbps TX
Average packet ~516 bytes
--
Denys Fedoryshchenko
Technical Manager
Virtual ISP S.A.L.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 23:24 ` Brandeburg, Jesse
2008-06-17 1:53 ` Denys Fedoryshchenko
@ 2008-06-17 6:46 ` Denys Fedoryshchenko
2008-06-17 17:22 ` Denys Fedoryshchenko
2008-06-17 21:04 ` Denys Fedoryshchenko
2 siblings, 1 reply; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-17 6:46 UTC (permalink / raw)
To: Brandeburg, Jesse; +Cc: Eric Dumazet, netdev
On Tuesday 17 June 2008 02:24, Brandeburg, Jesse wrote:
First, i catch when tx_deferred_ok increasing, rx_missed_errors also increasing. For example deffered +1 , missed_errors +10.
A bit strange.
> Try turning off Flow Control using ethtool -A ethX rx off tx off autoneg
> off
Well, seems it is a keypoint on new motherboard, but i have to see at peak time still (about 10-14 hours to go).
So the question is, or flow control is broken here, or it is unknown when i must use flow-control and when - not.
Once i had packetloss on e1000 gigabit link, without it.
Now it is packetloss - when i start to use it.
--
------
Technical Manager
Virtual ISP S.A.L.
Lebanon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-17 6:46 ` Denys Fedoryshchenko
@ 2008-06-17 17:22 ` Denys Fedoryshchenko
2008-06-18 0:58 ` Brandeburg, Jesse
0 siblings, 1 reply; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-17 17:22 UTC (permalink / raw)
To: netdev
More info. Now it is peak time, and flow control disabled. Before it was 1% packetloss on small packets and 5-6% on large, now it is 0.3% on small, 1-2% on large.
The flow-control required on this particular setup (probably linksys switches is too much crappy, no idea), and without it i experience "small" packetloss"
But seems on this motherboard, by some reasons flow-control not working, maybe it is related to PBA.
I will try to play with PBA now, but not sure i can do that.
--
------
Technical Manager
Virtual ISP S.A.L.
Lebanon
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-16 23:24 ` Brandeburg, Jesse
2008-06-17 1:53 ` Denys Fedoryshchenko
2008-06-17 6:46 ` Denys Fedoryshchenko
@ 2008-06-17 21:04 ` Denys Fedoryshchenko
2008-06-18 16:50 ` Brandeburg, Jesse
2 siblings, 1 reply; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-17 21:04 UTC (permalink / raw)
To: Brandeburg, Jesse; +Cc: Eric Dumazet, netdev
After trying everything, it looks like that problem in PBS size, and as result PBA (rx fifo) size.
On ICH8 it is small, only 16K PBS (0x10), and RX/TX set each 8k, even i can set 0xd/0x3 it doesn't help (i didn't measure if it make less packetloss).
As i understand, i need to set only RX, TX calculated automatically.
Both motherboards i tried had ICH8.
All other servers which i mention, and which have enough big load have:
1)Sun - PBA 48K (82546EB)
2)DP35DP - PBA 16K (ICH9)
Also ICH8 missing some features, that ICH9 supports, such as FLAG_HAS_ERT, but it looks ERT useful only for Jumbo frames.
And sure ICH8 doesn't support Jumbo frames, maybe because of limited PBS.
Is PBS size hardware limitation of ICH8?
Is it possible i am right in my conclusions?
Probably such details in network adapters will be useful for Vyatta guys, to choose proper network adapter for their systems :-)
On Tuesday 17 June 2008 02:24, Brandeburg, Jesse wrote:
> Denys Fedoryshchenko wrote:
> > On Mon, 16 Jun 2008 23:44:07 +0200, Eric Dumazet wrote
> >>
> >> ALso please try to lower the copybreak value of e1000e driver
> >> (module parameter copybreak=20 for example) Already tried, but over
> >> sysfs
> >
> > MegaRouter-KARAM ~ # echo 20 > /sys/module/e1000e/parameters/copybreak
> >
> > Doesn't change anything. I am preparing profiling now.
> >
> > I dont think it is CPU load issue. I have motherboard DP35DP working
> > on similar workload(less routes, but conntrack + ifb + htb without
> > hysteresis) perfectly, but it doesn't have Intel AMT, maybe that's
> > the key.
> >
> > Why CPU load not the issue? I don't think mpstat is lying. 98-99% CPU
> > is idle. But i will try oprofile, maybe it can help and will show
> > something.
> > I will try also PREEMPT kernel now, maybe it is some locking issue,
> > and preemption can help with that. Because "bursty" and clearly
> > periodic character of errors means or counters updated each 1-2
> > seconds or it is something happening each 1-2 seconds. I am not sure
> > oprofile can catch that.
>
> error stats update only every two seconds from e1000e watchdog timer.
>
> Please try using ethtool -C ethX rx-usecs 10 (100,000 interrupts per
> second possible)
>
> The realtek may not have the same kind of interrupt moderation as the
> e1000e part. Also, you mentioned elsewhere in this thread that PCIe
> should have plenty of bandwith but the reality is that x1 PCIe has a lot
> of latency so as you try to push a lot of small packets they may not be
> processed fast enough.
>
> Try turning off Flow Control using ethtool -A ethX rx off tx off autoneg
> off
>
> rx_missed_errors is because you're running out of RX FIFO in the
> adapter, and we can try to change some code to increase the RX fifo to a
> larger value, if you're using default 1500 MTU, you only need 4K TX
> fifo, so can have a little more RX fifo than what you have by default
> (it's the PBA register we need to change the lower 16 bits to 0x1c)
>
> Jesse
>
--
------
Technical Manager
Virtual ISP S.A.L.
Lebanon
^ permalink raw reply [flat|nested] 33+ messages in thread
* RE: packetloss, on e1000e worse than r8169?
2008-06-17 17:22 ` Denys Fedoryshchenko
@ 2008-06-18 0:58 ` Brandeburg, Jesse
0 siblings, 0 replies; 33+ messages in thread
From: Brandeburg, Jesse @ 2008-06-18 0:58 UTC (permalink / raw)
To: Denys Fedoryshchenko; +Cc: netdev
Denys Fedoryshchenko wrote:
> More info. Now it is peak time, and flow control disabled. Before it
> was 1% packetloss on small packets and 5-6% on large, now it is 0.3%
> on small, 1-2% on large. The flow-control required on this particular
> setup (probably linksys switches is too much crappy, no idea), and
> without it i experience "small" packetloss"
> But seems on this motherboard, by some reasons flow-control not
> working, maybe it is related to PBA.
>
> I will try to play with PBA now, but not sure i can do that.
yes, on ICH8 playing with PBA will not give you much, ICH8 was meant to
be a "client" part and so sacrificed some FIFO space to save cost/power.
Also, our adapters support variable flow control thresholds (but we have
no good way to tune), it could be that your length of wire to your
switch or the switch itself takes too long to respond to flow control.
This would mean that too many packets get enqueued to the wire/hardware
before the flow control gets reacted to at the switch.
the registers are FCRTL and FCRTH, the numbers are in bytes, the
hardware sends a flow control STOP (txoff) packet when the FIFO fullness
goes past FCRTH, and sends flow control GO (txon) packet when threshold
drops below FCRTL.
Maybe that will help, let me know what else I can help with.
^ permalink raw reply [flat|nested] 33+ messages in thread
* RE: packetloss, on e1000e worse than r8169?
2008-06-17 21:04 ` Denys Fedoryshchenko
@ 2008-06-18 16:50 ` Brandeburg, Jesse
2008-06-18 17:03 ` Denys Fedoryshchenko
0 siblings, 1 reply; 33+ messages in thread
From: Brandeburg, Jesse @ 2008-06-18 16:50 UTC (permalink / raw)
To: Denys Fedoryshchenko; +Cc: Eric Dumazet, netdev
Denys Fedoryshchenko wrote:
> After trying everything, it looks like that problem in PBS size, and
> as result PBA (rx fifo) size.
agreed
> On ICH8 it is small, only 16K PBS (0x10), and RX/TX set each 8k, even
> i can set 0xd/0x3 it doesn't help (i didn't measure if it make less
just to make sure, you set PBA=0xd, correct?
> packetloss). As i understand, i need to set only RX, TX calculated
> automatically. Both motherboards i tried had ICH8.
your understanding is correct, the lower 8 bits represent rx fifo size,
and the tx fifo size is computed based on the result of (PBS - (lower 8
bits of)PBA)
Please note this is mostly documented in the software developers manuals
posted both at intel.com and e1000.sourceforge.net. ICH8 documentation
is mostly covered in the chipset documentation.
> All other servers which i mention, and which have enough big load
> have: 1)Sun - PBA 48K (82546EB)
> 2)DP35DP - PBA 16K (ICH9)
>
> Also ICH8 missing some features, that ICH9 supports, such as
> FLAG_HAS_ERT, but it looks ERT useful only for Jumbo frames.
> And sure ICH8 doesn't support Jumbo frames, maybe because of limited
> PBS.
>
> Is PBS size hardware limitation of ICH8?
yes, working size of the FIFO is 16kB total (PBS)
> Is it possible i am right in my conclusions?
yes, client parts will not buffer as much data (generally) due to
smaller FIFO as the server parts, which typically have 64kB total FIFO.
> Probably such details in network adapters will be useful for Vyatta
> guys, to choose proper network adapter for their systems :-)
agreed, the rule here would be don't use client parts for server class
workloads, unfortunately we don't control what machines certain server
vendors put client parts like 82573, ICH8/ICH9 in, so sometimes you have
a "low end" server with a client gigabit ethernet part.
^ permalink raw reply [flat|nested] 33+ messages in thread
* Re: packetloss, on e1000e worse than r8169?
2008-06-18 16:50 ` Brandeburg, Jesse
@ 2008-06-18 17:03 ` Denys Fedoryshchenko
0 siblings, 0 replies; 33+ messages in thread
From: Denys Fedoryshchenko @ 2008-06-18 17:03 UTC (permalink / raw)
To: Brandeburg, Jesse; +Cc: Eric Dumazet, netdev
On Wednesday 18 June 2008 19:50, Brandeburg, Jesse wrote:
> Denys Fedoryshchenko wrote:
> > After trying everything, it looks like that problem in PBS size, and
> > as result PBA (rx fifo) size.
>
> agreed
>
> > On ICH8 it is small, only 16K PBS (0x10), and RX/TX set each 8k, even
> > i can set 0xd/0x3 it doesn't help (i didn't measure if it make less
>
> just to make sure, you set PBA=0xd, correct?
Yes
>
> > packetloss). As i understand, i need to set only RX, TX calculated
> > automatically. Both motherboards i tried had ICH8.
>
> your understanding is correct, the lower 8 bits represent rx fifo size,
> and the tx fifo size is computed based on the result of (PBS - (lower 8
> bits of)PBA)
>
> Please note this is mostly documented in the software developers manuals
> posted both at intel.com and e1000.sourceforge.net. ICH8 documentation
> is mostly covered in the chipset documentation.
>
> > All other servers which i mention, and which have enough big load
> > have: 1)Sun - PBA 48K (82546EB)
> > 2)DP35DP - PBA 16K (ICH9)
> >
> > Also ICH8 missing some features, that ICH9 supports, such as
> > FLAG_HAS_ERT, but it looks ERT useful only for Jumbo frames.
> > And sure ICH8 doesn't support Jumbo frames, maybe because of limited
> > PBS.
> >
> > Is PBS size hardware limitation of ICH8?
> yes, working size of the FIFO is 16kB total (PBS)
>
> > Is it possible i am right in my conclusions?
> yes, client parts will not buffer as much data (generally) due to
> smaller FIFO as the server parts, which typically have 64kB total FIFO.
>
> > Probably such details in network adapters will be useful for Vyatta
> > guys, to choose proper network adapter for their systems :-)
>
> agreed, the rule here would be don't use client parts for server class
> workloads, unfortunately we don't control what machines certain server
> vendors put client parts like 82573, ICH8/ICH9 in, so sometimes you have
> a "low end" server with a client gigabit ethernet part.
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
Yes, 0xd. I am using now onboard 82546GB on old Intel Xeon 3.0 Ghz, flow-control works flawlessly.
cpu family : 15
model : 4
model name : Intel(R) Xeon(TM) CPU 3.00GHz
stepping : 1
cpu MHz : 2992.650
cache size : 1024 KB
I have packetloss, but now it is 1.516-e06 %, which is acceptable number for me.
I had to increase ring size, otherwise i was getting rx_no_buffer_count in stats.
It is still famous rx_missed_errors: 3616 .
But as i reported in personal mail, before rx_missed_errors was larger than tx_deferred_ok, now it is:
rx_missed_errors: 3633
tx_deferred_ok: 19145380
Now server handling 180Kpps RX, 800Mbps RX+TX, 4 VLAN.
latest git kernel running. I will do probably soon profiling iptables on it and some other tasks to test.
Maybe i will try also just to compare ICH9, if will have chance and way to buy it.
MegaRouterXeon-KARAM ~ # mpstat 30
Linux 2.6.26-rc6-git4-build-0029 (MegaRouterXeon-KARAM) 06/18/08
20:04:02 CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
20:04:32 all 0.09 0.00 1.19 0.00 1.94 14.93 0.00 81.84 17775.00
20:05:02 all 0.09 0.00 1.32 0.00 1.72 14.70 0.00 82.17 17810.23
--
------
Technical Manager
Virtual ISP S.A.L.
Lebanon
^ permalink raw reply [flat|nested] 33+ messages in thread
end of thread, other threads:[~2008-06-18 17:03 UTC | newest]
Thread overview: 33+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-06-16 19:37 packetloss, on e1000e worse than r8169? Denys Fedoryshchenko
2008-06-16 19:48 ` Eric Dumazet
2008-06-16 20:20 ` Denys Fedoryshchenko
2008-06-16 20:23 ` Denys Fedoryshchenko
2008-06-16 20:36 ` Kok, Auke
2008-06-16 20:52 ` Denys Fedoryshchenko
[not found] ` <20080616204411.M52834@visp.net.lb>
2008-06-16 20:49 ` Kok, Auke
2008-06-16 20:59 ` Denys Fedoryshchenko
2008-06-16 21:13 ` Denys Fedoryshchenko
2008-06-16 21:29 ` Eric Dumazet
2008-06-16 21:41 ` Denys Fedoryshchenko
2008-06-16 21:41 ` Denys Fedoryshchenko
[not found] ` <20080616213419.M212@visp.net.lb>
2008-06-16 21:41 ` Eric Dumazet
2008-06-16 21:44 ` Eric Dumazet
2008-06-16 22:08 ` Denys Fedoryshchenko
2008-06-17 2:17 ` Eric Dumazet
2008-06-17 2:47 ` Denys Fedoryshchenko
2008-06-16 22:08 ` Denys Fedoryshchenko
2008-06-16 23:24 ` Brandeburg, Jesse
2008-06-17 1:53 ` Denys Fedoryshchenko
2008-06-17 6:46 ` Denys Fedoryshchenko
2008-06-17 17:22 ` Denys Fedoryshchenko
2008-06-18 0:58 ` Brandeburg, Jesse
2008-06-17 21:04 ` Denys Fedoryshchenko
2008-06-18 16:50 ` Brandeburg, Jesse
2008-06-18 17:03 ` Denys Fedoryshchenko
2008-06-16 21:17 ` Denys Fedoryshchenko
2008-06-16 21:05 ` Eric Dumazet
2008-06-16 20:15 ` Auke Kok
2008-06-16 20:29 ` Denys Fedoryshchenko
2008-06-16 20:26 ` Francois Romieu
2008-06-16 20:37 ` Waskiewicz Jr, Peter P
2008-06-16 20:42 ` Denys Fedoryshchenko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).