e100: inappropriate handling of shared interrupt ?

e100: inappropriate handling of shared interrupt ?

[Shaw Vrana]

Hello All,

I'm seeing some odd behavior using the e100 driver for an intel ethernet
controller 82557/8/9 (revv 10).  It appears as if the e100 driver is
handling interrupts generated by another device, though I am not certain
of this..

Using some printks, I see some odd packets received that are eventually
dropped somewhere up the stack.  The packets usually look something like
this:

SrcAddr: 8.0.69.0 (bogus source ip)
DstAddr: 0.40.226.169  (bogus dest ip)
Protocol: 6
InputInt: 2
SrcPort: 20
DstPort: 8793

The src address and dest. address are entirely bogus, the protocol is not
always TCP, but I've seen it be icmp or udp.  In addition, I see _nothing_
using tcpdump, which I also do not understand as I didn't think packets
were dropped before tcpdump.  I've seen this behavior on multiple machines
using the same hardware, but haven't been able to make much sense of it. 
These packets do not seem to affect the normal operation of the device,
i.e. it services correct ips/ports just as one would expect.

B/c I haven't been able to see the packets using tcpdump, I have been
thinking that the packets were generated by the box itself.  The packets
appear to be constantly arriving, though it does not appear as if a packet
with the same src ip/dst ip arrives more than once, though I could be
wrong about this.

>From dmesg I see that the e100 is sharing irq 12.

e100: Intel(R) PRO/100 Network Driver, 3.4.8-k2-NAPI
e100: Copyright(c) 1999-2005 Intel Corporation
PCI: Found IRQ 12 for device 0000:01:04.0
PCI: Sharing IRQ 12 with 0000:00:02.0
PCI: Sharing IRQ 12 with 0000:00:1d.0
divert: allocating divert_blk for eth0
e100: eth0: e100_probe: addr 0xe8083000, irq 12, MAC addr 00:0E:B6:26:95:05
(This is the only other message I see mentioning irq 12)
serio: i8042 AUX port at 0x60,0x64 irq 12

(output of ethtool -e)
Offset          Values
------          ------
0x0000          00 0e b6 26 95 05 1b 0d ff ff 01 02 01 47 ff ff
0x0010          ff ff ff ff 00 5f 70 00 86 80 7f 00 ff ff ff ff
0x0020          ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0x0030          ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0x0040          ff ff ff ff ff ff 29 12 ff ff ff ff ff ff ff ff
0x0050          ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0x0060          2c 01 00 40 06 41 ff ff ff ff ff ff ff ff ff ff
0x0070          ff ff ff ff ff ff ff ff ff ff ff ff ff ff b3 b5

eth1    Link encap:Ethernet  HWaddr 00:0E:B6:26:95:05
        UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
        RX packets:3959305 errors:0 dropped:0 overruns:0 frame:0
        TX packets:5337629 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:100
        RX bytes:801040171 (763.9 MiB)  TX bytes:797939498 (760.9 MiB)
        Interrupt:12 Base address:0xd000 Memory:e8083000-e8084000


Notice that 0 errors are reported..  How could this be?

ethtool eth1
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Advertised auto-negotiation: No
        Speed: 100Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: g
        Wake-on: d
        Current message level: 0x00000007 (7)
        Link detected: yes


Any ideas? or debugging info greatly appreciated.


Thanks,
Shaw

Re: e100: inappropriate handling of shared interrupt ?

[Auke Kok]

Shaw Vrana wrote:
> Hello All,
> 
> I'm seeing some odd behavior using the e100 driver for an intel ethernet
> controller 82557/8/9 (revv 10).  It appears as if the e100 driver is
> handling interrupts generated by another device, though I am not certain
> of this..
> 
> Using some printks, I see some odd packets received that are eventually
> dropped somewhere up the stack.  The packets usually look something like
> this:
> 
> SrcAddr: 8.0.69.0 (bogus source ip)
> DstAddr: 0.40.226.169  (bogus dest ip)
> Protocol: 6
> InputInt: 2
> SrcPort: 20
> DstPort: 8793
> 
> The src address and dest. address are entirely bogus, the protocol is not
> always TCP, but I've seen it be icmp or udp.  In addition, I see _nothing_
> using tcpdump, which I also do not understand as I didn't think packets
> were dropped before tcpdump.  I've seen this behavior on multiple machines
> using the same hardware, but haven't been able to make much sense of it. 
> These packets do not seem to affect the normal operation of the device,
> i.e. it services correct ips/ports just as one would expect.
> 
> B/c I haven't been able to see the packets using tcpdump, I have been
> thinking that the packets were generated by the box itself.  The packets
> appear to be constantly arriving, though it does not appear as if a packet
> with the same src ip/dst ip arrives more than once, though I could be
> wrong about this.
> 
> From dmesg I see that the e100 is sharing irq 12.
> 
> e100: Intel(R) PRO/100 Network Driver, 3.4.8-k2-NAPI
> e100: Copyright(c) 1999-2005 Intel Corporation
> PCI: Found IRQ 12 for device 0000:01:04.0
> PCI: Sharing IRQ 12 with 0000:00:02.0
> PCI: Sharing IRQ 12 with 0000:00:1d.0
> divert: allocating divert_blk for eth0
> e100: eth0: e100_probe: addr 0xe8083000, irq 12, MAC addr 00:0E:B6:26:95:05
> (This is the only other message I see mentioning irq 12)

what does /proc/interrupts say after the box is fully booted?

> serio: i8042 AUX port at 0x60,0x64 irq 12

so, proc/interrupts should show 2 devices using the same interrupt.

> (output of ethtool -e)
> Offset          Values
> ------          ------
> 0x0000          00 0e b6 26 95 05 1b 0d ff ff 01 02 01 47 ff ff
> 0x0010          ff ff ff ff 00 5f 70 00 86 80 7f 00 ff ff ff ff
> 0x0020          ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
> 0x0030          ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
> 0x0040          ff ff ff ff ff ff 29 12 ff ff ff ff ff ff ff ff
> 0x0050          ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
> 0x0060          2c 01 00 40 06 41 ff ff ff ff ff ff ff ff ff ff
> 0x0070          ff ff ff ff ff ff ff ff ff ff ff ff ff ff b3 b5


> 
> eth1    Link encap:Ethernet  HWaddr 00:0E:B6:26:95:05
>         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>         RX packets:3959305 errors:0 dropped:0 overruns:0 frame:0
>         TX packets:5337629 errors:0 dropped:0 overruns:0 carrier:0
>         collisions:0 txqueuelen:100
>         RX bytes:801040171 (763.9 MiB)  TX bytes:797939498 (760.9 MiB)
>         Interrupt:12 Base address:0xd000 Memory:e8083000-e8084000
> 
> 
> Notice that 0 errors are reported..  How could this be?

use ethtool -S eth1 to get more information on errors etc.

It's unlikely that an irq problem shows up in the ifconfig error stats. Those are 
completely different counters that don't interact.

> ethtool eth1
>         Supported ports: [ TP MII ]
>         Supported link modes:   10baseT/Half 10baseT/Full
>                                 100baseT/Half 100baseT/Full
>         Supports auto-negotiation: Yes
>         Advertised link modes:  10baseT/Half 10baseT/Full
>                                 100baseT/Half 100baseT/Full
>         Advertised auto-negotiation: No
>         Speed: 100Mb/s
>         Duplex: Full
>         Port: MII
>         PHYAD: 1
>         Transceiver: internal
>         Auto-negotiation: off
>         Supports Wake-on: g
>         Wake-on: d
>         Current message level: 0x00000007 (7)
>         Link detected: yes
> 
> 
> Any ideas?

can you try with the latest e100 driver from e1000.sf.net ? I don't think it solves it 
but it might help to try (doesn't hurt).

Cheers,

Auke

Re: e100: inappropriate handling of shared interrupt ?

[Shaw Vrana]

Hi Auke,

On Mon, Nov 27, 2006 at 11:18:13AM -0800, Auke Kok wrote:
> >I'm seeing some odd behavior using the e100 driver for an intel ethernet
> >controller 82557/8/9 (revv 10).  It appears as if the e100 driver is
> >handling interrupts generated by another device, though I am not certain
> >of this..
> >
> >Using some printks, I see some odd packets received that are eventually
> >dropped somewhere up the stack.  The packets usually look something like
> >this:
> >
> >SrcAddr: 8.0.69.0 (bogus source ip)
> >DstAddr: 0.40.226.169  (bogus dest ip)
> >Protocol: 6
> >InputInt: 2
> >SrcPort: 20
> >DstPort: 8793
> >
> >The src address and dest. address are entirely bogus, the protocol is not
> >always TCP, but I've seen it be icmp or udp.  In addition, I see _nothing_
> >using tcpdump, which I also do not understand as I didn't think packets
> >were dropped before tcpdump.  I've seen this behavior on multiple machines
> >using the same hardware, but haven't been able to make much sense of it. 
> >These packets do not seem to affect the normal operation of the device,
> >i.e. it services correct ips/ports just as one would expect.
> >
> >B/c I haven't been able to see the packets using tcpdump, I have been
> >thinking that the packets were generated by the box itself.  The packets
> >appear to be constantly arriving, though it does not appear as if a packet
> >with the same src ip/dst ip arrives more than once, though I could be
> >wrong about this.
> >
> >From dmesg I see that the e100 is sharing irq 12.
> >
> >e100: Intel(R) PRO/100 Network Driver, 3.4.8-k2-NAPI
> >e100: Copyright(c) 1999-2005 Intel Corporation
> >PCI: Found IRQ 12 for device 0000:01:04.0
> >PCI: Sharing IRQ 12 with 0000:00:02.0
> >PCI: Sharing IRQ 12 with 0000:00:1d.0
> >divert: allocating divert_blk for eth0
> >e100: eth0: e100_probe: addr 0xe8083000, irq 12, MAC addr 00:0E:B6:26:95:05
> >(This is the only other message I see mentioning irq 12)
> 
> what does /proc/interrupts say after the box is fully booted?

       CPU0       
  0: 1236112960          XT-PIC  timer
  2:          0          XT-PIC  cascade
  4:     144338          XT-PIC  serial
  5:    2109514          XT-PIC  primary
  8:          0          XT-PIC  rtc
  9:          0          XT-PIC  ehci_hcd
 10:   55010907          XT-PIC  lan0_0
 12:   57079668          XT-PIC  wan0_0
 14:   28456949          XT-PIC  ide0
NMI:          0 
ERR:          0


> >Notice that 0 errors are reported..  How could this be?
> 
> use ethtool -S eth1 to get more information on errors etc.
> 
> It's unlikely that an irq problem shows up in the ifconfig error stats. 
> Those are completely different counters that don't interact.

NIC statistics:
     rx_packets: 25896640
     tx_packets: 33721440
     rx_bytes: 391691733
     tx_bytes: 2804738076
     rx_errors: 0
     tx_errors: 0
     rx_dropped: 0
     tx_dropped: 0
     multicast: 0
     collisions: 0
     rx_length_errors: 0
     rx_over_errors: 0
     rx_crc_errors: 0
     rx_frame_errors: 0
     rx_fifo_errors: 0
     rx_missed_errors: 0
     tx_aborted_errors: 0
     tx_carrier_errors: 0
     tx_fifo_errors: 0
     tx_heartbeat_errors: 0
     tx_window_errors: 0
     tx_deferred: 0
     tx_single_collisions: 0
     tx_multi_collisions: 0
     tx_flow_control_pause: 40967
     rx_flow_control_pause: 0
     rx_flow_control_unsupported: 0
     tx_tco_packets: 0
     rx_tco_packets: 0


Unfortunately I do not currently have a machine in the lab on which I can
reproduce this problem, so this data toook a little while to get.  I did
find a box with the same version of the NIC, but the weird packets did not
appear on there at all.  I have seen this problem personally, but that box
seems to have disappeared.

Is there any reason these packets would not be reported in the above
statistics?  And if an interrupt sharing error is out, where could these
packets be coming from?

> can you try with the latest e100 driver from e1000.sf.net ? I don't think 
> it solves it but it might help to try (doesn't hurt).

I'll fly the NIC home and test the new driver if worse comes to worse.. :(


Thanks for your input,
Shaw