Mystery packet killing tg3

Mystery packet killing tg3

[Stephen Hemminger]

While I was on vacation, OSDL did some networking changes that seems to aggravate some
existing bug in the tg3 driver. Could be some VLAN related garbage, not sure.

System is 2 CPU AMD64 and the tg3 is on the motherboard.

I am seeing messages like:
 eth0: Tigon3 [partno(BCM95703A30) rev 1002 PHY(5703)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:0d:60:53:08:18
 eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] Split[0] WireSpeed[1] TSOcap[0] 
 tg3: tg3_stop_block timed out, ofs=4000 enable_bit=2

Any clues?

Re: Mystery packet killing tg3

[David S. Miller]

On Mon, 2 May 2005 16:24:05 -0700
Stephen Hemminger <shemminger@osdl.org> wrote:

> While I was on vacation, OSDL did some networking changes that seems to aggravate some
> existing bug in the tg3 driver. Could be some VLAN related garbage, not sure.
> 
> System is 2 CPU AMD64 and the tg3 is on the motherboard.
> 
> I am seeing messages like:
>  eth0: Tigon3 [partno(BCM95703A30) rev 1002 PHY(5703)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:0d:60:53:08:18
>  eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] Split[0] WireSpeed[1] TSOcap[0] 
>  tg3: tg3_stop_block timed out, ofs=4000 enable_bit=2
> 
> Any clues?

This usually means that there is some DMA corruption.
For example, some bug in the x86_64 IOMMU code or similar
causes a bogus DMA address to be fed to the tg3 or even
worse a DMA mapping is unmapped before tg3 is actually
done with it.

Please try to get some more debugging.  One thing that might
be useful would be a dump of the PCI config and PCI status
registers from PCI config space when that tg3_stop_block
event triggers.  It will tell us if there was a master or
slave abort on the PCI bus which would confirm my above
theory.

Also what PCI controller is in this box?  (ie. the north bridge,
lspci -v would tell)

Since AMD promised me an Opteron system last year, but never
made good on that promise, I've never been able to work on
fixing this bug myself. :-/

Re: Mystery packet killing tg3

[Michael Chan]

On Tue, 2005-05-03 at 14:13 -0700, David S. Miller wrote:
> On Tue, 3 May 2005 14:05:28 -0700
> Stephen Hemminger <shemminger@osdl.org> wrote:
> 
> > 00:0b.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X APIC (rev 01) (prog-if 10 [IO-APIC])
> 
> Buggy 8131 APIC, does the drivers/pci/quirks.c:quirk_amd_8131_ioapic()
> message:
> 
> PCI: MSI quirk detected. pci_msi_quirk set.
> 
> trigger?  MSI support recently got added to the tg3.c driver, but it should
> only register MSI interrupt handling if pci_enable_msi() returns zero.
> 
> See what it ends up doing, or if you even have a MSI capable tg3.c in
> your kernel.
> 
> 
MSI is buggy on 5703 and tg3 will not enable MSI on that chip. MSI only
works on the latest PCI Express chips.

Re: Mystery packet killing tg3

[Stephen Hemminger]

On Mon, 2 May 2005 20:02:51 -0700
"David S. Miller" <davem@davemloft.net> wrote:

> On Mon, 2 May 2005 16:24:05 -0700
> Stephen Hemminger <shemminger@osdl.org> wrote:
> 
> > While I was on vacation, OSDL did some networking changes that seems to aggravate some
> > existing bug in the tg3 driver. Could be some VLAN related garbage, not sure.
> > 
> > System is 2 CPU AMD64 and the tg3 is on the motherboard.
> > 
> > I am seeing messages like:
> >  eth0: Tigon3 [partno(BCM95703A30) rev 1002 PHY(5703)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:0d:60:53:08:18
> >  eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] Split[0] WireSpeed[1] TSOcap[0] 
> >  tg3: tg3_stop_block timed out, ofs=4000 enable_bit=2
> > 
> > Any clues?
> 
> This usually means that there is some DMA corruption.
> For example, some bug in the x86_64 IOMMU code or similar
> causes a bogus DMA address to be fed to the tg3 or even
> worse a DMA mapping is unmapped before tg3 is actually
> done with it.
> 
> Please try to get some more debugging.  One thing that might
> be useful would be a dump of the PCI config and PCI status
> registers from PCI config space when that tg3_stop_block
> event triggers.  It will tell us if there was a master or
> slave abort on the PCI bus which would confirm my above
> theory.

Added call to tg_dump_state()

tg3: tg3_stop_block timed out, ofs=4000 enable_bit=2
DEBUG: PCI status [02b0] TG3PCI state[000010e2]
DEBUG: MAC_MODE[00c04c08] MAC_STATUS[00400003]
       MAC_EVENT[00001000] MAC_LED_CTRL[00000880]
DEBUG: MAC_TX_MODE[00000010] MAC_TX_STATUS[00000008]
       MAC_RX_MODE[00000004] MAC_RX_STATUS[00000000]
DEBUG: SNDDATAI_MODE[00000000] SNDDATAI_STATUS[00000000]
       SNDDATAI_STATSCTRL[00000003]
DEBUG: SNDDATAC_MODE[00000000]
DEBUG: SNDBDS_MODE[00000004] SNDBDS_STATUS[00000000]
DEBUG: SNDBDI_MODE[00000004] SNDBDI_STATUS[00000000]
DEBUG: SNDBDC_MODE[00000000]
DEBUG: RCVLPC_MODE[00000000] RCVLPC_STATUS[00000000]
       RCVLPC_STATSCTRL[00000001]
DEBUG: RCVDBDI_MODE[00000010] RCVDBDI_STATUS[00000000]
DEBUG: RCVDCC_MODE[00000004]
DEBUG: RCVBDI_MODE[00000004] RCVBDI_STATUS[00000000]
DEBUG: RCVCC_MODE[00000004] RCVCC_STATUS[00000000]
DEBUG: RCVLSC_MODE[00000004] RCVLSC_STATUS[00000000]
DEBUG: MBFREE_MODE[00000000] MBFREE_STATUS[00000000]
DEBUG: HOSTCC_MODE[00000100] HOSTCC_STATUS[00000000]
DEBUG: HOSTCC_STATS_BLK_HOST_ADDR[0000000006bb8000]
DEBUG: HOSTCC_STATUS_BLK_HOST_ADDR[000000001ce8c000]
DEBUG: HOSTCC_STATS_BLK_NIC_ADDR[00000300]
DEBUG: HOSTCC_STATUS_BLK_NIC_ADDR[00000b00]
DEBUG: MEMARB_MODE[00000002] MEMARB_STATUS[00000000]
DEBUG: BUFMGR_MODE[00000004] BUFMGR_STATUS[00000000]
DEBUG: BUFMGR_MB_POOL_ADDR[00008000] BUFMGR_MB_POOL_SIZE[00018000]
DEBUG: BUFMGR_DMA_DESC_POOL_ADDR[00002000] BUFMGR_DMA_DESC_POOL_SIZE[00002000]
DEBUG: RDMAC_MODE[000003fc] RDMAC_STATUS[00000000]
DEBUG: WDMAC_MODE[000003fc] WDMAC_STATUS[00000000]
DEBUG: DMAC_MODE[00000000]
DEBUG: GRC_MODE[04130034] GRC_MISC_CFG[00081082]
DEBUG: GRC_LOCAL_CTRL[01000209]
DEBUG: RCVDBDI_JUMBO_BD[0000000000000000:00000002:00000000]
DEBUG: RCVDBDI_STD_BD[000000002d1c4000:06000000:00006000]
DEBUG: RCVDBDI_MINI_BD[0000000000000000:00000002:00000000]


> Also what PCI controller is in this box?  (ie. the north bridge,
> lspci -v would tell)

00:06.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8111 PCI (rev 07) (prog-if 00 [Normal decode])
	Flags: bus master, 66Mhz, medium devsel, latency 64
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=69
	I/O behind bridge: 00002000-00002fff
	Memory behind bridge: c0100000-c01fffff
	Capabilities: [c0] HyperTransport: Slave or Primary Interface
	Capabilities: [f0] HyperTransport: Interrupt Discovery and Configuration

00:07.0 ISA bridge: Advanced Micro Devices [AMD] AMD-8111 LPC (rev 05)
	Subsystem: IBM: Unknown device 02b7
	Flags: bus master, 66Mhz, medium devsel, latency 0

00:07.1 IDE interface: Advanced Micro Devices [AMD] AMD-8111 IDE (rev 03) (prog-if 8a [Master SecP PriP])
	Subsystem: IBM: Unknown device 02b7
	Flags: bus master, medium devsel, latency 64
	I/O ports at 1460 [size=16]

00:07.2 SMBus: Advanced Micro Devices [AMD] AMD-8111 SMBus 2.0 (rev 02)
	Subsystem: IBM: Unknown device 02b7
	Flags: medium devsel, IRQ 11
	I/O ports at 1440 [size=32]

00:07.3 Bridge: Advanced Micro Devices [AMD] AMD-8111 ACPI (rev 05)
	Subsystem: IBM: Unknown device 02b7
	Flags: medium devsel

00:07.5 Multimedia audio controller: Advanced Micro Devices [AMD] AMD-8111 AC97 Audio (rev 03)
	Subsystem: IBM: Unknown device 02b7
	Flags: bus master, medium devsel, latency 64, IRQ 169
	I/O ports at 1000 [size=256]
	I/O ports at 1400 [size=64]

00:0a.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 12) (prog-if 00 [Normal decode])
	Flags: bus master, 66Mhz, medium devsel, latency 64
	Bus: primary=00, secondary=02, subordinate=02, sec-latency=64
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: c0200000-c02fffff
	Capabilities: [a0] PCI-X bridge device.
	Capabilities: [b8] HyperTransport: Interrupt Discovery and Configuration
	Capabilities: [c0] HyperTransport: Slave or Primary Interface

00:0a.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X APIC (rev 01) (prog-if 10 [IO-APIC])
	Subsystem: IBM: Unknown device 02b7
	Flags: bus master, medium devsel, latency 0
	Memory at c0000000 (64-bit, non-prefetchable) [size=4K]

00:0b.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 12) (prog-if 00 [Normal decode])
	Flags: bus master, 66Mhz, medium devsel, latency 64
	Bus: primary=00, secondary=03, subordinate=03, sec-latency=64
	Memory behind bridge: c0300000-c03fffff
	Capabilities: [a0] PCI-X bridge device.
	Capabilities: [b8] HyperTransport: Interrupt Discovery and Configuration

00:0b.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X APIC (rev 01) (prog-if 10 [IO-APIC])
	Subsystem: IBM: Unknown device 02b7
	Flags: bus master, medium devsel, latency 0
	Memory at c0001000 (64-bit, non-prefetchable) [size=4K]

00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
	Flags: fast devsel
	Capabilities: [80] HyperTransport: Host or Secondary Interface
	Capabilities: [a0] HyperTransport: Host or Secondary Interface
	Capabilities: [c0] HyperTransport: Host or Secondary Interface

00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
	Flags: fast devsel

00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
	Flags: fast devsel

00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
	Flags: fast devsel

00:19.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
	Flags: fast devsel
	Capabilities: [80] HyperTransport: Host or Secondary Interface
	Capabilities: [a0] HyperTransport: Host or Secondary Interface
	Capabilities: [c0] HyperTransport: Host or Secondary Interface

00:19.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
	Flags: fast devsel

00:19.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
	Flags: fast devsel

00:19.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
	Flags: fast devsel

01:02.0 Unknown mass storage controller: Silicon Image, Inc. (formerly CMD Technology Inc) SiI 3512 [SATALink/SATARaid] Serial ATA Controller (rev 01) (prog-if 85)
	Subsystem: IBM: Unknown device 3512
	Flags: bus master, 66Mhz, medium devsel, latency 64, IRQ 169
	I/O ports at 2020 [size=8]
	I/O ports at 2014 [size=4]
	I/O ports at 2018 [size=8]
	I/O ports at 2010 [size=4]
	I/O ports at 2000 [size=16]
	Memory at c0104000 (32-bit, non-prefetchable) [size=512]
	Capabilities: [60] Power Management version 2

01:03.0 USB Controller: NEC Corporation USB (rev 43) (prog-if 10 [OHCI])
	Subsystem: IBM: Unknown device 02b7
	Flags: bus master, medium devsel, latency 64, IRQ 193
	Memory at c0102000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [40] Power Management version 2

01:03.1 USB Controller: NEC Corporation USB (rev 43) (prog-if 10 [OHCI])
	Subsystem: IBM: Unknown device 02b7
	Flags: bus master, medium devsel, latency 64, IRQ 201
	Memory at c0103000 (32-bit, non-prefetchable) [size=4K]
	Capabilities: [40] Power Management version 2

01:03.2 USB Controller: NEC Corporation USB 2.0 (rev 04) (prog-if 20 [EHCI])
	Subsystem: IBM: Unknown device 02b7
	Flags: bus master, medium devsel, latency 132, IRQ 185
	Memory at c0104400 (32-bit, non-prefetchable) [size=256]
	Capabilities: [40] Power Management version 2

02:01.0 Ethernet controller: Syskonnect (Schneider & Koch) SK-98xx Gigabit Ethernet Server Adapter (rev 13)
	Subsystem: Syskonnect (Schneider & Koch) SK-9821 Gigabit Ethernet Server Adapter (SK-NET GE-T)
	Flags: bus master, 66Mhz, medium devsel, latency 64, IRQ 10
	Memory at c0200000 (32-bit, non-prefetchable) [size=16K]
	I/O ports at 3000 [size=256]
	Capabilities: [48] Power Management version 1
	Capabilities: [50] Vital Product Data

03:02.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5703X Gigabit Ethernet (rev 02)
	Subsystem: IBM: Unknown device 026f
	Flags: bus master, 66Mhz, medium devsel, latency 64, IRQ 177
	Memory at c0300000 (64-bit, non-prefetchable) [size=64K]
	Capabilities: [40] PCI-X non-bridge device.
	Capabilities: [48] Power Management version 2
	Capabilities: [50] Vital Product Data
	Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable-

08:00.0 Host bridge: Advanced Micro Devices [AMD] AMD-8151 System Controller (rev 13)
	Subsystem: IBM: Unknown device 02b7
	Flags: medium devsel
	Memory at d0000000 (32-bit, prefetchable) [size=256M]
	Capabilities: [a0] AGP version 3.0
	Capabilities: [c0] HyperTransport: Slave or Primary Interface

08:01.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8151 AGP Bridge (rev 13) (prog-if 00 [Normal decode])
	Flags: bus master, 66Mhz, medium devsel, latency 99
	Bus: primary=08, secondary=09, subordinate=0d, sec-latency=68
	I/O behind bridge: 00004000-00004fff
	Memory behind bridge: c0700000-c07fffff
	Prefetchable memory behind bridge: e0000000-efffffff

08:03.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 12) (prog-if 00 [Normal decode])
	Flags: bus master, 66Mhz, medium devsel, latency 64
	Bus: primary=08, secondary=0e, subordinate=12, sec-latency=64
	Capabilities: [a0] PCI-X bridge device.
	Capabilities: [b8] HyperTransport: Interrupt Discovery and Configuration
	Capabilities: [c0] HyperTransport: Slave or Primary Interface

08:03.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X APIC (rev 01) (prog-if 10 [IO-APIC])
	Subsystem: IBM: Unknown device 02b7
	Flags: bus master, medium devsel, latency 0
	Memory at c0600000 (64-bit, non-prefetchable) [size=4K]

08:04.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 12) (prog-if 00 [Normal decode])
	Flags: bus master, 66Mhz, medium devsel, latency 64
	Bus: primary=08, secondary=13, subordinate=17, sec-latency=64
	Capabilities: [a0] PCI-X bridge device.
	Capabilities: [b8] HyperTransport: Interrupt Discovery and Configuration

08:04.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X APIC (rev 01) (prog-if 10 [IO-APIC])
	Subsystem: IBM: Unknown device 02b7
	Flags: bus master, medium devsel, latency 0
	Memory at c0601000 (64-bit, non-prefetchable) [size=4K]

09:00.0 VGA compatible controller: ATI Technologies Inc Radeon R300 NG [FireGL X1] (rev 80) (prog-if 00 [VGA])
	Subsystem: ATI Technologies Inc: Unknown device 0172
	Flags: bus master, stepping, 66Mhz, medium devsel, latency 66, IRQ 5
	Memory at e0000000 (32-bit, prefetchable) [size=128M]
	I/O ports at 4000 [size=256]
	Memory at c0700000 (32-bit, non-prefetchable) [size=64K]
	Capabilities: [58] AGP version 3.0
	Capabilities: [50] Power Management version 2

09:00.1 Display controller: ATI Technologies Inc Radeon R300 [FireGL X1] (Secondary) (rev 80)
	Subsystem: ATI Technologies Inc: Unknown device 0173
	Flags: bus master, stepping, 66Mhz, medium devsel, latency 66
	Memory at e8000000 (32-bit, prefetchable) [size=128M]
	Memory at c0710000 (32-bit, non-prefetchable) [size=64K]
	Capabilities: [50] Power Management version 2



-- 
Stephen Hemminger	<shemminger@osdl.org>


> Since AMD promised me an Opteron system last year, but never
> made good on that promise, I've never been able to work on
> fixing this bug myself. :-/

Re: Mystery packet killing tg3

[David S. Miller]

On Tue, 3 May 2005 14:05:28 -0700
Stephen Hemminger <shemminger@osdl.org> wrote:

> 00:0b.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X APIC (rev 01) (prog-if 10 [IO-APIC])

Buggy 8131 APIC, does the drivers/pci/quirks.c:quirk_amd_8131_ioapic()
message:

PCI: MSI quirk detected. pci_msi_quirk set.

trigger?  MSI support recently got added to the tg3.c driver, but it should
only register MSI interrupt handling if pci_enable_msi() returns zero.

See what it ends up doing, or if you even have a MSI capable tg3.c in
your kernel.

Re: Mystery packet killing tg3

[Michael Chan]

On Tue, 2005-05-03 at 15:03 -0700, David S. Miller wrote:

> Michael, there were no master/target abort bits set in the PCI status
> register from his dump.  If one of the DMA units locks up on the tg3,
> will it still be able to update the PCI_STATUS register appropriately
> when it encounters a DMA transaction error (ie. master or target abort)

I believe so. Also, the DMA read and write status registers showed all
zeros, meaning there were no DMA related errors:

DEBUG: RDMAC_MODE[000003fc] RDMAC_STATUS[00000000]
DEBUG: WDMAC_MODE[000003fc] WDMAC_STATUS[00000000]

Re: Mystery packet killing tg3

[Stephen Hemminger]

On Tue, 3 May 2005 14:13:14 -0700
"David S. Miller" <davem@davemloft.net> wrote:

> On Tue, 3 May 2005 14:05:28 -0700
> Stephen Hemminger <shemminger@osdl.org> wrote:
> 
> > 00:0b.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X APIC (rev 01) (prog-if 10 [IO-APIC])
> 
> Buggy 8131 APIC, does the drivers/pci/quirks.c:quirk_amd_8131_ioapic()
> message:
> 
> PCI: MSI quirk detected. pci_msi_quirk set.

yes. that is there.

> trigger?  MSI support recently got added to the tg3.c driver, but it should
> only register MSI interrupt handling if pci_enable_msi() returns zero.

There is no call to pci_enable_msi in the 2.6.12-rc3 source

> See what it ends up doing, or if you even have a MSI capable tg3.c in
> your kernel.

Re: Mystery packet killing tg3

[David S. Miller]

On Tue, 03 May 2005 13:41:47 -0700
"Michael Chan" <mchan@broadcom.com> wrote:

> MSI is buggy on 5703 and tg3 will not enable MSI on that chip. MSI only
> works on the latest PCI Express chips.

Ok, and he has a pre-MSI copy of the tg3 driver anyways.

Michael, there were no master/target abort bits set in the PCI status
register from his dump.  If one of the DMA units locks up on the tg3,
will it still be able to update the PCI_STATUS register appropriately
when it encounters a DMA transaction error (ie. master or target abort)
or would we also need to look at the PCI host bridge PCI config space
registers as well?

I strongly suspect one of two things in these AMD system cases of tg3
wedging:

1) some DMA problem
2) register write reordering

Lack of master/target abort indication in tg3's PCI status register
makes me feel that #2 is more likely to be the problem cause.

Just for fun, Stephen, can you make tg3_get_invariants() always set
the TG3_FLAG_MBOX_WRITE_REORDER bit in tp->tg3_flags and see if that
makes the problem go away?

Thanks.

Re: Mystery packet killing tg3

[David S. Miller]

On Tue, 3 May 2005 15:45:00 -0700
Stephen Hemminger <shemminger@osdl.org> wrote:

> No, I forced it to always be true and still dies (when taking network down).

You're reproducing this pretty fast.

Does the link jam up, or are you just simply downing the interface
and the stop block messages are printed out?

If it jams up, what kind of traffic are you using to trigger this
problem?

Re: Mystery packet killing tg3

[Stephen Hemminger]

On Tue, 3 May 2005 15:03:33 -0700
"David S. Miller" <davem@davemloft.net> wrote:

> On Tue, 03 May 2005 13:41:47 -0700
> "Michael Chan" <mchan@broadcom.com> wrote:
> 
> > MSI is buggy on 5703 and tg3 will not enable MSI on that chip. MSI only
> > works on the latest PCI Express chips.
> 
> Ok, and he has a pre-MSI copy of the tg3 driver anyways.
> 
> Michael, there were no master/target abort bits set in the PCI status
> register from his dump.  If one of the DMA units locks up on the tg3,
> will it still be able to update the PCI_STATUS register appropriately
> when it encounters a DMA transaction error (ie. master or target abort)
> or would we also need to look at the PCI host bridge PCI config space
> registers as well?
> 
> I strongly suspect one of two things in these AMD system cases of tg3
> wedging:
> 
> 1) some DMA problem
> 2) register write reordering
> 
> Lack of master/target abort indication in tg3's PCI status register
> makes me feel that #2 is more likely to be the problem cause.
> 
> Just for fun, Stephen, can you make tg3_get_invariants() always set
> the TG3_FLAG_MBOX_WRITE_REORDER bit in tp->tg3_flags and see if that
> makes the problem go away?

No, I forced it to always be true and still dies (when taking network down).

Re: Mystery packet killing tg3

[David S. Miller]

On Tue, 03 May 2005 14:28:10 -0700
"Michael Chan" <mchan@broadcom.com> wrote:

> On Tue, 2005-05-03 at 15:03 -0700, David S. Miller wrote:
> 
> > Michael, there were no master/target abort bits set in the PCI status
> > register from his dump.  If one of the DMA units locks up on the tg3,
> > will it still be able to update the PCI_STATUS register appropriately
> > when it encounters a DMA transaction error (ie. master or target abort)
> 
> I believe so. Also, the DMA read and write status registers showed all
> zeros, meaning there were no DMA related errors:
> 
> DEBUG: RDMAC_MODE[000003fc] RDMAC_STATUS[00000000]
> DEBUG: WDMAC_MODE[000003fc] WDMAC_STATUS[00000000]

Right, I noticed that too.

Stephen says that trying to force enable the mailbox write
reordering workaround doesn't solve the problem either.

I wonder exactly how it would show up if the x86_64 port
unmapped a DMA address in the IOMMU and tg3 (or a bridge
in the middle) tried to fetch that address again.

I remember some issue not too long ago where PCI bridges
could still prefetch a DMA address even after a device
was done with it.  Because of this, they added code to
the x86_64 IOMMU driver that kept a read-only dummy mapping
around all the time so that this would not cause faults.
I even added similar code to the PCI IOMMU handling on sparc64.

Indeed, if you look in arch/x86_64/kernel/pci-gart.c they use
a scratch mapping when the unmap DMA translations.

I can't think of what else could be wedging the tg3.  Michael,
any ideas?  There are some 5703 specific programming to consider:

1) Setting of PCIX_CAPS_BURST_MASK in PCI_X_CAPS register.
   Currently tg3 sets it to "PCIX_CAPS_MAX_BURST_CPIOB" which
   is 2 (shifted up PCIX_CAPS_BURST_SHIFT of course).

2) On Fibre, we force a write of 0x616000 to MAC_SERDES_CFG for
   5703 chips.

   Because the PCS synced indication is on in his dumps, I
   am assuming he is on a Fibre link, so this is relevant.

3) When the low 5 bits of TG3PCI_CLOCK_CTRL are 0x6 or 0x7
   we set DMA_RWCTL_ONE_DMA in the DMA R/W control register.
   On all 5703 we also set bit 23 to enable some hw bug workaround.

4) On 5703 (and 5704), we always clear the low 4 bits of DMA R/W
   control.

A quick perusal shows that these same exact things get done in
the bcm5700 driver too.

Re: Mystery packet killing tg3

[Stephen Hemminger]

On Tue, 3 May 2005 15:39:54 -0700
"David S. Miller" <davem@davemloft.net> wrote:

> On Tue, 3 May 2005 15:45:00 -0700
> Stephen Hemminger <shemminger@osdl.org> wrote:
> 
> > No, I forced it to always be true and still dies (when taking network down).
> 
> You're reproducing this pretty fast.
> 
> Does the link jam up, or are you just simply downing the interface
> and the stop block messages are printed out?

Initially, it reproduced everytime link came up, we reconfigured the VLAN to
have a mirror port into a laptop to try and capture what was happening, but when
we did that the bootup problem went away. It was in the tg3_reset_hw
during initial dev_open.

Today it only triggers when
I force it by doing something does a dev_close (like ifconfig or rmmod).
Since the message is the same, I assume it is the same or similar problem.

> If it jams up, what kind of traffic are you using to trigger this
> problem?

No traffic is present when the close occurs. But some DMA could be stuck
(forever) in flight at that point.

RE: Mystery packet killing tg3

[Michael Chan]

David S. Miller wrote:

> I can't think of what else could be wedging the tg3.  
> Michael, any ideas?  There are some 5703 specific programming 
> to consider:
> 
The 5703 related settings in tg3 seem ok to me too.

If Stephen says the stop_block error happens during normal ifdown and traffic
was otherwise working fine before ifdown, then I think it may not even be a
problem at all.

Stopping the various state machines can be tricky and I'm not at all
surprised that some state machines can fail to stop in some cases. They are
all interconnected and even if you follow the stopping sequence in the
programmer's reference manual, you may still end up with a situation where
one state machine is waiting for another that has been stopped already. This
is not a problem as tg3_stop_block() calls are always followed by a global
chip reset that will clean up the whole chip. The purpose of tg3_stop_block()
is to quiesce the chip and complete all DMA transactions before abruptly
resetting the chip. If the DMA blocks would not stop, then it would be a
bigger problem.

Other tg3_stop_block() errors that I've seen, such as the ones reported by
John Linville, were preceded by netdev watchdog timeouts. These netdev
watchdog timeouts were real problems that needed to be solved.

RE: Mystery packet killing tg3

[Michael Chan]

Stephen Hemminger wrote:

> Initially, it reproduced everytime link came up, we 
> reconfigured the VLAN to
> have a mirror port into a laptop to try and capture what was 
> happening, but when
> we did that the bootup problem went away. It was in the tg3_reset_hw
> during initial dev_open.
> 

During initial dev_open, the TG3_FLAG_INIT_COMPLETE flag is not set so
tg3_reset_hw() should not call tg3_abort_hw() where the stop_block calls are
made. So there should be no stop_block errors.

I think stop_block errors can only happen during dev_close, suspend, netdev
watchdog, or ethtool "set" calls.

Re: Mystery packet killing tg3

[Andi Kleen]

"David S. Miller" <davem@davemloft.net> writes:
>
> This usually means that there is some DMA corruption.
> For example, some bug in the x86_64 IOMMU code or similar
> causes a bogus DMA address to be fed to the tg3 or even
> worse a DMA mapping is unmapped before tg3 is actually
> done with it.

IOMMU code on x86-64 should be never active unless Stephen
used IOMMU_DEBUG or iommu=force.  THat is because the tg3
is a 64bit capable device and should always use bypass.

> Since AMD promised me an Opteron system last year, but never
> made good on that promise, I've never been able to work on
> fixing this bug myself. :-/

You could just buy one ,-) Lowend A64s boards+cpus are *really* cheap
these days. And it will likely be much quieter than a noisy
original AMD box.  And the IOMMU works on that too if you force
it (normally it is never active because they dont have enough
memory to need remapping) 

-Andi

Re: Mystery packet killing tg3

[Peter Buckingham]

Andi Kleen wrote:
> "David S. Miller" <davem@davemloft.net> writes:
> 
>>This usually means that there is some DMA corruption.
>>For example, some bug in the x86_64 IOMMU code or similar
>>causes a bogus DMA address to be fed to the tg3 or even
>>worse a DMA mapping is unmapped before tg3 is actually
>>done with it.
> 
> 
> IOMMU code on x86-64 should be never active unless Stephen
> used IOMMU_DEBUG or iommu=force.  THat is because the tg3
> is a 64bit capable device and should always use bypass.

how is this detected? we have a board that is using a 32bit e1000 and we 
have to use iommu=force otherwise it doesn't do the right thing with > 
4GB of RAM.

peter

Re: Mystery packet killing tg3

[Stephen Hemminger]

On Wed, 04 May 2005 20:30:00 +0200
Andi Kleen <ak@muc.de> wrote:

> "David S. Miller" <davem@davemloft.net> writes:
> >
> > This usually means that there is some DMA corruption.
> > For example, some bug in the x86_64 IOMMU code or similar
> > causes a bogus DMA address to be fed to the tg3 or even
> > worse a DMA mapping is unmapped before tg3 is actually
> > done with it.
> 
> IOMMU code on x86-64 should be never active unless Stephen
> used IOMMU_DEBUG or iommu=force.  THat is because the tg3
> is a 64bit capable device and should always use bypass.
> 
> > Since AMD promised me an Opteron system last year, but never
> > made good on that promise, I've never been able to work on
> > fixing this bug myself. :-/
> 
> You could just buy one ,-) Lowend A64s boards+cpus are *really* cheap
> these days. And it will likely be much quieter than a noisy
> original AMD box.  And the IOMMU works on that too if you force
> it (normally it is never active because they dont have enough
> memory to need remapping) 
> 
> -Andi

FYI the config of the system:
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.12-rc3
# Mon May  2 13:31:20 2005
#
CONFIG_X86_64=y
CONFIG_64BIT=y
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_RWSEM_GENERIC_SPINLOCK=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_X86_CMPXCHG=y
CONFIG_EARLY_PRINTK=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y

#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_CLEAN_COMPILE=y
CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32

#
# General setup
#
CONFIG_LOCALVERSION="-netem"
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
# CONFIG_BSD_PROCESS_ACCT is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
CONFIG_HOTPLUG=y
CONFIG_KOBJECT_UEVENT=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
# CONFIG_CPUSETS is not set
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SHMEM=y
CONFIG_CC_ALIGN_FUNCTIONS=0
CONFIG_CC_ALIGN_LABELS=0
CONFIG_CC_ALIGN_LOOPS=0
CONFIG_CC_ALIGN_JUMPS=0
# CONFIG_TINY_SHMEM is not set
CONFIG_BASE_SMALL=0

#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
CONFIG_OBSOLETE_MODPARM=y
CONFIG_MODVERSIONS=y
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y
CONFIG_STOP_MACHINE=y

#
# Processor type and features
#
CONFIG_MK8=y
# CONFIG_MPSC is not set
# CONFIG_GENERIC_CPU is not set
CONFIG_X86_L1_CACHE_BYTES=64
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_TSC=y
CONFIG_X86_GOOD_APIC=y
CONFIG_MICROCODE=m
CONFIG_X86_MSR=m
CONFIG_X86_CPUID=m
CONFIG_X86_IO_APIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_MTRR=y
CONFIG_SMP=y
# CONFIG_PREEMPT is not set
# CONFIG_SCHED_SMT is not set
# CONFIG_K8_NUMA is not set
# CONFIG_NUMA_EMU is not set
# CONFIG_NUMA is not set
CONFIG_HAVE_DEC_LOCK=y
CONFIG_NR_CPUS=8
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_GART_IOMMU=y
CONFIG_SWIOTLB=y
CONFIG_X86_MCE=y
# CONFIG_X86_MCE_INTEL is not set
CONFIG_SECCOMP=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y

#
# Power management options
#
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
# CONFIG_SOFTWARE_SUSPEND is not set

#
# ACPI (Advanced Configuration and Power Interface) Support
#
CONFIG_ACPI=y
CONFIG_ACPI_BOOT=y
CONFIG_ACPI_INTERPRETER=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_SLEEP_PROC_FS=y
CONFIG_ACPI_AC=m
CONFIG_ACPI_BATTERY=m
CONFIG_ACPI_BUTTON=m
CONFIG_ACPI_VIDEO=m
CONFIG_ACPI_FAN=m
CONFIG_ACPI_PROCESSOR=m
CONFIG_ACPI_THERMAL=m
# CONFIG_ACPI_ASUS is not set
# CONFIG_ACPI_IBM is not set
# CONFIG_ACPI_TOSHIBA is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_BUS=y
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_PCI=y
CONFIG_ACPI_SYSTEM=y
# CONFIG_ACPI_CONTAINER is not set

#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
# CONFIG_UNORDERED_IO is not set
CONFIG_PCIEPORTBUS=y
CONFIG_PCI_MSI=y
# CONFIG_PCI_LEGACY_PROC is not set
# CONFIG_PCI_NAMES is not set
# CONFIG_PCI_DEBUG is not set

#
# PCCARD (PCMCIA/CardBus) support
#
# CONFIG_PCCARD is not set

#
# PCI Hotplug Support
#
# CONFIG_HOTPLUG_PCI is not set

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_MISC=m
CONFIG_IA32_EMULATION=y
CONFIG_IA32_AOUT=y
CONFIG_COMPAT=y
CONFIG_SYSVIPC_COMPAT=y
CONFIG_UID16=y

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=m
# CONFIG_DEBUG_DRIVER is not set

#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set

#
# Parallel port support
#
# CONFIG_PARPORT is not set

#
# Plug and Play support
#
CONFIG_PNP=y
# CONFIG_PNP_DEBUG is not set

#
# Protocols
#
CONFIG_PNPACPI=y

#
# Block devices
#
CONFIG_BLK_DEV_FD=m
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_NBD=m
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=128000
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
# CONFIG_LBD is not set
# CONFIG_CDROM_PKTCDVD is not set

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_ATA_OVER_ETH is not set

#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y

#
# Please see Documentation/ide.txt for help/info on IDE drives
#
# CONFIG_BLK_DEV_IDE_SATA is not set
# CONFIG_BLK_DEV_HD_IDE is not set
CONFIG_BLK_DEV_IDEDISK=y
CONFIG_IDEDISK_MULTI_MODE=y
CONFIG_BLK_DEV_IDECD=y
CONFIG_BLK_DEV_IDETAPE=m
CONFIG_BLK_DEV_IDEFLOPPY=y
CONFIG_BLK_DEV_IDESCSI=m
# CONFIG_IDE_TASK_IOCTL is not set

#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=y
# CONFIG_BLK_DEV_CMD640 is not set
CONFIG_BLK_DEV_IDEPNP=y
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_SHARE_IRQ=y
CONFIG_BLK_DEV_OFFBOARD=y
CONFIG_BLK_DEV_GENERIC=y
# CONFIG_BLK_DEV_OPTI621 is not set
CONFIG_BLK_DEV_RZ1000=y
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
CONFIG_IDEDMA_ONLYDISK=y
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not set
# CONFIG_BLK_DEV_AMD74XX is not set
# CONFIG_BLK_DEV_ATIIXP is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5520 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_SC1200 is not set
# CONFIG_BLK_DEV_PIIX is not set
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_BLK_DEV_SVWKS is not set
# CONFIG_BLK_DEV_SIIMAGE is not set
# CONFIG_BLK_DEV_SIS5513 is not set
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
# CONFIG_BLK_DEV_VIA82CXXX is not set
# CONFIG_IDE_ARM is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set

#
# SCSI device support
#
CONFIG_SCSI=y
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
# CONFIG_BLK_DEV_SR is not set
CONFIG_CHR_DEV_SG=m

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y

#
# SCSI Transport Attributes
#
CONFIG_SCSI_SPI_ATTRS=m
CONFIG_SCSI_FC_ATTRS=m
# CONFIG_SCSI_ISCSI_ATTRS is not set

#
# SCSI low-level drivers
#
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AACRAID is not set
CONFIG_SCSI_AIC7XXX=m
CONFIG_AIC7XXX_CMDS_PER_DEVICE=32
CONFIG_AIC7XXX_RESET_DELAY_MS=5000
# CONFIG_AIC7XXX_DEBUG_ENABLE is not set
CONFIG_AIC7XXX_DEBUG_MASK=0
CONFIG_AIC7XXX_REG_PRETTY_PRINT=y
# CONFIG_SCSI_AIC7XXX_OLD is not set
CONFIG_SCSI_AIC79XX=m
CONFIG_AIC79XX_CMDS_PER_DEVICE=32
CONFIG_AIC79XX_RESET_DELAY_MS=15000
# CONFIG_AIC79XX_ENABLE_RD_STRM is not set
# CONFIG_AIC79XX_DEBUG_ENABLE is not set
CONFIG_AIC79XX_DEBUG_MASK=0
CONFIG_AIC79XX_REG_PRETTY_PRINT=y
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
CONFIG_SCSI_SATA=y
# CONFIG_SCSI_SATA_AHCI is not set
# CONFIG_SCSI_SATA_SVW is not set
# CONFIG_SCSI_ATA_PIIX is not set
# CONFIG_SCSI_SATA_NV is not set
# CONFIG_SCSI_SATA_PROMISE is not set
# CONFIG_SCSI_SATA_QSTOR is not set
# CONFIG_SCSI_SATA_SX4 is not set
CONFIG_SCSI_SATA_SIL=y
# CONFIG_SCSI_SATA_SIS is not set
# CONFIG_SCSI_SATA_ULI is not set
# CONFIG_SCSI_SATA_VIA is not set
# CONFIG_SCSI_SATA_VITESSE is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_SCSI_DMX3191D is not set
CONFIG_SCSI_EATA=m
CONFIG_SCSI_EATA_TAGGED_QUEUE=y
CONFIG_SCSI_EATA_LINKED_COMMANDS=y
CONFIG_SCSI_EATA_MAX_TAGS=16
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_QLOGIC_FC is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
CONFIG_SCSI_QLA2XXX=y
# CONFIG_SCSI_QLA21XX is not set
# CONFIG_SCSI_QLA22XX is not set
# CONFIG_SCSI_QLA2300 is not set
# CONFIG_SCSI_QLA2322 is not set
# CONFIG_SCSI_QLA6312 is not set
# CONFIG_SCSI_LPFC is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_DEBUG is not set

#
# Multi-device support (RAID and LVM)
#
# CONFIG_MD is not set

#
# Fusion MPT device support
#
# CONFIG_FUSION is not set

#
# IEEE 1394 (FireWire) support
#
# CONFIG_IEEE1394 is not set

#
# I2O device support
#
# CONFIG_I2O is not set

#
# Networking support
#
CONFIG_NET=y

#
# Networking options
#
CONFIG_PACKET=m
CONFIG_PACKET_MMAP=y
CONFIG_UNIX=y
CONFIG_NET_KEY=m
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y
# CONFIG_IP_ROUTE_MULTIPATH_CACHED is not set
CONFIG_IP_ROUTE_VERBOSE=y
CONFIG_IP_PNP=y
CONFIG_IP_PNP_DHCP=y
CONFIG_IP_PNP_BOOTP=y
CONFIG_IP_PNP_RARP=y
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE=m
CONFIG_NET_IPGRE_BROADCAST=y
CONFIG_IP_MROUTE=y
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_IPCOMP=m
CONFIG_INET_TUNNEL=m
CONFIG_IP_TCPDIAG=y
# CONFIG_IP_TCPDIAG_IPV6 is not set
CONFIG_IPV6=m
# CONFIG_IPV6_PRIVACY is not set
CONFIG_INET6_AH=m
CONFIG_INET6_ESP=m
CONFIG_INET6_IPCOMP=m
CONFIG_INET6_TUNNEL=m
# CONFIG_IPV6_TUNNEL is not set
# CONFIG_NETFILTER is not set
CONFIG_XFRM=y
# CONFIG_XFRM_USER is not set

#
# SCTP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_SCTP is not set
# CONFIG_ATM is not set
CONFIG_BRIDGE=m
CONFIG_VLAN_8021Q=m
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set

#
# QoS and/or fair queueing
#
CONFIG_NET_SCHED=y
# CONFIG_NET_SCH_CLK_JIFFIES is not set
# CONFIG_NET_SCH_CLK_GETTIMEOFDAY is not set
CONFIG_NET_SCH_CLK_CPU=y
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_HFSC=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_DSMARK=m
CONFIG_NET_SCH_NETEM=m
CONFIG_NET_SCH_INGRESS=m
CONFIG_NET_QOS=y
CONFIG_NET_ESTIMATOR=y
CONFIG_NET_CLS=y
# CONFIG_NET_CLS_BASIC is not set
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_ROUTE=y
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
CONFIG_CLS_U32_PERF=y
CONFIG_NET_CLS_IND=y
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
CONFIG_NET_EMATCH=y
CONFIG_NET_EMATCH_STACK=32
CONFIG_NET_EMATCH_CMP=m
CONFIG_NET_EMATCH_NBYTE=m
CONFIG_NET_EMATCH_U32=m
CONFIG_NET_EMATCH_META=m
CONFIG_NET_CLS_ACT=y
CONFIG_NET_ACT_POLICE=m
CONFIG_NET_ACT_GACT=m
CONFIG_GACT_PROB=y
CONFIG_NET_ACT_MIRRED=m
CONFIG_NET_ACT_PEDIT=m

#
# Network testing
#
CONFIG_NET_PKTGEN=m
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
CONFIG_TUN=y
# CONFIG_NET_SB1000 is not set

#
# ARCnet devices
#
# CONFIG_ARCNET is not set

#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
CONFIG_MII=m
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
# CONFIG_NET_VENDOR_3COM is not set

#
# Tulip family network device support
#
# CONFIG_NET_TULIP is not set
# CONFIG_HP100 is not set
CONFIG_NET_PCI=y
# CONFIG_PCNET32 is not set
# CONFIG_AMD8111_ETH is not set
# CONFIG_ADAPTEC_STARFIRE is not set
CONFIG_B44=m
# CONFIG_FORCEDETH is not set
# CONFIG_DGRS is not set
# CONFIG_EEPRO100 is not set
# CONFIG_E100 is not set
# CONFIG_FEALNX is not set
# CONFIG_NATSEMI is not set
# CONFIG_NE2K_PCI is not set
# CONFIG_8139CP is not set
# CONFIG_8139TOO is not set
# CONFIG_SIS900 is not set
# CONFIG_EPIC100 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_VIA_RHINE is not set

#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
CONFIG_R8169=m
CONFIG_R8169_NAPI=y
# CONFIG_R8169_VLAN is not set
# CONFIG_SK98LIN is not set
# CONFIG_VIA_VELOCITY is not set
CONFIG_TIGON3=m

#
# Ethernet (10000 Mbit)
#
# CONFIG_IXGB is not set
# CONFIG_S2IO is not set

#
# Token Ring devices
#
# CONFIG_TR is not set

#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set

#
# Wan interfaces
#
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
# CONFIG_SHAPER is not set
# CONFIG_NETCONSOLE is not set

#
# ISDN subsystem
#
# CONFIG_ISDN is not set

#
# Telephony Support
#
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_TSDEV is not set
CONFIG_INPUT_EVDEV=m
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_PCSPKR=y
CONFIG_INPUT_UINPUT=m

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
# CONFIG_GAMEPORT is not set
CONFIG_SOUND_GAMEPORT=y

#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_SERIAL_NONSTANDARD is not set

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_ACPI=y
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
# CONFIG_SERIAL_8250_MANY_PORTS is not set
CONFIG_SERIAL_8250_SHARE_IRQ=y
CONFIG_SERIAL_8250_DETECT_IRQ=y
# CONFIG_SERIAL_8250_MULTIPORT is not set
# CONFIG_SERIAL_8250_RSA is not set

#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
# CONFIG_SERIAL_JSM is not set
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256

#
# IPMI
#
# CONFIG_IPMI_HANDLER is not set

#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not set
CONFIG_HW_RANDOM=m
CONFIG_NVRAM=m
CONFIG_RTC=y
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set

#
# Ftape, the floppy tape device driver
#
CONFIG_AGP=y
CONFIG_AGP_AMD64=y
CONFIG_AGP_INTEL=m
# CONFIG_DRM is not set
# CONFIG_MWAVE is not set
# CONFIG_RAW_DRIVER is not set
# CONFIG_HPET is not set
# CONFIG_HANGCHECK_TIMER is not set

#
# TPM devices
#
# CONFIG_TCG_TPM is not set

#
# I2C support
#
CONFIG_I2C=m
# CONFIG_I2C_CHARDEV is not set

#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=m
# CONFIG_I2C_ALGOPCF is not set
# CONFIG_I2C_ALGOPCA is not set

#
# I2C Hardware Bus support
#
CONFIG_I2C_ALI1535=m
# CONFIG_I2C_ALI1563 is not set
CONFIG_I2C_ALI15X3=m
CONFIG_I2C_AMD756=m
# CONFIG_I2C_AMD756_S4882 is not set
CONFIG_I2C_AMD8111=m
CONFIG_I2C_I801=m
CONFIG_I2C_I810=m
# CONFIG_I2C_PIIX4 is not set
CONFIG_I2C_ISA=m
CONFIG_I2C_NFORCE2=m
CONFIG_I2C_PARPORT_LIGHT=m
CONFIG_I2C_PROSAVAGE=m
CONFIG_I2C_SAVAGE4=m
CONFIG_SCx200_ACB=m
CONFIG_I2C_SIS5595=m
CONFIG_I2C_SIS630=m
CONFIG_I2C_SIS96X=m
# CONFIG_I2C_STUB is not set
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m
CONFIG_I2C_VOODOO3=m
# CONFIG_I2C_PCA_ISA is not set

#
# Hardware Sensors Chip support
#
# CONFIG_I2C_SENSOR is not set
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ADM1025 is not set
# CONFIG_SENSORS_ADM1026 is not set
# CONFIG_SENSORS_ADM1031 is not set
# CONFIG_SENSORS_ASB100 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_FSCHER is not set
# CONFIG_SENSORS_FSCPOS is not set
# CONFIG_SENSORS_GL518SM is not set
# CONFIG_SENSORS_GL520SM is not set
# CONFIG_SENSORS_IT87 is not set
# CONFIG_SENSORS_LM63 is not set
# CONFIG_SENSORS_LM75 is not set
# CONFIG_SENSORS_LM77 is not set
# CONFIG_SENSORS_LM78 is not set
# CONFIG_SENSORS_LM80 is not set
# CONFIG_SENSORS_LM83 is not set
# CONFIG_SENSORS_LM85 is not set
# CONFIG_SENSORS_LM87 is not set
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_LM92 is not set
# CONFIG_SENSORS_MAX1619 is not set
# CONFIG_SENSORS_PC87360 is not set
# CONFIG_SENSORS_SMSC47B397 is not set
# CONFIG_SENSORS_SIS5595 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_VIA686A is not set
# CONFIG_SENSORS_W83781D is not set
# CONFIG_SENSORS_W83L785TS is not set
# CONFIG_SENSORS_W83627HF is not set

#
# Other I2C Chip support
#
# CONFIG_SENSORS_DS1337 is not set
# CONFIG_SENSORS_EEPROM is not set
# CONFIG_SENSORS_PCF8574 is not set
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_SENSORS_RTC8564 is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set

#
# Dallas's 1-wire bus
#
# CONFIG_W1 is not set

#
# Misc devices
#
# CONFIG_IBM_ASM is not set

#
# Multimedia devices
#
CONFIG_VIDEO_DEV=m

#
# Video For Linux
#

#
# Video Adapters
#
# CONFIG_VIDEO_BT848 is not set
# CONFIG_VIDEO_CPIA is not set
# CONFIG_VIDEO_SAA5246A is not set
# CONFIG_VIDEO_SAA5249 is not set
# CONFIG_TUNER_3036 is not set
# CONFIG_VIDEO_STRADIS is not set
# CONFIG_VIDEO_ZORAN is not set
# CONFIG_VIDEO_SAA7134 is not set
# CONFIG_VIDEO_MXB is not set
# CONFIG_VIDEO_DPC is not set
# CONFIG_VIDEO_HEXIUM_ORION is not set
# CONFIG_VIDEO_HEXIUM_GEMINI is not set
# CONFIG_VIDEO_CX88 is not set
# CONFIG_VIDEO_OVCAMCHIP is not set

#
# Radio Adapters
#
# CONFIG_RADIO_GEMTEK_PCI is not set
# CONFIG_RADIO_MAXIRADIO is not set
# CONFIG_RADIO_MAESTRO is not set

#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set

#
# Graphics support
#
CONFIG_FB=y
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
CONFIG_FB_SOFT_CURSOR=y
# CONFIG_FB_MACMODES is not set
CONFIG_FB_MODE_HELPERS=y
# CONFIG_FB_TILEBLITTING is not set
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
CONFIG_FB_VGA16=m
CONFIG_FB_VESA=y
CONFIG_VIDEO_SELECT=y
# CONFIG_FB_HGA is not set
# CONFIG_FB_NVIDIA is not set
# CONFIG_FB_RIVA is not set
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON_OLD is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_SAVAGE is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_GEODE is not set
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_VIRTUAL is not set

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y

#
# Logo configuration
#
# CONFIG_LOGO is not set
# CONFIG_BACKLIGHT_LCD_SUPPORT is not set

#
# Sound
#
CONFIG_SOUND=m

#
# Advanced Linux Sound Architecture
#
CONFIG_SND=m
CONFIG_SND_TIMER=m
CONFIG_SND_PCM=m
CONFIG_SND_RAWMIDI=m
CONFIG_SND_SEQUENCER=m
CONFIG_SND_SEQ_DUMMY=m
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=m
CONFIG_SND_PCM_OSS=m
CONFIG_SND_SEQUENCER_OSS=y
CONFIG_SND_RTCTIMER=m
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set

#
# Generic devices
#
CONFIG_SND_MPU401_UART=m
CONFIG_SND_DUMMY=m
CONFIG_SND_VIRMIDI=m
CONFIG_SND_MTPAV=m
CONFIG_SND_SERIAL_U16550=m
CONFIG_SND_MPU401=m

#
# PCI devices
#
CONFIG_SND_AC97_CODEC=m
# CONFIG_SND_ALI5451 is not set
# CONFIG_SND_ATIIXP is not set
# CONFIG_SND_ATIIXP_MODEM is not set
# CONFIG_SND_AU8810 is not set
# CONFIG_SND_AU8820 is not set
# CONFIG_SND_AU8830 is not set
# CONFIG_SND_AZT3328 is not set
# CONFIG_SND_BT87X is not set
# CONFIG_SND_CS46XX is not set
# CONFIG_SND_CS4281 is not set
# CONFIG_SND_EMU10K1 is not set
# CONFIG_SND_EMU10K1X is not set
# CONFIG_SND_CA0106 is not set
# CONFIG_SND_KORG1212 is not set
# CONFIG_SND_MIXART is not set
# CONFIG_SND_NM256 is not set
# CONFIG_SND_RME32 is not set
# CONFIG_SND_RME96 is not set
# CONFIG_SND_RME9652 is not set
# CONFIG_SND_HDSP is not set
# CONFIG_SND_TRIDENT is not set
# CONFIG_SND_YMFPCI is not set
# CONFIG_SND_ALS4000 is not set
# CONFIG_SND_CMIPCI is not set
# CONFIG_SND_ENS1370 is not set
# CONFIG_SND_ENS1371 is not set
# CONFIG_SND_ES1938 is not set
# CONFIG_SND_ES1968 is not set
# CONFIG_SND_MAESTRO3 is not set
# CONFIG_SND_FM801 is not set
# CONFIG_SND_ICE1712 is not set
# CONFIG_SND_ICE1724 is not set
CONFIG_SND_INTEL8X0=m
# CONFIG_SND_INTEL8X0M is not set
# CONFIG_SND_SONICVIBES is not set
# CONFIG_SND_VIA82XX is not set
# CONFIG_SND_VIA82XX_MODEM is not set
# CONFIG_SND_VX222 is not set
# CONFIG_SND_HDA_INTEL is not set

#
# USB devices
#
CONFIG_SND_USB_AUDIO=m
# CONFIG_SND_USB_USX2Y is not set

#
# Open Sound System
#
# CONFIG_SOUND_PRIME is not set

#
# USB support
#
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB=m
# CONFIG_USB_DEBUG is not set

#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
# CONFIG_USB_BANDWIDTH is not set
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_SUSPEND is not set
# CONFIG_USB_OTG is not set

#
# USB Host Controller Drivers
#
CONFIG_USB_EHCI_HCD=m
CONFIG_USB_EHCI_SPLIT_ISO=y
# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
CONFIG_USB_OHCI_HCD=m
# CONFIG_USB_OHCI_BIG_ENDIAN is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_UHCI_HCD=m
# CONFIG_USB_SL811_HCD is not set

#
# USB Device Class drivers
#
# CONFIG_USB_AUDIO is not set
# CONFIG_USB_BLUETOOTH_TTY is not set
# CONFIG_USB_MIDI is not set
# CONFIG_USB_ACM is not set
# CONFIG_USB_PRINTER is not set

#
# NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support' may also be needed; see USB_STORAGE Help for more information
#
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
CONFIG_USB_STORAGE_DATAFAB=y
CONFIG_USB_STORAGE_FREECOM=y
CONFIG_USB_STORAGE_ISD200=y
CONFIG_USB_STORAGE_DPCM=y
CONFIG_USB_STORAGE_USBAT=y
CONFIG_USB_STORAGE_SDDR09=y
CONFIG_USB_STORAGE_SDDR55=y
CONFIG_USB_STORAGE_JUMPSHOT=y

#
# USB Input Devices
#
CONFIG_USB_HID=m
CONFIG_USB_HIDINPUT=y
CONFIG_HID_FF=y
CONFIG_HID_PID=y
CONFIG_LOGITECH_FF=y
CONFIG_THRUSTMASTER_FF=y
CONFIG_USB_HIDDEV=y

#
# USB HID Boot Protocol drivers
#
# CONFIG_USB_KBD is not set
# CONFIG_USB_MOUSE is not set
# CONFIG_USB_AIPTEK is not set
# CONFIG_USB_WACOM is not set
# CONFIG_USB_KBTAB is not set
# CONFIG_USB_POWERMATE is not set
# CONFIG_USB_MTOUCH is not set
# CONFIG_USB_EGALAX is not set
# CONFIG_USB_XPAD is not set
# CONFIG_USB_ATI_REMOTE is not set

#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set

#
# USB Multimedia devices
#
# CONFIG_USB_DABUSB is not set
# CONFIG_USB_VICAM is not set
# CONFIG_USB_DSBR is not set
# CONFIG_USB_IBMCAM is not set
# CONFIG_USB_KONICAWC is not set
# CONFIG_USB_OV511 is not set
# CONFIG_USB_SE401 is not set
# CONFIG_USB_SN9C102 is not set
# CONFIG_USB_STV680 is not set
# CONFIG_USB_PWC is not set

#
# USB Network Adapters
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_USBNET is not set
CONFIG_USB_MON=m

#
# USB port drivers
#

#
# USB Serial Converter support
#
CONFIG_USB_SERIAL=m
CONFIG_USB_SERIAL_GENERIC=y
# CONFIG_USB_SERIAL_BELKIN is not set
# CONFIG_USB_SERIAL_DIGI_ACCELEPORT is not set
# CONFIG_USB_SERIAL_CP2101 is not set
# CONFIG_USB_SERIAL_CYPRESS_M8 is not set
# CONFIG_USB_SERIAL_EMPEG is not set
# CONFIG_USB_SERIAL_FTDI_SIO is not set
CONFIG_USB_SERIAL_VISOR=m
CONFIG_USB_SERIAL_IPAQ=m
# CONFIG_USB_SERIAL_IR is not set
# CONFIG_USB_SERIAL_EDGEPORT is not set
# CONFIG_USB_SERIAL_EDGEPORT_TI is not set
# CONFIG_USB_SERIAL_GARMIN is not set
# CONFIG_USB_SERIAL_IPW is not set
# CONFIG_USB_SERIAL_KEYSPAN_PDA is not set
# CONFIG_USB_SERIAL_KEYSPAN is not set
# CONFIG_USB_SERIAL_KLSI is not set
# CONFIG_USB_SERIAL_KOBIL_SCT is not set
# CONFIG_USB_SERIAL_MCT_U232 is not set
# CONFIG_USB_SERIAL_PL2303 is not set
# CONFIG_USB_SERIAL_SAFE is not set
# CONFIG_USB_SERIAL_TI is not set
# CONFIG_USB_SERIAL_CYBERJACK is not set
# CONFIG_USB_SERIAL_XIRCOM is not set
# CONFIG_USB_SERIAL_OMNINET is not set

#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_AUERSWALD is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_PHIDGETKIT is not set
# CONFIG_USB_PHIDGETSERVO is not set
# CONFIG_USB_IDMOUSE is not set
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_TEST is not set

#
# USB ATM/DSL drivers
#

#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set

#
# MMC/SD Card support
#
# CONFIG_MMC is not set

#
# InfiniBand support
#
# CONFIG_INFINIBAND is not set

#
# Firmware Drivers
#
# CONFIG_EDD is not set

#
# File systems
#
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_JBD=y
CONFIG_JBD_DEBUG=y
CONFIG_FS_MBCACHE=y
CONFIG_REISERFS_FS=m
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
CONFIG_REISERFS_FS_XATTR=y
CONFIG_REISERFS_FS_POSIX_ACL=y
CONFIG_REISERFS_FS_SECURITY=y
# CONFIG_JFS_FS is not set
CONFIG_FS_POSIX_ACL=y

#
# XFS support
#
# CONFIG_XFS_FS is not set
# CONFIG_MINIX_FS is not set
CONFIG_ROMFS_FS=m
# CONFIG_QUOTA is not set
CONFIG_DNOTIFY=y
CONFIG_AUTOFS_FS=m
CONFIG_AUTOFS4_FS=m

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_ZISOFS_FS=y
CONFIG_UDF_FS=m
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
# CONFIG_NTFS_FS is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
# CONFIG_DEVFS_FS is not set
CONFIG_DEVPTS_FS_XATTR=y
CONFIG_DEVPTS_FS_SECURITY=y
CONFIG_TMPFS=y
# CONFIG_TMPFS_XATTR is not set
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_RAMFS=y

#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set

#
# Network File Systems
#
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
# CONFIG_NFS_V4 is not set
CONFIG_NFS_DIRECTIO=y
CONFIG_NFSD=m
CONFIG_NFSD_V3=y
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
# CONFIG_ROOT_NFS is not set
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=m
CONFIG_SUNRPC=y
# CONFIG_RPCSEC_GSS_KRB5 is not set
# CONFIG_RPCSEC_GSS_SPKM3 is not set
# CONFIG_SMB_FS is not set
# CONFIG_CIFS is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set

#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y

#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=m
CONFIG_NLS_CODEPAGE_737=m
CONFIG_NLS_CODEPAGE_775=m
CONFIG_NLS_CODEPAGE_850=m
CONFIG_NLS_CODEPAGE_852=m
CONFIG_NLS_CODEPAGE_855=m
CONFIG_NLS_CODEPAGE_857=m
CONFIG_NLS_CODEPAGE_860=m
CONFIG_NLS_CODEPAGE_861=m
CONFIG_NLS_CODEPAGE_862=m
CONFIG_NLS_CODEPAGE_863=m
CONFIG_NLS_CODEPAGE_864=m
CONFIG_NLS_CODEPAGE_865=m
CONFIG_NLS_CODEPAGE_866=m
CONFIG_NLS_CODEPAGE_869=m
CONFIG_NLS_CODEPAGE_936=m
CONFIG_NLS_CODEPAGE_950=m
CONFIG_NLS_CODEPAGE_932=m
CONFIG_NLS_CODEPAGE_949=m
CONFIG_NLS_CODEPAGE_874=m
CONFIG_NLS_ISO8859_8=m
CONFIG_NLS_CODEPAGE_1250=m
CONFIG_NLS_CODEPAGE_1251=m
# CONFIG_NLS_ASCII is not set
CONFIG_NLS_ISO8859_1=m
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=m
CONFIG_NLS_ISO8859_4=m
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=m
CONFIG_NLS_ISO8859_7=m
CONFIG_NLS_ISO8859_9=m
CONFIG_NLS_ISO8859_13=m
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=m
CONFIG_NLS_KOI8_R=m
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_UTF8=m

#
# Profiling support
#
CONFIG_PROFILING=y
CONFIG_OPROFILE=m

#
# Kernel hacking
#
# CONFIG_PRINTK_TIME is not set
CONFIG_DEBUG_KERNEL=y
CONFIG_MAGIC_SYSRQ=y
CONFIG_LOG_BUF_SHIFT=18
CONFIG_SCHEDSTATS=y
# CONFIG_DEBUG_SLAB is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_FS=y
# CONFIG_INIT_DEBUG is not set
# CONFIG_IOMMU_DEBUG is not set
CONFIG_KPROBES=y

#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY is not set

#
# Cryptographic options
#
CONFIG_CRYPTO=y
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_SHA1=m
CONFIG_CRYPTO_SHA256=m
CONFIG_CRYPTO_SHA512=m
# CONFIG_CRYPTO_WP512 is not set
# CONFIG_CRYPTO_TGR192 is not set
CONFIG_CRYPTO_DES=y
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_AES=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST6=m
# CONFIG_CRYPTO_TEA is not set
CONFIG_CRYPTO_ARC4=m
# CONFIG_CRYPTO_KHAZAD is not set
CONFIG_CRYPTO_ANUBIS=m
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_MICHAEL_MIC=m
# CONFIG_CRYPTO_CRC32C is not set
CONFIG_CRYPTO_TEST=m

#
# Hardware crypto devices
#

#
# Library routines
#
CONFIG_CRC_CCITT=m
CONFIG_CRC32=y
# CONFIG_LIBCRC32C is not set
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=m

Re: Mystery packet killing tg3

[Michael Chan]

On Wed, 2005-05-04 at 15:51 -0700, Stephen Hemminger wrote:
> On Tue, 3 May 2005 23:27:38 -0700
> "Michael Chan" <mchan@broadcom.com> wrote:
> 
> > I think stop_block errors can only happen during dev_close, suspend, netdev
> > watchdog, or ethtool "set" calls.
> 
> It seems that dhclient was failing on bootup then taking the device down.
> When the device down happened it got the tg3_stop_block from
>  Call Trace:<ffffffff880178a9>{:tg3:tg3_stop_block+185} <ffffffff88017b1d>{:tg3:tg3_abort_hw+605}
>        <ffffffff88018178>{:tg3:tg3_halt+40} <ffffffff88019827>{:tg3:tg3_close+71}
>        <ffffffff802c1734>{dev_close+100} <ffffffff802c2b68>{dev_change_flags+104}
>        <ffffffff80300e35>{devinet_ioctl+773} <ffffffff803022dc>{inet_ioctl+92}
>        <ffffffff802b82bc>{sock_ioctl+556} <ffffffff801898fa>{do_ioctl+58}
>        <ffffffff80189c2b>{vfs_ioctl+715} <ffffffff80189cad>{sys_ioctl+77}
>        <ffffffff8010ea66>{system_call+126} 
> 

This makes sense. Before the dhclient closes the device, was the device
functioning properly? If not, was it not sending or not receiving?

If the device was functioning properly prior to the close, meaning that
the dhclient was closing because there was no response from the dhcp
server, then the stop_block error was inconsequential.

Re: Mystery packet killing tg3

[Stephen Hemminger]

On Tue, 3 May 2005 23:27:38 -0700
"Michael Chan" <mchan@broadcom.com> wrote:

> Stephen Hemminger wrote:
> 
> > Initially, it reproduced everytime link came up, we 
> > reconfigured the VLAN to
> > have a mirror port into a laptop to try and capture what was 
> > happening, but when
> > we did that the bootup problem went away. It was in the tg3_reset_hw
> > during initial dev_open.
> > 
> 
> During initial dev_open, the TG3_FLAG_INIT_COMPLETE flag is not set so
> tg3_reset_hw() should not call tg3_abort_hw() where the stop_block calls are
> made. So there should be no stop_block errors.
> 
> I think stop_block errors can only happen during dev_close, suspend, netdev
> watchdog, or ethtool "set" calls.

It seems that dhclient was failing on bootup then taking the device down.
When the device down happened it got the tg3_stop_block from
 Call Trace:<ffffffff880178a9>{:tg3:tg3_stop_block+185} <ffffffff88017b1d>{:tg3:tg3_abort_hw+605}
       <ffffffff88018178>{:tg3:tg3_halt+40} <ffffffff88019827>{:tg3:tg3_close+71}
       <ffffffff802c1734>{dev_close+100} <ffffffff802c2b68>{dev_change_flags+104}
       <ffffffff80300e35>{devinet_ioctl+773} <ffffffff803022dc>{inet_ioctl+92}
       <ffffffff802b82bc>{sock_ioctl+556} <ffffffff801898fa>{do_ioctl+58}
       <ffffffff80189c2b>{vfs_ioctl+715} <ffffffff80189cad>{sys_ioctl+77}
       <ffffffff8010ea66>{system_call+126} 

Re: Mystery packet killing tg3

[Andi Kleen]

> >IOMMU code on x86-64 should be never active unless Stephen
> >used IOMMU_DEBUG or iommu=force.  THat is because the tg3
> >is a 64bit capable device and should always use bypass.
> 
> how is this detected? we have a board that is using a 32bit e1000 and we 
> have to use iommu=force otherwise it doesn't do the right thing with > 
> 4GB of RAM.

"32bit e1000"? How did you get such a beast? AFAIK all e1000s are 64bit
address capable. Please supply a full boot log without iommu=force 
and describe what happens exactly.



-Andi

Re: Mystery packet killing tg3

[Stephen Hemminger]

Andi Kleen wrote:

>>>IOMMU code on x86-64 should be never active unless Stephen
>>>used IOMMU_DEBUG or iommu=force.  THat is because the tg3
>>>is a 64bit capable device and should always use bypass.
>>>      
>>>
>>how is this detected? we have a board that is using a 32bit e1000 and we 
>>have to use iommu=force otherwise it doesn't do the right thing with > 
>>4GB of RAM.
>>    
>>
>
>"32bit e1000"? How did you get such a beast? AFAIK all e1000s are 64bit
>address capable. Please supply a full boot log without iommu=force 
>and describe what happens exactly.
>  
>
One of the osdl e1000's in the lab is an IBM rebranded card that although
it uses a 64bit slot, is really only 32bit

Re: Mystery packet killing tg3

[Peter Buckingham]

Andi Kleen wrote:
> "32bit e1000"? How did you get such a beast? AFAIK all e1000s are 64bit
> address capable. Please supply a full boot log without iommu=force 
> and describe what happens exactly.

that was my initial impression too :-(

basically what happens is when there is more that 4GB of RAM in this 
system packets will start disappearing. ie ping will drop packets. 
Initially our bios was not configuring the IOMMU correctly, that has 
changed now.

I can make it work without the iommu=force by forcing the DMA to be 
32bit in the initialisation, but this seems to be a bit of a hack..

I've attached a dmesg output from a while ago (you may remember it from 
when i was tracking down a serial console problem ;-)

peter

---
Linux version 2.6.8-24.11-smp (geeko@buildhost) (gcc version 3.3.3 (SuSE 
Linux)) #2 SMP Wed Mar 16 09:22:34 PST 2005
BIOS-provided physical RAM map:
  BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
  BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
  BIOS-e820: 00000000000e6000 - 0000000000100000 (reserved)
  BIOS-e820: 0000000000100000 - 00000000b6ff0000 (usable)
  BIOS-e820: 00000000b6ff0000 - 00000000b6ffe000 (ACPI data)
  BIOS-e820: 00000000b6ffe000 - 00000000b7000000 (ACPI NVS)
  BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
  BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
  BIOS-e820: 00000000ff780000 - 0000000100000000 (reserved)
  BIOS-e820: 0000000100000000 - 0000000200000000 (usable)
Scanning NUMA topology in Northbridge 24
Number of nodes 4 (30030)
Node 0 MemBase 0000000000000000 Limit 000000007fffffff
Node 1 MemBase 0000000080000000 Limit 00000000ffffffff
Node 2 MemBase 0000000100000000 Limit 000000017fffffff
Node 3 MemBase 0000000180000000 Limit 00000001ffffffff
node 1 shift 24 addr ff000000 conflict 0
node 3 shift 25 addr 1fe000000 conflict 0
Using node hash shift of 26
Bootmem setup node 0 0000000000000000-000000007fffffff
Bootmem setup node 1 0000000080000000-00000000ffffffff
Bootmem setup node 2 0000000100000000-000000017fffffff
Bootmem setup node 3 0000000180000000-00000001ffffffff
No mptable found.
NVidia chipset found. Disabling timer override
ACPI: RSDP (v000 ACPIAM                                    ) @ 
0x00000000000f8510
ACPI: RSDT (v001 A M I  OEMRSDT  0x03000509 MSFT 0x00000097) @ 
0x00000000b6ff0000
ACPI: FADT (v002 A M I  OEMFACP  0x03000509 MSFT 0x00000097) @ 
0x00000000b6ff0200
ACPI: MADT (v001 A M I  OEMAPIC  0x03000509 MSFT 0x00000097) @ 
0x00000000b6ff0390
ACPI: OEMB (v001 A M I  AMI_OEM  0x03000509 MSFT 0x00000097) @ 
0x00000000b6ffe040
ACPI: MCFG (v001 A M I  OEMMCFG  0x03000509 MSFT 0x00000097) @ 
0x00000000b6ff65e0
ACPI: DSDT (v001  0ABGS 0ABGS020 0x00000020 INTL 0x02002026) @ 
0x0000000000000000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Processor #1 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
Processor #2 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
Processor #3 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x84] disabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x85] disabled)
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x86] disabled)
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x87] disabled)
ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 4, version 17, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Using ACPI (MADT) for SMP configuration information
Checking aperture...
CPU 0: aperture @ 16e0000000 size 64 MB
Aperture from northbridge cpu 0 beyond 4GB. Ignoring.
No AGP bridge found
Your BIOS doesn't leave a aperture memory hole
Please enable the IOMMU option in the BIOS setup
This costs you 64 MB of RAM
Mapping aperture over 65536 KB of RAM @ 4000000
Built 4 zonelists
Kernel command line: ip=dhcp nfsroot=10.2.128.1:/discovery iommu=force 
console=tty0 console=ttyS1,115200 BOOT_IMAGE=vmlinuz 
ip=10.2.135.253:10.2.128.1:0.0.0.0:255.255.128.0
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 131072 bytes)
time.c: Using 1.193182 MHz PIT timer.
time.c: Detected 2000.015 MHz processor.
Console: colour VGA+ 80x25
Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes)
Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Memory: 6978140k/8388608k available (3860k kernel code, 0k reserved, 
2106k data, 240k init)
Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
Using local APIC NMI watchdog using perfctr0
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU0: AMD Opteron(tm) Processor 846 HE stepping 0a
per-CPU timeslice cutoff: 1023.93 usecs.
task migration cache decay timeout: 2 msecs.
Booting processor 1/1 rip 6000 rsp 10101c3ff58
Initializing CPU#1
3940.35 BogoMIPS (lpj=1970176)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 846 HE stepping 0a
Booting processor 2/2 rip 6000 rsp 1017ffa5f58
Initializing CPU#2
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 846 HE stepping 0a
Booting processor 3/3 rip 6000 rsp 101fffb1f58
Initializing CPU#3
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 846 HE stepping 0a
Total of 4 processors activated (15785.98 BogoMIPS).
Using local APIC timer interrupts.
Detected 12.500 MHz APIC timer.
checking TSC synchronization across 4 CPUs: passed.
time.c: Using PIT/TSC based timekeeping.
Brought up 4 CPUs
NET: Registered protocol family 16
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
ACPI: Subsystem revision 20040715
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (00:00)
PCI: Probing PCI hardware (bus 00)
PCI: Transparent bridge - 0000:00:09.0
ACPI: PCI Interrupt Link [LNKA] (IRQs 16 17 18 19) *10
ACPI: PCI Interrupt Link [LNKB] (IRQs 16 17 18 19) *9
ACPI: PCI Interrupt Link [LNKC] (IRQs 16 17 18 19) *11
ACPI: PCI Interrupt Link [LNKD] (IRQs 16 17 18 19) *0, disabled.
ACPI: PCI Interrupt Link [LNKE] (IRQs 16 17 18 19) *0, disabled.
ACPI: PCI Interrupt Link [LUS0] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LUS1] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LUS2] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LKLN] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LAUI] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LKMO] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LKSM] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LTID] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LTIE] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LATA] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LN2A] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN2B] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN2C] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN2D] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LK2N] (IRQs 44 45 46 47) *0, disabled.
ACPI: PCI Interrupt Link [LT5D] (IRQs 44 45 46 47) *0, disabled.
ACPI: PCI Interrupt Link [LT2E] (IRQs 44 45 46 47) *0, disabled.
ACPI: PCI Interrupt Link [LN3A] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN3B] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN3C] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN3D] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN4A] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN4B] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN4C] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN4D] (IRQs 40 41 42 43) *0, disabled.
SCSI subsystem initialized
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
ACPI: PCI Interrupt Link [LKSM] enabled at IRQ 22
ACPI: PCI interrupt 0000:00:01.1[A] -> GSI 22 (level, low) -> IRQ 177
ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 19
ACPI: PCI interrupt 0000:05:06.0[A] -> GSI 19 (level, low) -> IRQ 185
ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 18
ACPI: PCI interrupt 0000:05:07.0[A] -> GSI 18 (level, low) -> IRQ 193
ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 17
ACPI: PCI interrupt 0000:01:00.0[A] -> GSI 17 (level, low) -> IRQ 201
PCI-DMA: Disabling AGP.
PCI-DMA: aperture base @ 4000000 size 65536 KB

Kernel panic - not syncing: Cannot allocate iommu bitmap

Re: Mystery packet killing tg3

[Rick Jones]

Peter Buckingham wrote:
> Andi Kleen wrote:
> 
>> "32bit e1000"? How did you get such a beast? AFAIK all e1000s are 64bit
>> address capable. Please supply a full boot log without iommu=force and 
>> describe what happens exactly.
> 
> 
> that was my initial impression too :-(

IIRC the HP Itanium2 workstations (zx2000, zx6000) have a 32-bit Intel (e1000) 
GbE on them.  I have an idea what the name of the chip was, but will defer to 
any and all Intel types on the list :)

rick jones

Re: Mystery packet killing tg3

[Peter Buckingham]

Rick Jones wrote:
> IIRC the HP Itanium2 workstations (zx2000, zx6000) have a 32-bit Intel 
> (e1000) GbE on them.  I have an idea what the name of the chip was, but 
> will defer to any and all Intel types on the list :)

this is the one we're using. in the next generation of our system we'll 
definitely be using a 64bit part...

peter

0000:05:07.0 Ethernet controller: Intel Corp. 82541GI/PI Gigabit 
Ethernet Controller (rev 05)
         Subsystem: Giga-byte Technology: Unknown device 3000
         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- 
ParErr- Stepping- SERR+ FastB2B-
         Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium 
 >TAbort- <TAbort- <MAbort- >SERR- <PERR-
         Latency: 64 (63750ns min), cache line size 10
         Interrupt: pin A routed to IRQ 201
         Region 0: Memory at beda0000 (32-bit, non-prefetchable)
         Region 2: I/O ports at fc00 [size=64]
         Capabilities: [dc] Power Management version 2
                 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA 
PME(D0+,D1-,D2-,D3hot+,D3cold+)
                 Status: D0 PME-Enable- DSel=0 DScale=1 PME-
         Capabilities: [e4] PCI-X non-bridge device.
                 Command: DPERE- ERO+ RBC=0 OST=0
                 Status: Bus=0 Dev=0 Func=0 64bit- 133MHz- SCD- USC-, 
DC=simple, DMMRBC=0, DMOST=0, DMCRS=0, RSCEM-

Re: Mystery packet killing tg3

[John Heffner]

On Thursday 05 May 2005 01:32 pm, Rick Jones wrote:
> Peter Buckingham wrote:
> > Andi Kleen wrote:
> >> "32bit e1000"? How did you get such a beast? AFAIK all e1000s are 64bit
> >> address capable. Please supply a full boot log without iommu=force and
> >> describe what happens exactly.
> >
> > that was my initial impression too :-(
>
> IIRC the HP Itanium2 workstations (zx2000, zx6000) have a 32-bit Intel
> (e1000) GbE on them.  I have an idea what the name of the chip was, but
> will defer to any and all Intel types on the list :)
>
> rick jones

>From a zx2000:

0000:a0:03.0 Ethernet controller: Intel Corp. 82540EM Gigabit Ethernet 
Controller (rev 02)
        Subsystem: Hewlett-Packard Company: Unknown device 1274
        Flags: bus master, 66MHz, medium devsel, latency 128, IRQ 56
        Memory at 00000000d0000000 (32-bit, non-prefetchable) [size=128K]
        I/O ports at a080 [size=64]
        Capabilities: [dc] Power Management version 2
        Capabilities: [e4] PCI-X non-bridge device.
        Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 
Enable-

  -John

Re: Mystery packet killing tg3

[Andi Kleen]

On Thu, May 05, 2005 at 09:20:25AM -0700, Stephen Hemminger wrote:
> Andi Kleen wrote:
> 
> >>>IOMMU code on x86-64 should be never active unless Stephen
> >>>used IOMMU_DEBUG or iommu=force.  THat is because the tg3
> >>>is a 64bit capable device and should always use bypass.
> >>>      
> >>>
> >>how is this detected? we have a board that is using a 32bit e1000 and we 
> >>have to use iommu=force otherwise it doesn't do the right thing with > 
> >>4GB of RAM.
> >>    
> >>
> >
> >"32bit e1000"? How did you get such a beast? AFAIK all e1000s are 64bit
> >address capable. Please supply a full boot log without iommu=force 
> >and describe what happens exactly.
> >  
> >
> One of the osdl e1000's in the lab is an IBM rebranded card that although
> it uses a 64bit slot, is really only 32bit

Ok, but it still supports DAC. The PCI bus width is a different
thing from the size of the address registers.

What we are talking about here is just if the card can directly
address physical memory above 4GB. What width the bus is 
is a hardware internal thing that we do not really care about
in the IOMMU code.

-Andi

Re: Mystery packet killing tg3

[Andi Kleen]

On Thu, May 05, 2005 at 10:09:55AM -0700, Peter Buckingham wrote:
> Andi Kleen wrote:
> >"32bit e1000"? How did you get such a beast? AFAIK all e1000s are 64bit
> >address capable. Please supply a full boot log without iommu=force 
> >and describe what happens exactly.
> 
> that was my initial impression too :-(
> 
> basically what happens is when there is more that 4GB of RAM in this 
> system packets will start disappearing. ie ping will drop packets. 
> Initially our bios was not configuring the IOMMU correctly, that has 
> changed now.

The kernel should detect incorrect IOMMU setups and work around
it. So even if your BIOS screws it up completely it should
work (at the cost of some lost memory). That  is the theory
at least, but perhaps you discovered some new way to screw
it up that is not handled yet ,-)

> 
> I can make it work without the iommu=force by forcing the DMA to be 
> 32bit in the initialisation, but this seems to be a bit of a hack..

In what initialization exactly? YOu mean changing the pci_set_dma_mask()
in the e1000 driver?

> Checking aperture...
> CPU 0: aperture @ 16e0000000 size 64 MB
> Aperture from northbridge cpu 0 beyond 4GB. Ignoring.
> No AGP bridge found
> Your BIOS doesn't leave a aperture memory hole
> Please enable the IOMMU option in the BIOS setup
> This costs you 64 MB of RAM
> Mapping aperture over 65536 KB of RAM @ 4000000
We try to fix up the IOMMU. Normally it should work now.

> PCI-DMA: Disabling AGP.
> PCI-DMA: aperture base @ 4000000 size 65536 KB
> 
> Kernel panic - not syncing: Cannot allocate iommu bitmap

Hmm, that must be a different problem in the IOMMU code
not related to the e1000. Probably it got confused somehow.
Why did you get the idea the ethernet driver has anything to do with
it?


-Andi

Re: Mystery packet killing tg3

[Peter Buckingham]

Andi Kleen wrote:
> In what initialization exactly? YOu mean changing the pci_set_dma_mask()
> in the e1000 driver?

exactly.
>>Kernel panic - not syncing: Cannot allocate iommu bitmap
> 
> 
> Hmm, that must be a different problem in the IOMMU code
> not related to the e1000. Probably it got confused somehow.
> Why did you get the idea the ethernet driver has anything to do with
> it?

i don't think it has anything to do with the e1000 actually. we only see 
this when the serial console redirection is enabled. It was the only 
dmesg I had lying around that had the IOMMU not forced. We run our 
systems diskless so it's not easy to get the boot log when the ethernet 
device is not working correctly. I can try to get a system with a hard 
drive and give you the dmesg from that instead (probably more useful ;-)

peter

Re: Mystery packet killing tg3

[Andi Kleen]

On Thu, May 05, 2005 at 11:21:26AM -0700, Peter Buckingham wrote:
> Andi Kleen wrote:
> >In what initialization exactly? YOu mean changing the pci_set_dma_mask()
> >in the e1000 driver?
> 
> exactly.


That should be impossible. Or it sounds like a serious
hardware problem. DAC should normally always work with all e1000 AFAIK.

Most likely you have some hardware problem and it is somehow magically
worked around by IOMMU remapping. One difference is that
the remapping makes all IO slower, perhaps the changed timing
works around some bug.

Or there is a posting problem somewhere that is worked
around by the additional config accesses in IOMMU.

Have you contacted the e1000 maintainters? 

> >>Kernel panic - not syncing: Cannot allocate iommu bitmap
> >
> >
> >Hmm, that must be a different problem in the IOMMU code
> >not related to the e1000. Probably it got confused somehow.
> >Why did you get the idea the ethernet driver has anything to do with
> >it?
> 
> i don't think it has anything to do with the e1000 actually. we only see 
> this when the serial console redirection is enabled. It was the only 
> dmesg I had lying around that had the IOMMU not forced. We run our 
> systems diskless so it's not easy to get the boot log when the ethernet 
> device is not working correctly. I can try to get a system with a hard 
> drive and give you the dmesg from that instead (probably more useful ;-)

It sounds like something is wrong with your hardware.

-Andi

Re: Mystery packet killing tg3

[Peter Buckingham]

[-- Attachment #1: Type: text/plain, Size: 955 bytes --]

Andi Kleen wrote:
> That should be impossible. Or it sounds like a serious
> hardware problem. DAC should normally always work with all e1000 AFAIK.

okay.

I basically just force it to take a 32bit dma mask. I admit i'm a little 
clueless as to what this truely means. I had assumed that it would 
result in only dma'ing to an area below 4GB, but I hadn't really 
validated that assumption :-(

> Most likely you have some hardware problem and it is somehow magically
> worked around by IOMMU remapping. One difference is that
> the remapping makes all IO slower, perhaps the changed timing
> works around some bug.

this is always a possibility, can you suggest some ways of isolating 
this problem?

> Have you contacted the e1000 maintainters? 

not as yet, if you think that it is valuable i will.

i've attached a fresh dmesg from a system running off a hard drive. this 
may or may not give you more clues about this.

thanks for your help,

peter

[-- Attachment #2: iommu_dmesg.txt --]
[-- Type: text/plain, Size: 16528 bytes --]

Bootdata ok (command line is vga=normal root=/dev/hda3 console=tty0)
Linux version 2.6.8-24.11-smp (geeko@buildhost) (gcc version 3.3.3 (SuSE Linux)) #1 SMP Tue Mar 1 18:17:49 PST 2005
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e6000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 0000000097ff0000 (usable)
 BIOS-e820: 0000000097ff0000 - 0000000097ffe000 (ACPI data)
 BIOS-e820: 0000000097ffe000 - 0000000098000000 (ACPI NVS)
 BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000ff780000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000240000000 (usable)
Scanning NUMA topology in Northbridge 24
Number of nodes 8 (70070)
Node 0 MemBase 0000000000000000 Limit 000000007fffffff
Node 1 MemBase 0000000080000000 Limit 00000000ffffffff
Node 2 MemBase 0000000100000000 Limit 000000017fffffff
Node 3 MemBase 0000000180000000 Limit 00000001ffffffff
Skipping disabled node 4
Skipping disabled node 5
Node 6 MemBase 0000000200000000 Limit 000000023fffffff
Skipping disabled node 7
node 1 shift 24 addr ff000000 conflict 0
node 3 shift 25 addr 1fe000000 conflict 0
Using node hash shift of 26
Bootmem setup node 0 0000000000000000-000000007fffffff
Bootmem setup node 1 0000000080000000-00000000ffffffff
Bootmem setup node 2 0000000100000000-000000017fffffff
Bootmem setup node 3 0000000180000000-00000001ffffffff
Bootmem setup node 6 0000000200000000-000000023fffffff
No mptable found.
On node 0 totalpages: 524287
  DMA zone: 4096 pages, LIFO batch:1
  Normal zone: 520191 pages, LIFO batch:16
  HighMem zone: 0 pages, LIFO batch:1
On node 1 totalpages: 524287
  DMA zone: 0 pages, LIFO batch:1
  Normal zone: 524287 pages, LIFO batch:16
  HighMem zone: 0 pages, LIFO batch:1
On node 2 totalpages: 524287
  DMA zone: 0 pages, LIFO batch:1
  Normal zone: 524287 pages, LIFO batch:16
  HighMem zone: 0 pages, LIFO batch:1
On node 3 totalpages: 524287
  DMA zone: 0 pages, LIFO batch:1
  Normal zone: 524287 pages, LIFO batch:16
  HighMem zone: 0 pages, LIFO batch:1
On node 6 totalpages: 262143
  DMA zone: 0 pages, LIFO batch:1
  Normal zone: 262143 pages, LIFO batch:16
  HighMem zone: 0 pages, LIFO batch:1
NVidia chipset found. Disabling timer override
ACPI: RSDP (v000 ACPIAM                                    ) @ 0x00000000000f8700
ACPI: RSDT (v001 A M I  OEMRSDT  0x03000518 MSFT 0x00000097) @ 0x0000000097ff0000
ACPI: FADT (v002 A M I  OEMFACP  0x03000518 MSFT 0x00000097) @ 0x0000000097ff0200
ACPI: MADT (v001 A M I  OEMAPIC  0x03000518 MSFT 0x00000097) @ 0x0000000097ff0390
ACPI: OEMB (v001 A M I  AMI_OEM  0x03000518 MSFT 0x00000097) @ 0x0000000097ffe040
ACPI: MCFG (v001 A M I  OEMMCFG  0x03000518 MSFT 0x00000097) @ 0x0000000097ff65e0
ACPI: DSDT (v001  0ABGS 0ABGS023 0x00000023 INTL 0x02002026) @ 0x0000000000000000
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Processor #1 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
Processor #2 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
Processor #3 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x04] enabled)
Processor #4 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x05] enabled)
Processor #5 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x07] lapic_id[0x06] enabled)
Processor #6 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
Processor #7 15:5 APIC version 16
ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 8, version 17, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x09] address[0xbeeff000] gsi_base[24])
IOAPIC[1]: apic_id 9, version 17, address 0xbeeff000, GSI 24-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
Checking aperture...
CPU 0: aperture @ 98000000 size 32 MB
Aperture from northbridge cpu 0 too small (32 MB)
No AGP bridge found
Your BIOS doesn't leave a aperture memory hole
Please enable the IOMMU option in the BIOS setup
This costs you 64 MB of RAM
Mapping aperture over 65536 KB of RAM @ 4000000
Built 8 zonelists
Kernel command line: vga=normal root=/dev/hda3 console=tty0
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 131072 bytes)
time.c: Using 1.193182 MHz PIT timer.
time.c: Detected 2000.032 MHz processor.
Console: colour VGA+ 80x25
Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes)
Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Memory: 7505340k/9437184k available (3226k kernel code, 0k reserved, 1578k data, 228k init)
Calibrating delay loop... 3964.92 BogoMIPS (lpj=1982464)
Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
Using local APIC NMI watchdog using perfctr0
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU0: AMD Opteron(tm) Processor 846 HE stepping 0a
per-CPU timeslice cutoff: 1023.93 usecs.
task migration cache decay timeout: 2 msecs.
Booting processor 1/1 rip 6000 rsp 10202647f58
Initializing CPU#1
Calibrating delay loop... <7>spurious 8259A interrupt: IRQ7.
3940.35 BogoMIPS (lpj=1970176)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 846 HE stepping 0a
Booting processor 2/2 rip 6000 rsp 101fff93f58
Initializing CPU#2
Calibrating delay loop... 3940.35 BogoMIPS (lpj=1970176)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 846 HE stepping 0a
Booting processor 3/3 rip 6000 rsp 1023ff85f58
Initializing CPU#3
Calibrating delay loop... 3940.35 BogoMIPS (lpj=1970176)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 846 HE stepping 0a
Booting processor 4/4 rip 6000 rsp 10097fbbf58
Initializing CPU#4
Calibrating delay loop... 3940.35 BogoMIPS (lpj=1970176)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 846 HE stepping 0a
Booting processor 5/5 rip 6000 rsp 10181c49f58
Initializing CPU#5
Calibrating delay loop... 3940.35 BogoMIPS (lpj=1970176)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 846 HE stepping 0a
Booting processor 6/6 rip 6000 rsp 10008069f58
Initializing CPU#6
Calibrating delay loop... 3940.35 BogoMIPS (lpj=1970176)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 846 HE stepping 0a
Booting processor 7/7 rip 6000 rsp 10081c7df58
Initializing CPU#7
Calibrating delay loop... 3940.35 BogoMIPS (lpj=1970176)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 846 HE stepping 0a
Total of 8 processors activated (31547.39 BogoMIPS).
Using local APIC timer interrupts.
Detected 12.500 MHz APIC timer.
checking TSC synchronization across 8 CPUs: passed.
time.c: Using PIT/TSC based timekeeping.
Brought up 8 CPUs
NET: Registered protocol family 16
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
ACPI: Subsystem revision 20040715
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (00:00)
PCI: Probing PCI hardware (bus 00)
PCI: Transparent bridge - 0000:00:09.0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
ACPI: PCI Root Bridge [PCIC] (00:40)
PCI: Probing PCI hardware (bus 40)
ACPI: PCI Interrupt Routing Table [\_SB_.PCIC._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 16 17 18 19) *10
ACPI: PCI Interrupt Link [LNKB] (IRQs 16 17 18 19) *9
ACPI: PCI Interrupt Link [LNKC] (IRQs 16 17 18 19) *11
ACPI: PCI Interrupt Link [LNKD] (IRQs 16 17 18 19) *0, disabled.
ACPI: PCI Interrupt Link [LNKE] (IRQs 16 17 18 19) *0, disabled.
ACPI: PCI Interrupt Link [LUS0] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LUS1] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LUS2] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LKLN] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LAUI] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LKMO] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LKSM] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LTID] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LTIE] (IRQs 20 21 22) *0, disabled.
ACPI: PCI Interrupt Link [LATA] (IRQs 20 21 22) *14
ACPI: PCI Interrupt Link [LN2A] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN2B] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN2C] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN2D] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LK2N] (IRQs 44 45 46 47) *0, disabled.
ACPI: PCI Interrupt Link [LT5D] (IRQs 44 45 46 47) *0, disabled.
ACPI: PCI Interrupt Link [LT2E] (IRQs 44 45 46 47) *0, disabled.
ACPI: PCI Interrupt Link [LN3A] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN3B] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN3C] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN3D] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN4A] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN4B] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN4C] (IRQs 40 41 42 43) *0, disabled.
ACPI: PCI Interrupt Link [LN4D] (IRQs 40 41 42 43) *0, disabled.
SCSI subsystem initialized
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
ACPI: PCI Interrupt Link [LKSM] enabled at IRQ 22
ACPI: PCI interrupt 0000:00:01.1[A] -> GSI 22 (level, low) -> IRQ 22
ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 19
ACPI: PCI interrupt 0000:05:06.0[A] -> GSI 19 (level, low) -> IRQ 19
ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 18
ACPI: PCI interrupt 0000:05:07.0[A] -> GSI 18 (level, low) -> IRQ 18
ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 17
ACPI: PCI interrupt 0000:01:00.0[A] -> GSI 17 (level, low) -> IRQ 17
ACPI: PCI Interrupt Link [LN3C] enabled at IRQ 43
ACPI: PCI interrupt 0000:41:00.0[A] -> GSI 43 (level, low) -> IRQ 43
PCI-DMA: Disabling AGP.
PCI-DMA: aperture base @ 4000000 size 65536 KB
PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture
IA32 emulation $Id: sys_ia32.c 72 2005-03-02 01:01:07Z peter $
Total HugeTLB memory allocated, 0
NTFS driver 2.1.17 [Flags: R/O].
subfs 0.9
Initializing Cryptographic API
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
vesafb: probe of vesafb0 failed with error -6
ACPI: Power Button (FF) [PWRF]
ACPI: Processor [CPU1] (supports C1)
ACPI: Processor [CPU2] (supports C1)
ACPI: Processor [CPU3] (supports C1)
ACPI: Processor [CPU4] (supports C1)
ACPI: Processor [CPU5] (supports C1)
ACPI: Processor [CPU6] (supports C1)
ACPI: Processor [CPU7] (supports C1)
ACPI: Processor [CPU8] (supports C1)
Real Time Clock Driver v1.12
Non-volatile memory driver v1.2
Linux agpgart interface v0.100 (c) Dave Jones
ipmi message handler version v33
ipmi device interface version v33
IPMI System Interface driver version v33, KCS version v33, SMIC version v33, BT version v33
ipmi_si: Found SMBIOS-specified state machine at I/O address 0x62
 Could not set up I/O space
Trying to free nonexistent resource <00000062-00000063>
ipmi_si: Unable to find any System Interface(s)
Copyright (C) 2004 MontaVista Software - IPMI Powerdown via sys_reboot version v33.
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
Serial: 8250/16550 driver $Revision: 1.1 $ 8 ports, IRQ sharing disabled
ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
RAMDISK driver initialized: 16 RAM disks of 65536K size 1024 blocksize
loop: loaded (max 8 devices)
nbd: Couldn't get a blowfish cipher context.
Intel(R) PRO/1000 Network Driver - version 5.3.19-k2
Copyright (c) 1999-2004 Intel Corporation.
ACPI: PCI interrupt 0000:05:07.0[A] -> GSI 18 (level, low) -> IRQ 18
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
eth0: TCP Segmentation Offload (TSO) disabled by default
pcnet32.c:v1.30i 06.28.2004 tsbogend@alpha.franken.de
e100: Intel(R) PRO/100 Network Driver, 3.0.27-k2-NAPI
e100: Copyright(c) 1999-2004 Intel Corporation
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
NFORCE-CK804: IDE controller at PCI slot 0000:00:06.0
NFORCE-CK804: chipset revision 162
NFORCE-CK804: not 100% native mode: will probe irqs later
NFORCE-CK804: 0000:00:06.0 (rev a2) UDMA133 controller
    ide0: BM-DMA at 0x6000-0x6007, BIOS settings: hda:DMA, hdb:DMA
    ide1: BM-DMA at 0x6008-0x600f, BIOS settings: hdc:DMA, hdd:DMA
Probing IDE interface ide0...
hda: FUJITSU MHT2040AS, ATA DISK drive
Using anticipatory io scheduler
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Probing IDE interface ide1...
Probing IDE interface ide1...
Probing IDE interface ide2...
ide2: Wait for ready failed before probe !
Probing IDE interface ide3...
ide3: Wait for ready failed before probe !
Probing IDE interface ide4...
ide4: Wait for ready failed before probe !
Probing IDE interface ide5...
ide5: Wait for ready failed before probe !
hda: max request size: 128KiB
hda: 78140160 sectors (40007 MB) w/8192KiB Cache, CHS=65535/16/63, UDMA(33)
hda: cache flushes supported
 hda: hda1 hda2 hda3
ohci_hcd: 2004 Feb 02 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI)
USB Universal Host Controller Interface driver v2.2
Initializing USB Mass Storage driver...
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
/panta-build/kernel-sources/SuSE-2.6.8/drivers/usb/input/hid-core.c: v2.0:USB HID core driver
mice: PS/2 mouse device common for all mice
input: AT Translated Set 2 keyboard on isa0060/serio0
NET: Registered protocol family 2
IP: routing cache hash table of 65536 buckets, 1024Kbytes
TCP: Hash tables configured (established 262144 bind 65536)
ip_conntrack version 2.1 (8192 buckets, 65536 max) - 520 bytes per conntrack
ip_tables: (C) 2000-2002 Netfilter core team
ipt_recent v0.3.1: Stephen Frost <sfrost@snowman.net>.  http://snowman.net/projects/ipt_recent/
ClusterIP Version 0.5 loaded successfully
arp_tables: (C) 2002 David S. Miller
NET: Registered protocol family 1
NET: Registered protocol family 17
ACPI: (supports S0 S1 S5)
ACPI wakeup devices: 
PS2K USB0 USB1 P0P1 P0P2 P0P3 P0P4 P0P5 BR84 BR83 BR82 BR81 PWRB 
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 228k freed
EXT3 FS on hda3, internal journal
FAT: utf8 is not a recommended IO charset for FAT filesystems, filesystem will be case sensitive!
Adding 1052248k swap on /dev/hda2.  Priority:42 extents:1
 --------------------- ioctl INIT: pci_find_slot(1, 0)
 --------------------- ioctl INIT: pci_find_slot(1, 0)
 --------------------- ioctl INIT: pci_find_slot(1, 0)
 --------------------- ioctl INIT: pci_find_slot(65, 0)
 --------------------- ioctl INIT: pci_find_slot(65, 0)
 --------------------- ioctl INIT: pci_find_slot(65, 0)
e1000: eth0: e1000_watchdog: NIC Link is Up 100 Mbps Full Duplex
PCI: Setting latency timer of device 0000:01:00.0 to 64
ACPI: PCI interrupt 0000:01:00.0[A] -> GSI 17 (level, low) -> IRQ 17
PCI: Setting latency timer of device 0000:41:00.0 to 64
ACPI: PCI interrupt 0000:41:00.0[A] -> GSI 43 (level, low) -> IRQ 43

Mellanox Tavor Device Driver is creating device "InfiniHost0" (bus=01, devfn=00)


Mellanox Tavor Device Driver is creating device "InfiniHost1" (bus=41, devfn=00)

THH kernel module initialized successfully
[KERNEL_IB][ib_mad_static_compute_base][/var/tmp/IBGD/lib/modules/2.6.8-24.11-smp/source/drivers/infiniband/core/mad_static.c:93]Couldn't find a suitable network device; setting lid_base to 1
NET: Registered protocol family 26

Re: Mystery packet killing tg3

[Andi Kleen]

On Thu, May 05, 2005 at 11:40:24AM -0700, Peter Buckingham wrote:
> Andi Kleen wrote:
> >That should be impossible. Or it sounds like a serious
> >hardware problem. DAC should normally always work with all e1000 AFAIK.
> 
> okay.
> 
> I basically just force it to take a 32bit dma mask. I admit i'm a little 
> clueless as to what this truely means. I had assumed that it would 
> result in only dma'ing to an area below 4GB, but I hadn't really 
> validated that assumption :-(

If you dont use iommu=force it will only cause IOMMU remapping
when the memory buffer is beyond 4GB. It is basically random
if that happens or not, depending on how fully your memory is.

With iommu=force or CONFIG_IOMMU_DEBUG all IO is foced 
through the IOMMU.

> 
> >Most likely you have some hardware problem and it is somehow magically
> >worked around by IOMMU remapping. One difference is that
> >the remapping makes all IO slower, perhaps the changed timing
> >works around some bug.
> 
> this is always a possibility, can you suggest some ways of isolating 
> this problem?

Hmm - if you want to hack the kernel you could add udelay()s
to the no IOMMU paths in arch/x86_64/kernel/pci-gart.c 
and see if that cures the problem too.  If yes then the timing
theory would be proved.
Actually the IOMMU code does more than just delaying, it also
does config space accesses which might flush or synchronize
things in the PCI bridges. Perhaps adding some dummy access
for that would be good too.

The dmesg looks similar to the previous one from the IOMMU
code perspective.

-Andi

Re: Mystery packet killing tg3

[Peter Buckingham]

Andi Kleen wrote:
> With iommu=force or CONFIG_IOMMU_DEBUG all IO is foced 
> through the IOMMU.

okay, that's definitely bad...

> Hmm - if you want to hack the kernel you could add udelay()s
> to the no IOMMU paths in arch/x86_64/kernel/pci-gart.c 
> and see if that cures the problem too.  If yes then the timing
> theory would be proved.
> Actually the IOMMU code does more than just delaying, it also
> does config space accesses which might flush or synchronize
> things in the PCI bridges. Perhaps adding some dummy access
> for that would be good too.

okay, i'll start to take a look at this, but it'll probably be a little 
while before i can really get to it. i'll let you know what i find out.

> The dmesg looks similar to the previous one from the IOMMU
> code perspective.

it actually looks like it's configure correctly now (at least from 
agp-gart messages), before we were getting errors. i guess the bios guys 
finally got it right ;-)

thanks,

peter

Re: Mystery packet killing tg3

[Andi Kleen]

> >The dmesg looks similar to the previous one from the IOMMU
> >code perspective.
> 
> it actually looks like it's configure correctly now (at least from 
> agp-gart messages), before we were getting errors. i guess the bios guys 
> finally got it right ;-)

No, it is still broken, the kernel fixed it up. Look at the early
messages. It was too small (32MB)

-Andi

Re: Mystery packet killing tg3

[Michael Chan]

On Thu, 2005-05-05 at 11:33 -0700, David S. Miller wrote:

> I'm very tempted to add a silencer to these messages in these
> cases.  Something like the patch below.  Michael, what do you
> think?
> 
> [TG3]: Elide tg3_stop_block messages when such events are normal.
> 
> Signed-off-by: David S. Miller <davem@davemloft.net>
> 

Looks good. I'm adding this patch to further ignore the tg3_stop_block
errors.

[TG3]: Ignore tg3_stop_block() errors.

tg3_stop_block() errors can be safely ignored since tg3_chip_reset()
always follows tg3_stop_block() calls.


Signed-off-by: Michael Chan <mchan@broadcom.com>

diff -Nru f/drivers/net/tg3.c g/drivers/net/tg3.c
--- f/drivers/net/tg3.c	2005-05-05 12:29:41.000000000 -0700
+++ g/drivers/net/tg3.c	2005-05-05 12:37:14.000000000 -0700
@@ -3725,8 +3725,6 @@
 	err |= tg3_stop_block(tp, SNDDATAC_MODE, SNDDATAC_MODE_ENABLE,
silent);
 	err |= tg3_stop_block(tp, DMAC_MODE, DMAC_MODE_ENABLE, silent);
 	err |= tg3_stop_block(tp, SNDBDC_MODE, SNDBDC_MODE_ENABLE, silent);
-	if (err)
-		goto out;
 
 	tp->mac_mode &= ~MAC_MODE_TDE_ENABLE;
 	tw32_f(MAC_MODE, tp->mac_mode);
@@ -3744,10 +3742,10 @@
 		printk(KERN_ERR PFX "tg3_abort_hw timed out for %s, "
 		       "TX_MODE_ENABLE will not clear MAC_TX_MODE=%08x\n",
 		       tp->dev->name, tr32(MAC_TX_MODE));
-		return -ENODEV;
+		err |= -ENODEV;
 	}
 
-	err  = tg3_stop_block(tp, HOSTCC_MODE, HOSTCC_MODE_ENABLE, silent);
+	err |= tg3_stop_block(tp, HOSTCC_MODE, HOSTCC_MODE_ENABLE, silent);
 	err |= tg3_stop_block(tp, WDMAC_MODE, WDMAC_MODE_ENABLE, silent);
 	err |= tg3_stop_block(tp, MBFREE_MODE, MBFREE_MODE_ENABLE, silent);
 
@@ -3756,15 +3754,12 @@
 
 	err |= tg3_stop_block(tp, BUFMGR_MODE, BUFMGR_MODE_ENABLE, silent);
 	err |= tg3_stop_block(tp, MEMARB_MODE, MEMARB_MODE_ENABLE, silent);
-	if (err)
-		goto out;
 
 	if (tp->hw_status)
 		memset(tp->hw_status, 0, TG3_HW_STATUS_SIZE);
 	if (tp->hw_stats)
 		memset(tp->hw_stats, 0, sizeof(struct tg3_hw_stats));
 
-out:
 	return err;
 }
 
@@ -5063,9 +5058,7 @@
 	tg3_write_sig_pre_reset(tp, RESET_KIND_INIT);
 
 	if (tp->tg3_flags & TG3_FLAG_INIT_COMPLETE) {
-		err = tg3_abort_hw(tp, 1);
-		if (err)
-			return err;
+		tg3_abort_hw(tp, 1);
 	}
 
 	err = tg3_chip_reset(tp);

Re: Mystery packet killing tg3

[David S. Miller]

On Thu, 05 May 2005 12:56:34 -0700
"Michael Chan" <mchan@broadcom.com> wrote:

> On Thu, 2005-05-05 at 11:33 -0700, David S. Miller wrote:
> 
> > I'm very tempted to add a silencer to these messages in these
> > cases.  Something like the patch below.  Michael, what do you
> > think?
> > 
> > [TG3]: Elide tg3_stop_block messages when such events are normal.
> > 
> > Signed-off-by: David S. Miller <davem@davemloft.net>
> > 
> 
> Looks good. I'm adding this patch to further ignore the tg3_stop_block
> errors.
> 
> [TG3]: Ignore tg3_stop_block() errors.
> 
> tg3_stop_block() errors can be safely ignored since tg3_chip_reset()
> always follows tg3_stop_block() calls.

Your email client wrapped the lines in the patch Michael, please use
attachments if you can't avoid this.

Anyways, I applied both my patch and your's (by hand) to my tree and
will push upstream.

Thanks again.