[e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Brandeburg, Jesse]

Hi all, I've identified you as people who have at some point in the past
emailed one of the Linux lists with problems with e1000 and
sk_forward_alloc.  It seems to be fairly widespread, but only seems to
have appeared with recent kernel changes (after 2.6.12...)

What I need from you is a reproducible test, and some information.  I
have never been able to reproduce this, and I'm trying to isolate the
problem a bit.  What motherboards are you using?  What seems to cause
this problem?  Are you all using iptables?  Are you all routing? From
the reports I assume none of you are using an 82571/2/3 (pci express)

As far as I know e1000 has the same requirement as tg3 and some others
where we have to modify the header of the skb in the case of transmits
using TSO.  I don't see anywhere else that the driver modifies the skb.
Tomorrow I'll generate a patch to try a more paranoid copying of the
skb, I hope some of you can test.

To do this we have code like so in e1000_tso:
2529         if (skb_shinfo(skb)->tso_size) {
2530                 if (skb_header_cloned(skb)) {
2531                         err = pskb_expand_head(skb, 0, 0,
GFP_ATOMIC);
2532                         if (err)
2533                                 return err;
2534                 }
2535 
2536                 hdr_len = ((skb->h.raw - skb->data) +
(skb->h.th->doff << 2));
2537                 mss = skb_shinfo(skb)->tso_size;
2538                 if (skb->protocol == ntohs(ETH_P_IP)) {
2539                         skb->nh.iph->tot_len = 0;
2540                         skb->nh.iph->check = 0;

Thanks for your assistance

Jesse


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x110944&bid$1720&dat\x121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Yoseph Basri]

Hi Jesse,

Thanks for your concern,

My server still send warning message regarding this KERNEL: assertion
(!sk_forward_alloc)  after upgrade kernel 2.6.12 or 2.6.15.

This is from dmesg server:

Linux version 2.6.15.4 (root@xxxxx) (gcc version 3.3.4 (Debian
1:3.3.4-13)) #1 SMP Tue Feb 21 17:12:27 SGT 2006
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009d400 (usable)
 BIOS-e820: 000000000009d400 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000f7ffb300 (usable)
 BIOS-e820: 00000000f7ffb300 - 00000000f8000000 (ACPI data)
 BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000108000000 (usable)
Warning only 4GB will be used.
Use a PAE enabled kernel.
3200MB HIGHMEM available.
896MB LOWMEM available.
found SMP MP-table at 0009d540
On node 0 totalpages: 1048576
  DMA zone: 4096 pages, LIFO batch:0
  DMA32 zone: 0 pages, LIFO batch:0
  Normal zone: 225280 pages, LIFO batch:31
  HighMem zone: 819200 pages, LIFO batch:31
DMI 2.3 present.
ACPI: RSDP (v000 IBM                                   ) @ 0x000fdfc0
ACPI: RSDT (v001 IBM    SERONYXP 0x00001000 IBM  0x45444f43) @ 0xf7ffff80
ACPI: FADT (v001 IBM    SERONYXP 0x00001000 IBM  0x45444f43) @ 0xf7ffff00
ACPI: MADT (v001 IBM    SERONYXP 0x00001000 IBM  0x45444f43) @ 0xf7fffe80
ACPI: ASF! (v016 IBM    SERONYXP 0x00000001 IBM  0x45444f43) @ 0xf7fffdc0
ACPI: DSDT (v001 IBM    SERGEODE 0x00001000 MSFT 0x0100000b) @ 0x00000000
ACPI: PM-Timer IO Port: 0x488
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 15:2 APIC version 20
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
Processor #1 15:2 APIC version 20
ACPI: LAPIC_NMI (acpi_id[0x00] dfl dfl lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] dfl dfl lint[0x1])
ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
ACPI: IOAPIC (id[0x0c] address[0xfec02000] gsi_base[32])
IOAPIC[2]: apic_id 12, version 17, address 0xfec02000, GSI 32-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ7 used by override.
Enabling APIC mode:  Flat.  Using 3 I/O APICs
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at f8800000 (gap: f8000000:06c00000)
Built 1 zonelists
Kernel command line: BOOT_IMAGE=linux-2.6.15.4 ro root=801
mapped APIC to ffffd000 (fee00000)
mapped IOAPIC to ffffc000 (fec00000)
mapped IOAPIC to ffffb000 (fec01000)
mapped IOAPIC to ffffa000 (fec02000)
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 65536 bytes)
Detected 2794.685 MHz processor.
Using pmtmr for high-res timesource
Console: colour VGA+ 80x25
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Memory: 4025292k/4194304k available (1906k kernel code, 36796k
reserved, 650k data, 240k init, 3145708k highmem)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
Calibrating delay using timer specific routine.. 5593.74 BogoMIPS (lpj=11187486)
Mount-cache hash table entries: 512
CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000
00004400 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 00000000 00000000 00000000
00004400 00000000 00000000
CPU: Trace cache: 12K uops, L1 D cache: 8K
CPU: L2 cache: 512K
CPU: Physical Processor ID: 0
CPU: After all inits, caps: bfebfbff 00000000 00000000 00000080
00004400 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU0: Intel P4/Xeon Extended MCE MSRs (12) available
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
CPU0: Intel(R) Xeon(TM) CPU 2.80GHz stepping 07
Booting processor 1/1 eip 2000
Initializing CPU#1
Calibrating delay using timer specific routine.. 5588.12 BogoMIPS (lpj=11176253)
CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000
00004400 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 00000000 00000000 00000000
00004400 00000000 00000000
CPU: Trace cache: 12K uops, L1 D cache: 8K
CPU: L2 cache: 512K
CPU: Physical Processor ID: 0
CPU: After all inits, caps: bfebfbff 00000000 00000000 00000080
00004400 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#1.
CPU1: Intel P4/Xeon Extended MCE MSRs (12) available
CPU1: Intel(R) Xeon(TM) CPU 2.80GHz stepping 07
Total of 2 processors activated (11181.86 BogoMIPS).
ENABLING IO-APIC IRQs
..TIMER: vector=0x31 apic1=0 pin1=2 apic2=-1 pin2=-1
checking TSC synchronization across 2 CPUs: passed.
Brought up 2 CPUs
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xfd7dc, last bus=8
PCI: Using configuration type 1
ACPI: Subsystem revision 20050902
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI: Probing PCI hardware (bus 00)
Boot video device is 0000:00:06.0
PCI: Ignoring BAR0-3 of IDE controller 0000:00:0f.1
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Root Bridge [PCI1] (0000:02)
PCI: Probing PCI hardware (bus 02)
ACPI: PCI Interrupt Routing Table [\_SB_.PCI1._PRT]
ACPI: PCI Root Bridge [PCI2] (0000:04)
PCI: Probing PCI hardware (bus 04)
ACPI: PCI Interrupt Routing Table [\_SB_.PCI2._PRT]
ACPI: PCI Root Bridge [PCI3] (0000:06)
PCI: Probing PCI hardware (bus 06)
ACPI: PCI Interrupt Routing Table [\_SB_.PCI3._PRT]
ACPI: PCI Root Bridge [PCI4] (0000:08)
PCI: Probing PCI hardware (bus 08)
ACPI: PCI Interrupt Routing Table [\_SB_.PCI4._PRT]
ACPI: PCI Interrupt Link [LP00] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP01] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP02] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP03] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP04] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP05] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP06] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP07] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP08] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP09] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP0A] (IRQs *10)
ACPI: PCI Interrupt Link [LP0B] (IRQs *9)
ACPI: PCI Interrupt Link [LP0C] (IRQs *9)
ACPI: PCI Interrupt Link [LP0D] (IRQs *3)
ACPI: PCI Interrupt Link [LP0E] (IRQs *5)
ACPI: PCI Interrupt Link [LP0F] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP10] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP11] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP12] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP13] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP14] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP15] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP16] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP17] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP18] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP19] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP1A] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP1B] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP1C] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP1D] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP1E] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP1F] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LPUS] (IRQs *11)
SCSI subsystem initialized
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
IA-32 Microcode Update Driver: v1.14 <tigran@veritas.com>
highmem bounce pool size: 64 pages
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered
Real Time Clock Driver v1.12
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
Floppy drive(s): fd0 is 1.44M
FDC 0 is a National Semiconductor PC87306
Intel(R) PRO/1000 Network Driver - version 6.1.16-k2
Copyright (c) 1999-2005 Intel Corporation.
ACPI: PCI Interrupt 0000:06:08.0[A] -> GSI 29 (level, low) -> IRQ 16
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
ACPI: PCI Interrupt 0000:06:08.1[B] -> GSI 30 (level, low) -> IRQ 17
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
e100: Intel(R) PRO/100 Network Driver, 3.4.14-k4-NAPI
e100: Copyright(c) 1999-2005 Intel Corporation
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
Probing IDE interface ide0...
hda: LG CD-ROM CRN-8245B, ATAPI CD/DVD-ROM drive
Probing IDE interface ide1...
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Fusion MPT base driver 3.03.04
Copyright (c) 1999-2005 LSI Logic Corporation
Fusion MPT SPI Host driver 3.03.04
Fusion MPT base driver 3.03.04
Copyright (c) 1999-2005 LSI Logic Corporation
Fusion MPT SPI Host driver 3.03.04
ACPI: PCI Interrupt 0000:08:07.0[A] -> GSI 27 (level, low) -> IRQ 18
mptbase: Initiating ioc0 bringup
ioc0: 53C1030: Capabilities={Initiator}
scsi0 : ioc0: LSI53C1030, FwRev=01000e00h, Ports=1, MaxQ=222, IRQ=18
  Vendor: IBM-ESXS  Model: MAP3735NC     FN  Rev: B109
  Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sda: 143374000 512-byte hdwr sectors (73407 MB)
SCSI device sda: drive cache: write through
SCSI device sda: 143374000 512-byte hdwr sectors (73407 MB)
SCSI device sda: drive cache: write through
 sda: sda1 sda2 sda3 < sda5 sda6 sda7 sda8 sda9 sda10 >
sd 0:0:0:0: Attached scsi disk sda
  Vendor: IBM-ESXS  Model: MAP3735NC     FN  Rev: B109
  Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sdb: 143374000 512-byte hdwr sectors (73407 MB)
SCSI device sdb: drive cache: write through
SCSI device sdb: 143374000 512-byte hdwr sectors (73407 MB)
SCSI device sdb: drive cache: write through
 sdb: sdb1 sdb2 sdb3 sdb4
sd 0:0:1:0: Attached scsi disk sdb
  Vendor: IBM-ESXS  Model: MAP3735NC     FN  Rev: B109
  Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sdc: 143374000 512-byte hdwr sectors (73407 MB)
SCSI device sdc: drive cache: write through
SCSI device sdc: 143374000 512-byte hdwr sectors (73407 MB)
SCSI device sdc: drive cache: write through
 sdc: sdc1 sdc2 sdc3 sdc4
sd 0:0:2:0: Attached scsi disk sdc
  Vendor: IBM-ESXS  Model: MAP3735NC     FN  Rev: B109
  Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sdd: 143374000 512-byte hdwr sectors (73407 MB)
SCSI device sdd: drive cache: write through
SCSI device sdd: 143374000 512-byte hdwr sectors (73407 MB)
SCSI device sdd: drive cache: write through
 sdd: sdd1 sdd2 sdd3 sdd4
sd 0:0:3:0: Attached scsi disk sdd
  Vendor: IBM-ESXS  Model: MAP3735NC     FN  Rev: B109
  Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sde: 143374000 512-byte hdwr sectors (73407 MB)
SCSI device sde: drive cache: write through
SCSI device sde: 143374000 512-byte hdwr sectors (73407 MB)
SCSI device sde: drive cache: write through
 sde: sde1 sde2 sde3 sde4
sd 0:0:4:0: Attached scsi disk sde
  Vendor: IBM-ESXS  Model: MAP3735NC     FN  Rev: B109
  Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sdf: 143374000 512-byte hdwr sectors (73407 MB)
SCSI device sdf: drive cache: write through
SCSI device sdf: 143374000 512-byte hdwr sectors (73407 MB)
SCSI device sdf: drive cache: write through
 sdf: sdf1 sdf2 sdf3 sdf4
sd 0:0:5:0: Attached scsi disk sdf
  Vendor: IBM       Model: 32P0032a S320  1  Rev: 1
  Type:   Processor                          ANSI SCSI revision: 02
ACPI: PCI Interrupt 0000:08:07.1[B] -> GSI 28 (level, low) -> IRQ 19
mptbase: Initiating ioc1 bringup
ioc1: 53C1030: Capabilities={Initiator}
scsi1 : ioc1: LSI53C1030, FwRev=01000e00h, Ports=1, MaxQ=222, IRQ=19
mice: PS/2 mouse device common for all mice
NET: Registered protocol family 2
IP route cache hash table entries: 262144 (order: 8, 1048576 bytes)
TCP established hash table entries: 524288 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
input: AT Translated Set 2 keyboard as /class/input/input0
TCP: Hash tables configured (established 524288 bind 65536)
TCP reno registered
ip_conntrack version 2.4 (8192 buckets, 65536 max) - 176 bytes per conntrack
ip_tables: (C) 2000-2002 Netfilter core team
TCP bic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
Starting balanced_irq
Using IPI Shortcut mode
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 240k freed
Adding 2104504k swap on /dev/sda2.  Priority:-1 extents:1 across:2104504k
EXT3 FS on sda1, internal journal
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda6, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda5, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda7, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda10, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda9, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda8, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdb1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdb2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdb3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdb4, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdc1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdc2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdc3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdc4, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdd1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdd2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdd3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdd4, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sde1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sde2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sde3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sde4, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdf1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdf2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdf3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sdf4, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
e1000: eth0: e1000_watchdog_task: NIC Link is Up 100 Mbps Full Duplex

Are you all using iptables?

YES

Are you all routing?

NO

>From the reports I assume none of you are using an 82571/2/3 (pci express)?

think so.

Thnaks again.

YB



On 3/30/06, Brandeburg, Jesse <jesse.brandeburg@intel.com> wrote:
> Hi all, I've identified you as people who have at some point in the past
> emailed one of the Linux lists with problems with e1000 and
> sk_forward_alloc.  It seems to be fairly widespread, but only seems to
> have appeared with recent kernel changes (after 2.6.12...)
>
> What I need from you is a reproducible test, and some information.  I
> have never been able to reproduce this, and I'm trying to isolate the
> problem a bit.  What motherboards are you using?  What seems to cause
> this problem?  Are you all using iptables?  Are you all routing? From
> the reports I assume none of you are using an 82571/2/3 (pci express)
>
> As far as I know e1000 has the same requirement as tg3 and some others
> where we have to modify the header of the skb in the case of transmits
> using TSO.  I don't see anywhere else that the driver modifies the skb.
> Tomorrow I'll generate a patch to try a more paranoid copying of the
> skb, I hope some of you can test.
>
> To do this we have code like so in e1000_tso:
> 2529         if (skb_shinfo(skb)->tso_size) {
> 2530                 if (skb_header_cloned(skb)) {
> 2531                         err = pskb_expand_head(skb, 0, 0,
> GFP_ATOMIC);
> 2532                         if (err)
> 2533                                 return err;
> 2534                 }
> 2535
> 2536                 hdr_len = ((skb->h.raw - skb->data) +
> (skb->h.th->doff << 2));
> 2537                 mss = skb_shinfo(skb)->tso_size;
> 2538                 if (skb->protocol == ntohs(ETH_P_IP)) {
> 2539                         skb->nh.iph->tot_len = 0;
> 2540                         skb->nh.iph->check = 0;
>
> Thanks for your assistance
>
> Jesse
>


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x110944&bid$1720&dat\x121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Phil Oester]

On Wed, Mar 29, 2006 at 06:53:57PM -0800, Brandeburg, Jesse wrote:
> Hi all, I've identified you as people who have at some point in the past
> emailed one of the Linux lists with problems with e1000 and
> sk_forward_alloc.  It seems to be fairly widespread, but only seems to
> have appeared with recent kernel changes (after 2.6.12...)
> 
> What I need from you is a reproducible test, and some information.  I
> have never been able to reproduce this, and I'm trying to isolate the
> problem a bit.  What motherboards are you using?  What seems to cause
> this problem?  Are you all using iptables?  Are you all routing? From
> the reports I assume none of you are using an 82571/2/3 (pci express)

Unfortunately it happens randomly, so I have no reproducible test.
Dell 1850s and 2850s here, no iptables, routing, or pci express.
lspci reports:

82541GI/PI Gigabit Ethernet Controller (rev 05)

> As far as I know e1000 has the same requirement as tg3 and some others
> where we have to modify the header of the skb in the case of transmits
> using TSO.  I don't see anywhere else that the driver modifies the skb.
> Tomorrow I'll generate a patch to try a more paranoid copying of the
> skb, I hope some of you can test.

I'll certainly try it as long as it doesn't blow things up :)

Phil


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[David S. Miller]

From: "Brandeburg, Jesse" <jesse.brandeburg@intel.com>
Date: Wed, 29 Mar 2006 18:53:57 -0800

> To do this we have code like so in e1000_tso:
> 2529         if (skb_shinfo(skb)->tso_size) {
> 2530                 if (skb_header_cloned(skb)) {
> 2531                         err = pskb_expand_head(skb, 0, 0,
> GFP_ATOMIC);
> 2532                         if (err)
> 2533                                 return err;
> 2534                 }

I was wondering if that call could somehow mess up the
sk->sk_forward_alloc value later on.

But it can't, sk_forward_alloc is modified based upon the
skb->truesize value, but pskb_expand_head() does not change that.

So the things left to check in the generic networking are the
skb_shinfo() contents and ->dataref handling.

I considered whether pskb_expand_head() could corrupt the TSO
information in skb_shinfo().  But that's clearly not the case because
pskb_expand_head() explicitly copies it over:

	memcpy(data + size, skb->end, sizeof(struct skb_shared_info));

And skb->end is set appropriately:

	skb->end      = data + size;

because skb_shinfo() is:

#define skb_shinfo(SKB)		((struct skb_shared_info *)((SKB)->end))

The only skb_shared_info that has to be explicitly setup is the
dataref, and pskb_expand_head() does that:

	atomic_set(&skb_shinfo(skb)->dataref, 1);

So that all checks out.

I wonder if something funky is going on wrt. the skb_release_data()
that pskb_expand_head() does.  We have that SKB_DATAREF_SHIFT thingy,
which will trigger in this case.

	if (!skb->cloned ||
	    !atomic_sub_return(skb->nohdr ? (1 << SKB_DATAREF_SHIFT) + 1 : 1,
			       &skb_shinfo(skb)->dataref)) {

When we enqueue a new TCP frame we do skb_header_release() which goes:

	skb->nohdr = 1;
	atomic_add(1 << SKB_DATAREF_SHIFT, &skb_shinfo(skb)->dataref);

Presumably the dataref is "1" already when we get here and do this.
We will clone, the clone will set ->nohdr to 0 and will increment the
dataref.

So at this point the dataref should be:

   1 /* initial reference */
 + (1 << SKB_DATAREF_SHIFT) /* from skb_header_release() */
 + 1 /* from skb_clone */

This all works out because when the clone is freed up, skb->nohdr will
be zero, so we will subtract "1" from dataref.  Later when the ACK
arrives we'll free up the non-clone and this will have skb->nohdr set
to "1" and thus we'll subtract

	(1 << SKB_DATAREF_SHIFT) + 1

from dataref, as per skb_release_data().

Although maybe relevant here, I just noticed that __skb_linearize()
does not clear skb->nohdr.  I bet that will cause a bunch of trouble
if the original SKB had skb->nohdr set, but I cannot see how that
can occur, we only send clones out to the device and those have
skb->nohdr clear (Herbert, double check this for me please).

Luckily that thing is used rarely.  Only in the dev_queue_xmit()
path when the SKB has been configured in such a way that the
transmitting device does not support so it should not be relevant
here.  Also I note that __skb_linearize() is not used at all
outside of net/core/dev.c, so we should mark it static some point
soon.  In fact we should do that while fixing this fringe "nohdr"
bug in __skb_linearize().

All the other dataref accesses look safe.

Herbert do you see any holes here?


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Herbert Xu]

[-- Attachment #1: Type: text/plain, Size: 706 bytes --]

On Wed, Mar 29, 2006 at 08:44:09PM -0800, David S. Miller wrote:
> 
> Herbert do you see any holes here?

Well I started from the beginning again, and found this.  This may be
the smoking gun that we're after :)

The xmit routine is lockless but checks last_tx_tso outside the locked
section.  So if a TSO packet wins a race against a non-TSO packet with
last_tx_tso == 0 then we'll have memory corruption.

Everyone, please try this patch and let us know whether the problem
goes away.

Thanks,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[-- Attachment #2: e1000.patch --]
[-- Type: text/plain, Size: 1229 bytes --]

diff --git a/drivers/net/e1000/e1000_main.c b/drivers/net/e1000/e1000_main.c
index 49cd096..96b7bef 100644
--- a/drivers/net/e1000/e1000_main.c
+++ b/drivers/net/e1000/e1000_main.c
@@ -2821,13 +2821,6 @@
 		count++;
 #endif
 
-#ifdef NETIF_F_TSO
-	/* Controller Erratum workaround */
-	if (!skb->data_len && tx_ring->last_tx_tso &&
-	    !skb_shinfo(skb)->tso_size)
-		count++;
-#endif
-
 	count += TXD_USE_COUNT(len, max_txd_pwr);
 
 	if (adapter->pcix_82544)
@@ -2846,11 +2839,6 @@
 				       max_txd_pwr);
 	if (adapter->pcix_82544)
 		count += nr_frags;
-
-
-	if (adapter->hw.tx_pkt_filtering &&
-	    (adapter->hw.mac_type == e1000_82573))
-		e1000_transfer_dhcp_info(adapter, skb);
 
 	local_irq_save(flags);
 	if (!spin_trylock(&tx_ring->tx_lock)) {
@@ -2858,6 +2846,17 @@
 		local_irq_restore(flags);
 		return NETDEV_TX_LOCKED;
 	}
+
+#ifdef NETIF_F_TSO
+	/* Controller Erratum workaround */
+	if (!skb->data_len && tx_ring->last_tx_tso &&
+	    !skb_shinfo(skb)->tso_size)
+		count++;
+#endif
+
+	if (adapter->hw.tx_pkt_filtering &&
+	    (adapter->hw.mac_type == e1000_82573))
+		e1000_transfer_dhcp_info(adapter, skb);
 
 	/* need: count + 2 desc gap to keep tail from touching
 	 * head, otherwise try next time */

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Herbert Xu.

On 30.03.2006 13:52 you said the following:

> On Wed, Mar 29, 2006 at 08:44:09PM -0800, David S. Miller wrote:
> 
>>Herbert do you see any holes here?
> 
> 
> Well I started from the beginning again, and found this.  This may be
> the smoking gun that we're after :)
> 
> The xmit routine is lockless but checks last_tx_tso outside the locked
> section.  So if a TSO packet wins a race against a non-TSO packet with
> last_tx_tso == 0 then we'll have memory corruption.
> 
> Everyone, please try this patch and let us know whether the problem
> goes away.
> 
> Thanks,



[zhmurov@builds linux-2.6.16]$ patch -p1 < 
../../../SOURCES/linux-2.6.16-e1000-try-to-fix-assertion_sk_forward_alloc_failed_by_Herbert_Xu.patch 

patching file drivers/net/e1000/e1000_main.c
Reversed (or previously applied) patch detected!  Assume -R? [n]


Herbert, is that patch already included in 2.6.16.1?

-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Herbert Xu]

[-- Attachment #1: Type: text/plain, Size: 706 bytes --]

On Thu, Mar 30, 2006 at 10:02:01AM +0000, Boris B. Zhmurov wrote:
> 
> [zhmurov@builds linux-2.6.16]$ patch -p1 < 
> ../../../SOURCES/linux-2.6.16-e1000-try-to-fix-assertion_sk_forward_alloc_failed_by_Herbert_Xu.patch 
> 
> patching file drivers/net/e1000/e1000_main.c
> Reversed (or previously applied) patch detected!  Assume -R? [n]
> 
> Herbert, is that patch already included in 2.6.16.1?

Not really.  It's just patch being silly (or too smart :)

Here it is again rediffed against 2.6.16.
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[-- Attachment #2: e1000-1.patch --]
[-- Type: text/plain, Size: 1210 bytes --]

diff --git a/drivers/net/e1000/e1000_main.c b/drivers/net/e1000/e1000_main.c
index 84dcca3..847d168 100644
--- a/drivers/net/e1000/e1000_main.c
+++ b/drivers/net/e1000/e1000_main.c
@@ -2932,13 +2932,6 @@
 		count++;
 #endif
 
-#ifdef NETIF_F_TSO
-	/* Controller Erratum workaround */
-	if (!skb->data_len && tx_ring->last_tx_tso &&
-		!skb_shinfo(skb)->tso_size)
-		count++;
-#endif
-
 	count += TXD_USE_COUNT(len, max_txd_pwr);
 
 	if (adapter->pcix_82544)
@@ -2957,9 +2950,6 @@
 				       max_txd_pwr);
 	if (adapter->pcix_82544)
 		count += nr_frags;
-
-	if (adapter->hw.tx_pkt_filtering && (adapter->hw.mac_type == e1000_82573) )
-		e1000_transfer_dhcp_info(adapter, skb);
 
 	local_irq_save(flags);
 	if (!spin_trylock(&tx_ring->tx_lock)) {
@@ -2967,6 +2957,16 @@
 		local_irq_restore(flags);
 		return NETDEV_TX_LOCKED;
 	}
+
+#ifdef NETIF_F_TSO
+	/* Controller Erratum workaround */
+	if (!skb->data_len && tx_ring->last_tx_tso &&
+		!skb_shinfo(skb)->tso_size)
+		count++;
+#endif
+
+	if (adapter->hw.tx_pkt_filtering && (adapter->hw.mac_type == e1000_82573) )
+		e1000_transfer_dhcp_info(adapter, skb);
 
 	/* need: count + 2 desc gap to keep tail from touching
 	 * head, otherwise try next time */

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[JaniD++]

Hello,

>Motherboard?

Intel SE7520AF2 (Dual Xeon)

>Are you all using iptables?

YES

>Are you all routing?

NO


>From the reports I assume none of you are using an 82571/2/3 (pci express)?

08:04.0 Ethernet controller: Intel Corp.: Unknown device 1079 (rev 03)
        Subsystem: Intel Corp.: Unknown device 341a
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr+
Stepping- SERR+ FastB2B-
        Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32 (63750ns min), cache line size 10
        Interrupt: pin A routed to IRQ 21
        Region 0: Memory at fe9c0000 (64-bit, non-prefetchable) [size=128K]
        Region 4: I/O ports at c800 [size=64]
        Capabilities: [dc] Power Management version 2
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [e4] PCI-X non-bridge device.
                Command: DPERE- ERO+ RBC=0 OST=0
                Status: Bus=0 Dev=0 Func=0 64bit- 133MHz- SCD- USC-,
DC=simple, DMMRBC=0, DMOST=0, DMCRS=0, RSCEM-      Capabi
lities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
                Address: 0000000000000000  Data: 0000

08:04.1 Ethernet controller: Intel Corp.: Unknown device 1079 (rev 03)
        Subsystem: Intel Corp.: Unknown device 341a
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr+
Stepping- SERR+ FastB2B-
        Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32 (63750ns min), cache line size 10
        Interrupt: pin B routed to IRQ 22
        Region 0: Memory at fe9e0000 (64-bit, non-prefetchable) [size=128K]
        Region 4: I/O ports at cc00 [size=64]
        Capabilities: [dc] Power Management version 2
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [e4] PCI-X non-bridge device.
                Command: DPERE- ERO+ RBC=0 OST=0
                Status: Bus=0 Dev=0 Func=0 64bit- 133MHz- SCD- USC-,
DC=simple, DMMRBC=0, DMOST=0, DMCRS=0, RSCEM-      Capabi
lities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
                Address: 0000000000000000  Data: 0000

And, i can see, all testers using the E1000 card to serving web (or web
proxy) directly to internet. (or not?)


My dmesg:  (kernel 2.6.15.7 + nbd-fix + E1000 driver from 2.6.16.1 version
6.3.9-k4)

Linux version 2.6.15.7-NBDFIX (root@dy-xeon-1) (gcc version 4.0.0 20050519
(Red Hat 4.0.0-8)) #2 SMP Wed Mar 29 15:05:41 CEST 2006
BIOS-provided physical RAM map:

BIOS-e820: 0000000000000000 - 000000000009f800 (usable)

BIOS-e820: 000000000009f800 - 00000000000a0000 (reserved)

BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)

BIOS-e820: 0000000000100000 - 00000000dffe0000 (usable)

BIOS-e820: 00000000dffe0000 - 00000000dffef000 (ACPI data)

BIOS-e820: 00000000dffef000 - 00000000dfffdc00 (ACPI NVS)

BIOS-e820: 00000000dfffdc00 - 00000000e0000000 (reserved)

BIOS-e820: 00000000fec00000 - 00000000fec86000 (reserved)

BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)

BIOS-e820: 00000000ffc00000 - 0000000100000000 (reserved)

BIOS-e820: 0000000100000000 - 0000000120000000 (usable)

3712MB HIGHMEM available.

896MB LOWMEM available.

found SMP MP-table at 000ff780

On node 0 totalpages: 1179648

DMA zone: 4096 pages, LIFO batch:0

DMA32 zone: 0 pages, LIFO batch:0

Normal zone: 225280 pages, LIFO batch:31

HighMem zone: 950272 pages, LIFO batch:31

DMI 2.3 present.

ACPI: RSDP (v000 ACPIAM ) @ 0x000f7f80

ACPI: RSDT (v001 A M I OEMRSDT 0x10000408 MSFT 0x00000097) @ 0xdffe0000

ACPI: FADT (v002 A M I OEMFACP 0x10000408 MSFT 0x00000097) @ 0xdffe0200

ACPI: MADT (v001 A M I OEMAPIC 0x10000408 MSFT 0x00000097) @ 0xdffe0390

ACPI: MCFG (v001 A M I OEMMCFG 0x10000408 MSFT 0x00000097) @ 0xdffe04a0

ACPI: OEMB (v001 A M I AMI_OEM 0x10000408 MSFT 0x00000097) @ 0xdffef040

ACPI: DSDT (v001 ALIEF ALIEF086 0x00000086 INTL 0x02002026) @ 0x00000000

ACPI: PM-Timer IO Port: 0x408

ACPI: Local APIC address 0xfee00000

ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)

Processor #0 15:4 APIC version 20

ACPI: LAPIC (acpi_id[0x02] lapic_id[0x06] enabled)

Processor #6 15:4 APIC version 20

ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)

Processor #1 15:4 APIC version 20

ACPI: LAPIC (acpi_id[0x04] lapic_id[0x07] enabled)

Processor #7 15:4 APIC version 20

ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])

ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])

ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])

ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])

ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])

IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23

ACPI: IOAPIC (id[0x09] address[0xfec80000] gsi_base[24])

IOAPIC[1]: apic_id 9, version 32, address 0xfec80000, GSI 24-47

ACPI: IOAPIC (id[0x0a] address[0xfec80400] gsi_base[48])

IOAPIC[2]: apic_id 10, version 32, address 0xfec80400, GSI 48-71

ACPI: IOAPIC (id[0x0b] address[0xfec85000] gsi_base[72])

IOAPIC[3]: apic_id 11, version 32, address 0xfec85000, GSI 72-95

ACPI: IOAPIC (id[0x0c] address[0xfec85400] gsi_base[96])

IOAPIC[4]: apic_id 12, version 32, address 0xfec85400, GSI 96-119

ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)

ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)

ACPI: IRQ0 used by override.

ACPI: IRQ2 used by override.

ACPI: IRQ9 used by override.

Enabling APIC mode: Flat. Using 5 I/O APICs

Using ACPI (MADT) for SMP configuration information

Allocating PCI resources starting at e2000000 (gap: e0000000:1ec00000)

Built 1 zonelists

Kernel command line: root=/dev/nfs,rw nfsroot=192.168.2.1://NFS/ROOT-XEON1/
ip=:::::eth0:autoconf pci=routeirq vga=0x0f05 notsc
BOOT_IMAGE=20060329_XEON_2.6.15.7-NBDFIX-NEWe1000

notsc: Kernel compiled with CONFIG_X86_TSC, cannot disable TSC.

mapped APIC to ffffd000 (fee00000)

mapped IOAPIC to ffffc000 (fec00000)

mapped IOAPIC to ffffb000 (fec80000)

mapped IOAPIC to ffffa000 (fec80400)

mapped IOAPIC to ffff9000 (fec85000)

mapped IOAPIC to ffff8000 (fec85400)

Initializing CPU#0

PID hash table entries: 4096 (order: 12, 65536 bytes)

Detected 2992.695 MHz processor.

Using pmtmr for high-res timesource

Console: colour VGA+ 80x30

Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)

Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)

Memory: 4149096k/4718592k available (3675k kernel code, 43856k reserved,
1225k data, 248k init, 3276672k highmem)

Checking if this processor honours the WP bit even in supervisor mode... Ok.

Calibrating delay using timer specific routine.. 5989.80 BogoMIPS
(lpj=29949038)

Security Framework v1.0.0 initialized

Capability LSM initialized

Mount-cache hash table entries: 512

CPU: After generic identify, caps: bfebfbff 20000000 00000000 00000000
0000641d 00000000 00000000

CPU: After vendor identify, caps: bfebfbff 20000000 00000000 00000000
0000641d 00000000 00000000

monitor/mwait feature present.

using mwait in idle threads.

CPU: Trace cache: 12K uops, L1 D cache: 16K

CPU: L2 cache: 1024K

CPU: Physical Processor ID: 0

CPU: After all inits, caps: bfebfbff 20000000 00000000 00000080 0000641d
00000000 00000000

Intel machine check architecture supported.

Intel machine check reporting enabled on CPU#0.

CPU0: Intel P4/Xeon Extended MCE MSRs (24) available

CPU0: Thermal monitoring enabled

mtrr: v2.0 (20020519)

Enabling fast FPU save and restore... done.

Enabling unmasked SIMD FPU exception support... done.

Checking 'hlt' instruction... OK.

CPU0: Intel(R) Xeon(TM) CPU 3.00GHz stepping 01

Booting processor 1/1 eip 2000

Initializing CPU#1

Calibrating delay using timer specific routine.. 5985.20 BogoMIPS
(lpj=29926009)

CPU: After generic identify, caps: bfebfbff 20000000 00000000 00000000
0000641d 00000000 00000000

CPU: After vendor identify, caps: bfebfbff 20000000 00000000 00000000
0000641d 00000000 00000000

monitor/mwait feature present.

CPU: Trace cache: 12K uops, L1 D cache: 16K

CPU: L2 cache: 1024K

CPU: Physical Processor ID: 0

CPU: After all inits, caps: bfebfbff 20000000 00000000 00000080 0000641d
00000000 00000000

Intel machine check architecture supported.

Intel machine check reporting enabled on CPU#1.

CPU1: Intel P4/Xeon Extended MCE MSRs (24) available

CPU1: Thermal monitoring enabled

CPU1: Intel(R) Xeon(TM) CPU 3.00GHz stepping 01

Booting processor 2/6 eip 2000

Initializing CPU#2

Calibrating delay using timer specific routine.. 5985.22 BogoMIPS
(lpj=29926134)

CPU: After generic identify, caps: bfebfbff 20000000 00000000 00000000
0000641d 00000000 00000000

CPU: After vendor identify, caps: bfebfbff 20000000 00000000 00000000
0000641d 00000000 00000000

monitor/mwait feature present.

CPU: Trace cache: 12K uops, L1 D cache: 16K

CPU: L2 cache: 1024K

CPU: Physical Processor ID: 3

CPU: After all inits, caps: bfebfbff 20000000 00000000 00000080 0000641d
00000000 00000000

Intel machine check architecture supported.

Intel machine check reporting enabled on CPU#2.

CPU2: Intel P4/Xeon Extended MCE MSRs (24) available

CPU2: Thermal monitoring enabled

CPU2: Intel(R) Xeon(TM) CPU 3.00GHz stepping 01

Booting processor 3/7 eip 2000

Initializing CPU#3

Calibrating delay using timer specific routine.. 5985.23 BogoMIPS
(lpj=29926183)

CPU: After generic identify, caps: bfebfbff 20000000 00000000 00000000
0000641d 00000000 00000000

CPU: After vendor identify, caps: bfebfbff 20000000 00000000 00000000
0000641d 00000000 00000000

monitor/mwait feature present.

CPU: Trace cache: 12K uops, L1 D cache: 16K

CPU: L2 cache: 1024K

CPU: Physical Processor ID: 3

CPU: After all inits, caps: bfebfbff 20000000 00000000 00000080 0000641d
00000000 00000000

Intel machine check architecture supported.

Intel machine check reporting enabled on CPU#3.

CPU3: Intel P4/Xeon Extended MCE MSRs (24) available

CPU3: Thermal monitoring enabled

CPU3: Intel(R) Xeon(TM) CPU 3.00GHz stepping 01

Total of 4 processors activated (23945.47 BogoMIPS).

ENABLING IO-APIC IRQs

..TIMER: vector=0x31 apic1=0 pin1=2 apic2=-1 pin2=-1

checking TSC synchronization across 4 CPUs: passed.

Brought up 4 CPUs

NET: Registered protocol family 16

ACPI: bus type pci registered

PCI: PCI BIOS revision 2.10 entry at 0xf0031, last bus=9

PCI: Using MMCONFIG

ACPI: Subsystem revision 20050902

ACPI: Interpreter enabled

ACPI: Using IOAPIC for interrupt routing

ACPI: PCI Root Bridge [PCI0] (0000:00)

PCI: Probing PCI hardware (bus 00)

PCI quirk: region 0400-047f claimed by ICH4 ACPI/GPIO/TCO

PCI quirk: region 0500-053f claimed by ICH4 GPIO

PCI: PXH quirk detected, disabling MSI for SHPC device

PCI: PXH quirk detected, disabling MSI for SHPC device

Boot video device is 0000:01:0c.0

PCI: Transparent bridge - 0000:00:1e.0

ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]

ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.EPA0.PXHA._PRT]

ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.EPA0.PXHB._PRT]

ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.EPB0._PRT]

ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.EPC0._PRT]

ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.EPC1.DBSA._PRT]

ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.EPC1.DBSB._PRT]

ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]

ACPI: PCI Interrupt Link [LNKA] (IRQs *7)

ACPI: PCI Interrupt Link [LNKB] (IRQs *7)

ACPI: PCI Interrupt Link [LNKC] (IRQs *7)

ACPI: PCI Interrupt Link [LNKD] (IRQs *7)

ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11 12 14 15) *0,
disabled.

ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12 14 15) *0,
disabled.

ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 11 12 14 15) *0,
disabled.

ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 *10 11 12 14 15)

Linux Plug and Play Support v0.97 (c) Adam Belay

pnp: PnP ACPI init

pnp: PnP ACPI: found 12 devices

SCSI subsystem initialized

usbcore: registered new driver usbfs

usbcore: registered new driver hub

PCI: Using ACPI for IRQ routing

PCI: Routing PCI interrupts for all devices because "pci=routeirq" specified

ACPI: PCI Interrupt 0000:00:01.0[A] -> GSI 16 (level, low) -> IRQ 16

ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 16 (level, low) -> IRQ 16

ACPI: PCI Interrupt 0000:00:04.0[A] -> GSI 16 (level, low) -> IRQ 16

ACPI: PCI Interrupt 0000:00:06.0[A] -> GSI 16 (level, low) -> IRQ 16

ACPI: PCI Interrupt 0000:00:07.0[A] -> GSI 16 (level, low) -> IRQ 16

ACPI: PCI Interrupt 0000:00:1d.0[A] -> GSI 16 (level, low) -> IRQ 16

ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 19 (level, low) -> IRQ 17

ACPI: PCI Interrupt 0000:00:1d.2[C] -> GSI 18 (level, low) -> IRQ 18

ACPI: PCI Interrupt 0000:00:1d.7[D] -> GSI 23 (level, low) -> IRQ 19

ACPI: PCI Interrupt 0000:00:1f.2[A] -> GSI 18 (level, low) -> IRQ 18

ACPI: PCI Interrupt 0000:00:1f.3[B] -> GSI 17 (level, low) -> IRQ 20

ACPI: PCI Interrupt 0000:08:04.0[A] -> GSI 52 (level, low) -> IRQ 21

ACPI: PCI Interrupt 0000:08:04.1[B] -> GSI 53 (level, low) -> IRQ 22

ACPI: PCI Interrupt 0000:01:0c.0[A] -> GSI 17 (level, low) -> IRQ 20

pnp: 00:09: ioport range 0x680-0x69f has been reserved

pnp: 00:09: ioport range 0x640-0x65f has been reserved

pnp: 00:09: ioport range 0x600-0x60f has been reserved

pnp: 00:09: ioport range 0x6c0-0x6df has been reserved

pnp: 00:09: ioport range 0x700-0x71f has been reserved

pnp: 00:09: ioport range 0x720-0x73f has been reserved

pnp: 00:09: ioport range 0x740-0x74f has been reserved

PCI: Bridge: 0000:07:00.0

IO window: disabled.

MEM window: disabled.

PREFETCH window: disabled.

PCI: Bridge: 0000:07:00.2

IO window: c000-cfff

MEM window: fe900000-fe9fffff

PREFETCH window: disabled.

PCI: Bridge: 0000:00:02.0

IO window: c000-cfff

MEM window: fe900000-feafffff

PREFETCH window: disabled.

PCI: Bridge: 0000:00:04.0

IO window: disabled.

MEM window: disabled.

PREFETCH window: disabled.

PCI: Bridge: 0000:00:06.0

IO window: disabled.

MEM window: disabled.

PREFETCH window: disabled.

PCI: Bridge: 0000:02:00.0

IO window: disabled.

MEM window: disabled.

PREFETCH window: disabled.

PCI: Bridge: 0000:02:00.2

IO window: disabled.

MEM window: disabled.

PREFETCH window: disabled.

PCI: Bridge: 0000:00:07.0

IO window: disabled.

MEM window: fe800000-fe8fffff

PREFETCH window: disabled.

PCI: Bridge: 0000:00:1e.0

IO window: b000-bfff

MEM window: fc700000-fe7fffff

PREFETCH window: e2000000-e20fffff

ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 16 (level, low) -> IRQ 16

PCI: Setting latency timer of device 0000:00:02.0 to 64

PCI: Setting latency timer of device 0000:07:00.0 to 64

PCI: Setting latency timer of device 0000:07:00.2 to 64

ACPI: PCI Interrupt 0000:00:04.0[A] -> GSI 16 (level, low) -> IRQ 16

PCI: Setting latency timer of device 0000:00:04.0 to 64

ACPI: PCI Interrupt 0000:00:06.0[A] -> GSI 16 (level, low) -> IRQ 16

PCI: Setting latency timer of device 0000:00:06.0 to 64

ACPI: PCI Interrupt 0000:00:07.0[A] -> GSI 16 (level, low) -> IRQ 16

PCI: Setting latency timer of device 0000:00:07.0 to 64

PCI: Setting latency timer of device 0000:02:00.0 to 64

PCI: Setting latency timer of device 0000:02:00.2 to 64

PCI: Setting latency timer of device 0000:00:1e.0 to 64

Machine check exception polling timer started.

apm: BIOS version 1.2 Flags 0x03 (Driver version 1.16ac)

apm: disabled - APM is not SMP safe.

audit: initializing netlink socket (disabled)

audit(1143712094.670:1): initialized

highmem bounce pool size: 64 pages

VFS: Disk quotas dquot_6.5.1

Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)

Coda Kernel/Venus communications, v5.3.20, coda@cs.cmu.edu

Installing knfsd (copyright (C) 1996 okir@monad.swb.de).

NTFS driver 2.1.25 [Flags: R/W].

SGI XFS with ACLs, large block numbers, no debug enabled

SGI XFS Quota Management subsystem

Initializing Cryptographic API

io scheduler noop registered

io scheduler anticipatory registered

io scheduler deadline registered

io scheduler cfq registered

ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 16 (level, low) -> IRQ 16

PCI: Setting latency timer of device 0000:00:02.0 to 64

Allocate Port Service[pcie00]

Allocate Port Service[pcie01]

ACPI: PCI Interrupt 0000:00:04.0[A] -> GSI 16 (level, low) -> IRQ 16

PCI: Setting latency timer of device 0000:00:04.0 to 64

Allocate Port Service[pcie00]

Allocate Port Service[pcie01]

ACPI: PCI Interrupt 0000:00:06.0[A] -> GSI 16 (level, low) -> IRQ 16

PCI: Setting latency timer of device 0000:00:06.0 to 64

Allocate Port Service[pcie00]

Allocate Port Service[pcie01]

ACPI: PCI Interrupt 0000:00:07.0[A] -> GSI 16 (level, low) -> IRQ 16

PCI: Setting latency timer of device 0000:00:07.0 to 64

Allocate Port Service[pcie00]

Allocate Port Service[pcie01]

kobject_register failed for radeonfb (-17)

[<c010360c>] dump_stack+0x16/0x1a

[<c02ad2d5>] kobject_register+0x37/0x4f

[<c0319cbb>] bus_add_driver+0x46/0xa6

[<c031a523>] driver_register+0x41/0x46

[<c02b5047>] __pci_register_driver+0x86/0x97

[<c05e01ef>] radeonfb_old_init+0x38/0x41

[<c05cc848>] do_initcalls+0x46/0x96

[<c05cc8cd>] do_basic_setup+0x21/0x23

[<c01003a5>] init+0xa2/0x199

[<c0100ea5>] kernel_thread_helper+0x5/0xb

Real Time Clock Driver v1.12

ipmi message handler version 38.0

PNP: PS/2 Controller [PNP0f03:PS2M] at 0x60,0x64 irq 12

PNP: PS/2 controller doesn't have KBD irq; using default 1

serio: i8042 AUX port at 0x60,0x64 irq 12

serio: i8042 KBD port at 0x60,0x64 irq 1

Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled

serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A

serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A

00:04: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A

00:05: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A

loop: loaded (max 8 devices)

ibmasm: IBM ASM Service Processor Driver version 1.0 loaded

Intel(R) PRO/1000 Network Driver - version 6.3.9-k4

Copyright (c) 1999-2005 Intel Corporation.

ACPI: PCI Interrupt 0000:08:04.0[A] -> GSI 52 (level, low) -> IRQ 21

e1000: 0000:08:04.0: e1000_probe: (PCI-X:100MHz:64-bit) 00:07:e9:32:e6:d8

e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection

ACPI: PCI Interrupt 0000:08:04.1[B] -> GSI 53 (level, low) -> IRQ 22

e1000: 0000:08:04.1: e1000_probe: (PCI-X:100MHz:64-bit) 00:07:e9:32:e6:d9

e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection

Ethernet Channel Bonding Driver: v2.6.5 (November 4, 2005)

bonding: Warning: either miimon or arp_interval and arp_ip_target module
parameters must be specified, otherwise bonding will not detect link
failures! see bonding.txt for details.

e100: Intel(R) PRO/100 Network Driver, 3.4.14-k4-NAPI

e100: Copyright(c) 1999-2005 Intel Corporation

tun: Universal TUN/TAP device driver, 1.6

tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>

Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2

ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx

Probing IDE interface ide0...

Probing IDE interface ide1...

3ware 9000 Storage Controller device driver for Linux v2.26.02.004.

libata version 1.20 loaded.

ata_piix 0000:00:1f.2: version 1.05

ACPI: PCI Interrupt 0000:00:1f.2[A] -> GSI 18 (level, low) -> IRQ 18

PCI: Setting latency timer of device 0000:00:1f.2 to 64

ata1: SATA max UDMA/133 cmd 0xEC00 ctl 0xE802 bmdma 0xDC00 irq 18

ata2: SATA max UDMA/133 cmd 0xE400 ctl 0xE002 bmdma 0xDC08 irq 18

ata1: dev 0 cfg 49:2f00 82:346b 83:7f61 84:4003 85:3469 86:3c41 87:4003
88:407f

ata1: dev 0 ATA-6, max UDMA/133, 390721968 sectors: LBA48

ata1: dev 0 configured for UDMA/133

scsi0 : ata_piix

ata2: dev 0 cfg 49:2f00 82:346b 83:7f61 84:4003 85:3469 86:3c41 87:4003
88:407f

ata2: dev 0 ATA-6, max UDMA/133, 390721968 sectors: LBA48

ata2: dev 0 configured for UDMA/133

scsi1 : ata_piix

Vendor: ATA Model: WDC WD2000JD-19H Rev: 08.0

Type: Direct-Access ANSI SCSI revision: 05

Vendor: ATA Model: WDC WD2000JD-19H Rev: 08.0

Type: Direct-Access ANSI SCSI revision: 05

SCSI device sda: 390721968 512-byte hdwr sectors (200050 MB)

SCSI device sda: drive cache: write back

SCSI device sda: 390721968 512-byte hdwr sectors (200050 MB)

SCSI device sda: drive cache: write back

sda: sda1 sda2

sd 0:0:0:0: Attached scsi disk sda

SCSI device sdb: 390721968 512-byte hdwr sectors (200050 MB)

SCSI device sdb: drive cache: write back

SCSI device sdb: 390721968 512-byte hdwr sectors (200050 MB)

SCSI device sdb: drive cache: write back

sdb: sdb1 sdb2

sd 1:0:0:0: Attached scsi disk sdb

sd 0:0:0:0: Attached scsi generic sg0 type 0

sd 1:0:0:0: Attached scsi generic sg1 type 0

aoe: aoe_init: AoE v2.6-14 initialised.

usbmon: debugfs is not available

ACPI: PCI Interrupt 0000:00:1d.7[D] -> GSI 23 (level, low) -> IRQ 19

PCI: Setting latency timer of device 0000:00:1d.7 to 64

ehci_hcd 0000:00:1d.7: EHCI Host Controller

ehci_hcd 0000:00:1d.7: debug port 1

PCI: cache line size of 128 is not supported by device 0000:00:1d.7

ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 1

ehci_hcd 0000:00:1d.7: irq 19, io mem 0xfebffc00

ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004

hub 1-0:1.0: USB hub found

hub 1-0:1.0: 6 ports detected

ohci_hcd: 2005 April 22 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI)

USB Universal Host Controller Interface driver v2.3

ACPI: PCI Interrupt 0000:00:1d.0[A] -> GSI 16 (level, low) -> IRQ 16

PCI: Setting latency timer of device 0000:00:1d.0 to 64

uhci_hcd 0000:00:1d.0: UHCI Host Controller

uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2

uhci_hcd 0000:00:1d.0: irq 16, io base 0x0000d000

hub 2-0:1.0: USB hub found

hub 2-0:1.0: 2 ports detected

ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 19 (level, low) -> IRQ 17

PCI: Setting latency timer of device 0000:00:1d.1 to 64

uhci_hcd 0000:00:1d.1: UHCI Host Controller

uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3

uhci_hcd 0000:00:1d.1: irq 17, io base 0x0000d400

hub 3-0:1.0: USB hub found

hub 3-0:1.0: 2 ports detected

ACPI: PCI Interrupt 0000:00:1d.2[C] -> GSI 18 (level, low) -> IRQ 18

PCI: Setting latency timer of device 0000:00:1d.2 to 64

uhci_hcd 0000:00:1d.2: UHCI Host Controller

uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4

uhci_hcd 0000:00:1d.2: irq 18, io base 0x0000d800

hub 4-0:1.0: USB hub found

hub 4-0:1.0: 2 ports detected

sl811: driver sl811-hcd, 19 May 2005

Initializing USB Mass Storage driver...

usbcore: registered new driver usb-storage

USB Mass Storage support registered.

mice: PS/2 mouse device common for all mice

I2O subsystem v1.288

i2o: max drivers = 8

I2O ProcFS OSM v1.145

i2c /dev entries driver

i2c-sis96x version 1.0.0

pc87360: PC8736x not detected, module not inserted.

md: linear personality registered as nr 1

md: raid0 personality registered as nr 2

md: raid1 personality registered as nr 3

md: raid5 personality registered as nr 4

raid5: automatically using best checksumming function: pIII_sse

pIII_sse : 4862.400 MB/sec

raid5: using function: pIII_sse (4862.400 MB/sec)

md: multipath personality registered as nr 7

md: faulty personality registered as nr 10

md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27

md: bitmap version 4.39

device-mapper: 4.4.0-ioctl (2005-01-12) initialised: dm-devel@redhat.com

device-mapper: dm-multipath version 1.0.4 loaded

device-mapper: dm-round-robin version 1.0.0 loaded

device-mapper: dm-emc version 0.0.3 loaded

NET: Registered protocol family 2

IP route cache hash table entries: 262144 (order: 8, 1048576 bytes)

TCP established hash table entries: 524288 (order: 10, 4194304 bytes)

TCP bind hash table entries: 65536 (order: 7, 524288 bytes)

TCP: Hash tables configured (established 524288 bind 65536)

TCP reno registered

ip_conntrack version 2.4 (8192 buckets, 65536 max) - 232 bytes per conntrack

ip_tables: (C) 2000-2002 Netfilter core team

ipt_recent v0.3.1: Stephen Frost <sfrost@snowman.net>.
http://snowman.net/projects/ipt_recent/

arp_tables: (C) 2002 David S. Miller

TCP bic registered

NET: Registered protocol family 1

NET: Registered protocol family 10

lo: Disabled Privacy Extensions

IPv6 over IPv4 tunneling driver

ip6_tables: (C) 2000-2002 Netfilter core team

registering ipv6 mark target

NET: Registered protocol family 17

p4-clockmod: P4/Xeon(TM) CPU On-Demand Clock Modulation available

Using IPI Shortcut mode

ADDRCONF(NETDEV_UP): eth0: link is not ready

e1000: eth0: e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex

ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

Sending DHCP requests ., OK

IP-Config: Got DHCP answer from 192.168.2.1, my address is 192.168.2.50

IP-Config: Complete:

device=eth0, addr=192.168.2.50, mask=255.255.255.0, gw=192.168.2.100,

host=xeon, domain=, nis-domain=(none),

bootserver=192.168.2.1, rootserver=192.168.2.1, rootpath=/NFS/ROOT-XEON1/

md: Autodetecting RAID arrays.

md: invalid raid superblock magic on sda2

md: sda2 has invalid sb, not importing!

md: invalid raid superblock magic on sdb2

md: sdb2 has invalid sb, not importing!

md: autorun ...

md: considering sdb1 ...

md: adding sdb1 ...

md: adding sda1 ...

md: created md10

md: bind<sda1>

md: bind<sdb1>

md: running: <sdb1><sda1>

md10: bitmap initialized from disk: read 9/9 pages, set 0 bits, status: 0

created bitmap (131 pages) for device md10

raid1: raid set md10 active with 2 out of 2 mirrors

md: ... autorun DONE.

Looking up port of RPC 100003/2 on 192.168.2.1

Looking up port of RPC 100005/1 on 192.168.2.1

VFS: Mounted root (nfs filesystem) readonly.

Freeing unused kernel memory: 248k freed

program scsi_unique_id is using a deprecated SCSI ioctl, please convert it
to SG_IO

program scsi_unique_id is using a deprecated SCSI ioctl, please convert it
to SG_IO

program scsi_unique_id is using a deprecated SCSI ioctl, please convert it
to SG_IO

program scsi_unique_id is using a deprecated SCSI ioctl, please convert it
to SG_IO

program scsi_unique_id is using a deprecated SCSI ioctl, please convert it
to SG_IO

program scsi_unique_id is using a deprecated SCSI ioctl, please convert it
to SG_IO

program scsi_unique_id is using a deprecated SCSI ioctl, please convert it
to SG_IO

program scsi_unique_id is using a deprecated SCSI ioctl, please convert it
to SG_IO

eth0: no IPv6 routers present


Cheers,
Janos



----- Original Message ----- 
From: "Herbert Xu" <herbert@gondor.apana.org.au>
To: "Boris B. Zhmurov" <bb@kernelpanic.ru>
Cc: "David S. Miller" <davem@davemloft.net>; <jesse.brandeburg@intel.com>;
<nipsy@bitgnome.net>; <jrlundgren@gmail.com>; <cat@zip.com.au>;
<djani22@dynamicweb.hu>; <yoseph.basri@gmail.com>; <mykleb@no.ibm.com>;
<olel@ans.pl>; <michal@feix.cz>; <chris@scorpion.nl>;
<netdev@vger.kernel.org>; <jesse.brandeburg@gmail.com>;
<E1000-devel@lists.sourceforge.net>
Sent: Thursday, March 30, 2006 12:12 PM
Subject: Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...


> On Thu, Mar 30, 2006 at 10:02:01AM +0000, Boris B. Zhmurov wrote:
> >
> > [zhmurov@builds linux-2.6.16]$ patch -p1 <
> >
../../../SOURCES/linux-2.6.16-e1000-try-to-fix-assertion_sk_forward_alloc_fa
iled_by_Herbert_Xu.patch
> >
> > patching file drivers/net/e1000/e1000_main.c
> > Reversed (or previously applied) patch detected!  Assume -R? [n]
> >
> > Herbert, is that patch already included in 2.6.16.1?
>
> Not really.  It's just patch being silly (or too smart :)
>
> Here it is again rediffed against 2.6.16.
> -- 
> Visit Openswan at http://www.openswan.org/
> Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
>
>
>  _____________ NOD32 1.584 (20031220) Információ _____________
>
> Az üzenetet a NOD32 Antivirus System megvizsgálta.
> http://www.nod32.hu
>
>



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Herbert Xu.

On 30.03.2006 14:12 you said the following:

> On Thu, Mar 30, 2006 at 10:02:01AM +0000, Boris B. Zhmurov wrote:
> 
>>[zhmurov@builds linux-2.6.16]$ patch -p1 < 
>>../../../SOURCES/linux-2.6.16-e1000-try-to-fix-assertion_sk_forward_alloc_failed_by_Herbert_Xu.patch 
>>
>>patching file drivers/net/e1000/e1000_main.c
>>Reversed (or previously applied) patch detected!  Assume -R? [n]
>>
>>Herbert, is that patch already included in 2.6.16.1?
> 
> 
> Not really.  It's just patch being silly (or too smart :)
> 
> Here it is again rediffed against 2.6.16.


Nope, with this patch the problem still exists. After 25 min. uptime 
with patched kernel 2.6.16.1, I have:

Mar 30 16:30:31 msk4 kernel: KERNEL: assertion (!sk->sk_forward_alloc) 
failed at net/core/stream.c (283)
Mar 30 16:30:31 msk4 kernel: KERNEL: assertion (!sk->sk_forward_alloc) 
failed at net/ipv4/af_inet.c (150)
Mar 30 17:05:42 msk4 kernel: KERNEL: assertion (!sk->sk_forward_alloc) 
failed at net/core/stream.c (283)
Mar 30 17:05:42 msk4 kernel: KERNEL: assertion (!sk->sk_forward_alloc) 
failed at net/ipv4/af_inet.c (150)


-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[David S. Miller]

From: "Boris B. Zhmurov" <bb@kernelpanic.ru>
Date: Thu, 30 Mar 2006 17:29:09 +0400

> Hello, Herbert Xu.
> 
> On 30.03.2006 14:12 you said the following:
> 
> > On Thu, Mar 30, 2006 at 10:02:01AM +0000, Boris B. Zhmurov wrote:
> > 
> >>[zhmurov@builds linux-2.6.16]$ patch -p1 < 
> >>../../../SOURCES/linux-2.6.16-e1000-try-to-fix-assertion_sk_forward_alloc_failed_by_Herbert_Xu.patch 
> >>
> >>patching file drivers/net/e1000/e1000_main.c
> >>Reversed (or previously applied) patch detected!  Assume -R? [n]
> >>
> >>Herbert, is that patch already included in 2.6.16.1?
> > 
> > 
> > Not really.  It's just patch being silly (or too smart :)
> > 
> > Here it is again rediffed against 2.6.16.
> 
> 
> Nope, with this patch the problem still exists. After 25 min. uptime 
> with patched kernel 2.6.16.1, I have:

Can you please double and triple check that you're really running a
kernel with the fix from Herbert applied?  I make this mistake all
the time :-)

Thanks.



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, David S. Miller.

On 31.03.2006 13:12 you said the following:

> From: "Boris B. Zhmurov" <bb@kernelpanic.ru>
> Date: Thu, 30 Mar 2006 17:29:09 +0400
> 
> 
>>Hello, Herbert Xu.
>>
>>On 30.03.2006 14:12 you said the following:
>>
>>
>>>On Thu, Mar 30, 2006 at 10:02:01AM +0000, Boris B. Zhmurov wrote:
>>>
>>>
>>>>[zhmurov@builds linux-2.6.16]$ patch -p1 < 
>>>>../../../SOURCES/linux-2.6.16-e1000-try-to-fix-assertion_sk_forward_alloc_failed_by_Herbert_Xu.patch 
>>>>
>>>>patching file drivers/net/e1000/e1000_main.c
>>>>Reversed (or previously applied) patch detected!  Assume -R? [n]
>>>>
>>>>Herbert, is that patch already included in 2.6.16.1?
>>>
>>>
>>>Not really.  It's just patch being silly (or too smart :)
>>>
>>>Here it is again rediffed against 2.6.16.
>>
>>
>>Nope, with this patch the problem still exists. After 25 min. uptime 
>>with patched kernel 2.6.16.1, I have:
> 
> 
> Can you please double and triple check that you're really running a
> kernel with the fix from Herbert applied?  I make this mistake all
> the time :-)
> 
> Thanks.
> 


[zhmurov@builds redhat]$ rpmbuild --sign --rebuild --target=i686 
/usr/src/redhat/SRPMS/kernel-2.6.16-1.14.bbel4.src.rpm
Enter pass phrase:
Pass phrase is good.
..... SKIP.....
+ echo 'Patch #295 
(linux-2.6.16-e1000-try-to-fix-assertion_sk_forward_alloc_failed_by_Herbert_Xu.patch):'
Patch #295 
(linux-2.6.16-e1000-try-to-fix-assertion_sk_forward_alloc_failed_by_Herbert_Xu.patch):
+ patch -p1 -s
...... SKIP .....


And xdelta tells, that e1000.ko was modified :)


P.S.
there is src.rpm of my kernel for RHEL4 and my RHEL4-clone:
ftp://builds.kernelpanic.ru/pub/linux/BBEL/updates/kernel_of_the_day/4/SRPMS/

and yum'able repo of 2.6.16 kernels _with_ Herbert's patch :), if 
anybody interested:
ftp://builds.kernelpanic.ru/pub/linux/BBEL/updates/kernel_of_the_day/4/


-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Herbert Xu]

[-- Attachment #1: Type: text/plain, Size: 890 bytes --]

On Fri, Mar 31, 2006 at 02:16:38PM +0400, Boris B. Zhmurov wrote:
> 
> And xdelta tells, that e1000.ko was modified :)

Thanks for checking again.

Anyway, it didn't take long to find another bug in the same area.
I'm afraid this driver does seem to be full of them :)

It sets last_tx_tso in between computing the number of descriptors and
calling e1000_tx_map.  This is bad because e1000_tx_map gets the wrong
value for last_tx_tso and therefore may corrupt memory for every TSO
packet when the ring is almost full.

This bug exists on UP as well as SMP.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Please try this in conjunction with the previous patch.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[-- Attachment #2: e1000-tso.patch --]
[-- Type: text/plain, Size: 645 bytes --]

diff --git a/drivers/net/e1000/e1000_main.c b/drivers/net/e1000/e1000_main.c
index 49cd096..38aeff9 100644
--- a/drivers/net/e1000/e1000_main.c
+++ b/drivers/net/e1000/e1000_main.c
@@ -2891,7 +2891,6 @@
 	}
 
 	if (likely(tso)) {
-		tx_ring->last_tx_tso = 1;
 		tx_flags |= E1000_TX_FLAGS_TSO;
 	} else if (likely(e1000_tx_csum(adapter, tx_ring, skb)))
 		tx_flags |= E1000_TX_FLAGS_CSUM;
@@ -2905,6 +2904,8 @@
 	e1000_tx_queue(adapter, tx_ring, tx_flags,
 	               e1000_tx_map(adapter, tx_ring, skb, first,
 	                            max_per_txd, nr_frags, mss));
+
+	tx_ring->last_tx_tso = tso;
 
 	netdev->trans_start = jiffies;
 

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[David S. Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 31 Mar 2006 21:39:56 +1100

> Anyway, it didn't take long to find another bug in the same area.
> I'm afraid this driver does seem to be full of them :)

Indeed.

Thanks for picking through this some more Herbert.  I hope we got it
this time.


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, David S. Miller.

On 31.03.2006 14:45 you said the following:

> From: Herbert Xu <herbert@gondor.apana.org.au>
> Date: Fri, 31 Mar 2006 21:39:56 +1100
> 
> 
>>Anyway, it didn't take long to find another bug in the same area.
>>I'm afraid this driver does seem to be full of them :)
> 
> 
> Indeed.
> 
> Thanks for picking through this some more Herbert.  I hope we got it
> this time.


Recompiling the kernel. I need about 2 hours to get the answer...

-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Herbert Xu]

Boris B. Zhmurov <bb@kernelpanic.ru> wrote:
> 
> Recompiling the kernel. I need about 2 hours to get the answer...

BTW, if you kept the built tree it is possible to apply the patch and
then do a make which should compile just the e1000 driver.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Herbert Xu.

On 31.03.2006 14:52 you said the following:

> BTW, if you kept the built tree it is possible to apply the patch and
> then do a make which should compile just the e1000 driver.
> 
> Cheers,

Thank's for the tip, actually I knew that :) First of, I've already 
applied some other new patches from bk-commits-head. Not for the e1000 
driver. And second - I didn't keep the tree, rpmbuild cleaned it up :) 
That's why I'm recompiling entire kernel.

-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Herbert Xu.

On 31.03.2006 14:39 you said the following:

> On Fri, Mar 31, 2006 at 02:16:38PM +0400, Boris B. Zhmurov wrote:
> 
>>And xdelta tells, that e1000.ko was modified :)
> 
> 
> Thanks for checking again.
> 
> Anyway, it didn't take long to find another bug in the same area.
> I'm afraid this driver does seem to be full of them :)
> 
> It sets last_tx_tso in between computing the number of descriptors and
> calling e1000_tx_map.  This is bad because e1000_tx_map gets the wrong
> value for last_tx_tso and therefore may corrupt memory for every TSO
> packet when the ring is almost full.
> 
> This bug exists on UP as well as SMP.
> 
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
> 
> Please try this in conjunction with the previous patch.
> 
> Cheers,


David, Herbert - FYI. One of my colleague confirmed, that idea "bug 
reproducible only if there is more then one e1000 adapter onboard" is 
true. He has a 3 servers with double intel pro 1000 adapters, and that 
bug occurs. Also, he has 4 servers with double intel pro 1000 adapters 
onboard, but _only one_ of them is up. And there is no such messages in 
dmesg at all! Inetresting...

-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Andi Kleen]

On Friday 31 March 2006 14:07, Boris B. Zhmurov wrote:

> David, Herbert - FYI. One of my colleague confirmed, that idea "bug
> reproducible only if there is more then one e1000 adapter onboard" is
> true. He has a 3 servers with double intel pro 1000 adapters, and that
> bug occurs. Also, he has 4 servers with double intel pro 1000 adapters
> onboard, but _only one_ of them is up. And there is no such messages in
> dmesg at all! Inetresting...

At least all our systems with troubles seem to have more than one e1000
though. Usually only one is active though.

We're still not 100% it is actually the E1000, it is a bit hard to reproduce 
the memory corruption :/

-Andi


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Mark Nipper]

On 31 Mar 2006, Boris B. Zhmurov wrote:
> David, Herbert - FYI. One of my colleague confirmed, that idea "bug 
> reproducible only if there is more then one e1000 adapter onboard" is 
> true. He has a 3 servers with double intel pro 1000 adapters, and that 
> bug occurs. Also, he has 4 servers with double intel pro 1000 adapters 
> onboard, but _only one_ of them is up. And there is no such messages in 
> dmesg at all! Inetresting...

        This unfortunately is not the case.  I have two e1000
interfaces but only eth1 is up and in use.  And I still had
assertions.  Hopefully the two already discovered problems will
fix things up for everyone though.

-- 
Mark Nipper                                                e-contacts:
832 Tanglewood Drive                                nipsy@bitgnome.net
Bryan, Texas 77802-4013                     http://nipsy.bitgnome.net/
(979)575-3193                      AIM/Yahoo: texasnipsy ICQ: 66971617

-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GG/IT d- s++:+ a- C++$ UBL++++$ P--->+++ L+++$ !E---
W++(--) N+ o K++ w(---) O++ M V(--) PS+++(+) PE(--)
Y+ PGP t+ 5 X R tv b+++@ DI+(++) D+ G e h r++ y+(**)
------END GEEK CODE BLOCK------

---begin random quote of the moment---
Generalizations are usually flawed by exceptions.
 -- seen at http://wunderland.com/
----end random quote of the moment----


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Mark Nipper.

On 31.03.2006 16:10 you said the following:

>         This unfortunately is not the case.  I have two e1000
> interfaces but only eth1 is up and in use.  And I still had
> assertions.


Can you switch to eth0? There is no problem with _eth0_, my friend says.


 > And I still had
 > assertions.


I'm already using kernel with second Herbert's patch. We'll see...


-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Herbert Xu]

[-- Attachment #1: Type: text/plain, Size: 459 bytes --]

On Fri, Mar 31, 2006 at 04:23:02PM +0400, Boris B. Zhmurov wrote:
> 
> I'm already using kernel with second Herbert's patch. We'll see...

If it still fails, here is a debugging patch which should tell us
whether we need to look elsewhere.

Thanks,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

[-- Attachment #2: e1000-debug.patch --]
[-- Type: text/plain, Size: 662 bytes --]

diff --git a/drivers/net/e1000/e1000_main.c b/drivers/net/e1000/e1000_main.c
index 49cd096..64ac6f4 100644
--- a/drivers/net/e1000/e1000_main.c
+++ b/drivers/net/e1000/e1000_main.c
@@ -2906,6 +2906,13 @@
 	               e1000_tx_map(adapter, tx_ring, skb, first,
 	                            max_per_txd, nr_frags, mss));
 
+	tso = tx_ring->next_to_use - first;
+	if (tso < 0)
+		tso += tx_ring->count;
+	if (unlikely(tso > count))
+		printk(KERN_ERR "e1000 bug: mss=%d, len=%d, frags=%d, est=%d, actual=%d\n",
+		       mss, skb->len, nr_frags, count, tso);
+
 	netdev->trans_start = jiffies;
 
 	/* Make sure there is space in the ring for the next send. */

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Herbert Xu.

On 31.03.2006 16:35 you said the following:

> On Fri, Mar 31, 2006 at 04:23:02PM +0400, Boris B. Zhmurov wrote:
> 
>>I'm already using kernel with second Herbert's patch. We'll see...
> 
> 
> If it still fails


Not yet. But give it a time :)

-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Mark Nipper]

On 31 Mar 2006, Herbert Xu wrote:
> If it still fails, here is a debugging patch which should tell us
> whether we need to look elsewhere.

        After three days and some hours, I finally saw another
event:
---
Apr  3 13:40:53 king kernel: KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (283)
Apr  3 13:40:53 king kernel: KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (150)
---

but as with the other person who reported recently, I also did
not receive the extra debugging output from Herbert's latest
patch.

        Anyway, this happened with 2.6.16.1 and Herbert's three
patches.  Let me know if you want me to try anything different.

        I guess all of this will just go away with the latest
driver version since those of us running at 100Mbps will no
longer have TSO enabled?

-- 
Mark Nipper                                                e-contacts:
832 Tanglewood Drive                                nipsy@bitgnome.net
Bryan, Texas 77802-4013                     http://nipsy.bitgnome.net/
(979)575-3193                      AIM/Yahoo: texasnipsy ICQ: 66971617

-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GG/IT d- s++:+ a- C++$ UBL++++$ P--->+++ L+++$ !E---
W++(--) N+ o K++ w(---) O++ M V(--) PS+++(+) PE(--)
Y+ PGP t+ 5 X R tv b+++@ DI+(++) D+ G e h r++ y+(**)
------END GEEK CODE BLOCK------

---begin random quote of the moment---
He hoped and prayed that there wasn't an afterlife. Then he
realized there was a contradiction involved here and merely
hoped that there wasn't an afterlife.
 -- Douglas Adams
----end random quote of the moment----


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Phil Oester]

On Mon, Apr 03, 2006 at 04:01:23PM -0500, Mark Nipper wrote:
>         After three days and some hours, I finally saw another
> event:

Ack, same here.  Looked hopeful, but finally saw the error today.

Phil


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Phil Oester.

On 04.04.2006 01:39 you said the following:

> On Mon, Apr 03, 2006 at 04:01:23PM -0500, Mark Nipper wrote:
> 
>>        After three days and some hours, I finally saw another
>>event:
> 
> 
> Ack, same here.  Looked hopeful, but finally saw the error today.
> 
> Phil


[root@msk4 ~]# dmesg |grep assertion |wc -l
176

[root@msk4 ~]# uptime
  02:00:01 up 3 days,  7:31,  2 users,  load average: 1.32, 0.59, 0.41

-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Jesse Brandeburg]

On Mon, 3 Apr 2006, Boris B. Zhmurov wrote:

> 
> Hello, Phil Oester.
> 
> On 04.04.2006 01:39 you said the following:
> 
> > On Mon, Apr 03, 2006 at 04:01:23PM -0500, Mark Nipper wrote:
> >
> >>        After three days and some hours, I finally saw another
> >>event:
> >
> >
> > Ack, same here.  Looked hopeful, but finally saw the error today.
> >
> > Phil
> 
> 
> [root@msk4 ~]# dmesg |grep assertion |wc -l
> 176
> 
> [root@msk4 ~]# uptime
>   02:00:01 up 3 days,  7:31,  2 users,  load average: 1.32, 0.59, 0.41
> 

Some earlier had proposed that this problem appeared in 2.6.12, which was 
the introduction of the 6.X series e1000 driver.

If someone would like to, can they try the 5.6.10.1 driver from the 
2.6.11.X kernel?

I'd like it if you can stick with your current kernel, and if you have 
trouble building the driver, go ahead and try the 5.6.10.1 driver from 
http://prdownloads.sf.net/e1000

I'll also send a patch today to back-rev the xmit routine to the 5.6.10.1 
state.

Jesse


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Jesse Brandeburg]

On Wed, 5 Apr 2006, Jesse Brandeburg wrote:

> I'll also send a patch today to back-rev the xmit routine to the 5.6.10.1 
> state.

I'm in a bit of a hurry, but I wanted to send these debug patches out.  
Forgive me if my mailer decides to munge them.
I'd suggest trying the first one and then both together.
The first fixes up the tso function only to be like 5.6.10.1-k2.
The second builds on the first, and incorporates more of the tx changes.
  
I built and tested the driver with patches on 2.6.16, with pci-x adapters.  
I removed some workarounds for PCIe adapters, but I dont think anyone 
having this problem has a PCIe adapter anyway.  I saw no TX hangs and ran 
some bi-directional tests, so i think the driver should work okay.  Just 
warning you I did minimal testing.

*********************
e1000: transmit the old fashioned way

It seems back in the day of 2.6.11, there were no sk_forward_alloc
asserions.  Forward port that transmit code to see if it fixes the issues
in today's kernel.  Unfortunately it doesn't have all the bug fixes that
the current code has, but if we get transmit timeouts we can add in
workarounds appropriately.

this changes only the e1000_tso function

Signed-off-by: Jesse Brandeburg <jesse.brandeburg@intel.com>

---

 drivers/net/e1000/e1000_main.c |   37 ++++++-------------------------------
 1 files changed, 6 insertions(+), 31 deletions(-)

diff --git a/drivers/net/e1000/e1000_main.c b/drivers/net/e1000/e1000_main.c
old mode 100644
new mode 100755
index e3d3778..2b8fb87
--- a/drivers/net/e1000/e1000_main.c
+++ b/drivers/net/e1000/e1000_main.c
@@ -2519,57 +2519,34 @@ e1000_tso(struct e1000_adapter *adapter,
 {
 #ifdef NETIF_F_TSO
 	struct e1000_context_desc *context_desc;
-	struct e1000_buffer *buffer_info;
 	unsigned int i;
 	uint32_t cmd_length = 0;
 	uint16_t ipcse = 0, tucse, mss;
 	uint8_t ipcss, ipcso, tucss, tucso, hdr_len;
-	int err;
 
 	if (skb_shinfo(skb)->tso_size) {
-		if (skb_header_cloned(skb)) {
-			err = pskb_expand_head(skb, 0, 0, GFP_ATOMIC);
-			if (err)
-				return err;
-		}
-
 		hdr_len = ((skb->h.raw - skb->data) + (skb->h.th->doff << 2));
 		mss = skb_shinfo(skb)->tso_size;
-		if (skb->protocol == ntohs(ETH_P_IP)) {
-			skb->nh.iph->tot_len = 0;
-			skb->nh.iph->check = 0;
-			skb->h.th->check =
-				~csum_tcpudp_magic(skb->nh.iph->saddr,
+		skb->nh.iph->tot_len = 0;
+		skb->nh.iph->check = 0;
+		skb->h.th->check = ~csum_tcpudp_magic(skb->nh.iph->saddr,
 						   skb->nh.iph->daddr,
 						   0,
 						   IPPROTO_TCP,
 						   0);
-			cmd_length = E1000_TXD_CMD_IP;
-			ipcse = skb->h.raw - skb->data - 1;
-#ifdef NETIF_F_TSO_IPV6
-		} else if (skb->protocol == ntohs(ETH_P_IPV6)) {
-			skb->nh.ipv6h->payload_len = 0;
-			skb->h.th->check =
-				~csum_ipv6_magic(&skb->nh.ipv6h->saddr,
-						 &skb->nh.ipv6h->daddr,
-						 0,
-						 IPPROTO_TCP,
-						 0);
-			ipcse = 0;
-#endif
-		}
 		ipcss = skb->nh.raw - skb->data;
 		ipcso = (void *)&(skb->nh.iph->check) - (void *)skb->data;
+		ipcse = skb->h.raw - skb->data - 1;
 		tucss = skb->h.raw - skb->data;
 		tucso = (void *)&(skb->h.th->check) - (void *)skb->data;
 		tucse = 0;
 
 		cmd_length |= (E1000_TXD_CMD_DEXT | E1000_TXD_CMD_TSE |
-			       E1000_TXD_CMD_TCP | (skb->len - (hdr_len)));
+			       E1000_TXD_CMD_IP | E1000_TXD_CMD_TCP |
+			       (skb->len - (hdr_len)));
 
 		i = tx_ring->next_to_use;
 		context_desc = E1000_CONTEXT_DESC(*tx_ring, i);
-		buffer_info = &tx_ring->buffer_info[i];
 
 		context_desc->lower_setup.ip_fields.ipcss  = ipcss;
 		context_desc->lower_setup.ip_fields.ipcso  = ipcso;
@@ -2581,8 +2558,6 @@ e1000_tso(struct e1000_adapter *adapter,
 		context_desc->tcp_seg_setup.fields.hdr_len = hdr_len;
 		context_desc->cmd_and_length = cpu_to_le32(cmd_length);
 
-		buffer_info->time_stamp = jiffies;
-
 		if (++i == tx_ring->count) i = 0;
 		tx_ring->next_to_use = i;
 



*************************************


e1000: implement old xmit_frame

It seems back in the day of 2.6.11, there were no sk_forward_alloc
asserions.  Forward port that transmit code to see if it fixes the issues
in today's kernel.  Unfortunately it doesn't have all the bug fixes that
the current code has, but if we get transmit timeouts we can add in
workarounds appropriately.

this changes the e1000_xmit_frame function, and some ancilliaries

Signed-off-by: Jesse Brandeburg <jesse.brandeburg@intel.com>

---

 drivers/net/e1000/e1000_main.c |   90 ++--------------------------------------
 1 files changed, 4 insertions(+), 86 deletions(-)

diff --git a/drivers/net/e1000/e1000_main.c b/drivers/net/e1000/e1000_main.c
index 2b8fb87..18790b6 100755
--- a/drivers/net/e1000/e1000_main.c
+++ b/drivers/net/e1000/e1000_main.c
@@ -2725,10 +2725,7 @@ e1000_tx_queue(struct e1000_adapter *ada
 	if (likely(tx_flags & E1000_TX_FLAGS_TSO)) {
 		txd_lower |= E1000_TXD_CMD_DEXT | E1000_TXD_DTYP_D |
 		             E1000_TXD_CMD_TSE;
-		txd_upper |= E1000_TXD_POPTS_TXSM << 8;
-
-		if (likely(tx_flags & E1000_TX_FLAGS_IPV4))
-			txd_upper |= E1000_TXD_POPTS_IXSM << 8;
+		txd_upper |= (E1000_TXD_POPTS_IXSM | E1000_TXD_POPTS_TXSM) << 8;
 	}
 
 	if (likely(tx_flags & E1000_TX_FLAGS_CSUM)) {
@@ -2803,41 +2800,6 @@ no_fifo_stall_required:
 	return 0;
 }
 
-#define MINIMUM_DHCP_PACKET_SIZE 282
-static inline int
-e1000_transfer_dhcp_info(struct e1000_adapter *adapter, struct sk_buff *skb)
-{
-	struct e1000_hw *hw =  &adapter->hw;
-	uint16_t length, offset;
-	if (vlan_tx_tag_present(skb)) {
-		if (!((vlan_tx_tag_get(skb) == adapter->hw.mng_cookie.vlan_id) &&
-			( adapter->hw.mng_cookie.status &
-			  E1000_MNG_DHCP_COOKIE_STATUS_VLAN_SUPPORT)) )
-			return 0;
-	}
-	if ((skb->len > MINIMUM_DHCP_PACKET_SIZE) && (!skb->protocol)) {
-		struct ethhdr *eth = (struct ethhdr *) skb->data;
-		if ((htons(ETH_P_IP) == eth->h_proto)) {
-			const struct iphdr *ip =
-				(struct iphdr *)((uint8_t *)skb->data+14);
-			if (IPPROTO_UDP == ip->protocol) {
-				struct udphdr *udp =
-					(struct udphdr *)((uint8_t *)ip +
-						(ip->ihl << 2));
-				if (ntohs(udp->dest) == 67) {
-					offset = (uint8_t *)udp + 8 - skb->data;
-					length = skb->len - offset;
-
-					return e1000_mng_write_dhcp_info(hw,
-							(uint8_t *)udp + 8,
-							length);
-				}
-			}
-		}
-	}
-	return 0;
-}
-
 #define TXD_USE_COUNT(S, X) (((S) >> (X)) + 1 )
 static int
 e1000_xmit_frame(struct sk_buff *skb, struct net_device *netdev)
@@ -2852,7 +2814,6 @@ e1000_xmit_frame(struct sk_buff *skb, st
 	unsigned int nr_frags = 0;
 	unsigned int mss = 0;
 	int count = 0;
- 	int tso;
 	unsigned int f;
 	len -= skb->data_len;
 
@@ -2876,44 +2837,18 @@ e1000_xmit_frame(struct sk_buff *skb, st
 	 * overrun the FIFO, adjust the max buffer len if mss
 	 * drops. */
 	if (mss) {
-		uint8_t hdr_len;
 		max_per_txd = min(mss << 2, max_per_txd);
 		max_txd_pwr = fls(max_per_txd) - 1;
-
-	/* TSO Workaround for 82571/2 Controllers -- if skb->data
-	 * points to just header, pull a few bytes of payload from
-	 * frags into skb->data */
-		hdr_len = ((skb->h.raw - skb->data) + (skb->h.th->doff << 2));
-		if (skb->data_len && (hdr_len == (skb->len - skb->data_len)) &&
-			(adapter->hw.mac_type == e1000_82571 ||
-			adapter->hw.mac_type == e1000_82572)) {
-			unsigned int pull_size;
-			pull_size = min((unsigned int)4, skb->data_len);
-			if (!__pskb_pull_tail(skb, pull_size)) {
-				printk(KERN_ERR "__pskb_pull_tail failed.\n");
-				dev_kfree_skb_any(skb);
-				return NETDEV_TX_OK;
-			}
-			len = skb->len - skb->data_len;
-		}
 	}
 
 	/* reserve a descriptor for the offload context */
 	if ((mss) || (skb->ip_summed == CHECKSUM_HW))
 		count++;
-	count++;
+	count++;	/* for sentinel desc */
 #else
 	if (skb->ip_summed == CHECKSUM_HW)
 		count++;
 #endif
-
-#ifdef NETIF_F_TSO
-	/* Controller Erratum workaround */
-	if (!skb->data_len && tx_ring->last_tx_tso &&
-		!skb_shinfo(skb)->tso_size)
-		count++;
-#endif
-
 	count += TXD_USE_COUNT(len, max_txd_pwr);
 
 	if (adapter->pcix_82544)
@@ -2933,9 +2868,6 @@ e1000_xmit_frame(struct sk_buff *skb, st
 	if (adapter->pcix_82544)
 		count += nr_frags;
 
-	if (adapter->hw.tx_pkt_filtering && (adapter->hw.mac_type == e1000_82573) )
-		e1000_transfer_dhcp_info(adapter, skb);
-
 	local_irq_save(flags);
 	if (!spin_trylock(&tx_ring->tx_lock)) {
 		/* Collision - tell upper layer to requeue */
@@ -2967,25 +2899,11 @@ e1000_xmit_frame(struct sk_buff *skb, st
 
 	first = tx_ring->next_to_use;
 
-	tso = e1000_tso(adapter, tx_ring, skb);
-	if (tso < 0) {
-		dev_kfree_skb_any(skb);
-		spin_unlock_irqrestore(&tx_ring->tx_lock, flags);
-		return NETDEV_TX_OK;
-	}
-
-	if (likely(tso)) {
-		tx_ring->last_tx_tso = 1;
+	if(likely(e1000_tso(adapter, tx_ring, skb)))
 		tx_flags |= E1000_TX_FLAGS_TSO;
-	} else if (likely(e1000_tx_csum(adapter, tx_ring, skb)))
+	else if(likely(e1000_tx_csum(adapter, tx_ring, skb)))
 		tx_flags |= E1000_TX_FLAGS_CSUM;
 
-	/* Old method was to assume IPv4 packet by default if TSO was enabled.
-	 * 82571 hardware supports TSO capabilities for IPv6 as well...
-	 * no longer assume, we must. */
-	if (likely(skb->protocol == ntohs(ETH_P_IP)))
-		tx_flags |= E1000_TX_FLAGS_IPV4;
-
 	e1000_tx_queue(adapter, tx_ring, tx_flags,
 	               e1000_tx_map(adapter, tx_ring, skb, first,
 	                            max_per_txd, nr_frags, mss));


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Jesse Brandeburg.

On 06.04.2006 04:42 you said the following:

> I built and tested the driver with patches on 2.6.16, with pci-x adapters.  
> I removed some workarounds for PCIe adapters, but I dont think anyone 
> having this problem has a PCIe adapter anyway.  I saw no TX hangs and ran 
> some bi-directional tests, so i think the driver should work okay.  Just 
> warning you I did minimal testing.
> 
> *********************
> e1000: transmit the old fashioned way
> 
> It seems back in the day of 2.6.11, there were no sk_forward_alloc
> asserions.  Forward port that transmit code to see if it fixes the issues
> in today's kernel.  Unfortunately it doesn't have all the bug fixes that
> the current code has, but if we get transmit timeouts we can add in
> workarounds appropriately.
> 
> this changes only the e1000_tso function

With this one still having:

TCP: Treason uncloaked! Peer 80.72.16.78:11460/80 shrinks window 
2223569515:2223569516. Repaired.
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (283)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (150)



> e1000: implement old xmit_frame
> 
> It seems back in the day of 2.6.11, there were no sk_forward_alloc
> asserions.  Forward port that transmit code to see if it fixes the issues
> in today's kernel.  Unfortunately it doesn't have all the bug fixes that
> the current code has, but if we get transmit timeouts we can add in
> workarounds appropriately.
> 
> this changes the e1000_xmit_frame function, and some ancilliaries
> 
> Signed-off-by: Jesse Brandeburg <jesse.brandeburg@intel.com>



Can't apply this one:

[zhmurov@builds linux-2.6.16]$ patch -p1 < 
../../../SOURCES/linux-2.6.16-e1000-implement_old_xmit_frame.patch
patching file drivers/net/e1000/e1000_main.c
Hunk #1 succeeded at 2620 (offset -105 lines).
Hunk #2 FAILED at 2695.
Hunk #4 FAILED at 2837.
Hunk #5 FAILED at 2868.
Hunk #6 FAILED at 2899.
4 out of 6 hunks FAILED -- saving rejects to file 
drivers/net/e1000/e1000_main.c.rej



-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Jesse Brandeburg]

Boris B. Zhmurov wrote:
> Hello, Jesse Brandeburg.
>
> On 06.04.2006 04:42 you said the following:
>
>> I built and tested the driver with patches on 2.6.16, with pci-x 
>> adapters.  I removed some workarounds for PCIe adapters, but I dont 
>> think anyone having this problem has a PCIe adapter anyway.  I saw no 
>> TX hangs and ran some bi-directional tests, so i think the driver 
>> should work okay.  Just warning you I did minimal testing.
>>
>> *********************
>> e1000: transmit the old fashioned way
>>
>> It seems back in the day of 2.6.11, there were no sk_forward_alloc
>> asserions.  Forward port that transmit code to see if it fixes the 
>> issues
>> in today's kernel.  Unfortunately it doesn't have all the bug fixes that
>> the current code has, but if we get transmit timeouts we can add in
>> workarounds appropriately.
>>
>> this changes only the e1000_tso function
>
> With this one still having:
>
> TCP: Treason uncloaked! Peer 80.72.16.78:11460/80 shrinks window 
> 2223569515:2223569516. Repaired.
> KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c 
> (283)
> KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c 
> (150)
This is a very important result.  It shows that the changes to the 
driver to call pskb_expand_head for TSO operations are not the cause of 
this problem.

We also have some new data from the last couple of days.  First, I think 
that this problem is likely not just E1000's fault.  We have multiple 
reports both in bugzilla.kernel.org and from a distro that show this 
problem has occurred on (at least) tg3 driven adapters as well as e1000.

I've been able to reliably reproduce this issue in house (finally) 
thanks to one of our testers.  The test is using the tbench application 
from the dbench package at samba.org.

on the server, start tbench_srv
on the machine you're trying to repro the issue on, start tbench 500 
<server ip>, on another client start tbench 50 <server ip>
I've seen sk_forward_alloc assertions on both server and client both 
running 2.6.16.  We're trying to figure out where there might be a stale 
pointer to an sk that accesses the data after free.  something seems to 
write ff ff ff ff 00 00 00 00 to memory after it is freed maybe?

It does seem that the load (the 500 threads) is important to this 
failure.  I've also seen a report that a memory poisoning kernel caught 
the failure.

Any investigation hints for me?
>> e1000: implement old xmit_frame
>>
>> It seems back in the day of 2.6.11, there were no sk_forward_alloc
>> asserions.  Forward port that transmit code to see if it fixes the 
>> issues
>> in today's kernel.  Unfortunately it doesn't have all the bug fixes that
>> the current code has, but if we get transmit timeouts we can add in
>> workarounds appropriately.
>>
>> this changes the e1000_xmit_frame function, and some ancilliaries
>>
>> Signed-off-by: Jesse Brandeburg <jesse.brandeburg@intel.com>
>
>
>
> Can't apply this one:
>
> [zhmurov@builds linux-2.6.16]$ patch -p1 < 
> ../../../SOURCES/linux-2.6.16-e1000-implement_old_xmit_frame.patch
> patching file drivers/net/e1000/e1000_main.c
> Hunk #1 succeeded at 2620 (offset -105 lines).
> Hunk #2 FAILED at 2695.
> Hunk #4 FAILED at 2837.
> Hunk #5 FAILED at 2868.
> Hunk #6 FAILED at 2899.
> 4 out of 6 hunks FAILED -- saving rejects to file 
> drivers/net/e1000/e1000_main.c.rej

well that seems kind of lame, but I think we got the data that we needed 
from the first patch.


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[David S. Miller]

From: Jesse Brandeburg <jesse.brandeburg@intel.com>
Date: Fri, 14 Apr 2006 13:28:10 -0700

> We also have some new data from the last couple of days.  First, I think 
> that this problem is likely not just E1000's fault.  We have multiple 
> reports both in bugzilla.kernel.org and from a distro that show this 
> problem has occurred on (at least) tg3 driven adapters as well as e1000.

That's interesting since tg3 does not enable TSO by default for any
chip until very recent versions of the tg3 driver.  And even with
those recent tg3 drivers which will enable TSO by default, it is only
done for a very specific selection of chip revisions.

Where are these reports that precisely implicate tg3?

Note that there were legitimate TSO retransmit bugs in the 2.6.14
timeframe that triggered those messages and did get fixed.  And yes
some of those reports were with tg3.

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Jesse Brandeburg]

On Fri, 14 Apr 2006, David S. Miller wrote:

> From: Jesse Brandeburg <jesse.brandeburg@intel.com>
> Date: Fri, 14 Apr 2006 13:28:10 -0700
> 
> > We also have some new data from the last couple of days.  First, I think 
> > that this problem is likely not just E1000's fault.  We have multiple 
> > reports both in bugzilla.kernel.org and from a distro that show this 
> > problem has occurred on (at least) tg3 driven adapters as well as e1000.
> 
> That's interesting since tg3 does not enable TSO by default for any
> chip until very recent versions of the tg3 driver.  And even with
> those recent tg3 drivers which will enable TSO by default, it is only
> done for a very specific selection of chip revisions.
> 
> Where are these reports that precisely implicate tg3?

well there was one of them here, but the tg3 bit may actually be due to 
the 2.6.14 problems.

http://bugzilla.kernel.org/show_bug.cgi?id=6279

the other one is buried at a distro's bugzilla, so I can't post it.

> Note that there were legitimate TSO retransmit bugs in the 2.6.14
> timeframe that triggered those messages and did get fixed.  And yes
> some of those reports were with tg3.

I've removed the call to pskb_expand_head in e1000 (yes i know its not 
quite right, but it shouldn't be fatal either) and the bug still occurs.

so its something else.  I'm not giving up on this, I just want to post the 
state of the investigation.

Jesse

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[David S. Miller]

From: Jesse Brandeburg <jesse.brandeburg@intel.com>
Date: Fri, 14 Apr 2006 15:32:55 -0700 (Pacific Daylight Time)

> well there was one of them here, but the tg3 bit may actually be due to 
> the 2.6.14 problems.
> 
> http://bugzilla.kernel.org/show_bug.cgi?id=6279

There are 2 e1000 gigabit devices in that person's system, and
not one tg3 device.

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Jesse Brandeburg]

On Fri, 14 Apr 2006, David S. Miller wrote:

> From: Jesse Brandeburg <jesse.brandeburg@intel.com>
> Date: Fri, 14 Apr 2006 15:32:55 -0700 (Pacific Daylight Time)
> 
> > well there was one of them here, but the tg3 bit may actually be due to 
> > the 2.6.14 problems.
> > 
> > http://bugzilla.kernel.org/show_bug.cgi?id=6279
> 
> There are 2 e1000 gigabit devices in that person's system, and
> not one tg3 device.

sure, thats fine, but we just reproduced it in two seperate systems 
without the e1000 driver loaded, using the instructions as mentioned in a 
previous email.  We used a 5704 with TSO enabled.

I'm willing to try any debug patches you might come up with.

Jesse

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[David S. Miller]

From: Jesse Brandeburg <jesse.brandeburg@intel.com>
Date: Fri, 14 Apr 2006 15:46:31 -0700 (Pacific Daylight Time)

> sure, thats fine, but we just reproduced it in two seperate systems 
> without the e1000 driver loaded, using the instructions as mentioned in a 
> previous email.  We used a 5704 with TSO enabled.

That I didn't notice, thanks for the datapoint.

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Jesse Brandeburg]

On Fri, 14 Apr 2006, David S. Miller wrote:

> From: Jesse Brandeburg <jesse.brandeburg@intel.com>
> Date: Fri, 14 Apr 2006 15:46:31 -0700 (Pacific Daylight Time)
> 
> > sure, thats fine, but we just reproduced it in two seperate systems 
> > without the e1000 driver loaded, using the instructions as mentioned in a 
> > previous email.  We used a 5704 with TSO enabled.
> 
> That I didn't notice, thanks for the datapoint.

you're welcome, its "hot off the presses" (we just reproduced it 5 minutes 
ago) 

I'm trying to isolate more of a reproduction case, I'll be sure to post if 
I can find anything with more detail.

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[David S. Miller]

From: Jesse Brandeburg <jesse.brandeburg@intel.com>
Date: Fri, 14 Apr 2006 15:55:10 -0700 (Pacific Daylight Time)

> I'm trying to isolate more of a reproduction case, I'll be sure to
> post if I can find anything with more detail.

I think I see the bug.

If tbench with large numbers of clients is part of what helps
reproduce it, the key might be hitting the memory limits in tcp_mem[]
and friends, or something to do with concurrent access to
sk->sk_forward_alloc.

I bet there is some race in there.

A lot of the action is in net/core/stream.c  We modify
sk->sk_forward_alloc non-atomically but that should be ok
since we ought to be holding all of the correct locks when
we hit these accesses.  But it is the first thing to audit.

Let's look at sk_stream_rfree() as that is invoked from SKB
freeing callbacks and is the most likely suspect for these
kinds of problems.

It is hooked up to the skb->destructor by sk_stream_set_owner_r() and
then invoked via __kfree_skb().

Nothing here takes any locks, and as stated above we modify
sk->sk_forward_alloc non-atomically, and this is therefore the bug.

Shit.

I'll think of how to fix this in the least invasive manner.  I also
want to search the changelog history to see if this race was always
present or if it was "introduced".

Making sk->sk_forward_alloc an atomic_t would be incredibly expensive
so I'll try to find a way to avoid that.  We may be able to just do
a bh_lock_sock()/bh_unlock_sock() around the body of sk_stream_rfree()
to fix this.

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Boris B. Zhmurov.

On 31.03.2006 16:23 you said the following:

> Hello, Mark Nipper.
> 
> On 31.03.2006 16:10 you said the following:
> 
>>         This unfortunately is not the case.  I have two e1000
>> interfaces but only eth1 is up and in use.  And I still had
>> assertions.
> 
> 
> 
> Can you switch to eth0? There is no problem with _eth0_, my friend says.

P.S. I have another high-load server as gateway. Same distro, same 
kernels, but less memory (512Mb lowmem). eth0 up - e100, eth1 up - 
e1000. No errors at all! It kinda looks like assertions happens on 
systems, where the _only_ interface _eth1_ e1000 is up.

-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Christiaan den Besten]

Hi !

> P.S. I have another high-load server as gateway. Same distro, same kernels, but less memory (512Mb lowmem). eth0 up - e100, eth1 
> up - e1000. No errors at all! It kinda looks like assertions happens on systems, where the _only_ interface _eth1_ e1000 is up.

No, we have a couple gateway's asserting.

2x : Usenet feeder : Onboard eth0 and eth1 "Ethernet controller: Intel Corporation 82546GB Gigabit Ethernet Controller (rev 03)" -> 
asserts (lot's of disk activity (writes) as well by the way ... ). SMP, 4Gb RAM. (2.6.14-mm2)

4x : Usenet cache : PCI-X eth0 "Ethernet controller: Intel Corporation 82545GM Gigabit Ethernet Controller (rev 04)" -> no asserts 
(no disk activity). Has 2 extra onboard e1000's, but are not used (Ethernet controller: Intel Corporation 82541GI/PI Gigabit 
Ethernet Controller). SMP, 2Gb RAM (2.6.15.1)

bye,
Chris



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Christiaan den Besten.

On 31.03.2006 17:12 you said the following:

> Hi !
> 
>> P.S. I have another high-load server as gateway. Same distro, same 
>> kernels, but less memory (512Mb lowmem). eth0 up - e100, eth1 up - 
>> e1000. No errors at all! It kinda looks like assertions happens on 
>> systems, where the _only_ interface _eth1_ e1000 is up.
> 
> 
> No, we have a couple gateway's asserting.


Yes, my mistake :( My server asserting with eth0 and eth1 is up both...


Herbert, with your second patch still no luck. After an hour of uptime I 
have assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (283) 
again...

Trying your debug patch.


-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Boris B. Zhmurov.

On 31.03.2006 17:30 you said the following:

> Herbert, with your second patch still no luck. After an hour of uptime I 
> have assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (283) 
> again...
> 
> Trying your debug patch.


Hmm... with lastest debug patch I can't see any of debug info:

e1000: eth0: e1000_watchdog_task: NIC Link is Up 100 Mbps Full Duplex
e1000: eth1: e1000_watchdog_task: NIC Link is Up 100 Mbps Full Duplex
e1000: eth1: e1000_watchdog_task: NIC Link is Up 100 Mbps Full Duplex
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (283)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (150)

Is it normal?

-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Boris B. Zhmurov.

On 31.03.2006 19:08 you said the following:

> Hmm... with lastest debug patch I can't see any of debug info:

But wait a minute. Two days ago, without Herbert's patches, assertion's 
errors was like this:

Mar 29 20:03:23 msk4 kernel: KERNEL: assertion (!sk->sk_forward_alloc) 
failed at net/core/stream.c (279)
Mar 29 20:03:23 msk4 kernel: KERNEL: assertion (!sk->sk_forward_alloc) 
failed at net/ipv4/af_inet.c (148)


and after appling patches, errors looks like this:

Mar 31 18:21:06 msk4 kernel: KERNEL: assertion (!sk->sk_forward_alloc) 
failed at net/core/stream.c (283)
Mar 31 18:21:06 msk4 kernel: KERNEL: assertion (!sk->sk_forward_alloc) 
failed at net/ipv4/af_inet.c (150)


stream.c (279) -> stream.c (283)
af_inet.c (148) -> af_inet.c (150)


Does it really matters?

-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Mark Nipper]

On 31 Mar 2006, Boris B. Zhmurov wrote:
> stream.c (279) -> stream.c (283)
> af_inet.c (148) -> af_inet.c (150)

        That will be because the patches changed the line numbers
in the source I believe.  Nothing helpful unfortunately.

-- 
Mark Nipper                                                e-contacts:
832 Tanglewood Drive                                nipsy@bitgnome.net
Bryan, Texas 77802-4013                     http://nipsy.bitgnome.net/
(979)575-3193                      AIM/Yahoo: texasnipsy ICQ: 66971617

-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GG/IT d- s++:+ a- C++$ UBL++++$ P--->+++ L+++$ !E---
W++(--) N+ o K++ w(---) O++ M V(--) PS+++(+) PE(--)
Y+ PGP t+ 5 X R tv b+++@ DI+(++) D+ G e h r++ y+(**)
------END GEEK CODE BLOCK------

---begin random quote of the moment---
"Whiskey-Tango-Foxtrot, over."
 -- anonymous
----end random quote of the moment----


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

Hello, Mark Nipper.

On 31.03.2006 20:01 you said the following:

> On 31 Mar 2006, Boris B. Zhmurov wrote:
> 
>>stream.c (279) -> stream.c (283)
>>af_inet.c (148) -> af_inet.c (150)
> 
> 
>         That will be because the patches changed the line numbers
> in the source I believe.  Nothing helpful unfortunately.
> 


Ok. Anyway, as assertion is 100% repeatable on my server, I'm ready to 
try any patches to get rid of this.

-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[JaniD++]

----- Original Message ----- 
From: "Boris B. Zhmurov" <bb@kernelpanic.ru>
To: "Herbert Xu" <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>; <jesse.brandeburg@intel.com>;
<nipsy@bitgnome.net>; <jrlundgren@gmail.com>; <cat@zip.com.au>;
<djani22@dynamicweb.hu>; <yoseph.basri@gmail.com>; <mykleb@no.ibm.com>;
<olel@ans.pl>; <michal@feix.cz>; <chris@scorpion.nl>;
<netdev@vger.kernel.org>; <jesse.brandeburg@gmail.com>;
<E1000-devel@lists.sourceforge.net>; "Andi Kleen" <ak@suse.de>; "Jeff
Garzik" <jgarzik@pobox.com>
Sent: Friday, March 31, 2006 2:07 PM
Subject: Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...


> Hello, Herbert Xu.
>
> On 31.03.2006 14:39 you said the following:
>
> > On Fri, Mar 31, 2006 at 02:16:38PM +0400, Boris B. Zhmurov wrote:
> >
> >>And xdelta tells, that e1000.ko was modified :)
> >
> >
> > Thanks for checking again.
> >
> > Anyway, it didn't take long to find another bug in the same area.
> > I'm afraid this driver does seem to be full of them :)
> >
> > It sets last_tx_tso in between computing the number of descriptors and
> > calling e1000_tx_map.  This is bad because e1000_tx_map gets the wrong
> > value for last_tx_tso and therefore may corrupt memory for every TSO
> > packet when the ring is almost full.
> >
> > This bug exists on UP as well as SMP.
> >
> > Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
> >
> > Please try this in conjunction with the previous patch.
> >
> > Cheers,
>
>
> David, Herbert - FYI. One of my colleague confirmed, that idea "bug
> reproducible only if there is more then one e1000 adapter onboard" is
> true. He has a 3 servers with double intel pro 1000 adapters, and that
> bug occurs. Also, he has 4 servers with double intel pro 1000 adapters
> onboard, but _only one_ of them is up. And there is no such messages in
> dmesg at all! Inetresting...

This is not an unique thing!

Only _one_ of my 2 equal NIC get this message

NETDEV WATCHDOG: eth0: transmit timed out
e1000: eth0: e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex

with the old 2.6.15.* e1000 driver!

Not the all e1000 chips ar really equal with the same P/N Number!

This can be hardware based problem, and needs workaround?

Cheers,



>
> -- 
> Boris B. Zhmurov
> mailto: bb@kernelpanic.ru
> "wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"
>
>  _____________ NOD32 1.584 (20031220) Információ _____________
>
> Az üzenetet a NOD32 Antivirus System megvizsgálta.
> http://www.nod32.hu
>
>



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[David S. Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Thu, 30 Mar 2006 20:52:45 +1100

> Well I started from the beginning again, and found this.  This may be
> the smoking gun that we're after :)
> 
> The xmit routine is lockless but checks last_tx_tso outside the locked
> section.  So if a TSO packet wins a race against a non-TSO packet with
> last_tx_tso == 0 then we'll have memory corruption.

Regardless of whether this fixes the bug being discussed, this
fix should go into the e1000 driver ASAP.

Good spotting Herbert.


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Christiaan den Besten]

Hi !

Yes, we still have these errors ... but then, we have not changed the running kernel version for some time now ;)

---
Linux version 2.6.14-rc2-mm2 (root@localhost) (gcc version 3.4.4 20050721 (Red Hat 3.4.4-2)) #2 SMP Thu Dec 15 19:06:21 CET 2005
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009b800 (usable)
 BIOS-e820: 000000000009b800 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e4000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000cff70000 (usable)
 BIOS-e820: 00000000cff70000 - 00000000cff78000 (ACPI data)
 BIOS-e820: 00000000cff78000 - 00000000cff80000 (ACPI NVS)
 BIOS-e820: 00000000cff80000 - 00000000d0000000 (reserved)
 BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
 BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000ff800000 - 00000000ffc00000 (reserved)
 BIOS-e820: 00000000fffffc00 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000130000000 (usable)
3968MB HIGHMEM available.
896MB LOWMEM available.
found SMP MP-table at 000f6820
NX (Execute Disable) protection: active
On node 0 totalpages: 1245184
  DMA zone: 4096 pages, LIFO batch:2
  DMA32 zone: 0 pages, LIFO batch:2
  Normal zone: 225280 pages, LIFO batch:64
  HighMem zone: 1015808 pages, LIFO batch:64
DMI present.
ACPI: RSDP (v000 PTLTD                                 ) @ 0x000f6880
ACPI: RSDT (v001 PTLTD    RSDT   0x06040000  LTP 0x00000000) @ 0xcff72d76
ACPI: FADT (v001 INTEL  LINDHRST 0x06040000 PTL  0x00000003) @ 0xcff77e20
ACPI: MADT (v001 PTLTD           APIC   0x06040000  LTP 0x00000000) @ 0xcff77e94
ACPI: BOOT (v001 PTLTD  $SBFTBL$ 0x06040000  LTP 0x00000001) @ 0xcff77f48
ACPI: SPCR (v001 PTLTD  $UCRTBL$ 0x06040000 PTL  0x00000001) @ 0xcff77f70
ACPI: MCFG (v001 PTLTD           MCFG   0x06040000  LTP 0x00000000) @ 0xcff77fc0
ACPI: SSDT (v001  PmRef    CpuPm 0x00003000 INTL 0x20030224) @ 0xcff72db2
ACPI: DSDT (v001  Intel LINDHRST 0x06040000 MSFT 0x0100000e) @ 0x00000000
ACPI: PM-Timer IO Port: 0x1008
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 15:4 APIC version 20
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x06] enabled)
Processor #6 15:4 APIC version 20
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Processor #1 15:4 APIC version 20
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x07] enabled)
Processor #7 15:4 APIC version 20
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x03] address[0xfec80000] gsi_base[24])
IOAPIC[1]: apic_id 3, version 32, address 0xfec80000, GSI 24-47
ACPI: IOAPIC (id[0x04] address[0xfec80400] gsi_base[48])
IOAPIC[2]: apic_id 4, version 32, address 0xfec80400, GSI 48-71
ACPI: IOAPIC (id[0x05] address[0xfec84000] gsi_base[72])
IOAPIC[3]: apic_id 5, version 32, address 0xfec84000, GSI 72-95
ACPI: IOAPIC (id[0x08] address[0xfec84400] gsi_base[96])
IOAPIC[4]: apic_id 8, version 32, address 0xfec84400, GSI 96-119
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Enabling APIC mode:  Flat.  Using 5 I/O APICs
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at d1000000 (gap: d0000000:10000000)
Built 1 zonelists
mapped APIC to ffffd000 (fee00000)
mapped IOAPIC to ffffc000 (fec00000)
mapped IOAPIC to ffffb000 (fec80000)
mapped IOAPIC to ffffa000 (fec80400)
mapped IOAPIC to ffff9000 (fec84000)
mapped IOAPIC to ffff8000 (fec84400)
Initializing CPU#0
Kernel command line: ro root=LABEL=/
PID hash table entries: 4096 (order: 12, 65536 bytes)
Detected 2801.358 MHz processor.
Using pmtmr for high-res timesource
Console: colour VGA+ 80x25
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Memory: 4146392k/4980736k available (2757k kernel code, 45216k reserved, 783k data, 216k init, 3276224k highmem)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
Calibrating delay using timer specific routine.. 5603.74 BogoMIPS (lpj=2801871)
Mount-cache hash table entries: 512
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
monitor/mwait feature present.
using mwait in idle threads.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 1024K
CPU: Physical Processor ID: 0
CPU: After all inits, caps: bfebfbff 20100000 00000000 00000080 0000641d 00000000 00000000
mtrr: v2.0 (20020519)
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
CPU0: Intel(R) Xeon(TM) CPU 2.80GHz stepping 01
Booting processor 1/1 eip 2000
Initializing CPU#1
Calibrating delay using timer specific routine.. 5599.54 BogoMIPS (lpj=2799774)
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
monitor/mwait feature present.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 1024K
CPU: Physical Processor ID: 0
CPU: After all inits, caps: bfebfbff 20100000 00000000 00000080 0000641d 00000000 00000000
CPU1: Intel(R) Xeon(TM) CPU 2.80GHz stepping 01
Booting processor 2/6 eip 2000
Initializing CPU#2
Calibrating delay using timer specific routine.. 5571.21 BogoMIPS (lpj=2785607)
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
monitor/mwait feature present.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 1024K
CPU: Physical Processor ID: 3
CPU: After all inits, caps: bfebfbff 20100000 00000000 00000080 0000641d 00000000 00000000
CPU2: Intel(R) Xeon(TM) CPU 2.80GHz stepping 01
Booting processor 3/7 eip 2000
Initializing CPU#3
Calibrating delay using timer specific routine.. 5599.55 BogoMIPS (lpj=2799778)
CPU: After generic identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 20100000 00000000 00000000 0000641d 00000000 00000000
monitor/mwait feature present.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 1024K
CPU: Physical Processor ID: 3
CPU: After all inits, caps: bfebfbff 20100000 00000000 00000080 0000641d 00000000 00000000
CPU3: Intel(R) Xeon(TM) CPU 2.80GHz stepping 01
Total of 4 processors activated (22374.06 BogoMIPS).
ENABLING IO-APIC IRQs
..TIMER: vector=0x31 pin1=2 pin2=-1
checking TSC synchronization across 4 CPUs: passed.
Brought up 4 CPUs
checking if image is initramfs... it is
Freeing initrd memory: 296k freed
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xfd830, last bus=9
PCI: Using MMCONFIG
usbcore: registered new driver usbfs
usbcore: registered new driver hub
ACPI: Subsystem revision 20050916
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI: Probing PCI hardware (bus 00)
PCI: Ignoring BAR0-3 of IDE controller 0000:00:1f.1
PCI: PXH quirk detected, disabling MSI for SHPC device
PCI: PXH quirk detected, disabling MSI for SHPC device
PCI: PXH quirk detected, disabling MSI for SHPC device
PCI: PXH quirk detected, disabling MSI for SHPC device
Boot video device is 0000:09:01.0
PCI: Transparent bridge - 0000:00:1e.0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX0.PXH0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX0.PXH1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEY0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEZ0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEZ0.PXH0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEZ0.PXH1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PCIB._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 *5 6 7 10 11 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 *10 11 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 *7 10 11 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 10 *11 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 4 5 6 7 10 11 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 11 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKH] (IRQs 4 5 6 7 *10 11 14 15)
ACPI: Device [PRT] status [0000000c]: functional but not present; setting present
SCSI subsystem initialized
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
PCI: Bridge: 0000:01:00.0
  IO window: disabled.
  MEM window: disabled.
  PREFETCH window: disabled.
PCI: Bridge: 0000:01:00.2
  IO window: 2000-2fff
  MEM window: dd200000-dd2fffff
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:02.0
  IO window: 2000-2fff
  MEM window: dd100000-dd2fffff
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:04.0
  IO window: disabled.
  MEM window: disabled.
  PREFETCH window: disabled.
PCI: Bridge: 0000:06:01.0
  IO window: disabled.
  MEM window: dd400000-dd4fffff
  PREFETCH window: d1000000-d10fffff
PCI: Bridge: 0000:05:00.0
  IO window: disabled.
  MEM window: dd400000-dd4fffff
  PREFETCH window: d1000000-d10fffff
PCI: Bridge: 0000:05:00.2
  IO window: disabled.
  MEM window: disabled.
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:06.0
  IO window: disabled.
  MEM window: dd300000-dd4fffff
  PREFETCH window: d1000000-d10fffff
PCI: Bridge: 0000:00:1e.0
  IO window: 3000-3fff
  MEM window: dd500000-deffffff
  PREFETCH window: d1100000-d11fffff
ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 16 (level, low) -> IRQ 16
PCI: Setting latency timer of device 0000:00:02.0 to 64
PCI: Setting latency timer of device 0000:01:00.0 to 64
PCI: Setting latency timer of device 0000:01:00.2 to 64
ACPI: PCI Interrupt 0000:00:04.0[A] -> GSI 16 (level, low) -> IRQ 16
PCI: Setting latency timer of device 0000:00:04.0 to 64
ACPI: PCI Interrupt 0000:00:06.0[A] -> GSI 16 (level, low) -> IRQ 16
PCI: Setting latency timer of device 0000:00:06.0 to 64
PCI: Setting latency timer of device 0000:05:00.0 to 64
PCI: Setting latency timer of device 0000:05:00.2 to 64
PCI: Setting latency timer of device 0000:00:1e.0 to 64
Simple Boot Flag at 0x39 set to 0x1
highmem bounce pool size: 64 pages
Total HugeTLB memory allocated, 0
SGI XFS with large block numbers, no debug enabled
Initializing Cryptographic API
ACPI: PCI Interrupt 0000:00:1d.7[D] -> GSI 23 (level, low) -> IRQ 17
PCI: Setting latency timer of device 0000:00:1d.7 to 64
ehci_hcd 0000:00:1d.7: EHCI Host Controller
ehci_hcd 0000:00:1d.7: debug port 1
ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:1d.7: irq 17, io mem 0xdd001000
PCI: cache line size of 128 is not supported by device 0000:00:1d.7
ehci_hcd 0000:00:1d.7: USB 2.0 initialized, EHCI 1.00, driver 10 Dec 2004
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 8 ports detected
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
usbcore: registered new driver usbserial
drivers/usb/serial/usb-serial.c: USB Serial support registered for Generic
usbcore: registered new driver usbserial_generic
drivers/usb/serial/usb-serial.c: USB Serial Driver core v2.0
ACPI: Power Button (FF) [PWRF]
ACPI: Power Button (CM) [PWRB]
ACPI: CPU0 (power states: C1[C1])
ACPI: CPU2 (power states: C1[C1])
ACPI: CPU1 (power states: C1[C1])
ACPI: CPU3 (power states: C1[C1])
Real Time Clock Driver v1.12
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
mice: PS/2 mouse device common for all mice
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered
Floppy drive(s): fd0 is 1.44M
FDC 0 is a National Semiconductor PC87306
RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
loop: loaded (max 8 devices)
Intel(R) PRO/1000 Network Driver - version 6.0.60-k2
Copyright (c) 1999-2005 Intel Corporation.
ACPI: PCI Interrupt 0000:03:02.0[A] -> GSI 54 (level, low) -> IRQ 18
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
ACPI: PCI Interrupt 0000:03:02.1[B] -> GSI 55 (level, low) -> IRQ 19
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
ICH5: IDE controller at PCI slot 0000:00:1f.1
ACPI: PCI Interrupt 0000:00:1f.1[A] -> GSI 18 (level, low) -> IRQ 20
ICH5: chipset revision 2
ICH5: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0x14a0-0x14a7, BIOS settings: hda:DMA, hdb:pio
    ide1: BM-DMA at 0x14a8-0x14af, BIOS settings: hdc:pio, hdd:pio
Probing IDE interface ide0...
hda: SR244W, ATAPI CD/DVD-ROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Probing IDE interface ide1...
Probing IDE interface ide1...
hda: ATAPI 24X CD-ROM drive, 128kB Cache, UDMA(33)
Uniform CD-ROM driver Revision: 3.20
ACPI: PCI Interrupt 0000:07:0e.0[A] -> GSI 74 (level, low) -> IRQ 21
ARECA RAID ADAPTER0: 64BITS PCI BUS DMA ADDRESSING SUPPORTED
ARECA RAID ADAPTER0: FIRMWARE VERSION V1.38 2005-10-4
scsi0 : ARECA ARC1160 PCI-X 16 PORTS SATA RAID CONTROLLER (RAID6-ENGINE Inside)
        Driver Version 1.20.00.12
  Vendor: Areca     Model: ARC-1160-VOL#00   Rev: R001
  Type:   Direct-Access                      ANSI SCSI revision: 03
  Vendor: Areca     Model: DATA1             Rev: R001
  Type:   Direct-Access                      ANSI SCSI revision: 03
  Vendor: Areca     Model: DATA2             Rev: R001
  Type:   Direct-Access                      ANSI SCSI revision: 03
arcmsr device major number 254
libata version 1.12 loaded.
SCSI device sda: 312499712 512-byte hdwr sectors (160000 MB)
SCSI device sda: drive cache: write back
SCSI device sda: 312499712 512-byte hdwr sectors (160000 MB)
SCSI device sda: drive cache: write back
 sda: sda1 sda2 sda3 sda4 < sda5 sda6 >
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
SCSI device sdb: 820310016 512-byte hdwr sectors (419999 MB)
SCSI device sdb: drive cache: write back
SCSI device sdb: 820310016 512-byte hdwr sectors (419999 MB)
SCSI device sdb: drive cache: write back
 sdb: sdb1
Attached scsi disk sdb at scsi0, channel 0, id 1, lun 0
SCSI device sdc: 820310016 512-byte hdwr sectors (419999 MB)
SCSI device sdc: drive cache: write back
SCSI device sdc: 820310016 512-byte hdwr sectors (419999 MB)
SCSI device sdc: drive cache: write back
 sdc: sdc1
Attached scsi disk sdc at scsi0, channel 0, id 2, lun 0
md: md driver 0.90.2 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 3.39
device-mapper: 4.4.0-ioctl (2005-01-12) initialised: dm-devel@redhat.com
NET: Registered protocol family 2
IP route cache hash table entries: 262144 (order: 8, 1048576 bytes)
TCP established hash table entries: 524288 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
TCP reno registered
ip_conntrack version 2.3 (8192 buckets, 65536 max) - 172 bytes per conntrack
ip_tables: (C) 2000-2002 Netfilter core team
TCP bic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
NET: Registered protocol family 15
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
Starting balanced_irq
Using IPI Shortcut mode
Freeing unused kernel memory: 216k freed
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting.  Commit interval 5 seconds
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
EXT3 FS on sda1, internal journal
hda: packet command error: status=0x51 { DriveReady SeekComplete Error }
hda: packet command error: error=0x50 { LastFailedSense=0x05 }
ide: failed opcode was: unknown
cdrom: open failed.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda5, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda6, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
XFS mounting filesystem sdb1
Starting XFS recovery on filesystem: sdb1 (dev: sdb1)
Ending XFS recovery on filesystem: sdb1 (dev: sdb1)
XFS mounting filesystem sdc1
Starting XFS recovery on filesystem: sdc1 (dev: sdc1)
Ending XFS recovery on filesystem: sdc1 (dev: sdc1)
e1000: eth0: e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex
e1000: eth1: e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex
e1000: eth1: e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex
e1000: eth0: e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
---

iptables : YES

routing : NO (3 routes at present)

traffic : A lot ;) ... 24x7 300mbit on eth0 and 450 mbit on eth1.

bye,
Chris

----- Original Message ----- 
From: "Brandeburg, Jesse" <jesse.brandeburg@intel.com>
To: <nipsy@bitgnome.net>; <jrlundgren@gmail.com>; <cat@zip.com.au>; <djani22@dynamicweb.hu>; <yoseph.basri@gmail.com>; 
<bb@kernelpanic.ru>; <mykleb@no.ibm.com>; <olel@ans.pl>; <michal@feix.cz>; <chris@scorpion.nl>
Cc: <netdev@vger.kernel.org>; "Jesse Brandeburg" <jesse.brandeburg@gmail.com>; <davem@davemloft.net>; 
<E1000-devel@lists.sourceforge.net>; "Brandeburg, Jesse" <jesse.brandeburg@intel.com>
Sent: Thursday, March 30, 2006 4:53 AM
Subject: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...


Hi all, I've identified you as people who have at some point in the past
emailed one of the Linux lists with problems with e1000 and
sk_forward_alloc.  It seems to be fairly widespread, but only seems to
have appeared with recent kernel changes (after 2.6.12...)

What I need from you is a reproducible test, and some information.  I
have never been able to reproduce this, and I'm trying to isolate the
problem a bit.  What motherboards are you using?  What seems to cause
this problem?  Are you all using iptables?  Are you all routing? From
the reports I assume none of you are using an 82571/2/3 (pci express)

As far as I know e1000 has the same requirement as tg3 and some others
where we have to modify the header of the skb in the case of transmits
using TSO.  I don't see anywhere else that the driver modifies the skb.
Tomorrow I'll generate a patch to try a more paranoid copying of the
skb, I hope some of you can test.

To do this we have code like so in e1000_tso:
2529         if (skb_shinfo(skb)->tso_size) {
2530                 if (skb_header_cloned(skb)) {
2531                         err = pskb_expand_head(skb, 0, 0,
GFP_ATOMIC);
2532                         if (err)
2533                                 return err;
2534                 }
2535
2536                 hdr_len = ((skb->h.raw - skb->data) +
(skb->h.th->doff << 2));
2537                 mss = skb_shinfo(skb)->tso_size;
2538                 if (skb->protocol == ntohs(ETH_P_IP)) {
2539                         skb->nh.iph->tot_len = 0;
2540                         skb->nh.iph->check = 0;

Thanks for your assistance

Jesse





-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Mark Nipper]

On 29 Mar 2006, Brandeburg, Jesse wrote:
> What I need from you is a reproducible test, and some information.  I
> have never been able to reproduce this, and I'm trying to isolate the
> problem a bit.  What motherboards are you using?  What seems to cause
> this problem?  Are you all using iptables?  Are you all routing? From
> the reports I assume none of you are using an 82571/2/3 (pci express)

        Unfortunately, my problem machine is a remote, leased
server, so I'd have to ask my provider for information on the
motherboard.  I have no specific idea what causes the problem as
the assertions simply show up after the fact in my logcheck
output.  I am not using iptables or routing.  And I'm fairly
certain the e1000 chip is just an integrated PCI device on the
motherboard.

> As far as I know e1000 has the same requirement as tg3 and some others
> where we have to modify the header of the skb in the case of transmits
> using TSO.  I don't see anywhere else that the driver modifies the skb.
> Tomorrow I'll generate a patch to try a more paranoid copying of the
> skb, I hope some of you can test.

        I'll be happy to test any patches you may have to narrow
down the problem.  I was actually considering running tcpdump or
ethereal or some such to try to capture the event on the network
side, but this probably isn't a wise idea considering it's a
production server and I do not have hands-on access to it.  A
patch which simply increased the verbosity of the event
(including counters and registers maybe?) would be preferable to
trying to capture an arbitrary amount of network traffic simply
waiting for the next time the assertion is triggered.

        Sorry for the real lack of data on this end.  But as I
said, any patch to help debug this is welcome.

-- 
Mark Nipper                                                e-contacts:
832 Tanglewood Drive                                nipsy@bitgnome.net
Bryan, Texas 77802-4013                     http://nipsy.bitgnome.net/
(979)575-3193                      AIM/Yahoo: texasnipsy ICQ: 66971617

-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GG/IT d- s++:+ a- C++$ UBL++++$ P--->+++ L+++$ !E---
W++(--) N+ o K++ w(---) O++ M V(--) PS+++(+) PE(--)
Y+ PGP t+ 5 X R tv b+++@ DI+(++) D+ G e h r++ y+(**)
------END GEEK CODE BLOCK------

---begin random quote of the moment---
I lost interest in "blade servers" when I found they didn't throw
knives at people who weren't supposed to be in your machine room.
  -- Anthony de Boer
----end random quote of the moment----


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Krzysztof Oledzki]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 706 bytes --]



On Thu, 30 Mar 2006, Mark Nipper wrote:

> On 29 Mar 2006, Brandeburg, Jesse wrote:
>> What I need from you is a reproducible test, and some information.  I
>> have never been able to reproduce this, and I'm trying to isolate the
>> problem a bit.  What motherboards are you using?  What seems to cause
>> this problem?  Are you all using iptables?  Are you all routing? From
>> the reports I assume none of you are using an 82571/2/3 (pci express)
>
>        Unfortunately, my problem machine is a remote, leased
> server, so I'd have to ask my provider for information on the
> motherboard.

You can probably check this with the dmidecode tool.

Best regards,

 				Krzysztof Olędzki

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Phil Oester]

> On 29 Mar 2006, Brandeburg, Jesse wrote:
> What I need from you is a reproducible test, and some information.  I

>From all the reports which have come in thus far, it seems everyone
has > 1 e1000.  One person even reported that removing one of the two
nics solved the problem for him.  Does this help narrow down the search
area?

Phil


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Krzysztof Oledzki]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 522 bytes --]



On Thu, 30 Mar 2006, Phil Oester wrote:

>> On 29 Mar 2006, Brandeburg, Jesse wrote:
>> What I need from you is a reproducible test, and some information.  I
>
> From all the reports which have come in thus far, it seems everyone
> has > 1 e1000.  One person even reported that removing one of the two
> nics solved the problem for him.  Does this help narrow down the search
> area?

I have only one. Anyway, this massage happens _very_ occasionally in my
case.


Best regrads,

 				Krzysztof Olędzki

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Boris B. Zhmurov]

[-- Attachment #1: Type: text/plain, Size: 2898 bytes --]

Hello, Brandeburg, Jesse.

On 30.03.2006 06:53 you said the following:

> Hi all, I've identified you as people who have at some point in the past
> emailed one of the Linux lists with problems with e1000 and
> sk_forward_alloc.  It seems to be fairly widespread, but only seems to
> have appeared with recent kernel changes (after 2.6.12...)
> 
> What I need from you is a reproducible test, and some information.  I
> have never been able to reproduce this, and I'm trying to isolate the
> problem a bit.  What motherboards are you using?  What seems to cause
> this problem?  Are you all using iptables?  Are you all routing? From
> the reports I assume none of you are using an 82571/2/3 (pci express)
> 
> As far as I know e1000 has the same requirement as tg3 and some others
> where we have to modify the header of the skb in the case of transmits
> using TSO.  I don't see anywhere else that the driver modifies the skb.
> Tomorrow I'll generate a patch to try a more paranoid copying of the
> skb, I hope some of you can test.


Jesse, I'd like to try your patches to help get rid of this annoying 
problem. I want to say, that this problem 100% reproucible on my 
hard-loading webserver based on RHEL4 with kernels 2.6.9 (rhel4 
original) - 2.6.15.7 (i.e. all releases from 2.6.9 to 2.6.15.7 affected).

  I have an asus 1unit server with double P4@2.8Ghz processors with 
enabled HyperThreading and 3Gb RAM, but with 1Gb RAM I have the same 
problem, thus it's not a RAM issue. This is really high load server, 
serving about 1000-1500 http requests per second plus about 500-1000 ftp 
requests per second.

  dmesg, lspci -vv, iptables -nL and ip route show in attached files. 
Wating for your instructions.


P.S. I use two e1000 adapters at the same time with advanced routing 
like this:

[root@msk4 ~]# cat /etc/rc.local |grep ip
# This script will be executed *after* all the other init scripts.
/sbin/ip rule add from 83.102.130.174 table NEW
/sbin/ip route add default via 83.102.130.173 dev eth0 table NEW



And also I have some hardcored sysctl options like this:

net.core.somaxconn=1024
net.ipv4.tcp_timestamps=0
net.ipv4.tcp_max_tw_buckets=720000
net.core.rmem_default=215040
net.core.rmem_max=262144
net.core.wmem_default=215040
net.core.wmem_max=262144
net.core.optmem_max=81920
net.core.netdev_max_backlog=8192
net.ipv4.neigh.default.gc_thresh1=512
net.ipv4.neigh.default.gc_thresh2=2048
net.ipv4.neigh.default.gc_thresh3=4096
net.ipv4.neigh.default.unres_qlen=64
net.ipv4.neigh.default.proxy_qlen=256
net.ipv4.tcp_rmem = 4096 131072 262144
net.ipv4.tcp_wmem = 4096 131072 262144
net.ipv4.tcp_keepalive_time=1800
net.ipv4.tcp_sack=0
net.ipv4.tcp_fin_timeout=30
net.ipv4.tcp_window_scaling=0
net.ipv4.tcp_keepalive_probes=3
kernel.sem=250 32000 100 128



-- 
Boris B. Zhmurov
mailto: bb@kernelpanic.ru
"wget http://kernelpanic.ru/bb_public_key.pgp -O - | gpg --import"


[-- Attachment #2: dmesg --]
[-- Type: text/plain, Size: 19109 bytes --]

Linux version 2.6.15-7.0.bbel4smp (zhmurov@builds.kernelpanic.ru) (gcc version 3.4.5 20051201 (Red Hat 3.4.5-2)) #1 SMP Tue Mar 28 14:52:13 MSD 2006
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000bfffb000 (usable)
 BIOS-e820: 00000000bfffb000 - 00000000bffff000 (ACPI data)
 BIOS-e820: 00000000bffff000 - 00000000c0000000 (ACPI NVS)
 BIOS-e820: 00000000fec00000 - 00000000fec01000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000ffff0000 - 0000000100000000 (reserved)
2175MB HIGHMEM available.
896MB LOWMEM available.
found SMP MP-table at 000f0830
On node 0 totalpages: 786427
  DMA zone: 4096 pages, LIFO batch:0
  DMA32 zone: 0 pages, LIFO batch:0
  Normal zone: 225280 pages, LIFO batch:31
  HighMem zone: 557051 pages, LIFO batch:31
DMI 2.3 present.
Using APIC driver default
ACPI: RSDP (v000 ASUS                                  ) @ 0x000f55a0
ACPI: RSDT (v001 ASUS   PR-DLSR  0x42302e31 MSFT 0x31313031) @ 0xbfffb000
ACPI: FADT (v001 ASUS   PR-DLSR  0x42302e31 MSFT 0x31313031) @ 0xbfffb145
ACPI: BOOT (v001 ASUS   PR-DLSR  0x42302e31 MSFT 0x31313031) @ 0xbfffb034
ACPI: SPCR (v001 ASUS   PR-DLSR  0x42302e31 MSFT 0x31313031) @ 0xbfffb05c
ACPI: MADT (v001 ASUS   PR-DLSR  0x42302e31 MSFT 0x31313031) @ 0xbfffb0a9
ACPI: DSDT (v001   ASUS PR-DLSR  0x00001000 MSFT 0x0100000b) @ 0x00000000
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 15:2 APIC version 20
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x06] enabled)
Processor #6 15:2 APIC version 20
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
Processor #1 15:2 APIC version 20
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x07] enabled)
Processor #7 15:2 APIC version 20
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-15
ACPI: IOAPIC (id[0x03] address[0xfec01000] gsi_base[16])
IOAPIC[1]: apic_id 3, version 17, address 0xfec01000, GSI 16-31
ACPI: IOAPIC (id[0x04] address[0xfec02000] gsi_base[32])
IOAPIC[2]: apic_id 4, version 17, address 0xfec02000, GSI 32-47
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl edge)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Enabling APIC mode:  Flat.  Using 3 I/O APICs
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at c4000000 (gap: c0000000:3ec00000)
Built 1 zonelists
Kernel command line: ro root=LABEL=/
mapped APIC to ffffd000 (fee00000)
mapped IOAPIC to ffffc000 (fec00000)
mapped IOAPIC to ffffb000 (fec01000)
mapped IOAPIC to ffffa000 (fec02000)
Initializing CPU#0
CPU 0 irqstacks, hard=c0391000 soft=c0371000
PID hash table entries: 4096 (order: 12, 65536 bytes)
Detected 2790.958 MHz processor.
Using tsc for high-res timesource
Console: colour VGA+ 80x25
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Memory: 3114260k/3145708k available (1599k kernel code, 30168k reserved, 685k data, 188k init, 2228204k highmem)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
Calibrating delay using timer specific routine.. 5589.74 BogoMIPS (lpj=27948717)
Mount-cache hash table entries: 512
CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000
CPU: Trace cache: 12K uops, L1 D cache: 8K
CPU: L2 cache: 512K
CPU: Physical Processor ID: 0
CPU: After all inits, caps: bfebfbff 00000000 00000000 00000080 00004400 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU0: Intel P4/Xeon Extended MCE MSRs (12) available
CPU0: Thermal monitoring enabled
mtrr: v2.0 (20020519)
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
CPU0: Intel(R) Xeon(TM) CPU 2.80GHz stepping 05
Booting processor 1/1 eip 2000
CPU 1 irqstacks, hard=c0392000 soft=c0372000
Initializing CPU#1
Calibrating delay using timer specific routine.. 5581.58 BogoMIPS (lpj=27907945)
CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000
CPU: Trace cache: 12K uops, L1 D cache: 8K
CPU: L2 cache: 512K
CPU: Physical Processor ID: 0
CPU: After all inits, caps: bfebfbff 00000000 00000000 00000080 00004400 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#1.
CPU1: Intel P4/Xeon Extended MCE MSRs (12) available
CPU1: Thermal monitoring enabled
CPU1: Intel(R) Xeon(TM) CPU 2.80GHz stepping 05
Booting processor 2/6 eip 2000
CPU 2 irqstacks, hard=c0393000 soft=c0373000
Initializing CPU#2
Calibrating delay using timer specific routine.. 5581.69 BogoMIPS (lpj=27908471)
CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000
CPU: Trace cache: 12K uops, L1 D cache: 8K
CPU: L2 cache: 512K
CPU: Physical Processor ID: 3
CPU: After all inits, caps: bfebfbff 00000000 00000000 00000080 00004400 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#2.
CPU2: Intel P4/Xeon Extended MCE MSRs (12) available
CPU2: Thermal monitoring enabled
CPU2: Intel(R) Xeon(TM) CPU 2.80GHz stepping 05
Booting processor 3/7 eip 2000
CPU 3 irqstacks, hard=c0394000 soft=c0374000
Initializing CPU#3
Calibrating delay using timer specific routine.. 5581.78 BogoMIPS (lpj=27908908)
CPU: After generic identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000
CPU: After vendor identify, caps: bfebfbff 00000000 00000000 00000000 00004400 00000000 00000000
CPU: Trace cache: 12K uops, L1 D cache: 8K
CPU: L2 cache: 512K
CPU: Physical Processor ID: 3
CPU: After all inits, caps: bfebfbff 00000000 00000000 00000080 00004400 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#3.
CPU3: Intel P4/Xeon Extended MCE MSRs (12) available
CPU3: Thermal monitoring enabled
CPU3: Intel(R) Xeon(TM) CPU 2.80GHz stepping 05
Total of 4 processors activated (22334.80 BogoMIPS).
ENABLING IO-APIC IRQs
..TIMER: vector=0x31 apic1=0 pin1=2 apic2=-1 pin2=-1
..MP-BIOS bug: 8254 timer not connected to IO-APIC
...trying to set up timer (IRQ0) through the 8259A ...  failed.
...trying to set up timer as Virtual Wire IRQ... works.
checking TSC synchronization across 4 CPUs: passed.
Brought up 4 CPUs
checking if image is initramfs... it is
Freeing initrd memory: 656k freed
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xf18e0, last bus=2
PCI: Using configuration type 1
ACPI: Subsystem revision 20050902
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 11 *12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKI] (IRQs 3 *4 5 6 7 9 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKJ] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKK] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKL] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0
ACPI: PCI Interrupt Link [LNKM] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKN] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKO] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKP] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKQ] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKR] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKS] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKT] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKU] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKV] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKW] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKX] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKY] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKZ] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNK0] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNK1] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNK2] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNK3] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNK4] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNK5] (IRQs 5 10 11 12 14 15) *0
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI: Probing PCI hardware (bus 00)
ACPI: _BBN and _CRS returns different value for \_SB_.PCI1. Select _CRS
ACPI: _BBN and _CRS returns different value for \_SB_.PCI2. Select _CRS
Boot video device is 0000:00:03.0
PCI: Ignoring BAR0-3 of IDE controller 0000:00:0f.1
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI _CRS 1 overrides _BBN 0
ACPI: PCI Root Bridge [PCI1] (0000:01)
PCI: Probing PCI hardware (bus 01)
ACPI: _BBN and _CRS returns different value for \_SB_.PCI1. Select _CRS
ACPI: _BBN and _CRS returns different value for \_SB_.PCI2. Select _CRS
ACPI: PCI Interrupt Routing Table [\_SB_.PCI1._PRT]
ACPI: PCI _CRS 2 overrides _BBN 0
ACPI: PCI Root Bridge [PCI2] (0000:02)
PCI: Probing PCI hardware (bus 02)
ACPI: _BBN and _CRS returns different value for \_SB_.PCI1. Select _CRS
ACPI: _BBN and _CRS returns different value for \_SB_.PCI2. Select _CRS
ACPI: PCI Interrupt Routing Table [\_SB_.PCI2._PRT]
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
Simple Boot Flag at 0x3a set to 0x80
highmem bounce pool size: 64 pages
Total HugeTLB memory allocated, 0
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered
Linux agpgart interface v0.101 (c) Dave Jones
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
SvrWks CSB5: IDE controller at PCI slot 0000:00:0f.1
SvrWks CSB5: chipset revision 147
SvrWks CSB5: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0xa800-0xa807, BIOS settings: hda:pio, hdb:pio
    ide1: BM-DMA at 0xa808-0xa80f, BIOS settings: hdc:pio, hdd:pio
Probing IDE interface ide0...
Probing IDE interface ide1...
Probing IDE interface ide0...
Probing IDE interface ide1...
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
NET: Registered protocol family 2
IP route cache hash table entries: 131072 (order: 7, 524288 bytes)
TCP established hash table entries: 524288 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
TCP reno registered
TCP bic registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
Starting balanced_irq
Using IPI Shortcut mode
Freeing unused kernel memory: 188k freed
SCSI subsystem initialized
megaraid cmm: 2.20.2.6 (Release Date: Mon Mar 7 00:01:03 EST 2005)
megaraid: 2.20.4.6 (Release Date: Mon Mar 07 12:27:22 EST 2005)
megaraid: probe new device 0x1000:0x1960:0x1000:0xa520: bus 1:slot 3:func 0
ACPI: PCI Interrupt 0000:01:03.0[A] -> GSI 27 (level, low) -> IRQ 16
megaraid: fw version:[1Z26] bios version:[G112]
scsi0 : LSI Logic MegaRAID driver
scsi[0]: scanning scsi channel 0 [Phy 0] for non-raid devices
  Vendor: SDR       Model: GEM318            Rev: 0   
  Type:   Processor                          ANSI SCSI revision: 02
scsi[0]: scanning scsi channel 1 [Phy 1] for non-raid devices
scsi[0]: scanning scsi channel 2 [virtual] for logical drives
  Vendor: MegaRAID  Model: LD0 RAID5 70090R  Rev: 1Z26
  Type:   Direct-Access                      ANSI SCSI revision: 02
SCSI device sda: 143544320 512-byte hdwr sectors (73495 MB)
sda: asking for cache data failed
sda: assuming drive cache: write through
SCSI device sda: 143544320 512-byte hdwr sectors (73495 MB)
sda: asking for cache data failed
sda: assuming drive cache: write through
 sda: sda1 sda2 sda3 sda4 < sda5 >
sd 0:2:0:0: Attached scsi disk sda
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
 0:0:11:0: Attached scsi generic sg0 type 3
sd 0:2:0:0: Attached scsi generic sg1 type 0
Floppy drive(s): fd0 is 1.44M
FDC 0 is a National Semiconductor PC87306
Intel(R) PRO/1000 Network Driver - version 6.1.16-k2-NAPI
Copyright (c) 1999-2005 Intel Corporation.
ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 18 (level, low) -> IRQ 17
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
ACPI: PCI Interrupt 0000:01:02.0[A] -> GSI 24 (level, low) -> IRQ 18
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
piix4_smbus 0000:00:0f.0: Found 0000:00:0f.0 device
piix4_smbus 0000:00:0f.0: Unusual config register value
piix4_smbus 0000:00:0f.0: Try using fix_hstcfg=1 if you experience problems
piix4_smbus 0000:00:0f.0: Illegal Interrupt configuration (or code out of date)!
ohci_hcd: 2005 April 22 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI)
ACPI: PCI Interrupt Link [LNK5] BIOS reported IRQ 0, using IRQ 11
ACPI: PCI Interrupt Link [LNK5] enabled at IRQ 11
ACPI: PCI Interrupt 0000:00:0f.2[A] -> Link [LNK5] -> GSI 11 (level, low) -> IRQ 11
ohci_hcd 0000:00:0f.2: OHCI Host Controller
ohci_hcd 0000:00:0f.2: new USB bus registered, assigned bus number 1
ohci_hcd 0000:00:0f.2: irq 11, io mem 0xf5000000
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 4 ports detected
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
ACPI: Power Button (FF) [PWRF]
ACPI: Power Button (CM) [PWRB]
ibm_acpi: ec object not found
EXT3 FS on sda1, internal journal
device-mapper: 4.4.0-ioctl (2005-01-12) initialised: dm-devel@redhat.com
program dmraid is using a deprecated SCSI ioctl, please convert it to SG_IO
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds
EXT3 FS on sda5, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
Adding 1052248k swap on /dev/sda2.  Priority:-1 extents:1 across:1052248k
ip_tables: (C) 2000-2002 Netfilter core team
ip_conntrack version 2.4 (8192 buckets, 65536 max) - 212 bytes per conntrack
e1000: eth0: e1000_watchdog_task: NIC Link is Up 100 Mbps Full Duplex
e1000: eth1: e1000_watchdog_task: NIC Link is Up 100 Mbps Full Duplex
e1000: eth1: e1000_watchdog_task: NIC Link is Up 100 Mbps Full Duplex
loop: loaded (max 8 devices)
TCP: Treason uncloaked! Peer 80.72.16.78:50766/80 shrinks window 2092777962:2092777963. Repaired.
TCP: Treason uncloaked! Peer 80.72.16.78:50769/80 shrinks window 2096690385:2096690386. Repaired.
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/core/stream.c (279)
KERNEL: assertion (!sk->sk_forward_alloc) failed at net/ipv4/af_inet.c (148)

[-- Attachment #3: ip_route_show --]
[-- Type: text/plain, Size: 222 bytes --]

83.102.130.172/30 dev eth0  proto kernel  scope link  src 83.102.130.174 
83.102.130.176/30 dev eth1  proto kernel  scope link  src 83.102.130.178 
169.254.0.0/16 dev eth1  scope link 
default via 83.102.130.177 dev eth1 

[-- Attachment #4: iptables --]
[-- Type: text/plain, Size: 1650 bytes --]

Chain INPUT (policy DROP)
target     prot opt source               destination         
DROP       all  --  212.176.84.130       0.0.0.0/0           
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           limit: avg 100/sec burst 5 
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
DROP       all  --  127.0.0.1            0.0.0.0/0           
DROP       all  --  0.0.0.0/0            127.0.0.1           
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:22 
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:25 
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:80 
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:3690 
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:5432 
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           multiport dports 20,21 
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpts:64000:65500 
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state ESTABLISHED 

Chain FORWARD (policy DROP)
target     prot opt source               destination         

Chain OUTPUT (policy DROP)
target     prot opt source               destination         
DROP       all  --  0.0.0.0/0            212.176.84.130      
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           limit: avg 100/sec burst 5 
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state ESTABLISHED 
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           state NEW 

[-- Attachment #5: lspci_vv --]
[-- Type: text/plain, Size: 6906 bytes --]

00:00.0 Host bridge: Broadcom CMIC-LE Host Bridge (GC-LE chipset) (rev 33)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-

00:00.1 Host bridge: Broadcom CMIC-LE Host Bridge (GC-LE chipset)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-

00:00.2 Host bridge: Broadcom CMIC-LE Host Bridge (GC-LE chipset)
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-

00:02.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)
	Subsystem: Intel Corporation 82540EM Gigabit Ethernet Controller
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
	Latency: 32 (63750ns min), Cache Line Size 08
	Interrupt: pin A routed to IRQ 17
	Region 0: Memory at f7000000 (32-bit, non-prefetchable) [size=128K]
	Region 2: I/O ports at d800 [size=64]
	Capabilities: [dc] Power Management version 2
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [e4] PCI-X non-bridge device.
		Command: DPERE- ERO+ RBC=0 OST=0
		Status: Bus=0 Dev=0 Func=0 64bit- 133MHz- SCD- USC-, DC=simple, DMMRBC=2, DMOST=0, DMCRS=1, RSCEM-
	Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
		Address: 0000000000000000  Data: 0000

00:03.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27) (prog-if 00 [VGA])
	Subsystem: ATI Technologies Inc Rage XL
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping+ SERR- FastB2B-
	Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
	Latency: 32 (2000ns min), Cache Line Size 08
	Interrupt: pin A routed to IRQ 10
	Region 0: Memory at f6000000 (32-bit, non-prefetchable) [size=16M]
	Region 1: I/O ports at d400 [size=256]
	Region 2: Memory at f5800000 (32-bit, non-prefetchable) [size=4K]
	Expansion ROM at febe0000 [disabled] [size=128K]
	Capabilities: [5c] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 PME-Enable- DSel=0 DScale=0 PME-

00:0f.0 ISA bridge: Broadcom CSB5 South Bridge (rev 93)
	Subsystem: Broadcom CSB5 South Bridge
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ >SERR- <PERR-
	Latency: 32

00:0f.1 IDE interface: Broadcom CSB5 IDE Controller (rev 93) (prog-if 8a [Master SecP PriP])
	Subsystem: Broadcom CSB5 IDE Controller
	Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
	Latency: 64, Cache Line Size 08
	Region 0: I/O ports at <ignored>
	Region 1: I/O ports at <ignored>
	Region 2: I/O ports at <ignored>
	Region 3: I/O ports at <ignored>
	Region 4: I/O ports at a800 [size=16]

00:0f.2 USB Controller: Broadcom OSB4/CSB5 OHCI USB Controller (rev 05) (prog-if 10 [OHCI])
	Subsystem: Broadcom OSB4/CSB5 OHCI USB Controller
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
	Latency: 32 (20000ns max), Cache Line Size 08
	Interrupt: pin A routed to IRQ 11
	Region 0: Memory at f5000000 (32-bit, non-prefetchable) [size=4K]

00:0f.3 Host bridge: Broadcom CSB5 LPC bridge
	Subsystem: Broadcom: Unknown device 0230
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
	Latency: 0

00:11.0 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 05)
	Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr+ DEVSEL=medium >TAbort- <TAbort- <MAbort+ >SERR- <PERR-
	Capabilities: [60] PCI-X non-bridge device.
		Command: DPERE- ERO- RBC=0 OST=4
		Status: Bus=0 Dev=0 Func=0 64bit+ 133MHz+ SCD- USC-, DC=bridge, DMMRBC=0, DMOST=4, DMCRS=0, RSCEM-

00:11.2 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 05)
	Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr+ DEVSEL=medium >TAbort- <TAbort- <MAbort+ >SERR- <PERR-
	Capabilities: [60] PCI-X non-bridge device.
		Command: DPERE- ERO- RBC=0 OST=4
		Status: Bus=0 Dev=0 Func=0 64bit+ 133MHz+ SCD- USC-, DC=bridge, DMMRBC=0, DMOST=4, DMCRS=0, RSCEM-

01:02.0 Ethernet controller: Intel Corporation 82544GC Gigabit Ethernet Controller (LOM) (rev 02)
	Subsystem: Intel Corporation 82544GC Based Network Connection
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap+ 66Mhz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
	Latency: 32 (63750ns min), Cache Line Size 08
	Interrupt: pin A routed to IRQ 18
	Region 0: Memory at f4800000 (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at f4000000 (32-bit, non-prefetchable) [size=128K]
	Region 2: I/O ports at a400 [size=32]
	Expansion ROM at feae0000 [disabled] [size=128K]
	Capabilities: [dc] Power Management version 2
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [e4] PCI-X non-bridge device.
		Command: DPERE- ERO+ RBC=0 OST=0
		Status: Bus=0 Dev=0 Func=0 64bit+ 133MHz+ SCD- USC-, DC=simple, DMMRBC=2, DMOST=0, DMCRS=1, RSCEM-
	Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
		Address: 0000000000000000  Data: 0000

01:03.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID (rev 01)
	Subsystem: LSI Logic / Symbios Logic MegaRAID ZCR SCSI 320-0 Controller
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B-
	Status: Cap+ 66Mhz+ UDF- FastB2B+ ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort+ >SERR- <PERR-
	Latency: 32, Cache Line Size 08
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at f8000000 (32-bit, prefetchable) [size=64M]
	[virtual] Expansion ROM at c4000000 [disabled] [size=32K]
	Capabilities: [80] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 PME-Enable- DSel=0 DScale=0 PME-

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Johan Lundgren]

Hi,

>What seems to cause this problem?

That I cannot say but the problem was fixed by removing one e1000 card
from the server (I initially had two e1000 cards installed in addition
to the two tg3 cards on the board).

Another fix was to disable TSO with ethtool.

>What motherboards are you using?

Supermicro H8DAE (dual Opteron)

>Are you all using iptables?  Are you all routing?

Iptables yes, routing no.

>none of you are using an 82571/2/3 (pci express)

Correct.

Regards,
Johan


On 3/30/06, Brandeburg, Jesse <jesse.brandeburg@intel.com> wrote:
> Hi all, I've identified you as people who have at some point in the past
> emailed one of the Linux lists with problems with e1000 and
> sk_forward_alloc.  It seems to be fairly widespread, but only seems to
> have appeared with recent kernel changes (after 2.6.12...)
>
> What I need from you is a reproducible test, and some information.  I
> have never been able to reproduce this, and I'm trying to isolate the
> problem a bit.  What motherboards are you using?  What seems to cause
> this problem?  Are you all using iptables?  Are you all routing? From
> the reports I assume none of you are using an 82571/2/3 (pci express)
>
> As far as I know e1000 has the same requirement as tg3 and some others
> where we have to modify the header of the skb in the case of transmits
> using TSO.  I don't see anywhere else that the driver modifies the skb.
> Tomorrow I'll generate a patch to try a more paranoid copying of the
> skb, I hope some of you can test.
>
> To do this we have code like so in e1000_tso:
> 2529         if (skb_shinfo(skb)->tso_size) {
> 2530                 if (skb_header_cloned(skb)) {
> 2531                         err = pskb_expand_head(skb, 0, 0,
> GFP_ATOMIC);
> 2532                         if (err)
> 2533                                 return err;
> 2534                 }
> 2535
> 2536                 hdr_len = ((skb->h.raw - skb->data) +
> (skb->h.th->doff << 2));
> 2537                 mss = skb_shinfo(skb)->tso_size;
> 2538                 if (skb->protocol == ntohs(ETH_P_IP)) {
> 2539                         skb->nh.iph->tot_len = 0;
> 2540                         skb->nh.iph->check = 0;
>
> Thanks for your assistance
>
> Jesse
>


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x110944&bid$1720&dat\x121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Krzysztof Oledzki]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2365 bytes --]



On Wed, 29 Mar 2006, Brandeburg, Jesse wrote:

> Hi all, I've identified you as people who have at some point in the past
> emailed one of the Linux lists with problems with e1000 and
> sk_forward_alloc.  It seems to be fairly widespread, but only seems to
> have appeared with recent kernel changes (after 2.6.12...)
>
> What I need from you is a reproducible test, and some information.  I
> have never been able to reproduce this, and I'm trying to isolate the
> problem a bit.  What motherboards are you using?
RIOWORKS/PDRCA.

# lspci
00:00.0 Host bridge: ServerWorks GCNB-LE Host Bridge (rev 32)
00:00.1 Host bridge: ServerWorks GCNB-LE Host Bridge
00:02.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)
00:04.0 I2O: Adaptec (formerly DPT) SmartRAID V Controller (rev 01)
00:04.1 PCI bridge: Adaptec (formerly DPT) PCI Bridge (rev 01)
00:06.0 Ethernet controller: D-Link System Inc DL2000-based Gigabit Ethernet (rev 0c)
00:08.0 SCSI storage controller: Initio Corporation INI-A100U2W (rev 01)
00:0e.0 IDE interface: ServerWorks CSB6 IDE Controller (rev a0)
00:0f.0 Host bridge: ServerWorks CSB6 South Bridge (rev a0)
00:0f.1 IDE interface: ServerWorks CSB6 RAID/IDE Controller (rev a0)
00:0f.2 USB Controller: ServerWorks CSB6 OHCI USB Controller (rev 05)
00:0f.3 ISA bridge: ServerWorks GCLE-2 Host Bridge

> What seems to cause this problem?
Don't known as this problem occurs only occasionally.

> Are you all using iptables?
Yes, this is a www proxy server with "-j REDIRECT".

>  Are you all routing?
Some kind as this is a transparent www proxy.

> From the reports I assume none of you are using an 82571/2/3 (pci 
> express)


00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)
         Subsystem: Rioworks: Unknown device 3011
         Flags: bus master, 66Mhz, medium devsel, latency 64, IRQ 169
         Memory at d0000000 (32-bit, non-prefetchable) [size=128K]
         I/O ports at 2c00 [size=64]
         Capabilities: [dc] Power Management version 2
         Capabilities: [e4] PCI-X non-bridge device.
         Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-

Thank you.

Best regards,

 			Krzysztof Olędzki

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Ingo Oeser]

[-- Attachment #1: Type: text/plain, Size: 3113 bytes --]

Hi Jesse,

More datapoints.

First of all, I don't see the problem, so this is an exclusion data point.

Machine is up 1 day, 19:02

I use 2.6.16 and I'm NBOT running at Gigabit speed.
(just couldn't get e100 cards anymore, they are not sold anymore here)

Version: vendor 00:aa:00, model 56 rev 0

Services: I'm routing, run IPsec, do firewalling/nat with iptables, do PPPoE
	on this machine but all not on that interface. 

The card is exposed to the local LAN interface.

[4294671.426000] e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
[4294671.447000] e100: eth1: e100_probe: addr 0xe3140000, irq 10, MAC addr 00:D0:B7:XX:XX:XX
[4294671.447000] eth2: VIA Rhine II at 0xe3142000, 00:0f:ea:XX:XX:XX, IRQ 11.
[4294671.448000] eth2: MII PHY found at address 1, status 0x786d advertising 05e1 Link 41e1.
[4294671.448000] forcedeth.c: Reverse Engineered nForce ethernet driver. Version 0.49.
[4294679.125000] e1000: eth0: e1000_watchdog_task: NIC Link is Up 100 Mbps Full Duplex
[4294679.165000] e100: eth1: e100_watchdog: link up, 10Mbps, half-duplex
[4294679.201000] eth2: link up, 100Mbps, full-duplex, lpa 0x41E1

lspci with info for this card.

0000:00:0c.0 Ethernet controller: Intel Corp. 82541GI/PI Gigabit Ethernet Controller
        Subsystem: Intel Corp. PRO/1000 MT Desktop Adapter
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
        Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR-
        Latency: 32 (63750ns min), Cache Line Size: 0x08 (32 bytes)
        Interrupt: pin A routed to IRQ 11
        Region 0: Memory at e3100000 (32-bit, non-prefetchable) [size=128K]
        Region 1: Memory at e3120000 (32-bit, non-prefetchable) [size=128K]
        Region 2: I/O ports at d400 [size=64]
        Expansion ROM at 20100000 [disabled] [size=128K]
        Capabilities: [dc] Power Management version 2
                Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                Status: D0 PME-Enable- DSel=0 DScale=1 PME-
        Capabilities: [e4] PCI-X non-bridge device.
                Command: DPERE- ERO+ RBC=0 OST=0
                Status: Bus=0 Dev=0 Func=0 64bit- 133MHz- SCD- USC-, DC=simple, DMMRBC=2, DMOST=0, DMCRS=0, RSCEM-
        Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
                Address: 0000000000000000  Data: 0000

ip -s -s link dev eth0
1: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0e:0c:XX:XX:XX brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast
    648259157  1081155  0       0       0       115
    RX errors: length  crc     frame   fifo    missed
               0        0       0       0       0
    TX: bytes  packets  errors  dropped carrier collsns
    393218241  933436   0       0       0       0
    TX errors: aborted fifo    window  heartbeat
               0        0       0       0

My config is attached, more data on request.
I can play with parameters, but cannot test patches.


Regards

Ingo Oeser

[-- Attachment #2: config-2.6.16 --]
[-- Type: text/plain, Size: 34769 bytes --]

#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.16
# Thu Mar 23 15:29:59 2006
#
CONFIG_X86_32=y
CONFIG_SEMAPHORE_SLEEPERS=y
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_DMI=y

#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_BROKEN_ON_SMP=y
CONFIG_INIT_ENV_ARG_LIMIT=32

#
# General setup
#
CONFIG_LOCALVERSION=""
CONFIG_LOCALVERSION_AUTO=y
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
# CONFIG_POSIX_MQUEUE is not set
# CONFIG_BSD_PROCESS_ACCT is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_UID16=y
# CONFIG_VM86 is not set
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_EMBEDDED=y
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SHMEM=y
CONFIG_CC_ALIGN_FUNCTIONS=0
CONFIG_CC_ALIGN_LABELS=0
CONFIG_CC_ALIGN_LOOPS=0
CONFIG_CC_ALIGN_JUMPS=0
CONFIG_SLAB=y
# CONFIG_TINY_SHMEM is not set
CONFIG_BASE_SMALL=0
# CONFIG_SLOB is not set

#
# Loadable module support
#
# CONFIG_MODULES is not set

#
# Block layer
#
# CONFIG_LBD is not set

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
CONFIG_DEFAULT_AS=y
# CONFIG_DEFAULT_DEADLINE is not set
# CONFIG_DEFAULT_CFQ is not set
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="anticipatory"

#
# Processor type and features
#
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
CONFIG_MK7=y
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MGEODEGX1 is not set
# CONFIG_MGEODE_LX is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
CONFIG_X86_GENERIC=y
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=7
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_USE_3DNOW=y
CONFIG_X86_TSC=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
# CONFIG_SMP is not set
CONFIG_PREEMPT_NONE=y
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT is not set
# CONFIG_X86_UP_APIC is not set
CONFIG_X86_MCE=y
CONFIG_X86_MCE_NONFATAL=y
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
CONFIG_X86_REBOOTFIXUPS=y
# CONFIG_MICROCODE is not set
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set

#
# Firmware Drivers
#
# CONFIG_EDD is not set
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set
CONFIG_NOHIGHMEM=y
# CONFIG_HIGHMEM4G is not set
# CONFIG_HIGHMEM64G is not set
# CONFIG_VMSPLIT_3G is not set
CONFIG_VMSPLIT_3G_OPT=y
# CONFIG_VMSPLIT_2G is not set
# CONFIG_VMSPLIT_1G is not set
CONFIG_PAGE_OFFSET=0xB0000000
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_FLATMEM_MANUAL=y
# CONFIG_DISCONTIGMEM_MANUAL is not set
# CONFIG_SPARSEMEM_MANUAL is not set
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
CONFIG_SPARSEMEM_STATIC=y
CONFIG_SPLIT_PTLOCK_CPUS=4
# CONFIG_MATH_EMULATION is not set
# CONFIG_MTRR is not set
# CONFIG_EFI is not set
CONFIG_REGPARM=y
CONFIG_SECCOMP=y
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
CONFIG_HZ_1000=y
CONFIG_HZ=1000
# CONFIG_KEXEC is not set
CONFIG_PHYSICAL_START=0x100000
CONFIG_DOUBLEFAULT=y

#
# Power management options (ACPI, APM)
#
CONFIG_PM=y
CONFIG_PM_LEGACY=y
# CONFIG_PM_DEBUG is not set
# CONFIG_SOFTWARE_SUSPEND is not set

#
# ACPI (Advanced Configuration and Power Interface) Support
#
CONFIG_ACPI=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_SLEEP_PROC_FS=y
# CONFIG_ACPI_SLEEP_PROC_SLEEP is not set
# CONFIG_ACPI_AC is not set
# CONFIG_ACPI_BATTERY is not set
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_VIDEO=y
# CONFIG_ACPI_HOTKEY is not set
CONFIG_ACPI_FAN=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_ASUS is not set
# CONFIG_ACPI_IBM is not set
# CONFIG_ACPI_TOSHIBA is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_SYSTEM=y
CONFIG_X86_PM_TIMER=y
# CONFIG_ACPI_CONTAINER is not set

#
# APM (Advanced Power Management) BIOS Support
#
CONFIG_APM=y
# CONFIG_APM_IGNORE_USER_SUSPEND is not set
# CONFIG_APM_DO_ENABLE is not set
# CONFIG_APM_CPU_IDLE is not set
# CONFIG_APM_DISPLAY_BLANK is not set
CONFIG_APM_RTC_IS_GMT=y
# CONFIG_APM_ALLOW_INTS is not set
# CONFIG_APM_REAL_MODE_POWER_OFF is not set

#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set

#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCIEPORTBUS=y
# CONFIG_PCI_LEGACY_PROC is not set
# CONFIG_PCI_DEBUG is not set
CONFIG_ISA_DMA_API=y
# CONFIG_ISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set

#
# PCCARD (PCMCIA/CardBus) support
#
# CONFIG_PCCARD is not set

#
# PCI Hotplug Support
#
# CONFIG_HOTPLUG_PCI is not set

#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
# CONFIG_BINFMT_AOUT is not set
# CONFIG_BINFMT_MISC is not set

#
# Networking
#
CONFIG_NET=y

#
# Networking options
#
# CONFIG_NETDEBUG is not set
CONFIG_PACKET=y
CONFIG_PACKET_MMAP=y
CONFIG_UNIX=y
CONFIG_XFRM=y
CONFIG_XFRM_USER=y
CONFIG_NET_KEY=y
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_ASK_IP_FIB_HASH=y
# CONFIG_IP_FIB_TRIE is not set
CONFIG_IP_FIB_HASH=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_FWMARK=y
# CONFIG_IP_ROUTE_MULTIPATH is not set
CONFIG_IP_ROUTE_VERBOSE=y
# CONFIG_IP_PNP is not set
CONFIG_NET_IPIP=y
CONFIG_NET_IPGRE=y
CONFIG_NET_IPGRE_BROADCAST=y
# CONFIG_IP_MROUTE is not set
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=y
CONFIG_INET_ESP=y
CONFIG_INET_IPCOMP=y
CONFIG_INET_TUNNEL=y
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_BIC=y

#
# IP: Virtual Server Configuration
#
# CONFIG_IP_VS is not set
# CONFIG_IPV6 is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set

#
# Core Netfilter Configuration
#
CONFIG_NETFILTER_NETLINK=y
# CONFIG_NETFILTER_NETLINK_QUEUE is not set
# CONFIG_NETFILTER_NETLINK_LOG is not set
CONFIG_NETFILTER_XTABLES=y
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=y
# CONFIG_NETFILTER_XT_TARGET_CONNMARK is not set
CONFIG_NETFILTER_XT_TARGET_MARK=y
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=y
CONFIG_NETFILTER_XT_MATCH_COMMENT=y
# CONFIG_NETFILTER_XT_MATCH_CONNBYTES is not set
CONFIG_NETFILTER_XT_MATCH_CONNMARK=y
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=y
# CONFIG_NETFILTER_XT_MATCH_DCCP is not set
CONFIG_NETFILTER_XT_MATCH_HELPER=y
CONFIG_NETFILTER_XT_MATCH_LENGTH=y
CONFIG_NETFILTER_XT_MATCH_LIMIT=y
CONFIG_NETFILTER_XT_MATCH_MAC=y
CONFIG_NETFILTER_XT_MATCH_MARK=y
CONFIG_NETFILTER_XT_MATCH_PKTTYPE=y
# CONFIG_NETFILTER_XT_MATCH_REALM is not set
# CONFIG_NETFILTER_XT_MATCH_SCTP is not set
CONFIG_NETFILTER_XT_MATCH_STATE=y
CONFIG_NETFILTER_XT_MATCH_STRING=y
CONFIG_NETFILTER_XT_MATCH_TCPMSS=y

#
# IP: Netfilter Configuration
#
CONFIG_IP_NF_CONNTRACK=y
CONFIG_IP_NF_CT_ACCT=y
CONFIG_IP_NF_CONNTRACK_MARK=y
# CONFIG_IP_NF_CONNTRACK_EVENTS is not set
# CONFIG_IP_NF_CONNTRACK_NETLINK is not set
# CONFIG_IP_NF_CT_PROTO_SCTP is not set
CONFIG_IP_NF_FTP=y
# CONFIG_IP_NF_IRC is not set
# CONFIG_IP_NF_NETBIOS_NS is not set
# CONFIG_IP_NF_TFTP is not set
# CONFIG_IP_NF_AMANDA is not set
# CONFIG_IP_NF_PPTP is not set
CONFIG_IP_NF_QUEUE=y
CONFIG_IP_NF_IPTABLES=y
CONFIG_IP_NF_MATCH_IPRANGE=y
# CONFIG_IP_NF_MATCH_MULTIPORT is not set
CONFIG_IP_NF_MATCH_TOS=y
CONFIG_IP_NF_MATCH_RECENT=y
CONFIG_IP_NF_MATCH_ECN=y
# CONFIG_IP_NF_MATCH_DSCP is not set
CONFIG_IP_NF_MATCH_AH_ESP=y
CONFIG_IP_NF_MATCH_TTL=y
CONFIG_IP_NF_MATCH_OWNER=y
CONFIG_IP_NF_MATCH_ADDRTYPE=y
# CONFIG_IP_NF_MATCH_HASHLIMIT is not set
CONFIG_IP_NF_MATCH_POLICY=y
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_REJECT=y
CONFIG_IP_NF_TARGET_LOG=y
# CONFIG_IP_NF_TARGET_ULOG is not set
CONFIG_IP_NF_TARGET_TCPMSS=y
CONFIG_IP_NF_NAT=y
CONFIG_IP_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=y
CONFIG_IP_NF_TARGET_REDIRECT=y
CONFIG_IP_NF_TARGET_NETMAP=y
CONFIG_IP_NF_TARGET_SAME=y
CONFIG_IP_NF_NAT_SNMP_BASIC=y
CONFIG_IP_NF_NAT_FTP=y
CONFIG_IP_NF_MANGLE=y
# CONFIG_IP_NF_TARGET_TOS is not set
# CONFIG_IP_NF_TARGET_ECN is not set
# CONFIG_IP_NF_TARGET_DSCP is not set
# CONFIG_IP_NF_TARGET_TTL is not set
CONFIG_IP_NF_TARGET_CLUSTERIP=y
# CONFIG_IP_NF_RAW is not set
# CONFIG_IP_NF_ARPTABLES is not set

#
# DCCP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_DCCP is not set

#
# SCTP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_SCTP is not set

#
# TIPC Configuration (EXPERIMENTAL)
#
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set

#
# QoS and/or fair queueing
#
CONFIG_NET_SCHED=y
# CONFIG_NET_SCH_CLK_JIFFIES is not set
# CONFIG_NET_SCH_CLK_GETTIMEOFDAY is not set
CONFIG_NET_SCH_CLK_CPU=y

#
# Queueing/Scheduling
#
CONFIG_NET_SCH_CBQ=y
CONFIG_NET_SCH_HTB=y
# CONFIG_NET_SCH_HFSC is not set
CONFIG_NET_SCH_PRIO=y
# CONFIG_NET_SCH_RED is not set
CONFIG_NET_SCH_SFQ=y
# CONFIG_NET_SCH_TEQL is not set
# CONFIG_NET_SCH_TBF is not set
# CONFIG_NET_SCH_GRED is not set
CONFIG_NET_SCH_DSMARK=y
# CONFIG_NET_SCH_NETEM is not set
CONFIG_NET_SCH_INGRESS=y

#
# Classification
#
CONFIG_NET_CLS=y
CONFIG_NET_CLS_BASIC=y
CONFIG_NET_CLS_TCINDEX=y
CONFIG_NET_CLS_ROUTE4=y
CONFIG_NET_CLS_ROUTE=y
CONFIG_NET_CLS_FW=y
CONFIG_NET_CLS_U32=y
# CONFIG_CLS_U32_PERF is not set
CONFIG_CLS_U32_MARK=y
CONFIG_NET_CLS_RSVP=y
# CONFIG_NET_CLS_RSVP6 is not set
CONFIG_NET_EMATCH=y
CONFIG_NET_EMATCH_STACK=32
# CONFIG_NET_EMATCH_CMP is not set
# CONFIG_NET_EMATCH_NBYTE is not set
# CONFIG_NET_EMATCH_U32 is not set
CONFIG_NET_EMATCH_META=y
# CONFIG_NET_EMATCH_TEXT is not set
CONFIG_NET_CLS_ACT=y
CONFIG_NET_ACT_POLICE=y
CONFIG_NET_ACT_GACT=y
CONFIG_GACT_PROB=y
CONFIG_NET_ACT_MIRRED=y
# CONFIG_NET_ACT_IPT is not set
# CONFIG_NET_ACT_PEDIT is not set
# CONFIG_NET_ACT_SIMP is not set
CONFIG_NET_CLS_IND=y
CONFIG_NET_ESTIMATOR=y

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_IEEE80211 is not set

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
# CONFIG_FW_LOADER is not set
# CONFIG_DEBUG_DRIVER is not set

#
# Connector - unified userspace <-> kernelspace linker
#
# CONFIG_CONNECTOR is not set

#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set

#
# Parallel port support
#
# CONFIG_PARPORT is not set

#
# Plug and Play support
#
# CONFIG_PNP is not set

#
# Block devices
#
CONFIG_BLK_DEV_FD=y
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_SX8 is not set
CONFIG_BLK_DEV_UB=y
# CONFIG_BLK_DEV_RAM is not set
CONFIG_BLK_DEV_RAM_COUNT=16
# CONFIG_CDROM_PKTCDVD is not set
# CONFIG_ATA_OVER_ETH is not set

#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y

#
# Please see Documentation/ide.txt for help/info on IDE drives
#
# CONFIG_BLK_DEV_IDE_SATA is not set
# CONFIG_BLK_DEV_HD_IDE is not set
CONFIG_BLK_DEV_IDEDISK=y
CONFIG_IDEDISK_MULTI_MODE=y
CONFIG_BLK_DEV_IDECD=y
# CONFIG_BLK_DEV_IDETAPE is not set
# CONFIG_BLK_DEV_IDEFLOPPY is not set
# CONFIG_BLK_DEV_IDESCSI is not set
# CONFIG_IDE_TASK_IOCTL is not set

#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=y
# CONFIG_BLK_DEV_CMD640 is not set
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_SHARE_IRQ=y
# CONFIG_BLK_DEV_OFFBOARD is not set
CONFIG_BLK_DEV_GENERIC=y
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_RZ1000 is not set
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not set
CONFIG_BLK_DEV_AMD74XX=y
CONFIG_BLK_DEV_ATIIXP=y
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5520 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_CS5535 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_SC1200 is not set
CONFIG_BLK_DEV_PIIX=y
# CONFIG_BLK_DEV_IT821X is not set
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_BLK_DEV_SVWKS is not set
CONFIG_BLK_DEV_SIIMAGE=y
CONFIG_BLK_DEV_SIS5513=y
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
CONFIG_BLK_DEV_VIA82CXXX=y
# CONFIG_IDE_ARM is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set

#
# SCSI device support
#
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=y
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=y
# CONFIG_BLK_DEV_SR_VENDOR is not set
# CONFIG_CHR_DEV_SG is not set
# CONFIG_CHR_DEV_SCH is not set

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
# CONFIG_SCSI_MULTI_LUN is not set
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y

#
# SCSI Transport Attributes
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_ATTRS is not set

#
# SCSI low-level drivers
#
# CONFIG_ISCSI_TCP is not set
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_SCSI_AIC7XXX is not set
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
# CONFIG_MEGARAID_SAS is not set
CONFIG_SCSI_SATA=y
CONFIG_SCSI_SATA_AHCI=y
CONFIG_SCSI_SATA_SVW=y
CONFIG_SCSI_ATA_PIIX=y
# CONFIG_SCSI_SATA_MV is not set
CONFIG_SCSI_SATA_NV=y
# CONFIG_SCSI_PDC_ADMA is not set
CONFIG_SCSI_SATA_QSTOR=y
CONFIG_SCSI_SATA_PROMISE=y
CONFIG_SCSI_SATA_SX4=y
CONFIG_SCSI_SATA_SIL=y
# CONFIG_SCSI_SATA_SIL24 is not set
CONFIG_SCSI_SATA_SIS=y
CONFIG_SCSI_SATA_ULI=y
CONFIG_SCSI_SATA_VIA=y
CONFIG_SCSI_SATA_VITESSE=y
CONFIG_SCSI_SATA_INTEL_COMBINED=y
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_QLOGIC_FC is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
# CONFIG_SCSI_QLA_FC is not set
# CONFIG_SCSI_LPFC is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_NSP32 is not set
# CONFIG_SCSI_DEBUG is not set

#
# Multi-device support (RAID and LVM)
#
CONFIG_MD=y
# CONFIG_BLK_DEV_MD is not set
CONFIG_BLK_DEV_DM=y
CONFIG_DM_CRYPT=y
CONFIG_DM_SNAPSHOT=y
CONFIG_DM_MIRROR=y
CONFIG_DM_ZERO=y
# CONFIG_DM_MULTIPATH is not set

#
# Fusion MPT device support
#
# CONFIG_FUSION is not set
# CONFIG_FUSION_SPI is not set
# CONFIG_FUSION_FC is not set
# CONFIG_FUSION_SAS is not set

#
# IEEE 1394 (FireWire) support
#
# CONFIG_IEEE1394 is not set

#
# I2O device support
#
# CONFIG_I2O is not set

#
# Network device support
#
CONFIG_NETDEVICES=y
# CONFIG_IFB is not set
# CONFIG_DUMMY is not set
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set

#
# ARCnet devices
#
# CONFIG_ARCNET is not set

#
# PHY device support
#
# CONFIG_PHYLIB is not set

#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
CONFIG_MII=y
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
# CONFIG_CASSINI is not set
CONFIG_NET_VENDOR_3COM=y
CONFIG_VORTEX=y
# CONFIG_TYPHOON is not set

#
# Tulip family network device support
#
# CONFIG_NET_TULIP is not set
# CONFIG_HP100 is not set
CONFIG_NET_PCI=y
CONFIG_PCNET32=y
# CONFIG_AMD8111_ETH is not set
# CONFIG_ADAPTEC_STARFIRE is not set
# CONFIG_B44 is not set
CONFIG_FORCEDETH=y
# CONFIG_DGRS is not set
# CONFIG_EEPRO100 is not set
CONFIG_E100=y
# CONFIG_FEALNX is not set
CONFIG_NATSEMI=y
# CONFIG_NE2K_PCI is not set
# CONFIG_8139CP is not set
CONFIG_8139TOO=y
# CONFIG_8139TOO_PIO is not set
CONFIG_8139TOO_TUNE_TWISTER=y
CONFIG_8139TOO_8129=y
# CONFIG_8139_OLD_RX_RESET is not set
CONFIG_SIS900=y
# CONFIG_EPIC100 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_TLAN is not set
CONFIG_VIA_RHINE=y
CONFIG_VIA_RHINE_MMIO=y

#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
CONFIG_E1000=y
CONFIG_E1000_NAPI=y
# CONFIG_E1000_DISABLE_PACKET_SPLIT is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SIS190 is not set
CONFIG_SKGE=y
# CONFIG_SKY2 is not set
# CONFIG_SK98LIN is not set
# CONFIG_VIA_VELOCITY is not set
CONFIG_TIGON3=y
# CONFIG_BNX2 is not set

#
# Ethernet (10000 Mbit)
#
# CONFIG_CHELSIO_T1 is not set
# CONFIG_IXGB is not set
# CONFIG_S2IO is not set

#
# Token Ring devices
#
# CONFIG_TR is not set

#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set

#
# Wan interfaces
#
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
CONFIG_PPP=y
# CONFIG_PPP_MULTILINK is not set
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=y
CONFIG_PPP_SYNC_TTY=y
CONFIG_PPP_DEFLATE=y
CONFIG_PPP_BSDCOMP=y
# CONFIG_PPP_MPPE is not set
CONFIG_PPPOE=y
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
# CONFIG_SHAPER is not set
# CONFIG_NETCONSOLE is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set

#
# ISDN subsystem
#
# CONFIG_ISDN is not set

#
# Telephony Support
#
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_TSDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
CONFIG_INPUT_MOUSE=y
# CONFIG_MOUSE_PS2 is not set
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
# CONFIG_GAMEPORT is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_SERIAL_NONSTANDARD is not set

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
# CONFIG_SERIAL_8250_ACPI is not set
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
CONFIG_SERIAL_8250_EXTENDED=y
# CONFIG_SERIAL_8250_MANY_PORTS is not set
CONFIG_SERIAL_8250_SHARE_IRQ=y
CONFIG_SERIAL_8250_DETECT_IRQ=y
# CONFIG_SERIAL_8250_RSA is not set

#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
# CONFIG_SERIAL_JSM is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_LEGACY_PTYS is not set

#
# IPMI
#
# CONFIG_IPMI_HANDLER is not set

#
# Watchdog Cards
#
CONFIG_WATCHDOG=y
# CONFIG_WATCHDOG_NOWAYOUT is not set

#
# Watchdog Device Drivers
#
CONFIG_SOFT_WATCHDOG=y
# CONFIG_ACQUIRE_WDT is not set
# CONFIG_ADVANTECH_WDT is not set
# CONFIG_ALIM1535_WDT is not set
# CONFIG_ALIM7101_WDT is not set
# CONFIG_SC520_WDT is not set
# CONFIG_EUROTECH_WDT is not set
# CONFIG_IB700_WDT is not set
# CONFIG_IBMASR is not set
# CONFIG_WAFER_WDT is not set
# CONFIG_I6300ESB_WDT is not set
# CONFIG_I8XX_TCO is not set
# CONFIG_SC1200_WDT is not set
# CONFIG_60XX_WDT is not set
# CONFIG_SBC8360_WDT is not set
# CONFIG_CPU5_WDT is not set
# CONFIG_W83627HF_WDT is not set
# CONFIG_W83877F_WDT is not set
# CONFIG_W83977F_WDT is not set
# CONFIG_MACHZ_WDT is not set
# CONFIG_SBC_EPX_C3_WATCHDOG is not set

#
# PCI-based Watchdog Cards
#
# CONFIG_PCIPCWATCHDOG is not set
# CONFIG_WDTPCI is not set

#
# USB-based Watchdog Cards
#
# CONFIG_USBPCWATCHDOG is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_NVRAM is not set
CONFIG_RTC=y
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set

#
# Ftape, the floppy tape device driver
#
# CONFIG_FTAPE is not set
# CONFIG_AGP is not set
# CONFIG_DRM is not set
# CONFIG_MWAVE is not set
# CONFIG_CS5535_GPIO is not set
# CONFIG_RAW_DRIVER is not set
CONFIG_HPET=y
# CONFIG_HPET_RTC_IRQ is not set
CONFIG_HPET_MMAP=y
CONFIG_HANGCHECK_TIMER=y

#
# TPM devices
#
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set

#
# I2C support
#
CONFIG_I2C=y
CONFIG_I2C_CHARDEV=y

#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=y
# CONFIG_I2C_ALGOPCF is not set
# CONFIG_I2C_ALGOPCA is not set

#
# I2C Hardware Bus support
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
# CONFIG_I2C_AMD756 is not set
# CONFIG_I2C_AMD8111 is not set
# CONFIG_I2C_I801 is not set
# CONFIG_I2C_I810 is not set
# CONFIG_I2C_PIIX4 is not set
CONFIG_I2C_ISA=y
CONFIG_I2C_NFORCE2=y
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_PROSAVAGE is not set
# CONFIG_I2C_SAVAGE4 is not set
# CONFIG_SCx200_ACB is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_VIA is not set
CONFIG_I2C_VIAPRO=y
# CONFIG_I2C_VOODOO3 is not set
# CONFIG_I2C_PCA_ISA is not set

#
# Miscellaneous I2C Chip support
#
# CONFIG_SENSORS_DS1337 is not set
# CONFIG_SENSORS_DS1374 is not set
CONFIG_SENSORS_EEPROM=y
# CONFIG_SENSORS_PCF8574 is not set
# CONFIG_SENSORS_PCA9539 is not set
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_SENSORS_RTC8564 is not set
# CONFIG_SENSORS_MAX6875 is not set
# CONFIG_RTC_X1205_I2C is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set

#
# SPI support
#
# CONFIG_SPI is not set
# CONFIG_SPI_MASTER is not set

#
# Dallas's 1-wire bus
#
# CONFIG_W1 is not set

#
# Hardware Monitoring support
#
CONFIG_HWMON=y
CONFIG_HWMON_VID=y
CONFIG_SENSORS_ADM1021=y
CONFIG_SENSORS_ADM1025=y
CONFIG_SENSORS_ADM1026=y
CONFIG_SENSORS_ADM1031=y
CONFIG_SENSORS_ADM9240=y
CONFIG_SENSORS_ASB100=y
CONFIG_SENSORS_ATXP1=y
CONFIG_SENSORS_DS1621=y
# CONFIG_SENSORS_F71805F is not set
CONFIG_SENSORS_FSCHER=y
CONFIG_SENSORS_FSCPOS=y
CONFIG_SENSORS_GL518SM=y
CONFIG_SENSORS_GL520SM=y
CONFIG_SENSORS_IT87=y
CONFIG_SENSORS_LM63=y
CONFIG_SENSORS_LM75=y
CONFIG_SENSORS_LM77=y
CONFIG_SENSORS_LM78=y
CONFIG_SENSORS_LM80=y
CONFIG_SENSORS_LM83=y
CONFIG_SENSORS_LM85=y
CONFIG_SENSORS_LM87=y
CONFIG_SENSORS_LM90=y
CONFIG_SENSORS_LM92=y
CONFIG_SENSORS_MAX1619=y
CONFIG_SENSORS_PC87360=y
CONFIG_SENSORS_SIS5595=y
CONFIG_SENSORS_SMSC47M1=y
CONFIG_SENSORS_SMSC47B397=y
CONFIG_SENSORS_VIA686A=y
# CONFIG_SENSORS_VT8231 is not set
CONFIG_SENSORS_W83781D=y
# CONFIG_SENSORS_W83792D is not set
CONFIG_SENSORS_W83L785TS=y
CONFIG_SENSORS_W83627HF=y
CONFIG_SENSORS_W83627EHF=y
# CONFIG_SENSORS_HDAPS is not set
# CONFIG_HWMON_DEBUG_CHIP is not set

#
# Misc devices
#
# CONFIG_IBM_ASM is not set

#
# Multimedia Capabilities Port drivers
#

#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set

#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set

#
# Graphics support
#
# CONFIG_FB is not set
# CONFIG_VIDEO_SELECT is not set

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_DUMMY_CONSOLE=y

#
# Sound
#
# CONFIG_SOUND is not set

#
# USB support
#
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB=y
# CONFIG_USB_DEBUG is not set

#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
# CONFIG_USB_BANDWIDTH is not set
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_SUSPEND is not set
# CONFIG_USB_OTG is not set

#
# USB Host Controller Drivers
#
CONFIG_USB_EHCI_HCD=y
# CONFIG_USB_EHCI_SPLIT_ISO is not set
# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
# CONFIG_USB_ISP116X_HCD is not set
CONFIG_USB_OHCI_HCD=y
# CONFIG_USB_OHCI_BIG_ENDIAN is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_UHCI_HCD=y
# CONFIG_USB_SL811_HCD is not set

#
# USB Device Class drivers
#
# CONFIG_USB_ACM is not set
# CONFIG_USB_PRINTER is not set

#
# NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support'
#

#
# may also be needed; see USB_STORAGE Help for more information
#
CONFIG_USB_STORAGE=y
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_DATAFAB is not set
# CONFIG_USB_STORAGE_FREECOM is not set
# CONFIG_USB_STORAGE_ISD200 is not set
# CONFIG_USB_STORAGE_DPCM is not set
# CONFIG_USB_STORAGE_USBAT is not set
# CONFIG_USB_STORAGE_SDDR09 is not set
# CONFIG_USB_STORAGE_SDDR55 is not set
# CONFIG_USB_STORAGE_JUMPSHOT is not set
# CONFIG_USB_STORAGE_ALAUDA is not set
# CONFIG_USB_LIBUSUAL is not set

#
# USB Input Devices
#
# CONFIG_USB_HID is not set

#
# USB HID Boot Protocol drivers
#
# CONFIG_USB_KBD is not set
# CONFIG_USB_MOUSE is not set
# CONFIG_USB_AIPTEK is not set
# CONFIG_USB_WACOM is not set
# CONFIG_USB_ACECAD is not set
# CONFIG_USB_KBTAB is not set
# CONFIG_USB_POWERMATE is not set
# CONFIG_USB_MTOUCH is not set
# CONFIG_USB_ITMTOUCH is not set
# CONFIG_USB_EGALAX is not set
# CONFIG_USB_YEALINK is not set
# CONFIG_USB_XPAD is not set
# CONFIG_USB_ATI_REMOTE is not set
# CONFIG_USB_ATI_REMOTE2 is not set
# CONFIG_USB_KEYSPAN_REMOTE is not set
# CONFIG_USB_APPLETOUCH is not set

#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set

#
# USB Multimedia devices
#
# CONFIG_USB_DABUSB is not set

#
# Video4Linux support is needed for USB Multimedia device support
#

#
# USB Network Adapters
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_USBNET is not set
CONFIG_USB_MON=y

#
# USB port drivers
#

#
# USB Serial Converter support
#
CONFIG_USB_SERIAL=y
CONFIG_USB_SERIAL_CONSOLE=y
CONFIG_USB_SERIAL_GENERIC=y
# CONFIG_USB_SERIAL_AIRPRIME is not set
# CONFIG_USB_SERIAL_ANYDATA is not set
# CONFIG_USB_SERIAL_BELKIN is not set
# CONFIG_USB_SERIAL_WHITEHEAT is not set
# CONFIG_USB_SERIAL_DIGI_ACCELEPORT is not set
# CONFIG_USB_SERIAL_CP2101 is not set
# CONFIG_USB_SERIAL_CYPRESS_M8 is not set
# CONFIG_USB_SERIAL_EMPEG is not set
CONFIG_USB_SERIAL_FTDI_SIO=y
# CONFIG_USB_SERIAL_VISOR is not set
# CONFIG_USB_SERIAL_IPAQ is not set
# CONFIG_USB_SERIAL_IR is not set
# CONFIG_USB_SERIAL_EDGEPORT is not set
# CONFIG_USB_SERIAL_EDGEPORT_TI is not set
# CONFIG_USB_SERIAL_GARMIN is not set
# CONFIG_USB_SERIAL_IPW is not set
# CONFIG_USB_SERIAL_KEYSPAN_PDA is not set
# CONFIG_USB_SERIAL_KEYSPAN is not set
# CONFIG_USB_SERIAL_KLSI is not set
# CONFIG_USB_SERIAL_KOBIL_SCT is not set
# CONFIG_USB_SERIAL_MCT_U232 is not set
# CONFIG_USB_SERIAL_PL2303 is not set
# CONFIG_USB_SERIAL_HP4X is not set
# CONFIG_USB_SERIAL_SAFE is not set
# CONFIG_USB_SERIAL_TI is not set
# CONFIG_USB_SERIAL_CYBERJACK is not set
# CONFIG_USB_SERIAL_XIRCOM is not set
# CONFIG_USB_SERIAL_OMNINET is not set

#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_AUERSWALD is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
CONFIG_USB_LCD=y
CONFIG_USB_LED=y
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_PHIDGETKIT is not set
# CONFIG_USB_PHIDGETSERVO is not set
# CONFIG_USB_IDMOUSE is not set
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_LD is not set
# CONFIG_USB_TEST is not set

#
# USB DSL modem support
#

#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set

#
# MMC/SD Card support
#
# CONFIG_MMC is not set

#
# InfiniBand support
#
# CONFIG_INFINIBAND is not set

#
# EDAC - error detection and reporting (RAS) (EXPERIMENTAL)
#
CONFIG_EDAC=y

#
# Reporting subsystems
#
# CONFIG_EDAC_DEBUG is not set
CONFIG_EDAC_MM_EDAC=y
CONFIG_EDAC_AMD76X=y
# CONFIG_EDAC_E7XXX is not set
# CONFIG_EDAC_E752X is not set
# CONFIG_EDAC_I82875P is not set
# CONFIG_EDAC_I82860 is not set
# CONFIG_EDAC_R82600 is not set
CONFIG_EDAC_POLL=y

#
# File systems
#
CONFIG_EXT2_FS=y
# CONFIG_EXT2_FS_XATTR is not set
# CONFIG_EXT2_FS_XIP is not set
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
# CONFIG_EXT3_FS_SECURITY is not set
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=y
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
CONFIG_FS_POSIX_ACL=y
# CONFIG_XFS_FS is not set
# CONFIG_OCFS2_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not set
CONFIG_INOTIFY=y
# CONFIG_QUOTA is not set
CONFIG_DNOTIFY=y
# CONFIG_AUTOFS_FS is not set
# CONFIG_AUTOFS4_FS is not set
# CONFIG_FUSE_FS is not set

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_ZISOFS_FS=y
# CONFIG_UDF_FS is not set

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
# CONFIG_NTFS_FS is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
# CONFIG_PROC_KCORE is not set
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_RAMFS=y
# CONFIG_RELAYFS_FS is not set
# CONFIG_CONFIGFS_FS is not set

#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set

#
# Network File Systems
#
# CONFIG_NFS_FS is not set
# CONFIG_NFSD is not set
# CONFIG_SMB_FS is not set
# CONFIG_CIFS is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
# CONFIG_9P_FS is not set

#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y

#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
CONFIG_NLS_CODEPAGE_850=y
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
CONFIG_NLS_ASCII=y
CONFIG_NLS_ISO8859_1=y
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
CONFIG_NLS_ISO8859_15=y
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
CONFIG_NLS_UTF8=y

#
# Instrumentation Support
#
# CONFIG_PROFILING is not set

#
# Kernel hacking
#
CONFIG_PRINTK_TIME=y
CONFIG_MAGIC_SYSRQ=y
CONFIG_DEBUG_KERNEL=y
CONFIG_LOG_BUF_SHIFT=17
CONFIG_DETECT_SOFTLOCKUP=y
# CONFIG_SCHEDSTATS is not set
# CONFIG_DEBUG_SLAB is not set
# CONFIG_DEBUG_MUTEXES is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_DEBUG_KOBJECT is not set
# CONFIG_DEBUG_BUGVERBOSE is not set
# CONFIG_DEBUG_INFO is not set
CONFIG_DEBUG_FS=y
# CONFIG_DEBUG_VM is not set
# CONFIG_FRAME_POINTER is not set
CONFIG_FORCED_INLINING=y
# CONFIG_RCU_TORTURE_TEST is not set
CONFIG_EARLY_PRINTK=y
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_PAGEALLOC is not set
CONFIG_DEBUG_RODATA=y
CONFIG_4KSTACKS=y

#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY is not set

#
# Cryptographic options
#
CONFIG_CRYPTO=y
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_NULL=y
CONFIG_CRYPTO_MD4=y
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_SHA256=y
CONFIG_CRYPTO_SHA512=y
CONFIG_CRYPTO_WP512=y
CONFIG_CRYPTO_TGR192=y
CONFIG_CRYPTO_DES=y
CONFIG_CRYPTO_BLOWFISH=y
CONFIG_CRYPTO_TWOFISH=y
CONFIG_CRYPTO_SERPENT=y
# CONFIG_CRYPTO_AES is not set
CONFIG_CRYPTO_AES_586=y
CONFIG_CRYPTO_CAST5=y
CONFIG_CRYPTO_CAST6=y
CONFIG_CRYPTO_TEA=y
CONFIG_CRYPTO_ARC4=y
CONFIG_CRYPTO_KHAZAD=y
CONFIG_CRYPTO_ANUBIS=y
CONFIG_CRYPTO_DEFLATE=y
CONFIG_CRYPTO_MICHAEL_MIC=y
# CONFIG_CRYPTO_CRC32C is not set
# CONFIG_CRYPTO_TEST is not set

#
# Hardware crypto devices
#
# CONFIG_CRYPTO_DEV_PADLOCK is not set

#
# Library routines
#
CONFIG_CRC_CCITT=y
# CONFIG_CRC16 is not set
CONFIG_CRC32=y
# CONFIG_LIBCRC32C is not set
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=y
CONFIG_TEXTSEARCH=y
CONFIG_TEXTSEARCH_KMP=y
CONFIG_TEXTSEARCH_BM=y
CONFIG_TEXTSEARCH_FSM=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_X86_BIOS_REBOOT=y
CONFIG_KTIME_SCALAR=y

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[David S. Miller]

From: Ingo Oeser <netdev@axxeo.de>
Date: Fri, 31 Mar 2006 10:57:06 +0200

> Hi Jesse,
> 
> More datapoints.
> 
> First of all, I don't see the problem, so this is an exclusion data point.
> 
> Machine is up 1 day, 19:02
> 
> I use 2.6.16 and I'm NBOT running at Gigabit speed.

If you're not running at gigabit speed, TSO is turned off
by the e1000 driver.

So that could by why you're not seeing the problem.


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Herbert Xu]

Ingo Oeser <netdev@axxeo.de> wrote:
> 
> More datapoints.
> 
> First of all, I don't see the problem, so this is an exclusion data point.

Great.  I think so far all the configurations that have this problem
are

e1000 + SMP + TSO

Since your machine is not SMP but has the other two things it would
indicate that this is an SMP race.

If there are anyone else out there who do have this problem and are
either using something other than e1000, have disabled TSO, or are UP,
please speak up now.

Thanks,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[David S. Miller]

From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri, 31 Mar 2006 20:16:53 +1100

> Ingo Oeser <netdev@axxeo.de> wrote:
> > 
> > More datapoints.
> > 
> > First of all, I don't see the problem, so this is an exclusion data point.
> 
> Great.  I think so far all the configurations that have this problem
> are
> 
> e1000 + SMP + TSO
> 
> Since your machine is not SMP but has the other two things it would
> indicate that this is an SMP race.

He does not have TSO enabled, e1000 disables TSO when on a link speed
slower than gigabit.

You'll see something like the following in your logs:

e1000: eth0: e1000_watchdog_task: 10/100 speed: disabling TSO


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Herbert Xu]

On Fri, Mar 31, 2006 at 01:35:40AM -0800, David S. Miller wrote:
> 
> He does not have TSO enabled, e1000 disables TSO when on a link speed
> slower than gigabit.

Indeed.  But I think that only happens on PCI Express and I don't think
Ingo is using PCI Express.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[JaniD++]

----- Original Message ----- 
From: "Herbert Xu" <herbert@gondor.apana.org.au>
To: "David S. Miller" <davem@davemloft.net>
Cc: <netdev@axxeo.de>; <jesse.brandeburg@intel.com>; <nipsy@bitgnome.net>;
<jrlundgren@gmail.com>; <cat@zip.com.au>; <djani22@dynamicweb.hu>;
<yoseph.basri@gmail.com>; <bb@kernelpanic.ru>; <mykleb@no.ibm.com>;
<olel@ans.pl>; <michal@feix.cz>; <chris@scorpion.nl>;
<netdev@vger.kernel.org>; <jesse.brandeburg@gmail.com>;
<E1000-devel@lists.sourceforge.net>
Sent: Friday, March 31, 2006 11:42 AM
Subject: Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...


> On Fri, Mar 31, 2006 at 01:35:40AM -0800, David S. Miller wrote:
> >
> > He does not have TSO enabled, e1000 disables TSO when on a link speed
> > slower than gigabit.
>
> Indeed.  But I think that only happens on PCI Express and I don't think
> Ingo is using PCI Express.

No, my card is "64-bit PCI-X Rev. 1.0 master interface".  - from the
datasheet
Number : "82546GB"
This is not PCI Express issue!

Cheers,

>
> Cheers,
> -- 
> Visit Openswan at http://www.openswan.org/
> Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>  _____________ NOD32 1.584 (20031220) Információ _____________
>
> Az üzenetet a NOD32 Antivirus System megvizsgálta.
> http://www.nod32.hu
>
>



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Ingo Oeser]

Hi,

Herbert Xu wrote:
> On Fri, Mar 31, 2006 at 01:35:40AM -0800, David S. Miller wrote:
> > He does not have TSO enabled, e1000 disables TSO when on a link speed
> > slower than gigabit.

dmesg|grep eth0
[4294671.426000] e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
[4294679.125000] e1000: eth0: e1000_watchdog_task: NIC Link is Up 100 Mbps Full Duplex


# ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on

So this theory doesn't seem to hold :-(

> Indeed.  But I think that only happens on PCI Express and I don't think
> Ingo is using PCI Express.

Right. PCI-Express is not available in this machine.

Maybe the traffic is not enough to trigger it. External connect is just a 6MBit DSL.


Regards

Ingo Oeser


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Jesse Brandeburg]

On Fri, 31 Mar 2006, Ingo Oeser wrote:

> Hi,
>
> Herbert Xu wrote:
>> On Fri, Mar 31, 2006 at 01:35:40AM -0800, David S. Miller wrote:
>>> He does not have TSO enabled, e1000 disables TSO when on a link speed
>>> slower than gigabit.
>
> dmesg|grep eth0
> [4294671.426000] e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
> [4294679.125000] e1000: eth0: e1000_watchdog_task: NIC Link is Up 100 Mbps Full Duplex
>
>
> # ethtool -k eth0
> Offload parameters for eth0:
> rx-checksumming: on
> tx-checksumming: on
> scatter-gather: on
> tcp segmentation offload: on
>
> So this theory doesn't seem to hold :-(
>
>> Indeed.  But I think that only happens on PCI Express and I don't think
>> Ingo is using PCI Express.
>
> Right. PCI-Express is not available in this machine.
>

First, thanks for all the responses.

6.3.9-k4 in 2.6.16 doesn't turn off TSO for 10/100, 7.0.33 in 2.6.17-pre 
does, I think that will help alleviate some of the confusion.

I've been working hard to try to reproduce here, no luck so far.

Herbert's fixes are interesting and appreciated.  I'm going to try to 
generate tests today that will show that the bugs he's mentioned could 
occur.

Jesse



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642

Re: [e1000 debug] KERNEL: assertion (!sk_forward_alloc) failed...

[Mark Nipper]

On 31 Mar 2006, David S. Miller wrote:
> He does not have TSO enabled, e1000 disables TSO when on a link speed
> slower than gigabit.
> 
> You'll see something like the following in your logs:
> 
> e1000: eth0: e1000_watchdog_task: 10/100 speed: disabling TSO

        Um...
---
$ uname -a
Linux king 2.6.16.1 #1 SMP Thu Mar 30 06:11:33 CST 2006 i686 GNU/Linux
$ dmesg | grep -i task
e1000: eth1: e1000_watchdog_task: NIC Link is Up 100 Mbps Full Duplex
$ ethtool -k eth1
Offload parameters for eth1:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on
---

        I know for a fact the link is 100Mbps (other than the
output from the driver itself) and I have been bitten by the
assertion.

        I've been running the first patch for about the last 24
hours and have not seen any assertions yet (although they don't
occur that frequently on this server).  I'll be adding the
second, most recent patch in a bit and rebooting again.
Hopefully between the two of them, that will have fixed the
problem.

-- 
Mark Nipper                                                e-contacts:
832 Tanglewood Drive                                nipsy@bitgnome.net
Bryan, Texas 77802-4013                     http://nipsy.bitgnome.net/
(979)575-3193                      AIM/Yahoo: texasnipsy ICQ: 66971617

-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GG/IT d- s++:+ a- C++$ UBL++++$ P--->+++ L+++$ !E---
W++(--) N+ o K++ w(---) O++ M V(--) PS+++(+) PE(--)
Y+ PGP t+ 5 X R tv b+++@ DI+(++) D+ G e h r++ y+(**)
------END GEEK CODE BLOCK------

---begin random quote of the moment---
And if I close my mind in fear, please pry it open.
----end random quote of the moment----


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642