tg3 timeouts with 2.6.17-rc6

tg3 timeouts with 2.6.17-rc6

[Juergen Kreileder]

Hi,

I'm seeing frequent network timeouts on my PowerMac G5 Quad with
2.6.17-rc6.  The timeouts are easily reproducible with moderate
network traffic, e.g. by using bittorrent.

,----
| NETDEV WATCHDOG: lan0: transmit timed out
| tg3: lan0: transmit timed out, resetting
| tg3: tg3_stop_block timed out, ofs=2c00 enable_bit=2
| tg3: tg3_stop_block timed out, ofs=1400 enable_bit=2
| tg3: lan0: Link is down.
| [...]
| tg3: lan0: Link is up at 1000 Mbps, full duplex.
| tg3: lan0: Flow control is on for TX and on for RX.
`----


        Juergen

-- 
Juergen Kreileder, Blackdown Java-Linux Team
http://blog.blackdown.de/

Re: tg3 timeouts with 2.6.17-rc6

[Michael Chan]

Juergen Kreileder

> I'm seeing frequent network timeouts on my PowerMac G5 Quad with
> 2.6.17-rc6.  The timeouts are easily reproducible with moderate
> network traffic, e.g. by using bittorrent.
> 

Did this use to work with an older kernel or older tg3 driver? If
yes, what version?

Please also provide the full tg3 probing output during modprobe and
ifconfig up. Thanks.

Re: tg3 timeouts with 2.6.17-rc6

[Juergen Kreileder]

Michael Chan <mchan@broadcom.com> writes:

> Juergen Kreileder
>
>> I'm seeing frequent network timeouts on my PowerMac G5 Quad with
>> 2.6.17-rc6.  The timeouts are easily reproducible with moderate
>> network traffic, e.g. by using bittorrent.
>>
>
> Did this use to work with an older kernel or older tg3 driver? If
> yes, what version?

Works fine with 2.6.16 and earlier.

> Please also provide the full tg3 probing output during modprobe and
> ifconfig up. Thanks.

,----
| tg3.c:v3.58 (May 22, 2006)
| eth0: Tigon3 [partno(BCM95780) rev 8003 PHY(5780)] (PCIX:133MHz:64-bit) 10/100/1000BaseT Ethernet 00:14:51:66:ff:b2
| eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1] 
| eth0: dma_rwctrl[76144000] dma_mask[40-bit]
| eth1: Tigon3 [partno(BCM95780) rev 8003 PHY(5780)] (PCIX:133MHz:64-bit) 10/100/1000BaseT Ethernet 00:14:51:66:ff:b3
| eth1: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1] 
| eth1: dma_rwctrl[76144000] dma_mask[40-bit]
| [...]
| tg3: lan0: Link is up at 1000 Mbps, full duplex.
| tg3: lan0: Flow control is on for TX and on for RX.
`----

eth0 and eth1 get renamed to lan0 and lan1 by udev.  eth0/lan0 is
connected to a Gigabit switch eth1/lan1 is not connected.
Both interface are controlled by ifplugd (using SIOCETHTOOL for link
beat detection).

Let me know if you need more debugging output.


        Juergen

-- 
Juergen Kreileder, Blackdown Java-Linux Team
http://blog.blackdown.de/

Re: tg3 timeouts with 2.6.17-rc6

[Michael Chan]

On Fri, 2006-06-16 at 23:20 +0200, Juergen Kreileder wrote:
> Michael Chan <mchan@broadcom.com> writes:
> 
> >
> > Did this use to work with an older kernel or older tg3 driver? If
> > yes, what version?
> 
> Works fine with 2.6.16 and earlier.
> 
> > Please also provide the full tg3 probing output during modprobe and
> > ifconfig up. Thanks.
> 
Looking at the patch history since 2.6.16, the only patch that could
have an impact is the one that enables TSO by default.

Please try turning TSO off to see if it makes a difference:

ethtool -K eth0 tso off

Re: tg3 timeouts with 2.6.17-rc6

[Juergen Kreileder]

Michael Chan <mchan@broadcom.com> writes:

> On Fri, 2006-06-16 at 23:20 +0200, Juergen Kreileder wrote:
>> Michael Chan <mchan@broadcom.com> writes:
>>
>>>
>>> Did this use to work with an older kernel or older tg3 driver? If
>>> yes, what version?
>>
>> Works fine with 2.6.16 and earlier.
>>
>>> Please also provide the full tg3 probing output during modprobe
>>> and ifconfig up. Thanks.
>>
> Looking at the patch history since 2.6.16, the only patch that could
> have an impact is the one that enables TSO by default.
>
> Please try turning TSO off to see if it makes a difference:
>
> ethtool -K eth0 tso off

Seems to work fine with TSO disabled.


        Juergen

-- 
Juergen Kreileder, Blackdown Java-Linux Team
http://blog.blackdown.de/

Re: tg3 timeouts with 2.6.17-rc6

[Michael Chan]

On Sat, 2006-06-17 at 02:05 +0200, Juergen Kreileder wrote:
> Michael Chan <mchan@broadcom.com> writes:
> > Please try turning TSO off to see if it makes a difference:
> >
> > ethtool -K eth0 tso off
> 
> Seems to work fine with TSO disabled.
> 
Thanks for the information. We'll look into it. If possible, please send
me (using private email) an ethereal trace of a sample of the packets
sent by the tg3 chip right before the NETDEV_WATCHDOG timeout.

In the meantime, I wonder if we should disable TSO by default on the
5780 chip for 2.6.17.

Re: tg3 timeouts with 2.6.17-rc6

[David Miller]

From: "Michael Chan" <mchan@broadcom.com>
Date: Fri, 16 Jun 2006 18:27:32 -0700

> In the meantime, I wonder if we should disable TSO by default on the
> 5780 chip for 2.6.17.

Sounds reasonable.  Would we disable it for all chips that set
TG3_FLG2_5780_CLASS or a specific variant?

Re: tg3 timeouts with 2.6.17-rc6

[Michael Chan]

David Miller wrote:

> From: "Michael Chan" <mchan@broadcom.com>
> Date: Fri, 16 Jun 2006 18:27:32 -0700
> 
> > In the meantime, I wonder if we should disable TSO by default on the
> > 5780 chip for 2.6.17.
> 
> Sounds reasonable.  Would we disable it for all chips that set
> TG3_FLG2_5780_CLASS or a specific variant?
> 
Yes, let's disable it for all TG3_FLG2_5780_CLASS chips for now
until we figure out what's going on.

Re: tg3 timeouts with 2.6.17-rc6

[Michael Chan]

On Fri, 2006-06-16 at 20:37 -0700, Michael Chan wrote:
> David Miller wrote:
> 
> > From: "Michael Chan" <mchan@broadcom.com>
> > Date: Fri, 16 Jun 2006 18:27:32 -0700
> > 
> > > In the meantime, I wonder if we should disable TSO by default on the
> > > 5780 chip for 2.6.17.
> > 
> > Sounds reasonable.  Would we disable it for all chips that set
> > TG3_FLG2_5780_CLASS or a specific variant?
> > 
> Yes, let's disable it for all TG3_FLG2_5780_CLASS chips for now
> until we figure out what's going on.

David, Here's the patch if you haven't already made one:

[TG3]: Disable TSO by default on 5780 class chips.

Disable TSO by default on 5780, 5714, and 5715 chips for now while we
investigate the reported tx timeouts by Juergen Kreileder.  Thanks to
Juergen for reporting the problem.

Update version to 3.60.

Signed-off-by: Michael Chan <mchan@broadcom.com>


diff --git a/drivers/net/tg3.c b/drivers/net/tg3.c
index 862c226..607d87e 100644
--- a/drivers/net/tg3.c
+++ b/drivers/net/tg3.c
@@ -69,8 +69,8 @@
 
 #define DRV_MODULE_NAME		"tg3"
 #define PFX DRV_MODULE_NAME	": "
-#define DRV_MODULE_VERSION	"3.59"
-#define DRV_MODULE_RELDATE	"June 8, 2006"
+#define DRV_MODULE_VERSION	"3.60"
+#define DRV_MODULE_RELDATE	"June 17, 2006"
 
 #define TG3_DEF_MAC_MODE	0
 #define TG3_DEF_RX_MODE		0
@@ -11385,7 +11385,11 @@ static int __devinit tg3_init_one(struct
 	 * Firmware TSO on older chips gives lower performance, so it
 	 * is off by default, but can be enabled using ethtool.
 	 */
-	if (tp->tg3_flags2 & TG3_FLG2_HW_TSO)
+	/* Disable TSO by default on all 5780 class chips because
+	 * of reported tx timeouts.
+	 */
+	if ((tp->tg3_flags2 & TG3_FLG2_HW_TSO) &&
+	   !(tp->tg3_flags2 & TG3_FLG2_5780_CLASS))
 		dev->features |= NETIF_F_TSO;
 
 #endif

Re: tg3 timeouts with 2.6.17-rc6

[David Miller]

From: "Michael Chan" <mchan@broadcom.com>
Date: Sat, 17 Jun 2006 11:23:28 -0700

> David, Here's the patch if you haven't already made one:
> 
> [TG3]: Disable TSO by default on 5780 class chips.

Sorry, I didn't get a chance to push this into 2.6.17
in time.  I will push it into the first 2.6.17.x -stable
release.

Re: tg3 timeouts with 2.6.17-rc6

[Michael M. Dwyer]

[-- Attachment #1: Type: text/plain, Size: 1302 bytes --]

> > David, Here's the patch if you haven't already made one:
> >
> > [TG3]: Disable TSO by default on 5780 class chips.
>
> Sorry, I didn't get a chance to push this into 2.6.17
> in time. I will push it into the first 2.6.17.x -stable
> release.

I just wanted to mention that I also have had this problem with tg3 on my Dell 
Optiplex 280.  Here's my dmesg info:

> tg3.c:v3.59 (June 8, 2006)
> ACPI: PCI Interrupt 0000:02:00.0[A] -> GSI 16 (level, low) -> IRQ 16
> PCI: Setting latency timer of device 0000:02:00.0 to 64
> eth0: Tigon3 [partno(BCM95751) rev 4001 PHY(5750)] (PCI Express)
> 10/100/1000BaseT E thernet 00:12:3f:4d:a5:bd
> eth0: RXcsums[1] LinkChgREG[1] MIirq[1] ASF[0] Split[0] WireSpeed[1]
> TSOcap[1] eth0: dma_rwctrl[76180000] dma_mask[64-bit]

I wasn't sure if BCM95751 was covered by "5780 class chips".

The problem didn't show up for >3 days after I upgraded to 2.6.17 
(Gentoo-sources, 2.6.17-r4 to be exact).  Problem was most evident during 
bittorrent download.  Setting TSO to off seems have solved it.  If there's 
any other information you'd like from me, please let me know.

Thanks,
Mike

-- 
Michael Dwyer, Programmer/Analyst II
Vice President for Research
Texas A&M University, College Station
mdwyer@vprmail.tamu.edu
979.845.3295

[-- Attachment #2: Type: application/pgp-signature, Size: 200 bytes --]

Re: tg3 timeouts with 2.6.17-rc6

[Michael Chan]

Michael M. Dwyer wrote:

> I wasn't sure if BCM95751 was covered by "5780 class chips".

No, it's not covered but it also has the same problem.

2.6.18 will have an elegant fix for this problem on all affected
chips without disabling TSO.  In the meantime, I will send another
-stable patch for 2.6.17 to disable TSO on other remaining chips
affected by this problem.

Thanks.