* [Intel-wired-lan] ixgbe Tx Unit Hangs seen when l2-fwd-offload is enabled
@ 2016-05-03 17:58 Samudrala, Sridhar
2016-05-03 20:34 ` Jeff Kirsher
0 siblings, 1 reply; 4+ messages in thread
From: Samudrala, Sridhar @ 2016-05-03 17:58 UTC (permalink / raw)
To: intel-wired-lan
With the latest davem's next-next or Jeff's dev-queue branch (4.6.0-rc5+), I am seeing Tx Unit Hangs when
using offloaded macvlans.
Here is a simple sequence of commands that can reproduce this issue.
ethtool -K p4p1 l2-fwd-offload on
ip link add link p4p1 name mvlan_1 type macvlan
ip link set mvlan_1 up
Here is the kernel log when this happens.
May 03 10:45:01 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
May 03 10:45:01 sri02 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): p4p1: link becomes ready
May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0: removed PHC on p4p1
May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0: Multiqueue Enabled: Rx Queue count = 2, Tx Queue count = 2
May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not enable Tx Queue 0
May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not enable Tx Queue 2
May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0: registered PHC device on p4p1
May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: detected SFP+: 5
May 03 10:45:31 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
May 03 10:45:33 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Detected Tx Unit Hang
Tx Queue <0>
TDH, TDT <0>, <4>
next_to_use <4>
next_to_clean <0>
tx_buffer_info[next_to_clean]
time_stamp <10005560a>
jiffies <100055df4>
May 03 10:45:33 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: tx hang 1 detected on queue 0, resetting adapter
May 03 10:45:33 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: initiating reset due to tx timeout
May 03 10:45:33 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Reset adapter
May 03 10:45:35 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not enable Tx Queue 0
May 03 10:45:35 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not enable Tx Queue 2
May 03 10:45:35 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: detected SFP+: 5
May 03 10:45:36 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
May 03 10:45:38 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Detected Tx Unit Hang
Tx Queue <0>
TDH, TDT <0>, <2>
next_to_use <2>
next_to_clean <0>
tx_buffer_info[next_to_clean]
time_stamp <1000569a0>
jiffies <1000571b0>
May 03 10:45:38 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: tx hang 2 detected on queue 0, resetting adapter
May 03 10:45:38 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: initiating reset due to tx timeout
May 03 10:45:38 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Reset adapter
May 03 10:45:40 sri02 kernel: Virtual device mvlan_1 asks to queue packet!
May 03 10:45:40 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not enable Tx Queue 0
May 03 10:45:40 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not enable Tx Queue 2
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20160503/6f45bdb4/attachment-0001.html>
^ permalink raw reply [flat|nested] 4+ messages in thread
* [Intel-wired-lan] ixgbe Tx Unit Hangs seen when l2-fwd-offload is enabled
2016-05-03 17:58 [Intel-wired-lan] ixgbe Tx Unit Hangs seen when l2-fwd-offload is enabled Samudrala, Sridhar
@ 2016-05-03 20:34 ` Jeff Kirsher
2016-05-03 21:13 ` Bowers, AndrewX
0 siblings, 1 reply; 4+ messages in thread
From: Jeff Kirsher @ 2016-05-03 20:34 UTC (permalink / raw)
To: intel-wired-lan
On Tue, 2016-05-03 at 10:58 -0700, Samudrala, Sridhar wrote:
> With the latest davem's next-next or Jeff's dev-queue branch (4.6.0-
> rc5+), I am seeing Tx Unit Hangs when
> using offloaded macvlans.
>
> Here is a simple sequence of commands that can reproduce this issue.
> ethtool -K p4p1 l2-fwd-offload on
> ip link add link p4p1 name mvlan_1 type macvlan
> ip link set mvlan_1 up
>
> Here is the kernel log when this happens.
>
> May 03 10:45:01 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: NIC Link is Up
> 10 Gbps, Flow Control: RX/TX
> May 03 10:45:01 sri02 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): p4p1:
> link becomes ready
> May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0: removed PHC on p4p1
> May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0: Multiqueue Enabled:
> Rx Queue count = 2, Tx Queue count = 2
> May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not
> enable Tx Queue 0
> May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not
> enable Tx Queue 2
> May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0: registered PHC
> device on p4p1
> May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: detected SFP+:
> 5
> May 03 10:45:31 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: NIC Link is Up
> 10 Gbps, Flow Control: RX/TX
> May 03 10:45:33 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Detected Tx
> Unit Hang
> ????????????????????????????????Tx Queue?????????????<0>
> ????????????????????????????????TDH, TDT?????????????<0>, <4>
> ????????????????????????????????next_to_use??????????<4>
> ????????????????????????????????next_to_clean????????<0>
> ??????????????????????????????tx_buffer_info[next_to_clean]
> ????????????????????????????????time_stamp???????????<10005560a>
> ????????????????????????????????jiffies??????????????<100055df4>
> May 03 10:45:33 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: tx hang 1
> detected on queue 0, resetting adapter
> May 03 10:45:33 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: initiating
> reset due to tx timeout
> May 03 10:45:33 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Reset adapter
> May 03 10:45:35 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not
> enable Tx Queue 0
> May 03 10:45:35 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not
> enable Tx Queue 2
> May 03 10:45:35 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: detected SFP+:
> 5
> May 03 10:45:36 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: NIC Link is Up
> 10 Gbps, Flow Control: RX/TX
> May 03 10:45:38 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Detected Tx
> Unit Hang
> ????????????????????????????????Tx Queue?????????????<0>
> ????????????????????????????????TDH, TDT?????????????<0>, <2>
> ????????????????????????????????next_to_use??????????<2>
> ????????????????????????????????next_to_clean????????<0>
> ??????????????????????????????tx_buffer_info[next_to_clean]
> ????????????????????????????????time_stamp???????????<1000569a0>
> ????????????????????????????????jiffies??????????????<1000571b0>
> May 03 10:45:38 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: tx hang 2
> detected on queue 0, resetting adapter
> May 03 10:45:38 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: initiating
> reset due to tx timeout
> May 03 10:45:38 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Reset adapter
> May 03 10:45:40 sri02 kernel: Virtual device mvlan_1 asks to queue
> packet!
> May 03 10:45:40 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not
> enable Tx Queue 0
> May 03 10:45:40 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not
> enable Tx Queue 2
I just updated my tree with Dave's latest just now, so if you want to
re-pull and see if it still exists, please.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20160503/cc1cbdc6/attachment.asc>
^ permalink raw reply [flat|nested] 4+ messages in thread
* [Intel-wired-lan] ixgbe Tx Unit Hangs seen when l2-fwd-offload is enabled
2016-05-03 20:34 ` Jeff Kirsher
@ 2016-05-03 21:13 ` Bowers, AndrewX
2016-05-03 23:16 ` Samudrala, Sridhar
0 siblings, 1 reply; 4+ messages in thread
From: Bowers, AndrewX @ 2016-05-03 21:13 UTC (permalink / raw)
To: intel-wired-lan
It still does it even with the latest tree pulled, however if I create the macvlan, bring it up, and THEN enable l2-fwd-offload, I don't get the tx hang. If I try to create another macvlan with l2-fwd-offload enabled, I get the tx hang.
> -----Original Message-----
> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
> Behalf Of Jeff Kirsher
> Sent: Tuesday, May 03, 2016 1:35 PM
> To: Samudrala, Sridhar <sridhar.samudrala@intel.com>; intel-wired-
> lan at lists.osuosl.org
> Subject: Re: [Intel-wired-lan] ixgbe Tx Unit Hangs seen when l2-fwd-offload
> is enabled
>
> On Tue, 2016-05-03 at 10:58 -0700, Samudrala, Sridhar wrote:
> > With the latest davem's next-next or Jeff's dev-queue branch (4.6.0-
> > rc5+), I am seeing Tx Unit Hangs when
> > using offloaded macvlans.
> >
> > Here is a simple sequence of commands that can reproduce this issue.
> > ethtool -K p4p1 l2-fwd-offload on
> > ip link add link p4p1 name mvlan_1 type macvlan
> > ip link set mvlan_1 up
> >
> > Here is the kernel log when this happens.
> >
> > May 03 10:45:01 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: NIC Link is Up
> > 10 Gbps, Flow Control: RX/TX
> > May 03 10:45:01 sri02 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): p4p1:
> > link becomes ready
> > May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0: removed PHC on p4p1
> > May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0: Multiqueue Enabled:
> > Rx Queue count = 2, Tx Queue count = 2 May 03 10:45:30 sri02 kernel:
> > ixgbe 0000:1f:00.0 p4p1: Could not enable Tx Queue 0 May 03 10:45:30
> > sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not enable Tx Queue 2 May
> > 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0: registered PHC device on
> > p4p1 May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: detected
> > SFP+:
> > 5
> > May 03 10:45:31 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: NIC Link is Up
> > 10 Gbps, Flow Control: RX/TX
> > May 03 10:45:33 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Detected Tx
> > Unit Hang
> > ????????????????????????????????Tx Queue?????????????<0>
> > ????????????????????????????????TDH, TDT?????????????<0>, <4>
> > ????????????????????????????????next_to_use??????????<4>
> > ????????????????????????????????next_to_clean????????<0>
> > ??????????????????????????????tx_buffer_info[next_to_clean]
> > ????????????????????????????????time_stamp???????????<10005560a>
> > ????????????????????????????????jiffies??????????????<100055df4> May
> > 03 10:45:33 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: tx hang 1 detected
> > on queue 0, resetting adapter May 03 10:45:33 sri02 kernel: ixgbe
> > 0000:1f:00.0 p4p1: initiating reset due to tx timeout May 03 10:45:33
> > sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Reset adapter May 03 10:45:35
> > sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not enable Tx Queue 0 May
> > 03 10:45:35 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not enable Tx
> > Queue 2 May 03 10:45:35 sri02 kernel: ixgbe 0000:1f:00.0 p4p1:
> > detected SFP+:
> > 5
> > May 03 10:45:36 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: NIC Link is Up
> > 10 Gbps, Flow Control: RX/TX
> > May 03 10:45:38 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Detected Tx
> > Unit Hang
> > ????????????????????????????????Tx Queue?????????????<0>
> > ????????????????????????????????TDH, TDT?????????????<0>, <2>
> > ????????????????????????????????next_to_use??????????<2>
> > ????????????????????????????????next_to_clean????????<0>
> > ??????????????????????????????tx_buffer_info[next_to_clean]
> > ????????????????????????????????time_stamp???????????<1000569a0>
> > ????????????????????????????????jiffies??????????????<1000571b0> May
> > 03 10:45:38 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: tx hang 2 detected
> > on queue 0, resetting adapter May 03 10:45:38 sri02 kernel: ixgbe
> > 0000:1f:00.0 p4p1: initiating reset due to tx timeout May 03 10:45:38
> > sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Reset adapter May 03 10:45:40
> > sri02 kernel: Virtual device mvlan_1 asks to queue packet!
> > May 03 10:45:40 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not
> > enable Tx Queue 0 May 03 10:45:40 sri02 kernel: ixgbe 0000:1f:00.0
> > p4p1: Could not enable Tx Queue 2
>
> I just updated my tree with Dave's latest just now, so if you want to re-pull
> and see if it still exists, please.
^ permalink raw reply [flat|nested] 4+ messages in thread
* [Intel-wired-lan] ixgbe Tx Unit Hangs seen when l2-fwd-offload is enabled
2016-05-03 21:13 ` Bowers, AndrewX
@ 2016-05-03 23:16 ` Samudrala, Sridhar
0 siblings, 0 replies; 4+ messages in thread
From: Samudrala, Sridhar @ 2016-05-03 23:16 UTC (permalink / raw)
To: intel-wired-lan
On 5/3/2016 2:13 PM, Bowers, AndrewX wrote:
> It still does it even with the latest tree pulled, however if I create the macvlan, bring it up, and THEN enable l2-fwd-offload, I don't get the tx hang. If I try to create another macvlan with l2-fwd-offload enabled, I get the tx hang.
Yes. I too see this issue even with the latest tree.
I remember not seeing this issue with 4.6.0-rc1 based tree.
>> -----Original Message-----
>> From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On
>> Behalf Of Jeff Kirsher
>> Sent: Tuesday, May 03, 2016 1:35 PM
>> To: Samudrala, Sridhar <sridhar.samudrala@intel.com>; intel-wired-
>> lan at lists.osuosl.org
>> Subject: Re: [Intel-wired-lan] ixgbe Tx Unit Hangs seen when l2-fwd-offload
>> is enabled
>>
>> On Tue, 2016-05-03 at 10:58 -0700, Samudrala, Sridhar wrote:
>>> With the latest davem's next-next or Jeff's dev-queue branch (4.6.0-
>>> rc5+), I am seeing Tx Unit Hangs when
>>> using offloaded macvlans.
>>>
>>> Here is a simple sequence of commands that can reproduce this issue.
>>> ethtool -K p4p1 l2-fwd-offload on
>>> ip link add link p4p1 name mvlan_1 type macvlan
>>> ip link set mvlan_1 up
>>>
>>> Here is the kernel log when this happens.
>>>
>>> May 03 10:45:01 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: NIC Link is Up
>>> 10 Gbps, Flow Control: RX/TX
>>> May 03 10:45:01 sri02 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): p4p1:
>>> link becomes ready
>>> May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0: removed PHC on p4p1
>>> May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0: Multiqueue Enabled:
>>> Rx Queue count = 2, Tx Queue count = 2 May 03 10:45:30 sri02 kernel:
>>> ixgbe 0000:1f:00.0 p4p1: Could not enable Tx Queue 0 May 03 10:45:30
>>> sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not enable Tx Queue 2 May
>>> 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0: registered PHC device on
>>> p4p1 May 03 10:45:30 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: detected
>>> SFP+:
>>> 5
>>> May 03 10:45:31 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: NIC Link is Up
>>> 10 Gbps, Flow Control: RX/TX
>>> May 03 10:45:33 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Detected Tx
>>> Unit Hang
>>> Tx Queue <0>
>>> TDH, TDT <0>, <4>
>>> next_to_use <4>
>>> next_to_clean <0>
>>> tx_buffer_info[next_to_clean]
>>> time_stamp <10005560a>
>>> jiffies <100055df4> May
>>> 03 10:45:33 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: tx hang 1 detected
>>> on queue 0, resetting adapter May 03 10:45:33 sri02 kernel: ixgbe
>>> 0000:1f:00.0 p4p1: initiating reset due to tx timeout May 03 10:45:33
>>> sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Reset adapter May 03 10:45:35
>>> sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not enable Tx Queue 0 May
>>> 03 10:45:35 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not enable Tx
>>> Queue 2 May 03 10:45:35 sri02 kernel: ixgbe 0000:1f:00.0 p4p1:
>>> detected SFP+:
>>> 5
>>> May 03 10:45:36 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: NIC Link is Up
>>> 10 Gbps, Flow Control: RX/TX
>>> May 03 10:45:38 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Detected Tx
>>> Unit Hang
>>> Tx Queue <0>
>>> TDH, TDT <0>, <2>
>>> next_to_use <2>
>>> next_to_clean <0>
>>> tx_buffer_info[next_to_clean]
>>> time_stamp <1000569a0>
>>> jiffies <1000571b0> May
>>> 03 10:45:38 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: tx hang 2 detected
>>> on queue 0, resetting adapter May 03 10:45:38 sri02 kernel: ixgbe
>>> 0000:1f:00.0 p4p1: initiating reset due to tx timeout May 03 10:45:38
>>> sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Reset adapter May 03 10:45:40
>>> sri02 kernel: Virtual device mvlan_1 asks to queue packet!
>>> May 03 10:45:40 sri02 kernel: ixgbe 0000:1f:00.0 p4p1: Could not
>>> enable Tx Queue 0 May 03 10:45:40 sri02 kernel: ixgbe 0000:1f:00.0
>>> p4p1: Could not enable Tx Queue 2
>> I just updated my tree with Dave's latest just now, so if you want to re-pull
>> and see if it still exists, please.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2016-05-03 23:16 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-05-03 17:58 [Intel-wired-lan] ixgbe Tx Unit Hangs seen when l2-fwd-offload is enabled Samudrala, Sridhar
2016-05-03 20:34 ` Jeff Kirsher
2016-05-03 21:13 ` Bowers, AndrewX
2016-05-03 23:16 ` Samudrala, Sridhar
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox