From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Nikita V. Shirokov" Subject: Re: ixgbe hangs when XDP_TX is enabled Date: Tue, 21 Aug 2018 09:58:58 -0700 Message-ID: <20180821165858.GA1507@maindev> References: <20180820193108.GA6390@maindev> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: netdev@vger.kernel.org, jeffrey.t.kirsher@intel.com To: Alexander Duyck Return-path: Received: from mail-pg1-f196.google.com ([209.85.215.196]:40455 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726780AbeHUUT5 (ORCPT ); Tue, 21 Aug 2018 16:19:57 -0400 Received: by mail-pg1-f196.google.com with SMTP id z25-v6so4033681pgu.7 for ; Tue, 21 Aug 2018 09:59:01 -0700 (PDT) Content-Disposition: inline In-Reply-To: Sender: netdev-owner@vger.kernel.org List-ID: On Tue, Aug 21, 2018 at 08:58:15AM -0700, Alexander Duyck wrote: > On Mon, Aug 20, 2018 at 12:32 PM Nikita V. Shirokov wrote: > > > > we are getting such errors: > > > > [ 408.737313] ixgbe 0000:03:00.0 eth0: Detected Tx Unit Hang (XDP) > > Tx Queue <46> > > TDH, TDT <0>, <2> > > next_to_use <2> > > next_to_clean <0> > > tx_buffer_info[next_to_clean] > > time_stamp <0> > > jiffies <1000197c0> > > [ 408.804438] ixgbe 0000:03:00.0 eth0: tx hang 1 detected on queue 46, resetting adapter > > [ 408.804440] ixgbe 0000:03:00.0 eth0: initiating reset due to tx timeout > > [ 408.817679] ixgbe 0000:03:00.0 eth0: Reset adapter > > [ 408.866091] ixgbe 0000:03:00.0 eth0: TXDCTL.ENABLE for one or more queues not cleared within the polling period > > [ 409.345289] ixgbe 0000:03:00.0 eth0: detected SFP+: 3 > > [ 409.497232] ixgbe 0000:03:00.0 eth0: NIC Link is Up 10 Gbps, Flow Control: RX/TX > > > > while running XDP prog on ixgbe nic. > > right now i'm seing this on bpfnext kernel > > (latest commit from Wed Aug 15 15:04:25 2018 -0700 ; > > 9a76aba02a37718242d7cdc294f0a3901928aa57) > > > > looks like this is the same issue as reported by Brenden in > > https://www.spinics.net/lists/netdev/msg439438.html > > > > -- > > Nikita V. Shirokov > > Could you provide some additional information about your setup. > Specifically useful would be "ethtool -i", "ethtool -l", and lspci > -vvv info for your device. The total number of CPUs on the system > would be useful to know as well. In addition could you try > reproducing sure: ethtool -l eth0 Channel parameters for eth0: Pre-set maximums: RX: 0 TX: 0 Other: 1 Combined: 63 Current hardware settings: RX: 0 TX: 0 Other: 1 Combined: 48 # ethtool -i eth0 driver: ixgbe version: 5.1.0-k firmware-version: 0x800006f1 expansion-rom-version: bus-info: 0000:03:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: yes # nproc 48 lspci: 03:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) Subsystem: Intel Corporation Device 000d Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- the issue with one of the sample XDP programs provided with the kernel > such as the xdp2 which I believe uses the XDP_TX function. We need to > try and create a similar setup in our own environment for > reproduction and debugging. will try but this could take a while, because i'm not sure that we have ixgbe in our test lab (and it would be hard to run such test in prod) > > Thanks. > > - Alex