linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Ivan Vecera <ivecera@redhat.com>,
	Jakub Kicinski <kuba@kernel.org>, Sasha Levin <sashal@kernel.org>
Subject: [PATCH 5.15 24/32] iavf: Fix hang during reboot/shutdown
Date: Mon, 21 Mar 2022 14:53:00 +0100	[thread overview]
Message-ID: <20220321133221.262713778@linuxfoundation.org> (raw)
In-Reply-To: <20220321133220.559554263@linuxfoundation.org>

From: Ivan Vecera <ivecera@redhat.com>

[ Upstream commit b04683ff8f0823b869c219c78ba0d974bddea0b5 ]

Recent commit 974578017fc1 ("iavf: Add waiting so the port is
initialized in remove") adds a wait-loop at the beginning of
iavf_remove() to ensure that port initialization is finished
prior unregistering net device. This causes a regression
in reboot/shutdown scenario because in this case callback
iavf_shutdown() is called and this callback detaches the device,
makes it down if it is running and sets its state to __IAVF_REMOVE.
Later shutdown callback of associated PF driver (e.g. ice_shutdown)
is called. That callback calls among other things sriov_disable()
that calls indirectly iavf_remove() (see stack trace below).
As the adapter state is already __IAVF_REMOVE then the mentioned
loop is end-less and shutdown process hangs.

The patch fixes this by checking adapter's state at the beginning
of iavf_remove() and skips the rest of the function if the adapter
is already in remove state (shutdown is in progress).

Reproducer:
1. Create VF on PF driven by ice or i40e driver
2. Ensure that the VF is bound to iavf driver
3. Reboot

[52625.981294] sysrq: SysRq : Show Blocked State
[52625.988377] task:reboot          state:D stack:    0 pid:17359 ppid:     1 f2
[52625.996732] Call Trace:
[52625.999187]  __schedule+0x2d1/0x830
[52626.007400]  schedule+0x35/0xa0
[52626.010545]  schedule_hrtimeout_range_clock+0x83/0x100
[52626.020046]  usleep_range+0x5b/0x80
[52626.023540]  iavf_remove+0x63/0x5b0 [iavf]
[52626.027645]  pci_device_remove+0x3b/0xc0
[52626.031572]  device_release_driver_internal+0x103/0x1f0
[52626.036805]  pci_stop_bus_device+0x72/0xa0
[52626.040904]  pci_stop_and_remove_bus_device+0xe/0x20
[52626.045870]  pci_iov_remove_virtfn+0xba/0x120
[52626.050232]  sriov_disable+0x2f/0xe0
[52626.053813]  ice_free_vfs+0x7c/0x340 [ice]
[52626.057946]  ice_remove+0x220/0x240 [ice]
[52626.061967]  ice_shutdown+0x16/0x50 [ice]
[52626.065987]  pci_device_shutdown+0x34/0x60
[52626.070086]  device_shutdown+0x165/0x1c5
[52626.074011]  kernel_restart+0xe/0x30
[52626.077593]  __do_sys_reboot+0x1d2/0x210
[52626.093815]  do_syscall_64+0x5b/0x1a0
[52626.097483]  entry_SYSCALL_64_after_hwframe+0x65/0xca

Fixes: 974578017fc1 ("iavf: Add waiting so the port is initialized in remove")
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
Link: https://lore.kernel.org/r/20220317104524.2802848-1-ivecera@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index 7fca9dd8dcf6..ca74824d40b8 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -4025,6 +4025,13 @@ static void iavf_remove(struct pci_dev *pdev)
 	struct iavf_hw *hw = &adapter->hw;
 	int err;
 
+	/* When reboot/shutdown is in progress no need to do anything
+	 * as the adapter is already REMOVE state that was set during
+	 * iavf_shutdown() callback.
+	 */
+	if (adapter->state == __IAVF_REMOVE)
+		return;
+
 	set_bit(__IAVF_IN_REMOVE_TASK, &adapter->crit_section);
 	/* Wait until port initialization is complete.
 	 * There are flows where register/unregister netdev may race.
-- 
2.34.1




  parent reply	other threads:[~2022-03-21 14:11 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-21 13:52 [PATCH 5.15 00/32] 5.15.31-rc1 review Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 01/32] crypto: qcom-rng - ensure buffer for generate is completely filled Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 02/32] ocfs2: fix crash when initialize filecheck kobj fails Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 03/32] mm: swap: get rid of livelock in swapin readahead Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 04/32] block: release rq qos structures for queue without disk Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 05/32] drm/mgag200: Fix PLL setup for g200wb and g200ew Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 06/32] efi: fix return value of __setup handlers Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 07/32] alx: acquire mutex for alx_reinit in alx_change_mtu Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 08/32] vsock: each transport cycles only on its own sockets Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 09/32] esp6: fix check on ipv6_skip_exthdrs return value Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 10/32] net: phy: marvell: Fix invalid comparison in the resume and suspend functions Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 11/32] net/packet: fix slab-out-of-bounds access in packet_recvmsg() Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 12/32] atm: eni: Add check for dma_map_single Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 13/32] iavf: Fix double free in iavf_reset_task Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 14/32] hv_netvsc: Add check for kvmalloc_array Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 15/32] drm/imx: parallel-display: Remove bus flags check in imx_pd_bridge_atomic_check() Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 16/32] drm/panel: simple: Fix Innolux G070Y2-L01 BPP settings Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 17/32] net: handle ARPHRD_PIMREG in dev_is_mac_header_xmit() Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 18/32] drm: Dont make DRM_PANEL_BRIDGE dependent on DRM_KMS_HELPERS Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 19/32] net: dsa: Add missing of_node_put() in dsa_port_parse_of Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 20/32] net: phy: mscc: Add MODULE_FIRMWARE macros Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 21/32] bnx2x: fix built-in kernel driver load failure Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 22/32] net: bcmgenet: skip invalid partial checksums Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 23/32] net: mscc: ocelot: fix backwards compatibility with single-chain tc-flower offload Greg Kroah-Hartman
2022-03-21 13:53 ` Greg Kroah-Hartman [this message]
2022-03-21 13:53 ` [PATCH 5.15 25/32] arm64: fix clang warning about TRAMP_VALIAS Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 26/32] usb: gadget: rndis: prevent integer overflow in rndis_set_response() Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 27/32] usb: gadget: Fix use-after-free bug by not setting udc->dev.driver Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 28/32] usb: usbtmc: Fix bug in pipe direction for control transfers Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 29/32] scsi: mpt3sas: Page fault in reply q processing Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 30/32] Input: aiptek - properly check endpoint type Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 31/32] perf symbols: Fix symbol size calculation condition Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 32/32] btrfs: skip reserved bytes warning on unmount after log cleanup failure Greg Kroah-Hartman
2022-03-21 18:42 ` [PATCH 5.15 00/32] 5.15.31-rc1 review Florian Fainelli
2022-03-21 22:48 ` Ron Economos
2022-03-21 23:22 ` Shuah Khan
2022-03-22  1:34 ` Fox Chen
2022-03-22  2:00 ` Guenter Roeck
2022-03-22  8:45 ` Bagas Sanjaya
2022-03-22 10:37 ` Naresh Kamboju
2022-03-22 12:49 ` Sudip Mukherjee

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220321133221.262713778@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=ivecera@redhat.com \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=sashal@kernel.org \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).