linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Ivan Vecera <ivecera@redhat.com>,
	Jakub Kicinski <kuba@kernel.org>, Sasha Levin <sashal@kernel.org>
Subject: [PATCH 5.16 26/37] iavf: Fix hang during reboot/shutdown
Date: Mon, 21 Mar 2022 14:53:08 +0100	[thread overview]
Message-ID: <20220321133222.053422632@linuxfoundation.org> (raw)
In-Reply-To: <20220321133221.290173884@linuxfoundation.org>

From: Ivan Vecera <ivecera@redhat.com>

[ Upstream commit b04683ff8f0823b869c219c78ba0d974bddea0b5 ]

Recent commit 974578017fc1 ("iavf: Add waiting so the port is
initialized in remove") adds a wait-loop at the beginning of
iavf_remove() to ensure that port initialization is finished
prior unregistering net device. This causes a regression
in reboot/shutdown scenario because in this case callback
iavf_shutdown() is called and this callback detaches the device,
makes it down if it is running and sets its state to __IAVF_REMOVE.
Later shutdown callback of associated PF driver (e.g. ice_shutdown)
is called. That callback calls among other things sriov_disable()
that calls indirectly iavf_remove() (see stack trace below).
As the adapter state is already __IAVF_REMOVE then the mentioned
loop is end-less and shutdown process hangs.

The patch fixes this by checking adapter's state at the beginning
of iavf_remove() and skips the rest of the function if the adapter
is already in remove state (shutdown is in progress).

Reproducer:
1. Create VF on PF driven by ice or i40e driver
2. Ensure that the VF is bound to iavf driver
3. Reboot

[52625.981294] sysrq: SysRq : Show Blocked State
[52625.988377] task:reboot          state:D stack:    0 pid:17359 ppid:     1 f2
[52625.996732] Call Trace:
[52625.999187]  __schedule+0x2d1/0x830
[52626.007400]  schedule+0x35/0xa0
[52626.010545]  schedule_hrtimeout_range_clock+0x83/0x100
[52626.020046]  usleep_range+0x5b/0x80
[52626.023540]  iavf_remove+0x63/0x5b0 [iavf]
[52626.027645]  pci_device_remove+0x3b/0xc0
[52626.031572]  device_release_driver_internal+0x103/0x1f0
[52626.036805]  pci_stop_bus_device+0x72/0xa0
[52626.040904]  pci_stop_and_remove_bus_device+0xe/0x20
[52626.045870]  pci_iov_remove_virtfn+0xba/0x120
[52626.050232]  sriov_disable+0x2f/0xe0
[52626.053813]  ice_free_vfs+0x7c/0x340 [ice]
[52626.057946]  ice_remove+0x220/0x240 [ice]
[52626.061967]  ice_shutdown+0x16/0x50 [ice]
[52626.065987]  pci_device_shutdown+0x34/0x60
[52626.070086]  device_shutdown+0x165/0x1c5
[52626.074011]  kernel_restart+0xe/0x30
[52626.077593]  __do_sys_reboot+0x1d2/0x210
[52626.093815]  do_syscall_64+0x5b/0x1a0
[52626.097483]  entry_SYSCALL_64_after_hwframe+0x65/0xca

Fixes: 974578017fc1 ("iavf: Add waiting so the port is initialized in remove")
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
Link: https://lore.kernel.org/r/20220317104524.2802848-1-ivecera@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index f2ce30366484..42e6675078cd 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -4028,6 +4028,13 @@ static void iavf_remove(struct pci_dev *pdev)
 	struct iavf_hw *hw = &adapter->hw;
 	int err;
 
+	/* When reboot/shutdown is in progress no need to do anything
+	 * as the adapter is already REMOVE state that was set during
+	 * iavf_shutdown() callback.
+	 */
+	if (adapter->state == __IAVF_REMOVE)
+		return;
+
 	set_bit(__IAVF_IN_REMOVE_TASK, &adapter->crit_section);
 	/* Wait until port initialization is complete.
 	 * There are flows where register/unregister netdev may race.
-- 
2.34.1




  parent reply	other threads:[~2022-03-21 14:15 UTC|newest]

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-21 13:52 [PATCH 5.16 00/37] 5.16.17-rc1 review Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 01/37] crypto: qcom-rng - ensure buffer for generate is completely filled Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 02/37] ocfs2: fix crash when initialize filecheck kobj fails Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 03/37] mm: swap: get rid of livelock in swapin readahead Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 04/37] block: release rq qos structures for queue without disk Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 05/37] drm/mgag200: Fix PLL setup for g200wb and g200ew Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 06/37] efi: fix return value of __setup handlers Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 07/37] alx: acquire mutex for alx_reinit in alx_change_mtu Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 08/37] vsock: each transport cycles only on its own sockets Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 09/37] esp6: fix check on ipv6_skip_exthdrs return value Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 10/37] net: phy: marvell: Fix invalid comparison in the resume and suspend functions Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 11/37] net/packet: fix slab-out-of-bounds access in packet_recvmsg() Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 12/37] nvmet: revert "nvmet: make discovery NQN configurable" Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 13/37] atm: eni: Add check for dma_map_single Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 14/37] ice: fix NULL pointer dereference in ice_update_vsi_tx_ring_stats() Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 15/37] iavf: Fix double free in iavf_reset_task Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 16/37] hv_netvsc: Add check for kvmalloc_array Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 17/37] drm/imx: parallel-display: Remove bus flags check in imx_pd_bridge_atomic_check() Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 18/37] drm/panel: simple: Fix Innolux G070Y2-L01 BPP settings Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 19/37] net: handle ARPHRD_PIMREG in dev_is_mac_header_xmit() Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 20/37] drm: Dont make DRM_PANEL_BRIDGE dependent on DRM_KMS_HELPERS Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 21/37] net: dsa: Add missing of_node_put() in dsa_port_parse_of Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 22/37] net: phy: mscc: Add MODULE_FIRMWARE macros Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 23/37] bnx2x: fix built-in kernel driver load failure Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 24/37] net: bcmgenet: skip invalid partial checksums Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 25/37] net: mscc: ocelot: fix backwards compatibility with single-chain tc-flower offload Greg Kroah-Hartman
2022-03-21 13:53 ` Greg Kroah-Hartman [this message]
2022-03-21 13:53 ` [PATCH 5.16 27/37] arm64: fix clang warning about TRAMP_VALIAS Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 28/37] usb: gadget: rndis: prevent integer overflow in rndis_set_response() Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 29/37] usb: gadget: Fix use-after-free bug by not setting udc->dev.driver Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 30/37] usb: usbtmc: Fix bug in pipe direction for control transfers Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 31/37] scsi: mpt3sas: Page fault in reply q processing Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 32/37] Input: aiptek - properly check endpoint type Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 33/37] arm64: errata: avoid duplicate field initializer Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 34/37] perf symbols: Fix symbol size calculation condition Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 35/37] Revert "arm64: dts: freescale: Fix interrupt-map parent address cells" Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 36/37] Revert "ath10k: drop beacon and probe response which leak from other channel" Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 37/37] btrfs: skip reserved bytes warning on unmount after log cleanup failure Greg Kroah-Hartman
2022-03-21 18:22 ` [PATCH 5.16 00/37] 5.16.17-rc1 review Florian Fainelli
2022-03-21 19:51 ` Jeffrin Thalakkottoor
2022-03-21 23:21 ` Shuah Khan
2022-03-21 23:28 ` Fox Chen
2022-03-22  1:53 ` Zan Aziz
2022-03-22  2:01 ` Guenter Roeck
2022-03-22  8:31 ` Ron Economos
2022-03-22  8:52 ` Naresh Kamboju
2022-03-22 11:23 ` Bagas Sanjaya

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220321133222.053422632@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=ivecera@redhat.com \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=sashal@kernel.org \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).