All of lore.kernel.org
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Ivan Vecera <ivecera@redhat.com>,
	Jakub Kicinski <kuba@kernel.org>, Sasha Levin <sashal@kernel.org>
Subject: [PATCH 5.15 24/32] iavf: Fix hang during reboot/shutdown
Date: Mon, 21 Mar 2022 14:53:00 +0100	[thread overview]
Message-ID: <20220321133221.262713778@linuxfoundation.org> (raw)
In-Reply-To: <20220321133220.559554263@linuxfoundation.org>

From: Ivan Vecera <ivecera@redhat.com>

[ Upstream commit b04683ff8f0823b869c219c78ba0d974bddea0b5 ]

Recent commit 974578017fc1 ("iavf: Add waiting so the port is
initialized in remove") adds a wait-loop at the beginning of
iavf_remove() to ensure that port initialization is finished
prior unregistering net device. This causes a regression
in reboot/shutdown scenario because in this case callback
iavf_shutdown() is called and this callback detaches the device,
makes it down if it is running and sets its state to __IAVF_REMOVE.
Later shutdown callback of associated PF driver (e.g. ice_shutdown)
is called. That callback calls among other things sriov_disable()
that calls indirectly iavf_remove() (see stack trace below).
As the adapter state is already __IAVF_REMOVE then the mentioned
loop is end-less and shutdown process hangs.

The patch fixes this by checking adapter's state at the beginning
of iavf_remove() and skips the rest of the function if the adapter
is already in remove state (shutdown is in progress).

Reproducer:
1. Create VF on PF driven by ice or i40e driver
2. Ensure that the VF is bound to iavf driver
3. Reboot

[52625.981294] sysrq: SysRq : Show Blocked State
[52625.988377] task:reboot          state:D stack:    0 pid:17359 ppid:     1 f2
[52625.996732] Call Trace:
[52625.999187]  __schedule+0x2d1/0x830
[52626.007400]  schedule+0x35/0xa0
[52626.010545]  schedule_hrtimeout_range_clock+0x83/0x100
[52626.020046]  usleep_range+0x5b/0x80
[52626.023540]  iavf_remove+0x63/0x5b0 [iavf]
[52626.027645]  pci_device_remove+0x3b/0xc0
[52626.031572]  device_release_driver_internal+0x103/0x1f0
[52626.036805]  pci_stop_bus_device+0x72/0xa0
[52626.040904]  pci_stop_and_remove_bus_device+0xe/0x20
[52626.045870]  pci_iov_remove_virtfn+0xba/0x120
[52626.050232]  sriov_disable+0x2f/0xe0
[52626.053813]  ice_free_vfs+0x7c/0x340 [ice]
[52626.057946]  ice_remove+0x220/0x240 [ice]
[52626.061967]  ice_shutdown+0x16/0x50 [ice]
[52626.065987]  pci_device_shutdown+0x34/0x60
[52626.070086]  device_shutdown+0x165/0x1c5
[52626.074011]  kernel_restart+0xe/0x30
[52626.077593]  __do_sys_reboot+0x1d2/0x210
[52626.093815]  do_syscall_64+0x5b/0x1a0
[52626.097483]  entry_SYSCALL_64_after_hwframe+0x65/0xca

Fixes: 974578017fc1 ("iavf: Add waiting so the port is initialized in remove")
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
Link: https://lore.kernel.org/r/20220317104524.2802848-1-ivecera@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/net/ethernet/intel/iavf/iavf_main.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index 7fca9dd8dcf6..ca74824d40b8 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -4025,6 +4025,13 @@ static void iavf_remove(struct pci_dev *pdev)
 	struct iavf_hw *hw = &adapter->hw;
 	int err;
 
+	/* When reboot/shutdown is in progress no need to do anything
+	 * as the adapter is already REMOVE state that was set during
+	 * iavf_shutdown() callback.
+	 */
+	if (adapter->state == __IAVF_REMOVE)
+		return;
+
 	set_bit(__IAVF_IN_REMOVE_TASK, &adapter->crit_section);
 	/* Wait until port initialization is complete.
 	 * There are flows where register/unregister netdev may race.
-- 
2.34.1




  parent reply	other threads:[~2022-03-21 14:11 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-21 13:52 [PATCH 5.15 00/32] 5.15.31-rc1 review Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 01/32] crypto: qcom-rng - ensure buffer for generate is completely filled Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 02/32] ocfs2: fix crash when initialize filecheck kobj fails Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 03/32] mm: swap: get rid of livelock in swapin readahead Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 04/32] block: release rq qos structures for queue without disk Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 05/32] drm/mgag200: Fix PLL setup for g200wb and g200ew Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 06/32] efi: fix return value of __setup handlers Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 07/32] alx: acquire mutex for alx_reinit in alx_change_mtu Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 08/32] vsock: each transport cycles only on its own sockets Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 09/32] esp6: fix check on ipv6_skip_exthdrs return value Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 10/32] net: phy: marvell: Fix invalid comparison in the resume and suspend functions Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 11/32] net/packet: fix slab-out-of-bounds access in packet_recvmsg() Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 12/32] atm: eni: Add check for dma_map_single Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 13/32] iavf: Fix double free in iavf_reset_task Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 14/32] hv_netvsc: Add check for kvmalloc_array Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 15/32] drm/imx: parallel-display: Remove bus flags check in imx_pd_bridge_atomic_check() Greg Kroah-Hartman
2022-03-21 13:52   ` Greg Kroah-Hartman
2022-03-21 13:52   ` Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 16/32] drm/panel: simple: Fix Innolux G070Y2-L01 BPP settings Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 17/32] net: handle ARPHRD_PIMREG in dev_is_mac_header_xmit() Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 18/32] drm: Dont make DRM_PANEL_BRIDGE dependent on DRM_KMS_HELPERS Greg Kroah-Hartman
2022-03-21 13:52   ` Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 19/32] net: dsa: Add missing of_node_put() in dsa_port_parse_of Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 20/32] net: phy: mscc: Add MODULE_FIRMWARE macros Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 21/32] bnx2x: fix built-in kernel driver load failure Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 22/32] net: bcmgenet: skip invalid partial checksums Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.15 23/32] net: mscc: ocelot: fix backwards compatibility with single-chain tc-flower offload Greg Kroah-Hartman
2022-03-21 13:53 ` Greg Kroah-Hartman [this message]
2022-03-21 13:53 ` [PATCH 5.15 25/32] arm64: fix clang warning about TRAMP_VALIAS Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 26/32] usb: gadget: rndis: prevent integer overflow in rndis_set_response() Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 27/32] usb: gadget: Fix use-after-free bug by not setting udc->dev.driver Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 28/32] usb: usbtmc: Fix bug in pipe direction for control transfers Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 29/32] scsi: mpt3sas: Page fault in reply q processing Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 30/32] Input: aiptek - properly check endpoint type Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 31/32] perf symbols: Fix symbol size calculation condition Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.15 32/32] btrfs: skip reserved bytes warning on unmount after log cleanup failure Greg Kroah-Hartman
2022-03-21 18:42 ` [PATCH 5.15 00/32] 5.15.31-rc1 review Florian Fainelli
2022-03-21 19:08 ` Jon Hunter
2022-03-21 22:48 ` Ron Economos
2022-03-21 23:22 ` Shuah Khan
2022-03-22  1:34 ` Fox Chen
2022-03-22  2:00 ` Guenter Roeck
2022-03-22  8:45 ` Bagas Sanjaya
2022-03-22 10:37 ` Naresh Kamboju
2022-03-22 12:49 ` Sudip Mukherjee

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220321133221.262713778@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=ivecera@redhat.com \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=sashal@kernel.org \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.