From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, Ivan Vecera <ivecera@redhat.com>,
Jakub Kicinski <kuba@kernel.org>, Sasha Levin <sashal@kernel.org>
Subject: [PATCH 5.16 26/37] iavf: Fix hang during reboot/shutdown
Date: Mon, 21 Mar 2022 14:53:08 +0100 [thread overview]
Message-ID: <20220321133222.053422632@linuxfoundation.org> (raw)
In-Reply-To: <20220321133221.290173884@linuxfoundation.org>
From: Ivan Vecera <ivecera@redhat.com>
[ Upstream commit b04683ff8f0823b869c219c78ba0d974bddea0b5 ]
Recent commit 974578017fc1 ("iavf: Add waiting so the port is
initialized in remove") adds a wait-loop at the beginning of
iavf_remove() to ensure that port initialization is finished
prior unregistering net device. This causes a regression
in reboot/shutdown scenario because in this case callback
iavf_shutdown() is called and this callback detaches the device,
makes it down if it is running and sets its state to __IAVF_REMOVE.
Later shutdown callback of associated PF driver (e.g. ice_shutdown)
is called. That callback calls among other things sriov_disable()
that calls indirectly iavf_remove() (see stack trace below).
As the adapter state is already __IAVF_REMOVE then the mentioned
loop is end-less and shutdown process hangs.
The patch fixes this by checking adapter's state at the beginning
of iavf_remove() and skips the rest of the function if the adapter
is already in remove state (shutdown is in progress).
Reproducer:
1. Create VF on PF driven by ice or i40e driver
2. Ensure that the VF is bound to iavf driver
3. Reboot
[52625.981294] sysrq: SysRq : Show Blocked State
[52625.988377] task:reboot state:D stack: 0 pid:17359 ppid: 1 f2
[52625.996732] Call Trace:
[52625.999187] __schedule+0x2d1/0x830
[52626.007400] schedule+0x35/0xa0
[52626.010545] schedule_hrtimeout_range_clock+0x83/0x100
[52626.020046] usleep_range+0x5b/0x80
[52626.023540] iavf_remove+0x63/0x5b0 [iavf]
[52626.027645] pci_device_remove+0x3b/0xc0
[52626.031572] device_release_driver_internal+0x103/0x1f0
[52626.036805] pci_stop_bus_device+0x72/0xa0
[52626.040904] pci_stop_and_remove_bus_device+0xe/0x20
[52626.045870] pci_iov_remove_virtfn+0xba/0x120
[52626.050232] sriov_disable+0x2f/0xe0
[52626.053813] ice_free_vfs+0x7c/0x340 [ice]
[52626.057946] ice_remove+0x220/0x240 [ice]
[52626.061967] ice_shutdown+0x16/0x50 [ice]
[52626.065987] pci_device_shutdown+0x34/0x60
[52626.070086] device_shutdown+0x165/0x1c5
[52626.074011] kernel_restart+0xe/0x30
[52626.077593] __do_sys_reboot+0x1d2/0x210
[52626.093815] do_syscall_64+0x5b/0x1a0
[52626.097483] entry_SYSCALL_64_after_hwframe+0x65/0xca
Fixes: 974578017fc1 ("iavf: Add waiting so the port is initialized in remove")
Signed-off-by: Ivan Vecera <ivecera@redhat.com>
Link: https://lore.kernel.org/r/20220317104524.2802848-1-ivecera@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
drivers/net/ethernet/intel/iavf/iavf_main.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c
index f2ce30366484..42e6675078cd 100644
--- a/drivers/net/ethernet/intel/iavf/iavf_main.c
+++ b/drivers/net/ethernet/intel/iavf/iavf_main.c
@@ -4028,6 +4028,13 @@ static void iavf_remove(struct pci_dev *pdev)
struct iavf_hw *hw = &adapter->hw;
int err;
+ /* When reboot/shutdown is in progress no need to do anything
+ * as the adapter is already REMOVE state that was set during
+ * iavf_shutdown() callback.
+ */
+ if (adapter->state == __IAVF_REMOVE)
+ return;
+
set_bit(__IAVF_IN_REMOVE_TASK, &adapter->crit_section);
/* Wait until port initialization is complete.
* There are flows where register/unregister netdev may race.
--
2.34.1
next prev parent reply other threads:[~2022-03-21 14:15 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-21 13:52 [PATCH 5.16 00/37] 5.16.17-rc1 review Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 01/37] crypto: qcom-rng - ensure buffer for generate is completely filled Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 02/37] ocfs2: fix crash when initialize filecheck kobj fails Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 03/37] mm: swap: get rid of livelock in swapin readahead Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 04/37] block: release rq qos structures for queue without disk Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 05/37] drm/mgag200: Fix PLL setup for g200wb and g200ew Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 06/37] efi: fix return value of __setup handlers Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 07/37] alx: acquire mutex for alx_reinit in alx_change_mtu Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 08/37] vsock: each transport cycles only on its own sockets Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 09/37] esp6: fix check on ipv6_skip_exthdrs return value Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 10/37] net: phy: marvell: Fix invalid comparison in the resume and suspend functions Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 11/37] net/packet: fix slab-out-of-bounds access in packet_recvmsg() Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 12/37] nvmet: revert "nvmet: make discovery NQN configurable" Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 13/37] atm: eni: Add check for dma_map_single Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 14/37] ice: fix NULL pointer dereference in ice_update_vsi_tx_ring_stats() Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 15/37] iavf: Fix double free in iavf_reset_task Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 16/37] hv_netvsc: Add check for kvmalloc_array Greg Kroah-Hartman
2022-03-21 13:52 ` [PATCH 5.16 17/37] drm/imx: parallel-display: Remove bus flags check in imx_pd_bridge_atomic_check() Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 18/37] drm/panel: simple: Fix Innolux G070Y2-L01 BPP settings Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 19/37] net: handle ARPHRD_PIMREG in dev_is_mac_header_xmit() Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 20/37] drm: Dont make DRM_PANEL_BRIDGE dependent on DRM_KMS_HELPERS Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 21/37] net: dsa: Add missing of_node_put() in dsa_port_parse_of Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 22/37] net: phy: mscc: Add MODULE_FIRMWARE macros Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 23/37] bnx2x: fix built-in kernel driver load failure Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 24/37] net: bcmgenet: skip invalid partial checksums Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 25/37] net: mscc: ocelot: fix backwards compatibility with single-chain tc-flower offload Greg Kroah-Hartman
2022-03-21 13:53 ` Greg Kroah-Hartman [this message]
2022-03-21 13:53 ` [PATCH 5.16 27/37] arm64: fix clang warning about TRAMP_VALIAS Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 28/37] usb: gadget: rndis: prevent integer overflow in rndis_set_response() Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 29/37] usb: gadget: Fix use-after-free bug by not setting udc->dev.driver Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 30/37] usb: usbtmc: Fix bug in pipe direction for control transfers Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 31/37] scsi: mpt3sas: Page fault in reply q processing Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 32/37] Input: aiptek - properly check endpoint type Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 33/37] arm64: errata: avoid duplicate field initializer Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 34/37] perf symbols: Fix symbol size calculation condition Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 35/37] Revert "arm64: dts: freescale: Fix interrupt-map parent address cells" Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 36/37] Revert "ath10k: drop beacon and probe response which leak from other channel" Greg Kroah-Hartman
2022-03-21 13:53 ` [PATCH 5.16 37/37] btrfs: skip reserved bytes warning on unmount after log cleanup failure Greg Kroah-Hartman
2022-03-21 18:22 ` [PATCH 5.16 00/37] 5.16.17-rc1 review Florian Fainelli
2022-03-21 19:51 ` Jeffrin Thalakkottoor
2022-03-21 23:21 ` Shuah Khan
2022-03-21 23:28 ` Fox Chen
2022-03-22 1:53 ` Zan Aziz
2022-03-22 2:01 ` Guenter Roeck
2022-03-22 8:31 ` Ron Economos
2022-03-22 8:52 ` Naresh Kamboju
2022-03-22 11:23 ` Bagas Sanjaya
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220321133222.053422632@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=ivecera@redhat.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=sashal@kernel.org \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).