All of lore.kernel.org
 help / color / mirror / Atom feed
From: saeed@kernel.org
To: "David S. Miller" <davem@davemloft.net>,
	Jakub Kicinski <kuba@kernel.org>
Cc: netdev@vger.kernel.org, Shay Drory <shayd@mellanox.com>,
	Saeed Mahameed <saeedm@mellanox.com>,
	Saeed Mahameed <saeedm@nvidia.com>
Subject: [net V2 01/15] net/mlx5: Don't allow health work when device is uninitialized
Date: Thu,  1 Oct 2020 12:52:33 -0700	[thread overview]
Message-ID: <20201001195247.66636-2-saeed@kernel.org> (raw)
In-Reply-To: <20201001195247.66636-1-saeed@kernel.org>

From: Shay Drory <shayd@mellanox.com>

On error flow due to failure on driver load, driver can be
un-initializing while a health work is running in the background,
health work shouldn't be allowed at this point, as it needs resources to
be initialized and there is no point to recover on driver load failures.

Therefore, introducing a new state bit to indicated if device is
initialized, for health work to check before trying to recover the driver.

Fixes: b6e0b6bebe07 ("net/mlx5: Fix fatal error handling during device load")
Signed-off-by: Shay Drory <shayd@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/health.c | 11 +++++++++++
 drivers/net/ethernet/mellanox/mlx5/core/main.c   |  2 ++
 include/linux/mlx5/driver.h                      |  1 +
 3 files changed, 14 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c
index b31f769d2df9..bbaffe7af619 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/health.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c
@@ -190,6 +190,11 @@ static bool reset_fw_if_needed(struct mlx5_core_dev *dev)
 	return true;
 }
 
+static bool mlx5_is_device_initialized(struct mlx5_core_dev *dev)
+{
+	return test_bit(MLX5_INTERFACE_STATE_INITIALIZED, &dev->intf_state);
+}
+
 void mlx5_enter_error_state(struct mlx5_core_dev *dev, bool force)
 {
 	bool err_detected = false;
@@ -201,6 +206,9 @@ void mlx5_enter_error_state(struct mlx5_core_dev *dev, bool force)
 		err_detected = true;
 	}
 	mutex_lock(&dev->intf_state_mutex);
+	if (!mlx5_is_device_initialized(dev))
+		goto unlock;
+
 	if (!err_detected && dev->state == MLX5_DEVICE_STATE_INTERNAL_ERROR)
 		goto unlock;/* a previous error is still being handled */
 	if (dev->state == MLX5_DEVICE_STATE_UNINITIALIZED) {
@@ -609,6 +617,9 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work)
 	dev = container_of(priv, struct mlx5_core_dev, priv);
 
 	mlx5_enter_error_state(dev, false);
+	if (!mlx5_is_device_initialized(dev))
+		return;
+
 	if (IS_ERR_OR_NULL(health->fw_fatal_reporter)) {
 		if (mlx5_health_try_recover(dev))
 			mlx5_core_err(dev, "health recovery failed\n");
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index ce43e3feccd9..8ebb8d27d3ac 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -878,6 +878,7 @@ static int mlx5_init_once(struct mlx5_core_dev *dev)
 	dev->tracer = mlx5_fw_tracer_create(dev);
 	dev->hv_vhca = mlx5_hv_vhca_create(dev);
 	dev->rsc_dump = mlx5_rsc_dump_create(dev);
+	set_bit(MLX5_INTERFACE_STATE_INITIALIZED, &dev->intf_state);
 
 	return 0;
 
@@ -906,6 +907,7 @@ static int mlx5_init_once(struct mlx5_core_dev *dev)
 
 static void mlx5_cleanup_once(struct mlx5_core_dev *dev)
 {
+	clear_bit(MLX5_INTERFACE_STATE_INITIALIZED, &dev->intf_state);
 	mlx5_rsc_dump_destroy(dev);
 	mlx5_hv_vhca_destroy(dev->hv_vhca);
 	mlx5_fw_tracer_destroy(dev->tracer);
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index c145de0473bc..223aaaaaf8a6 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -594,6 +594,7 @@ enum mlx5_device_state {
 
 enum mlx5_interface_state {
 	MLX5_INTERFACE_STATE_UP = BIT(0),
+	MLX5_INTERFACE_STATE_INITIALIZED = BIT(1),
 };
 
 enum mlx5_pci_status {
-- 
2.26.2


  reply	other threads:[~2020-10-01 19:53 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-01 19:52 [pull request][net V2 00/15] mlx5 fixes 2020-09-30 saeed
2020-10-01 19:52 ` saeed [this message]
2020-10-01 23:15   ` [net V2 01/15] net/mlx5: Don't allow health work when device is uninitialized Jakub Kicinski
2020-10-02 16:57     ` Saeed Mahameed
2020-10-01 19:52 ` [net V2 02/15] net/mlx5: Fix a race when moving command interface to polling mode saeed
2020-10-01 19:52 ` [net V2 03/15] net/mlx5: Avoid possible free of command entry while timeout comp handler saeed
2020-10-01 19:52 ` [net V2 04/15] net/mlx5: poll cmd EQ in case of command timeout saeed
2020-10-01 19:52 ` [net V2 05/15] net/mlx5: Add retry mechanism to the command entry index allocation saeed
2020-10-01 23:23   ` Jakub Kicinski
2020-10-02 17:03     ` Saeed Mahameed
2020-10-01 19:52 ` [net V2 06/15] net/mlx5: cmdif, Avoid skipping reclaim pages if FW is not accessible saeed
2020-10-01 19:52 ` [net V2 07/15] net/mlx5: Fix request_irqs error flow saeed
2020-10-01 23:24   ` Jakub Kicinski
2020-10-02 17:05     ` Saeed Mahameed
2020-10-02 17:19       ` Mark Bloch
2020-10-02 17:27         ` Saeed Mahameed
2020-10-01 19:52 ` [net V2 08/15] net/mlx5e: Fix error path for RQ alloc saeed
2020-10-01 19:52 ` [net V2 09/15] net/mlx5e: Add resiliency in Striding RQ mode for packets larger than MTU saeed
2020-10-01 23:27   ` Jakub Kicinski
2020-10-02 17:06     ` Saeed Mahameed
2020-10-01 19:52 ` [net V2 10/15] net/mlx5e: CT, Fix coverity issue saeed
2020-10-01 19:52 ` [net V2 11/15] net/mlx5e: Fix driver's declaration to support GRE offload saeed
2020-10-01 19:52 ` [net V2 12/15] net/mlx5e: Fix return status when setting unsupported FEC mode saeed
2020-10-01 19:52 ` [net V2 13/15] net/mlx5e: Fix VLAN cleanup flow saeed
2020-10-01 19:52 ` [net V2 14/15] net/mlx5e: Fix VLAN create flow saeed
2020-10-01 19:52 ` [net V2 15/15] net/mlx5e: Fix race condition on nhe->n pointer in neigh update saeed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201001195247.66636-2-saeed@kernel.org \
    --to=saeed@kernel.org \
    --cc=davem@davemloft.net \
    --cc=kuba@kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=saeedm@mellanox.com \
    --cc=saeedm@nvidia.com \
    --cc=shayd@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.