From: Saeed Mahameed <saeedm@mellanox.com>
To: "David S. Miller" <davem@davemloft.net>
Cc: "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
Moshe Shemesh <moshe@mellanox.com>,
Eran Ben Elisha <eranbe@mellanox.com>,
Saeed Mahameed <saeedm@mellanox.com>
Subject: [net 2/4] net/mlx5: Fix delay in fw fatal report handling due to fw report
Date: Thu, 22 Aug 2019 20:41:38 +0000 [thread overview]
Message-ID: <20190822204121.16954-3-saeedm@mellanox.com> (raw)
In-Reply-To: <20190822204121.16954-1-saeedm@mellanox.com>
From: Moshe Shemesh <moshe@mellanox.com>
When fw fatal error occurs, poll health() first detects and reports on a
fw error. Afterwards, it detects and reports on the fw fatal error
itself.
That can cause a long delay in fw fatal error handling which waits in a
queue for the fw error handling to be finished. The fw error handle will
try asking for fw core dump command while fw in fatal state may not
respond and driver will wait for command timeout.
Changing the flow to detect and handle first fw fatal errors and only if
no fatal error detected look for a fw error to handle.
Fixes: d1bf0e2cc4a6 ("net/mlx5: Report devlink health on FW issues")
Signed-off-by: Moshe Shemesh <moshe@mellanox.com>
Reviewed-by: Eran Ben Elisha <eranbe@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../net/ethernet/mellanox/mlx5/core/health.c | 19 ++++++++++---------
1 file changed, 10 insertions(+), 9 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c
index cc5887f52679..d685122d9ff7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/health.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c
@@ -701,6 +701,16 @@ static void poll_health(struct timer_list *t)
if (dev->state == MLX5_DEVICE_STATE_INTERNAL_ERROR)
goto out;
+ fatal_error = check_fatal_sensors(dev);
+
+ if (fatal_error && !health->fatal_error) {
+ mlx5_core_err(dev, "Fatal error %u detected\n", fatal_error);
+ dev->priv.health.fatal_error = fatal_error;
+ print_health_info(dev);
+ mlx5_trigger_health_work(dev);
+ goto out;
+ }
+
count = ioread32be(health->health_counter);
if (count == health->prev)
++health->miss_counter;
@@ -719,15 +729,6 @@ static void poll_health(struct timer_list *t)
if (health->synd && health->synd != prev_synd)
queue_work(health->wq, &health->report_work);
- fatal_error = check_fatal_sensors(dev);
-
- if (fatal_error && !health->fatal_error) {
- mlx5_core_err(dev, "Fatal error %u detected\n", fatal_error);
- dev->priv.health.fatal_error = fatal_error;
- print_health_info(dev);
- mlx5_trigger_health_work(dev);
- }
-
out:
mod_timer(&health->timer, get_next_poll_jiffies());
}
--
2.21.0
next prev parent reply other threads:[~2019-08-22 20:41 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-22 20:41 [pull request][net 0/4] Mellanox, mlx5 fixes 2019-08-22 Saeed Mahameed
2019-08-22 20:41 ` [net 1/4] net/mlx5: Fix crdump chunks print Saeed Mahameed
2019-08-22 20:41 ` Saeed Mahameed [this message]
2019-08-22 20:41 ` [net 3/4] net/mlx5e: Add num bytes metadata to WQE info Saeed Mahameed
2019-08-22 20:41 ` [net 4/4] net/mlx5e: Remove ethernet segment from dump WQE Saeed Mahameed
2019-08-24 23:27 ` [pull request][net 0/4] Mellanox, mlx5 fixes 2019-08-22 David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190822204121.16954-3-saeedm@mellanox.com \
--to=saeedm@mellanox.com \
--cc=davem@davemloft.net \
--cc=eranbe@mellanox.com \
--cc=moshe@mellanox.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox