From: Saeed Mahameed <saeed@kernel.org>
To: "David S. Miller" <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Eric Dumazet <edumazet@google.com>
Cc: Saeed Mahameed <saeedm@nvidia.com>,
netdev@vger.kernel.org, Tariq Toukan <tariqt@nvidia.com>,
Moshe Shemesh <moshe@nvidia.com>, Shay Drory <shayd@nvidia.com>
Subject: [net 04/16] net/mlx5: Fix command stats access after free
Date: Mon, 9 Jan 2023 22:11:11 -0800 [thread overview]
Message-ID: <20230110061123.338427-5-saeed@kernel.org> (raw)
In-Reply-To: <20230110061123.338427-1-saeed@kernel.org>
From: Moshe Shemesh <moshe@nvidia.com>
Command may fail while driver is reloading and can't accept FW commands
till command interface is reinitialized. Such command failure is being
logged to command stats. This results in NULL pointer access as command
stats structure is being freed and reallocated during mlx5 devlink
reload (see kernel log below).
Fix it by making command stats statically allocated on driver probe.
Kernel log:
[ 2394.808802] BUG: unable to handle kernel paging request at 000000000002a9c0
[ 2394.810610] PGD 0 P4D 0
[ 2394.811811] Oops: 0002 [#1] SMP NOPTI
...
[ 2394.815482] RIP: 0010:native_queued_spin_lock_slowpath+0x183/0x1d0
...
[ 2394.829505] Call Trace:
[ 2394.830667] _raw_spin_lock_irq+0x23/0x26
[ 2394.831858] cmd_status_err+0x55/0x110 [mlx5_core]
[ 2394.833020] mlx5_access_reg+0xe7/0x150 [mlx5_core]
[ 2394.834175] mlx5_query_port_ptys+0x78/0xa0 [mlx5_core]
[ 2394.835337] mlx5e_ethtool_get_link_ksettings+0x74/0x590 [mlx5_core]
[ 2394.836454] ? kmem_cache_alloc_trace+0x140/0x1c0
[ 2394.837562] __rh_call_get_link_ksettings+0x33/0x100
[ 2394.838663] ? __rtnl_unlock+0x25/0x50
[ 2394.839755] __ethtool_get_link_ksettings+0x72/0x150
[ 2394.840862] duplex_show+0x6e/0xc0
[ 2394.841963] dev_attr_show+0x1c/0x40
[ 2394.843048] sysfs_kf_seq_show+0x9b/0x100
[ 2394.844123] seq_read+0x153/0x410
[ 2394.845187] vfs_read+0x91/0x140
[ 2394.846226] ksys_read+0x4f/0xb0
[ 2394.847234] do_syscall_64+0x5b/0x1a0
[ 2394.848228] entry_SYSCALL_64_after_hwframe+0x65/0xca
Fixes: 34f46ae0d4b3 ("net/mlx5: Add command failures data to debugfs")
Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Reviewed-by: Shay Drory <shayd@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/cmd.c | 13 ++-----------
include/linux/mlx5/driver.h | 2 +-
2 files changed, 3 insertions(+), 12 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
index d3ca745d107d..c837103a9ee3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c
@@ -2176,15 +2176,9 @@ int mlx5_cmd_init(struct mlx5_core_dev *dev)
return -EINVAL;
}
- cmd->stats = kvcalloc(MLX5_CMD_OP_MAX, sizeof(*cmd->stats), GFP_KERNEL);
- if (!cmd->stats)
- return -ENOMEM;
-
cmd->pool = dma_pool_create("mlx5_cmd", mlx5_core_dma_dev(dev), size, align, 0);
- if (!cmd->pool) {
- err = -ENOMEM;
- goto dma_pool_err;
- }
+ if (!cmd->pool)
+ return -ENOMEM;
err = alloc_cmd_page(dev, cmd);
if (err)
@@ -2268,8 +2262,6 @@ int mlx5_cmd_init(struct mlx5_core_dev *dev)
err_free_pool:
dma_pool_destroy(cmd->pool);
-dma_pool_err:
- kvfree(cmd->stats);
return err;
}
@@ -2282,7 +2274,6 @@ void mlx5_cmd_cleanup(struct mlx5_core_dev *dev)
destroy_msg_cache(dev);
free_cmd_page(dev, cmd);
dma_pool_destroy(cmd->pool);
- kvfree(cmd->stats);
}
void mlx5_cmd_set_state(struct mlx5_core_dev *dev,
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index d476255c9a3f..76ef2e4fde38 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -315,7 +315,7 @@ struct mlx5_cmd {
struct mlx5_cmd_debug dbg;
struct cmd_msg_cache cache[MLX5_NUM_COMMAND_CACHES];
int checksum_disabled;
- struct mlx5_cmd_stats *stats;
+ struct mlx5_cmd_stats stats[MLX5_CMD_OP_MAX];
};
struct mlx5_cmd_mailbox {
--
2.39.0
next prev parent reply other threads:[~2023-01-10 6:11 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-10 6:11 [pull request][net 00/16] mlx5 fixes 2023-01-09 Saeed Mahameed
2023-01-10 6:11 ` [net 01/16] net/mlx5: DR, Fix 'stack frame size exceeds limit' error in dr_rule Saeed Mahameed
2023-01-11 13:00 ` patchwork-bot+netdevbpf
2023-01-10 6:11 ` [net 02/16] net/mlx5: check attr pointer validity before dereferencing it Saeed Mahameed
2023-01-10 6:11 ` [net 03/16] net/mlx5e: TC, Keep mod hdr actions after mod hdr alloc Saeed Mahameed
2023-01-10 6:11 ` Saeed Mahameed [this message]
2023-01-10 6:11 ` [net 05/16] net/mlx5e: Verify dev is present for fix features ndo Saeed Mahameed
2023-01-10 6:11 ` [net 06/16] net/mlx5e: IPoIB, Block queue count configuration when sub interfaces are present Saeed Mahameed
2023-01-10 6:11 ` [net 07/16] net/mlx5e: IPoIB, Block PKEY interfaces with less rx queues than parent Saeed Mahameed
2023-01-10 6:11 ` [net 08/16] net/mlx5e: IPoIB, Fix child PKEY interface stats on rx path Saeed Mahameed
2023-01-10 6:11 ` [net 09/16] net/mlx5e: TC, ignore match level for post meter rules Saeed Mahameed
2023-01-10 6:11 ` [net 10/16] net/mlx5e: TC, Restore pkt rate policing support Saeed Mahameed
2023-01-10 6:11 ` [net 11/16] net/mlx5e: Fix memory leak on updating vport counters Saeed Mahameed
2023-01-10 6:11 ` [net 12/16] net/mlx5: Fix ptp max frequency adjustment range Saeed Mahameed
2023-01-10 6:11 ` [net 13/16] net/mlx5e: Don't support encap rules with gbp option Saeed Mahameed
2023-01-10 6:11 ` [net 14/16] net/mlx5: E-switch, Coverity: overlapping copy Saeed Mahameed
2023-01-10 6:11 ` [net 15/16] net/mlx5e: Fix macsec ssci attribute handling in offload path Saeed Mahameed
2023-01-10 6:11 ` [net 16/16] net/mlx5e: Fix macsec possible null dereference when updating MAC security entity (SecY) Saeed Mahameed
2023-01-11 2:01 ` [pull request][net 00/16] mlx5 fixes 2023-01-09 Jakub Kicinski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230110061123.338427-5-saeed@kernel.org \
--to=saeed@kernel.org \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=kuba@kernel.org \
--cc=moshe@nvidia.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=saeedm@nvidia.com \
--cc=shayd@nvidia.com \
--cc=tariqt@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).