* [net 01/10] net/mlx5e: IPSEC RX, enable checksum complete
2021-09-30 23:14 [pull request][net 00/10] mlx5 fixes 2021-09-30 Saeed Mahameed
@ 2021-09-30 23:14 ` Saeed Mahameed
2021-10-01 13:10 ` patchwork-bot+netdevbpf
2021-09-30 23:14 ` [net 02/10] net/mlx5e: Keep the value for maximum number of channels in-sync Saeed Mahameed
` (8 subsequent siblings)
9 siblings, 1 reply; 14+ messages in thread
From: Saeed Mahameed @ 2021-09-30 23:14 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski; +Cc: netdev, Raed Salem, Saeed Mahameed
From: Raed Salem <raeds@nvidia.com>
Currently in Rx data path IPsec crypto offloaded packets uses
csum_none flag, so checksum is handled by the stack, this naturally
have some performance/cpu utilization impact on such flows. As Nvidia
NIC starting from ConnectX6DX provides checksum complete value out of
the box also for such flows there is no sense in taking csum_none path,
furthermore the stack (xfrm) have the method to handle checksum complete
corrections for such flows i.e. IPsec trailer removal and consequently
checksum value adjustment.
Because of the above and in addition the ConnectX6DX is the first HW
which supports IPsec crypto offload then it is safe to report csum
complete for IPsec offloaded traffic.
Fixes: b2ac7541e377 ("net/mlx5e: IPsec: Add Connect-X IPsec Rx data path offload")
Signed-off-by: Raed Salem <raeds@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 7 +------
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 3c65fd0bcf31..29a6586ef28d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -1001,14 +1001,9 @@ static inline void mlx5e_handle_csum(struct net_device *netdev,
goto csum_unnecessary;
if (likely(is_last_ethertype_ip(skb, &network_depth, &proto))) {
- u8 ipproto = get_ip_proto(skb, network_depth, proto);
-
- if (unlikely(ipproto == IPPROTO_SCTP))
+ if (unlikely(get_ip_proto(skb, network_depth, proto) == IPPROTO_SCTP))
goto csum_unnecessary;
- if (unlikely(mlx5_ipsec_is_rx_flow(cqe)))
- goto csum_none;
-
stats->csum_complete++;
skb->ip_summed = CHECKSUM_COMPLETE;
skb->csum = csum_unfold((__force __sum16)cqe->check_sum);
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* Re: [net 01/10] net/mlx5e: IPSEC RX, enable checksum complete
2021-09-30 23:14 ` [net 01/10] net/mlx5e: IPSEC RX, enable checksum complete Saeed Mahameed
@ 2021-10-01 13:10 ` patchwork-bot+netdevbpf
2021-10-01 18:27 ` Saeed Mahameed
0 siblings, 1 reply; 14+ messages in thread
From: patchwork-bot+netdevbpf @ 2021-10-01 13:10 UTC (permalink / raw)
To: Saeed Mahameed; +Cc: davem, kuba, netdev, raeds, saeedm
Hello:
This series was applied to netdev/net-next.git (refs/heads/master):
On Thu, 30 Sep 2021 16:14:52 -0700 you wrote:
> From: Raed Salem <raeds@nvidia.com>
>
> Currently in Rx data path IPsec crypto offloaded packets uses
> csum_none flag, so checksum is handled by the stack, this naturally
> have some performance/cpu utilization impact on such flows. As Nvidia
> NIC starting from ConnectX6DX provides checksum complete value out of
> the box also for such flows there is no sense in taking csum_none path,
> furthermore the stack (xfrm) have the method to handle checksum complete
> corrections for such flows i.e. IPsec trailer removal and consequently
> checksum value adjustment.
>
> [...]
Here is the summary with links:
- [net,01/10] net/mlx5e: IPSEC RX, enable checksum complete
https://git.kernel.org/netdev/net-next/c/f9a10440f0b1
- [net,02/10] net/mlx5e: Keep the value for maximum number of channels in-sync
https://git.kernel.org/netdev/net-next/c/9d758d4a3a03
- [net,03/10] net/mlx5e: Improve MQPRIO resiliency
https://git.kernel.org/netdev/net-next/c/7dbc849b2ab3
- [net,04/10] net/mlx5: E-Switch, Fix double allocation of acl flow counter
https://git.kernel.org/netdev/net-next/c/a586775f83bd
- [net,05/10] net/mlx5: Force round second at 1PPS out start time
https://git.kernel.org/netdev/net-next/c/64728294703e
- [net,06/10] net/mlx5: Avoid generating event after PPS out in Real time mode
https://git.kernel.org/netdev/net-next/c/99b9a678b2e4
- [net,07/10] net/mlx5: Fix length of irq_index in chars
https://git.kernel.org/netdev/net-next/c/ac8b7d50ae4c
- [net,08/10] net/mlx5: Fix setting number of EQs of SFs
https://git.kernel.org/netdev/net-next/c/f88c48763474
- [net,09/10] net/mlx5e: Fix the presented RQ index in PTP stats
https://git.kernel.org/netdev/net-next/c/dd1979cf3c71
- [net,10/10] net/mlx5e: Mutually exclude setting of TX-port-TS and MQPRIO in channel mode
https://git.kernel.org/netdev/net-next/c/3bf1742f3c69
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: [net 01/10] net/mlx5e: IPSEC RX, enable checksum complete
2021-10-01 13:10 ` patchwork-bot+netdevbpf
@ 2021-10-01 18:27 ` Saeed Mahameed
2021-10-02 11:21 ` David Miller
0 siblings, 1 reply; 14+ messages in thread
From: Saeed Mahameed @ 2021-10-01 18:27 UTC (permalink / raw)
To: patchwork-bot+netdevbpf@kernel.org
Cc: davem@davemloft.net, kuba@kernel.org, Raed Salem,
netdev@vger.kernel.org
On Fri, 2021-10-01 at 13:10 +0000, patchwork-bot+netdevbpf@kernel.org
wrote:
> Hello:
>
> This series was applied to netdev/net-next.git (refs/heads/master):
this was for -net.
I see it applied to both net and net-next, why the bot says it was
net-next ?
Any mistakes on my end ?
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [net 01/10] net/mlx5e: IPSEC RX, enable checksum complete
2021-10-01 18:27 ` Saeed Mahameed
@ 2021-10-02 11:21 ` David Miller
0 siblings, 0 replies; 14+ messages in thread
From: David Miller @ 2021-10-02 11:21 UTC (permalink / raw)
To: saeedm; +Cc: patchwork-bot+netdevbpf, kuba, raeds, netdev
From: Saeed Mahameed <saeedm@nvidia.com>
Date: Fri, 1 Oct 2021 18:27:34 +0000
> On Fri, 2021-10-01 at 13:10 +0000, patchwork-bot+netdevbpf@kernel.org
> wrote:
>> Hello:
>>
>> This series was applied to netdev/net-next.git (refs/heads/master):
>
> this was for -net.
> I see it applied to both net and net-next, why the bot says it was
> net-next ?
>
> Any mistakes on my end ?
I accidently pulled it initially into net-next and puished back out before I could undo that.
So I just pushed it to both trees in the end, not your fault.
Thanks!
^ permalink raw reply [flat|nested] 14+ messages in thread
* [net 02/10] net/mlx5e: Keep the value for maximum number of channels in-sync
2021-09-30 23:14 [pull request][net 00/10] mlx5 fixes 2021-09-30 Saeed Mahameed
2021-09-30 23:14 ` [net 01/10] net/mlx5e: IPSEC RX, enable checksum complete Saeed Mahameed
@ 2021-09-30 23:14 ` Saeed Mahameed
2021-09-30 23:14 ` [net 03/10] net/mlx5e: Improve MQPRIO resiliency Saeed Mahameed
` (7 subsequent siblings)
9 siblings, 0 replies; 14+ messages in thread
From: Saeed Mahameed @ 2021-09-30 23:14 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Tariq Toukan, Aya Levin, Maxim Mikityanskiy,
Saeed Mahameed
From: Tariq Toukan <tariqt@nvidia.com>
The value for maximum number of channels is first calculated based
on the netdev's profile and current function resources (specifically,
number of MSIX vectors, which depends among other things on the number
of online cores in the system).
This value is then used to calculate the netdev's number of rxqs/txqs.
Once created (by alloc_etherdev_mqs), the number of netdev's rxqs/txqs
is constant and we must not exceed it.
To achieve this, keep the maximum number of channels in sync upon any
netdevice re-attach.
Use mlx5e_get_max_num_channels() for calculating the number of netdev's
rxqs/txqs. After netdev is created, use mlx5e_calc_max_nch() (which
coinsiders core device resources, profile, and netdev) to init or
update priv->max_nch.
Before this patch, the value of priv->max_nch might get out of sync,
mistakenly allowing accesses to out-of-bounds objects, which would
crash the system.
Track the number of channels stats structures used in a separate
field, as they are persistent to suspend/resume operations. All the
collected stats of every channel index that ever existed should be
preserved. They are reset only when struct mlx5e_priv is,
in mlx5e_priv_cleanup(), which is part of the profile changing flow.
There is no point anymore in blocking a profile change due to max_nch
mismatch in mlx5e_netdev_change_profile(). Remove the limitation.
Fixes: a1f240f18017 ("net/mlx5e: Adjust to max number of channles when re-attaching")
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
Reviewed-by: Aya Levin <ayal@nvidia.com>
Reviewed-by: Maxim Mikityanskiy <maximmi@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en.h | 11 ++--
.../mellanox/mlx5/core/en/hv_vhca_stats.c | 6 +-
.../net/ethernet/mellanox/mlx5/core/en_main.c | 59 ++++++++++++++-----
.../net/ethernet/mellanox/mlx5/core/en_rep.c | 3 +-
.../ethernet/mellanox/mlx5/core/en_stats.c | 8 +--
.../ethernet/mellanox/mlx5/core/ipoib/ipoib.c | 4 +-
6 files changed, 57 insertions(+), 34 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 7b8c8187543a..2dca9219ca71 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -845,6 +845,7 @@ struct mlx5e_priv {
struct mlx5e_channel_stats channel_stats[MLX5E_MAX_NUM_CHANNELS];
struct mlx5e_channel_stats trap_stats;
struct mlx5e_ptp_stats ptp_stats;
+ u16 stats_nch;
u16 max_nch;
u8 max_opened_tc;
bool tx_ptp_opened;
@@ -1100,12 +1101,6 @@ int mlx5e_ethtool_set_pauseparam(struct mlx5e_priv *priv,
struct ethtool_pauseparam *pauseparam);
/* mlx5e generic netdev management API */
-static inline unsigned int
-mlx5e_calc_max_nch(struct mlx5e_priv *priv, const struct mlx5e_profile *profile)
-{
- return priv->netdev->num_rx_queues / max_t(u8, profile->rq_groups, 1);
-}
-
static inline bool
mlx5e_tx_mpwqe_supported(struct mlx5_core_dev *mdev)
{
@@ -1114,11 +1109,13 @@ mlx5e_tx_mpwqe_supported(struct mlx5_core_dev *mdev)
}
int mlx5e_priv_init(struct mlx5e_priv *priv,
+ const struct mlx5e_profile *profile,
struct net_device *netdev,
struct mlx5_core_dev *mdev);
void mlx5e_priv_cleanup(struct mlx5e_priv *priv);
struct net_device *
-mlx5e_create_netdev(struct mlx5_core_dev *mdev, unsigned int txqs, unsigned int rxqs);
+mlx5e_create_netdev(struct mlx5_core_dev *mdev, const struct mlx5e_profile *profile,
+ unsigned int txqs, unsigned int rxqs);
int mlx5e_attach_netdev(struct mlx5e_priv *priv);
void mlx5e_detach_netdev(struct mlx5e_priv *priv);
void mlx5e_destroy_netdev(struct mlx5e_priv *priv);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c
index ac44bbe95c5c..d290d7276b8d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/hv_vhca_stats.c
@@ -35,7 +35,7 @@ static void mlx5e_hv_vhca_fill_stats(struct mlx5e_priv *priv, void *data,
{
int ch, i = 0;
- for (ch = 0; ch < priv->max_nch; ch++) {
+ for (ch = 0; ch < priv->stats_nch; ch++) {
void *buf = data + i;
if (WARN_ON_ONCE(buf +
@@ -51,7 +51,7 @@ static void mlx5e_hv_vhca_fill_stats(struct mlx5e_priv *priv, void *data,
static int mlx5e_hv_vhca_stats_buf_size(struct mlx5e_priv *priv)
{
return (sizeof(struct mlx5e_hv_vhca_per_ring_stats) *
- priv->max_nch);
+ priv->stats_nch);
}
static void mlx5e_hv_vhca_stats_work(struct work_struct *work)
@@ -100,7 +100,7 @@ static void mlx5e_hv_vhca_stats_control(struct mlx5_hv_vhca_agent *agent,
sagent = &priv->stats_agent;
block->version = MLX5_HV_VHCA_STATS_VERSION;
- block->rings = priv->max_nch;
+ block->rings = priv->stats_nch;
if (!block->command) {
cancel_delayed_work_sync(&priv->stats_agent.work);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 3fd515e7bf30..774ce88d80cd 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -3065,7 +3065,7 @@ void mlx5e_fold_sw_stats64(struct mlx5e_priv *priv, struct rtnl_link_stats64 *s)
{
int i;
- for (i = 0; i < priv->max_nch; i++) {
+ for (i = 0; i < priv->stats_nch; i++) {
struct mlx5e_channel_stats *channel_stats = &priv->channel_stats[i];
struct mlx5e_rq_stats *xskrq_stats = &channel_stats->xskrq;
struct mlx5e_rq_stats *rq_stats = &channel_stats->rq;
@@ -4186,8 +4186,6 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16
struct mlx5_core_dev *mdev = priv->mdev;
u8 rx_cq_period_mode;
- priv->max_nch = mlx5e_calc_max_nch(priv, priv->profile);
-
params->sw_mtu = mtu;
params->hard_mtu = MLX5E_ETH_HARD_MTU;
params->num_channels = min_t(unsigned int, MLX5E_MAX_NUM_CHANNELS / 2,
@@ -4682,8 +4680,35 @@ static const struct mlx5e_profile mlx5e_nic_profile = {
.rx_ptp_support = true,
};
+static unsigned int
+mlx5e_calc_max_nch(struct mlx5_core_dev *mdev, struct net_device *netdev,
+ const struct mlx5e_profile *profile)
+
+{
+ unsigned int max_nch, tmp;
+
+ /* core resources */
+ max_nch = mlx5e_get_max_num_channels(mdev);
+
+ /* netdev rx queues */
+ tmp = netdev->num_rx_queues / max_t(u8, profile->rq_groups, 1);
+ max_nch = min_t(unsigned int, max_nch, tmp);
+
+ /* netdev tx queues */
+ tmp = netdev->num_tx_queues;
+ if (mlx5_qos_is_supported(mdev))
+ tmp -= mlx5e_qos_max_leaf_nodes(mdev);
+ if (MLX5_CAP_GEN(mdev, ts_cqe_to_dest_cqn))
+ tmp -= profile->max_tc;
+ tmp = tmp / profile->max_tc;
+ max_nch = min_t(unsigned int, max_nch, tmp);
+
+ return max_nch;
+}
+
/* mlx5e generic netdev management API (move to en_common.c) */
int mlx5e_priv_init(struct mlx5e_priv *priv,
+ const struct mlx5e_profile *profile,
struct net_device *netdev,
struct mlx5_core_dev *mdev)
{
@@ -4691,6 +4716,8 @@ int mlx5e_priv_init(struct mlx5e_priv *priv,
priv->mdev = mdev;
priv->netdev = netdev;
priv->msglevel = MLX5E_MSG_LEVEL;
+ priv->max_nch = mlx5e_calc_max_nch(mdev, netdev, profile);
+ priv->stats_nch = priv->max_nch;
priv->max_opened_tc = 1;
if (!alloc_cpumask_var(&priv->scratchpad.cpumask, GFP_KERNEL))
@@ -4734,7 +4761,8 @@ void mlx5e_priv_cleanup(struct mlx5e_priv *priv)
}
struct net_device *
-mlx5e_create_netdev(struct mlx5_core_dev *mdev, unsigned int txqs, unsigned int rxqs)
+mlx5e_create_netdev(struct mlx5_core_dev *mdev, const struct mlx5e_profile *profile,
+ unsigned int txqs, unsigned int rxqs)
{
struct net_device *netdev;
int err;
@@ -4745,7 +4773,7 @@ mlx5e_create_netdev(struct mlx5_core_dev *mdev, unsigned int txqs, unsigned int
return NULL;
}
- err = mlx5e_priv_init(netdev_priv(netdev), netdev, mdev);
+ err = mlx5e_priv_init(netdev_priv(netdev), profile, netdev, mdev);
if (err) {
mlx5_core_err(mdev, "mlx5e_priv_init failed, err=%d\n", err);
goto err_free_netdev;
@@ -4787,7 +4815,7 @@ int mlx5e_attach_netdev(struct mlx5e_priv *priv)
clear_bit(MLX5E_STATE_DESTROYING, &priv->state);
/* max number of channels may have changed */
- max_nch = mlx5e_get_max_num_channels(priv->mdev);
+ max_nch = mlx5e_calc_max_nch(priv->mdev, priv->netdev, profile);
if (priv->channels.params.num_channels > max_nch) {
mlx5_core_warn(priv->mdev, "MLX5E: Reducing number of channels to %d\n", max_nch);
/* Reducing the number of channels - RXFH has to be reset, and
@@ -4796,6 +4824,13 @@ int mlx5e_attach_netdev(struct mlx5e_priv *priv)
priv->netdev->priv_flags &= ~IFF_RXFH_CONFIGURED;
priv->channels.params.num_channels = max_nch;
}
+ if (max_nch != priv->max_nch) {
+ mlx5_core_warn(priv->mdev,
+ "MLX5E: Updating max number of channels from %u to %u\n",
+ priv->max_nch, max_nch);
+ priv->max_nch = max_nch;
+ }
+
/* 1. Set the real number of queues in the kernel the first time.
* 2. Set our default XPS cpumask.
* 3. Build the RQT.
@@ -4860,7 +4895,7 @@ mlx5e_netdev_attach_profile(struct net_device *netdev, struct mlx5_core_dev *mde
struct mlx5e_priv *priv = netdev_priv(netdev);
int err;
- err = mlx5e_priv_init(priv, netdev, mdev);
+ err = mlx5e_priv_init(priv, new_profile, netdev, mdev);
if (err) {
mlx5_core_err(mdev, "mlx5e_priv_init failed, err=%d\n", err);
return err;
@@ -4886,20 +4921,12 @@ mlx5e_netdev_attach_profile(struct net_device *netdev, struct mlx5_core_dev *mde
int mlx5e_netdev_change_profile(struct mlx5e_priv *priv,
const struct mlx5e_profile *new_profile, void *new_ppriv)
{
- unsigned int new_max_nch = mlx5e_calc_max_nch(priv, new_profile);
const struct mlx5e_profile *orig_profile = priv->profile;
struct net_device *netdev = priv->netdev;
struct mlx5_core_dev *mdev = priv->mdev;
void *orig_ppriv = priv->ppriv;
int err, rollback_err;
- /* sanity */
- if (new_max_nch != priv->max_nch) {
- netdev_warn(netdev, "%s: Replacing profile with different max channels\n",
- __func__);
- return -EINVAL;
- }
-
/* cleanup old profile */
mlx5e_detach_netdev(priv);
priv->profile->cleanup(priv);
@@ -4995,7 +5022,7 @@ static int mlx5e_probe(struct auxiliary_device *adev,
nch = mlx5e_get_max_num_channels(mdev);
txqs = nch * profile->max_tc + ptp_txqs + qos_sqs;
rxqs = nch * profile->rq_groups;
- netdev = mlx5e_create_netdev(mdev, txqs, rxqs);
+ netdev = mlx5e_create_netdev(mdev, profile, txqs, rxqs);
if (!netdev) {
mlx5_core_err(mdev, "mlx5e_create_netdev failed\n");
return -ENOMEM;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
index ae71a17fdb27..3dd1101cc693 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
@@ -596,7 +596,6 @@ static void mlx5e_build_rep_params(struct net_device *netdev)
MLX5_CQ_PERIOD_MODE_START_FROM_CQE :
MLX5_CQ_PERIOD_MODE_START_FROM_EQE;
- priv->max_nch = mlx5e_calc_max_nch(priv, priv->profile);
params = &priv->channels.params;
params->num_channels = MLX5E_REP_PARAMS_DEF_NUM_CHANNELS;
@@ -1169,7 +1168,7 @@ mlx5e_vport_vf_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep)
nch = mlx5e_get_max_num_channels(dev);
txqs = nch * profile->max_tc;
rxqs = nch * profile->rq_groups;
- netdev = mlx5e_create_netdev(dev, txqs, rxqs);
+ netdev = mlx5e_create_netdev(dev, profile, txqs, rxqs);
if (!netdev) {
mlx5_core_warn(dev,
"Failed to create representor netdev for vport %d\n",
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
index e4f5b6395148..46bf78169f63 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
@@ -450,7 +450,7 @@ static MLX5E_DECLARE_STATS_GRP_OP_UPDATE_STATS(sw)
memset(s, 0, sizeof(*s));
- for (i = 0; i < priv->max_nch; i++) {
+ for (i = 0; i < priv->stats_nch; i++) {
struct mlx5e_channel_stats *channel_stats =
&priv->channel_stats[i];
int j;
@@ -2119,7 +2119,7 @@ static MLX5E_DECLARE_STATS_GRP_OP_UPDATE_STATS(ptp) { return; }
static MLX5E_DECLARE_STATS_GRP_OP_NUM_STATS(channels)
{
- int max_nch = priv->max_nch;
+ int max_nch = priv->stats_nch;
return (NUM_RQ_STATS * max_nch) +
(NUM_CH_STATS * max_nch) +
@@ -2133,7 +2133,7 @@ static MLX5E_DECLARE_STATS_GRP_OP_NUM_STATS(channels)
static MLX5E_DECLARE_STATS_GRP_OP_FILL_STRS(channels)
{
bool is_xsk = priv->xsk.ever_used;
- int max_nch = priv->max_nch;
+ int max_nch = priv->stats_nch;
int i, j, tc;
for (i = 0; i < max_nch; i++)
@@ -2175,7 +2175,7 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STRS(channels)
static MLX5E_DECLARE_STATS_GRP_OP_FILL_STATS(channels)
{
bool is_xsk = priv->xsk.ever_used;
- int max_nch = priv->max_nch;
+ int max_nch = priv->stats_nch;
int i, j, tc;
for (i = 0; i < max_nch; i++)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
index 67571e5040d6..269ebb53eda6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
@@ -113,7 +113,7 @@ static void mlx5i_grp_sw_update_stats(struct mlx5e_priv *priv)
struct mlx5e_sw_stats s = { 0 };
int i, j;
- for (i = 0; i < priv->max_nch; i++) {
+ for (i = 0; i < priv->stats_nch; i++) {
struct mlx5e_channel_stats *channel_stats;
struct mlx5e_rq_stats *rq_stats;
@@ -711,7 +711,7 @@ static int mlx5_rdma_setup_rn(struct ib_device *ibdev, u32 port_num,
goto destroy_ht;
}
- err = mlx5e_priv_init(epriv, netdev, mdev);
+ err = mlx5e_priv_init(epriv, prof, netdev, mdev);
if (err)
goto destroy_mdev_resources;
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* [net 03/10] net/mlx5e: Improve MQPRIO resiliency
2021-09-30 23:14 [pull request][net 00/10] mlx5 fixes 2021-09-30 Saeed Mahameed
2021-09-30 23:14 ` [net 01/10] net/mlx5e: IPSEC RX, enable checksum complete Saeed Mahameed
2021-09-30 23:14 ` [net 02/10] net/mlx5e: Keep the value for maximum number of channels in-sync Saeed Mahameed
@ 2021-09-30 23:14 ` Saeed Mahameed
2021-09-30 23:14 ` [net 04/10] net/mlx5: E-Switch, Fix double allocation of acl flow counter Saeed Mahameed
` (6 subsequent siblings)
9 siblings, 0 replies; 14+ messages in thread
From: Saeed Mahameed @ 2021-09-30 23:14 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Tariq Toukan, Maxim Mikityanskiy, Saeed Mahameed
From: Tariq Toukan <tariqt@nvidia.com>
* Add netdev->tc_to_txq rollback in case of failure in
mlx5e_update_netdev_queues().
* Fix broken transition between the two modes:
MQPRIO DCB mode with tc==8, and MQPRIO channel mode.
* Disable MQPRIO channel mode if re-attaching with a different number
of channels.
* Improve code sharing.
Fixes: ec60c4581bd9 ("net/mlx5e: Support MQPRIO channel mode")
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
Reviewed-by: Maxim Mikityanskiy <maximmi@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 +
.../net/ethernet/mellanox/mlx5/core/en_main.c | 111 +++++++++++++-----
2 files changed, 80 insertions(+), 32 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 2dca9219ca71..03a7a4ce5cd5 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -252,6 +252,7 @@ struct mlx5e_params {
struct {
u16 mode;
u8 num_tc;
+ struct netdev_tc_txq tc_to_txq[TC_MAX_QUEUE];
} mqprio;
bool rx_cqe_compress_def;
bool tunneled_offload_en;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 774ce88d80cd..0390395f421f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -2264,7 +2264,7 @@ void mlx5e_set_netdev_mtu_boundaries(struct mlx5e_priv *priv)
}
static int mlx5e_netdev_set_tcs(struct net_device *netdev, u16 nch, u8 ntc,
- struct tc_mqprio_qopt_offload *mqprio)
+ struct netdev_tc_txq *tc_to_txq)
{
int tc, err;
@@ -2282,11 +2282,8 @@ static int mlx5e_netdev_set_tcs(struct net_device *netdev, u16 nch, u8 ntc,
for (tc = 0; tc < ntc; tc++) {
u16 count, offset;
- /* For DCB mode, map netdev TCs to offset 0
- * We have our own UP to TXQ mapping for QoS
- */
- count = mqprio ? mqprio->qopt.count[tc] : nch;
- offset = mqprio ? mqprio->qopt.offset[tc] : 0;
+ count = tc_to_txq[tc].count;
+ offset = tc_to_txq[tc].offset;
netdev_set_tc_queue(netdev, tc, count, offset);
}
@@ -2315,19 +2312,24 @@ int mlx5e_update_tx_netdev_queues(struct mlx5e_priv *priv)
static int mlx5e_update_netdev_queues(struct mlx5e_priv *priv)
{
+ struct netdev_tc_txq old_tc_to_txq[TC_MAX_QUEUE], *tc_to_txq;
struct net_device *netdev = priv->netdev;
int old_num_txqs, old_ntc;
int num_rxqs, nch, ntc;
int err;
+ int i;
old_num_txqs = netdev->real_num_tx_queues;
old_ntc = netdev->num_tc ? : 1;
+ for (i = 0; i < ARRAY_SIZE(old_tc_to_txq); i++)
+ old_tc_to_txq[i] = netdev->tc_to_txq[i];
nch = priv->channels.params.num_channels;
- ntc = mlx5e_get_dcb_num_tc(&priv->channels.params);
+ ntc = priv->channels.params.mqprio.num_tc;
num_rxqs = nch * priv->profile->rq_groups;
+ tc_to_txq = priv->channels.params.mqprio.tc_to_txq;
- err = mlx5e_netdev_set_tcs(netdev, nch, ntc, NULL);
+ err = mlx5e_netdev_set_tcs(netdev, nch, ntc, tc_to_txq);
if (err)
goto err_out;
err = mlx5e_update_tx_netdev_queues(priv);
@@ -2350,11 +2352,14 @@ static int mlx5e_update_netdev_queues(struct mlx5e_priv *priv)
WARN_ON_ONCE(netif_set_real_num_tx_queues(netdev, old_num_txqs));
err_tcs:
- mlx5e_netdev_set_tcs(netdev, old_num_txqs / old_ntc, old_ntc, NULL);
+ WARN_ON_ONCE(mlx5e_netdev_set_tcs(netdev, old_num_txqs / old_ntc, old_ntc,
+ old_tc_to_txq));
err_out:
return err;
}
+static MLX5E_DEFINE_PREACTIVATE_WRAPPER_CTX(mlx5e_update_netdev_queues);
+
static void mlx5e_set_default_xps_cpumasks(struct mlx5e_priv *priv,
struct mlx5e_params *params)
{
@@ -2861,6 +2866,58 @@ static int mlx5e_modify_channels_vsd(struct mlx5e_channels *chs, bool vsd)
return 0;
}
+static void mlx5e_mqprio_build_default_tc_to_txq(struct netdev_tc_txq *tc_to_txq,
+ int ntc, int nch)
+{
+ int tc;
+
+ memset(tc_to_txq, 0, sizeof(*tc_to_txq) * TC_MAX_QUEUE);
+
+ /* Map netdev TCs to offset 0.
+ * We have our own UP to TXQ mapping for DCB mode of QoS
+ */
+ for (tc = 0; tc < ntc; tc++) {
+ tc_to_txq[tc] = (struct netdev_tc_txq) {
+ .count = nch,
+ .offset = 0,
+ };
+ }
+}
+
+static void mlx5e_mqprio_build_tc_to_txq(struct netdev_tc_txq *tc_to_txq,
+ struct tc_mqprio_qopt *qopt)
+{
+ int tc;
+
+ for (tc = 0; tc < TC_MAX_QUEUE; tc++) {
+ tc_to_txq[tc] = (struct netdev_tc_txq) {
+ .count = qopt->count[tc],
+ .offset = qopt->offset[tc],
+ };
+ }
+}
+
+static void mlx5e_params_mqprio_dcb_set(struct mlx5e_params *params, u8 num_tc)
+{
+ params->mqprio.mode = TC_MQPRIO_MODE_DCB;
+ params->mqprio.num_tc = num_tc;
+ mlx5e_mqprio_build_default_tc_to_txq(params->mqprio.tc_to_txq, num_tc,
+ params->num_channels);
+}
+
+static void mlx5e_params_mqprio_channel_set(struct mlx5e_params *params,
+ struct tc_mqprio_qopt *qopt)
+{
+ params->mqprio.mode = TC_MQPRIO_MODE_CHANNEL;
+ params->mqprio.num_tc = qopt->num_tc;
+ mlx5e_mqprio_build_tc_to_txq(params->mqprio.tc_to_txq, qopt);
+}
+
+static void mlx5e_params_mqprio_reset(struct mlx5e_params *params)
+{
+ mlx5e_params_mqprio_dcb_set(params, 1);
+}
+
static int mlx5e_setup_tc_mqprio_dcb(struct mlx5e_priv *priv,
struct tc_mqprio_qopt *mqprio)
{
@@ -2874,8 +2931,7 @@ static int mlx5e_setup_tc_mqprio_dcb(struct mlx5e_priv *priv,
return -EINVAL;
new_params = priv->channels.params;
- new_params.mqprio.mode = TC_MQPRIO_MODE_DCB;
- new_params.mqprio.num_tc = tc ? tc : 1;
+ mlx5e_params_mqprio_dcb_set(&new_params, tc ? tc : 1);
err = mlx5e_safe_switch_params(priv, &new_params,
mlx5e_num_channels_changed_ctx, NULL, true);
@@ -2926,25 +2982,12 @@ static int mlx5e_mqprio_channel_validate(struct mlx5e_priv *priv,
return 0;
}
-static int mlx5e_mqprio_channel_set_tcs_ctx(struct mlx5e_priv *priv, void *ctx)
-{
- struct tc_mqprio_qopt_offload *mqprio = (struct tc_mqprio_qopt_offload *)ctx;
- struct net_device *netdev = priv->netdev;
- u8 num_tc;
-
- if (priv->channels.params.mqprio.mode != TC_MQPRIO_MODE_CHANNEL)
- return -EINVAL;
-
- num_tc = priv->channels.params.mqprio.num_tc;
- mlx5e_netdev_set_tcs(netdev, 0, num_tc, mqprio);
-
- return 0;
-}
-
static int mlx5e_setup_tc_mqprio_channel(struct mlx5e_priv *priv,
struct tc_mqprio_qopt_offload *mqprio)
{
+ mlx5e_fp_preactivate preactivate;
struct mlx5e_params new_params;
+ bool nch_changed;
int err;
err = mlx5e_mqprio_channel_validate(priv, mqprio);
@@ -2952,12 +2995,12 @@ static int mlx5e_setup_tc_mqprio_channel(struct mlx5e_priv *priv,
return err;
new_params = priv->channels.params;
- new_params.mqprio.mode = TC_MQPRIO_MODE_CHANNEL;
- new_params.mqprio.num_tc = mqprio->qopt.num_tc;
- err = mlx5e_safe_switch_params(priv, &new_params,
- mlx5e_mqprio_channel_set_tcs_ctx, mqprio, true);
+ mlx5e_params_mqprio_channel_set(&new_params, &mqprio->qopt);
- return err;
+ nch_changed = mlx5e_get_dcb_num_tc(&priv->channels.params) > 1;
+ preactivate = nch_changed ? mlx5e_num_channels_changed_ctx :
+ mlx5e_update_netdev_queues_ctx;
+ return mlx5e_safe_switch_params(priv, &new_params, preactivate, NULL, true);
}
static int mlx5e_setup_tc_mqprio(struct mlx5e_priv *priv,
@@ -4190,7 +4233,7 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16
params->hard_mtu = MLX5E_ETH_HARD_MTU;
params->num_channels = min_t(unsigned int, MLX5E_MAX_NUM_CHANNELS / 2,
priv->max_nch);
- params->mqprio.num_tc = 1;
+ mlx5e_params_mqprio_reset(params);
/* Set an initial non-zero value, so that mlx5e_select_queue won't
* divide by zero if called before first activating channels.
@@ -4823,6 +4866,10 @@ int mlx5e_attach_netdev(struct mlx5e_priv *priv)
*/
priv->netdev->priv_flags &= ~IFF_RXFH_CONFIGURED;
priv->channels.params.num_channels = max_nch;
+ if (priv->channels.params.mqprio.mode == TC_MQPRIO_MODE_CHANNEL) {
+ mlx5_core_warn(priv->mdev, "MLX5E: Disabling MQPRIO channel mode\n");
+ mlx5e_params_mqprio_reset(&priv->channels.params);
+ }
}
if (max_nch != priv->max_nch) {
mlx5_core_warn(priv->mdev,
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* [net 04/10] net/mlx5: E-Switch, Fix double allocation of acl flow counter
2021-09-30 23:14 [pull request][net 00/10] mlx5 fixes 2021-09-30 Saeed Mahameed
` (2 preceding siblings ...)
2021-09-30 23:14 ` [net 03/10] net/mlx5e: Improve MQPRIO resiliency Saeed Mahameed
@ 2021-09-30 23:14 ` Saeed Mahameed
2021-09-30 23:14 ` [net 05/10] net/mlx5: Force round second at 1PPS out start time Saeed Mahameed
` (5 subsequent siblings)
9 siblings, 0 replies; 14+ messages in thread
From: Saeed Mahameed @ 2021-09-30 23:14 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Moshe Shemesh, Tariq Toukan, Saeed Mahameed
From: Moshe Shemesh <moshe@nvidia.com>
Flow counter is allocated in eswitch legacy acl setting functions
without checking if already allocated by previous setting. Add a check
to avoid such double allocation.
Fixes: 07bab9502641 ("net/mlx5: E-Switch, Refactor eswitch ingress acl codes")
Fixes: ea651a86d468 ("net/mlx5: E-Switch, Refactor eswitch egress acl codes")
Signed-off-by: Moshe Shemesh <moshe@nvidia.com>
Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
.../mellanox/mlx5/core/esw/acl/egress_lgcy.c | 12 ++++++++----
.../mellanox/mlx5/core/esw/acl/ingress_lgcy.c | 4 +++-
2 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_lgcy.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_lgcy.c
index 0399a396d166..60a73990017c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_lgcy.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_lgcy.c
@@ -79,12 +79,16 @@ int esw_acl_egress_lgcy_setup(struct mlx5_eswitch *esw,
int dest_num = 0;
int err = 0;
- if (MLX5_CAP_ESW_EGRESS_ACL(esw->dev, flow_counter)) {
+ if (vport->egress.legacy.drop_counter) {
+ drop_counter = vport->egress.legacy.drop_counter;
+ } else if (MLX5_CAP_ESW_EGRESS_ACL(esw->dev, flow_counter)) {
drop_counter = mlx5_fc_create(esw->dev, false);
- if (IS_ERR(drop_counter))
+ if (IS_ERR(drop_counter)) {
esw_warn(esw->dev,
"vport[%d] configure egress drop rule counter err(%ld)\n",
vport->vport, PTR_ERR(drop_counter));
+ drop_counter = NULL;
+ }
vport->egress.legacy.drop_counter = drop_counter;
}
@@ -123,7 +127,7 @@ int esw_acl_egress_lgcy_setup(struct mlx5_eswitch *esw,
flow_act.action = MLX5_FLOW_CONTEXT_ACTION_DROP;
/* Attach egress drop flow counter */
- if (!IS_ERR_OR_NULL(drop_counter)) {
+ if (drop_counter) {
flow_act.action |= MLX5_FLOW_CONTEXT_ACTION_COUNT;
drop_ctr_dst.type = MLX5_FLOW_DESTINATION_TYPE_COUNTER;
drop_ctr_dst.counter_id = mlx5_fc_id(drop_counter);
@@ -162,7 +166,7 @@ void esw_acl_egress_lgcy_cleanup(struct mlx5_eswitch *esw,
esw_acl_egress_table_destroy(vport);
clean_drop_counter:
- if (!IS_ERR_OR_NULL(vport->egress.legacy.drop_counter)) {
+ if (vport->egress.legacy.drop_counter) {
mlx5_fc_destroy(esw->dev, vport->egress.legacy.drop_counter);
vport->egress.legacy.drop_counter = NULL;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_lgcy.c b/drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_lgcy.c
index f75b86abaf1c..b1a5199260f6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_lgcy.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_lgcy.c
@@ -160,7 +160,9 @@ int esw_acl_ingress_lgcy_setup(struct mlx5_eswitch *esw,
esw_acl_ingress_lgcy_rules_destroy(vport);
- if (MLX5_CAP_ESW_INGRESS_ACL(esw->dev, flow_counter)) {
+ if (vport->ingress.legacy.drop_counter) {
+ counter = vport->ingress.legacy.drop_counter;
+ } else if (MLX5_CAP_ESW_INGRESS_ACL(esw->dev, flow_counter)) {
counter = mlx5_fc_create(esw->dev, false);
if (IS_ERR(counter)) {
esw_warn(esw->dev,
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* [net 05/10] net/mlx5: Force round second at 1PPS out start time
2021-09-30 23:14 [pull request][net 00/10] mlx5 fixes 2021-09-30 Saeed Mahameed
` (3 preceding siblings ...)
2021-09-30 23:14 ` [net 04/10] net/mlx5: E-Switch, Fix double allocation of acl flow counter Saeed Mahameed
@ 2021-09-30 23:14 ` Saeed Mahameed
2021-09-30 23:14 ` [net 06/10] net/mlx5: Avoid generating event after PPS out in Real time mode Saeed Mahameed
` (4 subsequent siblings)
9 siblings, 0 replies; 14+ messages in thread
From: Saeed Mahameed @ 2021-09-30 23:14 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Aya Levin, Eran Ben Elisha, Saeed Mahameed
From: Aya Levin <ayal@nvidia.com>
Allow configuration of 1PPS start time only with time-stamp representing
a round second. Prior to this patch driver allowed setting of a
non-round-second which is not supported by the device. Avoid unexpected
behavior by restricting start-time configuration to a round-second.
Fixes: 4272f9b88db9 ("net/mlx5e: Change 1PPS out scheme")
Signed-off-by: Aya Levin <ayal@nvidia.com>
Reviewed-by: Eran Ben Elisha <eranbe@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
.../ethernet/mellanox/mlx5/core/lib/clock.c | 25 ++++++++-----------
1 file changed, 11 insertions(+), 14 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
index ffac8a0e7a23..d2ed7b0a18ea 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
@@ -448,22 +448,20 @@ static u64 find_target_cycles(struct mlx5_core_dev *mdev, s64 target_ns)
return cycles_now + cycles_delta;
}
-static u64 perout_conf_internal_timer(struct mlx5_core_dev *mdev,
- s64 sec, u32 nsec)
+static u64 perout_conf_internal_timer(struct mlx5_core_dev *mdev, s64 sec)
{
- struct timespec64 ts;
+ struct timespec64 ts = {};
s64 target_ns;
ts.tv_sec = sec;
- ts.tv_nsec = nsec;
target_ns = timespec64_to_ns(&ts);
return find_target_cycles(mdev, target_ns);
}
-static u64 perout_conf_real_time(s64 sec, u32 nsec)
+static u64 perout_conf_real_time(s64 sec)
{
- return (u64)nsec | (u64)sec << 32;
+ return (u64)sec << 32;
}
static int mlx5_perout_configure(struct ptp_clock_info *ptp,
@@ -501,8 +499,10 @@ static int mlx5_perout_configure(struct ptp_clock_info *ptp,
if (on) {
bool rt_mode = mlx5_real_time_mode(mdev);
- u32 nsec;
- s64 sec;
+ s64 sec = rq->perout.start.sec;
+
+ if (rq->perout.start.nsec)
+ return -EINVAL;
pin_mode = MLX5_PIN_MODE_OUT;
pattern = MLX5_OUT_PATTERN_PERIODIC;
@@ -513,14 +513,11 @@ static int mlx5_perout_configure(struct ptp_clock_info *ptp,
if ((ns >> 1) != 500000000LL)
return -EINVAL;
- nsec = rq->perout.start.nsec;
- sec = rq->perout.start.sec;
-
if (rt_mode && sec > U32_MAX)
return -EINVAL;
- time_stamp = rt_mode ? perout_conf_real_time(sec, nsec) :
- perout_conf_internal_timer(mdev, sec, nsec);
+ time_stamp = rt_mode ? perout_conf_real_time(sec) :
+ perout_conf_internal_timer(mdev, sec);
field_select |= MLX5_MTPPS_FS_PIN_MODE |
MLX5_MTPPS_FS_PATTERN |
@@ -717,7 +714,7 @@ static u64 perout_conf_next_event_timer(struct mlx5_core_dev *mdev,
ts_next_sec(&ts);
target_ns = timespec64_to_ns(&ts);
- return rt_mode ? perout_conf_real_time(ts.tv_sec, ts.tv_nsec) :
+ return rt_mode ? perout_conf_real_time(ts.tv_sec) :
find_target_cycles(mdev, target_ns);
}
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* [net 06/10] net/mlx5: Avoid generating event after PPS out in Real time mode
2021-09-30 23:14 [pull request][net 00/10] mlx5 fixes 2021-09-30 Saeed Mahameed
` (4 preceding siblings ...)
2021-09-30 23:14 ` [net 05/10] net/mlx5: Force round second at 1PPS out start time Saeed Mahameed
@ 2021-09-30 23:14 ` Saeed Mahameed
2021-09-30 23:14 ` [net 07/10] net/mlx5: Fix length of irq_index in chars Saeed Mahameed
` (3 subsequent siblings)
9 siblings, 0 replies; 14+ messages in thread
From: Saeed Mahameed @ 2021-09-30 23:14 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Aya Levin, Eran Ben Elisha, Saeed Mahameed
From: Aya Levin <ayal@nvidia.com>
When in Real-time mode, HW clock is synced with the PTP daemon. Hence
driver should not re-calibrate the next pulse (via MTPPSE repetitive
events mechanism).
This patch arms repetitive events only in free-running mode.
Fixes: 432119de33d9 ("net/mlx5: Add cyc2time HW translation mode support")
Signed-off-by: Aya Levin <ayal@nvidia.com>
Reviewed-by: Eran Ben Elisha <eranbe@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
.../net/ethernet/mellanox/mlx5/core/lib/clock.c | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
index d2ed7b0a18ea..91e806c1aa21 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c
@@ -472,6 +472,7 @@ static int mlx5_perout_configure(struct ptp_clock_info *ptp,
container_of(ptp, struct mlx5_clock, ptp_info);
struct mlx5_core_dev *mdev =
container_of(clock, struct mlx5_core_dev, clock);
+ bool rt_mode = mlx5_real_time_mode(mdev);
u32 in[MLX5_ST_SZ_DW(mtpps_reg)] = {0};
struct timespec64 ts;
u32 field_select = 0;
@@ -535,6 +536,9 @@ static int mlx5_perout_configure(struct ptp_clock_info *ptp,
if (err)
return err;
+ if (rt_mode)
+ return 0;
+
return mlx5_set_mtppse(mdev, pin, 0,
MLX5_EVENT_MODE_REPETETIVE & on);
}
@@ -702,20 +706,14 @@ static void ts_next_sec(struct timespec64 *ts)
static u64 perout_conf_next_event_timer(struct mlx5_core_dev *mdev,
struct mlx5_clock *clock)
{
- bool rt_mode = mlx5_real_time_mode(mdev);
struct timespec64 ts;
s64 target_ns;
- if (rt_mode)
- ts = mlx5_ptp_gettimex_real_time(mdev, NULL);
- else
- mlx5_ptp_gettimex(&clock->ptp_info, &ts, NULL);
-
+ mlx5_ptp_gettimex(&clock->ptp_info, &ts, NULL);
ts_next_sec(&ts);
target_ns = timespec64_to_ns(&ts);
- return rt_mode ? perout_conf_real_time(ts.tv_sec) :
- find_target_cycles(mdev, target_ns);
+ return find_target_cycles(mdev, target_ns);
}
static int mlx5_pps_event(struct notifier_block *nb,
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* [net 07/10] net/mlx5: Fix length of irq_index in chars
2021-09-30 23:14 [pull request][net 00/10] mlx5 fixes 2021-09-30 Saeed Mahameed
` (5 preceding siblings ...)
2021-09-30 23:14 ` [net 06/10] net/mlx5: Avoid generating event after PPS out in Real time mode Saeed Mahameed
@ 2021-09-30 23:14 ` Saeed Mahameed
2021-09-30 23:14 ` [net 08/10] net/mlx5: Fix setting number of EQs of SFs Saeed Mahameed
` (2 subsequent siblings)
9 siblings, 0 replies; 14+ messages in thread
From: Saeed Mahameed @ 2021-09-30 23:14 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Shay Drory, Parav Pandit, Saeed Mahameed
From: Shay Drory <shayd@nvidia.com>
The maximum irq_index can be 2047, This means irq_name should have 4
characters reserve for the irq_index. Hence, increase it to 4.
Fixes: 3af26495a247 ("net/mlx5: Enlarge interrupt field in CREATE_EQ")
Signed-off-by: Shay Drory <shayd@nvidia.com>
Reviewed-by: Parav Pandit <parav@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c
index c79a10b3454d..df54f62a38ac 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c
@@ -13,8 +13,8 @@
#endif
#define MLX5_MAX_IRQ_NAME (32)
-/* max irq_index is 255. three chars */
-#define MLX5_MAX_IRQ_IDX_CHARS (3)
+/* max irq_index is 2047, so four chars */
+#define MLX5_MAX_IRQ_IDX_CHARS (4)
#define MLX5_SFS_PER_CTRL_IRQ 64
#define MLX5_IRQ_CTRL_SF_MAX 8
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* [net 08/10] net/mlx5: Fix setting number of EQs of SFs
2021-09-30 23:14 [pull request][net 00/10] mlx5 fixes 2021-09-30 Saeed Mahameed
` (6 preceding siblings ...)
2021-09-30 23:14 ` [net 07/10] net/mlx5: Fix length of irq_index in chars Saeed Mahameed
@ 2021-09-30 23:14 ` Saeed Mahameed
2021-09-30 23:15 ` [net 09/10] net/mlx5e: Fix the presented RQ index in PTP stats Saeed Mahameed
2021-09-30 23:15 ` [net 10/10] net/mlx5e: Mutually exclude setting of TX-port-TS and MQPRIO in channel mode Saeed Mahameed
9 siblings, 0 replies; 14+ messages in thread
From: Saeed Mahameed @ 2021-09-30 23:14 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Shay Drory, Parav Pandit, Saeed Mahameed
From: Shay Drory <shayd@nvidia.com>
When setting number of completion EQs of the SF, consider number of
online CPUs.
Without this consideration, when number of online cpus are less than 8,
unnecessary 8 completion EQs are allocated.
Fixes: c36326d38d93 ("net/mlx5: Round-Robin EQs over IRQs")
Signed-off-by: Shay Drory <shayd@nvidia.com>
Reviewed-by: Parav Pandit <parav@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c
index df54f62a38ac..763c83a02380 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c
@@ -633,8 +633,9 @@ void mlx5_irq_table_destroy(struct mlx5_core_dev *dev)
int mlx5_irq_table_get_sfs_vec(struct mlx5_irq_table *table)
{
if (table->sf_comp_pool)
- return table->sf_comp_pool->xa_num_irqs.max -
- table->sf_comp_pool->xa_num_irqs.min + 1;
+ return min_t(int, num_online_cpus(),
+ table->sf_comp_pool->xa_num_irqs.max -
+ table->sf_comp_pool->xa_num_irqs.min + 1);
else
return mlx5_irq_table_get_num_comp(table);
}
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* [net 09/10] net/mlx5e: Fix the presented RQ index in PTP stats
2021-09-30 23:14 [pull request][net 00/10] mlx5 fixes 2021-09-30 Saeed Mahameed
` (7 preceding siblings ...)
2021-09-30 23:14 ` [net 08/10] net/mlx5: Fix setting number of EQs of SFs Saeed Mahameed
@ 2021-09-30 23:15 ` Saeed Mahameed
2021-09-30 23:15 ` [net 10/10] net/mlx5e: Mutually exclude setting of TX-port-TS and MQPRIO in channel mode Saeed Mahameed
9 siblings, 0 replies; 14+ messages in thread
From: Saeed Mahameed @ 2021-09-30 23:15 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Lama Kayal, Tariq Toukan, Saeed Mahameed
From: Lama Kayal <lkayal@nvidia.com>
PTP-RQ counters title format contains PTP-RQ identifier, which is
mistakenly not passed to sprinft().
This leads to unexpected garbage values instead.
This patch fixes it.
Before applying the patch:
ethtool -S eth3 | grep ptp_rq
ptp_rq15_packets: 0
ptp_rq8_bytes: 0
ptp_rq6_csum_complete: 0
ptp_rq14_csum_complete_tail: 0
ptp_rq3_csum_complete_tail_slow : 0
ptp_rq9_csum_unnecessary: 0
ptp_rq1_csum_unnecessary_inner: 0
ptp_rq7_csum_none: 0
ptp_rq10_xdp_drop: 0
ptp_rq9_xdp_redirect: 0
ptp_rq13_lro_packets: 0
ptp_rq12_lro_bytes: 0
ptp_rq10_ecn_mark: 0
ptp_rq9_removed_vlan_packets: 0
ptp_rq5_wqe_err: 0
ptp_rq8_mpwqe_filler_cqes: 0
ptp_rq2_mpwqe_filler_strides: 0
ptp_rq5_oversize_pkts_sw_drop: 0
ptp_rq6_buff_alloc_err: 0
ptp_rq15_cqe_compress_blks: 0
ptp_rq2_cqe_compress_pkts: 0
ptp_rq2_cache_reuse: 0
ptp_rq12_cache_full: 0
ptp_rq11_cache_empty: 256
ptp_rq12_cache_busy: 0
ptp_rq11_cache_waive: 0
ptp_rq12_congst_umr: 0
ptp_rq11_arfs_err: 0
ptp_rq9_recover: 0
After applying the patch:
ethtool -S eth3 | grep ptp_rq
ptp_rq0_packets: 0
ptp_rq0_bytes: 0
ptp_rq0_csum_complete: 0
ptp_rq0_csum_complete_tail: 0
ptp_rq0_csum_complete_tail_slow : 0
ptp_rq0_csum_unnecessary: 0
ptp_rq0_csum_unnecessary_inner: 0
ptp_rq0_csum_none: 0
ptp_rq0_xdp_drop: 0
ptp_rq0_xdp_redirect: 0
ptp_rq0_lro_packets: 0
ptp_rq0_lro_bytes: 0
ptp_rq0_ecn_mark: 0
ptp_rq0_removed_vlan_packets: 0
ptp_rq0_wqe_err: 0
ptp_rq0_mpwqe_filler_cqes: 0
ptp_rq0_mpwqe_filler_strides: 0
ptp_rq0_oversize_pkts_sw_drop: 0
ptp_rq0_buff_alloc_err: 0
ptp_rq0_cqe_compress_blks: 0
ptp_rq0_cqe_compress_pkts: 0
ptp_rq0_cache_reuse: 0
ptp_rq0_cache_full: 0
ptp_rq0_cache_empty: 256
ptp_rq0_cache_busy: 0
ptp_rq0_cache_waive: 0
ptp_rq0_congst_umr: 0
ptp_rq0_arfs_err: 0
ptp_rq0_recover: 0
Fixes: a28359e922c6 ("net/mlx5e: Add PTP-RX statistics")
Signed-off-by: Lama Kayal <lkayal@nvidia.com>
Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c | 3 +--
drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h | 2 ++
drivers/net/ethernet/mellanox/mlx5/core/en_stats.c | 3 ++-
3 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c
index ee688dec67a9..3a86f66d1295 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c
@@ -13,8 +13,6 @@ struct mlx5e_ptp_fs {
bool valid;
};
-#define MLX5E_PTP_CHANNEL_IX 0
-
struct mlx5e_ptp_params {
struct mlx5e_params params;
struct mlx5e_sq_param txq_sq_param;
@@ -509,6 +507,7 @@ static int mlx5e_init_ptp_rq(struct mlx5e_ptp *c, struct mlx5e_params *params,
rq->mdev = mdev;
rq->hw_mtu = MLX5E_SW2HW_MTU(params, params->sw_mtu);
rq->stats = &c->priv->ptp_stats.rq;
+ rq->ix = MLX5E_PTP_CHANNEL_IX;
rq->ptp_cyc2time = mlx5_rq_ts_translator(mdev);
err = mlx5e_rq_set_handlers(rq, params, false);
if (err)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h
index c96668bd701c..a71a32e00ebb 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h
@@ -8,6 +8,8 @@
#include "en_stats.h"
#include <linux/ptp_classify.h>
+#define MLX5E_PTP_CHANNEL_IX 0
+
struct mlx5e_ptpsq {
struct mlx5e_txqsq txqsq;
struct mlx5e_cq ts_cq;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
index 46bf78169f63..e1dd17019030 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c
@@ -34,6 +34,7 @@
#include "en.h"
#include "en_accel/tls.h"
#include "en_accel/en_accel.h"
+#include "en/ptp.h"
static unsigned int stats_grps_num(struct mlx5e_priv *priv)
{
@@ -2076,7 +2077,7 @@ static MLX5E_DECLARE_STATS_GRP_OP_FILL_STRS(ptp)
if (priv->rx_ptp_opened) {
for (i = 0; i < NUM_PTP_RQ_STATS; i++)
sprintf(data + (idx++) * ETH_GSTRING_LEN,
- ptp_rq_stats_desc[i].format);
+ ptp_rq_stats_desc[i].format, MLX5E_PTP_CHANNEL_IX);
}
return idx;
}
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread* [net 10/10] net/mlx5e: Mutually exclude setting of TX-port-TS and MQPRIO in channel mode
2021-09-30 23:14 [pull request][net 00/10] mlx5 fixes 2021-09-30 Saeed Mahameed
` (8 preceding siblings ...)
2021-09-30 23:15 ` [net 09/10] net/mlx5e: Fix the presented RQ index in PTP stats Saeed Mahameed
@ 2021-09-30 23:15 ` Saeed Mahameed
9 siblings, 0 replies; 14+ messages in thread
From: Saeed Mahameed @ 2021-09-30 23:15 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Aya Levin, Tariq Toukan, Saeed Mahameed
From: Aya Levin <ayal@nvidia.com>
TX-port-TS hijacks the PTP traffic to a specific HW TX-queue. This
conflicts with MQPRIO in channel mode, which specifies explicitly which
TC accepts the packet. This patch mutually excludes the above
configuration.
Fixes: ec60c4581bd9 ("net/mlx5e: Support MQPRIO channel mode")
Signed-off-by: Aya Levin <ayal@nvidia.com>
Reviewed-by: Tariq Toukan <tariqt@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c | 11 +++++++++++
drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 8 ++++++++
2 files changed, 19 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
index 306fb5d6a36d..9d451b8ee467 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
@@ -2036,6 +2036,17 @@ static int set_pflag_tx_port_ts(struct net_device *netdev, bool enable)
}
new_params = priv->channels.params;
+ /* Don't allow enabling TX-port-TS if MQPRIO mode channel offload is
+ * active, since it defines explicitly which TC accepts the packet.
+ * This conflicts with TX-port-TS hijacking the PTP traffic to a specific
+ * HW TX-queue.
+ */
+ if (enable && new_params.mqprio.mode == TC_MQPRIO_MODE_CHANNEL) {
+ netdev_err(priv->netdev,
+ "%s: MQPRIO mode channel offload is active, cannot set the TX-port-TS\n",
+ __func__);
+ return -EINVAL;
+ }
MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_TX_PORT_TS, enable);
/* No need to verify SQ stop room as
* ptpsq.txqsq.stop_room <= generic_sq->stop_room, and both
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 0390395f421f..0c5197f9cea3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -2945,9 +2945,17 @@ static int mlx5e_mqprio_channel_validate(struct mlx5e_priv *priv,
struct tc_mqprio_qopt_offload *mqprio)
{
struct net_device *netdev = priv->netdev;
+ struct mlx5e_ptp *ptp_channel;
int agg_count = 0;
int i;
+ ptp_channel = priv->channels.ptp;
+ if (ptp_channel && test_bit(MLX5E_PTP_STATE_TX, ptp_channel->state)) {
+ netdev_err(netdev,
+ "Cannot activate MQPRIO mode channel since it conflicts with TX port TS\n");
+ return -EINVAL;
+ }
+
if (mqprio->qopt.offset[0] != 0 || mqprio->qopt.num_tc < 1 ||
mqprio->qopt.num_tc > MLX5E_MAX_NUM_MQPRIO_CH_TC)
return -EINVAL;
--
2.31.1
^ permalink raw reply related [flat|nested] 14+ messages in thread