* [PATCH rdma-next 0/2] RoCE ICRC counter
@ 2018-06-21 12:37 Leon Romanovsky
2018-06-21 12:37 ` [PATCH mlx5-next 1/2] net/mlx5: Add RoCE RX ICRC encapsulated counter Leon Romanovsky
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Leon Romanovsky @ 2018-06-21 12:37 UTC (permalink / raw)
To: Doug Ledford, Jason Gunthorpe
Cc: Leon Romanovsky, RDMA mailing list, Mark Bloch, Talat Batheesh,
Saeed Mahameed, linux-netdev
From: Leon Romanovsky <leonro@mellanox.com>
Hi,
This series exposes RoCE ICRC counter through existing RDMA hw_counters
sysfs interface.
First patch has all HW definitions in mlx5_ifc.h file and second patch is
actual counter implementation.
Thanks
Talat Batheesh (2):
net/mlx5: Add RoCE RX ICRC encapsulated counter
IB/mlx5: Support RoCE ICRC encapsulated error counter
drivers/infiniband/hw/mlx5/cmd.c | 12 +++++++
drivers/infiniband/hw/mlx5/cmd.h | 1 +
drivers/infiniband/hw/mlx5/main.c | 62 ++++++++++++++++++++++++++++++++++--
drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 +
include/linux/mlx5/mlx5_ifc.h | 11 +++++--
5 files changed, 81 insertions(+), 6 deletions(-)
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH mlx5-next 1/2] net/mlx5: Add RoCE RX ICRC encapsulated counter 2018-06-21 12:37 [PATCH rdma-next 0/2] RoCE ICRC counter Leon Romanovsky @ 2018-06-21 12:37 ` Leon Romanovsky 2018-06-21 17:53 ` Leon Romanovsky 2018-06-21 12:37 ` [PATCH rdma-next 2/2] IB/mlx5: Support RoCE ICRC encapsulated error counter Leon Romanovsky ` (2 subsequent siblings) 3 siblings, 1 reply; 6+ messages in thread From: Leon Romanovsky @ 2018-06-21 12:37 UTC (permalink / raw) To: Doug Ledford, Jason Gunthorpe Cc: Leon Romanovsky, RDMA mailing list, Mark Bloch, Talat Batheesh, Saeed Mahameed, linux-netdev From: Talat Batheesh <talatb@mellanox.com> Add capability bit in PCAM register and RoCE ICRC error counter to PPCNT register. Signed-off-by: Talat Batheesh <talatb@mellanox.com> Reviewed-by: Mark Bloch <markb@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@mellanox.com> --- include/linux/mlx5/mlx5_ifc.h | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index b4302ccb63a6..9e8682489951 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -1687,7 +1687,11 @@ struct mlx5_ifc_eth_extended_cntrs_grp_data_layout_bits { u8 rx_buffer_full_low[0x20]; - u8 reserved_at_1c0[0x600]; + u8 rx_icrc_encapsulated_high[0x20]; + + u8 rx_icrc_encapsulated_low[0x20]; + + u8 reserved_at_3c0[0x5c0]; }; struct mlx5_ifc_eth_3635_cntrs_grp_data_layout_bits { @@ -8050,8 +8054,9 @@ struct mlx5_ifc_peir_reg_bits { }; struct mlx5_ifc_pcam_enhanced_features_bits { - u8 reserved_at_0[0x76]; - + u8 reserved_at_0[0x6d]; + u8 rx_icrc_encapsulated_counter[0x1]; + u8 reserved_at_6e[0x8]; u8 pfcc_mask[0x1]; u8 reserved_at_77[0x4]; u8 rx_buffer_fullness_counters[0x1]; -- 2.14.4 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH mlx5-next 1/2] net/mlx5: Add RoCE RX ICRC encapsulated counter 2018-06-21 12:37 ` [PATCH mlx5-next 1/2] net/mlx5: Add RoCE RX ICRC encapsulated counter Leon Romanovsky @ 2018-06-21 17:53 ` Leon Romanovsky 0 siblings, 0 replies; 6+ messages in thread From: Leon Romanovsky @ 2018-06-21 17:53 UTC (permalink / raw) To: Doug Ledford, Jason Gunthorpe Cc: RDMA mailing list, Mark Bloch, Talat Batheesh, Saeed Mahameed, linux-netdev [-- Attachment #1: Type: text/plain, Size: 1117 bytes --] On Thu, Jun 21, 2018 at 03:37:55PM +0300, Leon Romanovsky wrote: > From: Talat Batheesh <talatb@mellanox.com> > > Add capability bit in PCAM register and RoCE ICRC error counter > to PPCNT register. > > Signed-off-by: Talat Batheesh <talatb@mellanox.com> > Reviewed-by: Mark Bloch <markb@mellanox.com> > Signed-off-by: Leon Romanovsky <leonro@mellanox.com> > --- > include/linux/mlx5/mlx5_ifc.h | 11 ++++++++--- > 1 file changed, 8 insertions(+), 3 deletions(-) > > diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h > index b4302ccb63a6..9e8682489951 100644 > --- a/include/linux/mlx5/mlx5_ifc.h > +++ b/include/linux/mlx5/mlx5_ifc.h > @@ -1687,7 +1687,11 @@ struct mlx5_ifc_eth_extended_cntrs_grp_data_layout_bits { > > u8 rx_buffer_full_low[0x20]; > > - u8 reserved_at_1c0[0x600]; > + u8 rx_icrc_encapsulated_high[0x20]; > + > + u8 rx_icrc_encapsulated_low[0x20]; > + > + u8 reserved_at_3c0[0x5c0]; reserved_at_3c0 should be reserved_at_200, fixed and applied to mlx5-next. Commit 0af5107cd0640ee3424e337b492e4b11b450ce28 in mlx5-next. Thanks [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 801 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH rdma-next 2/2] IB/mlx5: Support RoCE ICRC encapsulated error counter 2018-06-21 12:37 [PATCH rdma-next 0/2] RoCE ICRC counter Leon Romanovsky 2018-06-21 12:37 ` [PATCH mlx5-next 1/2] net/mlx5: Add RoCE RX ICRC encapsulated counter Leon Romanovsky @ 2018-06-21 12:37 ` Leon Romanovsky 2018-06-21 17:43 ` [PATCH rdma-next 0/2] RoCE ICRC counter Jason Gunthorpe 2018-06-22 15:03 ` Jason Gunthorpe 3 siblings, 0 replies; 6+ messages in thread From: Leon Romanovsky @ 2018-06-21 12:37 UTC (permalink / raw) To: Doug Ledford, Jason Gunthorpe Cc: Leon Romanovsky, RDMA mailing list, Mark Bloch, Talat Batheesh, Saeed Mahameed, linux-netdev From: Talat Batheesh <talatb@mellanox.com> This patch adds support to query the counter that counts the RoCE packets with corrupted ICRC (Invariant Cyclic Redundancy Code). This counter will be under /sys/class/infiniband/<mlx5-dev>/ports/<port>/hw_counters/ rx_icrc_encapsulated - The number of RoCE packets with ICRC error. Signed-off-by: Talat Batheesh <talatb@mellanox.com> Reviewed-by: Mark Bloch <markb@mellanox.com> Signed-off-by: Leon Romanovsky <leonro@mellanox.com> --- drivers/infiniband/hw/mlx5/cmd.c | 12 +++++++ drivers/infiniband/hw/mlx5/cmd.h | 1 + drivers/infiniband/hw/mlx5/main.c | 62 ++++++++++++++++++++++++++++++++++-- drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 + 4 files changed, 73 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/cmd.c b/drivers/infiniband/hw/mlx5/cmd.c index ccc0b5d06a7d..c84fef9a8a08 100644 --- a/drivers/infiniband/hw/mlx5/cmd.c +++ b/drivers/infiniband/hw/mlx5/cmd.c @@ -185,3 +185,15 @@ int mlx5_cmd_dealloc_memic(struct mlx5_memic *memic, u64 addr, u64 length) return err; } + +int mlx5_cmd_query_ext_ppcnt_counters(struct mlx5_core_dev *dev, void *out) +{ + u32 in[MLX5_ST_SZ_DW(ppcnt_reg)] = {}; + int sz = MLX5_ST_SZ_BYTES(ppcnt_reg); + + MLX5_SET(ppcnt_reg, in, local_port, 1); + + MLX5_SET(ppcnt_reg, in, grp, MLX5_ETHERNET_EXTENDED_COUNTERS_GROUP); + return mlx5_core_access_reg(dev, in, sz, out, sz, MLX5_REG_PPCNT, + 0, 0); +} diff --git a/drivers/infiniband/hw/mlx5/cmd.h b/drivers/infiniband/hw/mlx5/cmd.h index 98ea4648c655..88cbb1c41703 100644 --- a/drivers/infiniband/hw/mlx5/cmd.h +++ b/drivers/infiniband/hw/mlx5/cmd.h @@ -41,6 +41,7 @@ int mlx5_cmd_dump_fill_mkey(struct mlx5_core_dev *dev, u32 *mkey); int mlx5_cmd_null_mkey(struct mlx5_core_dev *dev, u32 *null_mkey); int mlx5_cmd_query_cong_params(struct mlx5_core_dev *dev, int cong_point, void *out, int out_size); +int mlx5_cmd_query_ext_ppcnt_counters(struct mlx5_core_dev *dev, void *out); int mlx5_cmd_modify_cong_params(struct mlx5_core_dev *mdev, void *in, int in_size); int mlx5_cmd_alloc_memic(struct mlx5_memic *memic, phys_addr_t *addr, diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index 7c3956f8e0f3..32eae0d651cf 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -4691,6 +4691,15 @@ static const struct mlx5_ib_counter extended_err_cnts[] = { INIT_Q_COUNTER(req_cqe_flush_error), }; +#define INIT_EXT_PPCNT_COUNTER(_name) \ + { .name = #_name, .offset = \ + MLX5_BYTE_OFF(ppcnt_reg, \ + counter_set.eth_extended_cntrs_grp_data_layout._name##_high)} + +static const struct mlx5_ib_counter ext_ppcnt_cnts[] = { + INIT_EXT_PPCNT_COUNTER(rx_icrc_encapsulated), +}; + static void mlx5_ib_dealloc_counters(struct mlx5_ib_dev *dev) { int i; @@ -4726,7 +4735,10 @@ static int __mlx5_ib_alloc_counters(struct mlx5_ib_dev *dev, cnts->num_cong_counters = ARRAY_SIZE(cong_cnts); num_counters += ARRAY_SIZE(cong_cnts); } - + if (MLX5_CAP_PCAM_FEATURE(dev->mdev, rx_icrc_encapsulated_counter)) { + cnts->num_ext_ppcnt_counters = ARRAY_SIZE(ext_ppcnt_cnts); + num_counters += ARRAY_SIZE(ext_ppcnt_cnts); + } cnts->names = kcalloc(num_counters, sizeof(cnts->names), GFP_KERNEL); if (!cnts->names) return -ENOMEM; @@ -4783,6 +4795,13 @@ static void mlx5_ib_fill_counters(struct mlx5_ib_dev *dev, offsets[j] = cong_cnts[i].offset; } } + + if (MLX5_CAP_PCAM_FEATURE(dev->mdev, rx_icrc_encapsulated_counter)) { + for (i = 0; i < ARRAY_SIZE(ext_ppcnt_cnts); i++, j++) { + names[j] = ext_ppcnt_cnts[i].name; + offsets[j] = ext_ppcnt_cnts[i].offset; + } + } } static int mlx5_ib_alloc_counters(struct mlx5_ib_dev *dev) @@ -4828,7 +4847,8 @@ static struct rdma_hw_stats *mlx5_ib_alloc_hw_stats(struct ib_device *ibdev, return rdma_alloc_hw_stats_struct(port->cnts.names, port->cnts.num_q_counters + - port->cnts.num_cong_counters, + port->cnts.num_cong_counters + + port->cnts.num_ext_ppcnt_counters, RDMA_HW_STATS_DEFAULT_LIFESPAN); } @@ -4861,6 +4881,34 @@ static int mlx5_ib_query_q_counters(struct mlx5_core_dev *mdev, return ret; } +static int mlx5_ib_query_ext_ppcnt_counters(struct mlx5_ib_dev *dev, + struct mlx5_ib_port *port, + struct rdma_hw_stats *stats) +{ + int offset = port->cnts.num_q_counters + port->cnts.num_cong_counters; + int sz = MLX5_ST_SZ_BYTES(ppcnt_reg); + int ret, i; + void *out; + + out = kvzalloc(sz, GFP_KERNEL); + if (!out) + return -ENOMEM; + + ret = mlx5_cmd_query_ext_ppcnt_counters(dev->mdev, out); + if (ret) + goto free; + + for (i = 0; i < port->cnts.num_ext_ppcnt_counters; i++) { + stats->value[i + offset] = + be64_to_cpup((__be64 *)(out + + port->cnts.offsets[i + offset])); + } + +free: + kvfree(out); + return ret; +} + static int mlx5_ib_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats, u8 port_num, int index) @@ -4874,13 +4922,21 @@ static int mlx5_ib_get_hw_stats(struct ib_device *ibdev, if (!stats) return -EINVAL; - num_counters = port->cnts.num_q_counters + port->cnts.num_cong_counters; + num_counters = port->cnts.num_q_counters + + port->cnts.num_cong_counters + + port->cnts.num_ext_ppcnt_counters; /* q_counters are per IB device, query the master mdev */ ret = mlx5_ib_query_q_counters(dev->mdev, port, stats); if (ret) return ret; + if (MLX5_CAP_PCAM_FEATURE(dev->mdev, rx_icrc_encapsulated_counter)) { + ret = mlx5_ib_query_ext_ppcnt_counters(dev, port, stats); + if (ret) + return ret; + } + if (MLX5_CAP_GEN(dev->mdev, cc_query_allowed)) { mdev = mlx5_ib_get_native_port_mdev(dev, port_num, &mdev_port_num); diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index 0f95453b11db..67e86c8304a2 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -666,6 +666,7 @@ struct mlx5_ib_counters { size_t *offsets; u32 num_q_counters; u32 num_cong_counters; + u32 num_ext_ppcnt_counters; u16 set_id; bool set_id_valid; }; -- 2.14.4 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH rdma-next 0/2] RoCE ICRC counter 2018-06-21 12:37 [PATCH rdma-next 0/2] RoCE ICRC counter Leon Romanovsky 2018-06-21 12:37 ` [PATCH mlx5-next 1/2] net/mlx5: Add RoCE RX ICRC encapsulated counter Leon Romanovsky 2018-06-21 12:37 ` [PATCH rdma-next 2/2] IB/mlx5: Support RoCE ICRC encapsulated error counter Leon Romanovsky @ 2018-06-21 17:43 ` Jason Gunthorpe 2018-06-22 15:03 ` Jason Gunthorpe 3 siblings, 0 replies; 6+ messages in thread From: Jason Gunthorpe @ 2018-06-21 17:43 UTC (permalink / raw) To: Leon Romanovsky Cc: Doug Ledford, Leon Romanovsky, RDMA mailing list, Mark Bloch, Talat Batheesh, Saeed Mahameed, linux-netdev On Thu, Jun 21, 2018 at 03:37:54PM +0300, Leon Romanovsky wrote: > From: Leon Romanovsky <leonro@mellanox.com> > > Hi, > > This series exposes RoCE ICRC counter through existing RDMA hw_counters > sysfs interface. > > First patch has all HW definitions in mlx5_ifc.h file and second patch is > actual counter implementation. The RDMA parts are OK, can you please send me the commit for the mlx5 patch when applied? Thanks, Jason ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH rdma-next 0/2] RoCE ICRC counter 2018-06-21 12:37 [PATCH rdma-next 0/2] RoCE ICRC counter Leon Romanovsky ` (2 preceding siblings ...) 2018-06-21 17:43 ` [PATCH rdma-next 0/2] RoCE ICRC counter Jason Gunthorpe @ 2018-06-22 15:03 ` Jason Gunthorpe 3 siblings, 0 replies; 6+ messages in thread From: Jason Gunthorpe @ 2018-06-22 15:03 UTC (permalink / raw) To: Leon Romanovsky Cc: Doug Ledford, Leon Romanovsky, RDMA mailing list, Mark Bloch, Talat Batheesh, Saeed Mahameed, linux-netdev On Thu, Jun 21, 2018 at 03:37:54PM +0300, Leon Romanovsky wrote: > From: Leon Romanovsky <leonro@mellanox.com> > > Hi, > > This series exposes RoCE ICRC counter through existing RDMA hw_counters > sysfs interface. > > First patch has all HW definitions in mlx5_ifc.h file and second patch is > actual counter implementation. > > Thanks > > Talat Batheesh (2): > net/mlx5: Add RoCE RX ICRC encapsulated counter > IB/mlx5: Support RoCE ICRC encapsulated error counter > > drivers/infiniband/hw/mlx5/cmd.c | 12 +++++++ > drivers/infiniband/hw/mlx5/cmd.h | 1 + > drivers/infiniband/hw/mlx5/main.c | 62 ++++++++++++++++++++++++++++++++++-- > drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 + > include/linux/mlx5/mlx5_ifc.h | 11 +++++-- > 5 files changed, 81 insertions(+), 6 deletions(-) Applied to rdma for-next with the mellanox/mlx5-next branch Thanks, Jason ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-06-22 15:03 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2018-06-21 12:37 [PATCH rdma-next 0/2] RoCE ICRC counter Leon Romanovsky 2018-06-21 12:37 ` [PATCH mlx5-next 1/2] net/mlx5: Add RoCE RX ICRC encapsulated counter Leon Romanovsky 2018-06-21 17:53 ` Leon Romanovsky 2018-06-21 12:37 ` [PATCH rdma-next 2/2] IB/mlx5: Support RoCE ICRC encapsulated error counter Leon Romanovsky 2018-06-21 17:43 ` [PATCH rdma-next 0/2] RoCE ICRC counter Jason Gunthorpe 2018-06-22 15:03 ` Jason Gunthorpe
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).