* [PATCH net v2 0/2] mlx5: bugfixes for ptp fifo queue @ 2023-01-24 0:08 Vadim Fedorenko 2023-01-24 0:08 ` [PATCH net v2 1/2] mlx5: fix possible ptp queue fifo overflow Vadim Fedorenko 2023-01-24 0:08 ` [PATCH net v2 2/2] mlx5: fix skb leak while fifo resync Vadim Fedorenko 0 siblings, 2 replies; 7+ messages in thread From: Vadim Fedorenko @ 2023-01-24 0:08 UTC (permalink / raw) To: Vadim Fedorenko, Aya Levin, Saeed Mahameed, Jakub Kicinski, Gal Pressman Cc: Vadim Fedorenko, netdev From: Vadim Fedorenko <vadfed@meta.com> Simple FIFO implementation for PTP queue has several bug which lead to use-after-free and skb leaks. This series fixes the issues and adds new counters of out-of-order CQEs for this queue. v1 -> v2: Update Fixes tag to proper commit. Change debug line to avoid double print of function name Vadim Fedorenko (2): mlx5: fix possible ptp queue fifo overflow mlx5: fix skb leak while fifo resync .../net/ethernet/mellanox/mlx5/core/en/ptp.c | 29 ++++++++++++++----- .../net/ethernet/mellanox/mlx5/core/en/txrx.h | 6 +++- .../ethernet/mellanox/mlx5/core/en_stats.c | 2 ++ .../ethernet/mellanox/mlx5/core/en_stats.h | 2 ++ 4 files changed, 31 insertions(+), 8 deletions(-) -- 2.27.0 ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH net v2 1/2] mlx5: fix possible ptp queue fifo overflow 2023-01-24 0:08 [PATCH net v2 0/2] mlx5: bugfixes for ptp fifo queue Vadim Fedorenko @ 2023-01-24 0:08 ` Vadim Fedorenko 2023-01-24 2:05 ` Rahul Rameshbabu 2023-01-24 4:19 ` Jakub Kicinski 2023-01-24 0:08 ` [PATCH net v2 2/2] mlx5: fix skb leak while fifo resync Vadim Fedorenko 1 sibling, 2 replies; 7+ messages in thread From: Vadim Fedorenko @ 2023-01-24 0:08 UTC (permalink / raw) To: Vadim Fedorenko, Aya Levin, Saeed Mahameed, Jakub Kicinski, Gal Pressman Cc: Vadim Fedorenko, netdev From: Vadim Fedorenko <vadfed@meta.com> Fifo pointers are not checked for overflow and this could potentially lead to overflow and double free under heavy PTP traffic. Also there were accidental OOO cqe which lead to absolutely broken fifo. Add checks to workaround OOO cqe and add counters to show the amount of such events. Fixes: 58a518948f60 ("net/mlx5e: Add resiliency for PTP TX port timestamp") Signed-off-by: Vadim Fedorenko <vadfed@meta.com> --- .../net/ethernet/mellanox/mlx5/core/en/ptp.c | 28 ++++++++++++++----- .../net/ethernet/mellanox/mlx5/core/en/txrx.h | 6 +++- .../ethernet/mellanox/mlx5/core/en_stats.c | 2 ++ .../ethernet/mellanox/mlx5/core/en_stats.h | 2 ++ 4 files changed, 30 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c index 8469e9c38670..32d6b387af61 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c @@ -86,20 +86,31 @@ static bool mlx5e_ptp_ts_cqe_drop(struct mlx5e_ptpsq *ptpsq, u16 skb_cc, u16 skb return (ptpsq->ts_cqe_ctr_mask && (skb_cc != skb_id)); } -static void mlx5e_ptp_skb_fifo_ts_cqe_resync(struct mlx5e_ptpsq *ptpsq, u16 skb_cc, u16 skb_id) +static bool mlx5e_ptp_skb_fifo_ts_cqe_resync(struct mlx5e_ptpsq *ptpsq, u16 skb_cc, u16 skb_id) { struct skb_shared_hwtstamps hwts = {}; struct sk_buff *skb; ptpsq->cq_stats->resync_event++; - while (skb_cc != skb_id) { - skb = mlx5e_skb_fifo_pop(&ptpsq->skb_fifo); + if (skb_cc > skb_id || PTP_WQE_CTR2IDX(ptpsq->skb_fifo_pc) < skb_id) { + ptpsq->cq_stats->ooo_cqe++; + return false; + } + + while (skb_cc != skb_id && (skb = mlx5e_skb_fifo_pop(&ptpsq->skb_fifo))) { hwts.hwtstamp = mlx5e_skb_cb_get_hwts(skb)->cqe_hwtstamp; skb_tstamp_tx(skb, &hwts); ptpsq->cq_stats->resync_cqe++; skb_cc = PTP_WQE_CTR2IDX(ptpsq->skb_fifo_cc); } + + if (!skb) { + ptpsq->cq_stats->fifo_empty++; + return false; + } + + return true; } static void mlx5e_ptp_handle_ts_cqe(struct mlx5e_ptpsq *ptpsq, @@ -109,7 +120,7 @@ static void mlx5e_ptp_handle_ts_cqe(struct mlx5e_ptpsq *ptpsq, u16 skb_id = PTP_WQE_CTR2IDX(be16_to_cpu(cqe->wqe_counter)); u16 skb_cc = PTP_WQE_CTR2IDX(ptpsq->skb_fifo_cc); struct mlx5e_txqsq *sq = &ptpsq->txqsq; - struct sk_buff *skb; + struct sk_buff *skb = NULL; ktime_t hwtstamp; if (unlikely(MLX5E_RX_ERR_CQE(cqe))) { @@ -118,8 +129,10 @@ static void mlx5e_ptp_handle_ts_cqe(struct mlx5e_ptpsq *ptpsq, goto out; } - if (mlx5e_ptp_ts_cqe_drop(ptpsq, skb_cc, skb_id)) - mlx5e_ptp_skb_fifo_ts_cqe_resync(ptpsq, skb_cc, skb_id); + if (mlx5e_ptp_ts_cqe_drop(ptpsq, skb_cc, skb_id) && + !mlx5e_ptp_skb_fifo_ts_cqe_resync(ptpsq, skb_cc, skb_id)) { + goto out; + } skb = mlx5e_skb_fifo_pop(&ptpsq->skb_fifo); hwtstamp = mlx5e_cqe_ts_to_ns(sq->ptp_cyc2time, sq->clock, get_cqe_ts(cqe)); @@ -128,7 +141,8 @@ static void mlx5e_ptp_handle_ts_cqe(struct mlx5e_ptpsq *ptpsq, ptpsq->cq_stats->cqe++; out: - napi_consume_skb(skb, budget); + if (skb) + napi_consume_skb(skb, budget); } static bool mlx5e_ptp_poll_ts_cq(struct mlx5e_cq *cq, int budget) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h index 853f312cd757..5fb58764c923 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h @@ -81,7 +81,7 @@ void mlx5e_free_txqsq_descs(struct mlx5e_txqsq *sq); static inline bool mlx5e_skb_fifo_has_room(struct mlx5e_skb_fifo *fifo) { - return (*fifo->pc - *fifo->cc) < fifo->mask; + return (u16)(*fifo->pc - *fifo->cc) < fifo->mask; } static inline bool @@ -291,12 +291,16 @@ void mlx5e_skb_fifo_push(struct mlx5e_skb_fifo *fifo, struct sk_buff *skb) { struct sk_buff **skb_item = mlx5e_skb_fifo_get(fifo, (*fifo->pc)++); + WARN_ONCE((u16)(*fifo->pc - *fifo->cc) > fifo->mask, "ptp fifo overflow"); *skb_item = skb; } static inline struct sk_buff *mlx5e_skb_fifo_pop(struct mlx5e_skb_fifo *fifo) { + if (*fifo->pc == *fifo->cc) + return NULL; + return *mlx5e_skb_fifo_get(fifo, (*fifo->cc)++); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c index 6687b8136e44..6fbd58d1722a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c @@ -2138,6 +2138,8 @@ static const struct counter_desc ptp_cq_stats_desc[] = { { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, abort_abs_diff_ns) }, { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, resync_cqe) }, { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, resync_event) }, + { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, ooo_cqe) }, + { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, fifo_empty) }, }; static const struct counter_desc ptp_rq_stats_desc[] = { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h index 375752d6546d..51da492169c2 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h @@ -461,6 +461,8 @@ struct mlx5e_ptp_cq_stats { u64 abort_abs_diff_ns; u64 resync_cqe; u64 resync_event; + u64 ooo_cqe; + u64 fifo_empty; }; struct mlx5e_rep_stats { -- 2.27.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH net v2 1/2] mlx5: fix possible ptp queue fifo overflow 2023-01-24 0:08 ` [PATCH net v2 1/2] mlx5: fix possible ptp queue fifo overflow Vadim Fedorenko @ 2023-01-24 2:05 ` Rahul Rameshbabu 2023-01-24 4:19 ` Jakub Kicinski 1 sibling, 0 replies; 7+ messages in thread From: Rahul Rameshbabu @ 2023-01-24 2:05 UTC (permalink / raw) To: Vadim Fedorenko Cc: Vadim Fedorenko, Aya Levin, Saeed Mahameed, Jakub Kicinski, Gal Pressman, Vadim Fedorenko, netdev On Tue, 24 Jan, 2023 03:08:35 +0300 Vadim Fedorenko <vfedorenko@novek.ru> wrote: > From: Vadim Fedorenko <vadfed@meta.com> > > Fifo pointers are not checked for overflow and this could potentially > lead to overflow and double free under heavy PTP traffic. > > Also there were accidental OOO cqe which lead to absolutely broken fifo. > Add checks to workaround OOO cqe and add counters to show the amount of > such events. > > Fixes: 58a518948f60 ("net/mlx5e: Add resiliency for PTP TX port timestamp") > Signed-off-by: Vadim Fedorenko <vadfed@meta.com> > --- > .../net/ethernet/mellanox/mlx5/core/en/ptp.c | 28 ++++++++++++++----- > .../net/ethernet/mellanox/mlx5/core/en/txrx.h | 6 +++- > .../ethernet/mellanox/mlx5/core/en_stats.c | 2 ++ > .../ethernet/mellanox/mlx5/core/en_stats.h | 2 ++ > 4 files changed, 30 insertions(+), 8 deletions(-) > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c > index 8469e9c38670..32d6b387af61 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c > @@ -86,20 +86,31 @@ static bool mlx5e_ptp_ts_cqe_drop(struct mlx5e_ptpsq *ptpsq, u16 skb_cc, u16 skb > return (ptpsq->ts_cqe_ctr_mask && (skb_cc != skb_id)); > } > > -static void mlx5e_ptp_skb_fifo_ts_cqe_resync(struct mlx5e_ptpsq *ptpsq, u16 skb_cc, u16 skb_id) > +static bool mlx5e_ptp_skb_fifo_ts_cqe_resync(struct mlx5e_ptpsq *ptpsq, u16 skb_cc, u16 skb_id) > { > struct skb_shared_hwtstamps hwts = {}; > struct sk_buff *skb; > > ptpsq->cq_stats->resync_event++; > > - while (skb_cc != skb_id) { > - skb = mlx5e_skb_fifo_pop(&ptpsq->skb_fifo); > + if (skb_cc > skb_id || PTP_WQE_CTR2IDX(ptpsq->skb_fifo_pc) < skb_id) { > + ptpsq->cq_stats->ooo_cqe++; > + return false; > + } > + > + while (skb_cc != skb_id && (skb = mlx5e_skb_fifo_pop(&ptpsq->skb_fifo))) { > hwts.hwtstamp = mlx5e_skb_cb_get_hwts(skb)->cqe_hwtstamp; > skb_tstamp_tx(skb, &hwts); > ptpsq->cq_stats->resync_cqe++; > skb_cc = PTP_WQE_CTR2IDX(ptpsq->skb_fifo_cc); > } > + > + if (!skb) { > + ptpsq->cq_stats->fifo_empty++; > + return false; > + } > + > + return true; > } > > static void mlx5e_ptp_handle_ts_cqe(struct mlx5e_ptpsq *ptpsq, > @@ -109,7 +120,7 @@ static void mlx5e_ptp_handle_ts_cqe(struct mlx5e_ptpsq *ptpsq, > u16 skb_id = PTP_WQE_CTR2IDX(be16_to_cpu(cqe->wqe_counter)); > u16 skb_cc = PTP_WQE_CTR2IDX(ptpsq->skb_fifo_cc); > struct mlx5e_txqsq *sq = &ptpsq->txqsq; > - struct sk_buff *skb; > + struct sk_buff *skb = NULL; > ktime_t hwtstamp; > > if (unlikely(MLX5E_RX_ERR_CQE(cqe))) { > @@ -118,8 +129,10 @@ static void mlx5e_ptp_handle_ts_cqe(struct mlx5e_ptpsq *ptpsq, > goto out; > } > > - if (mlx5e_ptp_ts_cqe_drop(ptpsq, skb_cc, skb_id)) > - mlx5e_ptp_skb_fifo_ts_cqe_resync(ptpsq, skb_cc, skb_id); > + if (mlx5e_ptp_ts_cqe_drop(ptpsq, skb_cc, skb_id) && > + !mlx5e_ptp_skb_fifo_ts_cqe_resync(ptpsq, skb_cc, skb_id)) { > + goto out; > + } > > skb = mlx5e_skb_fifo_pop(&ptpsq->skb_fifo); > hwtstamp = mlx5e_cqe_ts_to_ns(sq->ptp_cyc2time, sq->clock, get_cqe_ts(cqe)); > @@ -128,7 +141,8 @@ static void mlx5e_ptp_handle_ts_cqe(struct mlx5e_ptpsq *ptpsq, > ptpsq->cq_stats->cqe++; > > out: > - napi_consume_skb(skb, budget); > + if (skb) > + napi_consume_skb(skb, budget); > } > > static bool mlx5e_ptp_poll_ts_cq(struct mlx5e_cq *cq, int budget) > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h > index 853f312cd757..5fb58764c923 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h > @@ -81,7 +81,7 @@ void mlx5e_free_txqsq_descs(struct mlx5e_txqsq *sq); > static inline bool > mlx5e_skb_fifo_has_room(struct mlx5e_skb_fifo *fifo) > { > - return (*fifo->pc - *fifo->cc) < fifo->mask; > + return (u16)(*fifo->pc - *fifo->cc) < fifo->mask; > } > > static inline bool > @@ -291,12 +291,16 @@ void mlx5e_skb_fifo_push(struct mlx5e_skb_fifo *fifo, struct sk_buff *skb) > { > struct sk_buff **skb_item = mlx5e_skb_fifo_get(fifo, (*fifo->pc)++); > > + WARN_ONCE((u16)(*fifo->pc - *fifo->cc) > fifo->mask, "ptp fifo overflow"); I found this pretty tough to read/understand since it needed to account for the fact that the overflow already occurred. Instead I would refactor into the following. static inline void mlx5e_skb_fifo_push(struct mlx5e_skb_fifo *fifo, struct sk_buff *skb) { struct sk_buff **skb_item; WARN_ONCE(!mlx5e_skb_fifo_has_room(fifo), "ptp fifo overflow"); skb_item = mlx5e_skb_fifo_get(fifo, (*fifo->pc)++); *skb_item = skb; } > *skb_item = skb; > } > > static inline > struct sk_buff *mlx5e_skb_fifo_pop(struct mlx5e_skb_fifo *fifo) > { > + if (*fifo->pc == *fifo->cc) > + return NULL; > + > return *mlx5e_skb_fifo_get(fifo, (*fifo->cc)++); > } > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c > index 6687b8136e44..6fbd58d1722a 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c > @@ -2138,6 +2138,8 @@ static const struct counter_desc ptp_cq_stats_desc[] = { > { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, abort_abs_diff_ns) }, > { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, resync_cqe) }, > { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, resync_event) }, > + { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, ooo_cqe) }, > + { MLX5E_DECLARE_PTP_CQ_STAT(struct mlx5e_ptp_cq_stats, fifo_empty) }, > }; > > static const struct counter_desc ptp_rq_stats_desc[] = { > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h > index 375752d6546d..51da492169c2 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h > @@ -461,6 +461,8 @@ struct mlx5e_ptp_cq_stats { > u64 abort_abs_diff_ns; > u64 resync_cqe; > u64 resync_event; > + u64 ooo_cqe; > + u64 fifo_empty; > }; > > struct mlx5e_rep_stats { ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH net v2 1/2] mlx5: fix possible ptp queue fifo overflow 2023-01-24 0:08 ` [PATCH net v2 1/2] mlx5: fix possible ptp queue fifo overflow Vadim Fedorenko 2023-01-24 2:05 ` Rahul Rameshbabu @ 2023-01-24 4:19 ` Jakub Kicinski 2023-01-24 16:03 ` Vadim Fedorenko 1 sibling, 1 reply; 7+ messages in thread From: Jakub Kicinski @ 2023-01-24 4:19 UTC (permalink / raw) To: Vadim Fedorenko Cc: Vadim Fedorenko, Aya Levin, Saeed Mahameed, Gal Pressman, Vadim Fedorenko, netdev On Tue, 24 Jan 2023 03:08:35 +0300 Vadim Fedorenko wrote: > From: Vadim Fedorenko <vadfed@meta.com> > > Fifo pointers are not checked for overflow and this could potentially > lead to overflow and double free under heavy PTP traffic. > > Also there were accidental OOO cqe which lead to absolutely broken fifo. > Add checks to workaround OOO cqe and add counters to show the amount of > such events. May be worth adding a mention of the brokenness of the empty() check. Comparing free running counters works well unless C promotes the types to something wider than the counters themselves. So unsigned or u32 works, but comparing two u16s or u8s needs a explicit cast. > -static void mlx5e_ptp_skb_fifo_ts_cqe_resync(struct mlx5e_ptpsq *ptpsq, u16 skb_cc, u16 skb_id) > +static bool mlx5e_ptp_skb_fifo_ts_cqe_resync(struct mlx5e_ptpsq *ptpsq, u16 skb_cc, u16 skb_id) > { > struct skb_shared_hwtstamps hwts = {}; > struct sk_buff *skb; > > ptpsq->cq_stats->resync_event++; > > - while (skb_cc != skb_id) { > - skb = mlx5e_skb_fifo_pop(&ptpsq->skb_fifo); > + if (skb_cc > skb_id || PTP_WQE_CTR2IDX(ptpsq->skb_fifo_pc) < skb_id) { Are you sure this works for all cases? Directly comparing indexes of a ring buffer seems dangerous. We'd need to compare like this: (s16)(skb_cc - skb_id) < 0 > + ptpsq->cq_stats->ooo_cqe++; > + return false; > + } > @@ -128,7 +141,8 @@ static void mlx5e_ptp_handle_ts_cqe(struct mlx5e_ptpsq *ptpsq, > ptpsq->cq_stats->cqe++; > > out: > - napi_consume_skb(skb, budget); > + if (skb) > + napi_consume_skb(skb, budget); I think napi_consume_skb() takes NULLs. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH net v2 1/2] mlx5: fix possible ptp queue fifo overflow 2023-01-24 4:19 ` Jakub Kicinski @ 2023-01-24 16:03 ` Vadim Fedorenko 2023-01-25 22:02 ` Jakub Kicinski 0 siblings, 1 reply; 7+ messages in thread From: Vadim Fedorenko @ 2023-01-24 16:03 UTC (permalink / raw) To: Jakub Kicinski, Vadim Fedorenko Cc: Vadim Fedorenko, Aya Levin, Saeed Mahameed, Gal Pressman, netdev@vger.kernel.org On 24/01/2023 04:19, Jakub Kicinski wrote: > On Tue, 24 Jan 2023 03:08:35 +0300 Vadim Fedorenko wrote: >> From: Vadim Fedorenko <vadfed@meta.com> >> >> Fifo pointers are not checked for overflow and this could potentially >> lead to overflow and double free under heavy PTP traffic. >> >> Also there were accidental OOO cqe which lead to absolutely broken fifo. >> Add checks to workaround OOO cqe and add counters to show the amount of >> such events. > > May be worth adding a mention of the brokenness of the empty() check. > Comparing free running counters works well unless C promotes the types > to something wider than the counters themselves. So unsigned or u32 > works, but comparing two u16s or u8s needs a explicit cast. > Yep, sure, will add it to the next version >> -static void mlx5e_ptp_skb_fifo_ts_cqe_resync(struct mlx5e_ptpsq *ptpsq, u16 skb_cc, u16 skb_id) >> +static bool mlx5e_ptp_skb_fifo_ts_cqe_resync(struct mlx5e_ptpsq *ptpsq, u16 skb_cc, u16 skb_id) >> { >> struct skb_shared_hwtstamps hwts = {}; >> struct sk_buff *skb; >> >> ptpsq->cq_stats->resync_event++; >> >> - while (skb_cc != skb_id) { >> - skb = mlx5e_skb_fifo_pop(&ptpsq->skb_fifo); >> + if (skb_cc > skb_id || PTP_WQE_CTR2IDX(ptpsq->skb_fifo_pc) < skb_id) { > > Are you sure this works for all cases? > Directly comparing indexes of a ring buffer seems dangerous. > We'd need to compare like this: > > (s16)(skb_cc - skb_id) < 0 > Here I would like to count (and skip re-syncing) all the packets that are not going to be in FIFO. Your suggestion will not work for the simplest example. Imagine we have FIFO for 16 elements, and current counters are: (consumer) skb_cc = 13, (producer) skb_pc = 15, so 3 packets are in. Then skb_id = 10 arrives out-of-order. It will be counted because of (skb_cc > skb_id), but will not be catched by (skb_cc - skb_id) < 0. To cover all other cases let's continue. Let's think that 2 more packets landed in the queue, now we have skb_cc = 13, skb_pc = 1 (because of wraparound). skb_id = 11 comes and it's still out of order and will be catched by the same check. Then let's assume that skb_id 13,14,15 arrived and moved our consumer pointer, now we have skb_cc = 0, skb_pc = 1. If skb_id = 12 arrives, it will be catched by PTP_WQE_CTR2IDX(ptpsq->skb_fifo_pc) < skb_id. So I believe we should be fine here and catch all out-of-order (older than skb_cc) skbs. >> + ptpsq->cq_stats->ooo_cqe++; >> + return false; >> + } > >> @@ -128,7 +141,8 @@ static void mlx5e_ptp_handle_ts_cqe(struct mlx5e_ptpsq *ptpsq, >> ptpsq->cq_stats->cqe++; >> >> out: >> - napi_consume_skb(skb, budget); >> + if (skb) >> + napi_consume_skb(skb, budget); > > I think napi_consume_skb() takes NULLs. Yep, will remove this piece. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH net v2 1/2] mlx5: fix possible ptp queue fifo overflow 2023-01-24 16:03 ` Vadim Fedorenko @ 2023-01-25 22:02 ` Jakub Kicinski 0 siblings, 0 replies; 7+ messages in thread From: Jakub Kicinski @ 2023-01-25 22:02 UTC (permalink / raw) To: Vadim Fedorenko Cc: Vadim Fedorenko, Aya Levin, Saeed Mahameed, Gal Pressman, netdev@vger.kernel.org On Tue, 24 Jan 2023 16:03:42 +0000 Vadim Fedorenko wrote: > > Are you sure this works for all cases? > > Directly comparing indexes of a ring buffer seems dangerous. > > We'd need to compare like this: > > > > (s16)(skb_cc - skb_id) < 0 > > > > Here I would like to count (and skip re-syncing) all the packets that > are not going to be in FIFO. Your suggestion will not work for the > simplest example. Imagine we have FIFO for 16 elements, and current > counters are: > (consumer) skb_cc = 13, (producer) skb_pc = 15, so 3 packets are in. > Then skb_id = 10 arrives out-of-order. It will be counted because of > (skb_cc > skb_id), but will not be catched by (skb_cc - skb_id) < 0. Oh, I may be confused about what the producer and consumer are. The point I was trying to make is that comparing indexes on rings is hard. Instead of writing: if (a < b) you need to write: if ((signed)(a - b) < 0) "mathematically" it's the same, but in "wrapping logic" it works because if you're further than half a ring around then it counts as a second negation.. ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH net v2 2/2] mlx5: fix skb leak while fifo resync 2023-01-24 0:08 [PATCH net v2 0/2] mlx5: bugfixes for ptp fifo queue Vadim Fedorenko 2023-01-24 0:08 ` [PATCH net v2 1/2] mlx5: fix possible ptp queue fifo overflow Vadim Fedorenko @ 2023-01-24 0:08 ` Vadim Fedorenko 1 sibling, 0 replies; 7+ messages in thread From: Vadim Fedorenko @ 2023-01-24 0:08 UTC (permalink / raw) To: Vadim Fedorenko, Aya Levin, Saeed Mahameed, Jakub Kicinski, Gal Pressman Cc: Vadim Fedorenko, netdev From: Vadim Fedorenko <vadfed@meta.com> During ptp resync operation SKBs were poped from the fifo but were never freed neither by napi_consume nor by dev_kfree_skb_any. Add call to napi_consume_skb to properly free SKBs. Fixes: 58a518948f60 ("net/mlx5e: Add resiliency for PTP TX port timestamp") Reviewed-by: Gal Pressman <gal@nvidia.com> Signed-off-by: Vadim Fedorenko <vadfed@meta.com> --- drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c index 32d6b387af61..2797028608a3 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c @@ -102,6 +102,7 @@ static bool mlx5e_ptp_skb_fifo_ts_cqe_resync(struct mlx5e_ptpsq *ptpsq, u16 skb_ hwts.hwtstamp = mlx5e_skb_cb_get_hwts(skb)->cqe_hwtstamp; skb_tstamp_tx(skb, &hwts); ptpsq->cq_stats->resync_cqe++; + napi_consume_skb(skb, 1); skb_cc = PTP_WQE_CTR2IDX(ptpsq->skb_fifo_cc); } -- 2.27.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2023-01-25 22:02 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2023-01-24 0:08 [PATCH net v2 0/2] mlx5: bugfixes for ptp fifo queue Vadim Fedorenko 2023-01-24 0:08 ` [PATCH net v2 1/2] mlx5: fix possible ptp queue fifo overflow Vadim Fedorenko 2023-01-24 2:05 ` Rahul Rameshbabu 2023-01-24 4:19 ` Jakub Kicinski 2023-01-24 16:03 ` Vadim Fedorenko 2023-01-25 22:02 ` Jakub Kicinski 2023-01-24 0:08 ` [PATCH net v2 2/2] mlx5: fix skb leak while fifo resync Vadim Fedorenko
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).