Re: [PATCH net-next v4 2/8] bnxt_en: add support for tcp-data-split ethtool command

[Andy Gospodarek <andrew.gospodarek@broadcom.com>]

On Thu, Oct 24, 2024 at 10:02:30PM -0700, Michael Chan wrote:
> On Tue, Oct 22, 2024 at 9:24 AM Taehee Yoo <ap420073@gmail.com> wrote:
> >
> > NICs that uses bnxt_en driver supports tcp-data-split feature by the
> > name of HDS(header-data-split).
> > But there is no implementation for the HDS to enable or disable by
> > ethtool.
> > Only getting the current HDS status is implemented and The HDS is just
> > automatically enabled only when either LRO, HW-GRO, or JUMBO is enabled.
> > The hds_threshold follows rx-copybreak value. and it was unchangeable.
> >
> > This implements `ethtool -G <interface name> tcp-data-split <value>`
> > command option.
> > The value can be <on>, <off>, and <auto> but the <auto> will be
> > automatically changed to <on>.
> >
> > HDS feature relies on the aggregation ring.
> > So, if HDS is enabled, the bnxt_en driver initializes the aggregation
> > ring.
> > This is the reason why BNXT_FLAG_AGG_RINGS contains HDS condition.
> >
> > Tested-by: Stanislav Fomichev <sdf@fomichev.me>
> > Signed-off-by: Taehee Yoo <ap420073@gmail.com>
> > ---
> >
> > v4:
> >  - Do not support disable tcp-data-split.
> >  - Add Test tag from Stanislav.
> >
> > v3:
> >  - No changes.
> >
> > v2:
> >  - Do not set hds_threshold to 0.
> >
> >  drivers/net/ethernet/broadcom/bnxt/bnxt.c         |  8 +++-----
> >  drivers/net/ethernet/broadcom/bnxt/bnxt.h         |  5 +++--
> >  drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 13 +++++++++++++
> >  3 files changed, 19 insertions(+), 7 deletions(-)
> >
> > diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> > index 0f5fe9ba691d..91ea42ff9b17 100644
> > --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> > +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
> 
> > @@ -6420,15 +6420,13 @@ static int bnxt_hwrm_vnic_set_hds(struct bnxt *bp, struct bnxt_vnic_info *vnic)
> >
> >         req->flags = cpu_to_le32(VNIC_PLCMODES_CFG_REQ_FLAGS_JUMBO_PLACEMENT);
> >         req->enables = cpu_to_le32(VNIC_PLCMODES_CFG_REQ_ENABLES_JUMBO_THRESH_VALID);
> > +       req->jumbo_thresh = cpu_to_le16(bp->rx_buf_use_size);
> >
> > -       if (BNXT_RX_PAGE_MODE(bp)) {
> > -               req->jumbo_thresh = cpu_to_le16(bp->rx_buf_use_size);
> 
> Please explain why this "if" condition is removed.
> BNXT_RX_PAGE_MODE() means that we are in XDP mode and we currently
> don't support HDS in XDP mode.  Added Andy Gospo to CC so he can also
> comment.
> 

In bnxt_set_rx_skb_mode we set BNXT_FLAG_RX_PAGE_MODE and clear
BNXT_FLAG_AGG_RINGS, so this should work.  The only issue is that we
have spots in the driver where we check BNXT_RX_PAGE_MODE(bp) to
indicate that XDP single-buffer mode is enabled on the device.

If you need to respin this series I would prefer that the change is like
below to key off the page mode being disabled and BNXT_FLAG_AGG_RINGS
being enabled to setup HDS.  This will serve as a reminder that this is
for XDP.

@@ -6418,15 +6418,13 @@ static int bnxt_hwrm_vnic_set_hds(struct bnxt *bp, struct bnxt_vnic_info *vnic)
 
        req->flags = cpu_to_le32(VNIC_PLCMODES_CFG_REQ_FLAGS_JUMBO_PLACEMENT);
        req->enables = cpu_to_le32(VNIC_PLCMODES_CFG_REQ_ENABLES_JUMBO_THRESH_VALID);
+       req->jumbo_thresh = cpu_to_le16(bp->rx_buf_use_size);
 
-       if (BNXT_RX_PAGE_MODE(bp)) {
-               req->jumbo_thresh = cpu_to_le16(bp->rx_buf_use_size);
-       } else {
+       if (!BNXT_RX_PAGE_MODE(bp) && (bp->flags & BNXT_FLAG_AGG_RINGS)) {
                req->flags |= cpu_to_le32(VNIC_PLCMODES_CFG_REQ_FLAGS_HDS_IPV4 |
                                          VNIC_PLCMODES_CFG_REQ_FLAGS_HDS_IPV6);
                req->enables |=
                        cpu_to_le32(VNIC_PLCMODES_CFG_REQ_ENABLES_HDS_THRESHOLD_VALID);
-               req->jumbo_thresh = cpu_to_le16(bp->rx_copy_thresh);
                req->hds_threshold = cpu_to_le16(bp->rx_copy_thresh);
        }
        req->vnic_id = cpu_to_le32(vnic->fw_vnic_id);

> > -       } else {
> > +       if (bp->flags & BNXT_FLAG_AGG_RINGS) {
> >                 req->flags |= cpu_to_le32(VNIC_PLCMODES_CFG_REQ_FLAGS_HDS_IPV4 |
> >                                           VNIC_PLCMODES_CFG_REQ_FLAGS_HDS_IPV6);
> >                 req->enables |=
> >                         cpu_to_le32(VNIC_PLCMODES_CFG_REQ_ENABLES_HDS_THRESHOLD_VALID);
> > -               req->jumbo_thresh = cpu_to_le16(bp->rx_copybreak);
> >                 req->hds_threshold = cpu_to_le16(bp->rx_copybreak);
> >         }
> >         req->vnic_id = cpu_to_le32(vnic->fw_vnic_id);