Re: [PATCH net-next 7/8] net/mlx5e: take into account device reconfiguration for xdp_features flag

[Tariq Toukan <ttoukan.linux@gmail.com>]

On 08/03/2023 17:47, Lorenzo Bianconi wrote:
>>
>>
>> On 07/03/2023 16:54, Lorenzo Bianconi wrote:
>>> Take into account LRO and GRO configuration setting device xdp_features
>>> flag. Moreover consider channel rq_wq_type enabling rx scatter-gatter
>>> support in xdp_features flag.
>>>
>>> Fixes: 66c0e13ad236 ("drivers: net: turn on XDP features")
>>> Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
>>> ---
>>>    drivers/net/ethernet/mellanox/mlx5/core/en.h  |  1 +
>>>    .../ethernet/mellanox/mlx5/core/en_ethtool.c  | 10 ++++-
>>>    .../net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++++++++++++---
>>>    .../net/ethernet/mellanox/mlx5/core/en_rep.c  |  3 ++
>>>    4 files changed, 51 insertions(+), 8 deletions(-)
>>>
>>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
>>> index 88460b7796e5..4276c6eb6820 100644
>>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
>>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
>>> @@ -1243,6 +1243,7 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16
>>>    void mlx5e_rx_dim_work(struct work_struct *work);
>>>    void mlx5e_tx_dim_work(struct work_struct *work);
>>> +void mlx5e_set_xdp_feature(struct net_device *netdev);
>>>    netdev_features_t mlx5e_features_check(struct sk_buff *skb,
>>>    				       struct net_device *netdev,
>>>    				       netdev_features_t features);
>>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
>>> index 7708acc9b2ab..79fd21ecb9cb 100644
>>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
>>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
>>> @@ -1985,6 +1985,7 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
>>>    	struct mlx5e_priv *priv = netdev_priv(netdev);
>>>    	struct mlx5_core_dev *mdev = priv->mdev;
>>>    	struct mlx5e_params new_params;
>>> +	int err;
>>>    	if (enable) {
>>>    		/* Checking the regular RQ here; mlx5e_validate_xsk_param called
>>> @@ -2005,7 +2006,14 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable)
>>>    	MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ, enable);
>>>    	mlx5e_set_rq_type(mdev, &new_params);
>>> -	return mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true);
>>> +	err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true);
>>> +	if (err)
>>> +		return err;
>>> +
>>> +	/* update XDP supported features */
>>> +	mlx5e_set_xdp_feature(netdev);
>>> +
>>> +	return 0;
>>>    }
>>>    static int set_pflag_rx_no_csum_complete(struct net_device *netdev, bool enable)
>>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>>> index 76a9c5194a70..1b68dd2be2c5 100644
>>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>>> @@ -4004,6 +4004,30 @@ static int mlx5e_handle_feature(struct net_device *netdev,
>>>    	return 0;
>>>    }
>>> +void mlx5e_set_xdp_feature(struct net_device *netdev)
>>> +{
>>> +	struct mlx5e_priv *priv = netdev_priv(netdev);
>>> +	bool ndo_xmit = test_bit(MLX5E_STATE_XDP_ACTIVE, &priv->state);
>>
>> Our driver doesn't require loading a dummy XDP program to have the
>> redirect-in ability. It's always there.
>>
>> I actually have a bug fix under internal review with Saeed that addresses
>> this.
>>
>> In addition, it cleans up the NETDEV_XDP_ACT_NDO_XMIT_SG as we do not
>> support it yet. I have a series that's adding support and will submit it
>> soon.
>>
>> Any reason you're submitting these fixes to net-next rather than net?
> 
> Hi Tariq,
> 
> I am fine to repost this series for net instead of net-next. Any downsides about
> it?

Let's repost to net.
It's a fixes series, and 6.3 is still in its RCs.
If you don't post it to net then the xdp-features in 6.3 will be broken.

> 
>> Maybe it'd be better if we integrate the patches, here's my fix (still under
>> review...):
>>
>> Author: Tariq Toukan <tariqt@nvidia.com>
>> Date:   Thu Feb 23 08:58:04 2023 +0200
>>
>>      net/mlx5e: Fix exposed xdp_features
>>
>>      Always declare NETDEV_XDP_ACT_NDO_XMIT as the ndo_xdp_xmit callback
>>      is always functional per our design, and does not require loading
>>      a dummy xdp program.
>>
>>      Although non-linear XDP buffer is supported for XDP_TX flow, do not
>>      declare NETDEV_XDP_ACT_NDO_XMIT_SG as it is yet supported for
>>      redirected-in frames.
>>
>>      Fixes: 66c0e13ad236 ("drivers: net: turn on XDP features")
>>      Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
>>
>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> index 53feb0529943..9a5d3ce1fbcd 100644
>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> @@ -4741,13 +4741,6 @@ static int mlx5e_xdp_set(struct net_device *netdev,
>> struct bpf_prog *prog)
>>          if (old_prog)
>>                  bpf_prog_put(old_prog);
>>
>> -       if (reset) {
>> -               if (prog)
>> -                       xdp_features_set_redirect_target(netdev, true);
>> -               else
>> -                       xdp_features_clear_redirect_target(netdev);
>> -       }
>> -
>>          if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset)
>>                  goto unlock;
>>
>> @@ -5144,6 +5137,7 @@ static void mlx5e_build_nic_netdev(struct net_device
>> *netdev)
>>          netdev->features         |= NETIF_F_HW_VLAN_STAG_FILTER;
>>
>>          netdev->xdp_features = NETDEV_XDP_ACT_BASIC |
>> NETDEV_XDP_ACT_REDIRECT |
>> +                              NETDEV_XDP_ACT_NDO_XMIT |
>>                                 NETDEV_XDP_ACT_XSK_ZEROCOPY |
>>                                 NETDEV_XDP_ACT_RX_SG;
> 
> I am fine to drop this my patch and rely on the one you provided but it depends
> on the eta about the described patches because otherwise real capabilities and
> xdp-features will not be aligned. Any inputs on it?
> 

My patch doesn't replace yours, as it doesn't fix the missing 
features_update according to striding RQ and HW LRO/GRO.

I think we should combine them, either take mine as-is into your series, 
or squash it into this patch. I'm fine with both.

>>
>>
>>> +	struct mlx5e_params *params = &priv->channels.params;
>>> +	xdp_features_t val;
>>> +
>>> +	if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) {
>>> +		xdp_clear_features_flag(netdev);
>>> +		return;
>>> +	}
>>> +
>>> +	val = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT |
>>> +	      NETDEV_XDP_ACT_XSK_ZEROCOPY;
>>> +	if (ndo_xmit)
>>> +		val |= NETDEV_XDP_ACT_NDO_XMIT;
>>> +	if (params->rq_wq_type == MLX5_WQ_TYPE_CYCLIC) {
>>> +		val |= NETDEV_XDP_ACT_RX_SG;
>>> +		if (ndo_xmit)
>>> +			val |= NETDEV_XDP_ACT_NDO_XMIT_SG;
>>
>> This NETDEV_XDP_ACT_NDO_XMIT_SG capability is not related to the RQ type.
>> It's still not supported at this point.
> 
> ack, I will fix it.
> 
>>
>> BTW, I have a series completing all the missing capabilities (multibuf on
>> Striding + multibuf redirect-in), should be submitted in this kernel.
> 
> cool :)
> 
> Regards,
> Lorenzo
>