[PATCH iwl-next v2] ice: Reset VF on Tx MDD event

[PATCH iwl-next v2] ice: Reset VF on Tx MDD event

[Pawel Chmielewski]

From: Liang-Min Wang <liang-min.wang@intel.com>

In cases when VF sends malformed packets that are classified as malicious,
sometimes it causes Tx queue to freeze. This frozen queue can be stuck
for several minutes being unusable. This behavior can be reproduced with
DPDK application, testpmd.

When Malicious Driver Detection event occurs, perform graceful VF reset
to quickly bring VF back to operational state. Add a log message to
notify about the cause of the reset.

Signed-off-by: Liang-Min Wang <liang-min.wang@intel.com>
Signed-off-by: Pawel Chmielewski <pawel.chmielewski@intel.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
---
Changelog
v1->v2:
Reverted unneeded formatting change, fixed commit message, fixed a log
message with a correct event name.
---

 drivers/net/ethernet/intel/ice/ice_main.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 3c9419b05a2a..ee9752af6397 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -1839,6 +1839,10 @@ static void ice_handle_mdd_event(struct ice_pf *pf)
 			if (netif_msg_tx_err(pf))
 				dev_info(dev, "Malicious Driver Detection event TX_TCLAN detected on VF %d\n",
 					 vf->vf_id);
+			dev_info(dev,
+				 "PF-to-VF reset on VF %d due to Tx MDD TX_TCLAN event\n",
+				 vf->vf_id);
+			ice_reset_vf(vf, ICE_VF_RESET_NOTIFY);
 		}
 
 		reg = rd32(hw, VP_MDET_TX_TDPU(vf->vf_id));
@@ -1849,6 +1853,10 @@ static void ice_handle_mdd_event(struct ice_pf *pf)
 			if (netif_msg_tx_err(pf))
 				dev_info(dev, "Malicious Driver Detection event TX_TDPU detected on VF %d\n",
 					 vf->vf_id);
+			dev_info(dev,
+				 "PF-to-VF reset on VF %d due to Tx MDD TX_TDPU event\n",
+				 vf->vf_id);
+			ice_reset_vf(vf, ICE_VF_RESET_NOTIFY);
 		}
 
 		reg = rd32(hw, VP_MDET_RX(vf->vf_id));
-- 
2.37.3

Re: [PATCH iwl-next v2] ice: Reset VF on Tx MDD event

[Simon Horman]

On Thu, Nov 02, 2023 at 04:51:49PM +0100, Pawel Chmielewski wrote:
> From: Liang-Min Wang <liang-min.wang@intel.com>
> 
> In cases when VF sends malformed packets that are classified as malicious,
> sometimes it causes Tx queue to freeze. This frozen queue can be stuck
> for several minutes being unusable. This behavior can be reproduced with
> DPDK application, testpmd.
> 
> When Malicious Driver Detection event occurs, perform graceful VF reset
> to quickly bring VF back to operational state. Add a log message to
> notify about the cause of the reset.
> 
> Signed-off-by: Liang-Min Wang <liang-min.wang@intel.com>
> Signed-off-by: Pawel Chmielewski <pawel.chmielewski@intel.com>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>

Reviewed-by: Simon Horman <horms@kernel.org>

Re: [PATCH iwl-next v2] ice: Reset VF on Tx MDD event

[Michal Schmidt]

On Thu, Nov 2, 2023 at 4:56 PM Pawel Chmielewski
<pawel.chmielewski@intel.com> wrote:
> From: Liang-Min Wang <liang-min.wang@intel.com>
>
> In cases when VF sends malformed packets that are classified as malicious,
> sometimes it causes Tx queue to freeze. This frozen queue can be stuck
> for several minutes being unusable. This behavior can be reproduced with
> DPDK application, testpmd.
>
> When Malicious Driver Detection event occurs, perform graceful VF reset
> to quickly bring VF back to operational state. Add a log message to
> notify about the cause of the reset.

Sorry for bringing this up so late, but I have just now realized this:
Wasn't freezing of the queue originally the intended behavior, as a
penalty for being malicious?
Shouldn't these resets at least be guarded by ICE_FLAG_MDD_AUTO_RESET_VF?

Michal

> Signed-off-by: Liang-Min Wang <liang-min.wang@intel.com>
> Signed-off-by: Pawel Chmielewski <pawel.chmielewski@intel.com>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> ---
> Changelog
> v1->v2:
> Reverted unneeded formatting change, fixed commit message, fixed a log
> message with a correct event name.
> ---
>
>  drivers/net/ethernet/intel/ice/ice_main.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
> index 3c9419b05a2a..ee9752af6397 100644
> --- a/drivers/net/ethernet/intel/ice/ice_main.c
> +++ b/drivers/net/ethernet/intel/ice/ice_main.c
> @@ -1839,6 +1839,10 @@ static void ice_handle_mdd_event(struct ice_pf *pf)
>                         if (netif_msg_tx_err(pf))
>                                 dev_info(dev, "Malicious Driver Detection event TX_TCLAN detected on VF %d\n",
>                                          vf->vf_id);
> +                       dev_info(dev,
> +                                "PF-to-VF reset on VF %d due to Tx MDD TX_TCLAN event\n",
> +                                vf->vf_id);
> +                       ice_reset_vf(vf, ICE_VF_RESET_NOTIFY);
>                 }
>
>                 reg = rd32(hw, VP_MDET_TX_TDPU(vf->vf_id));
> @@ -1849,6 +1853,10 @@ static void ice_handle_mdd_event(struct ice_pf *pf)
>                         if (netif_msg_tx_err(pf))
>                                 dev_info(dev, "Malicious Driver Detection event TX_TDPU detected on VF %d\n",
>                                          vf->vf_id);
> +                       dev_info(dev,
> +                                "PF-to-VF reset on VF %d due to Tx MDD TX_TDPU event\n",
> +                                vf->vf_id);
> +                       ice_reset_vf(vf, ICE_VF_RESET_NOTIFY);
>                 }
>
>                 reg = rd32(hw, VP_MDET_RX(vf->vf_id));
> --
> 2.37.3
>
>

Re: [PATCH iwl-next v2] ice: Reset VF on Tx MDD event

[Pawel Chmielewski]

On Thu, Dec 14, 2023 at 09:37:32AM +0100, Michal Schmidt wrote:
> On Thu, Nov 2, 2023 at 4:56 PM Pawel Chmielewski
> <pawel.chmielewski@intel.com> wrote:
> > From: Liang-Min Wang <liang-min.wang@intel.com>
> >
> > In cases when VF sends malformed packets that are classified as malicious,
> > sometimes it causes Tx queue to freeze. This frozen queue can be stuck
> > for several minutes being unusable. This behavior can be reproduced with
> > DPDK application, testpmd.
> >
> > When Malicious Driver Detection event occurs, perform graceful VF reset
> > to quickly bring VF back to operational state. Add a log message to
> > notify about the cause of the reset.
> 
> Sorry for bringing this up so late, but I have just now realized this:
> Wasn't freezing of the queue originally the intended behavior, as a
> penalty for being malicious?
> Shouldn't these resets at least be guarded by ICE_FLAG_MDD_AUTO_RESET_VF?
> 
> Michal

In some cases, the MDD can be caused also by a regular software error
(like the one mentioned in commit message), and not the actual malicious
action. There was decision to change the default behavior to avoid denial
of service. 

Re: [PATCH iwl-next v2] ice: Reset VF on Tx MDD event

[Tony Nguyen]

On 12/14/2023 8:51 AM, Pawel Chmielewski wrote:
> On Thu, Dec 14, 2023 at 09:37:32AM +0100, Michal Schmidt wrote:
>> On Thu, Nov 2, 2023 at 4:56 PM Pawel Chmielewski
>> <pawel.chmielewski@intel.com> wrote:
>>> From: Liang-Min Wang <liang-min.wang@intel.com>

...

>>> When Malicious Driver Detection event occurs, perform graceful VF reset
>>> to quickly bring VF back to operational state. Add a log message to
>>> notify about the cause of the reset.
>>
>> Sorry for bringing this up so late, but I have just now realized this:
>> Wasn't freezing of the queue originally the intended behavior, as a
>> penalty for being malicious?
>> Shouldn't these resets at least be guarded by ICE_FLAG_MDD_AUTO_RESET_VF?
>>
>> Michal
> 
> In some cases, the MDD can be caused also by a regular software error
> (like the one mentioned in commit message), and not the actual malicious
> action. There was decision to change the default behavior to avoid denial
> of service.

Michal brings up some valid questions. I'd like to clarify the 
expectations between how the two should work together before moving 
forward with this.

Thanks,
Tony