Linux kernel -stable discussions
 help / color / mirror / Atom feed
From: Sahil Chandna <sahilchandna@linux.microsoft.com>
To: linux-hyperv@vger.kernel.org, netdev@vger.kernel.org,
	Haiyang Zhang <haiyangz@microsoft.com>,
	Wei Liu <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
	Andrew Lunn <andrew+netdev@lunn.ch>,
	"David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@kernel.org>,
	Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
	Wei Hu <weh@microsoft.com>,
	Konstantin Taranov <kotaranov@microsoft.com>,
	Simon Horman <horms@kernel.org>,
	Erni Sri Satya Vennela <ernis@linux.microsoft.com>,
	Aditya Garg <gargaditya@linux.microsoft.com>,
	Manish Awasthi <mawasthi@linux.microsoft.com>,
	Breno Leitao <leitao@debian.org>,
	linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org
Cc: paulros@microsoft.com, stable@vger.kernel.org
Subject: Re: [PATCH net] net: mana: Set 1 usec / 1 comp as default and minimum interrupt moderation
Date: Thu, 19 Nov 2026 15:52:40 +0530	[thread overview]
Message-ID: <cc705453-0794-494f-adb9-b6fb71254ba7@linux.microsoft.com> (raw)
In-Reply-To: <20261113204427.1922848-1-haiyangz@linux.microsoft.com>



On 14/11/2026 02:14, Haiyang Zhang wrote:
> From: Haiyang Zhang <haiyangz@microsoft.com>
> 
> The existing code sets interrupt moderation defaults to 0 usec and 0
> completions, which may cause CQ starvation of our NIC HW, and TX time
> out error like:
> mana 7870:00:00.0 ens1: NETDEV WATCHDOG: CPU: 14: transmit queue 4 timed out 63434 ms
> 
> To fix, change the default and minimal rx/tx usecs and frames to 1.
> When the HW supports interrupt moderation, reject ethtool rx/tx usecs
> and frames below 1.
> Also clamp the values written by mana_gd_ring_dim() to [1, max], so
> no caller, including the DIM work, programs a zero value.
> On the HW without interrupt moderation support, do not request CQ
> moderation, the usecs and frames stay at 0, and cannot be modified by
> ethtool.
> The PF capability is queried again when the device resumes from a
> reset or suspend, so re-initialize these settings before re-attaching
> the vPorts.
> 
> Advertise this change to the PF with the new driver capability flag
> GDMA_DRV_CAP_FLAG_1_SET_MIN_INTR_MOD_VALUE.
> 
> Cc: stable@vger.kernel.org
> Fixes: 433f482add31 ("net: mana: Add Interrupt Moderation support")
> Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
> ---
>   .../net/ethernet/microsoft/mana/gdma_main.c   |  5 ++
>   drivers/net/ethernet/microsoft/mana/mana_en.c | 76 +++++++++++++------
>   .../ethernet/microsoft/mana/mana_ethtool.c    | 26 +++++--
>   include/net/mana/gdma.h                       |  6 +-
>   include/net/mana/mana.h                       |  9 ++-
>   5 files changed, 88 insertions(+), 34 deletions(-)
> 
> diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c
> index ab942d5bfe05..10be69e97ceb 100644
> --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c
> +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c
> @@ -646,6 +646,11 @@ void mana_gd_ring_dim(struct gdma_queue *cq, u32 mod_usec, bool mod_usec_vld,
>   	struct gdma_context *gc = cq->gdma_dev->gdma_context;
>   	u32 dim_val;
>   
> +	mod_usec = clamp_t(u32, mod_usec, MANA_INTR_MODR_USEC_MIN,
> +			   MANA_INTR_MODR_USEC_MAX);
> +	mod_comps = clamp_t(u32, mod_comps, MANA_INTR_MODR_COMP_MIN,
> +			    MANA_INTR_MODR_COMP_MAX);
> +
>   	/* Convert the DIM values to doorbell parameters */
>   	dim_val = FIELD_PREP(MANA_INTR_MODR_USEC_MAX, mod_usec) |
>   		  FIELD_PREP(MANA_INTR_MODR_COMP_MASK, mod_comps);
> diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
> index 7e9223143fc7..95bf153c3e1a 100644
> --- a/drivers/net/ethernet/microsoft/mana/mana_en.c
> +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
> @@ -2515,9 +2515,6 @@ static void mana_rx_dim_work(struct work_struct *work)
>   	cur_moder = net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
>   	cq = container_of(dim, struct mana_cq, dim);
>   
> -	cur_moder.usec = min_t(u16, cur_moder.usec, MANA_INTR_MODR_USEC_MAX);
> -	cur_moder.pkts = min_t(u16, cur_moder.pkts, MANA_INTR_MODR_COMP_MAX);
> -
>   	mana_gd_ring_dim(cq->gdma_cq, cur_moder.usec, true,
>   			 cur_moder.pkts, true);
>   
> @@ -2533,9 +2530,6 @@ static void mana_tx_dim_work(struct work_struct *work)
>   	cur_moder = net_dim_get_tx_moderation(dim->mode, dim->profile_ix);
>   	cq = container_of(dim, struct mana_cq, dim);
>   
> -	cur_moder.usec = min_t(u16, cur_moder.usec, MANA_INTR_MODR_USEC_MAX);
> -	cur_moder.pkts = min_t(u16, cur_moder.pkts, MANA_INTR_MODR_COMP_MAX);
> -
>   	mana_gd_ring_dim(cq->gdma_cq, cur_moder.usec, true,
>   			 cur_moder.pkts, true);
>   
> @@ -2871,10 +2865,15 @@ static int mana_create_txq(struct mana_port_context *apc,
>   		cq_spec.modr_ctx_id = 0;
>   		cq_spec.attached_eq = cq->gdma_cq->cq.parent->id;
>   
> -		/* DIM setting can be changed at runtime */
> -		cq_spec.req_cq_moderation = true;
> -		cq_spec.cq_moderation_usec = apc->intr_modr_tx_usec;
> -		cq_spec.cq_moderation_comp = apc->intr_modr_tx_comp;
> +		/* Request CQ moderation only if HW supports it.
> +		 * DIM setting can be changed at runtime.
> +		 */
> +		if (gc->pf_cap_flags1 &
> +		    GDMA_PF_CAP_FLAG_1_DYN_INTERRUPT_MODERATION) {
> +			cq_spec.req_cq_moderation = true;
> +			cq_spec.cq_moderation_usec = apc->intr_modr_tx_usec;
> +			cq_spec.cq_moderation_comp = apc->intr_modr_tx_comp;
> +		}
>   
>   		err = mana_create_wq_obj(apc, apc->port_handle, GDMA_SQ,
>   					 &wq_spec, &cq_spec,
> @@ -3206,10 +3205,14 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,
>   	cq_spec.modr_ctx_id = 0;
>   	cq_spec.attached_eq = cq->gdma_cq->cq.parent->id;
>   
> -	/* DIM setting can be changed at runtime */
> -	cq_spec.req_cq_moderation = true;
> -	cq_spec.cq_moderation_usec = apc->intr_modr_rx_usec;
> -	cq_spec.cq_moderation_comp = apc->intr_modr_rx_comp;
> +	/* Request CQ moderation only if HW supports it.
> +	 * DIM setting can be changed at runtime.
> +	 */
> +	if (gc->pf_cap_flags1 & GDMA_PF_CAP_FLAG_1_DYN_INTERRUPT_MODERATION) {
> +		cq_spec.req_cq_moderation = true;
> +		cq_spec.cq_moderation_usec = apc->intr_modr_rx_usec;
> +		cq_spec.cq_moderation_comp = apc->intr_modr_rx_comp;
> +	}
>   
>   	err = mana_create_wq_obj(apc, apc->port_handle, GDMA_RQ,
>   				 &wq_spec, &cq_spec, &rxq->rxobj);
> @@ -3894,6 +3897,40 @@ int mana_detach(struct net_device *ndev, bool from_close)
>   	return 0;
>   }
>   
> +/* Match the interrupt moderation settings to the PF capability, which is
> + * queried again when the device resumes from a reset or suspend. Must be
> + * called before the queues are created.
> + */
> +static void mana_init_intr_modr(struct mana_port_context *apc)
> +{
> +	struct gdma_context *gc = apc->ac->gdma_dev->gdma_context;
> +
> +	if (!(gc->pf_cap_flags1 &
> +	      GDMA_PF_CAP_FLAG_1_DYN_INTERRUPT_MODERATION)) {
> +		apc->intr_modr_rx_usec = 0;
> +		apc->intr_modr_rx_comp = 0;
> +		apc->intr_modr_tx_usec = 0;
> +		apc->intr_modr_tx_comp = 0;
> +		apc->rx_dim_enabled = false;
> +		apc->tx_dim_enabled = false;
> +		return;
> +	}
> +
> +	/* Keep the current settings, unless they were never initialized
> +	 * because the port was probed without the capability.
> +	 */
> +	if (apc->intr_modr_rx_usec && apc->intr_modr_rx_comp &&
> +	    apc->intr_modr_tx_usec && apc->intr_modr_tx_comp)
> +		return;
> +
> +	apc->intr_modr_rx_usec = MANA_INTR_MODR_USEC_DEF;
> +	apc->intr_modr_rx_comp = MANA_INTR_MODR_COMP_DEF;
> +	apc->intr_modr_tx_usec = MANA_INTR_MODR_USEC_DEF;
> +	apc->intr_modr_tx_comp = MANA_INTR_MODR_COMP_DEF;
> +	apc->rx_dim_enabled = MANA_ADAPTIVE_RX_DEF;
> +	apc->tx_dim_enabled = MANA_ADAPTIVE_TX_DEF;
> +}
> +
>   static int mana_probe_port(struct mana_context *ac, int port_idx,
>   			   struct net_device **ndev_storage)
>   {
> @@ -3924,15 +3961,7 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
>   	apc->cqe_coalescing_enable = 0;
>   	apc->cqe8_coalescing_enable = 0;
>   
> -	/* Initialize interrupt moderation settings if supported by HW */
> -	if (gc->pf_cap_flags1 & GDMA_PF_CAP_FLAG_1_DYN_INTERRUPT_MODERATION) {
> -		apc->intr_modr_rx_usec = MANA_INTR_MODR_USEC_DEF;
> -		apc->intr_modr_rx_comp = MANA_INTR_MODR_COMP_DEF;
> -		apc->intr_modr_tx_usec = MANA_INTR_MODR_USEC_DEF;
> -		apc->intr_modr_tx_comp = MANA_INTR_MODR_COMP_DEF;
> -		apc->rx_dim_enabled = MANA_ADAPTIVE_RX_DEF;
> -		apc->tx_dim_enabled = MANA_ADAPTIVE_TX_DEF;
> -	}
> +	mana_init_intr_modr(apc);
>   
>   	mutex_init(&apc->vport_mutex);
>   	apc->vport_use_count = 0;
> @@ -4256,6 +4285,7 @@ int mana_probe(struct gdma_dev *gd, bool resuming)
>   			enable_work(&apc->queue_reset_work);
>   			netdev_lock(ac->ports[i]);
>   			apc->link_cfg_error = 1;
> +			mana_init_intr_modr(apc);
>   			netdev_unlock(ac->ports[i]);
>   			err = mana_attach(ac->ports[i]);
>   			rtnl_unlock();
> diff --git a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
> index 4068e08acb16..96aabf7a8f08 100644
> --- a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
> +++ b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
> @@ -505,9 +505,12 @@ static int mana_set_coalesce(struct net_device *ndev,
>   	bool dim_changed = false;
>   	struct gdma_context *gc;
>   	u32 max_cqe_frames;
> +	bool modr_sup;
>   	int err;
>   
>   	gc = apc->ac->gdma_dev->gdma_context;
> +	modr_sup = gc->pf_cap_flags1 &
> +		   GDMA_PF_CAP_FLAG_1_DYN_INTERRUPT_MODERATION;
>   	max_cqe_frames = gc->cqe8_coalescing_sup ? MANA_CQE_COAL_PKTS_8 :
>   						   MANA_RXCOMP_OOB_NUM_PPI;
>   
> @@ -517,7 +520,7 @@ static int mana_set_coalesce(struct net_device *ndev,
>   	if ((ec->use_adaptive_rx_coalesce || ec->use_adaptive_tx_coalesce ||
>   	     ec->rx_coalesce_usecs || ec->tx_coalesce_usecs ||
>   	     ec->rx_max_coalesced_frames || ec->tx_max_coalesced_frames) &&
> -	    !(gc->pf_cap_flags1 & GDMA_PF_CAP_FLAG_1_DYN_INTERRUPT_MODERATION)) {
> +	    !modr_sup) {
>   		NL_SET_ERR_MSG(extack,
>   			       "Interrupt Moderation is not supported by HW");
>   		return -EOPNOTSUPP;
> @@ -534,18 +537,27 @@ static int mana_set_coalesce(struct net_device *ndev,
>   		return -EINVAL;
>   	}
>   
> -	if (ec->rx_coalesce_usecs > MANA_INTR_MODR_USEC_MAX ||
> -	    ec->tx_coalesce_usecs > MANA_INTR_MODR_USEC_MAX) {
> +	/* Without the HW capability, only zero values pass the check above */
> +	if (modr_sup &&
> +	    (ec->rx_coalesce_usecs < MANA_INTR_MODR_USEC_MIN ||
> +	     ec->tx_coalesce_usecs < MANA_INTR_MODR_USEC_MIN ||
> +	     ec->rx_coalesce_usecs > MANA_INTR_MODR_USEC_MAX ||
> +	     ec->tx_coalesce_usecs > MANA_INTR_MODR_USEC_MAX)) {
>   		NL_SET_ERR_MSG_FMT(extack,
> -				   "coalesce usecs must be <= %lu",
> +				   "coalesce usecs must be between %d and %lu",
> +				   MANA_INTR_MODR_USEC_MIN,
>   				   MANA_INTR_MODR_USEC_MAX);
>   		return -EINVAL;
>   	}
>   
> -	if (ec->rx_max_coalesced_frames > MANA_INTR_MODR_COMP_MAX ||
> -	    ec->tx_max_coalesced_frames > MANA_INTR_MODR_COMP_MAX) {
> +	if (modr_sup &&
> +	    (ec->rx_max_coalesced_frames < MANA_INTR_MODR_COMP_MIN ||
> +	     ec->tx_max_coalesced_frames < MANA_INTR_MODR_COMP_MIN ||
> +	     ec->rx_max_coalesced_frames > MANA_INTR_MODR_COMP_MAX ||
> +	     ec->tx_max_coalesced_frames > MANA_INTR_MODR_COMP_MAX)) {
>   		NL_SET_ERR_MSG_FMT(extack,
> -				   "coalesce frames must be <= %lu",
> +				   "coalesce frames must be between %d and %lu",
> +				   MANA_INTR_MODR_COMP_MIN,
>   				   MANA_INTR_MODR_COMP_MAX);
>   		return -EINVAL;
>   	}
> diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
> index 29d152e18596..6191ddddc27f 100644
> --- a/include/net/mana/gdma.h
> +++ b/include/net/mana/gdma.h
> @@ -776,6 +776,9 @@ enum {
>   /* Driver supports handling throttled EQEs */
>   #define GDMA_DRV_CAP_FLAG_1_THROTTLED_EVENT_QUEUE BIT_ULL(32)
>   
> +/* Enforce min 1 usec, 1 frames for Interrupt Moderation */
> +#define GDMA_DRV_CAP_FLAG_1_SET_MIN_INTR_MOD_VALUE BIT_ULL(33)
> +
>   /* Capabilities in the PCI-only group below rely on dynamic MSI-X allocation
>    * and on the servicing and reset paths reached through
>    * mana_schedule_serv_work(). Transports that provide neither leave
> @@ -801,7 +804,8 @@ enum {
>   	 GDMA_DRV_CAP_FLAG_1_EQ_MSI_UNSHARE_MULTI_VPORT | \
>   	 GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION | \
>   	 GDMA_DRV_CAP_FLAG_1_NON_CONTIGUOUS_BUFFERS | \
> -	 GDMA_DRV_CAP_FLAG_1_THROTTLED_EVENT_QUEUE)
> +	 GDMA_DRV_CAP_FLAG_1_THROTTLED_EVENT_QUEUE | \
> +	 GDMA_DRV_CAP_FLAG_1_SET_MIN_INTR_MOD_VALUE)
>   
>   #define GDMA_DRV_CAP_FLAGS2 0
>   
> diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h
> index 218d540bc994..84afc4199e87 100644
> --- a/include/net/mana/mana.h
> +++ b/include/net/mana/mana.h
> @@ -74,9 +74,12 @@ enum mana_priv_flag_bits {
>   /* 8-pkt mode packs up to 2 packets per PPI entry */
>   #define MANA_CQE_COAL_PKTS_8 8
>   
> -/* Default/max interrupt moderation settings */
> -#define MANA_INTR_MODR_USEC_DEF 0
> -#define MANA_INTR_MODR_COMP_DEF 0
> +/* Default/min interrupt moderation settings */
> +#define MANA_INTR_MODR_USEC_DEF 1
> +#define MANA_INTR_MODR_COMP_DEF 1
> +
> +#define MANA_INTR_MODR_USEC_MIN 1
> +#define MANA_INTR_MODR_COMP_MIN 1
>   
>   #define MANA_ADAPTIVE_RX_DEF true
>   #define MANA_ADAPTIVE_TX_DEF true
LGTM, Reviewed-by: Sahil Chandna <sahilchandna@linux.microsoft.com>

      parent reply	other threads:[~2026-11-19 10:22 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-13 20:44 [PATCH net] net: mana: Set 1 usec / 1 comp as default and minimum interrupt moderation Haiyang Zhang
2026-11-17 20:49 ` Hamza Mahfooz
2026-11-19 10:22 ` Sahil Chandna [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cc705453-0794-494f-adb9-b6fb71254ba7@linux.microsoft.com \
    --to=sahilchandna@linux.microsoft.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=davem@davemloft.net \
    --cc=decui@microsoft.com \
    --cc=edumazet@kernel.org \
    --cc=ernis@linux.microsoft.com \
    --cc=gargaditya@linux.microsoft.com \
    --cc=haiyangz@microsoft.com \
    --cc=horms@kernel.org \
    --cc=kotaranov@microsoft.com \
    --cc=kuba@kernel.org \
    --cc=leitao@debian.org \
    --cc=linux-hyperv@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=mawasthi@linux.microsoft.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=paulros@microsoft.com \
    --cc=stable@vger.kernel.org \
    --cc=weh@microsoft.com \
    --cc=wei.liu@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox