[PATCH net] gve: use gve_schedule_reset() on AdminQ flow rule timeout

[PATCH net] gve: use gve_schedule_reset() on AdminQ flow rule timeout

[Rénich Bon Ćirić]

When an AdminQ command to configure a flow rule times out in
gve_adminq_configure_flow_rule(), the driver currently calls
gve_reset(priv, true) synchronously.

When flow rule configuration is invoked under netlink or ethtool
callbacks, gve_reset() attempts to acquire rtnl_lock while already
executing within an inner driver lock context. This violates the driver's
lock hierarchy and causes an AB-BA circular lock inversion deadlock.

Resolve this lock inversion by replacing synchronous gve_reset() with
gve_schedule_reset(). This defers reset execution safely to
gve_service_task out-of-band under the proper lock ordering.

Fixes: 8ffade77b633 ("gve: Flow steering trigger reset only for timeout error")
Link: https://github.com/GoogleCloudPlatform/compute-virtual-ethernet-linux/issues/93
Signed-off-by: Rénich Bon Ćirić <renich@woralelandia.com>
---
 drivers/net/ethernet/google/gve/gve_adminq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c
index 08587bf12345..2037b1767890 100644
--- a/drivers/net/ethernet/google/gve/gve_adminq.c
+++ b/drivers/net/ethernet/google/gve/gve_adminq.c
@@ -1322,7 +1322,7 @@ gve_adminq_configure_flow_rule(struct gve_priv *priv,
 
 	if (err == -ETIME) {
 		dev_err(&priv->pdev->dev, "Timeout to configure the flow rule, trigger reset");
-		gve_reset(priv, true);
+		gve_schedule_reset(priv);
 	} else if (!err) {
 		priv->flow_rules_cache.rules_cache_synced = false;
 	}
-- 
2.45.2

Re: [PATCH net] gve: use gve_schedule_reset() on AdminQ flow rule timeout

[Przemek Kitszel]

On 8/6/26 10:51, Rénich Bon Ćirić wrote:
> When an AdminQ command to configure a flow rule times out in
> gve_adminq_configure_flow_rule(), the driver currently calls
> gve_reset(priv, true) synchronously.
> 
> When flow rule configuration is invoked under netlink or ethtool
> callbacks, gve_reset() attempts to acquire rtnl_lock while already
> executing within an inner driver lock context. This violates the driver's
> lock hierarchy and causes an AB-BA circular lock inversion deadlock.
> 
> Resolve this lock inversion by replacing synchronous gve_reset() with
> gve_schedule_reset(). This defers reset execution safely to
> gve_service_task out-of-band under the proper lock ordering.
> 
> Fixes: 8ffade77b633 ("gve: Flow steering trigger reset only for timeout error")

this is not the commit that introduced the bug

initial implementation was in commit 57718b60df9b ("gve: Add flow 
steering adminq commands"), so this could be used as "wrong code intro",
but likely it was the addition of netdev_lock() around ethtool call,
that exposed the issue, and should be the actual Fixes tag

> Link: https://github.com/GoogleCloudPlatform/compute-virtual-ethernet-linux/issues/93
> Signed-off-by: Rénich Bon Ćirić <renich@woralelandia.com>
> ---
>   drivers/net/ethernet/google/gve/gve_adminq.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c
> index 08587bf12345..2037b1767890 100644
> --- a/drivers/net/ethernet/google/gve/gve_adminq.c
> +++ b/drivers/net/ethernet/google/gve/gve_adminq.c
> @@ -1322,7 +1322,7 @@ gve_adminq_configure_flow_rule(struct gve_priv *priv,
>   
>   	if (err == -ETIME) {
>   		dev_err(&priv->pdev->dev, "Timeout to configure the flow rule, trigger reset");
> -		gve_reset(priv, true);
> +		gve_schedule_reset(priv);
>   	} else if (!err) {
>   		priv->flow_rules_cache.rules_cache_synced = false;
>   	}

code looks good, after updating Fixes tag, feel free to add my:
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>

Re: [PATCH net] gve: use gve_schedule_reset() on AdminQ flow rule timeout

[Harshitha Ramamurthy]

On Thu, Aug 6, 2026 at 1:51 AM Rénich Bon Ćirić <renich@evalinux.com> wrote:
>
> When an AdminQ command to configure a flow rule times out in
> gve_adminq_configure_flow_rule(), the driver currently calls
> gve_reset(priv, true) synchronously.
>
> When flow rule configuration is invoked under netlink or ethtool
> callbacks, gve_reset() attempts to acquire rtnl_lock while already
> executing within an inner driver lock context. This violates the driver's
> lock hierarchy and causes an AB-BA circular lock inversion deadlock.
>
> Resolve this lock inversion by replacing synchronous gve_reset() with
> gve_schedule_reset(). This defers reset execution safely to
> gve_service_task out-of-band under the proper lock ordering.
>
> Fixes: 8ffade77b633 ("gve: Flow steering trigger reset only for timeout error")
> Link: https://github.com/GoogleCloudPlatform/compute-virtual-ethernet-linux/issues/93

Hi Renich,

Did you encounter this issue on the upstream net kernel? Because the
Github issue you have linked is referencing the CentOS Stream 10
kernel version 6.12.0-248.el10.x86_64. We scanned our backend within
Google to see if this reset causing error is encountered on VMs and we
did not see such an occurrence, at least recently. If you ran into
this on a non-upstream kernel, would be great if you could provide
more information on the Github issue or file a support ticket so we
could look into it off the mailing list.

If you did hit this issue on an upstream kernel, could you also please
provide a splat here?

Thanks,
Harshitha

> Signed-off-by: Rénich Bon Ćirić <renich@woralelandia.com>
> ---
>  drivers/net/ethernet/google/gve/gve_adminq.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c
> index 08587bf12345..2037b1767890 100644
> --- a/drivers/net/ethernet/google/gve/gve_adminq.c
> +++ b/drivers/net/ethernet/google/gve/gve_adminq.c
> @@ -1322,7 +1322,7 @@ gve_adminq_configure_flow_rule(struct gve_priv *priv,
>
>         if (err == -ETIME) {
>                 dev_err(&priv->pdev->dev, "Timeout to configure the flow rule, trigger reset");
> -               gve_reset(priv, true);
> +               gve_schedule_reset(priv);
>         } else if (!err) {
>                 priv->flow_rules_cache.rules_cache_synced = false;
>         }
> --
> 2.45.2

Re: [PATCH net] gve: use gve_schedule_reset() on AdminQ flow rule timeout

[Rénich Bon Ćirić]

Issue #93 on GitHub was opened for the CentOS Stream 10 gve_add_napi deadlock, which MR !3007 addresses. The link was included in v1 in error.

However, gve_adminq_configure_flow_rule() calling gve_reset(priv, true) synchronously on AdminQ timeout remains an AB-BA lock inversion bug under ethtool/netlink callbacks.

I have posted [PATCH net v2] with the GitHub link removed, the Fixes tag updated to 57718b60df9b ("gve: Add flow steering adminq commands"), and Przemek Kitszel's Reviewed-by tag added.