[PATCH net/gve] gve: fix early boot deadlock between rtnl_lock and service_task_mutex

[PATCH net/gve] gve: fix early boot deadlock between rtnl_lock and service_task_mutex

[Rénich Bon Ćirić]

Hello Netdev Team,

We hit a reproducible hard deadlock (AB-BA circular lock inversion) during early boot on EL10 / RHEL 10 kernels when running Google Cloud Platform VMs with the gve driver:

1. Thread A (NetworkManager): Holds rtnl_lock during __dev_open -> gve_open -> gve_queues_start -> gve_add_napi, and attempts to acquire service_task_mutex.
2. Thread B (gve_service_task worker): Holds service_task_mutex, and calls reset recovery which attempts to acquire rtnl_lock.

Serial Console Trace:
"INFO: task NetworkManager:856 is blocked on a mutex likely owned by task NetworkManager:856" at RIP: copy_net_ns+0x1a4 / selinux_netlbl_sk_security_free.

- Working Kernels: 6.12.0-246.el10 and 6.12.0-233.el10.
- GitHub Issue #93: https://github.com/GoogleCloudPlatform/compute-virtual-ethernet-linux/issues/93
- Proposed PR #95: https://github.com/GoogleCloudPlatform/compute-virtual-ethernet-linux/pull/95

Best regards,

Rénich Bon Ćirić
Principal Infrastructure Architect
renich@woralelandia.com | renich@evalinux.com

Re: [PATCH net/gve] gve: fix early boot deadlock between rtnl_lock and service_task_mutex

[Jakub Kicinski]

On Tue, 4 Aug 2026 07:18:21 +0000 Rénich Bon Ćirić wrote:
> We hit a reproducible hard deadlock (AB-BA circular lock inversion) during early boot on EL10 / RHEL 10 kernels when running Google Cloud Platform VMs with the gve driver:
> 
> 1. Thread A (NetworkManager): Holds rtnl_lock during __dev_open -> gve_open -> gve_queues_start -> gve_add_napi, and attempts to acquire service_task_mutex.
> 2. Thread B (gve_service_task worker): Holds service_task_mutex, and calls reset recovery which attempts to acquire rtnl_lock.
> 
> Serial Console Trace:
> "INFO: task NetworkManager:856 is blocked on a mutex likely owned by task NetworkManager:856" at RIP: copy_net_ns+0x1a4 / selinux_netlbl_sk_security_free.

You emailed the upstream Linux list.
If the bug exists in the upstream kernel we will happily review the fix
- please post it to the list.

Re: [PATCH net/gve] gve: fix early boot deadlock between rtnl_lock and service_task_mutex

[Harshitha Ramamurthy]

On Wed, Aug 5, 2026 at 5:01 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Tue, 4 Aug 2026 07:18:21 +0000 Rénich Bon Ćirić wrote:
> > We hit a reproducible hard deadlock (AB-BA circular lock inversion) during early boot on EL10 / RHEL 10 kernels when running Google Cloud Platform VMs with the gve driver:
> >
> > 1. Thread A (NetworkManager): Holds rtnl_lock during __dev_open -> gve_open -> gve_queues_start -> gve_add_napi, and attempts to acquire service_task_mutex.
> > 2. Thread B (gve_service_task worker): Holds service_task_mutex, and calls reset recovery which attempts to acquire rtnl_lock.
> >
> > Serial Console Trace:
> > "INFO: task NetworkManager:856 is blocked on a mutex likely owned by task NetworkManager:856" at RIP: copy_net_ns+0x1a4 / selinux_netlbl_sk_security_free.

Hi,

Thanks for the report. We've communicated re: this on the Github issue
also but just an update that a fix is now merged for those kernels.
Like Jakub mentioned, if you also see this issue on the upstream
kernels, please let us know along with a splat and the exact kernel
version or commit ID so we could look into it.

Thanks,
Harshitha

>
> You emailed the upstream Linux list.
> If the bug exists in the upstream kernel we will happily review the fix
> - please post it to the list.