[REGRESSION] net: usb: asix: deadlock on interface setup

[REGRESSION] net: usb: asix: deadlock on interface setup

[Hubert Wiśniewski]

Trying to bring an AX88772B-based USB-Ethernet adapter up results in a
deadlock if the adapter was suspended at the time. Most network-related
software hangs up indefinitely as a result. This can happen on systems
which configure USB power control to 'auto' by default, e.g. laptops
running `tlp`.


Steps to reproduce:
  Try to bring the interface up while the adapter is suspended. For
  example, assuming that the device is on bus 1, port 1:
    root@usb-eth-test:/sys/bus/usb/devices/1-1/power# echo auto > control
    root@usb-eth-test:/sys/bus/usb/devices/1-1/power# cat runtime_status
    suspended
    root@usb-eth-test:/sys/bus/usb/devices/1-1/power# ip link set enp0s1u1 up


Expectations vs reality:
  The interface should be brought up and be able to operate, but instead
  the `ip` command hangs up and never returns, and lockdep emits the
  following warning (decoded here):
    ============================================
    WARNING: possible recursive locking detected
    6.17.0-rc3 #1 Not tainted
    --------------------------------------------
    ip/273 is trying to acquire lock:
    ffffffffb906e748 (rtnl_mutex){+.+.}-{4:4}, at: ax88772_resume (drivers/net/usb/asix_devices.c:650)

    but task is already holding lock:
    ffffffffb906e748 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink (net/core/rtnetlink.c:3893 net/core/rtnetlink.c:4057)

    other info that might help us debug this:
    Possible unsafe locking scenario:

    CPU0
    ----
    lock(rtnl_mutex);
    lock(rtnl_mutex);

    *** DEADLOCK ***

    May be due to missing lock nesting notation

    1 lock held by ip/273:
    #0: ffffffffb906e748 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink (net/core/rtnetlink.c:3893 net/core/rtnetlink.c:4057)

    stack backtrace:
    CPU: 0 UID: 0 PID: 273 Comm: ip Not tainted 6.17.0-rc3 #1 PREEMPT(voluntary)
    Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
    Call Trace:
    <TASK>
    dump_stack_lvl (lib/dump_stack.c:122)
    print_deadlock_bug.cold (kernel/locking/lockdep.c:3044)
    __lock_acquire (kernel/locking/lockdep.c:3897 kernel/locking/lockdep.c:5237)
    ? usb_start_wait_urb (drivers/usb/core/message.c:83)
    lock_acquire (kernel/locking/lockdep.c:470 (discriminator 4) kernel/locking/lockdep.c:5870 (discriminator 4) kernel/locking/lockdep.c:5825 (discriminator 4))
    ? ax88772_resume (drivers/net/usb/asix_devices.c:650)
    __mutex_lock (arch/x86/include/asm/jump_label.h:36 include/trace/events/lock.h:95 kernel/locking/mutex.c:600 kernel/locking/mutex.c:760)
    ? ax88772_resume (drivers/net/usb/asix_devices.c:650)
    ? ax88772_resume (drivers/net/usb/asix_devices.c:650)
    ? __usbnet_read_cmd (drivers/net/usb/usbnet.c:2065)
    ? ax88772_resume (drivers/net/usb/asix_devices.c:650)
    ? srso_alias_return_thunk (arch/x86/lib/retpoline.S:183)
    ax88772_resume (drivers/net/usb/asix_devices.c:650)
    asix_resume (drivers/net/usb/asix_devices.c:663)
    usb_resume_interface.isra.0 (drivers/usb/core/driver.c:1375)
    usb_resume_both (drivers/usb/core/driver.c:1532 (discriminator 1))
    ? __pfx_usb_runtime_resume (drivers/usb/core/driver.c:1981)
    __rpm_callback (drivers/base/power/runtime.c:406)
    ? srso_alias_return_thunk (arch/x86/lib/retpoline.S:183)
    ? srso_alias_return_thunk (arch/x86/lib/retpoline.S:183)
    rpm_callback (include/linux/sched/mm.h:339 (discriminator 1) include/linux/sched/mm.h:369 (discriminator 1) drivers/base/power/runtime.c:458 (discriminator 1))
    ? __pfx_usb_runtime_resume (drivers/usb/core/driver.c:1981)
    rpm_resume (drivers/base/power/runtime.c:934)
    ? srso_alias_return_thunk (arch/x86/lib/retpoline.S:183)
    ? lock_release (kernel/locking/lockdep.c:5536 kernel/locking/lockdep.c:5889 kernel/locking/lockdep.c:5875)
    rpm_resume (drivers/base/power/runtime.c:913)
    __pm_runtime_resume (include/linux/spinlock.h:406 drivers/base/power/runtime.c:1193)
    usb_autopm_get_interface (include/linux/pm_runtime.h:532 drivers/usb/core/driver.c:1828)
    usbnet_open (drivers/net/usb/usbnet.c:899)
    __dev_open (net/core/dev.c:1684)
    __dev_change_flags (net/core/dev.c:9549)
    netif_change_flags (net/core/dev.c:9612)
    do_setlink.isra.0 (net/core/rtnetlink.c:3143 (discriminator 1))
    ? lock_release (kernel/locking/lockdep.c:5536 kernel/locking/lockdep.c:5889 kernel/locking/lockdep.c:5875)
    ? rtnl_newlink (net/core/rtnetlink.c:3893 net/core/rtnetlink.c:4057)
    ? srso_alias_return_thunk (arch/x86/lib/retpoline.S:183)
    ? __mutex_lock (include/trace/events/lock.h:122 (discriminator 2) kernel/locking/mutex.c:607 (discriminator 2) kernel/locking/mutex.c:760 (discriminator 2))
    ? srso_alias_return_thunk (arch/x86/lib/retpoline.S:183)
    ? rtnl_newlink (net/core/rtnetlink.c:3893 net/core/rtnetlink.c:4057)
    ? lock_release (kernel/locking/lockdep.c:5536 kernel/locking/lockdep.c:5889 kernel/locking/lockdep.c:5875)
    rtnl_newlink (net/core/rtnetlink.c:3761 (discriminator 1) net/core/rtnetlink.c:3920 (discriminator 1) net/core/rtnetlink.c:4057 (discriminator 1))
    ? lock_acquire (kernel/locking/lockdep.c:470 (discriminator 4) kernel/locking/lockdep.c:5870 (discriminator 4) kernel/locking/lockdep.c:5825 (discriminator 4))
    ? srso_alias_return_thunk (arch/x86/lib/retpoline.S:183)
    ? find_held_lock (kernel/locking/lockdep.c:5350 (discriminator 1))
    ? rtnetlink_rcv_msg (include/linux/rcupdate.h:341 (discriminator 1) include/linux/rcupdate.h:871 (discriminator 1) net/core/rtnetlink.c:6944 (discriminator 1))
    ? __pfx_rtnl_newlink (net/core/rtnetlink.c:3948)
    ? srso_alias_return_thunk (arch/x86/lib/retpoline.S:183)
    ? rtnetlink_rcv_msg (net/core/rtnetlink.c:6945)
    ? srso_alias_return_thunk (arch/x86/lib/retpoline.S:183)
    ? __pfx_rtnetlink_rcv_msg (net/core/rtnetlink.c:6849)
    netlink_rcv_skb (net/netlink/af_netlink.c:2552)
    netlink_unicast (net/netlink/af_netlink.c:1321 net/netlink/af_netlink.c:1346)
    netlink_sendmsg (net/netlink/af_netlink.c:1896)
    ____sys_sendmsg (net/socket.c:714 (discriminator 1) net/socket.c:729 (discriminator 1) net/socket.c:2614 (discriminator 1))
    ___sys_sendmsg (net/socket.c:2670)
    ? lock_release (kernel/locking/lockdep.c:5536 kernel/locking/lockdep.c:5889 kernel/locking/lockdep.c:5875)
    __sys_sendmsg (net/socket.c:2700 (discriminator 1))
    ? srso_alias_return_thunk (arch/x86/lib/retpoline.S:183)
    do_syscall_64 (arch/x86/entry/syscall_64.c:63 (discriminator 1) arch/x86/entry/syscall_64.c:94 (discriminator 1))
    entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130)
    RIP: 0033:0x7fddcbcb2687
    Code: 48 89 fa 4c 89 df e8 58 b3 00 00 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 1a 5b c3 0f 1f 84 00 00 00 00 00 48 8b 44 24 10 0f 05 <5b> c3 0f 1f 80 00 00 00 00 83 e2 39 83 fa 08 75 de e8 23 ff ff ff
    All code
    ========
       0:	48 89 fa             	mov    %rdi,%rdx
       3:	4c 89 df             	mov    %r11,%rdi
       6:	e8 58 b3 00 00       	call   0xb363
       b:	8b 93 08 03 00 00    	mov    0x308(%rbx),%edx
      11:	59                   	pop    %rcx
      12:	5e                   	pop    %rsi
      13:	48 83 f8 fc          	cmp    $0xfffffffffffffffc,%rax
      17:	74 1a                	je     0x33
      19:	5b                   	pop    %rbx
      1a:	c3                   	ret
      1b:	0f 1f 84 00 00 00 00 	nopl   0x0(%rax,%rax,1)
      22:	00
      23:	48 8b 44 24 10       	mov    0x10(%rsp),%rax
      28:	0f 05                	syscall
      2a:*	5b                   	pop    %rbx		<-- trapping instruction
      2b:	c3                   	ret
      2c:	0f 1f 80 00 00 00 00 	nopl   0x0(%rax)
      33:	83 e2 39             	and    $0x39,%edx
      36:	83 fa 08             	cmp    $0x8,%edx
      39:	75 de                	jne    0x19
      3b:	e8 23 ff ff ff       	call   0xffffffffffffff63

    Code starting with the faulting instruction
    ===========================================
       0:	5b                   	pop    %rbx
       1:	c3                   	ret
       2:	0f 1f 80 00 00 00 00 	nopl   0x0(%rax)
       9:	83 e2 39             	and    $0x39,%edx
       c:	83 fa 08             	cmp    $0x8,%edx
       f:	75 de                	jne    0xffffffffffffffef
      11:	e8 23 ff ff ff       	call   0xffffffffffffff39
    RSP: 002b:00007fffad1c7b60 EFLAGS: 00000202 ORIG_RAX: 000000000000002e
    RAX: ffffffffffffffda RBX: 00007fddcba85840 RCX: 00007fddcbcb2687
    RDX: 0000000000000000 RSI: 00007fffad1c7c10 RDI: 0000000000000003
    RBP: 00007fffad1c7c10 R08: 0000000000000000 R09: 0000000000000000
    R10: 0000000000000000 R11: 0000000000000202 R12: 00007fffad1c8300
    R13: 0000000000000000 R14: 0000555b0fa13020 R15: 0000000000000000
    </TASK>


Details:
  I have used QEMU with USB passthrough to catch the issue, but this
  happens on real hardware as well.

  The bug has already been reported on bugzilla
  (https://bugzilla.kernel.org/show_bug.cgi?id=215199), but this was
  probably not the right place.

  USB device: Edimax USB 2.0 Fast Ethernet Adapter, model no. EU-4208
  USB device IDs: 0b95:772b ASIX Electronics Corp. AX88772B

  Kernel version (/proc/version):
    Linux version 6.17.0-rc3 (hubert25632@B550M-AE) (gcc (Debian 14.2.0-19)
    14.2.0, GNU ld (GNU Binutils for Debian) 2.44) #1 SMP PREEMPT_DYNAMIC
    Sat Aug 30 21:43:30 CEST 2025

  iproute2 version: ip utility, iproute2-6.16.0, libbpf 1.6.2
  OS: Debian GNU/Linux forky/sid
  CPU architecture: x86_64
  Kernel config: https://pastebin.com/MiBZnCgC
  dmesg log: https://pastebin.com/JXiZTiAT

  Last good kernel version: v5.13
  First bad commit: 4a2c7217cd5a ("net: usb: asix: ax88772: manage PHY PM from MAC")


Possible workaroud:
  Users of `tlp` can add the following statement to `/etc/tlp.conf` to
  prevent the adapter from being suspended automatically:
    USB_DENYLIST="0b95:772b"


#regzbot introduced: 4a2c7217cd5a

Re: [REGRESSION] net: usb: asix: deadlock on interface setup

[Jakub Kicinski]

On Sun, 31 Aug 2025 10:50:35 +0200 Hubert Wiśniewski wrote:
> Trying to bring an AX88772B-based USB-Ethernet adapter up results in a
> deadlock if the adapter was suspended at the time. Most network-related
> software hangs up indefinitely as a result. This can happen on systems
> which configure USB power control to 'auto' by default, e.g. laptops
> running `tlp`.
 
Oleksij, this seems to date back to commit e0bffe3e6894 ("net: asix:
ax88772: migrate to phylink"). Taking rtnl_lock in runtime resume
callbacks is known to result in unhappiness :(

Could you check if commit e110bc825897 ("net: usb:
lan78xx: Convert to PHYLINK for improved PHY and MAC management")
isn't similarly flawed?

Re: [REGRESSION] net: usb: asix: deadlock on interface setup

[Oleksij Rempel]

On Tue, Sep 02, 2025 at 04:35:11PM -0700, Jakub Kicinski wrote:
> On Sun, 31 Aug 2025 10:50:35 +0200 Hubert Wiśniewski wrote:
> > Trying to bring an AX88772B-based USB-Ethernet adapter up results in a
> > deadlock if the adapter was suspended at the time. Most network-related
> > software hangs up indefinitely as a result. This can happen on systems
> > which configure USB power control to 'auto' by default, e.g. laptops
> > running `tlp`.
>  
> Oleksij, this seems to date back to commit e0bffe3e6894 ("net: asix:
> ax88772: migrate to phylink"). Taking rtnl_lock in runtime resume
> callbacks is known to result in unhappiness :(
> 
> Could you check if commit e110bc825897 ("net: usb:
> lan78xx: Convert to PHYLINK for improved PHY and MAC management")
> isn't similarly flawed?

Yes, sorry for delay answer.  I'll investigate it at Friday.

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |