All of lore.kernel.org
 help / color / mirror / Atom feed
* [BUG] cdc_ncm: soft lockup with netconsole over an AX88179B dock
@ 2026-10-19 13:48 Mikkel Eskildsen
  2026-10-19 13:56 ` Mikkel Eskildsen
  2026-10-19 14:42 ` Nicolai Buchwitz
  0 siblings, 2 replies; 4+ messages in thread
From: Mikkel Eskildsen @ 2026-10-19 13:48 UTC (permalink / raw)
  To: Oliver Neukum, Breno Leitao
  Cc: netdev@vger.kernel.org, linux-usb@vger.kernel.org, Jakub Kicinski,
	Eric Dumazet, Paolo Abeni

Hi,

My laptop panicked with a soft lockup on 2026-10-16 and again on 2026-10-19 while netconsole was running over a USB dock NIC that uses cdc_ncm. Both times the laptop was docked and netconsole was on that interface. I have turned netconsole off for now.

The trace below is from pstore. The netconsole thread on CPU15 is in netpoll_send_skb -> usbnet_start_xmit -> cdc_ncm_tx_fixup. The spin_unlock_bh() in cdc_ncm_tx_fixup runs pending softirqs, TCP RX sends a packet and it spins in sch_direct_xmit. My guess is that it waits for the eth0 tx queue lock that netpoll already holds. The other CPUs then wait on the socket lock until the watchdog panics.

The first panic looks the same, but pstore lost the part with the netconsole thread.

I have not tried mainline, but cdc_ncm_tx_fixup and netpoll_send_skb look the same there.

Kernel: 7.2.9-1-cachyos, PREEMPT(full)
Machine: Lenovo Yoga Book 9 14IAH10 (83KJ), BIOS QECN29WW
NIC: ASIX AX88179B (0b95:1790) on cdc_ncm, in a USB dock
netconsole: one target on eth0, kernel.printk = 7 4 1 7, kernel.softlockup_panic = 1
Taint O/E is a patched out of tree wacom HID module for the touch panels.

[271447.258658] watchdog: BUG: soft lockup - CPU#9 stuck for 13s! [python:3582585]
[271447.258667] CPU#9 Utilization every 2000ms during lockup:
[271447.258669] 	#1: 100% system,	  0% softirq,	  1% hardirq,	  0% idle
[271447.258672] 	#2: 100% system,	  0% softirq,	  1% hardirq,	  0% idle
[271447.258674] 	#3: 100% system,	  0% softirq,	  2% hardirq,	  0% idle
[271447.258675] 	#4: 100% system,	  0% softirq,	  1% hardirq,	  0% idle
[271447.258676] 	#5: 100% system,	  0% softirq,	  1% hardirq,	  0% idle
[271447.258678] Modules linked in: ntfs3 wireguard libcurve25519 ip6_udp_tunnel udp_tunnel cp210x tcp_diag inet_diag uinput rfcomm tun ccm snd_seq_dummy snd_hrtimer snd_seq nft_masq nft_ct nft_reject_ipv4 nf_reject_ipv4 nft_reject act_csum cls_u32 sch_htb nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables bridge stp llc netconsole algif_hash algif_skcipher af_alg bnep snd_usb_audio snd_usbmidi_lib snd_ump snd_rawmidi snd_seq_device hid_sensor_custom_intel_hinge hid_sensor_gyro_3d hid_sensor_accel_3d hid_sensor_custom intel_ishtp_hid snd_ctl_led snd_soc_skl_hda_dsp snd_soc_intel_sof_board_helpers snd_sof_probes snd_soc_intel_hda_dsp_common snd_hda_codec_intelhdmi snd_hda_codec_hdmi snd_hda_codec_alc269 snd_hda_codec_realtek_lib snd_hda_scodec_component snd_hda_codec_generic snd_soc_dmic snd_hda_intel snd_sof_pci_intel_mtl snd_sof_intel_hda_generic soundwire_intel snd_sof_intel_hda_sdw_bpt hid_sensor_als hid_sensor_prox hid_sensor_trigger snd_sof_intel_hda_common intel_rapl_msr kfifo_buf
[271447.258727]  hid_sensor_iio_common snd_soc_hdac_hda intel_uncore_frequency intel_uncore_frequency_common snd_sof_intel_hda industrialio snd_sof_intel_hda_mlink intel_tcc_cooling soundwire_cadence snd_sof_pci snd_sof_xtensa_dsp x86_pkg_temp_thermal intel_powerclamp snd_sof coretemp snd_sof_utils snd_hda_ext_core vfat snd_hda_codec fat snd_hda_core ucsi_acpi kvm_intel typec_ucsi snd_soc_acpi_intel_match snd_soc_acpi_intel_sdca_quirks roles iwlmld snd_intel_dspcfg typec snd_intel_sdw_acpi soundwire_generic_allocation snd_hda_scodec_cs35l56_spi mac80211 mei_gsc_proxy snd_soc_sdw_utils snd_soc_acpi kvm snd_hwdep soundwire_bus ptp snd_soc_sdca irqbypass yogafan pps_core aesni_intel libarc4 snd_soc_core gf128mul ac97_bus aead snd_hda_scodec_cs35l56_i2c snd_pcm_dmaengine snd_hda_scodec_cs35l56 snd_compress rapl snd_hda_cirrus_scodec lenovo_ymc cdc_mbim intel_cstate snd_pcm snd_soc_cs35l56_shared iwlwifi lenovo_wmi_gamezone processor_thermal_device_pci snd_soc_cs_amp_lib lenovo_wmi_other spi_nor processor_thermal_device
[271447.258771]  snd_timer cs_dsp lenovo_wmi_events cdc_wdm intel_uncore processor_thermal_power_floor lenovo_wmi_capdata uvcvideo pcspkr processor_thermal_wt_hint i2c_i801 snd firmware_attributes_class wmi_bmof lenovo_wmi_helpers lenovo_wmi_hotkey_utilities mtd uvc processor_thermal_wt_req mei_me btusb i2c_smbus videobuf2_vmalloc soundcore cfg80211 intel_pmc_core intel_ish_ipc crc8 btmtk i2c_mux intel_vpu processor_thermal_rfim videobuf2_memops mei thunderbolt intel_ishtp btbcm processor_thermal_mbox serial_multi_instantiate cdc_ncm videobuf2_v4l2 intel_pmc_ssram_telemetry btintel platform_temperature_control videobuf2_common cdc_ether processor_thermal_rapl intel_pmc_pwrm_telemetry btrtl ideapad_laptop soc_button_array int3403_thermal intel_rapl_common usbnet psmouse pmt_telemetry videodev processor_thermal_soc_slider bluetooth int3400_thermal pmt_discovery joydev intel_hid mousedev mii pmt_class mc acpi_thermal_rel sparse_keymap hid_sensor_hub pinctrl_meteorlake igen6_edac int340x_thermal_zone cdc_acm mac_hid wacom(OE)
[271447.258818]  acpi_pad acpi_tad rfkill dm_mod usbip_host iTCO_wdt usbip_core i2c_dev pkcs8_key_parser sg ntsync crypto_user intel_pmc_bxt zram 842_decompress 842_compress lz4hc_compress xe gpu_sched drm_ttm_helper drm_gpuvm drm_exec drm_gpusvm_helper drm_suballoc_helper i915 drm_buddy nvme intel_gtt spi_intel_pci nvme_core ttm spi_intel i2c_algo_bit nvme_keyring video nvme_auth intel_lpss_pci drm_display_helper wmi intel_lpss idma64 cec intel_vsec serio_raw hid_multitouch
[271447.258850] CPU: 9 UID: 1000 PID: 3582585 Comm: python Tainted: G           OE       7.2.9-1-cachyos #1 PREEMPT(full)  458eaf81e04e4bc0eb2f59b85134c9935c8c6b28
[271447.258858] Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
[271447.258859] Hardware name: LENOVO 83KJ/LNVNB161216, BIOS QECN29WW 04/16/2026
[271447.258862] RIP: 0010:native_queued_spin_lock_slowpath+0x55/0x260
[271447.258874] Code: c0 f0 0f ba 2f 08 0f 92 c0 89 c2 c1 e2 08 b9 ff 00 ff ff 23 0f 09 ca 81 fa 00 01 00 00 73 5a 85 c9 74 0e 66 90 80 3f 00 74 07 <f3> 90 80 3f 00 75 f9 66 c7 07 01 00 65 48 ff 05 1f 51 1c 02 5b 41
[271447.258877] RSP: 0018:ffffd294037c7980 EFLAGS: 00000202
[271447.258880] RAX: 0000000000000000 RBX: ffff8acb2f082700 RCX: 0000000000000001
[271447.258882] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8acb2f08284c
[271447.258883] RBP: 0000000000000040 R08: 0000000000002000 R09: 0000000000000000
[271447.258885] R10: 0000000000000000 R11: ffffffffb45d8eb0 R12: ffffd294037c7a30
[271447.258887] R13: 0000000000002656 R14: ffff8acb2f08284c R15: 0000000000002656
[271447.258889] FS:  00007ff425ffb6c0(0000) GS:ffff8ad128893000(0000) knlGS:0000000000000000
[271447.258892] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[271447.258893] CR2: 0000555febdd8030 CR3: 0000000147562002 CR4: 0000000008f72ef0
[271447.258896] PKRU: 55555554
[271447.258896] Call Trace:
[271447.258899]  <TASK>
[271447.258905]  _raw_spin_lock_bh+0x2e/0x40
[271447.258910]  lock_sock_nested+0x35/0x70
[271447.258916]  tcp_recvmsg+0x68/0x1d0
[271447.258921]  sock_read_iter+0x1c7/0x1d0
[271447.258928]  __x64_sys_read+0x233/0x450
[271447.258933]  do_syscall_64+0xa6/0x3e0
[271447.258941]  ? arch_exit_to_user_mode_prepare.cold+0x5/0x79
[271447.258946]  ? do_syscall_64+0xe4/0x3e0
[271447.258951]  ? __x64_sys_read+0x2ca/0x450
[271447.258955]  ? do_syscall_64+0xe4/0x3e0
[271447.258959]  ? arch_exit_to_user_mode_prepare.cold+0x5/0x79
[271447.258962]  ? do_syscall_64+0xe4/0x3e0
[271447.258966]  ? do_syscall_64+0x65/0x3e0
[271447.258971]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
[271447.258974] RIP: 0033:0x7ff478ab5a22
[271447.259027] Code: 08 0f 85 b1 32 ff ff 49 89 fb 48 89 f0 48 89 d7 48 89 ce 4c 89 c2 4d 89 ca 4c 8b 44 24 08 4c 8b 4c 24 10 4c 89 5c 24 08 0f 05 <c3> 66 2e 0f 1f 84 00 00 00 00 00 66 2e 0f 1f 84 00 00 00 00 00 66
[271447.259029] RSP: 002b:00007ff425ffa2f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[271447.259032] RAX: ffffffffffffffda RBX: 00007ff4140322b0 RCX: 00007ff478ab5a22
[271447.259034] RDX: 0000000000002656 RSI: 00007ff414156013 RDI: 0000000000000017
[271447.259035] RBP: 00007ff425ffa350 R08: 0000000000000000 R09: 0000000000000000
[271447.259036] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000002656
[271447.259038] R13: 00007ff425ffa3c8 R14: 00007ff425ffb660 R15: 00007ff414156013
[271447.259040]  </TASK>
[271447.259098] NMI backtrace for cpu 15
[271447.259105] CPU: 15 UID: 0 PID: 1781 Comm: pr/netcon0 Tainted: G           OE       7.2.9-1-cachyos #1 PREEMPT(full)  458eaf81e04e4bc0eb2f59b85134c9935c8c6b28
[271447.259111] Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
[271447.259112] Hardware name: LENOVO 83KJ/LNVNB161216, BIOS QECN29WW 04/16/2026
[271447.259114] RIP: 0010:native_queued_spin_lock_slowpath+0x57/0x260
[271447.259122] Code: 0f ba 2f 08 0f 92 c0 89 c2 c1 e2 08 b9 ff 00 ff ff 23 0f 09 ca 81 fa 00 01 00 00 73 5a 85 c9 74 0e 66 90 80 3f 00 74 07 f3 90 <80> 3f 00 75 f9 66 c7 07 01 00 65 48 ff 05 1f 51 1c 02 5b 41 5e 41
[271447.259124] RSP: 0018:ffffd293c05588d8 EFLAGS: 00000202
[271447.259127] RAX: 0000000000000000 RBX: ffff8acb65f57000 RCX: 0000000000000001
[271447.259128] RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8acb8ece5f00
[271447.259129] RBP: 000000000000000f R08: 000000006916127b R09: 0000000000000001
[271447.259131] R10: ffffd293c0558a80 R11: 0000000000000000 R12: ffff8acb65f570d4
[271447.259132] R13: ffff8acb8ece5e00 R14: ffff8acf6e25ef00 R15: ffff8aca3363b000
[271447.259134] FS:  0000000000000000(0000) GS:ffff8ad128a13000(0000) knlGS:0000000000000000
[271447.259136] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[271447.259137] CR2: 00007f16f88ef00b CR3: 0000000296412001 CR4: 0000000008f72ef0
[271447.259139] PKRU: 55555554
[271447.259140] Call Trace:
[271447.259142]  <IRQ>
[271447.259146]  _raw_spin_lock+0x2a/0x40
[271447.259150]  sch_direct_xmit+0x98/0x1d0
[271447.259156]  __dev_queue_xmit+0x415/0xc10
[271447.259159]  ? __dev_queue_xmit+0x7f/0xc10
[271447.259165]  ? nf_conntrack_in+0x379/0x780 [nf_conntrack a48fe096bf19282d81b91569c939d30474edf370]
[271447.259188]  ip_finish_output2+0x262/0x450
[271447.259192]  ip_output+0xaf/0xf0
[271447.259194]  ? __pfx_ip_finish_output+0x10/0x10
[271447.259197]  __ip_queue_xmit+0x263/0x3b0
[271447.259201]  __tcp_transmit_skb+0x553/0x1210
[271447.259205]  ? __alloc_skb+0x30d/0x370
[271447.259207]  tcp_v4_do_rcv+0x87/0x340
[271447.259211]  tcp_v4_rcv+0x46b/0xde0
[271447.259214]  ip_protocol_deliver_rcu+0x4c/0x120
[271447.259218]  ip_local_deliver_finish+0x7d/0xf0
[271447.259221]  ip_local_deliver+0xa6/0x100
[271447.259223]  ? __pfx_ip_local_deliver_finish+0x10/0x10
[271447.259226]  ip_rcv+0x98/0xd0
[271447.259228]  ? __pfx_ip_rcv_finish+0x10/0x10
[271447.259231]  process_backlog+0x17d/0x270
[271447.259234]  __napi_poll+0x2c/0x150
[271447.259238]  net_rx_action+0x17d/0x270
[271447.259242]  handle_softirqs+0x126/0x1b0
[271447.259247]  __local_bh_enable_ip+0x9c/0xc0
[271447.259251]  </IRQ>
[271447.259251]  <TASK>
[271447.259253]  cdc_ncm_tx_fixup+0x5e/0x90 [cdc_ncm 795c10151ac1efda840e316dcec923133205dc76]
[271447.259261]  usbnet_start_xmit+0x6e/0x880 [usbnet 9bb700a5a823900feca1a3fc00259009646b31b2]
[271447.259268]  netpoll_start_xmit+0x1d8/0x240
[271447.259273]  netpoll_send_skb+0x182/0x230
[271447.259277]  send_udp+0x510/0x550 [netconsole 3dd9ae90ed093db9d831cffd8ca5dea61ecdd887]
[271447.259283]  netconsole_write_basic+0xbe/0xd0 [netconsole 3dd9ae90ed093db9d831cffd8ca5dea61ecdd887]
[271447.259289]  nbcon_emit_next_record+0x6e9/0xc50
[271447.259294]  nbcon_kthread_func+0x2b4/0x360
[271447.259298]  ? __pfx_nbcon_kthread_func+0x10/0x10
[271447.259301]  kthread+0x119/0x120
[271447.259305]  ? __pfx_kthread+0x10/0x10
[271447.259308]  ret_from_fork+0xee/0x2b0
[271447.259312]  ? __pfx_kthread+0x10/0x10
[271447.259314]  ret_from_fork_asm+0x1a/0x30
[271447.259319]  </TASK>

I can test a patch if that helps. Is this a known issue?

Mikkel

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2026-10-19 17:37 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-19 13:48 [BUG] cdc_ncm: soft lockup with netconsole over an AX88179B dock Mikkel Eskildsen
2026-10-19 13:56 ` Mikkel Eskildsen
2026-10-19 14:42 ` Nicolai Buchwitz
2026-10-19 17:37   ` Mikkel Eskildsen

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.