Linux block layer
 help / color / mirror / Atom feed
* [bug report] RIP: 0010:throtl_trim_slice+0xc6/0x320 caused kernel panic
@ 2023-09-22  2:10 Changhui Zhong
  2023-09-22  7:50 ` Ming Lei
                   ` (2 more replies)
  0 siblings, 3 replies; 12+ messages in thread
From: Changhui Zhong @ 2023-09-22  2:10 UTC (permalink / raw)
  To: linux-block; +Cc: Ming Lei

Hello,

triggered below issue with branch 'for-next',please help check,

INFO: HEAD of cloned kernel:
commit d975b468819a142a49c8e7db83feb07c3018c550
Merge: 58fac6aac7a2 df137dff93fc
Author: Jens Axboe <axboe@kernel.dk>
Date:   Wed Sep 13 13:16:19 2023 -0600

    Merge branch 'io_uring-futex' into for-next

    * io_uring-futex:
      io_uring: add support for vectored futex waits
      futex: make the vectored futex operations available
      futex: make futex_parse_waitv() available as a helper
      futex: add wake_data to struct futex_q
      io_uring: add support for futex wake and wait
      futex: abstract out a __futex_wake_mark() helper
      futex: factor out the futex wake handling
      futex: move FUTEX2_VALID_MASK to futex.h
      futex: Validate futex value against futex size
      futex: Flag conversion
      futex: Extend the FUTEX2 flags
      futex: Clarify FUTEX2 flags


reproduce steps:
echo "+cpuset +cpu +io" > /sys/fs/cgroup/cgroup.subtree_control
mkdir /sys/fs/cgroup/test
MAJ=$(ls -l /dev/"$disk" | awk -F ',' '{print $1}' | awk -F ' ' '{print $NF}')
MIN=$(ls -l /dev/"$disk" | awk -F ',' '{print $2}' | awk -F ' ' '{print $1}')
echo "$MAJ:$MIN wbps=1024" > /sys/fs/cgroup/test/io.max
echo $$ > /sys/fs/cgroup/test/cgroup.procs
dd if=/dev/zero of=/dev/$disk bs=10k count=1 oflag=direct &
dd if=/dev/zero of=/dev/$disk bs=10k count=1 oflag=direct &
wait

console log:
[ 1979.303330] divide error: 0000 [#1] PREEMPT SMP NOPTI
[ 1979.308393] CPU: 28 PID: 0 Comm: swapper/28 Not tainted 6.6.0-rc1+ #1
[ 1979.314837] Hardware name: Dell Inc. PowerEdge R7525/0590KW, BIOS
2.6.6 01/13/2022
[ 1979.322402] RIP: 0010:throtl_trim_slice+0xc6/0x320
[ 1979.327195] Code: 00 00 48 89 e8 48 f7 f1 48 29 d5 74 9f 40 0f b6
f6 48 89 df 89 34 24 e8 f8 f5 ff ff 8b 34 24 b9 e8 03 00 00 48 89 df
48 f7 e5 <48> f7 f1 49 03 85 f8 01 00 00 49 89 c7 e8 78 e0 ff ff ba ff
ff ff
[ 1979.345942] RSP: 0018:ffffadf9c2484dc0 EFLAGS: 00010847
[ 1979.351169] RAX: ffffffffffffd8f0 RBX: ffff8f5486a99800 RCX: 00000000000003e8
[ 1979.358300] RDX: 000000000000270f RSI: 0000000000000001 RDI: ffff8f5486a99800
[ 1979.365435] RBP: 0000000000002710 R08: ffffffffffffffff R09: ffff8f565a67b818
[ 1979.372565] R10: ffff8f5486a99810 R11: ffff8f565a67b830 R12: 0000000000000001
[ 1979.379698] R13: ffff8f5486a99808 R14: 0000000000000001 R15: 0000000000000021
[ 1979.386833] FS:  0000000000000000(0000) GS:ffff8f5677d00000(0000)
knlGS:0000000000000000
[ 1979.394916] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1979.400663] CR2: 00007f7db5443250 CR3: 0000000459420000 CR4: 0000000000350ee0
[ 1979.407797] Call Trace:
[ 1979.410250]  <IRQ>
[ 1979.412270]  ? die+0x33/0x90
[ 1979.415157]  ? do_trap+0xe0/0x110
[ 1979.418476]  ? throtl_trim_slice+0xc6/0x320
[ 1979.422662]  ? do_error_trap+0x65/0x80
[ 1979.426412]  ? throtl_trim_slice+0xc6/0x320
[ 1979.430600]  ? exc_divide_error+0x36/0x50
[ 1979.434612]  ? throtl_trim_slice+0xc6/0x320
[ 1979.438800]  ? asm_exc_divide_error+0x16/0x20
[ 1979.443163]  ? throtl_trim_slice+0xc6/0x320
[ 1979.447347]  tg_dispatch_one_bio+0xf0/0x1e0
[ 1979.451533]  throtl_pending_timer_fn+0x1e5/0x510
[ 1979.456152]  ? __pfx_throtl_pending_timer_fn+0x10/0x10
[ 1979.461287]  ? __pfx_throtl_pending_timer_fn+0x10/0x10
[ 1979.466427]  call_timer_fn+0x27/0x130
[ 1979.470096]  __run_timers.part.0+0x1ee/0x280
[ 1979.474368]  ? srso_return_thunk+0x5/0x10
[ 1979.478379]  ? __hrtimer_run_queues+0x121/0x2b0
[ 1979.482914]  ? srso_return_thunk+0x5/0x10
[ 1979.486925]  ? srso_return_thunk+0x5/0x10
[ 1979.490939]  ? srso_return_thunk+0x5/0x10
[ 1979.494953]  run_timer_softirq+0x26/0x50
[ 1979.498878]  __do_softirq+0xcb/0x2ab
[ 1979.502459]  __irq_exit_rcu+0xa1/0xc0
[ 1979.506124]  sysvec_apic_timer_interrupt+0x72/0x90
[ 1979.510917]  </IRQ>
[ 1979.513021]  <TASK>
[ 1979.515128]  asm_sysvec_apic_timer_interrupt+0x16/0x20
[ 1979.520265] RIP: 0010:mwait_idle+0x4f/0x70
[ 1979.524365] Code: 31 d2 48 89 d1 65 48 8b 04 25 80 18 03 00 0f 01
c8 48 8b 00 a8 08 75 13 eb 07 0f 00 2d 96 f3 36 00 31 c0 48 89 c1 fb
0f 01 c9 <fa> 65 48 8b 04 25 80 18 03 00 f0 80 60 02 df e9 6d 3f 01 00
0f ae
[ 1979.543111] RSP: 0018:ffffadf9c04a7ed0 EFLAGS: 00000246
[ 1979.548338] RAX: 0000000000000000 RBX: ffff8f548019cd40 RCX: 0000000000000000
[ 1979.555471] RDX: 0000000000000000 RSI: 0000000000000087 RDI: 000000000016e484
[ 1979.562605] RBP: 0000000000000000 R08: ffff8f5677d22a80 R09: 0000000000000000
[ 1979.569738] R10: 00000000000001dc R11: 0000000000000000 R12: 0000000000000000
[ 1979.576868] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 1979.584009]  default_idle_call+0x28/0xd0
[ 1979.587938]  cpuidle_idle_call+0x125/0x160
[ 1979.592036]  ? sched_clock_cpu+0xb/0x190
[ 1979.595963]  do_idle+0x7b/0xe0
[ 1979.599022]  cpu_startup_entry+0x19/0x20
[ 1979.602949]  start_secondary+0x115/0x140
[ 1979.606875]  secondary_startup_64_no_verify+0x17d/0x18b
[ 1979.612104]  </TASK>
[ 1979.614293] Modules linked in: binfmt_misc dm_crypt raid10 raid1
raid0 dm_raid raid456 async_raid6_recov async_memcpy async_pq
async_xor xor async_tx raid6_pq loop tls rpcsec_gss_krb5 auth_rpcgss
nfsv4 dns_resolver nfs lockd grace fscache netfs rfkill sunrpc
dm_multipath ipmi_ssif intel_rapl_msr intel_rapl_common amd64_edac
edac_mce_amd kvm_amd kvm mgag200 i2c_algo_bit dcdbas acpi_ipmi
drm_shmem_helper irqbypass ipmi_si drm_kms_helper dell_smbios rapl
wmi_bmof pcspkr dell_wmi_descriptor ipmi_devintf k10temp ptdma
i2c_piix4 ipmi_msghandler acpi_power_meter drm fuse xfs libcrc32c
sd_mod t10_pi sg ahci libahci crct10dif_pclmul crc32_pclmul
crc32c_intel libata ghash_clmulni_intel tg3 ccp sp5100_tco wmi
dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_debug]
[ 1979.681931] ---[ end trace 0000000000000000 ]---
[ 1979.706620] pstore: backend (erst) writing error (-22)
[ 1979.711761] RIP: 0010:throtl_trim_slice+0xc6/0x320
[ 1979.716555] Code: 00 00 48 89 e8 48 f7 f1 48 29 d5 74 9f 40 0f b6
f6 48 89 df 89 34 24 e8 f8 f5 ff ff 8b 34 24 b9 e8 03 00 00 48 89 df
48 f7 e5 <48> f7 f1 49 03 85 f8 01 00 00 49 89 c7 e8 78 e0 ff ff ba ff
ff ff
[ 1979.735301] RSP: 0018:ffffadf9c2484dc0 EFLAGS: 00010847
[ 1979.740527] RAX: ffffffffffffd8f0 RBX: ffff8f5486a99800 RCX: 00000000000003e8
[ 1979.747659] RDX: 000000000000270f RSI: 0000000000000001 RDI: ffff8f5486a99800
[ 1979.754793] RBP: 0000000000002710 R08: ffffffffffffffff R09: ffff8f565a67b818
[ 1979.761924] R10: ffff8f5486a99810 R11: ffff8f565a67b830 R12: 0000000000000001
[ 1979.769056] R13: ffff8f5486a99808 R14: 0000000000000001 R15: 0000000000000021
[ 1979.776190] FS:  0000000000000000(0000) GS:ffff8f5677d00000(0000)
knlGS:0000000000000000
[ 1979.784278] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1979.790023] CR2: 00007f7db5443250 CR3: 0000000459420000 CR4: 0000000000350ee0
[ 1979.797155] Kernel panic - not syncing: Fatal exception in interrupt
[ 1979.805797] Kernel Offset: 0x30600000 from 0xffffffff81000000
(relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 1979.835382] ---[ end Kernel panic - not syncing: Fatal exception in
interrupt ]---

Thanks,


^ permalink raw reply	[flat|nested] 12+ messages in thread
* [bug report] RIP: 0010:throtl_trim_slice+0xc6/0x320 caused kernel panic
@ 2023-10-31  3:08 Changhui Zhong
  2023-10-31  3:28 ` Yu Kuai
  0 siblings, 1 reply; 12+ messages in thread
From: Changhui Zhong @ 2023-10-31  3:08 UTC (permalink / raw)
  To: Linux Block Devices

Hello,

triggered below issue with branch 'master',please help check,

INFO: HEAD of cloned kernel
commit 888cf78c29e223fd808682f477c18cf8f61ad995
Merge: 09a4a03c073b 6e6c6d6bc6c9
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Fri Oct 27 05:43:05 2023 -1000

    Merge tag 'iommu-fix-v6.6-rc7' of
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

    Pull iommu fix from Joerg Roedel:

     - Fix boot regression for Sapphire Rapids with Intel VT-d driver

    * tag 'iommu-fix-v6.6-rc7' of
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu:
      iommu: Avoid unnecessary cache invalidations

steps:
echo "+cpuset +cpu +io" > /sys/fs/cgroup/cgroup.subtree_control
mkdir /sys/fs/cgroup/test
MAJ=$(ls -l /dev/"$disk" | awk -F ',' '{print $1}' | awk -F ' ' '{print $NF}')
MIN=$(ls -l /dev/"$disk" | awk -F ',' '{print $2}' | awk -F ' ' '{print $1}')
echo "$MAJ:$MIN wbps=1024" > /sys/fs/cgroup/test/io.max
echo $$ > /sys/fs/cgroup/test/cgroup.procs
dd if=/dev/zero of=/dev/$disk bs=10k count=1 oflag=direct &
dd if=/dev/zero of=/dev/$disk bs=10k count=1 oflag=direct &
wait

console log:
[ 6502.907379] divide error: 0000 [#1] PREEMPT SMP NOPTI
[ 6502.912447] CPU: 6 PID: 0 Comm: swapper/6 Not tainted 6.6.0-rc7+ #1
[ 6502.918711] Hardware name: Dell Inc. PowerEdge R650xs/0PPTY2, BIOS
1.4.4 10/07/2021
[ 6502.926364] RIP: 0010:throtl_trim_slice+0xc6/0x320
[ 6502.931156] Code: 00 00 48 89 e8 48 f7 f1 48 29 d5 74 9f 40 0f b6
f6 48 89 df 89 34 24 e8 f8 f5 ff ff 8b 34 24 b9 e8 03 00 00 48 89 df
48 f7 e5 <48> f7 f1 49 03 85 f8 01 00 00 49 89 c7 e8 78 e0 ff ff ba ff
ff ff
[ 6502.949902] RSP: 0018:ffa00000006ccdc0 EFLAGS: 00010887
[ 6502.955128] RAX: ffffffffffffd8f0 RBX: ff110001061aa000 RCX: 00000000000003e8
[ 6502.962260] RDX: 000000000000270f RSI: 0000000000000001 RDI: ff110001061aa000
[ 6502.969394] RBP: 0000000000002710 R08: ffffffffffffffff R09: ff1100010fa74818
[ 6502.976527] R10: ff110001061aa010 R11: ff1100010fa74830 R12: 0000000000000001
[ 6502.983660] R13: ff110001061aa008 R14: 0000000000000001 R15: 0000000000000021
[ 6502.990793] FS:  0000000000000000(0000) GS:ff1100046fd80000(0000)
knlGS:0000000000000000
[ 6502.998879] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6503.004624] CR2: 000055d60b15b000 CR3: 00000001dce20005 CR4: 0000000000771ee0
[ 6503.011757] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 6503.018891] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 6503.026023] PKRU: 55555554
[ 6503.028735] Call Trace:
[ 6503.031187]  <IRQ>
[ 6503.033207]  ? die+0x33/0x90
[ 6503.036093]  ? do_trap+0xe0/0x110
[ 6503.039413]  ? throtl_trim_slice+0xc6/0x320
[ 6503.043599]  ? do_error_trap+0x65/0x80
[ 6503.047351]  ? throtl_trim_slice+0xc6/0x320
[ 6503.051538]  ? exc_divide_error+0x36/0x50
[ 6503.055549]  ? throtl_trim_slice+0xc6/0x320
[ 6503.059735]  ? asm_exc_divide_error+0x16/0x20
[ 6503.064098]  ? throtl_trim_slice+0xc6/0x320
[ 6503.068281]  tg_dispatch_one_bio+0xf0/0x1e0
[ 6503.072469]  throtl_pending_timer_fn+0x1e5/0x510
[ 6503.077086]  ? __pfx_throtl_pending_timer_fn+0x10/0x10
[ 6503.082226]  ? __pfx_throtl_pending_timer_fn+0x10/0x10
[ 6503.087365]  call_timer_fn+0x24/0x130
[ 6503.091034]  __run_timers.part.0+0x1ee/0x280
[ 6503.095304]  ? __hrtimer_run_queues+0x121/0x2b0
[ 6503.099839]  ? sched_clock+0xc/0x30
[ 6503.103329]  run_timer_softirq+0x26/0x50
[ 6503.107256]  __do_softirq+0xc8/0x2ab
[ 6503.110834]  __irq_exit_rcu+0xa1/0xc0
[ 6503.114501]  sysvec_apic_timer_interrupt+0x72/0x90
[ 6503.119295]  </IRQ>
[ 6503.121400]  <TASK>
[ 6503.123507]  asm_sysvec_apic_timer_interrupt+0x16/0x20
[ 6503.128646] RIP: 0010:cpuidle_enter_state+0xc2/0x420
[ 6503.133611] Code: 00 e8 22 cf 4e ff e8 4d f4 ff ff 8b 53 04 49 89
c5 0f 1f 44 00 00 31 ff e8 0b a8 4d ff 45 84 ff 0f 85 3a 02 00 00 fb
45 85 f6 <0f> 88 6e 01 00 00 49 63 d6 4c 2b 2c 24 48 8d 04 52 48 8d 04
82 49
[ 6503.152357] RSP: 0018:ffa00000001ebe80 EFLAGS: 00000206
[ 6503.157582] RAX: ff1100046fdb2100 RBX: ff1100046fdbc7e0 RCX: 000000000000001f
[ 6503.164707] RDX: 0000000000000006 RSI: 000000003d1877c2 RDI: 0000000000000000
[ 6503.171842] RBP: 0000000000000003 R08: 000005ea137b9428 R09: 0000000000000000
[ 6503.178972] R10: 00000000000003dc R11: ff1100046fdb0be4 R12: ffffffffbe2b1900
[ 6503.186104] R13: 000005ea137b9428 R14: 0000000000000003 R15: 0000000000000000
[ 6503.193232]  cpuidle_enter+0x29/0x40
[ 6503.196811]  cpuidle_idle_call+0xfa/0x160
[ 6503.200822]  do_idle+0x7b/0xe0
[ 6503.203882]  cpu_startup_entry+0x26/0x30
[ 6503.207808]  start_secondary+0x115/0x140
[ 6503.211735]  secondary_startup_64_no_verify+0x17d/0x18b
[ 6503.216961]  </TASK>
[ 6503.219154] Modules linked in: binfmt_misc dm_crypt raid10 raid1
raid0 dm_raid raid456 async_raid6_recov async_memcpy async_pq
async_xor xor async_tx raid6_pq loop nf_tables nfnetlink tls
rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache
netfs rfkill sunrpc ipmi_ssif vfat fat intel_rapl_msr
intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common
i10nm_edac nfit libnvdimm x86_pkg_temp_thermal intel_powerclamp
coretemp kvm_intel kvm mgag200 irqbypass rapl iTCO_wdt i2c_algo_bit
acpi_ipmi drm_shmem_helper iTCO_vendor_support dax_hmem intel_cstate
ipmi_si cxl_acpi drm_kms_helper mei_me dell_smbios i2c_i801
ipmi_devintf isst_if_mbox_pci isst_if_mmio dcdbas cxl_core
intel_uncore dell_wmi_descriptor wmi_bmof pcspkr mei isst_if_common
intel_pch_thermal intel_vsec i2c_smbus ipmi_msghandler
acpi_power_meter drm fuse xfs libcrc32c sd_mod t10_pi sg ahci libahci
crct10dif_pclmul crc32_pclmul crc32c_intel libata tg3
ghash_clmulni_intel wmi dm_mirror dm_region_hash dm_log dm_mod
[ 6503.219202]  [last unloaded: scsi_debug]
[ 6503.311377] ---[ end trace 0000000000000000 ]---
[ 6503.346804] RIP: 0010:throtl_trim_slice+0xc6/0x320
[ 6503.351597] Code: 00 00 48 89 e8 48 f7 f1 48 29 d5 74 9f 40 0f b6
f6 48 89 df 89 34 24 e8 f8 f5 ff ff 8b 34 24 b9 e8 03 00 00 48 89 df
48 f7 e5 <48> f7 f1 49 03 85 f8 01 00 00 49 89 c7 e8 78 e0 ff ff ba ff
ff ff
[ 6503.370344] RSP: 0018:ffa00000006ccdc0 EFLAGS: 00010887
[ 6503.375569] RAX: ffffffffffffd8f0 RBX: ff110001061aa000 RCX: 00000000000003e8
[ 6503.382703] RDX: 000000000000270f RSI: 0000000000000001 RDI: ff110001061aa000
[ 6503.389836] RBP: 0000000000002710 R08: ffffffffffffffff R09: ff1100010fa74818
[ 6503.396969] R10: ff110001061aa010 R11: ff1100010fa74830 R12: 0000000000000001
[ 6503.404100] R13: ff110001061aa008 R14: 0000000000000001 R15: 0000000000000021
[ 6503.411235] FS:  0000000000000000(0000) GS:ff1100046fd80000(0000)
knlGS:0000000000000000
[ 6503.419320] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6503.425067] CR2: 000055d60b15b000 CR3: 00000001dce20005 CR4: 0000000000771ee0
[ 6503.432199] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 6503.439332] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 6503.446463] PKRU: 55555554
[ 6503.449176] Kernel panic - not syncing: Fatal exception in interrupt
[ 6503.455610] Kernel Offset: disabled
[ 6503.488281] ---[ end Kernel panic - not syncing: Fatal exception in
interrupt ]---

Thanks,


^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2023-11-01  3:29 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-09-22  2:10 [bug report] RIP: 0010:throtl_trim_slice+0xc6/0x320 caused kernel panic Changhui Zhong
2023-09-22  7:50 ` Ming Lei
2023-09-22  8:53 ` Bagas Sanjaya
2023-09-22  9:34   ` Changhui Zhong
2023-09-25  8:16   ` Changhui Zhong
2023-09-22 10:17 ` Li Nan
2023-09-26  6:41   ` Yu Kuai
  -- strict thread matches above, loose matches on Subject: below --
2023-10-31  3:08 Changhui Zhong
2023-10-31  3:28 ` Yu Kuai
2023-10-31 10:28   ` Changhui Zhong
2023-10-31 12:30     ` Yu Kuai
2023-11-01  3:28       ` Changhui Zhong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox