[bug] unable to handle kernel paging request when running btrfs/011

[bug] unable to handle kernel paging request when running btrfs/011

[Anand Jain]

  Saw below warn leading to bug when running btrfs/011, not
  reproducible. Any idea ?

------
static void  __queue_work(..)
{
::
       /* if draining, only works from the same workqueue are allowed */
         if (unlikely(wq->flags & __WQ_DRAINING) &&
             WARN_ON_ONCE(!is_chained_work(wq)))       <---
                 return;

and then btrfs reported..
   BUG: unable to handle kernel paging request

(gdb) list *btrfs_destroy_workqueue+0x1f
0x6a43f is in btrfs_destroy_workqueue (fs/btrfs/async-thread.c:350).
345	}
346	
347	static inline void
348	__btrfs_destroy_workqueue(struct __btrfs_workqueue *wq)
349	{
350		destroy_workqueue(wq->normal_wq); <---
351		trace_btrfs_workqueue_destroy(wq);
352		kfree(wq);
353	}
354	
------


Thanks, Anand


[   61.038867] ------------[ cut here ]------------
[   61.038876] WARNING: CPU: 0 PID: 117 at kernel/workqueue.c:1381 
__queue_work+0x37f/0x470()
[   61.038878] Modules linked in: btrfs xor raid6_pq cfg80211 rfkill 
ppdev pcspkr psmouse serio_raw evdev joydev mousedev mac_hid 
acpi_cpufreq processor e1000 battery intel_agp intel_gtt video 
parport_pc parport ac button sch_fq_codel nfs lockd grace sunrpc fscache 
sr_mod cdrom sd_mod ata_generic pata_acpi hid_generic usbhid hid atkbd 
libps2 ata_piix ahci libahci ohci_pci ohci_hcd usbcore usb_common libata 
scsi_mod i8042 serio [last unloaded: xor]
[   61.038908] CPU: 0 PID: 117 Comm: kworker/u2:5 Not tainted 
4.5.0-rc3asj+ #10
[   61.038909] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS 
VirtualBox 12/01/2006
[   61.038931] Workqueue: btrfs-readahead btrfs_readahead_helper [btrfs]
[   61.038933]  0000000000000086 00000000bc179d46 ffff88011f403c40 
ffffffff813a2651
[   61.038935]  0000000000000000 ffffffff817f7268 ffff88011f403c78 
ffffffff81079892
[   61.038937]  0000000000000246 0000000000000080 ffff8800da4fcb20 
ffff88011a59db90
[   61.038939] Call Trace:
[   61.038941]  <IRQ>  [<ffffffff813a2651>] dump_stack+0x63/0x82
[   61.038946]  [<ffffffff81079892>] warn_slowpath_common+0x82/0xc0
[   61.038948]  [<ffffffff810799da>] warn_slowpath_null+0x1a/0x20
[   61.038949]  [<ffffffff810910ef>] __queue_work+0x37f/0x470
[   61.038951]  [<ffffffff81091207>] queue_work_on+0x27/0x40
[   61.038964]  [<ffffffffa00933a0>] btrfs_queue_work+0x90/0xe0 [btrfs]
[   61.038974]  [<ffffffffa0056785>] end_workqueue_bio+0x85/0xe0 [btrfs]
[   61.038978]  [<ffffffff8136c4af>] bio_endio+0x3f/0x60
[   61.038990]  [<ffffffffa008d075>] btrfs_end_bio+0x95/0x140 [btrfs]
[   61.038992]  [<ffffffff8136c4af>] bio_endio+0x3f/0x60
[   61.038995]  [<ffffffff8137390b>] blk_update_request+0x8b/0x310
[   61.039002]  [<ffffffffa016ab43>] scsi_end_request+0x33/0x1d0 [scsi_mod]
[   61.039008]  [<ffffffffa016d26a>] scsi_io_completion+0xba/0x5f0 
[scsi_mod]
[   61.039012]  [<ffffffffa0164fae>] scsi_finish_command+0xbe/0xf0 
[scsi_mod]
[   61.039017]  [<ffffffffa016cbc8>] scsi_softirq_done+0xf8/0x120 [scsi_mod]
[   61.039019]  [<ffffffff8137b36b>] blk_done_softirq+0x8b/0xb0
[   61.039021]  [<ffffffff8107dadf>] __do_softirq+0xcf/0x290
[   61.039023]  [<ffffffff8107de13>] irq_exit+0xa3/0xb0
[   61.039026]  [<ffffffff815f2bc4>] do_IRQ+0x54/0xd0
[   61.039028]  [<ffffffff815f0d42>] common_interrupt+0x82/0x82
[   61.039028]  <EOI>  [<ffffffffa00c09d8>] ? 
reada_extent_put+0x128/0x1a0 [btrfs]
[   61.039043]  [<ffffffff81387101>] ? set_task_ioprio+0x21/0xb0
[   61.039054]  [<ffffffffa00c1b18>] 
reada_start_machine_worker+0x228/0x360 [btrfs]
[   61.039065]  [<ffffffffa0092df7>] btrfs_scrubparity_helper+0x77/0x2f0 
[btrfs]
[   61.039076]  [<ffffffffa00930ce>] btrfs_readahead_helper+0xe/0x10 [btrfs]
[   61.039078]  [<ffffffff8109223e>] process_one_work+0x14e/0x440
[   61.039080]  [<ffffffff81092578>] worker_thread+0x48/0x4e0
[   61.039081]  [<ffffffff81092530>] ? process_one_work+0x440/0x440
[   61.039083]  [<ffffffff810980c8>] kthread+0xd8/0xf0
[   61.039085]  [<ffffffff81097ff0>] ? kthread_worker_fn+0x180/0x180
[   61.039087]  [<ffffffff815f060f>] ret_from_fork+0x3f/0x70
[   61.039089]  [<ffffffff81097ff0>] ? kthread_worker_fn+0x180/0x180
[   61.039090] ---[ end trace 1531237816dde88b ]---
[   61.039119] BUG: unable to handle kernel paging request at 
0000000100000000
[   61.039121] IP: [<ffffffffa009340f>] 
btrfs_destroy_workqueue+0x1f/0xf0 [btrfs]
[   61.039140] PGD 116827067 PUD 0
[   61.039143] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
[   61.039145] Modules linked in: btrfs xor raid6_pq cfg80211 rfkill 
ppdev pcspkr psmouse serio_raw evdev joydev mousedev mac_hid 
acpi_cpufreq processor e1000 battery intel_agp intel_gtt video 
parport_pc parport ac button sch_fq_codel nfs lockd grace sunrpc fscache 
sr_mod cdrom sd_mod ata_generic pata_acpi hid_generic usbhid hid atkbd 
libps2 ata_piix ahci libahci ohci_pci ohci_hcd usbcore usb_common libata 
scsi_mod i8042 serio [last unloaded: xor]
[   61.039163] CPU: 0 PID: 11203 Comm: umount Tainted: G        W 
4.5.0-rc3asj+ #10
[   61.039164] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS 
VirtualBox 12/01/2006
[   61.039166] task: ffff8800d8d82ac0 ti: ffff8800be6a0000 task.ti: 
ffff8800be6a0000
[   61.039167] RIP: 0010:[<ffffffffa009340f>]  [<ffffffffa009340f>] 
btrfs_destroy_workqueue+0x1f/0xf0 [btrfs]
[   61.039178] RSP: 0018:ffff8800be6a3d70  EFLAGS: 00010206
[   61.039179] RAX: ffff8800da4fcf01 RBX: ffff8800d9372000 RCX: 
000000000011b080
[   61.039180] RDX: 000000000011b000 RSI: ffff88011f418a40 RDI: 
ffff8800da4fcf90
[   61.039181] RBP: ffff8800be6a3d88 R08: 0000000000018a40 R09: 
ffffffffa0093447
[   61.039182] R10: ffffea0003693f00 R11: ffff8800da31d700 R12: 
ffff8800da4fcf90
[   61.039183] R13: 0000000100000000 R14: 0000000000000000 R15: 
ffff8800d8d82ac0
[   61.039185] FS:  00007f28c0315780(0000) GS:ffff88011f400000(0000) 
knlGS:0000000000000000
[   61.039186] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   61.039187] CR2: 0000000100000000 CR3: 0000000119b0f000 CR4: 
00000000000406f0
[   61.039190] Stack:
[   61.039191]  ffff8800d9372000 ffff8800d9372b58 ffff8800daf04800 
ffff8800be6a3da0
[   61.039193]  ffffffffa0057d6d ffff8800d9372000 ffff8800be6a3df8 
ffffffffa005b8f2
[   61.039194]  ffff8800daf06800 ffff8800daf06e00 ffff8800be6a4000 
ffff8800be6a3dc8
[   61.039196] Call Trace:
[   61.039207]  [<ffffffffa0057d6d>] btrfs_stop_all_workers+0xcd/0xf0 
[btrfs]
[   61.039218]  [<ffffffffa005b8f2>] close_ctree+0x182/0x360 [btrfs]
[   61.039225]  [<ffffffffa002ec69>] btrfs_put_super+0x19/0x20 [btrfs]
[   61.039231]  [<ffffffff811f695a>] generic_shutdown_super+0x6a/0xf0
[   61.039233]  [<ffffffff811f6c72>] kill_anon_super+0x12/0x20
[   61.039241]  [<ffffffffa002e908>] btrfs_kill_super+0x18/0x110 [btrfs]
[   61.039243]  [<ffffffff811f6e13>] deactivate_locked_super+0x43/0x70
[   61.039244]  [<ffffffff811f79fc>] deactivate_super+0x5c/0x60
[   61.039247]  [<ffffffff8121328f>] cleanup_mnt+0x3f/0x90
[   61.039248]  [<ffffffff81213322>] __cleanup_mnt+0x12/0x20
[   61.039250]  [<ffffffff810963a3>] task_work_run+0x73/0x90
[   61.039252]  [<ffffffff8100369e>] exit_to_usermode_loop+0xae/0xb0
[   61.039254]  [<ffffffff81003b9e>] syscall_return_slowpath+0x4e/0x60
[   61.039256]  [<ffffffff815f040c>] int_ret_from_sys_call+0x25/0x8f
[   61.039257] Code: ff e8 2a fc f6 e0 eb a1 0f 1f 40 00 0f 1f 44 00 00 
48 85 ff 74 53 55 48 89 e5 41 55 41 54 53 4c 8b 6f 08 49 89 fc 4d 85 ed 
74 16 <49> 8b 7d 00 e8 d8 e4 ff e0 0f 1f 44 00 00 4c 89 ef e8 0b 06 14
[   61.039276] RIP  [<ffffffffa009340f>] 
btrfs_destroy_workqueue+0x1f/0xf0 [btrfs]
[   61.039287]  RSP <ffff8800be6a3d70>
[   61.039288] CR2: 0000000100000000
[   61.039290] ---[ end trace 1531237816dde88c ]---

Re: [bug] unable to handle kernel paging request when running btrfs/011

[Qu Wenruo]

On 02/19/2016 09:41 PM, Anand Jain wrote:
>
>
>   Saw below warn leading to bug when running btrfs/011, not
>   reproducible. Any idea ?
>
Seems like another wq_destroy race.

But it's hard to locate which wq is the cause from backtrace only.

What's the wq btrfs_stop_all_workers+0xcd is going to free?
And what's the wq end_workqueue_bio+0x85 is going to add?

Thanks,
Qu
> ------
> static void  __queue_work(..)
> {
> ::
>        /* if draining, only works from the same workqueue are allowed */
>          if (unlikely(wq->flags & __WQ_DRAINING) &&
>              WARN_ON_ONCE(!is_chained_work(wq)))       <---
>                  return;
>
> and then btrfs reported..
>    BUG: unable to handle kernel paging request
>
> (gdb) list *btrfs_destroy_workqueue+0x1f
> 0x6a43f is in btrfs_destroy_workqueue (fs/btrfs/async-thread.c:350).
> 345    }
> 346
> 347    static inline void
> 348    __btrfs_destroy_workqueue(struct __btrfs_workqueue *wq)
> 349    {
> 350        destroy_workqueue(wq->normal_wq); <---
> 351        trace_btrfs_workqueue_destroy(wq);
> 352        kfree(wq);
> 353    }
> 354
> ------
>
>
> Thanks, Anand
>
>
> [   61.038867] ------------[ cut here ]------------
> [   61.038876] WARNING: CPU: 0 PID: 117 at kernel/workqueue.c:1381
> __queue_work+0x37f/0x470()
> [   61.038878] Modules linked in: btrfs xor raid6_pq cfg80211 rfkill
> ppdev pcspkr psmouse serio_raw evdev joydev mousedev mac_hid
> acpi_cpufreq processor e1000 battery intel_agp intel_gtt video
> parport_pc parport ac button sch_fq_codel nfs lockd grace sunrpc fscache
> sr_mod cdrom sd_mod ata_generic pata_acpi hid_generic usbhid hid atkbd
> libps2 ata_piix ahci libahci ohci_pci ohci_hcd usbcore usb_common libata
> scsi_mod i8042 serio [last unloaded: xor]
> [   61.038908] CPU: 0 PID: 117 Comm: kworker/u2:5 Not tainted
> 4.5.0-rc3asj+ #10
> [   61.038909] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS
> VirtualBox 12/01/2006
> [   61.038931] Workqueue: btrfs-readahead btrfs_readahead_helper [btrfs]
> [   61.038933]  0000000000000086 00000000bc179d46 ffff88011f403c40
> ffffffff813a2651
> [   61.038935]  0000000000000000 ffffffff817f7268 ffff88011f403c78
> ffffffff81079892
> [   61.038937]  0000000000000246 0000000000000080 ffff8800da4fcb20
> ffff88011a59db90
> [   61.038939] Call Trace:
> [   61.038941]  <IRQ>  [<ffffffff813a2651>] dump_stack+0x63/0x82
> [   61.038946]  [<ffffffff81079892>] warn_slowpath_common+0x82/0xc0
> [   61.038948]  [<ffffffff810799da>] warn_slowpath_null+0x1a/0x20
> [   61.038949]  [<ffffffff810910ef>] __queue_work+0x37f/0x470
> [   61.038951]  [<ffffffff81091207>] queue_work_on+0x27/0x40
> [   61.038964]  [<ffffffffa00933a0>] btrfs_queue_work+0x90/0xe0 [btrfs]
> [   61.038974]  [<ffffffffa0056785>] end_workqueue_bio+0x85/0xe0 [btrfs]
> [   61.038978]  [<ffffffff8136c4af>] bio_endio+0x3f/0x60
> [   61.038990]  [<ffffffffa008d075>] btrfs_end_bio+0x95/0x140 [btrfs]
> [   61.038992]  [<ffffffff8136c4af>] bio_endio+0x3f/0x60
> [   61.038995]  [<ffffffff8137390b>] blk_update_request+0x8b/0x310
> [   61.039002]  [<ffffffffa016ab43>] scsi_end_request+0x33/0x1d0 [scsi_mod]
> [   61.039008]  [<ffffffffa016d26a>] scsi_io_completion+0xba/0x5f0
> [scsi_mod]
> [   61.039012]  [<ffffffffa0164fae>] scsi_finish_command+0xbe/0xf0
> [scsi_mod]
> [   61.039017]  [<ffffffffa016cbc8>] scsi_softirq_done+0xf8/0x120
> [scsi_mod]
> [   61.039019]  [<ffffffff8137b36b>] blk_done_softirq+0x8b/0xb0
> [   61.039021]  [<ffffffff8107dadf>] __do_softirq+0xcf/0x290
> [   61.039023]  [<ffffffff8107de13>] irq_exit+0xa3/0xb0
> [   61.039026]  [<ffffffff815f2bc4>] do_IRQ+0x54/0xd0
> [   61.039028]  [<ffffffff815f0d42>] common_interrupt+0x82/0x82
> [   61.039028]  <EOI>  [<ffffffffa00c09d8>] ?
> reada_extent_put+0x128/0x1a0 [btrfs]
> [   61.039043]  [<ffffffff81387101>] ? set_task_ioprio+0x21/0xb0
> [   61.039054]  [<ffffffffa00c1b18>]
> reada_start_machine_worker+0x228/0x360 [btrfs]
> [   61.039065]  [<ffffffffa0092df7>] btrfs_scrubparity_helper+0x77/0x2f0
> [btrfs]
> [   61.039076]  [<ffffffffa00930ce>] btrfs_readahead_helper+0xe/0x10
> [btrfs]
> [   61.039078]  [<ffffffff8109223e>] process_one_work+0x14e/0x440
> [   61.039080]  [<ffffffff81092578>] worker_thread+0x48/0x4e0
> [   61.039081]  [<ffffffff81092530>] ? process_one_work+0x440/0x440
> [   61.039083]  [<ffffffff810980c8>] kthread+0xd8/0xf0
> [   61.039085]  [<ffffffff81097ff0>] ? kthread_worker_fn+0x180/0x180
> [   61.039087]  [<ffffffff815f060f>] ret_from_fork+0x3f/0x70
> [   61.039089]  [<ffffffff81097ff0>] ? kthread_worker_fn+0x180/0x180
> [   61.039090] ---[ end trace 1531237816dde88b ]---
> [   61.039119] BUG: unable to handle kernel paging request at
> 0000000100000000
> [   61.039121] IP: [<ffffffffa009340f>]
> btrfs_destroy_workqueue+0x1f/0xf0 [btrfs]
> [   61.039140] PGD 116827067 PUD 0
> [   61.039143] Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
> [   61.039145] Modules linked in: btrfs xor raid6_pq cfg80211 rfkill
> ppdev pcspkr psmouse serio_raw evdev joydev mousedev mac_hid
> acpi_cpufreq processor e1000 battery intel_agp intel_gtt video
> parport_pc parport ac button sch_fq_codel nfs lockd grace sunrpc fscache
> sr_mod cdrom sd_mod ata_generic pata_acpi hid_generic usbhid hid atkbd
> libps2 ata_piix ahci libahci ohci_pci ohci_hcd usbcore usb_common libata
> scsi_mod i8042 serio [last unloaded: xor]
> [   61.039163] CPU: 0 PID: 11203 Comm: umount Tainted: G        W
> 4.5.0-rc3asj+ #10
> [   61.039164] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS
> VirtualBox 12/01/2006
> [   61.039166] task: ffff8800d8d82ac0 ti: ffff8800be6a0000 task.ti:
> ffff8800be6a0000
> [   61.039167] RIP: 0010:[<ffffffffa009340f>]  [<ffffffffa009340f>]
> btrfs_destroy_workqueue+0x1f/0xf0 [btrfs]
> [   61.039178] RSP: 0018:ffff8800be6a3d70  EFLAGS: 00010206
> [   61.039179] RAX: ffff8800da4fcf01 RBX: ffff8800d9372000 RCX:
> 000000000011b080
> [   61.039180] RDX: 000000000011b000 RSI: ffff88011f418a40 RDI:
> ffff8800da4fcf90
> [   61.039181] RBP: ffff8800be6a3d88 R08: 0000000000018a40 R09:
> ffffffffa0093447
> [   61.039182] R10: ffffea0003693f00 R11: ffff8800da31d700 R12:
> ffff8800da4fcf90
> [   61.039183] R13: 0000000100000000 R14: 0000000000000000 R15:
> ffff8800d8d82ac0
> [   61.039185] FS:  00007f28c0315780(0000) GS:ffff88011f400000(0000)
> knlGS:0000000000000000
> [   61.039186] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   61.039187] CR2: 0000000100000000 CR3: 0000000119b0f000 CR4:
> 00000000000406f0
> [   61.039190] Stack:
> [   61.039191]  ffff8800d9372000 ffff8800d9372b58 ffff8800daf04800
> ffff8800be6a3da0
> [   61.039193]  ffffffffa0057d6d ffff8800d9372000 ffff8800be6a3df8
> ffffffffa005b8f2
> [   61.039194]  ffff8800daf06800 ffff8800daf06e00 ffff8800be6a4000
> ffff8800be6a3dc8
> [   61.039196] Call Trace:
> [   61.039207]  [<ffffffffa0057d6d>] btrfs_stop_all_workers+0xcd/0xf0
> [btrfs]
> [   61.039218]  [<ffffffffa005b8f2>] close_ctree+0x182/0x360 [btrfs]
> [   61.039225]  [<ffffffffa002ec69>] btrfs_put_super+0x19/0x20 [btrfs]
> [   61.039231]  [<ffffffff811f695a>] generic_shutdown_super+0x6a/0xf0
> [   61.039233]  [<ffffffff811f6c72>] kill_anon_super+0x12/0x20
> [   61.039241]  [<ffffffffa002e908>] btrfs_kill_super+0x18/0x110 [btrfs]
> [   61.039243]  [<ffffffff811f6e13>] deactivate_locked_super+0x43/0x70
> [   61.039244]  [<ffffffff811f79fc>] deactivate_super+0x5c/0x60
> [   61.039247]  [<ffffffff8121328f>] cleanup_mnt+0x3f/0x90
> [   61.039248]  [<ffffffff81213322>] __cleanup_mnt+0x12/0x20
> [   61.039250]  [<ffffffff810963a3>] task_work_run+0x73/0x90
> [   61.039252]  [<ffffffff8100369e>] exit_to_usermode_loop+0xae/0xb0
> [   61.039254]  [<ffffffff81003b9e>] syscall_return_slowpath+0x4e/0x60
> [   61.039256]  [<ffffffff815f040c>] int_ret_from_sys_call+0x25/0x8f
> [   61.039257] Code: ff e8 2a fc f6 e0 eb a1 0f 1f 40 00 0f 1f 44 00 00
> 48 85 ff 74 53 55 48 89 e5 41 55 41 54 53 4c 8b 6f 08 49 89 fc 4d 85 ed
> 74 16 <49> 8b 7d 00 e8 d8 e4 ff e0 0f 1f 44 00 00 4c 89 ef e8 0b 06 14
> [   61.039276] RIP  [<ffffffffa009340f>]
> btrfs_destroy_workqueue+0x1f/0xf0 [btrfs]
> [   61.039287]  RSP <ffff8800be6a3d70>
> [   61.039288] CR2: 0000000100000000
> [   61.039290] ---[ end trace 1531237816dde88c ]---
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [bug] unable to handle kernel paging request when running btrfs/011

[David Sterba]

On Sun, Feb 21, 2016 at 09:36:44AM +0800, Qu Wenruo wrote:
> 
> 
> On 02/19/2016 09:41 PM, Anand Jain wrote:
> >
> >
> >   Saw below warn leading to bug when running btrfs/011, not
> >   reproducible. Any idea ?
> >
> Seems like another wq_destroy race.
> 
> But it's hard to locate which wq is the cause from backtrace only.

Name of the workqueue is in the first stacktrasce, it's readahead. The
crash happens in 'umount'.

Re: [bug] unable to handle kernel paging request when running btrfs/011

[David Sterba]

On Sun, Feb 21, 2016 at 09:36:44AM +0800, Qu Wenruo wrote:
> 
> 
> On 02/19/2016 09:41 PM, Anand Jain wrote:
> >
> >
> >   Saw below warn leading to bug when running btrfs/011, not
> >   reproducible. Any idea ?
> >
> Seems like another wq_destroy race.
> 
> But it's hard to locate which wq is the cause from backtrace only.
> 
> What's the wq btrfs_stop_all_workers+0xcd is going to free?

You can always try to guess on your locally built sources, provided the
configs do not enable "too much" debugging. What I get and looks
reasonable:

(gdb) l *(btrfs_stop_all_workers+0xcd)
0x2bc7d is in btrfs_stop_all_workers (fs/btrfs/disk-io.c:2154).
warning: Source file is more recent than executable.
2149            btrfs_destroy_workqueue(fs_info->delayed_workers);
2150            btrfs_destroy_workqueue(fs_info->caching_workers);
2151            btrfs_destroy_workqueue(fs_info->readahead_workers);
2152            btrfs_destroy_workqueue(fs_info->flush_workers);
2153            btrfs_destroy_workqueue(fs_info->qgroup_rescan_workers);
2154            btrfs_destroy_workqueue(fs_info->extent_workers);
2155    }
2156
2157    static void free_root_extent_buffers(struct btrfs_root *root)
2158    {

> And what's the wq end_workqueue_bio+0x85 is going to add?

(gdb) l *(end_workqueue_bio+0x85)
0x2ab55 is in end_workqueue_bio (fs/btrfs/disk-io.c:726).
721
722             if (bio->bi_rw & REQ_WRITE) {
723                     if (end_io_wq->metadata == BTRFS_WQ_ENDIO_METADATA) {
724                             wq = fs_info->endio_meta_write_workers;
725                             func = btrfs_endio_meta_write_helper;
726                     } else if (end_io_wq->metadata == BTRFS_WQ_ENDIO_FREE_SPACE) {
727                             wq = fs_info->endio_freespace_worker;
728                             func = btrfs_freespace_write_helper;
729                     } else if (end_io_wq->metadata == BTRFS_WQ_ENDIO_RAID56) {
730                             wq = fs_info->endio_raid56_workers;

Re: [bug] unable to handle kernel paging request when running btrfs/011

[Qu Wenruo]

David Sterba wrote on 2016/02/23 15:45 +0100:
> On Sun, Feb 21, 2016 at 09:36:44AM +0800, Qu Wenruo wrote:
>>
>>
>> On 02/19/2016 09:41 PM, Anand Jain wrote:
>>>
>>>
>>>    Saw below warn leading to bug when running btrfs/011, not
>>>    reproducible. Any idea ?
>>>
>> Seems like another wq_destroy race.
>>
>> But it's hard to locate which wq is the cause from backtrace only.
>>
>> What's the wq btrfs_stop_all_workers+0xcd is going to free?
>
> You can always try to guess on your locally built sources, provided the
> configs do not enable "too much" debugging. What I get and looks
> reasonable:

Never though even on different system, the binary could be so identical.

This should provide good enough info to investigate further.

Great thanks for the advice.
Qu

>
> (gdb) l *(btrfs_stop_all_workers+0xcd)
> 0x2bc7d is in btrfs_stop_all_workers (fs/btrfs/disk-io.c:2154).
> warning: Source file is more recent than executable.
> 2149            btrfs_destroy_workqueue(fs_info->delayed_workers);
> 2150            btrfs_destroy_workqueue(fs_info->caching_workers);
> 2151            btrfs_destroy_workqueue(fs_info->readahead_workers);
> 2152            btrfs_destroy_workqueue(fs_info->flush_workers);
> 2153            btrfs_destroy_workqueue(fs_info->qgroup_rescan_workers);
> 2154            btrfs_destroy_workqueue(fs_info->extent_workers);
> 2155    }
> 2156
> 2157    static void free_root_extent_buffers(struct btrfs_root *root)
> 2158    {
>
>> And what's the wq end_workqueue_bio+0x85 is going to add?
>
> (gdb) l *(end_workqueue_bio+0x85)
> 0x2ab55 is in end_workqueue_bio (fs/btrfs/disk-io.c:726).
> 721
> 722             if (bio->bi_rw & REQ_WRITE) {
> 723                     if (end_io_wq->metadata == BTRFS_WQ_ENDIO_METADATA) {
> 724                             wq = fs_info->endio_meta_write_workers;
> 725                             func = btrfs_endio_meta_write_helper;
> 726                     } else if (end_io_wq->metadata == BTRFS_WQ_ENDIO_FREE_SPACE) {
> 727                             wq = fs_info->endio_freespace_worker;
> 728                             func = btrfs_freespace_write_helper;
> 729                     } else if (end_io_wq->metadata == BTRFS_WQ_ENDIO_RAID56) {
> 730                             wq = fs_info->endio_raid56_workers;
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>