From: "Theodore Y. Ts'o" <tytso-3s7WtUTddSA@public.gmane.org>
To: Ross Zwisler
<ross.zwisler-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>,
Matthew Wilcox <mawilcox-0li6OtcxBFHby3iVrkZq2A@public.gmane.org>
Cc: linux-ext4-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org
Subject: Help trying to use /dev/pmem for dax debugging?
Date: Mon, 30 Jul 2018 19:53:12 -0400 [thread overview]
Message-ID: <20180730235312.GA5089@thunk.org> (raw)
In newer kernels, it looks like you can't use /dev/pmem0 for DAX
unless it's marked as being DAX capable. This appears to require
CONFIG_NVDIMM_PFN. But when I tried to build a kernel with that
configured, I get the following BUG:
[ 0.000000] Linux version 4.18.0-rc4-xfstests-00031-g7c2d77aa7d80 (tytso@cwcc) (gcc version 7.3.0 (Debian 7.3.0-27)) #460 SMP Mon Jul 30 19:38:44 EDT 2018
[ 0.000000] Command line: systemd.show_status=auto systemd.log_level=crit root=/dev/vda console=ttyS0,115200 cmd=maint fstesttz=America/New_York fstesttyp=ext4 fstestapi=1.4 memmap=4G!9G memmap=9G!14G
...
[ 16.544707] BUG: unable to handle kernel paging request at ffffed0048000000
[ 16.546132] PGD 6bffe9067 P4D 6bffe9067 PUD 6bfbec067 PMD 0
[ 16.547174] Oops: 0000 [#1] SMP KASAN PTI
[ 16.547923] CPU: 0 PID: 81 Comm: kworker/u8:1 Not tainted 4.18.0-rc4-xfstests-00031-g7c2d77aa7d80 #460
[ 16.549706] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.1-1 04/01/2014
[ 16.551285] Workqueue: events_unbound async_run_entry_fn
[ 16.552309] RIP: 0010:check_memory_region+0xdd/0x190
[ 16.553264] Code: 74 0b 41 80 38 00 74 f0 4d 85 c0 75 56 4c 01 c8 49 89 e8 49 29 c0 4d 8d 48 07 4d 85 c0 4d 0f 49 c8 49 c1 f9 03 45 85 c9 74 5b <48> 83 38 00 75 18 45 8d 41 ff 4e 8d 44 c0 08 48 83 c0 08 49 39 c0
[ 16.556872] RSP: 0000:ffff8806469b6bb8 EFLAGS: 00010202
[ 16.557861] RAX: ffffed0048000000 RBX: ffff880240000fff RCX: ffffffffa8a2f9bc
[ 16.559500] RDX: 0000000000000000 RSI: 0000000000001000 RDI: ffff880240000000
[ 16.561255] RBP: ffffed0048000200 R08: 0000000000000200 R09: 0000000000000040
[ 16.563245] R10: 0000000000000200 R11: ffffed00480001ff R12: ffff880240000000
[ 16.565186] R13: dffffc0000000000 R14: fffffbfff5361562 R15: ffffea0015d34bd8
[ 16.567119] FS: 0000000000000000(0000) GS:ffff88064b600000(0000) knlGS:0000000000000000
[ 16.569331] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 16.570927] CR2: ffffed0048000000 CR3: 0000000212416001 CR4: 0000000000360ef0
[ 16.572839] Call Trace:
[ 16.573493] memcpy+0x1f/0x50
[ 16.574050] pmem_do_bvec+0x1dc/0x670
[ 16.575086] ? pmem_release_pgmap_ops+0x10/0x10
[ 16.576392] ? rcu_read_lock_sched_held+0x110/0x130
[ 16.577785] ? generic_make_request_checks+0xf87/0x1520
[ 16.579310] ? do_read_cache_page+0x219/0x8b0
[ 16.580551] pmem_make_request+0x306/0x9e0
[ 16.581714] generic_make_request+0x565/0xd30
[ 16.582947] ? mempool_alloc+0xf7/0x2d0
[ 16.584032] ? blk_plug_queued_count+0x150/0x150
[ 16.585339] ? sched_clock_cpu+0x18/0x180
[ 16.586473] ? debug_show_all_locks+0x2d0/0x2d0
[ 16.587803] ? submit_bio+0x139/0x3a0
[ 16.588864] submit_bio+0x139/0x3a0
[ 16.589896] ? lock_downgrade+0x5e0/0x5e0
[ 16.591031] ? lock_acquire+0x106/0x3e0
[ 16.592123] ? direct_make_request+0x1e0/0x1e0
[ 16.593428] ? guard_bio_eod+0x19d/0x570
[ 16.594547] submit_bh_wbc.isra.12+0x409/0x5a0
[ 16.595804] block_read_full_page+0x526/0x800
[ 16.597032] ? block_llseek+0xd0/0xd0
[ 16.598072] ? block_page_mkwrite+0x270/0x270
[ 16.599317] ? add_to_page_cache_lru+0x119/0x210
[ 16.600621] ? add_to_page_cache_locked+0x40/0x40
[ 16.601943] ? pagecache_get_page+0x44/0x6b0
[ 16.603153] do_read_cache_page+0x219/0x8b0
[ 16.604338] ? blkdev_writepages+0x10/0x10
[ 16.605500] read_dev_sector+0xbb/0x390
[ 16.606606] read_lba.isra.0+0x2f0/0x5c0
[ 16.607735] ? compare_gpts+0x1500/0x1500
[ 16.608870] ? efi_partition+0x2bc/0x1bb0
[ 16.610021] ? rcu_read_lock_sched_held+0x110/0x130
[ 16.611387] efi_partition+0x2e6/0x1bb0
[ 16.612468] ? __isolate_free_page+0x530/0x530
[ 16.613717] ? rcu_read_lock_sched_held+0x110/0x130
[ 16.615103] ? is_gpt_valid.part.1+0xdc0/0xdc0
[ 16.616396] ? string+0x14c/0x220
[ 16.617344] ? string+0x14c/0x220
[ 16.618285] ? format_decode+0x3be/0x760
[ 16.619409] ? vsnprintf+0x1ff/0x10a0
[ 16.620439] ? num_to_str+0x220/0x220
[ 16.621472] ? snprintf+0x8f/0xc0
[ 16.622411] ? vscnprintf+0x30/0x30
[ 16.623402] ? is_gpt_valid.part.1+0xdc0/0xdc0
[ 16.624650] ? check_partition+0x308/0x660
[ 16.625818] check_partition+0x308/0x660
[ 16.626966] rescan_partitions+0x187/0x8d0
[ 16.628123] ? lock_acquire+0x106/0x3e0
[ 16.629219] ? up_write+0x1d/0x150
[ 16.630185] ? bd_set_size+0x24e/0x2e0
[ 16.631244] __blkdev_get+0x696/0xfd0
[ 16.632276] ? bd_set_size+0x2e0/0x2e0
[ 16.633337] ? kvm_sched_clock_read+0x21/0x30
[ 16.634570] ? sched_clock+0x5/0x10
[ 16.635563] ? sched_clock_cpu+0x18/0x180
[ 16.636706] blkdev_get+0x28f/0x850
[ 16.637714] ? lockdep_rcu_suspicious+0x150/0x150
[ 16.639032] ? __blkdev_get+0xfd0/0xfd0
[ 16.640144] ? refcount_sub_and_test+0xcd/0x160
[ 16.641415] ? refcount_inc+0x30/0x30
[ 16.642453] ? do_raw_spin_unlock+0x144/0x220
[ 16.643680] ? kobject_put+0x50/0x410
[ 16.644711] __device_add_disk+0xbe5/0xe40
[ 16.645916] ? bdget_disk+0x60/0x60
[ 16.646919] ? alloc_dax+0x2b2/0x5b0
[ 16.647939] ? kill_dax+0x140/0x140
[ 16.648928] ? nvdimm_badblocks_populate+0x47/0x360
[ 16.649904] ? __raw_spin_lock_init+0x2d/0x100
[ 16.650720] pmem_attach_disk+0x944/0xf90
[ 16.651477] ? nd_pmem_notify+0x4a0/0x4a0
[ 16.652233] ? kfree+0xd4/0x210
[ 16.652822] ? nd_dax_probe+0x1d0/0x240
[ 16.653526] nvdimm_bus_probe+0xd4/0x370
[ 16.654261] driver_probe_device+0x56d/0xbe0
[ 16.655432] ? __driver_attach+0x2c0/0x2c0
[ 16.656548] bus_for_each_drv+0x10d/0x1a0
[ 16.657414] ? subsys_find_device_by_id+0x2e0/0x2e0
[ 16.658385] __device_attach+0x19c/0x230
[ 16.659225] ? device_bind_driver+0xa0/0xa0
[ 16.660135] ? kobject_uevent_env+0x223/0xfb0
[ 16.661072] bus_probe_device+0x1ad/0x260
[ 16.661852] ? sysfs_create_groups+0x86/0x130
[ 16.662826] device_add+0x9fe/0x1340
[ 16.663814] ? device_private_init+0x180/0x180
[ 16.664786] nd_async_device_register+0xe/0x40
[ 16.665621] async_run_entry_fn+0xc3/0x630
[ 16.666400] process_one_work+0x767/0x1670
[ 16.667221] ? debug_show_all_locks+0x2d0/0x2d0
[ 16.668137] ? pwq_dec_nr_in_flight+0x2c0/0x2c0
[ 16.669011] worker_thread+0x87/0xb90
[ 16.669730] ? __kthread_parkme+0xb6/0x180
[ 16.670515] ? process_one_work+0x1670/0x1670
[ 16.671339] kthread+0x314/0x3d0
[ 16.671963] ? kthread_flush_work_fn+0x10/0x10
[ 16.672810] ret_from_fork+0x3a/0x50
[ 16.673502] CR2: ffffed0048000000
[ 16.674436] ---[ end trace ac6b16a57e0c48ad ]---
Does this ring any bells? Any suggestions about how I get ext4 dax
testing working again? Many thanks!!
(full log and config attached below, compressed for size reasons)
- Ted
WARNING: multiple messages have this Message-ID (diff)
From: "Theodore Y. Ts'o" <tytso@mit.edu>
To: Ross Zwisler <ross.zwisler@linux.intel.com>,
Matthew Wilcox <mawilcox@microsoft.com>
Cc: linux-ext4@vger.kernel.org, linux-nvdimm@lists.01.org
Subject: Help trying to use /dev/pmem for dax debugging?
Date: Mon, 30 Jul 2018 19:53:12 -0400 [thread overview]
Message-ID: <20180730235312.GA5089@thunk.org> (raw)
In newer kernels, it looks like you can't use /dev/pmem0 for DAX
unless it's marked as being DAX capable. This appears to require
CONFIG_NVDIMM_PFN. But when I tried to build a kernel with that
configured, I get the following BUG:
[ 0.000000] Linux version 4.18.0-rc4-xfstests-00031-g7c2d77aa7d80 (tytso@cwcc) (gcc version 7.3.0 (Debian 7.3.0-27)) #460 SMP Mon Jul 30 19:38:44 EDT 2018
[ 0.000000] Command line: systemd.show_status=auto systemd.log_level=crit root=/dev/vda console=ttyS0,115200 cmd=maint fstesttz=America/New_York fstesttyp=ext4 fstestapi=1.4 memmap=4G!9G memmap=9G!14G
...
[ 16.544707] BUG: unable to handle kernel paging request at ffffed0048000000
[ 16.546132] PGD 6bffe9067 P4D 6bffe9067 PUD 6bfbec067 PMD 0
[ 16.547174] Oops: 0000 [#1] SMP KASAN PTI
[ 16.547923] CPU: 0 PID: 81 Comm: kworker/u8:1 Not tainted 4.18.0-rc4-xfstests-00031-g7c2d77aa7d80 #460
[ 16.549706] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.1-1 04/01/2014
[ 16.551285] Workqueue: events_unbound async_run_entry_fn
[ 16.552309] RIP: 0010:check_memory_region+0xdd/0x190
[ 16.553264] Code: 74 0b 41 80 38 00 74 f0 4d 85 c0 75 56 4c 01 c8 49 89 e8 49 29 c0 4d 8d 48 07 4d 85 c0 4d 0f 49 c8 49 c1 f9 03 45 85 c9 74 5b <48> 83 38 00 75 18 45 8d 41 ff 4e 8d 44 c0 08 48 83 c0 08 49 39 c0
[ 16.556872] RSP: 0000:ffff8806469b6bb8 EFLAGS: 00010202
[ 16.557861] RAX: ffffed0048000000 RBX: ffff880240000fff RCX: ffffffffa8a2f9bc
[ 16.559500] RDX: 0000000000000000 RSI: 0000000000001000 RDI: ffff880240000000
[ 16.561255] RBP: ffffed0048000200 R08: 0000000000000200 R09: 0000000000000040
[ 16.563245] R10: 0000000000000200 R11: ffffed00480001ff R12: ffff880240000000
[ 16.565186] R13: dffffc0000000000 R14: fffffbfff5361562 R15: ffffea0015d34bd8
[ 16.567119] FS: 0000000000000000(0000) GS:ffff88064b600000(0000) knlGS:0000000000000000
[ 16.569331] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 16.570927] CR2: ffffed0048000000 CR3: 0000000212416001 CR4: 0000000000360ef0
[ 16.572839] Call Trace:
[ 16.573493] memcpy+0x1f/0x50
[ 16.574050] pmem_do_bvec+0x1dc/0x670
[ 16.575086] ? pmem_release_pgmap_ops+0x10/0x10
[ 16.576392] ? rcu_read_lock_sched_held+0x110/0x130
[ 16.577785] ? generic_make_request_checks+0xf87/0x1520
[ 16.579310] ? do_read_cache_page+0x219/0x8b0
[ 16.580551] pmem_make_request+0x306/0x9e0
[ 16.581714] generic_make_request+0x565/0xd30
[ 16.582947] ? mempool_alloc+0xf7/0x2d0
[ 16.584032] ? blk_plug_queued_count+0x150/0x150
[ 16.585339] ? sched_clock_cpu+0x18/0x180
[ 16.586473] ? debug_show_all_locks+0x2d0/0x2d0
[ 16.587803] ? submit_bio+0x139/0x3a0
[ 16.588864] submit_bio+0x139/0x3a0
[ 16.589896] ? lock_downgrade+0x5e0/0x5e0
[ 16.591031] ? lock_acquire+0x106/0x3e0
[ 16.592123] ? direct_make_request+0x1e0/0x1e0
[ 16.593428] ? guard_bio_eod+0x19d/0x570
[ 16.594547] submit_bh_wbc.isra.12+0x409/0x5a0
[ 16.595804] block_read_full_page+0x526/0x800
[ 16.597032] ? block_llseek+0xd0/0xd0
[ 16.598072] ? block_page_mkwrite+0x270/0x270
[ 16.599317] ? add_to_page_cache_lru+0x119/0x210
[ 16.600621] ? add_to_page_cache_locked+0x40/0x40
[ 16.601943] ? pagecache_get_page+0x44/0x6b0
[ 16.603153] do_read_cache_page+0x219/0x8b0
[ 16.604338] ? blkdev_writepages+0x10/0x10
[ 16.605500] read_dev_sector+0xbb/0x390
[ 16.606606] read_lba.isra.0+0x2f0/0x5c0
[ 16.607735] ? compare_gpts+0x1500/0x1500
[ 16.608870] ? efi_partition+0x2bc/0x1bb0
[ 16.610021] ? rcu_read_lock_sched_held+0x110/0x130
[ 16.611387] efi_partition+0x2e6/0x1bb0
[ 16.612468] ? __isolate_free_page+0x530/0x530
[ 16.613717] ? rcu_read_lock_sched_held+0x110/0x130
[ 16.615103] ? is_gpt_valid.part.1+0xdc0/0xdc0
[ 16.616396] ? string+0x14c/0x220
[ 16.617344] ? string+0x14c/0x220
[ 16.618285] ? format_decode+0x3be/0x760
[ 16.619409] ? vsnprintf+0x1ff/0x10a0
[ 16.620439] ? num_to_str+0x220/0x220
[ 16.621472] ? snprintf+0x8f/0xc0
[ 16.622411] ? vscnprintf+0x30/0x30
[ 16.623402] ? is_gpt_valid.part.1+0xdc0/0xdc0
[ 16.624650] ? check_partition+0x308/0x660
[ 16.625818] check_partition+0x308/0x660
[ 16.626966] rescan_partitions+0x187/0x8d0
[ 16.628123] ? lock_acquire+0x106/0x3e0
[ 16.629219] ? up_write+0x1d/0x150
[ 16.630185] ? bd_set_size+0x24e/0x2e0
[ 16.631244] __blkdev_get+0x696/0xfd0
[ 16.632276] ? bd_set_size+0x2e0/0x2e0
[ 16.633337] ? kvm_sched_clock_read+0x21/0x30
[ 16.634570] ? sched_clock+0x5/0x10
[ 16.635563] ? sched_clock_cpu+0x18/0x180
[ 16.636706] blkdev_get+0x28f/0x850
[ 16.637714] ? lockdep_rcu_suspicious+0x150/0x150
[ 16.639032] ? __blkdev_get+0xfd0/0xfd0
[ 16.640144] ? refcount_sub_and_test+0xcd/0x160
[ 16.641415] ? refcount_inc+0x30/0x30
[ 16.642453] ? do_raw_spin_unlock+0x144/0x220
[ 16.643680] ? kobject_put+0x50/0x410
[ 16.644711] __device_add_disk+0xbe5/0xe40
[ 16.645916] ? bdget_disk+0x60/0x60
[ 16.646919] ? alloc_dax+0x2b2/0x5b0
[ 16.647939] ? kill_dax+0x140/0x140
[ 16.648928] ? nvdimm_badblocks_populate+0x47/0x360
[ 16.649904] ? __raw_spin_lock_init+0x2d/0x100
[ 16.650720] pmem_attach_disk+0x944/0xf90
[ 16.651477] ? nd_pmem_notify+0x4a0/0x4a0
[ 16.652233] ? kfree+0xd4/0x210
[ 16.652822] ? nd_dax_probe+0x1d0/0x240
[ 16.653526] nvdimm_bus_probe+0xd4/0x370
[ 16.654261] driver_probe_device+0x56d/0xbe0
[ 16.655432] ? __driver_attach+0x2c0/0x2c0
[ 16.656548] bus_for_each_drv+0x10d/0x1a0
[ 16.657414] ? subsys_find_device_by_id+0x2e0/0x2e0
[ 16.658385] __device_attach+0x19c/0x230
[ 16.659225] ? device_bind_driver+0xa0/0xa0
[ 16.660135] ? kobject_uevent_env+0x223/0xfb0
[ 16.661072] bus_probe_device+0x1ad/0x260
[ 16.661852] ? sysfs_create_groups+0x86/0x130
[ 16.662826] device_add+0x9fe/0x1340
[ 16.663814] ? device_private_init+0x180/0x180
[ 16.664786] nd_async_device_register+0xe/0x40
[ 16.665621] async_run_entry_fn+0xc3/0x630
[ 16.666400] process_one_work+0x767/0x1670
[ 16.667221] ? debug_show_all_locks+0x2d0/0x2d0
[ 16.668137] ? pwq_dec_nr_in_flight+0x2c0/0x2c0
[ 16.669011] worker_thread+0x87/0xb90
[ 16.669730] ? __kthread_parkme+0xb6/0x180
[ 16.670515] ? process_one_work+0x1670/0x1670
[ 16.671339] kthread+0x314/0x3d0
[ 16.671963] ? kthread_flush_work_fn+0x10/0x10
[ 16.672810] ret_from_fork+0x3a/0x50
[ 16.673502] CR2: ffffed0048000000
[ 16.674436] ---[ end trace ac6b16a57e0c48ad ]---
Does this ring any bells? Any suggestions about how I get ext4 dax
testing working again? Many thanks!!
(full log and config attached below, compressed for size reasons)
- Ted
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
next reply other threads:[~2018-07-30 23:53 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-30 23:53 Theodore Y. Ts'o [this message]
2018-07-30 23:53 ` Help trying to use /dev/pmem for dax debugging? Theodore Y. Ts'o
[not found] ` <20180730235312.GA5089-AKGzg7BKzIDYtjvyW6yDsg@public.gmane.org>
2018-07-31 19:36 ` Ross Zwisler
2018-07-31 19:36 ` Ross Zwisler
[not found] ` <20180731193642.GA3473-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2018-07-31 20:27 ` Dave Jiang
2018-07-31 20:27 ` Dave Jiang
[not found] ` <30247efc-88f8-3e0b-fcd8-83801be5f041-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2018-08-10 2:53 ` Theodore Y. Ts'o
2018-08-10 2:53 ` Theodore Y. Ts'o
[not found] ` <20180810025339.GB21087-AKGzg7BKzIDYtjvyW6yDsg@public.gmane.org>
2018-08-10 16:18 ` Dave Jiang
2018-08-10 16:18 ` Dave Jiang
[not found] ` <4222843a-13ea-e88f-ca74-4e3fe53b4105-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2018-08-10 17:28 ` Dan Williams
2018-08-10 17:28 ` Dan Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180730235312.GA5089@thunk.org \
--to=tytso-3s7wtutddsa@public.gmane.org \
--cc=linux-ext4-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org \
--cc=mawilcox-0li6OtcxBFHby3iVrkZq2A@public.gmane.org \
--cc=ross.zwisler-VuQAYsv1563Yd54FQh9/CA@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.