* Fwd: Fw: Panic when make check for ndctl [not found] <20170529155643.5848.E1E9C6FF@jp.fujitsu.com> @ 2017-05-29 14:21 ` Yasunori Goto 2017-05-29 23:04 ` Dan Williams 0 siblings, 1 reply; 14+ messages in thread From: Yasunori Goto @ 2017-05-29 14:21 UTC (permalink / raw) To: linux-nvdimm, y-goto Hello, I tried "make check" of ndctl command on the kernel of libnvdimm-for-next branch(*), but I feel its kernel is unstable when nfit_test.ko module is loaded. (*) git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm libnvdimm-for-next The newest commit is f5705aa8cfed Here is console logs when the kernel was paniced, and I'll attach the kernel config file. Does anyone know how to solve this? Am I something wrong? ------- CentOS Linux 7 (Core) Kernel 4.12.0-rc1+ on an x86_64 goto-guest login: [ 75.737769] random: crng init done [ 189.205125] nfit_test_iomap: loading out-of-tree module taints kernel. [ 189.586917] nfit_test nfit_test.0: found a zero length table '0' parsing nfit [ 191.644655] nfit_test nfit_test.0: failed to evaluate _FIT [ 191.692708] nfit_test nfit_test.1: nmem4 flags: save_fail restore_fail flush_fail not_armed [ 191.694838] nfit_test nfit_test.1: nmem5 flags: map_fail [ 192.734002] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only [ 192.743577] pmem7: detected capacity change from 0 to 4194304 [ 192.747323] pmem6: detected capacity change from 0 to 33554432 CentOS Linux 7 (Core) Kernel 4.12.0-rc1+ on an x86_64 goto-guest login: [ 342.168046] pmem5: detected capacity change from 0 to 67108864 [ 342.253861] pmem5: detected capacity change from 0 to 63590400 [ 342.332442] pmem5: detected capacity change from 0 to 67108864 [ 342.376596] nd_pmem btt5.0: No existing arenas [ 342.382494] pmem5s: detected capacity change from 0 to 65961984 [ 342.461783] pmem5: detected capacity change from 0 to 67108864 [ 342.542996] ndblk1.0: detected capacity change from 0 to 33554432 [ 342.614674] nd_blk btt1.0: No existing arenas [ 342.629830] ndblk1.0s: detected capacity change from 0 to 32440320 [ 342.850342] pmem5: detected capacity change from 0 to 67108864 [ 342.935050] pmem5: detected capacity change from 0 to 63590400 [ 343.059219] pfn5.1: bad offset: 0x35b000 dax disabled align: 0x200000 [ 343.062980] pmem5: detected capacity change from 0 to 67108864 [ 343.240452] list_del corruption. next->prev should be ffff92f06f5ba940, but was (null) [ 343.247955] ------------[ cut here ]------------ [ 343.249663] WARNING: CPU: 2 PID: 15846 at lib/list_debug.c:56 __list_del_entry_valid+0x6c/0xa0 [ 343.252757] Modules linked in: dax_pmem(O) device_dax(O) nd_pmem(O) nd_blk(O) nd_btt(O) nfit_test(O-) nfit(O) libnvdimm(O) nfit_test_iomap(O) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache nfsd ppdev auth_rpcgss nfs_acl lockd parport_pc parport virtio_balloon grace sg pcspkr sunrpc i2c_piix4 i2c_core acpi_cpufreq ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_blk ata_piix libata virtio_pci virtio_ring crc32c_intel serio_raw virtio floppy [ 343.267337] CPU: 2 PID: 15846 Comm: modprobe Tainted: G O 4.12.0-rc1+ #3 [ 343.269805] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [ 343.271694] task: ffff92f068bed500 task.stack: ffff9ff05d4ac000 [ 343.273626] RIP: 0010:__list_del_entry_valid+0x6c/0xa0 [ 343.275271] RSP: 0018:ffff9ff05d4afd38 EFLAGS: 00010046 [ 343.276969] RAX: 0000000000000054 RBX: ffff9ff042d09000 RCX: 0000000000000000 [ 343.279206] RDX: 0000000000000000 RSI: ffff92f06f28e0a8 RDI: ffff92f06f28e0a8 [ 343.281545] RBP: ffff9ff05d4afd38 R08: 00000000fffffffe R09: 0000000000000236 [ 343.283737] R10: 0000000000000005 R11: 0000000000000235 R12: ffff9ff05d4afd70 [ 343.285799] R13: ffff9ff042d09000 R14: 0000000000000000 R15: ffff92f066ed3c00 [ 343.287859] FS: 00007f9a9c471740(0000) GS:ffff92f06f280000(0000) knlGS:0000000000000000 [ 343.290175] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 343.291852] CR2: 00007f9a9b914eb0 CR3: 00000003b58cd000 CR4: 00000000000006e0 [ 343.293975] Call Trace: [ 343.294751] release_nodes+0x76/0x260 [ 343.295881] devres_release_all+0x3c/0x50 [ 343.297070] device_release_driver_internal+0x159/0x200 [ 343.298598] device_release_driver+0x12/0x20 [ 343.299919] bus_remove_device+0xfd/0x170 [ 343.301133] device_del+0x1e8/0x330 [ 343.302180] ? __getnstimeofday64+0x3c/0xd0 [ 343.303412] platform_device_del+0x28/0x90 [ 343.304626] platform_device_unregister+0x12/0x30 [ 343.306065] nfit_test_exit+0x2a/0x93b [nfit_test] [ 343.307477] SyS_delete_module+0x171/0x250 [ 343.308716] do_syscall_64+0x67/0x150 [ 343.309832] entry_SYSCALL64_slow_path+0x25/0x25 [ 343.311204] RIP: 0033:0x7f9a9b945c27 [ 343.312263] RSP: 002b:00007ffef08b9728 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0 [ 343.314468] RAX: ffffffffffffffda RBX: 0000000001888490 RCX: 00007f9a9b945c27 [ 343.316535] RDX: 0000000000000000 RSI: 0000000000000800 RDI: 00000000018884f8 [ 343.318615] RBP: 0000000000000000 R08: 00007f9a9bc09060 R09: 00007f9a9b9b6000 [ 343.320682] R10: 00007ffef08b94b0 R11: 0000000000000206 R12: 0000000000000000 [ 343.322758] R13: 0000000000000001 R14: 00000000018884f8 R15: 0000000000000000 [ 343.324771] Code: 48 89 c2 48 89 fe 31 c0 48 c7 c7 48 6a a7 b9 e8 fa 37 e0 ff 0f ff 31 c0 5d c3 48 89 fe 31 c0 48 c7 c7 f8 6a a7 b9 e8 e3 37 e0 ff <0f> ff 31 c0 5d c3 48 89 fe 31 c0 48 c7 c7 b8 6a a7 b9 e8 cc 37 [ 343.329870] ---[ end trace 7b42652be22fe7db ]--- [ 343.331207] BUG: unable to handle kernel NULL pointer dereference at (null) [ 343.333310] IP: __list_del_entry_valid+0x29/0xa0 [ 343.334618] PGD 3b5a8b067 [ 343.334619] P4D 3b5a8b067 [ 343.335365] PUD 3b5ad1067 [ 343.336109] PMD 0 [ 343.336856] [ 343.337886] Oops: 0000 [#1] SMP [ 343.338809] Modules linked in: dax_pmem(O) device_dax(O) nd_pmem(O) nd_blk(O) nd_btt(O) nfit_test(O-) nfit(O) libnvdimm(O) nfit_test_iomap(O) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache nfsd ppdev auth_rpcgss nfs_acl lockd parport_pc parport virtio_balloon grace sg pcspkr sunrpc i2c_piix4 i2c_core acpi_cpufreq ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_blk ata_piix libata virtio_pci virtio_ring crc32c_intel serio_raw virtio floppy [ 343.350897] CPU: 2 PID: 15846 Comm: modprobe Tainted: G W O 4.12.0-rc1+ #3 [ 343.352954] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [ 343.354403] task: ffff92f068bed500 task.stack: ffff9ff05d4ac000 [ 343.355876] RIP: 0010:__list_del_entry_valid+0x29/0xa0 [ 343.357213] RSP: 0018:ffff9ff05d4afd38 EFLAGS: 00010007 [ 343.358515] RAX: 0000000000000000 RBX: 0000000000000000 RCX: dead000000000200 [ 343.360275] RDX: 0000000000000000 RSI: ffff9ff05d4afd70 RDI: ffff9ff042d09000 [ 343.362026] RBP: ffff9ff05d4afd38 R08: 00000000fffffffe R09: ffff9ff042d09000 [ 343.363830] R10: 0000000000000005 R11: ffff92f06f5ba920 R12: ffff9ff05d4afd70 [ 343.365498] R13: 0000000000000000 R14: 0000000000000000 R15: ffff92f066ed3c00 [ 343.367137] FS: 00007f9a9c471740(0000) GS:ffff92f06f280000(0000) knlGS:0000000000000000 [ 343.369017] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 343.370362] CR2: 0000000000000000 CR3: 00000003b58cd000 CR4: 00000000000006e0 [ 343.372108] Call Trace: [ 343.372700] release_nodes+0x76/0x260 [ 343.373560] devres_release_all+0x3c/0x50 [ 343.374532] device_release_driver_internal+0x159/0x200 [ 343.375771] device_release_driver+0x12/0x20 [ 343.376834] bus_remove_device+0xfd/0x170 [ 343.377799] device_del+0x1e8/0x330 [ 343.378670] ? __getnstimeofday64+0x3c/0xd0 [ 343.379657] platform_device_del+0x28/0x90 [ 343.380687] platform_device_unregister+0x12/0x30 [ 343.381796] nfit_test_exit+0x2a/0x93b [nfit_test] [ 343.382926] SyS_delete_module+0x171/0x250 [ 343.383904] do_syscall_64+0x67/0x150 [ 343.384779] entry_SYSCALL64_slow_path+0x25/0x25 [ 343.385870] RIP: 0033:0x7f9a9b945c27 [ 343.386716] RSP: 002b:00007ffef08b9728 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0 [ 343.388484] RAX: ffffffffffffffda RBX: 0000000001888490 RCX: 00007f9a9b945c27 [ 343.390217] RDX: 0000000000000000 RSI: 0000000000000800 RDI: 00000000018884f8 [ 343.391875] RBP: 0000000000000000 R08: 00007f9a9bc09060 R09: 00007f9a9b9b6000 [ 343.393619] R10: 00007ffef08b94b0 R11: 0000000000000206 R12: 0000000000000000 [ 343.395291] R13: 0000000000000001 R14: 00000000018884f8 R15: 0000000000000000 [ 343.397010] Code: 00 00 55 48 8b 07 48 b9 00 01 00 00 00 00 ad de 48 8b 57 08 48 89 e5 48 39 c8 74 27 48 b9 00 02 00 00 00 00 ad de 48 39 ca 74 60 <48> 8b 12 48 39 d7 75 41 48 8b 50 08 48 39 d7 75 21 b8 01 00 00 [ 343.401444] RIP: __list_del_entry_valid+0x29/0xa0 RSP: ffff9ff05d4afd38 [ 343.402972] CR2: 0000000000000000 [ 343.403774] ---[ end trace 7b42652be22fe7dc ]--- [ 343.404872] Kernel panic - not syncing: Fatal exception [ 343.438096] Kernel Offset: 0x38000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) [ 343.440605] ---[ end Kernel panic - not syncing: Fatal exception -- Yasunori Goto <y-goto@jp.fujitsu.com> _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Fw: Panic when make check for ndctl 2017-05-29 14:21 ` Fwd: Fw: Panic when make check for ndctl Yasunori Goto @ 2017-05-29 23:04 ` Dan Williams 2017-05-30 7:34 ` Yasunori Goto 0 siblings, 1 reply; 14+ messages in thread From: Dan Williams @ 2017-05-29 23:04 UTC (permalink / raw) To: Yasunori Goto; +Cc: linux-nvdimm@lists.01.org On Mon, May 29, 2017 at 7:21 AM, Yasunori Goto <ygotopersonal@gmail.com> wrote: > > Hello, > I tried "make check" of ndctl command on the kernel of libnvdimm-for-next > branch(*), > but I feel its kernel is unstable when nfit_test.ko module is loaded. > > (*) git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm > libnvdimm-for-next > The newest commit is f5705aa8cfed > > Here is console logs when the kernel was paniced, and I'll attach the kernel > config file. > > Does anyone know how to solve this? Am I something wrong? No, you're not doing anything wrong. This is a known issue that I am investigating. The reports of this usually come from distributions with older module loading infrastructure. If you can try with a recent Fedora that would mimic the environment I normally use to test, but hopefully in the next week or so I'll root cause what is triggering this on CentOS and Debian. _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Fw: Panic when make check for ndctl 2017-05-29 23:04 ` Dan Williams @ 2017-05-30 7:34 ` Yasunori Goto 2017-05-30 11:44 ` Dan Williams 2017-06-06 5:51 ` Yasunori Goto 0 siblings, 2 replies; 14+ messages in thread From: Yasunori Goto @ 2017-05-30 7:34 UTC (permalink / raw) To: Dan Williams; +Cc: linux-nvdimm@lists.01.org On 2017年05月30日 08:04, Dan Williams wrote: > On Mon, May 29, 2017 at 7:21 AM, Yasunori Goto <ygotopersonal@gmail.com> wrote: >> Hello, >> I tried "make check" of ndctl command on the kernel of libnvdimm-for-next >> branch(*), >> but I feel its kernel is unstable when nfit_test.ko module is loaded. >> >> (*) git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm >> libnvdimm-for-next >> The newest commit is f5705aa8cfed >> >> Here is console logs when the kernel was paniced, and I'll attach the kernel >> config file. >> >> Does anyone know how to solve this? Am I something wrong? > No, you're not doing anything wrong. This is a known issue that I am > investigating. The reports of this usually come from distributions > with older module loading infrastructure. If you can try with a recent > Fedora that would mimic the environment I normally use to test, Oh, I see. Certainly, I used CentOS7. Ok, I'll use Fedora environment. > but hopefully in the next week or so I'll root cause what is triggering > this on CentOS and Debian. Thanks, BTW, I could take the crash dump when this panic occurred. Is the crash dump helpful for your investigation of root cause? If yes, I'll upload the dump to google storage. Regards, _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Fw: Panic when make check for ndctl 2017-05-30 7:34 ` Yasunori Goto @ 2017-05-30 11:44 ` Dan Williams 2017-06-06 5:51 ` Yasunori Goto 1 sibling, 0 replies; 14+ messages in thread From: Dan Williams @ 2017-05-30 11:44 UTC (permalink / raw) To: Yasunori Goto; +Cc: linux-nvdimm@lists.01.org On Tue, May 30, 2017 at 12:34 AM, Yasunori Goto <ygotopersonal@gmail.com> wrote: > > > On 2017年05月30日 08:04, Dan Williams wrote: >> >> On Mon, May 29, 2017 at 7:21 AM, Yasunori Goto <ygotopersonal@gmail.com> >> wrote: >>> >>> Hello, >>> I tried "make check" of ndctl command on the kernel of libnvdimm-for-next >>> branch(*), >>> but I feel its kernel is unstable when nfit_test.ko module is loaded. >>> >>> (*) git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm >>> libnvdimm-for-next >>> The newest commit is f5705aa8cfed >>> >>> Here is console logs when the kernel was paniced, and I'll attach the >>> kernel >>> config file. >>> >>> Does anyone know how to solve this? Am I something wrong? >> >> No, you're not doing anything wrong. This is a known issue that I am >> investigating. The reports of this usually come from distributions >> with older module loading infrastructure. If you can try with a recent >> Fedora that would mimic the environment I normally use to test, > > > Oh, I see. Certainly, I used CentOS7. > Ok, I'll use Fedora environment. > >> but hopefully in the next week or so I'll root cause what is triggering >> this on CentOS and Debian. > > Thanks, > > BTW, I could take the crash dump when this panic occurred. > Is the crash dump helpful for your investigation of root cause? > If yes, I'll upload the dump to google storage. Thanks for the offer. I do have access to a system that can reproduce this so no need to collect the crash dump. _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Panic when make check for ndctl 2017-05-30 7:34 ` Yasunori Goto 2017-05-30 11:44 ` Dan Williams @ 2017-06-06 5:51 ` Yasunori Goto 2017-06-06 13:32 ` Linda Knippers 2017-06-06 16:42 ` Dan Williams 1 sibling, 2 replies; 14+ messages in thread From: Yasunori Goto @ 2017-06-06 5:51 UTC (permalink / raw) To: Dan Williams; +Cc: Yasunori Goto, linux-nvdimm@lists.01.org Hi, > > No, you're not doing anything wrong. This is a known issue that I am > > investigating. The reports of this usually come from distributions > > with older module loading infrastructure. If you can try with a recent > > Fedora that would mimic the environment I normally use to test, > > Oh, I see. Certainly, I used CentOS7. > Ok, I'll use Fedora environment. Hmmm, though I made Fedora 25 environment, this panic still occurs... I'll attach syslog and .config again. [ 105.119978] nfit_test_iomap: loading out-of-tree module taints kernel. [ 105.306302] nfit_test nfit_test.0: found a zero length table '0' parsing nfit [ 106.573244] random: crng init done [ 107.339579] nfit_test nfit_test.0: failed to evaluate _FIT [ 107.358018] nfit_test nfit_test.1: nmem4 flags: save_fail restore_fail flush_fail not_armed [ 107.360586] nfit_test nfit_test.1: nmem5 flags: map_fail [ 107.480603] pmem4: detected capacity change from 0 to 18874368 [ 107.510795] pmem4: detected capacity change from 0 to 18874368 [ 107.539968] pmem4: detected capacity change from 0 to 18874368 [ 107.630683] pmem5: detected capacity change from 0 to 20971520 [ 107.662595] pmem5: detected capacity change from 0 to 20971520 [ 107.697813] pmem5: detected capacity change from 0 to 20971520 [ 107.738166] ndblk0.0: detected capacity change from 0 to 7340032 [ 107.768583] ndblk0.0: detected capacity change from 0 to 7340032 [ 107.797455] ndblk0.0: detected capacity change from 0 to 6524416 [ 107.823556] ndblk0.0: detected capacity change from 0 to 6524416 [ 107.853451] ndblk0.0: detected capacity change from 0 to 6524416 [ 107.883705] ndblk0.0: detected capacity change from 0 to 6524416 [ 107.913563] ndblk0.0: detected capacity change from 0 to 7340032 [ 107.948093] ndblk0.0: detected capacity change from 0 to 7340032 [ 107.982461] ndblk0.0: detected capacity change from 0 to 7225344 [ 108.015363] ndblk0.0: detected capacity change from 0 to 7225344 [ 108.046480] ndblk0.0: detected capacity change from 0 to 7225344 [ 108.071713] ndblk0.0: detected capacity change from 0 to 7225344 [ 108.098458] ndblk0.0: detected capacity change from 0 to 7114752 [ 108.127799] ndblk0.0: detected capacity change from 0 to 7114752 [ 108.133122] ndblk0.1: detected capacity change from 0 to 11534336 [ 108.162370] ndblk0.1: detected capacity change from 0 to 11534336 [ 108.192647] ndblk0.1: detected capacity change from 0 to 10252288 [ 108.218613] ndblk0.1: detected capacity change from 0 to 10252288 [ 108.248650] ndblk0.1: detected capacity change from 0 to 10252288 [ 108.276487] ndblk0.1: detected capacity change from 0 to 10252288 [ 108.307132] ndblk0.1: detected capacity change from 0 to 11534336 [ 108.336644] ndblk0.1: detected capacity change from 0 to 11534336 [ 108.366668] ndblk0.1: detected capacity change from 0 to 11354112 [ 108.399347] ndblk0.1: detected capacity change from 0 to 11354112 [ 108.428403] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only [ 108.428679] pmem7: detected capacity change from 0 to 4194304 [ 108.439346] pmem6: detected capacity change from 0 to 33554432 [ 108.448567] ndblk0.1: detected capacity change from 0 to 11354112 [ 108.475539] ndblk0.1: detected capacity change from 0 to 11354112 [ 108.501562] ndblk0.1: detected capacity change from 0 to 11182080 [ 108.529544] ndblk0.1: detected capacity change from 0 to 11182080 [ 108.580854] ndblk0.1: detected capacity change from 0 to 7340032 [ 108.580861] ndblk0.0: detected capacity change from 0 to 11534336 [ 108.619809] ndblk0.1: detected capacity change from 0 to 7340032 [ 108.644717] ndblk0.0: detected capacity change from 0 to 11534336 [ 108.706521] ndblk1.0: detected capacity change from 0 to 7340032 [ 108.735577] ndblk1.0: detected capacity change from 0 to 7340032 [ 108.763659] ndblk1.0: detected capacity change from 0 to 6524416 [ 108.793613] ndblk1.0: detected capacity change from 0 to 6524416 [ 108.822612] ndblk1.0: detected capacity change from 0 to 6524416 [ 108.852674] ndblk1.0: detected capacity change from 0 to 6524416 [ 108.881684] ndblk1.0: detected capacity change from 0 to 7340032 [ 108.911788] ndblk1.0: detected capacity change from 0 to 7340032 [ 108.941352] ndblk1.0: detected capacity change from 0 to 7225344 [ 108.970375] ndblk1.0: detected capacity change from 0 to 7225344 [ 108.998504] ndblk1.0: detected capacity change from 0 to 7225344 [ 109.031416] ndblk1.0: detected capacity change from 0 to 7225344 [ 109.062783] ndblk1.0: detected capacity change from 0 to 7114752 [ 109.088454] ndblk1.0: detected capacity change from 0 to 7114752 [ 109.093521] ndblk1.1: detected capacity change from 0 to 11534336 [ 109.127332] ndblk1.1: detected capacity change from 0 to 11534336 [ 109.157533] ndblk1.1: detected capacity change from 0 to 10252288 [ 109.183291] ndblk1.1: detected capacity change from 0 to 10252288 [ 109.212336] ndblk1.1: detected capacity change from 0 to 10252288 [ 109.238434] ndblk1.1: detected capacity change from 0 to 10252288 [ 109.271471] ndblk1.1: detected capacity change from 0 to 11534336 [ 109.297386] ndblk1.1: detected capacity change from 0 to 11534336 [ 109.328372] ndblk1.1: detected capacity change from 0 to 11354112 [ 109.361400] ndblk1.1: detected capacity change from 0 to 11354112 [ 109.392430] ndblk1.1: detected capacity change from 0 to 11354112 [ 109.428406] ndblk1.1: detected capacity change from 0 to 11354112 [ 109.454491] ndblk1.1: detected capacity change from 0 to 11182080 [ 109.487353] ndblk1.1: detected capacity change from 0 to 11182080 [ 109.537879] ndblk1.1: detected capacity change from 0 to 7340032 [ 109.537967] ndblk1.0: detected capacity change from 0 to 11534336 [ 109.565350] ndblk1.1: detected capacity change from 0 to 7340032 [ 109.598463] ndblk1.0: detected capacity change from 0 to 11534336 [ 109.660689] ndblk2.0: detected capacity change from 0 to 28311552 [ 109.690525] ndblk2.0: detected capacity change from 0 to 28311552 [ 109.716581] ndblk2.0: detected capacity change from 0 to 25165824 [ 109.745327] ndblk2.0: detected capacity change from 0 to 25165824 [ 109.770396] ndblk2.0: detected capacity change from 0 to 25165824 [ 109.803360] ndblk2.0: detected capacity change from 0 to 25165824 [ 109.832330] ndblk2.0: detected capacity change from 0 to 28311552 [ 109.858490] ndblk2.0: detected capacity change from 0 to 28311552 [ 109.888490] ndblk2.0: detected capacity change from 0 to 27873280 [ 109.917227] ndblk2.0: detected capacity change from 0 to 27873280 [ 109.945591] ndblk2.0: detected capacity change from 0 to 27873280 [ 109.974347] ndblk2.0: detected capacity change from 0 to 27873280 [ 110.003344] ndblk2.0: detected capacity change from 0 to 27451392 [ 110.032411] ndblk2.0: detected capacity change from 0 to 27451392 [ 110.061529] ndblk2.0: detected capacity change from 0 to 28311552 [ 110.095333] ndblk2.0: detected capacity change from 0 to 28311552 [ 110.129210] ndblk3.0: detected capacity change from 0 to 28311552 [ 110.156337] ndblk3.0: detected capacity change from 0 to 28311552 [ 110.186749] ndblk3.0: detected capacity change from 0 to 25165824 [ 110.216575] ndblk3.0: detected capacity change from 0 to 25165824 [ 110.246734] ndblk3.0: detected capacity change from 0 to 25165824 [ 110.274606] ndblk3.0: detected capacity change from 0 to 25165824 [ 110.303706] ndblk3.0: detected capacity change from 0 to 28311552 [ 110.332341] ndblk3.0: detected capacity change from 0 to 28311552 [ 110.362347] ndblk3.0: detected capacity change from 0 to 27873280 [ 110.391318] ndblk3.0: detected capacity change from 0 to 27873280 [ 110.419590] ndblk3.0: detected capacity change from 0 to 27873280 [ 110.449602] ndblk3.0: detected capacity change from 0 to 27873280 [ 110.475609] ndblk3.0: detected capacity change from 0 to 27451392 [ 110.509744] ndblk3.0: detected capacity change from 0 to 27451392 [ 110.539967] ndblk3.0: detected capacity change from 0 to 28311552 [ 110.568702] ndblk3.0: detected capacity change from 0 to 28311552 [ 110.637689] pmem4: detected capacity change from 0 to 18866176 [ 110.669552] pmem4: detected capacity change from 0 to 14700544 [ 110.700010] pmem4: detected capacity change from 0 to 14700544 [ 110.736370] pmem4: detected capacity change from 0 to 14700544 [ 110.767042] pmem4: detected capacity change from 0 to 18874368 [ 110.798402] pmem4: detected capacity change from 0 to 18874368 [ 110.828564] pmem4: detected capacity change from 0 to 18874368 [ 110.864839] pmem5: detected capacity change from 0 to 20963328 [ 110.899520] pmem5: detected capacity change from 0 to 16801792 [ 110.931500] pmem5: detected capacity change from 0 to 16801792 [ 110.965372] pmem5: detected capacity change from 0 to 16801792 [ 110.996902] pmem5: detected capacity change from 0 to 20971520 [ 111.028319] pmem5: detected capacity change from 0 to 20971520 [ 111.061723] pmem5: detected capacity change from 0 to 20971520 [ 111.096534] ndblk0.0: detected capacity change from 0 to 7340032 [ 111.126313] ndblk0.0: detected capacity change from 0 to 7340032 [ 111.155570] ndblk0.0: detected capacity change from 0 to 6524416 [ 111.188508] ndblk0.0: detected capacity change from 0 to 6524416 [ 111.218726] ndblk0.0: detected capacity change from 0 to 6524416 [ 111.248807] ndblk0.0: detected capacity change from 0 to 6524416 [ 111.279621] ndblk0.0: detected capacity change from 0 to 7340032 [ 111.309380] ndblk0.0: detected capacity change from 0 to 7340032 [ 111.343599] ndblk0.0: detected capacity change from 0 to 7225344 [ 111.371350] ndblk0.0: detected capacity change from 0 to 7225344 [ 111.396568] ndblk0.0: detected capacity change from 0 to 7225344 [ 111.422790] ndblk0.0: detected capacity change from 0 to 7225344 [ 111.449014] ndblk0.0: detected capacity change from 0 to 7114752 [ 111.479696] ndblk0.0: detected capacity change from 0 to 7114752 [ 111.484436] ndblk0.1: detected capacity change from 0 to 11534336 [ 111.511601] ndblk0.1: detected capacity change from 0 to 11534336 [ 111.537681] ndblk0.1: detected capacity change from 0 to 10252288 [ 111.562396] ndblk0.1: detected capacity change from 0 to 10252288 [ 111.589685] ndblk0.1: detected capacity change from 0 to 10252288 [ 111.619815] ndblk0.1: detected capacity change from 0 to 10252288 [ 111.646740] ndblk0.1: detected capacity change from 0 to 11534336 [ 111.676715] ndblk0.1: detected capacity change from 0 to 11534336 [ 111.702806] ndblk0.1: detected capacity change from 0 to 11354112 [ 111.729349] ndblk0.1: detected capacity change from 0 to 11354112 [ 111.756575] ndblk0.1: detected capacity change from 0 to 11354112 [ 111.787806] ndblk0.1: detected capacity change from 0 to 11354112 [ 111.817861] ndblk0.1: detected capacity change from 0 to 11182080 [ 111.846802] ndblk0.1: detected capacity change from 0 to 11182080 [ 111.898048] ndblk0.1: detected capacity change from 0 to 7340032 [ 111.898069] ndblk0.0: detected capacity change from 0 to 11534336 [ 111.926589] ndblk0.1: detected capacity change from 0 to 7340032 [ 111.954610] ndblk0.0: detected capacity change from 0 to 11534336 [ 112.013684] ndblk1.0: detected capacity change from 0 to 7340032 [ 112.041567] ndblk1.0: detected capacity change from 0 to 7340032 [ 112.067506] ndblk1.0: detected capacity change from 0 to 6524416 [ 112.099657] ndblk1.0: detected capacity change from 0 to 6524416 [ 112.128561] ndblk1.0: detected capacity change from 0 to 6524416 [ 112.160662] ndblk1.0: detected capacity change from 0 to 6524416 [ 112.190756] ndblk1.0: detected capacity change from 0 to 7340032 [ 112.220656] ndblk1.0: detected capacity change from 0 to 7340032 [ 112.253664] ndblk1.0: detected capacity change from 0 to 7225344 [ 112.283519] ndblk1.0: detected capacity change from 0 to 7225344 [ 112.309701] ndblk1.0: detected capacity change from 0 to 7225344 [ 112.342567] ndblk1.0: detected capacity change from 0 to 7225344 [ 112.372528] ndblk1.0: detected capacity change from 0 to 7114752 [ 112.405435] ndblk1.0: detected capacity change from 0 to 7114752 [ 112.409926] ndblk1.1: detected capacity change from 0 to 11534336 [ 112.438331] ndblk1.1: detected capacity change from 0 to 11534336 [ 112.466617] ndblk1.1: detected capacity change from 0 to 10252288 [ 112.497657] ndblk1.1: detected capacity change from 0 to 10252288 [ 112.528683] ndblk1.1: detected capacity change from 0 to 10252288 [ 112.562610] ndblk1.1: detected capacity change from 0 to 10252288 [ 112.595812] ndblk1.1: detected capacity change from 0 to 11534336 [ 112.625853] ndblk1.1: detected capacity change from 0 to 11534336 [ 112.655840] ndblk1.1: detected capacity change from 0 to 11354112 [ 112.685837] ndblk1.1: detected capacity change from 0 to 11354112 [ 112.715739] ndblk1.1: detected capacity change from 0 to 11354112 [ 112.743641] ndblk1.1: detected capacity change from 0 to 11354112 [ 112.775647] ndblk1.1: detected capacity change from 0 to 11182080 [ 112.803627] ndblk1.1: detected capacity change from 0 to 11182080 [ 112.856962] ndblk1.0: detected capacity change from 0 to 11534336 [ 112.871023] ndblk1.1: detected capacity change from 0 to 7340032 [ 112.898543] ndblk1.1: detected capacity change from 0 to 7340032 [ 112.928733] ndblk1.0: detected capacity change from 0 to 11534336 [ 112.988305] ndblk2.0: detected capacity change from 0 to 28311552 [ 113.015628] ndblk2.0: detected capacity change from 0 to 28311552 [ 113.047834] ndblk2.0: detected capacity change from 0 to 25165824 [ 113.077795] ndblk2.0: detected capacity change from 0 to 25165824 [ 113.119912] ndblk2.0: detected capacity change from 0 to 25165824 [ 113.148630] ndblk2.0: detected capacity change from 0 to 25165824 [ 113.177691] ndblk2.0: detected capacity change from 0 to 28311552 [ 113.206434] ndblk2.0: detected capacity change from 0 to 28311552 [ 113.235665] ndblk2.0: detected capacity change from 0 to 27873280 [ 113.265569] ndblk2.0: detected capacity change from 0 to 27873280 [ 113.293662] ndblk2.0: detected capacity change from 0 to 27873280 [ 113.322613] ndblk2.0: detected capacity change from 0 to 27873280 [ 113.351621] ndblk2.0: detected capacity change from 0 to 27451392 [ 113.380707] ndblk2.0: detected capacity change from 0 to 27451392 [ 113.414204] ndblk2.0: detected capacity change from 0 to 28311552 [ 113.445534] ndblk2.0: detected capacity change from 0 to 28311552 [ 113.479150] ndblk3.0: detected capacity change from 0 to 28311552 [ 113.508569] ndblk3.0: detected capacity change from 0 to 28311552 [ 113.539519] ndblk3.0: detected capacity change from 0 to 25165824 [ 113.571755] ndblk3.0: detected capacity change from 0 to 25165824 [ 113.601544] ndblk3.0: detected capacity change from 0 to 25165824 [ 113.631450] ndblk3.0: detected capacity change from 0 to 25165824 [ 113.661343] ndblk3.0: detected capacity change from 0 to 28311552 [ 113.690698] ndblk3.0: detected capacity change from 0 to 28311552 [ 113.720616] ndblk3.0: detected capacity change from 0 to 27873280 [ 113.746796] ndblk3.0: detected capacity change from 0 to 27873280 [ 113.775765] ndblk3.0: detected capacity change from 0 to 27873280 [ 113.804548] ndblk3.0: detected capacity change from 0 to 27873280 [ 113.834762] ndblk3.0: detected capacity change from 0 to 27451392 [ 113.865715] ndblk3.0: detected capacity change from 0 to 27451392 [ 113.900500] ndblk3.0: detected capacity change from 0 to 28311552 [ 113.929600] ndblk3.0: detected capacity change from 0 to 28311552 [ 113.980479] nd_pmem btt4.0: No existing arenas [ 113.983934] pmem4s: detected capacity change from 0 to 18564096 [ 114.020170] pmem4s: detected capacity change from 0 to 18564096 [ 114.053960] pmem4s: detected capacity change from 0 to 18564096 [ 114.088905] pmem4s: detected capacity change from 0 to 148512768 [ 114.123022] pmem4s: detected capacity change from 0 to 148512768 [ 114.156559] pmem4s: detected capacity change from 0 to 148512768 [ 114.192973] pmem4s: detected capacity change from 0 to 148512768 [ 114.227204] pmem4s: detected capacity change from 0 to 18564096 [ 114.262795] pmem4s: detected capacity change from 0 to 18564096 [ 114.296797] pmem4: detected capacity change from 0 to 18874368 [ 114.332711] pmem4: detected capacity change from 0 to 18874368 [ 114.362463] pmem4: detected capacity change from 0 to 18874368 [ 114.400217] nd_pmem btt5.0: No existing arenas [ 114.404187] pmem5s: detected capacity change from 0 to 20644864 [ 114.442874] pmem5s: detected capacity change from 0 to 20644864 [ 114.480026] pmem5s: detected capacity change from 0 to 20644864 [ 114.515993] pmem5s: detected capacity change from 0 to 165158912 [ 114.552262] pmem5s: detected capacity change from 0 to 165158912 [ 114.592288] pmem5s: detected capacity change from 0 to 165158912 [ 114.628456] pmem5s: detected capacity change from 0 to 165158912 [ 114.667815] pmem5s: detected capacity change from 0 to 20644864 [ 114.698337] pmem5s: detected capacity change from 0 to 20644864 [ 114.727759] pmem5: detected capacity change from 0 to 20971520 [ 114.759815] pmem5: detected capacity change from 0 to 20971520 [ 114.792048] pmem5: detected capacity change from 0 to 20971520 [ 114.827093] ndblk0.0: detected capacity change from 0 to 7340032 [ 114.855851] ndblk0.0: detected capacity change from 0 to 7340032 [ 114.882503] ndblk0.0: detected capacity change from 0 to 6524416 [ 114.912396] ndblk0.0: detected capacity change from 0 to 6524416 [ 114.942545] ndblk0.0: detected capacity change from 0 to 6524416 [ 114.972699] ndblk0.0: detected capacity change from 0 to 6524416 [ 115.003739] ndblk0.0: detected capacity change from 0 to 7340032 [ 115.032844] ndblk0.0: detected capacity change from 0 to 7340032 [ 115.058561] ndblk0.0: detected capacity change from 0 to 7225344 [ 115.084344] ndblk0.0: detected capacity change from 0 to 7225344 [ 115.113604] ndblk0.0: detected capacity change from 0 to 7225344 [ 115.144313] ndblk0.0: detected capacity change from 0 to 7225344 [ 115.173577] ndblk0.0: detected capacity change from 0 to 7114752 [ 115.201318] ndblk0.0: detected capacity change from 0 to 7114752 [ 115.206551] ndblk0.1: detected capacity change from 0 to 11534336 [ 115.233525] ndblk0.1: detected capacity change from 0 to 11534336 [ 115.261761] ndblk0.1: detected capacity change from 0 to 10252288 [ 115.290866] ndblk0.1: detected capacity change from 0 to 10252288 [ 115.319724] ndblk0.1: detected capacity change from 0 to 10252288 [ 115.348726] ndblk0.1: detected capacity change from 0 to 10252288 [ 115.375692] ndblk0.1: detected capacity change from 0 to 11534336 [ 115.402375] ndblk0.1: detected capacity change from 0 to 11534336 [ 115.428721] ndblk0.1: detected capacity change from 0 to 11354112 [ 115.457458] ndblk0.1: detected capacity change from 0 to 11354112 [ 115.487723] ndblk0.1: detected capacity change from 0 to 11354112 [ 115.517554] ndblk0.1: detected capacity change from 0 to 11354112 [ 115.547857] ndblk0.1: detected capacity change from 0 to 11182080 [ 115.576733] ndblk0.1: detected capacity change from 0 to 11182080 [ 115.627671] ndblk0.0: detected capacity change from 0 to 11534336 [ 115.627688] ndblk0.1: detected capacity change from 0 to 7340032 [ 115.659725] ndblk0.1: detected capacity change from 0 to 7340032 [ 115.689442] ndblk0.0: detected capacity change from 0 to 11534336 [ 115.745343] ndblk1.0: detected capacity change from 0 to 7340032 [ 115.775727] ndblk1.0: detected capacity change from 0 to 7340032 [ 115.805728] ndblk1.0: detected capacity change from 0 to 6524416 [ 115.831536] ndblk1.0: detected capacity change from 0 to 6524416 [ 115.861891] ndblk1.0: detected capacity change from 0 to 6524416 [ 115.891832] ndblk1.0: detected capacity change from 0 to 6524416 [ 115.921839] ndblk1.0: detected capacity change from 0 to 7340032 [ 115.951448] ndblk1.0: detected capacity change from 0 to 7340032 [ 115.977690] ndblk1.0: detected capacity change from 0 to 7225344 [ 116.010474] ndblk1.0: detected capacity change from 0 to 7225344 [ 116.039340] ndblk1.0: detected capacity change from 0 to 7225344 [ 116.065740] ndblk1.0: detected capacity change from 0 to 7225344 [ 116.095687] ndblk1.0: detected capacity change from 0 to 7114752 [ 116.124474] ndblk1.0: detected capacity change from 0 to 7114752 [ 116.130332] ndblk1.1: detected capacity change from 0 to 11534336 [ 116.156512] ndblk1.1: detected capacity change from 0 to 11534336 [ 116.186542] ndblk1.1: detected capacity change from 0 to 10252288 [ 116.223188] ndblk1.1: detected capacity change from 0 to 10252288 [ 116.250695] ndblk1.1: detected capacity change from 0 to 10252288 [ 116.280524] ndblk1.1: detected capacity change from 0 to 10252288 [ 116.306568] ndblk1.1: detected capacity change from 0 to 11534336 [ 116.335389] ndblk1.1: detected capacity change from 0 to 11534336 [ 116.365484] ndblk1.1: detected capacity change from 0 to 11354112 [ 116.390303] ndblk1.1: detected capacity change from 0 to 11354112 [ 116.418332] ndblk1.1: detected capacity change from 0 to 11354112 [ 116.444226] ndblk1.1: detected capacity change from 0 to 11354112 [ 116.469322] ndblk1.1: detected capacity change from 0 to 11182080 [ 116.500557] ndblk1.1: detected capacity change from 0 to 11182080 [ 116.554723] ndblk1.0: detected capacity change from 0 to 11534336 [ 116.554895] ndblk1.1: detected capacity change from 0 to 7340032 [ 116.583641] ndblk1.1: detected capacity change from 0 to 7340032 [ 116.613324] ndblk1.0: detected capacity change from 0 to 11534336 [ 116.671228] nd_blk btt2.0: No existing arenas [ 116.675514] ndblk2.0s: detected capacity change from 0 to 27928064 [ 116.711302] ndblk2.0s: detected capacity change from 0 to 27928064 [ 116.750457] ndblk2.0s: detected capacity change from 0 to 27928064 [ 116.786165] ndblk2.0s: detected capacity change from 0 to 223424512 [ 116.821733] ndblk2.0s: detected capacity change from 0 to 223424512 [ 116.857652] ndblk2.0s: detected capacity change from 0 to 223424512 [ 116.890435] ndblk2.0s: detected capacity change from 0 to 223424512 [ 116.925862] ndblk2.0s: detected capacity change from 0 to 27928064 [ 116.967171] ndblk2.0s: detected capacity change from 0 to 27928064 [ 117.000647] ndblk2.0: detected capacity change from 0 to 28311552 [ 117.026754] ndblk2.0: detected capacity change from 0 to 28311552 [ 117.056780] ndblk2.0: detected capacity change from 0 to 28311552 [ 117.091005] nd_blk btt3.0: No existing arenas [ 117.094760] ndblk3.0s: detected capacity change from 0 to 27928064 [ 117.129367] ndblk3.0s: detected capacity change from 0 to 27928064 [ 117.165070] ndblk3.0s: detected capacity change from 0 to 27928064 [ 117.200090] ndblk3.0s: detected capacity change from 0 to 223424512 [ 117.235165] ndblk3.0s: detected capacity change from 0 to 223424512 [ 117.274015] ndblk3.0s: detected capacity change from 0 to 223424512 [ 117.316141] ndblk3.0s: detected capacity change from 0 to 223424512 [ 117.350107] ndblk3.0s: detected capacity change from 0 to 27928064 [ 117.380054] ndblk3.0s: detected capacity change from 0 to 27928064 [ 117.414741] ndblk3.0: detected capacity change from 0 to 28311552 [ 117.442741] ndblk3.0: detected capacity change from 0 to 28311552 [ 117.471744] ndblk3.0: detected capacity change from 0 to 28311552 [ 117.529686] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only [ 117.535374] pmem6: detected capacity change from 0 to 33554432 [ 117.541192] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only [ 117.543681] pmem6: detected capacity change from 33554432 to 0 [ 117.569779] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only [ 117.573036] pmem6: detected capacity change from 0 to 33554432 [ 117.604734] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only [ 117.607796] pmem6: detected capacity change from 0 to 33554432 [ 117.613821] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only [ 117.616799] pmem6: detected capacity change from 33554432 to 0 [ 117.641595] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only [ 117.644598] pmem6: detected capacity change from 0 to 33554432 [ 117.719438] list_del corruption. next->prev should be ffff88041cac41e0, but was 65745f7469666e3a [ 117.722832] ------------[ cut here ]------------ [ 117.724477] WARNING: CPU: 4 PID: 2233 at lib/list_debug.c:56 __list_del_entry_valid+0x6c/0xa0 [ 117.727507] Modules linked in: dax_pmem(O) device_dax(O) nd_pmem(O) nd_blk(O) nd_btt(O) nfit_test(O-) nfit(O) libnvdimm(O) nfit_test_iomap(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc ppdev parport_pc pcspkr i2c_piix4 parport floppy i2c_core virtio_net virtio_balloon acpi_cpufreq sch_fq_codel ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk ata_piix libata virtio_pci crc32c_intel virtio_ring serio_raw virtio [ 117.741905] CPU: 4 PID: 2233 Comm: lt-libndctl Tainted: G O 4.12.0-rc1+ #12 [ 117.744461] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [ 117.746310] task: ffff88041d685500 task.stack: ffffc90002b50000 [ 117.748198] RIP: 0010:__list_del_entry_valid+0x6c/0xa0 [ 117.751084] RSP: 0018:ffffc90002b53d60 EFLAGS: 00010046 [ 117.753142] RAX: 0000000000000054 RBX: ffffc9001700e000 RCX: 0000000000000000 [ 117.755955] RDX: 0000000000000000 RSI: ffff88043310e0a8 RDI: ffff88043310e0a8 [ 117.758745] RBP: ffffc90002b53d60 R08: 00000000fffffffe R09: 0000000000000382 [ 117.760801] R10: 0000000000000005 R11: 0000000000000381 R12: ffffc90002b53d98 [ 117.762949] R13: ffffc9001700e000 R14: 0000000000000000 R15: ffff88041beefc08 [ 117.765000] FS: 00007f7d70082c80(0000) GS:ffff880433100000(0000) knlGS:0000000000000000 [ 117.767300] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 117.768962] CR2: 000055c0db522cc0 CR3: 0000000422156000 CR4: 00000000000006e0 [ 117.770975] Call Trace: [ 117.771716] release_nodes+0x76/0x260 [ 117.772797] devres_release_all+0x3c/0x50 [ 117.773981] device_release_driver_internal+0x159/0x200 [ 117.775494] device_release_driver+0x12/0x20 [ 117.776763] bus_remove_device+0xfd/0x170 [ 117.777924] device_del+0x1e8/0x330 [ 117.778943] platform_device_del+0x28/0x90 [ 117.780129] platform_device_unregister+0x12/0x30 [ 117.781483] nfit_test_exit+0x2a/0x93b [nfit_test] [ 117.782858] SyS_delete_module+0x171/0x250 [ 117.784044] entry_SYSCALL_64_fastpath+0x1a/0xa5 [ 117.785346] RIP: 0033:0x7f7d6f399487 [ 117.786377] RSP: 002b:00007ffc604502a8 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0 [ 117.788503] RAX: ffffffffffffffda RBX: 0000000002000000 RCX: 00007f7d6f399487 [ 117.790525] RDX: 00000000000000e8 RSI: 0000000000000800 RDI: 00000000020838f8 [ 117.792635] RBP: 00007ffc6044fc60 R08: 000000000208fb00 R09: 0000000000000037 [ 117.794641] R10: 000000000000006d R11: 0000000000000206 R12: 0000000000404e60 [ 117.796656] R13: 00007ffc60450440 R14: 0000000000000000 R15: 0000000000000000 [ 117.798699] Code: 48 89 c2 48 89 fe 31 c0 48 c7 c7 38 3f a7 81 e8 1e 34 e0 ff 0f ff 31 c0 5d c3 48 89 fe 31 c0 48 c7 c7 e8 3f a7 81 e8 07 34 e0 ff <0f> ff 31 c0 5d c3 48 89 fe 31 c0 48 c7 c7 a8 3f a7 81 e8 f0 33 [ 117.803715] ---[ end trace 06a87dde16f89e24 ]--- [ 117.804948] general protection fault: 0000 [#1] SMP [ 117.806233] Modules linked in: dax_pmem(O) device_dax(O) nd_pmem(O) nd_blk(O) nd_btt(O) nfit_test(O-) nfit(O) libnvdimm(O) nfit_test_iomap(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc ppdev parport_pc pcspkr i2c_piix4 parport floppy i2c_core virtio_net virtio_balloon acpi_cpufreq sch_fq_codel ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk ata_piix libata virtio_pci crc32c_intel virtio_ring serio_raw virtio [ 117.818454] CPU: 4 PID: 2233 Comm: lt-libndctl Tainted: G W O 4.12.0-rc1+ #12 [ 117.820866] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [ 117.822487] task: ffff88041d685500 task.stack: ffffc90002b50000 [ 117.824178] RIP: 0010:__list_del_entry_valid+0x29/0xa0 [ 117.825647] RSP: 0018:ffffc90002b53d60 EFLAGS: 00010887 [ 117.827130] RAX: 6d726f6674616c70 RBX: 6d726f6674616c70 RCX: dead000000000200 [ 117.829142] RDX: 65745f7469666e3a RSI: ffffc90002b53d98 RDI: ffffc9001700e000 [ 117.831266] RBP: ffffc90002b53d60 R08: 00000000fffffffe R09: ffffc9001700e000 [ 117.833276] R10: 0000000000000005 R11: ffff88041cac41c0 R12: ffffc90002b53d98 [ 117.835285] R13: 6d726f6674616c70 R14: 0000000000000000 R15: ffff88041beefc08 [ 117.837285] FS: 00007f7d70082c80(0000) GS:ffff880433100000(0000) knlGS:0000000000000000 [ 117.839579] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 117.841230] CR2: 000055c0db522cc0 CR3: 0000000422156000 CR4: 00000000000006e0 [ 117.843262] Call Trace: [ 117.843985] release_nodes+0x76/0x260 [ 117.845062] devres_release_all+0x3c/0x50 [ 117.846225] device_release_driver_internal+0x159/0x200 [ 117.847748] device_release_driver+0x12/0x20 [ 117.849029] bus_remove_device+0xfd/0x170 [ 117.850192] device_del+0x1e8/0x330 [ 117.851284] platform_device_del+0x28/0x90 [ 117.852485] platform_device_unregister+0x12/0x30 [ 117.853846] nfit_test_exit+0x2a/0x93b [nfit_test] [ 117.855219] SyS_delete_module+0x171/0x250 [ 117.856403] entry_SYSCALL_64_fastpath+0x1a/0xa5 [ 117.857769] RIP: 0033:0x7f7d6f399487 [ 117.858809] RSP: 002b:00007ffc604502a8 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0 [ 117.860877] RAX: ffffffffffffffda RBX: 0000000002000000 RCX: 00007f7d6f399487 [ 117.862752] RDX: 00000000000000e8 RSI: 0000000000000800 RDI: 00000000020838f8 [ 117.864645] RBP: 00007ffc6044fc60 R08: 000000000208fb00 R09: 0000000000000037 [ 117.866495] R10: 000000000000006d R11: 0000000000000206 R12: 0000000000404e60 [ 117.868359] R13: 00007ffc60450440 R14: 0000000000000000 R15: 0000000000000000 [ 117.870242] Code: 00 00 55 48 8b 07 48 b9 00 01 00 00 00 00 ad de 48 8b 57 08 48 89 e5 48 39 c8 74 27 48 b9 00 02 00 00 00 00 ad de 48 39 ca 74 60 <48> 8b 12 48 39 d7 75 41 48 8b 50 08 48 39 d7 75 21 b8 01 00 00 [ 117.875198] RIP: __list_del_entry_valid+0x29/0xa0 RSP: ffffc90002b53d60 ---- Yasunori Goto _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Panic when make check for ndctl 2017-06-06 5:51 ` Yasunori Goto @ 2017-06-06 13:32 ` Linda Knippers 2017-06-06 16:42 ` Dan Williams 1 sibling, 0 replies; 14+ messages in thread From: Linda Knippers @ 2017-06-06 13:32 UTC (permalink / raw) To: Yasunori Goto, Dan Williams; +Cc: Yasunori Goto, linux-nvdimm@lists.01.org On 06/06/2017 01:51 AM, Yasunori Goto wrote: > Hi, > >>> No, you're not doing anything wrong. This is a known issue that I am >>> investigating. The reports of this usually come from distributions >>> with older module loading infrastructure. If you can try with a recent >>> Fedora that would mimic the environment I normally use to test, >> >> Oh, I see. Certainly, I used CentOS7. >> Ok, I'll use Fedora environment. > > Hmmm, though I made Fedora 25 environment, this panic still occurs... > I'll attach syslog and .config again. It's not you or your config. Your panic looks much like the one I reported in April. https://lists.01.org/pipermail/linux-nvdimm/2017-April/009406.html In my case I didn't even need to run the tests to get the panic. The panic is related to loading and unloading the nfit_test module, which is something the tests do. -- ljk > > > [ 105.119978] nfit_test_iomap: loading out-of-tree module taints kernel. > [ 105.306302] nfit_test nfit_test.0: found a zero length table '0' parsing nfit > [ 106.573244] random: crng init done > [ 107.339579] nfit_test nfit_test.0: failed to evaluate _FIT > [ 107.358018] nfit_test nfit_test.1: nmem4 flags: save_fail restore_fail flush_fail not_armed > [ 107.360586] nfit_test nfit_test.1: nmem5 flags: map_fail > [ 107.480603] pmem4: detected capacity change from 0 to 18874368 > [ 107.510795] pmem4: detected capacity change from 0 to 18874368 > [ 107.539968] pmem4: detected capacity change from 0 to 18874368 > [ 107.630683] pmem5: detected capacity change from 0 to 20971520 > [ 107.662595] pmem5: detected capacity change from 0 to 20971520 > [ 107.697813] pmem5: detected capacity change from 0 to 20971520 > [ 107.738166] ndblk0.0: detected capacity change from 0 to 7340032 > [ 107.768583] ndblk0.0: detected capacity change from 0 to 7340032 > [ 107.797455] ndblk0.0: detected capacity change from 0 to 6524416 > [ 107.823556] ndblk0.0: detected capacity change from 0 to 6524416 > [ 107.853451] ndblk0.0: detected capacity change from 0 to 6524416 > [ 107.883705] ndblk0.0: detected capacity change from 0 to 6524416 > [ 107.913563] ndblk0.0: detected capacity change from 0 to 7340032 > [ 107.948093] ndblk0.0: detected capacity change from 0 to 7340032 > [ 107.982461] ndblk0.0: detected capacity change from 0 to 7225344 > [ 108.015363] ndblk0.0: detected capacity change from 0 to 7225344 > [ 108.046480] ndblk0.0: detected capacity change from 0 to 7225344 > [ 108.071713] ndblk0.0: detected capacity change from 0 to 7225344 > [ 108.098458] ndblk0.0: detected capacity change from 0 to 7114752 > [ 108.127799] ndblk0.0: detected capacity change from 0 to 7114752 > [ 108.133122] ndblk0.1: detected capacity change from 0 to 11534336 > [ 108.162370] ndblk0.1: detected capacity change from 0 to 11534336 > [ 108.192647] ndblk0.1: detected capacity change from 0 to 10252288 > [ 108.218613] ndblk0.1: detected capacity change from 0 to 10252288 > [ 108.248650] ndblk0.1: detected capacity change from 0 to 10252288 > [ 108.276487] ndblk0.1: detected capacity change from 0 to 10252288 > [ 108.307132] ndblk0.1: detected capacity change from 0 to 11534336 > [ 108.336644] ndblk0.1: detected capacity change from 0 to 11534336 > [ 108.366668] ndblk0.1: detected capacity change from 0 to 11354112 > [ 108.399347] ndblk0.1: detected capacity change from 0 to 11354112 > [ 108.428403] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only > [ 108.428679] pmem7: detected capacity change from 0 to 4194304 > [ 108.439346] pmem6: detected capacity change from 0 to 33554432 > [ 108.448567] ndblk0.1: detected capacity change from 0 to 11354112 > [ 108.475539] ndblk0.1: detected capacity change from 0 to 11354112 > [ 108.501562] ndblk0.1: detected capacity change from 0 to 11182080 > [ 108.529544] ndblk0.1: detected capacity change from 0 to 11182080 > [ 108.580854] ndblk0.1: detected capacity change from 0 to 7340032 > [ 108.580861] ndblk0.0: detected capacity change from 0 to 11534336 > [ 108.619809] ndblk0.1: detected capacity change from 0 to 7340032 > [ 108.644717] ndblk0.0: detected capacity change from 0 to 11534336 > [ 108.706521] ndblk1.0: detected capacity change from 0 to 7340032 > [ 108.735577] ndblk1.0: detected capacity change from 0 to 7340032 > [ 108.763659] ndblk1.0: detected capacity change from 0 to 6524416 > [ 108.793613] ndblk1.0: detected capacity change from 0 to 6524416 > [ 108.822612] ndblk1.0: detected capacity change from 0 to 6524416 > [ 108.852674] ndblk1.0: detected capacity change from 0 to 6524416 > [ 108.881684] ndblk1.0: detected capacity change from 0 to 7340032 > [ 108.911788] ndblk1.0: detected capacity change from 0 to 7340032 > [ 108.941352] ndblk1.0: detected capacity change from 0 to 7225344 > [ 108.970375] ndblk1.0: detected capacity change from 0 to 7225344 > [ 108.998504] ndblk1.0: detected capacity change from 0 to 7225344 > [ 109.031416] ndblk1.0: detected capacity change from 0 to 7225344 > [ 109.062783] ndblk1.0: detected capacity change from 0 to 7114752 > [ 109.088454] ndblk1.0: detected capacity change from 0 to 7114752 > [ 109.093521] ndblk1.1: detected capacity change from 0 to 11534336 > [ 109.127332] ndblk1.1: detected capacity change from 0 to 11534336 > [ 109.157533] ndblk1.1: detected capacity change from 0 to 10252288 > [ 109.183291] ndblk1.1: detected capacity change from 0 to 10252288 > [ 109.212336] ndblk1.1: detected capacity change from 0 to 10252288 > [ 109.238434] ndblk1.1: detected capacity change from 0 to 10252288 > [ 109.271471] ndblk1.1: detected capacity change from 0 to 11534336 > [ 109.297386] ndblk1.1: detected capacity change from 0 to 11534336 > [ 109.328372] ndblk1.1: detected capacity change from 0 to 11354112 > [ 109.361400] ndblk1.1: detected capacity change from 0 to 11354112 > [ 109.392430] ndblk1.1: detected capacity change from 0 to 11354112 > [ 109.428406] ndblk1.1: detected capacity change from 0 to 11354112 > [ 109.454491] ndblk1.1: detected capacity change from 0 to 11182080 > [ 109.487353] ndblk1.1: detected capacity change from 0 to 11182080 > [ 109.537879] ndblk1.1: detected capacity change from 0 to 7340032 > [ 109.537967] ndblk1.0: detected capacity change from 0 to 11534336 > [ 109.565350] ndblk1.1: detected capacity change from 0 to 7340032 > [ 109.598463] ndblk1.0: detected capacity change from 0 to 11534336 > [ 109.660689] ndblk2.0: detected capacity change from 0 to 28311552 > [ 109.690525] ndblk2.0: detected capacity change from 0 to 28311552 > [ 109.716581] ndblk2.0: detected capacity change from 0 to 25165824 > [ 109.745327] ndblk2.0: detected capacity change from 0 to 25165824 > [ 109.770396] ndblk2.0: detected capacity change from 0 to 25165824 > [ 109.803360] ndblk2.0: detected capacity change from 0 to 25165824 > [ 109.832330] ndblk2.0: detected capacity change from 0 to 28311552 > [ 109.858490] ndblk2.0: detected capacity change from 0 to 28311552 > [ 109.888490] ndblk2.0: detected capacity change from 0 to 27873280 > [ 109.917227] ndblk2.0: detected capacity change from 0 to 27873280 > [ 109.945591] ndblk2.0: detected capacity change from 0 to 27873280 > [ 109.974347] ndblk2.0: detected capacity change from 0 to 27873280 > [ 110.003344] ndblk2.0: detected capacity change from 0 to 27451392 > [ 110.032411] ndblk2.0: detected capacity change from 0 to 27451392 > [ 110.061529] ndblk2.0: detected capacity change from 0 to 28311552 > [ 110.095333] ndblk2.0: detected capacity change from 0 to 28311552 > [ 110.129210] ndblk3.0: detected capacity change from 0 to 28311552 > [ 110.156337] ndblk3.0: detected capacity change from 0 to 28311552 > [ 110.186749] ndblk3.0: detected capacity change from 0 to 25165824 > [ 110.216575] ndblk3.0: detected capacity change from 0 to 25165824 > [ 110.246734] ndblk3.0: detected capacity change from 0 to 25165824 > [ 110.274606] ndblk3.0: detected capacity change from 0 to 25165824 > [ 110.303706] ndblk3.0: detected capacity change from 0 to 28311552 > [ 110.332341] ndblk3.0: detected capacity change from 0 to 28311552 > [ 110.362347] ndblk3.0: detected capacity change from 0 to 27873280 > [ 110.391318] ndblk3.0: detected capacity change from 0 to 27873280 > [ 110.419590] ndblk3.0: detected capacity change from 0 to 27873280 > [ 110.449602] ndblk3.0: detected capacity change from 0 to 27873280 > [ 110.475609] ndblk3.0: detected capacity change from 0 to 27451392 > [ 110.509744] ndblk3.0: detected capacity change from 0 to 27451392 > [ 110.539967] ndblk3.0: detected capacity change from 0 to 28311552 > [ 110.568702] ndblk3.0: detected capacity change from 0 to 28311552 > [ 110.637689] pmem4: detected capacity change from 0 to 18866176 > [ 110.669552] pmem4: detected capacity change from 0 to 14700544 > [ 110.700010] pmem4: detected capacity change from 0 to 14700544 > [ 110.736370] pmem4: detected capacity change from 0 to 14700544 > [ 110.767042] pmem4: detected capacity change from 0 to 18874368 > [ 110.798402] pmem4: detected capacity change from 0 to 18874368 > [ 110.828564] pmem4: detected capacity change from 0 to 18874368 > [ 110.864839] pmem5: detected capacity change from 0 to 20963328 > [ 110.899520] pmem5: detected capacity change from 0 to 16801792 > [ 110.931500] pmem5: detected capacity change from 0 to 16801792 > [ 110.965372] pmem5: detected capacity change from 0 to 16801792 > [ 110.996902] pmem5: detected capacity change from 0 to 20971520 > [ 111.028319] pmem5: detected capacity change from 0 to 20971520 > [ 111.061723] pmem5: detected capacity change from 0 to 20971520 > [ 111.096534] ndblk0.0: detected capacity change from 0 to 7340032 > [ 111.126313] ndblk0.0: detected capacity change from 0 to 7340032 > [ 111.155570] ndblk0.0: detected capacity change from 0 to 6524416 > [ 111.188508] ndblk0.0: detected capacity change from 0 to 6524416 > [ 111.218726] ndblk0.0: detected capacity change from 0 to 6524416 > [ 111.248807] ndblk0.0: detected capacity change from 0 to 6524416 > [ 111.279621] ndblk0.0: detected capacity change from 0 to 7340032 > [ 111.309380] ndblk0.0: detected capacity change from 0 to 7340032 > [ 111.343599] ndblk0.0: detected capacity change from 0 to 7225344 > [ 111.371350] ndblk0.0: detected capacity change from 0 to 7225344 > [ 111.396568] ndblk0.0: detected capacity change from 0 to 7225344 > [ 111.422790] ndblk0.0: detected capacity change from 0 to 7225344 > [ 111.449014] ndblk0.0: detected capacity change from 0 to 7114752 > [ 111.479696] ndblk0.0: detected capacity change from 0 to 7114752 > [ 111.484436] ndblk0.1: detected capacity change from 0 to 11534336 > [ 111.511601] ndblk0.1: detected capacity change from 0 to 11534336 > [ 111.537681] ndblk0.1: detected capacity change from 0 to 10252288 > [ 111.562396] ndblk0.1: detected capacity change from 0 to 10252288 > [ 111.589685] ndblk0.1: detected capacity change from 0 to 10252288 > [ 111.619815] ndblk0.1: detected capacity change from 0 to 10252288 > [ 111.646740] ndblk0.1: detected capacity change from 0 to 11534336 > [ 111.676715] ndblk0.1: detected capacity change from 0 to 11534336 > [ 111.702806] ndblk0.1: detected capacity change from 0 to 11354112 > [ 111.729349] ndblk0.1: detected capacity change from 0 to 11354112 > [ 111.756575] ndblk0.1: detected capacity change from 0 to 11354112 > [ 111.787806] ndblk0.1: detected capacity change from 0 to 11354112 > [ 111.817861] ndblk0.1: detected capacity change from 0 to 11182080 > [ 111.846802] ndblk0.1: detected capacity change from 0 to 11182080 > [ 111.898048] ndblk0.1: detected capacity change from 0 to 7340032 > [ 111.898069] ndblk0.0: detected capacity change from 0 to 11534336 > [ 111.926589] ndblk0.1: detected capacity change from 0 to 7340032 > [ 111.954610] ndblk0.0: detected capacity change from 0 to 11534336 > [ 112.013684] ndblk1.0: detected capacity change from 0 to 7340032 > [ 112.041567] ndblk1.0: detected capacity change from 0 to 7340032 > [ 112.067506] ndblk1.0: detected capacity change from 0 to 6524416 > [ 112.099657] ndblk1.0: detected capacity change from 0 to 6524416 > [ 112.128561] ndblk1.0: detected capacity change from 0 to 6524416 > [ 112.160662] ndblk1.0: detected capacity change from 0 to 6524416 > [ 112.190756] ndblk1.0: detected capacity change from 0 to 7340032 > [ 112.220656] ndblk1.0: detected capacity change from 0 to 7340032 > [ 112.253664] ndblk1.0: detected capacity change from 0 to 7225344 > [ 112.283519] ndblk1.0: detected capacity change from 0 to 7225344 > [ 112.309701] ndblk1.0: detected capacity change from 0 to 7225344 > [ 112.342567] ndblk1.0: detected capacity change from 0 to 7225344 > [ 112.372528] ndblk1.0: detected capacity change from 0 to 7114752 > [ 112.405435] ndblk1.0: detected capacity change from 0 to 7114752 > [ 112.409926] ndblk1.1: detected capacity change from 0 to 11534336 > [ 112.438331] ndblk1.1: detected capacity change from 0 to 11534336 > [ 112.466617] ndblk1.1: detected capacity change from 0 to 10252288 > [ 112.497657] ndblk1.1: detected capacity change from 0 to 10252288 > [ 112.528683] ndblk1.1: detected capacity change from 0 to 10252288 > [ 112.562610] ndblk1.1: detected capacity change from 0 to 10252288 > [ 112.595812] ndblk1.1: detected capacity change from 0 to 11534336 > [ 112.625853] ndblk1.1: detected capacity change from 0 to 11534336 > [ 112.655840] ndblk1.1: detected capacity change from 0 to 11354112 > [ 112.685837] ndblk1.1: detected capacity change from 0 to 11354112 > [ 112.715739] ndblk1.1: detected capacity change from 0 to 11354112 > [ 112.743641] ndblk1.1: detected capacity change from 0 to 11354112 > [ 112.775647] ndblk1.1: detected capacity change from 0 to 11182080 > [ 112.803627] ndblk1.1: detected capacity change from 0 to 11182080 > [ 112.856962] ndblk1.0: detected capacity change from 0 to 11534336 > [ 112.871023] ndblk1.1: detected capacity change from 0 to 7340032 > [ 112.898543] ndblk1.1: detected capacity change from 0 to 7340032 > [ 112.928733] ndblk1.0: detected capacity change from 0 to 11534336 > [ 112.988305] ndblk2.0: detected capacity change from 0 to 28311552 > [ 113.015628] ndblk2.0: detected capacity change from 0 to 28311552 > [ 113.047834] ndblk2.0: detected capacity change from 0 to 25165824 > [ 113.077795] ndblk2.0: detected capacity change from 0 to 25165824 > [ 113.119912] ndblk2.0: detected capacity change from 0 to 25165824 > [ 113.148630] ndblk2.0: detected capacity change from 0 to 25165824 > [ 113.177691] ndblk2.0: detected capacity change from 0 to 28311552 > [ 113.206434] ndblk2.0: detected capacity change from 0 to 28311552 > [ 113.235665] ndblk2.0: detected capacity change from 0 to 27873280 > [ 113.265569] ndblk2.0: detected capacity change from 0 to 27873280 > [ 113.293662] ndblk2.0: detected capacity change from 0 to 27873280 > [ 113.322613] ndblk2.0: detected capacity change from 0 to 27873280 > [ 113.351621] ndblk2.0: detected capacity change from 0 to 27451392 > [ 113.380707] ndblk2.0: detected capacity change from 0 to 27451392 > [ 113.414204] ndblk2.0: detected capacity change from 0 to 28311552 > [ 113.445534] ndblk2.0: detected capacity change from 0 to 28311552 > [ 113.479150] ndblk3.0: detected capacity change from 0 to 28311552 > [ 113.508569] ndblk3.0: detected capacity change from 0 to 28311552 > [ 113.539519] ndblk3.0: detected capacity change from 0 to 25165824 > [ 113.571755] ndblk3.0: detected capacity change from 0 to 25165824 > [ 113.601544] ndblk3.0: detected capacity change from 0 to 25165824 > [ 113.631450] ndblk3.0: detected capacity change from 0 to 25165824 > [ 113.661343] ndblk3.0: detected capacity change from 0 to 28311552 > [ 113.690698] ndblk3.0: detected capacity change from 0 to 28311552 > [ 113.720616] ndblk3.0: detected capacity change from 0 to 27873280 > [ 113.746796] ndblk3.0: detected capacity change from 0 to 27873280 > [ 113.775765] ndblk3.0: detected capacity change from 0 to 27873280 > [ 113.804548] ndblk3.0: detected capacity change from 0 to 27873280 > [ 113.834762] ndblk3.0: detected capacity change from 0 to 27451392 > [ 113.865715] ndblk3.0: detected capacity change from 0 to 27451392 > [ 113.900500] ndblk3.0: detected capacity change from 0 to 28311552 > [ 113.929600] ndblk3.0: detected capacity change from 0 to 28311552 > [ 113.980479] nd_pmem btt4.0: No existing arenas > [ 113.983934] pmem4s: detected capacity change from 0 to 18564096 > [ 114.020170] pmem4s: detected capacity change from 0 to 18564096 > [ 114.053960] pmem4s: detected capacity change from 0 to 18564096 > [ 114.088905] pmem4s: detected capacity change from 0 to 148512768 > [ 114.123022] pmem4s: detected capacity change from 0 to 148512768 > [ 114.156559] pmem4s: detected capacity change from 0 to 148512768 > [ 114.192973] pmem4s: detected capacity change from 0 to 148512768 > [ 114.227204] pmem4s: detected capacity change from 0 to 18564096 > [ 114.262795] pmem4s: detected capacity change from 0 to 18564096 > [ 114.296797] pmem4: detected capacity change from 0 to 18874368 > [ 114.332711] pmem4: detected capacity change from 0 to 18874368 > [ 114.362463] pmem4: detected capacity change from 0 to 18874368 > [ 114.400217] nd_pmem btt5.0: No existing arenas > [ 114.404187] pmem5s: detected capacity change from 0 to 20644864 > [ 114.442874] pmem5s: detected capacity change from 0 to 20644864 > [ 114.480026] pmem5s: detected capacity change from 0 to 20644864 > [ 114.515993] pmem5s: detected capacity change from 0 to 165158912 > [ 114.552262] pmem5s: detected capacity change from 0 to 165158912 > [ 114.592288] pmem5s: detected capacity change from 0 to 165158912 > [ 114.628456] pmem5s: detected capacity change from 0 to 165158912 > [ 114.667815] pmem5s: detected capacity change from 0 to 20644864 > [ 114.698337] pmem5s: detected capacity change from 0 to 20644864 > [ 114.727759] pmem5: detected capacity change from 0 to 20971520 > [ 114.759815] pmem5: detected capacity change from 0 to 20971520 > [ 114.792048] pmem5: detected capacity change from 0 to 20971520 > [ 114.827093] ndblk0.0: detected capacity change from 0 to 7340032 > [ 114.855851] ndblk0.0: detected capacity change from 0 to 7340032 > [ 114.882503] ndblk0.0: detected capacity change from 0 to 6524416 > [ 114.912396] ndblk0.0: detected capacity change from 0 to 6524416 > [ 114.942545] ndblk0.0: detected capacity change from 0 to 6524416 > [ 114.972699] ndblk0.0: detected capacity change from 0 to 6524416 > [ 115.003739] ndblk0.0: detected capacity change from 0 to 7340032 > [ 115.032844] ndblk0.0: detected capacity change from 0 to 7340032 > [ 115.058561] ndblk0.0: detected capacity change from 0 to 7225344 > [ 115.084344] ndblk0.0: detected capacity change from 0 to 7225344 > [ 115.113604] ndblk0.0: detected capacity change from 0 to 7225344 > [ 115.144313] ndblk0.0: detected capacity change from 0 to 7225344 > [ 115.173577] ndblk0.0: detected capacity change from 0 to 7114752 > [ 115.201318] ndblk0.0: detected capacity change from 0 to 7114752 > [ 115.206551] ndblk0.1: detected capacity change from 0 to 11534336 > [ 115.233525] ndblk0.1: detected capacity change from 0 to 11534336 > [ 115.261761] ndblk0.1: detected capacity change from 0 to 10252288 > [ 115.290866] ndblk0.1: detected capacity change from 0 to 10252288 > [ 115.319724] ndblk0.1: detected capacity change from 0 to 10252288 > [ 115.348726] ndblk0.1: detected capacity change from 0 to 10252288 > [ 115.375692] ndblk0.1: detected capacity change from 0 to 11534336 > [ 115.402375] ndblk0.1: detected capacity change from 0 to 11534336 > [ 115.428721] ndblk0.1: detected capacity change from 0 to 11354112 > [ 115.457458] ndblk0.1: detected capacity change from 0 to 11354112 > [ 115.487723] ndblk0.1: detected capacity change from 0 to 11354112 > [ 115.517554] ndblk0.1: detected capacity change from 0 to 11354112 > [ 115.547857] ndblk0.1: detected capacity change from 0 to 11182080 > [ 115.576733] ndblk0.1: detected capacity change from 0 to 11182080 > [ 115.627671] ndblk0.0: detected capacity change from 0 to 11534336 > [ 115.627688] ndblk0.1: detected capacity change from 0 to 7340032 > [ 115.659725] ndblk0.1: detected capacity change from 0 to 7340032 > [ 115.689442] ndblk0.0: detected capacity change from 0 to 11534336 > [ 115.745343] ndblk1.0: detected capacity change from 0 to 7340032 > [ 115.775727] ndblk1.0: detected capacity change from 0 to 7340032 > [ 115.805728] ndblk1.0: detected capacity change from 0 to 6524416 > [ 115.831536] ndblk1.0: detected capacity change from 0 to 6524416 > [ 115.861891] ndblk1.0: detected capacity change from 0 to 6524416 > [ 115.891832] ndblk1.0: detected capacity change from 0 to 6524416 > [ 115.921839] ndblk1.0: detected capacity change from 0 to 7340032 > [ 115.951448] ndblk1.0: detected capacity change from 0 to 7340032 > [ 115.977690] ndblk1.0: detected capacity change from 0 to 7225344 > [ 116.010474] ndblk1.0: detected capacity change from 0 to 7225344 > [ 116.039340] ndblk1.0: detected capacity change from 0 to 7225344 > [ 116.065740] ndblk1.0: detected capacity change from 0 to 7225344 > [ 116.095687] ndblk1.0: detected capacity change from 0 to 7114752 > [ 116.124474] ndblk1.0: detected capacity change from 0 to 7114752 > [ 116.130332] ndblk1.1: detected capacity change from 0 to 11534336 > [ 116.156512] ndblk1.1: detected capacity change from 0 to 11534336 > [ 116.186542] ndblk1.1: detected capacity change from 0 to 10252288 > [ 116.223188] ndblk1.1: detected capacity change from 0 to 10252288 > [ 116.250695] ndblk1.1: detected capacity change from 0 to 10252288 > [ 116.280524] ndblk1.1: detected capacity change from 0 to 10252288 > [ 116.306568] ndblk1.1: detected capacity change from 0 to 11534336 > [ 116.335389] ndblk1.1: detected capacity change from 0 to 11534336 > [ 116.365484] ndblk1.1: detected capacity change from 0 to 11354112 > [ 116.390303] ndblk1.1: detected capacity change from 0 to 11354112 > [ 116.418332] ndblk1.1: detected capacity change from 0 to 11354112 > [ 116.444226] ndblk1.1: detected capacity change from 0 to 11354112 > [ 116.469322] ndblk1.1: detected capacity change from 0 to 11182080 > [ 116.500557] ndblk1.1: detected capacity change from 0 to 11182080 > [ 116.554723] ndblk1.0: detected capacity change from 0 to 11534336 > [ 116.554895] ndblk1.1: detected capacity change from 0 to 7340032 > [ 116.583641] ndblk1.1: detected capacity change from 0 to 7340032 > [ 116.613324] ndblk1.0: detected capacity change from 0 to 11534336 > [ 116.671228] nd_blk btt2.0: No existing arenas > [ 116.675514] ndblk2.0s: detected capacity change from 0 to 27928064 > [ 116.711302] ndblk2.0s: detected capacity change from 0 to 27928064 > [ 116.750457] ndblk2.0s: detected capacity change from 0 to 27928064 > [ 116.786165] ndblk2.0s: detected capacity change from 0 to 223424512 > [ 116.821733] ndblk2.0s: detected capacity change from 0 to 223424512 > [ 116.857652] ndblk2.0s: detected capacity change from 0 to 223424512 > [ 116.890435] ndblk2.0s: detected capacity change from 0 to 223424512 > [ 116.925862] ndblk2.0s: detected capacity change from 0 to 27928064 > [ 116.967171] ndblk2.0s: detected capacity change from 0 to 27928064 > [ 117.000647] ndblk2.0: detected capacity change from 0 to 28311552 > [ 117.026754] ndblk2.0: detected capacity change from 0 to 28311552 > [ 117.056780] ndblk2.0: detected capacity change from 0 to 28311552 > [ 117.091005] nd_blk btt3.0: No existing arenas > [ 117.094760] ndblk3.0s: detected capacity change from 0 to 27928064 > [ 117.129367] ndblk3.0s: detected capacity change from 0 to 27928064 > [ 117.165070] ndblk3.0s: detected capacity change from 0 to 27928064 > [ 117.200090] ndblk3.0s: detected capacity change from 0 to 223424512 > [ 117.235165] ndblk3.0s: detected capacity change from 0 to 223424512 > [ 117.274015] ndblk3.0s: detected capacity change from 0 to 223424512 > [ 117.316141] ndblk3.0s: detected capacity change from 0 to 223424512 > [ 117.350107] ndblk3.0s: detected capacity change from 0 to 27928064 > [ 117.380054] ndblk3.0s: detected capacity change from 0 to 27928064 > [ 117.414741] ndblk3.0: detected capacity change from 0 to 28311552 > [ 117.442741] ndblk3.0: detected capacity change from 0 to 28311552 > [ 117.471744] ndblk3.0: detected capacity change from 0 to 28311552 > [ 117.529686] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only > [ 117.535374] pmem6: detected capacity change from 0 to 33554432 > [ 117.541192] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only > [ 117.543681] pmem6: detected capacity change from 33554432 to 0 > [ 117.569779] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only > [ 117.573036] pmem6: detected capacity change from 0 to 33554432 > [ 117.604734] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only > [ 117.607796] pmem6: detected capacity change from 0 to 33554432 > [ 117.613821] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only > [ 117.616799] pmem6: detected capacity change from 33554432 to 0 > [ 117.641595] nd_pmem namespace6.0: region6 read-only, marking pmem6 read-only > [ 117.644598] pmem6: detected capacity change from 0 to 33554432 > [ 117.719438] list_del corruption. next->prev should be ffff88041cac41e0, but was 65745f7469666e3a > [ 117.722832] ------------[ cut here ]------------ > [ 117.724477] WARNING: CPU: 4 PID: 2233 at lib/list_debug.c:56 __list_del_entry_valid+0x6c/0xa0 > [ 117.727507] Modules linked in: dax_pmem(O) device_dax(O) nd_pmem(O) nd_blk(O) nd_btt(O) nfit_test(O-) nfit(O) libnvdimm(O) nfit_test_iomap(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc ppdev parport_pc pcspkr i2c_piix4 parport floppy i2c_core virtio_net virtio_balloon acpi_cpufreq sch_fq_codel ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk ata_piix libata virtio_pci crc32c_intel virtio_ring serio_raw virtio > [ 117.741905] CPU: 4 PID: 2233 Comm: lt-libndctl Tainted: G O 4.12.0-rc1+ #12 > [ 117.744461] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 > [ 117.746310] task: ffff88041d685500 task.stack: ffffc90002b50000 > [ 117.748198] RIP: 0010:__list_del_entry_valid+0x6c/0xa0 > [ 117.751084] RSP: 0018:ffffc90002b53d60 EFLAGS: 00010046 > [ 117.753142] RAX: 0000000000000054 RBX: ffffc9001700e000 RCX: 0000000000000000 > [ 117.755955] RDX: 0000000000000000 RSI: ffff88043310e0a8 RDI: ffff88043310e0a8 > [ 117.758745] RBP: ffffc90002b53d60 R08: 00000000fffffffe R09: 0000000000000382 > [ 117.760801] R10: 0000000000000005 R11: 0000000000000381 R12: ffffc90002b53d98 > [ 117.762949] R13: ffffc9001700e000 R14: 0000000000000000 R15: ffff88041beefc08 > [ 117.765000] FS: 00007f7d70082c80(0000) GS:ffff880433100000(0000) knlGS:0000000000000000 > [ 117.767300] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 117.768962] CR2: 000055c0db522cc0 CR3: 0000000422156000 CR4: 00000000000006e0 > [ 117.770975] Call Trace: > [ 117.771716] release_nodes+0x76/0x260 > [ 117.772797] devres_release_all+0x3c/0x50 > [ 117.773981] device_release_driver_internal+0x159/0x200 > [ 117.775494] device_release_driver+0x12/0x20 > [ 117.776763] bus_remove_device+0xfd/0x170 > [ 117.777924] device_del+0x1e8/0x330 > [ 117.778943] platform_device_del+0x28/0x90 > [ 117.780129] platform_device_unregister+0x12/0x30 > [ 117.781483] nfit_test_exit+0x2a/0x93b [nfit_test] > [ 117.782858] SyS_delete_module+0x171/0x250 > [ 117.784044] entry_SYSCALL_64_fastpath+0x1a/0xa5 > [ 117.785346] RIP: 0033:0x7f7d6f399487 > [ 117.786377] RSP: 002b:00007ffc604502a8 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0 > [ 117.788503] RAX: ffffffffffffffda RBX: 0000000002000000 RCX: 00007f7d6f399487 > [ 117.790525] RDX: 00000000000000e8 RSI: 0000000000000800 RDI: 00000000020838f8 > [ 117.792635] RBP: 00007ffc6044fc60 R08: 000000000208fb00 R09: 0000000000000037 > [ 117.794641] R10: 000000000000006d R11: 0000000000000206 R12: 0000000000404e60 > [ 117.796656] R13: 00007ffc60450440 R14: 0000000000000000 R15: 0000000000000000 > [ 117.798699] Code: 48 89 c2 48 89 fe 31 c0 48 c7 c7 38 3f a7 81 e8 1e 34 e0 ff 0f ff 31 c0 5d c3 48 89 fe 31 c0 48 c7 c7 e8 3f a7 81 e8 07 34 e0 ff <0f> ff 31 c0 5d c3 48 89 fe 31 c0 48 c7 c7 a8 3f a7 81 e8 f0 33 > [ 117.803715] ---[ end trace 06a87dde16f89e24 ]--- > [ 117.804948] general protection fault: 0000 [#1] SMP > [ 117.806233] Modules linked in: dax_pmem(O) device_dax(O) nd_pmem(O) nd_blk(O) nd_btt(O) nfit_test(O-) nfit(O) libnvdimm(O) nfit_test_iomap(O) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc ppdev parport_pc pcspkr i2c_piix4 parport floppy i2c_core virtio_net virtio_balloon acpi_cpufreq sch_fq_codel ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk ata_piix libata virtio_pci crc32c_intel virtio_ring serio_raw virtio > [ 117.818454] CPU: 4 PID: 2233 Comm: lt-libndctl Tainted: G W O 4.12.0-rc1+ #12 > [ 117.820866] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 > [ 117.822487] task: ffff88041d685500 task.stack: ffffc90002b50000 > [ 117.824178] RIP: 0010:__list_del_entry_valid+0x29/0xa0 > [ 117.825647] RSP: 0018:ffffc90002b53d60 EFLAGS: 00010887 > [ 117.827130] RAX: 6d726f6674616c70 RBX: 6d726f6674616c70 RCX: dead000000000200 > [ 117.829142] RDX: 65745f7469666e3a RSI: ffffc90002b53d98 RDI: ffffc9001700e000 > [ 117.831266] RBP: ffffc90002b53d60 R08: 00000000fffffffe R09: ffffc9001700e000 > [ 117.833276] R10: 0000000000000005 R11: ffff88041cac41c0 R12: ffffc90002b53d98 > [ 117.835285] R13: 6d726f6674616c70 R14: 0000000000000000 R15: ffff88041beefc08 > [ 117.837285] FS: 00007f7d70082c80(0000) GS:ffff880433100000(0000) knlGS:0000000000000000 > [ 117.839579] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 117.841230] CR2: 000055c0db522cc0 CR3: 0000000422156000 CR4: 00000000000006e0 > [ 117.843262] Call Trace: > [ 117.843985] release_nodes+0x76/0x260 > [ 117.845062] devres_release_all+0x3c/0x50 > [ 117.846225] device_release_driver_internal+0x159/0x200 > [ 117.847748] device_release_driver+0x12/0x20 > [ 117.849029] bus_remove_device+0xfd/0x170 > [ 117.850192] device_del+0x1e8/0x330 > [ 117.851284] platform_device_del+0x28/0x90 > [ 117.852485] platform_device_unregister+0x12/0x30 > [ 117.853846] nfit_test_exit+0x2a/0x93b [nfit_test] > [ 117.855219] SyS_delete_module+0x171/0x250 > [ 117.856403] entry_SYSCALL_64_fastpath+0x1a/0xa5 > [ 117.857769] RIP: 0033:0x7f7d6f399487 > [ 117.858809] RSP: 002b:00007ffc604502a8 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0 > [ 117.860877] RAX: ffffffffffffffda RBX: 0000000002000000 RCX: 00007f7d6f399487 > [ 117.862752] RDX: 00000000000000e8 RSI: 0000000000000800 RDI: 00000000020838f8 > [ 117.864645] RBP: 00007ffc6044fc60 R08: 000000000208fb00 R09: 0000000000000037 > [ 117.866495] R10: 000000000000006d R11: 0000000000000206 R12: 0000000000404e60 > [ 117.868359] R13: 00007ffc60450440 R14: 0000000000000000 R15: 0000000000000000 > [ 117.870242] Code: 00 00 55 48 8b 07 48 b9 00 01 00 00 00 00 ad de 48 8b 57 08 48 89 e5 48 39 c8 74 27 48 b9 00 02 00 00 00 00 ad de 48 39 ca 74 60 <48> 8b 12 48 39 d7 75 41 48 8b 50 08 48 39 d7 75 21 b8 01 00 00 > [ 117.875198] RIP: __list_del_entry_valid+0x29/0xa0 RSP: ffffc90002b53d60 > > ---- > Yasunori Goto > > _______________________________________________ > Linux-nvdimm mailing list > Linux-nvdimm@lists.01.org > https://lists.01.org/mailman/listinfo/linux-nvdimm > _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Panic when make check for ndctl 2017-06-06 5:51 ` Yasunori Goto 2017-06-06 13:32 ` Linda Knippers @ 2017-06-06 16:42 ` Dan Williams 2017-06-07 0:33 ` Yasunori Goto 1 sibling, 1 reply; 14+ messages in thread From: Dan Williams @ 2017-06-06 16:42 UTC (permalink / raw) To: Yasunori Goto; +Cc: Yasunori Goto, linux-nvdimm@lists.01.org On Mon, Jun 5, 2017 at 10:51 PM, Yasunori Goto <y-goto@jp.fujitsu.com> wrote: > Hi, > >> > No, you're not doing anything wrong. This is a known issue that I am >> > investigating. The reports of this usually come from distributions >> > with older module loading infrastructure. If you can try with a recent >> > Fedora that would mimic the environment I normally use to test, >> >> Oh, I see. Certainly, I used CentOS7. >> Ok, I'll use Fedora environment. > > Hmmm, though I made Fedora 25 environment, this panic still occurs... > I'll attach syslog and .config again. > > [..] > [ 117.804948] general protection fault: 0000 [#1] SMP [..] > [ 117.820866] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [..] > [ 117.843262] Call Trace: > [ 117.843985] release_nodes+0x76/0x260 > [ 117.845062] devres_release_all+0x3c/0x50 > [ 117.846225] device_release_driver_internal+0x159/0x200 > [ 117.847748] device_release_driver+0x12/0x20 > [ 117.849029] bus_remove_device+0xfd/0x170 > [ 117.850192] device_del+0x1e8/0x330 > [ 117.851284] platform_device_del+0x28/0x90 > [ 117.852485] platform_device_unregister+0x12/0x30 > [ 117.853846] nfit_test_exit+0x2a/0x93b [nfit_test] > [ 117.855219] SyS_delete_module+0x171/0x250 > [ 117.856403] entry_SYSCALL_64_fastpath+0x1a/0xa5 Can you also attach the qemu-kvm command line you are using? ps aux | grep qemu _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Panic when make check for ndctl 2017-06-06 16:42 ` Dan Williams @ 2017-06-07 0:33 ` Yasunori Goto 2017-06-14 2:41 ` Yasunori Goto 0 siblings, 1 reply; 14+ messages in thread From: Yasunori Goto @ 2017-06-07 0:33 UTC (permalink / raw) To: Dan Williams; +Cc: linux-nvdimm@lists.01.org > > Hmmm, though I made Fedora 25 environment, this panic still occurs... > > I'll attach syslog and .config again. > > > > > [..] > > [ 117.804948] general protection fault: 0000 [#1] SMP > [..] > > [ 117.820866] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 > [..] > > [ 117.843262] Call Trace: > > [ 117.843985] release_nodes+0x76/0x260 > > [ 117.845062] devres_release_all+0x3c/0x50 > > [ 117.846225] device_release_driver_internal+0x159/0x200 > > [ 117.847748] device_release_driver+0x12/0x20 > > [ 117.849029] bus_remove_device+0xfd/0x170 > > [ 117.850192] device_del+0x1e8/0x330 > > [ 117.851284] platform_device_del+0x28/0x90 > > [ 117.852485] platform_device_unregister+0x12/0x30 > > [ 117.853846] nfit_test_exit+0x2a/0x93b [nfit_test] > > [ 117.855219] SyS_delete_module+0x171/0x250 > > [ 117.856403] entry_SYSCALL_64_fastpath+0x1a/0xa5 > > Can you also attach the qemu-kvm command line you are using? > > ps aux | grep qemu Sure! --- $ ps auxw |grep qemu |grep goto-fedora qemu 45082 2.1 16.9 17408108 5547508 ? Sl Jun06 30:37 /usr/libexec/qemu-kvm -name goto-fedora -S -machine pc-i440fx-rhel7.0.0,accel=kvm,usb=off -cpu Nehalem -m 16384 -realtime mlock=off -smp 12,sockets=12,cores=1,threads=1 -uuid b9802ae7-404e-453b-8998-24dfc81567f2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-26-goto-fedora/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive f ile=/var/lib/libvirt/images/goto-fedora.img,format=qcow2,if=none,id=drive-virtio-disk0 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,id=drive-ide0-0-0,readonly=on -device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:fa:94:a9,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-26-goto-fedora/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0,port=1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on ---- --- Yasunori Goto _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Panic when make check for ndctl 2017-06-07 0:33 ` Yasunori Goto @ 2017-06-14 2:41 ` Yasunori Goto 2017-06-14 3:05 ` Dan Williams 2017-06-14 16:06 ` Dan Williams 0 siblings, 2 replies; 14+ messages in thread From: Yasunori Goto @ 2017-06-14 2:41 UTC (permalink / raw) To: Dan Williams; +Cc: linux-nvdimm@lists.01.org Hi, Dan-san, Linda-san, I had chased the root cause of this panic problem, and maybe I found it. > > > Hmmm, though I made Fedora 25 environment, this panic still occurs... > > > I'll attach syslog and .config again. > > > > > > > > [..] > > > [ 117.804948] general protection fault: 0000 [#1] SMP > > [..] > > > [ 117.820866] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 > > [..] > > > [ 117.843262] Call Trace: > > > [ 117.843985] release_nodes+0x76/0x260 > > > [ 117.845062] devres_release_all+0x3c/0x50 > > > [ 117.846225] device_release_driver_internal+0x159/0x200 > > > [ 117.847748] device_release_driver+0x12/0x20 > > > [ 117.849029] bus_remove_device+0xfd/0x170 > > > [ 117.850192] device_del+0x1e8/0x330 > > > [ 117.851284] platform_device_del+0x28/0x90 > > > [ 117.852485] platform_device_unregister+0x12/0x30 > > > [ 117.853846] nfit_test_exit+0x2a/0x93b [nfit_test] > > > [ 117.855219] SyS_delete_module+0x171/0x250 > > > [ 117.856403] entry_SYSCALL_64_fastpath+0x1a/0xa5 > > > > Can you also attach the qemu-kvm command line you are using? > > > > ps aux | grep qemu The cause of this problem is the num_pm of nfit_test1 is wrong. Though 1 is specified for num_pm at nfit_test_init(), it must be 2. ---- static __init int nfit_test_init(void) { int rc, i; : : case 1: nfit_test->num_pm = 1; <---- !!! nfit_test->dcr_idx = NUM_DCR; ----- The num_pm affects size of devm_kcalloc() at nfit_test_probe(). ---- static int nfit_test_probe(struct platform_device *pdev) { if (nfit_test->num_pm) { int num = nfit_test->num_pm; <----!!! nfit_test->spa_set = devm_kcalloc(dev, num, sizeof(void *), GFP_KERNEL); <---!!!! nfit_test->spa_set_dma = devm_kcalloc(dev, num, sizeof(dma_addr_t), GFP_KERNEL); ----- However, spa_set[] array needs 2 elements at nfit_test1_alloc(). --- static int nfit_test1_alloc(struct nfit_test *t) { : t->spa_set[0] = test_alloc(t, SPA2_SIZE, &t->spa_set_dma[0]); <--- first element if (!t->spa_set[0]) return -ENOMEM; : t->spa_set[1] = test_alloc(t, SPA_VCD_SIZE, &t->spa_set_dma[1]); <---- The second element!!!! ----- This breaks other area, and the area is often the link list of devres. As a result, the panic occured on release_nodes(). I confirmed that this panic never occurred with the following patch. --- diff --git a/tools/testing/nvdimm/test/nfit.c b/tools/testing/nvdimm/test/nfit.c index c218717..548b6d4 100644 --- a/tools/testing/nvdimm/test/nfit.c +++ b/tools/testing/nvdimm/test/nfit.c @@ -1943,7 +1943,7 @@ static __init int nfit_test_init(void) nfit_test->setup = nfit_test0_setup; break; case 1: - nfit_test->num_pm = 1; + nfit_test->num_pm = 2; nfit_test->dcr_idx = NUM_DCR; nfit_test->num_dcr = 2; nfit_test->alloc = nfit_test1_alloc; ---- Thanks, Yasunori Goto _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: Panic when make check for ndctl 2017-06-14 2:41 ` Yasunori Goto @ 2017-06-14 3:05 ` Dan Williams 2017-06-14 15:31 ` Linda Knippers 2017-06-14 16:06 ` Dan Williams 1 sibling, 1 reply; 14+ messages in thread From: Dan Williams @ 2017-06-14 3:05 UTC (permalink / raw) To: Yasunori Goto; +Cc: linux-nvdimm@lists.01.org On Tue, Jun 13, 2017 at 7:41 PM, Yasunori Goto <y-goto@jp.fujitsu.com> wrote: > Hi, Dan-san, Linda-san, > > I had chased the root cause of this panic problem, and maybe I found it. > >> > > Hmmm, though I made Fedora 25 environment, this panic still occurs... >> > > I'll attach syslog and .config again. >> > > >> > > >> > [..] >> > > [ 117.804948] general protection fault: 0000 [#1] SMP >> > [..] >> > > [ 117.820866] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 >> > [..] >> > > [ 117.843262] Call Trace: >> > > [ 117.843985] release_nodes+0x76/0x260 >> > > [ 117.845062] devres_release_all+0x3c/0x50 >> > > [ 117.846225] device_release_driver_internal+0x159/0x200 >> > > [ 117.847748] device_release_driver+0x12/0x20 >> > > [ 117.849029] bus_remove_device+0xfd/0x170 >> > > [ 117.850192] device_del+0x1e8/0x330 >> > > [ 117.851284] platform_device_del+0x28/0x90 >> > > [ 117.852485] platform_device_unregister+0x12/0x30 >> > > [ 117.853846] nfit_test_exit+0x2a/0x93b [nfit_test] >> > > [ 117.855219] SyS_delete_module+0x171/0x250 >> > > [ 117.856403] entry_SYSCALL_64_fastpath+0x1a/0xa5 >> > >> > Can you also attach the qemu-kvm command line you are using? >> > >> > ps aux | grep qemu > > > The cause of this problem is the num_pm of nfit_test1 is wrong. > Though 1 is specified for num_pm at nfit_test_init(), it must be 2. > > ---- > static __init int nfit_test_init(void) > { > int rc, i; > : > : > case 1: > nfit_test->num_pm = 1; <---- !!! > nfit_test->dcr_idx = NUM_DCR; > ----- > > The num_pm affects size of devm_kcalloc() at nfit_test_probe(). > > ---- > static int nfit_test_probe(struct platform_device *pdev) > { > if (nfit_test->num_pm) { > int num = nfit_test->num_pm; <----!!! > > nfit_test->spa_set = devm_kcalloc(dev, num, sizeof(void *), > GFP_KERNEL); <---!!!! > nfit_test->spa_set_dma = devm_kcalloc(dev, num, > sizeof(dma_addr_t), GFP_KERNEL); > ----- > > However, spa_set[] array needs 2 elements at nfit_test1_alloc(). > > --- > static int nfit_test1_alloc(struct nfit_test *t) > { > : > t->spa_set[0] = test_alloc(t, SPA2_SIZE, &t->spa_set_dma[0]); <--- first element > if (!t->spa_set[0]) > return -ENOMEM; > : > > t->spa_set[1] = test_alloc(t, SPA_VCD_SIZE, &t->spa_set_dma[1]); <---- The second element!!!! > ----- > > This breaks other area, and the area is often the link list of devres. > As a result, the panic occured on release_nodes(). > > I confirmed that this panic never occurred with the following patch. Wow! > > --- > diff --git a/tools/testing/nvdimm/test/nfit.c b/tools/testing/nvdimm/test/nfit.c > index c218717..548b6d4 100644 > --- a/tools/testing/nvdimm/test/nfit.c > +++ b/tools/testing/nvdimm/test/nfit.c > @@ -1943,7 +1943,7 @@ static __init int nfit_test_init(void) > nfit_test->setup = nfit_test0_setup; > break; > case 1: > - nfit_test->num_pm = 1; > + nfit_test->num_pm = 2; > nfit_test->dcr_idx = NUM_DCR; > nfit_test->num_dcr = 2; > nfit_test->alloc = nfit_test1_alloc; This change looks correct to me. I'm going to try it out. ...but, "Wow!" again and a big "Thank You!". _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Panic when make check for ndctl 2017-06-14 3:05 ` Dan Williams @ 2017-06-14 15:31 ` Linda Knippers 0 siblings, 0 replies; 14+ messages in thread From: Linda Knippers @ 2017-06-14 15:31 UTC (permalink / raw) To: Dan Williams, Yasunori Goto; +Cc: linux-nvdimm@lists.01.org On 6/13/2017 11:05 PM, Dan Williams wrote: > On Tue, Jun 13, 2017 at 7:41 PM, Yasunori Goto <y-goto@jp.fujitsu.com> wrote: >> Hi, Dan-san, Linda-san, >> >> I had chased the root cause of this panic problem, and maybe I found it. Very good! Thank you! -- ljk >> >>>>> Hmmm, though I made Fedora 25 environment, this panic still occurs... >>>>> I'll attach syslog and .config again. >>>>> >>>>> >>>> [..] >>>>> [ 117.804948] general protection fault: 0000 [#1] SMP >>>> [..] >>>>> [ 117.820866] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 >>>> [..] >>>>> [ 117.843262] Call Trace: >>>>> [ 117.843985] release_nodes+0x76/0x260 >>>>> [ 117.845062] devres_release_all+0x3c/0x50 >>>>> [ 117.846225] device_release_driver_internal+0x159/0x200 >>>>> [ 117.847748] device_release_driver+0x12/0x20 >>>>> [ 117.849029] bus_remove_device+0xfd/0x170 >>>>> [ 117.850192] device_del+0x1e8/0x330 >>>>> [ 117.851284] platform_device_del+0x28/0x90 >>>>> [ 117.852485] platform_device_unregister+0x12/0x30 >>>>> [ 117.853846] nfit_test_exit+0x2a/0x93b [nfit_test] >>>>> [ 117.855219] SyS_delete_module+0x171/0x250 >>>>> [ 117.856403] entry_SYSCALL_64_fastpath+0x1a/0xa5 >>>> >>>> Can you also attach the qemu-kvm command line you are using? >>>> >>>> ps aux | grep qemu >> >> >> The cause of this problem is the num_pm of nfit_test1 is wrong. >> Though 1 is specified for num_pm at nfit_test_init(), it must be 2. >> >> ---- >> static __init int nfit_test_init(void) >> { >> int rc, i; >> : >> : >> case 1: >> nfit_test->num_pm = 1; <---- !!! >> nfit_test->dcr_idx = NUM_DCR; >> ----- >> >> The num_pm affects size of devm_kcalloc() at nfit_test_probe(). >> >> ---- >> static int nfit_test_probe(struct platform_device *pdev) >> { >> if (nfit_test->num_pm) { >> int num = nfit_test->num_pm; <----!!! >> >> nfit_test->spa_set = devm_kcalloc(dev, num, sizeof(void *), >> GFP_KERNEL); <---!!!! >> nfit_test->spa_set_dma = devm_kcalloc(dev, num, >> sizeof(dma_addr_t), GFP_KERNEL); >> ----- >> >> However, spa_set[] array needs 2 elements at nfit_test1_alloc(). >> >> --- >> static int nfit_test1_alloc(struct nfit_test *t) >> { >> : >> t->spa_set[0] = test_alloc(t, SPA2_SIZE, &t->spa_set_dma[0]); <--- first element >> if (!t->spa_set[0]) >> return -ENOMEM; >> : >> >> t->spa_set[1] = test_alloc(t, SPA_VCD_SIZE, &t->spa_set_dma[1]); <---- The second element!!!! >> ----- >> >> This breaks other area, and the area is often the link list of devres. >> As a result, the panic occured on release_nodes(). >> >> I confirmed that this panic never occurred with the following patch. > > Wow! > >> >> --- >> diff --git a/tools/testing/nvdimm/test/nfit.c b/tools/testing/nvdimm/test/nfit.c >> index c218717..548b6d4 100644 >> --- a/tools/testing/nvdimm/test/nfit.c >> +++ b/tools/testing/nvdimm/test/nfit.c >> @@ -1943,7 +1943,7 @@ static __init int nfit_test_init(void) >> nfit_test->setup = nfit_test0_setup; >> break; >> case 1: >> - nfit_test->num_pm = 1; >> + nfit_test->num_pm = 2; >> nfit_test->dcr_idx = NUM_DCR; >> nfit_test->num_dcr = 2; >> nfit_test->alloc = nfit_test1_alloc; > > This change looks correct to me. I'm going to try it out. > > ...but, "Wow!" again and a big "Thank You!". > _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Panic when make check for ndctl 2017-06-14 2:41 ` Yasunori Goto 2017-06-14 3:05 ` Dan Williams @ 2017-06-14 16:06 ` Dan Williams 2017-06-15 5:04 ` Yasunori Goto 1 sibling, 1 reply; 14+ messages in thread From: Dan Williams @ 2017-06-14 16:06 UTC (permalink / raw) To: Yasunori Goto; +Cc: linux-nvdimm@lists.01.org On Tue, Jun 13, 2017 at 7:41 PM, Yasunori Goto <y-goto@jp.fujitsu.com> wrote: > Hi, Dan-san, Linda-san, > > I had chased the root cause of this panic problem, and maybe I found it. > >> > > Hmmm, though I made Fedora 25 environment, this panic still occurs... >> > > I'll attach syslog and .config again. >> > > >> > > >> > [..] >> > > [ 117.804948] general protection fault: 0000 [#1] SMP >> > [..] >> > > [ 117.820866] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 >> > [..] >> > > [ 117.843262] Call Trace: >> > > [ 117.843985] release_nodes+0x76/0x260 >> > > [ 117.845062] devres_release_all+0x3c/0x50 >> > > [ 117.846225] device_release_driver_internal+0x159/0x200 >> > > [ 117.847748] device_release_driver+0x12/0x20 >> > > [ 117.849029] bus_remove_device+0xfd/0x170 >> > > [ 117.850192] device_del+0x1e8/0x330 >> > > [ 117.851284] platform_device_del+0x28/0x90 >> > > [ 117.852485] platform_device_unregister+0x12/0x30 >> > > [ 117.853846] nfit_test_exit+0x2a/0x93b [nfit_test] >> > > [ 117.855219] SyS_delete_module+0x171/0x250 >> > > [ 117.856403] entry_SYSCALL_64_fastpath+0x1a/0xa5 >> > >> > Can you also attach the qemu-kvm command line you are using? >> > >> > ps aux | grep qemu > > > The cause of this problem is the num_pm of nfit_test1 is wrong. > Though 1 is specified for num_pm at nfit_test_init(), it must be 2. > > ---- > static __init int nfit_test_init(void) > { > int rc, i; > : > : > case 1: > nfit_test->num_pm = 1; <---- !!! > nfit_test->dcr_idx = NUM_DCR; > ----- > > The num_pm affects size of devm_kcalloc() at nfit_test_probe(). > > ---- > static int nfit_test_probe(struct platform_device *pdev) > { > if (nfit_test->num_pm) { > int num = nfit_test->num_pm; <----!!! > > nfit_test->spa_set = devm_kcalloc(dev, num, sizeof(void *), > GFP_KERNEL); <---!!!! > nfit_test->spa_set_dma = devm_kcalloc(dev, num, > sizeof(dma_addr_t), GFP_KERNEL); > ----- > > However, spa_set[] array needs 2 elements at nfit_test1_alloc(). > > --- > static int nfit_test1_alloc(struct nfit_test *t) > { > : > t->spa_set[0] = test_alloc(t, SPA2_SIZE, &t->spa_set_dma[0]); <--- first element > if (!t->spa_set[0]) > return -ENOMEM; > : > > t->spa_set[1] = test_alloc(t, SPA_VCD_SIZE, &t->spa_set_dma[1]); <---- The second element!!!! > ----- > > This breaks other area, and the area is often the link list of devres. > As a result, the panic occured on release_nodes(). > > I confirmed that this panic never occurred with the following patch. Care to resend this as a formal patch with a "Signed-off-by:"? I'll get it applied. _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Panic when make check for ndctl 2017-06-14 16:06 ` Dan Williams @ 2017-06-15 5:04 ` Yasunori Goto 2017-06-15 21:29 ` Dan Williams 0 siblings, 1 reply; 14+ messages in thread From: Yasunori Goto @ 2017-06-15 5:04 UTC (permalink / raw) To: Dan Williams; +Cc: linux-nvdimm@lists.01.org > > Care to resend this as a formal patch with a "Signed-off-by:"? I'll > get it applied. Ok! ----- Fix the panic when nfit_test.ko is used. The root cause of panic is the num_pm of nfit_test1 is wrong. Though 1 is specified for num_pm at nfit_test_init(), it must be 2, because nfit_test1->spa_set[] array has 2 elements. Since the array is smaller than expected, the driver breaks other area. (it is often the link list of devres). As a result, panic occurs like the following example. --- 117.741905] CPU: 4 PID: 2233 Comm: lt-libndctl Tainted: G O 4.12.0-rc1+ #12 [ 117.744461] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [ 117.746310] task: ffff88041d685500 task.stack: ffffc90002b50000 [ 117.748198] RIP: 0010:__list_del_entry_valid+0x6c/0xa0 [ 117.751084] RSP: 0018:ffffc90002b53d60 EFLAGS: 00010046 [ 117.753142] RAX: 0000000000000054 RBX: ffffc9001700e000 RCX: 0000000000000000 [ 117.755955] RDX: 0000000000000000 RSI: ffff88043310e0a8 RDI: ffff88043310e0a8 [ 117.758745] RBP: ffffc90002b53d60 R08: 00000000fffffffe R09: 0000000000000382 [ 117.760801] R10: 0000000000000005 R11: 0000000000000381 R12: ffffc90002b53d98 [ 117.762949] R13: ffffc9001700e000 R14: 0000000000000000 R15: ffff88041beefc08 [ 117.765000] FS: 00007f7d70082c80(0000) GS:ffff880433100000(0000) knlGS:0000000000000000 [ 117.767300] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 117.768962] CR2: 000055c0db522cc0 CR3: 0000000422156000 CR4: 00000000000006e0 [ 117.770975] Call Trace: [ 117.771716] release_nodes+0x76/0x260 [ 117.772797] devres_release_all+0x3c/0x50 [ 117.773981] device_release_driver_internal+0x159/0x200 [ 117.775494] device_release_driver+0x12/0x20 [ 117.776763] bus_remove_device+0xfd/0x170 [ 117.777924] device_del+0x1e8/0x330 [ 117.778943] platform_device_del+0x28/0x90 [ 117.780129] platform_device_unregister+0x12/0x30 [ 117.781483] nfit_test_exit+0x2a/0x93b [nfit_test] [ 117.782858] SyS_delete_module+0x171/0x250 [ 117.784044] entry_SYSCALL_64_fastpath+0x1a/0xa5 --- Signed-off-by: Yasunori Goto <y-goto@jp.fujitsu.com> --- tools/testing/nvdimm/test/nfit.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/nvdimm/test/nfit.c b/tools/testing/nvdimm/test/nfit.c index c218717..548b6d4 100644 --- a/tools/testing/nvdimm/test/nfit.c +++ b/tools/testing/nvdimm/test/nfit.c @@ -1943,7 +1943,7 @@ static __init int nfit_test_init(void) nfit_test->setup = nfit_test0_setup; break; case 1: - nfit_test->num_pm = 1; + nfit_test->num_pm = 2; nfit_test->dcr_idx = NUM_DCR; nfit_test->num_dcr = 2; nfit_test->alloc = nfit_test1_alloc; --- Yasunori Goto _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: Panic when make check for ndctl 2017-06-15 5:04 ` Yasunori Goto @ 2017-06-15 21:29 ` Dan Williams 0 siblings, 0 replies; 14+ messages in thread From: Dan Williams @ 2017-06-15 21:29 UTC (permalink / raw) To: Yasunori Goto; +Cc: linux-nvdimm@lists.01.org On Wed, Jun 14, 2017 at 10:04 PM, Yasunori Goto <y-goto@jp.fujitsu.com> wrote: >> >> Care to resend this as a formal patch with a "Signed-off-by:"? I'll >> get it applied. > > Ok! > > ----- > Fix the panic when nfit_test.ko is used. > > The root cause of panic is the num_pm of nfit_test1 is wrong. > Though 1 is specified for num_pm at nfit_test_init(), it must be 2, > because nfit_test1->spa_set[] array has 2 elements. > > Since the array is smaller than expected, the driver breaks other area. > (it is often the link list of devres). > > As a result, panic occurs like the following example. > --- > 117.741905] CPU: 4 PID: 2233 Comm: lt-libndctl Tainted: G O 4.12.0-rc1+ #12 > [ 117.744461] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 > [ 117.746310] task: ffff88041d685500 task.stack: ffffc90002b50000 > [ 117.748198] RIP: 0010:__list_del_entry_valid+0x6c/0xa0 > [ 117.751084] RSP: 0018:ffffc90002b53d60 EFLAGS: 00010046 > [ 117.753142] RAX: 0000000000000054 RBX: ffffc9001700e000 RCX: 0000000000000000 > [ 117.755955] RDX: 0000000000000000 RSI: ffff88043310e0a8 RDI: ffff88043310e0a8 > [ 117.758745] RBP: ffffc90002b53d60 R08: 00000000fffffffe R09: 0000000000000382 > [ 117.760801] R10: 0000000000000005 R11: 0000000000000381 R12: ffffc90002b53d98 > [ 117.762949] R13: ffffc9001700e000 R14: 0000000000000000 R15: ffff88041beefc08 > [ 117.765000] FS: 00007f7d70082c80(0000) GS:ffff880433100000(0000) knlGS:0000000000000000 > [ 117.767300] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 117.768962] CR2: 000055c0db522cc0 CR3: 0000000422156000 CR4: 00000000000006e0 > [ 117.770975] Call Trace: > [ 117.771716] release_nodes+0x76/0x260 > [ 117.772797] devres_release_all+0x3c/0x50 > [ 117.773981] device_release_driver_internal+0x159/0x200 > [ 117.775494] device_release_driver+0x12/0x20 > [ 117.776763] bus_remove_device+0xfd/0x170 > [ 117.777924] device_del+0x1e8/0x330 > [ 117.778943] platform_device_del+0x28/0x90 > [ 117.780129] platform_device_unregister+0x12/0x30 > [ 117.781483] nfit_test_exit+0x2a/0x93b [nfit_test] > [ 117.782858] SyS_delete_module+0x171/0x250 > [ 117.784044] entry_SYSCALL_64_fastpath+0x1a/0xa5 > --- > Thanks, applied. _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2017-06-15 21:28 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20170529155643.5848.E1E9C6FF@jp.fujitsu.com>
2017-05-29 14:21 ` Fwd: Fw: Panic when make check for ndctl Yasunori Goto
2017-05-29 23:04 ` Dan Williams
2017-05-30 7:34 ` Yasunori Goto
2017-05-30 11:44 ` Dan Williams
2017-06-06 5:51 ` Yasunori Goto
2017-06-06 13:32 ` Linda Knippers
2017-06-06 16:42 ` Dan Williams
2017-06-07 0:33 ` Yasunori Goto
2017-06-14 2:41 ` Yasunori Goto
2017-06-14 3:05 ` Dan Williams
2017-06-14 15:31 ` Linda Knippers
2017-06-14 16:06 ` Dan Williams
2017-06-15 5:04 ` Yasunori Goto
2017-06-15 21:29 ` Dan Williams
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.